SBIR/STTR at the National Institutes of Health (2015)

8

Findings and Recommendations

The findings and recommendations in this chapter reflect the performance of the NIH SBIR/STTR programs against the broad congressional objectives for the SBIR and STTR programs.¹

For SBIR, these objectives were reiterated in the 2011 program reauthorization and elaborated in the subsequent policy directive of the Small Business Administration.² Section 1c of the Small Business Administration (SBA) SBIR Directive states program objectives as follows:

The parallel language from the SBA’s STTR Policy Directive is as follows:

_______________

¹See Box 1-2 and the discussion of the Committee’s task in Chapter 1 (Introduction).

²SBA SBIR/STTR Policy Directive, October 18, 2012.

³Ibid., 3.

The findings below review the extent to which each of these program objectives is being addressed at NIH, as well as examine some specific aspects of NIH’s management of the program.

FINDINGS

The Small Business Innovation Research (SBIR) program at the National Institutes of Health is having a positive overall impact. It is meeting three of the four legislative objectives of the program with regard to stimulating technological innovation, using small businesses to meet federal research and development (R&D) needs, and increasing private-sector commercialization of innovations derived from federal R&D. However, we find that more needs to be done to “foster and encourage participation by socially and economically disadvantaged small businesses (SDBs), and by women-owned small businesses (WOSBs), in technological innovation.” The Small Business Technology Transfer (STTR) program at the National Institutes of Health is meeting the program’s statutory objectives, defined above.

The order in which the findings below are presented reflects the committee’s relative emphasis. The first set of findings focus on the commercialization of SBIR/STTR funded projects. This is followed by findings concerning the participation of women and minorities in the program. The third and fourth sets of findings address how well the NIH SBIR/STTR programs are stimulating technological innovation and fostering innovative companies. The final set of findings concern the management of the programs at NIH.

Sources of Findings

The committee’s findings are based on a complement of quantitative and qualitative tools including a survey, case studies of award recipients, agency data, public workshops, and agency meetings. The methodology is described in Chapter 1 and Appendix A of this report. In reviewing the findings below, it is important to note that the Academies’ 2014 Survey—hereafter referred to as the 2014 Survey—was sent to every principal investigator (PI) who received a Phase II award from NIH, FY2001-2010. PIs were asked to complete a maximum

_______________

⁴Small Business Administration, Office of Investment and Innovation, “Small Business Technology Transfer (STTR) Program—Policy Guidance,” updated February 24, 2014.

of two questionnaires. The preliminary population prior to contact was 3,375. Of these, 1,723 were determined to be not contactable at the SBIR/STTR company listed in the NIH awards database. The remaining 1,652 awards constitute the effective population for this study. We received 726 responses, for a preliminary population response rate of 21.5 percent and an effective population response rate of 43.9 percent.

I. Commercialization

The focus at NIH has primarily been on the commercialization of SBIR/STTR funded projects and on the development of technologies that help to meet the agency’s mission (discussed separately below). The committee recognizes that issue of commercialization is complex.⁵ For NIH, these objectives are primarily met when projects are commercially successful in private-sector markets. Keeping in mind the low response rate for the 2014 survey, the key findings are as follows:

1. SBIR/STTR projects at NIH commercialize at a substantial rate.⁶

1. Sales are reported by a substantial fraction of the survey respondents: Forty-nine percent of SBIR and STTR respondents reported some sales or licensing revenues at the time of the survey, and a further 25 percent expected sales in the future, according to the 2014 Survey.⁷ This is similar to the rates reported in the 2005 Survey (46 percent and 19 percent, respectively) as reported in the Academies’ 2009 report on the NIH SBIR program.⁸
2. Sales anticipated: These rates inevitably undercount the eventual share of projects that generate sales, and an additional 25 percent of Phase II respondents reported that they anticipate future sales.⁹
3. There is room for improvement: The large number of companies with small-scale revenues suggests that while many companies reach the market, fewer can be described as successful in commercial terms.¹⁰ Despite the high percentage of SBIR/STTR projects with sales, the amount of sales was often small: of those with some sales, 39 percent had sales less than $100,000. Six percent had sales over $10 million.¹¹

_______________

⁵See the discussion on Defining “Commercialization” in Chapter 5.

⁶NIH does not yet have in place internal capacity to track project outcomes.

⁷See Table 5-7.

⁸National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, Washington, DC: The National Academies Press, 2009, Figure 4-4, p. 88.

⁹See Table 5-7.

¹⁰See Table 5-8.

¹¹See Table 5-8.

2. NIH SBIR/STTR projects are primarily commercializing in the domestic private sector.

1. According to the 2014 Survey, 57 percent of responses with sales reported revenues from domestic private-sector customers.¹²
2. Seventeen percent reported export customers.¹³
3. Eleven percent of responses identified customers in the public sector (primarily state and local governments or other federal agencies).¹⁴

3. Further investment in NIH SBIR/STTR projects is additional evidence of commercial activity. Subsequent investment in NIH SBIR/STTR projects is an indicator that they are expected to generate substantial commercial value even if they have not yet reached the market. The 2014 Survey shows that:

1. About 80 percent of 2014 Survey respondents reported additional investment funding.¹⁵
2. The most likely source of additional funding (other than their own company and personal funds) was the U.S. private sector (44 percent of responses). This included funding from strategic investors (21 percent), angel investors (14 percent), and venture capital (10 percent).¹⁶

4. NIH SBIR/STTR awards are not overall associated with substantial direct job growth, but some awardees grew rapidly.

1. The 2014 Survey indicates, based on responses received, that the median size of firms with NIH awards remained flat on average at seven employees between the time of award and late 2014. Other things being equal, larger employment gains are more typically associated with the long-term commercialization phase of the resulting innovation, rather than with the research phase.
2. Some firms grew rapidly, however, as mean employment grew from 19 at the time of award to 88 employees on average at the time of survey.¹⁷

5. For small innovative firms, SBIR/STTR funding makes an important difference to project outcomes. SBIR/STTR funding makes a substantial

_______________

¹²See Table 5-9.

¹³See Table 5-9.

¹⁴See Table 5-9.

¹⁵2014 Survey, Question 30. N=572.

¹⁶See Table 5-13.

¹⁷See Tables 5-10 and 5-11.

difference in determining project initiation, scope, and timing. The 2014 Survey data show that:

1. Seventy-four percent of respondents reported that the project probably or definitely would not have proceeded without SBIR/STTR funding.¹⁸
2. Fifty-seven percent of those who would likely have proceeded anyway reported that the project would have been narrower in scope.¹⁹
3. About one-third of those who would likely have proceeded anyway reported that the project would have been delayed by at least 1 year.²⁰

6. Venture capital funding plays only a modest role for NIH SBIR/STTR firms.

1. Venture capital funding plays an instrumental role in the eventual commercialization of many medical therapeutics and devices. The 2011 reauthorization permitted NIH to use up to 25 percent of its SBIR/STTR funding for projects submitted by companies that are majority owned by venture capitalists.
2. However, while 80 percent of respondents surveyed reported that they raised additional investment funds, only 10 percent reported that the funding included funds from venture capitalists (VCs).²¹ Most NIH SBIR/STTR projects do not meet the narrow criteria set by VC firms, including timeline to market exit, size of opportunity, amount of funding required, capabilities of the management team, and industry sector.

7. NIH SBIR/STTR firms face challenges particular to the commercialization of biomedical technologies.

1. A large proportion of NIH SBIR/STTR projects must receive approval from the Food and Drug Administration (FDA) before they can reach the market. Responses to the 2014 Survey indicated that 45 percent of funded SBIR and STTR Phase II project products required FDA approval.²²
2. The FDA regulatory process is often a barrier for SBIR/STTR companies. While costs vary, the overall cost of a Phase 3 clinical trial is normally much more than is available through SBIR/STTR, even with additional funding through Phase IIB. Funding in the tens of millions of

_______________

¹⁸See Table 5-31.

¹⁹2014 Survey, Question 26A.

²⁰2014 Survey, Question 26A.

²¹See Table 5-2.

²²See section on “Survey Data about FDA Approval” in Chapter 2 (Program Management).

3. dollars is not exceptional.²³ SBIR/STTR companies often do not have sufficient expertise in dealing with the FDA.²⁴

4. Extensive evidence from open-ended survey responses solicited in the 2014 Survey and from case studies indicates that small innovative companies do not have a straightforward pathway to funding for clinical trials. As a result, about one-half of the NIH SBIR/STTR awardees face a major funding challenge before they can reach the market.
   • NIH has no dedicated mechanism to fund clinical trials.
   • 15 percent of respondents who were engaged in clinical trials mentioned venture funding.
   • Less than 10 percent of respondents mentioned “other company” funding, which in this case would include large pharmaceutical companies.

II. Fostering the Participation of Women and Other Underserved Groups in the SBIR/STTR Programs

1. Current outcomes data show that the objective of fostering the participation of women and underserved minorities has not been met by the NIH SBIR/STTR programs.

1. Levels of participation by underserved groups are low and declining.
   • Data from NIH indicate that the share of Phase I awards made to SBIR/STTR Minority-Owned Small Businesses (MOSBs) has declined from a peak of 3.5 percent in 2006 to less than 2 percent in 2014.²⁵ The pattern for Phase II awards is more variable, but the trend there too is downward.²⁶
   • MOSBs also show lower success rates for both Phase I and Phase II applications than non-MOSB applicants. The success for Phase I MOSB applicants was lower in every year, and averaged 10.1 percent across the period, while the rate for non-MOSB applications was 18.3 percent.
   • The success rate of Phase II MOSBs averaged 6 percentage points lower over the period, and was lower for every year except 2010 and

_______________

²³For example, Sanaria recently announced an agreement for clinical trial funding for $48.5 million. See Sanaria case study in Appendix E.

²⁴See Chapter 2 (Program Management).

²⁵See Figure 6-13.

²⁶See Figure 6-16.

2. 2012.²⁷ NIH does not maintain data on woman and minority Principal Investigators (PIs). Data from the 2014 Survey indicate that these numbers were also low. The survey indicated that 15 percent of PIs were female.²⁸

3. Participation by Black, Hispanic, and Native Americans in NIH SBIR/STTRs program is low.
   • The 2014 Survey indicates that Blackowned small businesses accounted for only 0.7 percent of all respondents; Hispanic-owned small businesses, about 1.7 percent.²⁹
   • Seven percent of survey respondents indicated that the Principal Investigator on the surveyed project was from a minority. However, more detailed analysis indicates that 0.5 percent were Black American, 1.6 percent Hispanic, and 0.2 percent American Indian.³⁰
4. Levels of participation by women are also low.

   NIH data show that 10 percent of SBIR/STTR Phase I awards were to WOSBs and that these firms receive 12 percent of Phase II awards.³¹ However WOSB success rates were persistently lower than those for non-WOSBs for both Phase I and Phase II.³²

2. NIH efforts to “foster and encourage” the participation of women-owned and minority-owned small businesses are not adequate.³³

1. NIH outreach efforts have focused more heavily on efforts to attract participation from low-award states than from women-owned and minority-owned small businesses.
   • The SBAsponsored Road Show is a primary outreach activity and is targeted at low award states.

_______________

²⁷See Figure 6-12 for Phase I MOSB comparative success rates for applications receiving awards, and Figure 6-16 for Phase II MOSB comparative success rates.

²⁸2014 Survey, Question 16.

²⁹See Table 6-2.

³⁰See Table 6-1.

³¹See Figure 6-4 for percentages of SBIR/STTR Phase I awards going to WOSBs, and Figure 6-8 for percentage of SBIR/STTR Phase II awards going to WOSBs.

³²See Figures 6-4 and 6-8.

³³Information in this section is based on the “Outreach” section in Chapter 6 (Participation of Women and Minorities).

2. • The NIH Annual Report to SBA for FY 2014 mentions a considerable catalog of outreach activities—but mentions under-represented groups only as a part of one activity.
   • Most NIH Program outreach is conducted in conjunction with other partners. This means that NIH has limited capacity to attune these events to its own needs. NIH is now working to improve reporting on outreach activities with these partners, especially in relation to women and minorities.
3. NIH is developing outreach activities focused on women and minorities.
   • NIH now holds a wellattended workshop focused on women and minorities at its regular annual SBIR/STTR conference.
   • NIH is planning to work more closely with woman and minority professional societies in the life sciences and science and engineering more generally.

3. NIH’s efforts to understand the patterns of woman and minority participation in the SBIR program are not adequate. Concerted analytic effort is needed to determine what practical steps can be taken to improve participation and hence both meet congressional objectives for the program and expand the pool of qualified applicants and capabilities.³⁴

1. NIH maintains no separate data on African American-, Hispanic-, or Native American-owned small businesses.
2. NIH has not followed up with a review of application and award patterns for women and minorities. These patterns would show differences between woman/minority applications and other applications on a variety of metrics.³⁵
3. NIH has not sought to contextualize observed patterns against larger patterns of participation in life sciences. Participation rates are low especially for minorities. NIH needs to determine whether this is a function of the life sciences sector, of the SBIR/STTR programs, of the financial and business communities, or of a combination of these factors. Furthermore, trends are an especially important indicator: declining participation rates are especially a matter of concern.³⁶

_______________

³⁴A discussion of women and minority participation and NIH’s limited efforts to address the issue is provided in more detail in the section, “Summary: Woman and Minority Participants in the NIH SBIR/STTR Program,” in Chapter 6.

³⁵See Figures 6-8 and 6-9.

³⁶See Figure 6-14.

III. Stimulating Technological Innovation and Meeting Agency Mission Needs

NIH’s agency mission is to “seek fundamental knowledge about the nature and behavior of living systems and the application of that knowledge to enhance health, lengthen life, and reduce illness and disability.”³⁷ Thus the twin objectives of using small business to meet federal agency needs and to stimulate technological innovation are closely intertwined and are therefore discussed together in this section.

1. The SBIR/STTR programs at NIH support the development and adoption of technological innovations that advance the agency’s mission.³⁸

1. NIH topics focus on new research and technological opportunities. Some SBIR/STTR topics are generated by program officers and are then approved by the NIH’s Institutes and Centers (IC) and some are applicant-generated. The former are funded via proposals resulting in contract arrangements; the latter via applications resulting in awards. Because the agency does not typically buy or use the outputs from SBIR/STTR research, the topics for contract research are likely to be driven by the research priorities of the agency.³⁹
2. Awards are not limited to problems and technologies described in the NIH solicitations. Under the terms of the solicitation, non-matching applications can also be accepted, in an effort to ensure that potentially important innovations are not excluded by the topic structure. However, with both the recent shift toward more targeted contracts at the National Cancer Institute (NCI) and the general increase in more targeted funding opportunities, this hitherto central characteristic of the NIH program may be eroding. NIH does not track whether applications match the solicitation topics, so assessing the actual impact of these shifts on SBIR/STTR applicants is difficult.⁴⁰
3. NIH scoring selects for novelty.
   • Selection scoring for individual projects utilizes the standard NIH scoring criteria, which are heavily weighted toward the novelty of the proposed project.⁴¹

_______________

³⁷National Institutes of Health, “Mission,” http://www.nih.gov/about/mission.htm, accessed July 9, 2015.

³⁸See Chapter 2 (Program Management) for a discussion of NIH topic selection and funding selection procedures.

³⁹See Chapter 2 (Program Management).

⁴⁰See Chapter 2 (Program Management).

⁴¹See Chapter 2 (Program Management).

2. The NIH SBIR/STTR programs continue to connect companies to universities and research institutions.

1. Faculty and student participation: For the SBIR program alone, 63 percent of survey respondents reported a link to some kind to a research institution. In 39 percent of responses, faculty worked on the project (not as a PI); 22 percent employed graduate students; and 37 percent used universities and research institutions as subcontractors.⁴² These figures are in all cases up from those reported from the 2005 Survey.⁴³
2. Project partners: 255 different universities were identified by survey respondents as project partners; 34 were mentioned by more than four respondents.⁴⁴
3. Academic founders: Eighty-five percent of SBIR/STTR companies responding to the 2014 Survey reported at least one academic founder, and 59 percent reported that the most recent prior employment of the founder was at a university.⁴⁵

3. NIH SBIR/STTR projects generate substantial knowledge-based outputs such as patents and peer-reviewed publications.

1. Patents: Patenting remains an important component of knowledge diffusion (and protection).
   • About twothirds of companies overall (and more than 80 percent of Phase IIB recipients) responding to the 2014 Survey claimed to have been awarded at least one patent related to any SBIR/STTR-funded technology.⁴⁶
   • Fifty-three percent of SBIR respondents reported receiving at least one patent related to the surveyed technology. Nine percent reported receiving 5 or more related patents, and 4 percent reported 10 or more. STTR awardees were not more likely to report receiving

_______________

⁴²See Table 5-20.

⁴³National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 259.

⁴⁴See Table 5-21.

⁴⁵See Tables 5-22 and 5-23.

⁴⁶See Table 5-17.

2. patents. These figures are down slightly from those reported in the 2005 Survey, where 57 percent reported at least one patent.^47,48

3. Peer-reviewed publications: Publication of peer-reviewed articles remains the primary currency of scientific discourse, and despite the need to protect ideas in the commercial environment of small businesses, the 2014 Survey shows that SBIR/STTR firms continue to pursue and achieve scientific publication.
   • Seventy-nine percent of SBIR respondents and 85 percent of STTR respondents indicated that an author at the surveyed company had published at least one related scientific paper.⁴⁹
   • Forty-two percent reported publishing three or more related papers.⁵⁰ Many of the companies interviewed for case studies made a point of indicating that they take a great deal of pride in the number of peer-reviewed publications developed by their scientists and engineers, both within and outside of the SBIR/STTR programs.⁵¹

4. The NIH SBIR/STTR programs fund projects with social benefits that may not be attractive to commercial sources of funding.⁵² The NIH SBIR/STTR programs fund some projects that are high risk, socially desirable, and market oriented but that are unlikely to generate the high returns needed to attract venture-type funding. Companies working on projects with these characteristics are often not attractive to commercial investors:

1. Small markets. The FDA orphan drug designation recognizes that some markets are not commercially attractive. Companies like ArmaGen are seeking to develop therapies for small populations.⁵³
2. Long cycle research. NIH SBIR/STTR help support the development of innovations that will take many years to reach the market. While these projects may hold great potential for positive impact, the time taken to get to market can be a major barrier for commercial investors.⁵⁴

_______________

⁴⁷National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 265.

⁴⁸See Table 5-18.

⁴⁹See Table 5-19.

⁵⁰See Table 5-19.

⁵¹See Chapter 5 (Outcomes).

⁵²See section on “Funding Otherwise Un-fundable Projects” in Chapter 5 (Outcomes).

⁵³See ArmaGen case study in Appendix E.

⁵⁴For example, Dr. Pardridge (Armagen) has been working on the blood-brain barriers for more than 30 years. Dr. Sabbadini founded Lpath in 1997. See section on “Long Cycle Research” and Box 7-7 in Chapter 7 (Insights).

5. SBIR/STTR funds the development of research tools that leverage its impact many times. Case studies show that the impact of awards can be multiplied if SBIR/STTR technologies are used to develop innovative tools and services for biomedical researchers working in partnership with larger pharmaceutical companies to dramatically reduce the cost and increase the success rate of clinical trials. Avanti Lipids has become a core provider of lipids to the research market.⁵⁵ Another company, Invitrogen, has now become the world’s largest provider of genetic testing to the research community.⁵⁶

IV. Fostering Innovative Companies

1. The NIH SBIR/STTR programs support the foundation of new innovative firms: Many of the survey respondents reported that SBIR/STTR funding was instrumental in the founding of the company. The formation of new innovative companies is a positive outcome for the program.

1. Forty-four percent of survey respondents said that the company was founded entirely or in part because of the SBIR/STTR programs.⁵⁷
2. For some companies included among the case studies, SBIR/STTR funding permitted the shift from an exploratory to a professional operation.⁵⁸ And for some STTR companies in particular, funding permitted university faculty to retain their positions while initially building the company.⁵⁹

2. Funding provided by the NIH SBIR/STTR programs reduces the risk for subsequent investors: Leveraging of SBIR/STTR funding helps small innovative firms lower their risks while retaining the power of markets to make final decisions about funding.

1. Early stage: Many respondents to the 2014 Survey and a number of companies contacted for case studies said that NIH SBIR/STTR funding was provided at a stage when the project was simply too risky for commercial sources of funding. Once the project proceeded further, risk was lower and additional funding could be acquired.⁶⁰

_______________

⁵⁵See Avanti Lipids case study in Appendix E.

⁵⁶See Invitrogen case study in National Research Council, An Assessment of the SBIR Program at the National Institutes of Health.

⁵⁷See Table 5-24.

⁵⁸See section on “Validation Effects” and Box 7-4 in Chapter 7 (Insights).

⁵⁹See, for example, Stratatech case study in Appendix E.

⁶⁰See section on “Company Formation and Very Early Stage Funding” and Box 7-2 in Chapter 7 (Insights).

2. Support for core technology development: NIH SBIR/STTR funding supports technology development, which can be supported through commercial funding further downstream. SBIR/STTR is particularly important for funding proof of concept for new technologies, as described in several case studies (Appendix E) as well as in survey responses.⁶¹
3. Validation: NIH SBIR/STTR funding has itself been important validation for companies seeking further investments, according to discussions with representatives of case study companies and survey responses. The strength of the selection process and growing understanding of SBIR/STTR among both equity and strategic investors may be strengthening this effect.⁶²
4. Exploit technology platforms: In some cases, companies use SBIR/STTR funding to build off existing platform technologies specifically to enter new markets. This platform-driven approach is used by a number of the companies highlighted in the case studies. (See Chapter 7.)
5. Strategic corrections: Innovative companies must often make mid-course corrections to their business strategy. NIH funding has—according to respondents to the 2014 Survey—helped a number of companies successfully make what are often difficult changes that are hard to fund.

3. The NIH SBIR/STTR programs have supported the development of small innovative companies in the United States.

1. The 2014 Survey provided SBIR/STTR companies with the opportunity both to report the overall impact of SBIR/STTR on the company and to identify specific kinds of impacts. Sixty-two percent of Phase II winning recipients indicated that the NIH SBIR/STTR programs had a “highly positive or transformative” effect on their company. Another 35 percent said that it had a “substantial positive long term effect.”⁶³
2. The 393 detailed comments received in the 2014 Survey offered widely differing kinds of impacts, as summarized in Box 8-1.⁶⁴

4. Company dependence on the NIH SBIR/STTR programs is limited: SBIR/STTR is not designed to provide permanent life support for companies. Congress has—through changes under the 2011 reauthorization—indicated that it expects companies to use SBIR/STTR to move on to a commercialization phase that is not funded by SBIR/STTR. And NIH is, according to discussions with representatives of case study companies as well as com-

_______________

⁶¹See section “Innovative Technologies and Product Development” and Box 7-6 in Chapter 7 (Insights).

⁶²See section on “Validation Effects” and Box 7-4 in Chapter 7 (Insights).

⁶³See Table 5-29.

⁶⁴See section on “Key Aspects of SBIR-Driven Transformation”” in Chapter 5 (Outcomes), as well as Chapter 7 (Insights) and Appendix E (Case Studies).

ments from several survey respondents, now tightening its requirements, to the point that some respondents indicated that they no longer seek awards because their commercialization record is not strong enough.

1. No formal limits on multiple awards. NIH does not limit either the number of awards or the number of applications from a company. This practice contrasts with that of the National Science Foundation (NSF) SBIR/STTR programs.
2. Awards are spread widely across the applicant pool. At NIH, the most prolific winners account for a relatively low percentage of all awards (compared to SBIR/STTR programs at other agencies). The most prolific winners received 24 Phase II awards over the 10-year study period, and the top 21 winners (top 20 plus ties) received 9.9 percent of Phase II SBIR/STTR awards and 8 percent of funding during this period.
3. The company’s commercialization track record is of growing importance. While there are no formal metrics or benchmarks, companies believe that NIH is increasingly focused on outcomes and that companies without commercialization from previous awards will find it harder to garner new ones.⁶⁵
4. Most NIH firms are not dependent on SBIR/STTR awards. Only 27 percent of companies responding to the 2014 Survey report that SBIR/STTR accounts for more than half of current revenues.⁶⁶ However, a considerable number of surveyed firms reported in textual responses that SBIR/STTR has been the most important source of funding prior to reaching the market.⁶⁷
5. NIH SBIR/STTR innovation is non-linear. Most projects at most companies do not proceed directly from Phase I to Phase II to commercialization.⁶⁸
   • About 80 percent of Phase II survey respondents reported at least one additional SBIR/STTR Phase II award related to the surveyed project.⁶⁹
   • About onethird reported at least two additional related Phase II awards.⁷⁰
   • As noted above, more than 60 percent of Phase II respondents reported additional investment funding related to the project subsequent to the SBIR/STTR award.⁷¹

_______________

⁶⁵See Chapter 7 (Insights from Case Studies and Survey Responses).

⁶⁶See Table 5-26.

⁶⁷See Chapter 7 (Insights) and Appendix E (Case Studies).

⁶⁸See section on “Prior Use of the SBIR/STTR Program” in Chapter 5 (Outcomes).

⁶⁹See Table 5-27.

⁷⁰See Table 5-28.

⁷¹See Chapter 5.

V. Program Management

1. The NIH SBIR/STTR programs are managed in a flexible way in terms of application topics, dates, and funding.⁷²

1. Focus is on investigator-initiated research: NIH makes clear that topics listed in the Omnibus Solicitation are guides for applicants, not boundaries. And while targeted solicitations have become more common, research conducted under SBIR/STTR at NIH is still largely investigator driven.
2. Multiple applications dates: NIH offers three submission dates annually, which provides investigators with a much reduced timeline to funding.
3. Significant funding flexibility:
   • Funding amounts. Amounts are not preset, and selection panels do not compare funding requests among applications. NIH has in many cases, and with appropriate SBA waivers, provided funding that goes beyond the standard SBA guidelines.
   • Supplementary funding. NIH provides small amounts of supplementary funding in cases where the completion of research plans can be accomplished with a minor increase in support.
   • Nocost extensions. NIH will normally extend the timeline for an award.
   • Multiple support mechanisms. The introduction of Phase IIB and direct to Phase II indicates that NIH continues to seek ways to match available funding with the needs of companies and investigators.
4. Resubmission of applications: The ability to resubmit applications after addressing flaws identified by selection panels is a unique feature of the NIH SBIR/STTR programs and, while highly commendable, can be further improved.

2. The NIH application review system can be improved: Case studies, survey responses, and discussions with agency managers all indicate that although the NIH application review system is highly regarded and has many positive characteristics, it is not serving the SBIR/STTR community as well as it could.⁷³

1. NIH’s commercialization review is overly weighted toward the views of academic reviewers.
   • Based on information gathered through case study and agency discussions and survey responses, a large majority of reviewers appear

_______________

⁷²See section on “Program Flexibility” in Chapter 2 (Program Management).

⁷³See section on “The Peer Review Process” in Chapter 2 (Program Management).

2. • to be academics. (NIH does not track the composition of review panels.)

   • NIH’s Center for Scientific Review counts as “commercially experienced” all reviewers working in the private sector. This is not appropriate because most industry scientists have no more knowledge of commercialization than academic scientists. Recently issued guidelines are insufficient to ensure that each application is assessed by at least one reviewer with commercialization expertise. This is a minimal requirement for effective review.
   • Academic reviewers are permitted to review commercial potential, for which they may have no valid expertise.
   • Selection of awards is based on standard NIH criteria, which do not include commercial potential. Thus, commercial potential is not part of the formal scoring system.
   • Scientific Review Officers, assigned to SBIR/STTR panels, are not usually specialists in SBIR/STTR.
3. Reviewers may have a weak or misinformed view of innovation in the context of SBIR/STTR.
   • Grants at NIH are reviewed against standard NIH criteria including innovation, defined as “novel theoretical concepts, approaches or methodologies, instrumentation, or interventions.”⁷⁴ However, “novelty” in the context of an academic grant application is not the same as “innovation” for SBIR/STTR. The latter includes all of the steps necessary to reach the market, many of which have no “novelty” in academic terms.⁷⁵
   • The result is that the SBIR/STTR Phase II applications may be subject to misapplied innovation criteria.
4. Timelines for NIH review cause unnecessary and expensive delays for small businesses.
   • There are three NIH deadlines for submission annually. This is a positive feature of the NIH program, especially compared to agencies that provide a single annual deadline. However, NIH debriefings for rejected applications are usually delivered too late for resubmission at the next deadline, effectively imposing a minimum 4-month additional delay on applicants.

_______________

⁷⁴See section on “Peer Review Process” in Chapter 2 (Program Management).

⁷⁵See Chapter 7 (Insights).

2. • Delays can be a major challenge for small companies—especially startups—who must often use up their own limited capital to stay in business until they can receive funding and begin their SBIR/STTR project, which may eventually lead to revenue generation and a growing self-sufficiency for the company.
   • Case studies show that projects that are rejected based on minor defects and misunderstandings, and then allowed to be revised and resubmitted, may in fact impose heavy penalties on small innovative firms if the wait time for resubmission is too long.⁷⁶
3. NIH does not employ methods used by other agencies to help reduce the number of poor-quality applications.⁷⁷
   • Discouraging the submission of poorquality proposals can improve program efficiency and may allow reviewers to focus more carefully on a smaller number of better proposals.

     For example: NSF limits the number of applications that a given firm can submit, ensuring that firms put forward only what they consider their most promising projects. The Department of Energy (DoE) requires the submission of a white paper before application and provides applicants with rapid feedback on their chances of success. Some agencies use pre-review screenings as triage to remove obviously fatally flawed and highly deficient proposals.

3. Survey data shows that the NIH Phase IIB program supports the accelerated commercialization of SBIR-funded research through the provision of funding for clinical trials.

1. About 45 percent of those responding to the 2014 Survey indicated that their project will require FDA approval; the costs of clinical trials pose a large barrier to commercialization of SBIR/STTR technologies.
   • Full completion of clinical trials’ costs range widely, but they now account for approximately 40 percent of drug development costs.
   • The size, scope, and complexity of required clinical trials have increased sharply.⁷⁸

_______________

⁷⁶See Chapter 7 (Insights).

⁷⁷See Chapter 2 (Program Management).

⁷⁸See Table 1 and Table 2 in S. Avik and A. Roy, “Stifling New Cures: The True Cost of Lengthy Clinical Drug Trials,” Manhattan Institute of Policy Research, April 2012.

2. Survey data indicate that Phase IIB funding is associated with improved outcomes in relation to clinical trials.
3. • Twenty-one percent of Phase IIB projects report having completed clinical trials, compared with 9 percent for other SBIR and STTR Phase II awards.
   • Conversely, only 5 percent of Phase IIB projects report having abandoned the clinical trials process, compared to 36 percent of other Phase II SBIR/STTR projects.⁷⁹
4. The National Cancer Institute (NCI) has been an active proponent of Phase IIB awards, and its Bridge program offers a potential new model for Phase IIB at NIH.
   • The NCI Bridge program is based on the NSF Phase IIB matching funds model.
   • It differs from NSF’s Phase IIB, in that it is focused—as with other Phase IIB programs at NIH—on providing support for clinical trials.
5. Both case-study companies and survey respondents had positive views of the impact of Phase IIB funding on their projects and their companies as a whole.
   • Threequarters of the 21 Phase IIB respondents indicated that the funding had made a “tremendous difference to the project.” The case studies conducted for this assessment support this view.⁸⁰
   • Phase IIB had a positive longterm impact on the company. Nearly two-thirds stated that the program had a substantial positive impact, and nearly 1 in 10 responded that the Phase IIB program had a “transformative effect” on the company.⁸¹
   • Comments from respondents indicate that Phase IIB supports a wide range of activities tied to a considerable variety of commercialization strategies and approaches.⁸²
6. Other aspects of the Phase IIB program.
   • Current policies on matching funds may exclude worthy projects from Phase IIB funding.

_______________

⁷⁹See Table 5-3.

⁸⁰See Appendix E (Case Studies).

⁸¹See Table 5-29.

⁸²See section on “Phase IIB” in Chapter 7 (Insights).

4. The flexibility of award patterns at NIH helps address the diverse needs of small innovative companies in the biomedical sector.

1. NIH continues to make extra-large awards. These awards conform to recent changes stipulating maximum award size, and in particular the new requirement that SBA apply waivers to specific topics.
   • NIH began to make extralarge Phase II awards in FY 2003, when 20 were made totaling $69 million.⁸⁵ The number of extra-large Phase II awards remained flat until 2009, when it grew to 30, and then to more than 40 in 2011-2012.⁸⁶
   • Phase II awards above $1.5 million accounted for $647 million in funding from FY2001-2014, equivalent to 430 additional awards at $1.5 million each. The cost of these awards grew as a share of total Phase II awards across the study period until the impact of 2011 SBIR reauthorization limits was felt in FY2013. The share peaked at 26 percent in 2012.⁸⁷
   • Respondents with larger awards account for more positive outcomes, based on preliminary data from the 2014 Survey. And a higher percentage of companies with large awards report large positive sales outcomes ($20 million or more).⁸⁸ These conclusions are based on small samples and should be viewed cautiously.
2. New entrants account for a substantial portion of awards at NIH, and the share of awards made to multiple award winners does not seem excessive.
   • New participants: New companies have accounted for more than 30 percent of companies applying for Phase I funding in almost all

_______________

⁸³See section on “Phase IIB” in Chapter 2 (Program Management).

⁸⁴See section on “Phase IIB” in Chapter 2 (Program Management).

⁸⁵See Table 3-5.

⁸⁶See Table 3-5.

⁸⁷See Table 3-4.

⁸⁸See Figure 3-9.

2. • years of the study period.⁸⁹ New companies account for between 20-25 percent of awards and 25-30 percent of successful FY20052013 applications during the study period. The access provided for new entrants to the program is appropriate.

   • Multiple award winners: The top 20 awards winners at NIH account for 7.7 percent of Phase I SBIR/STTR awards and 8.1 percent of Phase I funding.⁹⁰ They account for 9.6 percent of awards and 8.1 percent of funding for SBIR/STTR Phase II awards. This level of concentration is lower than that for other agencies. For example, at the Department of Defense (DoD) the top 20 winners account for 14.4 percent of awards and 14.3 percent of Phase I SBIR funding.⁹¹
3. Fast Track awards. NIH is increasing the number of Fast Track awards (combined Phase I and Phase II).
   • The number of Fast Track awards has grown steadily and in FY 2014 reached 70 awards.⁹²
   • The share of Fast Track awards as percentage of all regular Phase II + Fast Track awards has increased from 8 percent in 2005 to 24 percent in 2014.⁹³
4. Use of contracts is expanding at the National Cancer Institute (the largest IC). NCI has made a substantial shift from grants—the traditional funding mechanism—to contracts.
   • Approximately onethird of NCI SBIR/STTR funding is now awarded as contracts.
   • Aside from NCI there has been little change in contracting patterns across NIH over the study period.
   • NCI is focused on contracts in part because this mechanism leaves control of selection entirely with NCI. (NIH’s Center for Scientific Review [CSR] is not involved in selection.) Contracting by NCI also offers tighter control of the project itself, where payments can be linked to agreed milestones.⁹⁴

_______________

⁸⁹See Figure 4-2.

⁹⁰See Table 4-3.

⁹¹National Research Council, SBIR at the Department of Defense, Washington, DC: The National Academies Press, 2014, Table 2-3.

⁹²See Figure 4-23.

⁹³See Figure 4-24.

⁹⁴See section on “SBIR Phase I Contracts” in Chapter 4 (Awards).

5. NIH Institutes and Centers are pioneering new models of program management (e.g., the National Cancer Institute (NCI) and the National Heart, Lung, and Blood Institute (NHLBI)).

The SBIR/STTR programs at NIH are implemented by the ICs. Therefore, they have leeway to experiment with different ways of managing the program. Under the prevailing model at NIH, ICs simply add SBIR/STTR responsibilities to the other responsibilities of NIH technical staff, but NCI and NHLBI have developed new approaches.

1. The NHLBI model—professionalized consulting support and support across the entire SBIR/STTR timeline.

   The NHLBI model is based on the detailed recommendations of an assessment team that reported to the Institute Director in 2007. Implementation of the model is too recent to determine impacts. Key features include:⁹⁵

   • Dedicated fulltime consulting and advisory staff for SBIR/STTR applicants and winners, and for NHLBI program staff.
   • New preSBIR activities aimed at improving and expanding the pool of applicants through new accelerators/hubs located at universities and through innovative use of social media for outreach.
   • Improving the effectiveness of Phase I/Phase II in meeting agency priorities especially through more targeted Funding Opportunity Announcements (FOAs).
   • Better training and support for awardees, including participation in I-Corps.
   • Two postPhase II award programs: Bridge awards (like NCI, see below) and Small Market awards targeted at socially beneficial but less commercial technologies.
   • NHLBI has also commissioned the Research Triangle Institute (RTI) to support data collection and analysis for outcomes research.
2. The NCI model—replacement of part-time program officers with SBIR/STTR staff.

   The NCI model is closer in concept to the NSF model, in that it replaces NCI technical staff rather than providing them with consulting and support services. Key features include:⁹⁶

_______________

⁹⁵See section on “The NHLBI Model” in Chapter 3 (Program Initiatives).

⁹⁶See section on “The NCI Model: Building on the NSF Management Model” in Chapter 3 (Program Initiatives).

2. • Staff at the NCI SBIR Development Center run the SBIR/STTR programs. They develop solicitations, to a considerable extent manage the selection process (more so than for most ICs), and make funding decisions.
   • Staff are hired because they have a combination of scientific and entrepreneurial skills.
   • Bridge awards, though comprising a small share of the budget thus far, are a focus—a variant on Phase IIB, which highly recommends (though does not formally require) an investor match.
   • Bridge awards generated approximately a 2:1 match for NCI investments, as of November 2014.⁹⁷
   • NCI has also pioneered the introduction of the ICorp executive training program at NIH. This is a promising program in use at NSF that provides a coherent framework and program for entrepreneurs. Initial results are positive, and expansion is planned.⁹⁸
3. The new management models at NCI and NHLBI are encouraging but require data tracking and analysis.⁹⁹

   NIH and the participating Institutes should be commended for experimenting with promising new ways to manage the SBIR/STTR programs. The NCI and NHLBI models have different strengths and weaknesses—as does the standard model.

   • The new models are too recent to permit quantitative analysis.
   • NHLBI’s commitment to better data collection and outcomes research is commendable.
   • NCI’s existing data collection is helpful but also illustrates pitfalls (see above). Two-thirds of the additional investment reported from a reportedly successful 2010 investor forum may not finally eventuate.
   • Building on the 2014 Survey, additional data collection can be specifically tailored to provide comparisons of the program officer model of program management with the newer forms that have been recently introduced.

6. NIH is seeking to improve its data collection and tracking.

1. Data collection and tracking is necessary for effective program management.

_______________

⁹⁷See Figure 3-2.

⁹⁸See section on “I-Corps” in Chapter 3 (Initiatives).

⁹⁹See section on “New Management Models” in Chapter 3 (Initiatives).

2. • NIH faces broad challenges in tracking commercialization, at both the company and project levels. Companies move in and out of the program, and tracking is harder once companies have left. More generally, commercialization may come years after an award, and may involve multiple awards plus considerable additional funding. All this makes it difficult to assert that any specific outcome “results from” an SBIR/STTR award, particularly if developments are tracked for only a limited time.
   • Longer term tracking of outcomes is essential for effective program management: without outcomes data and analysis it is impossible to determine what is working and what is not. The previous report by the Academies (2009) recommended that NIH improve its tracking and evaluation of SBIR awards and in particular their outcomes.¹⁰⁰
3. Data collection at NIH is still limited.
   • Only outcomes from participants in the CAP are currently captured in the NIH Performance Outcome Data System.¹⁰¹
   • Tracking of CAP participants is limited in duration (18 months).
   • Tracking does not currently capture some important characteristics of company activities, such as interactions with FDA, clinical trials, and relationships with the Centers for Medicare & Medicaid Services (CMS).
   • Tracking addresses only commercialization outcomes. Other program objectives are not tracked.
4. NIH is taking steps to improve outcomes tracking and analysis.¹⁰²
   • NIH is working with SBA to utilize the anticipated commercialization database being developed by SBA. The new database will require that companies provide updated information on all awards whenever they apply for new funding, and also that they voluntarily report for 5 years after the end of the award.
   • NIH is also improving its own Performance Outcome Data System (PODS) to capture data on all awards, for areas not covered by the SBA database such as outcomes from clinical trials.
   • NHLBI is also focused on improved tracking as a prerequisite for better program management decisions. It has engaged RTI to provide further recommendations.¹⁰³

_______________

¹⁰⁰See section on “Data Collection, Tracking, and Analysis” in Chapter 2 (Program Management).

¹⁰¹See section on “Data Collection, Tracking, and Analysis” in Chapter 2 (Program Management).

¹⁰²See section on “Data Collection, Tracking, and Analysis” in Chapter 2 (Program Management).

¹⁰³See section on “NHLBI” in Chapter 3 (Program Initiatives).

7. NIH is seeking to gauge the effectiveness of its commercialization support initiatives.

1. NIH provides considerable commercialization support and training to its awardees.
   • NIH was among the first SBIR agencies to provide commercialization training and support to awardees.
   • The Commercialization Assistance Program has been operated by a third-party provider, LARTA, since its inception 10 years ago. It provides support to selected Phase II awardees.¹⁰⁴ According to the 2014 Survey about one-third of respondents reported that they had been through commercialization training.¹⁰⁵
   • The NICHE program, operated by a different thirdparty provider (Foresight), provides market research to a limited number of Phase I awardees. Providing early support for commercialization is an unusual feature of the NIH program and should be commended.¹⁰⁶
2. The effectiveness of this support is not yet established.
   • Data provided by NIH regarding outcomes from the CAP are not sufficient to determine its value. NIH is now moving to bring assessment of the program in-house, having relied on LARTA for analysis in the past.
   • Opinion on training was mixed. Thirty-eight percent of STTR respondents reported that it was valuable or very valuable; 26 percent that it was not at all or not very valuable.¹⁰⁷

8. A substantial gap remains between the end of Phase I and the beginning of funding for Phase II.

1. Sixty-eight percent of STTR respondents to the 2014 Survey indicated that they had experienced a gap.¹⁰⁸
2. The funding gap reportedly had a significant effect on company work on the funded project.

_______________

¹⁰⁴See section on “The Commercialization Assistance Program (CAP)” in Chapter 2 (Program Management).

¹⁰⁵See section on “Commercialization Training and Marketing” in Chapter 5 (Outcomes).

¹⁰⁶See section on “Niche Program for Phase I Participants” in Chapter 2 (Program Management).

¹⁰⁷See Table 5-15.

¹⁰⁸See section on “Funding Gaps and Award Timelines” in Chapter 2 (Program Management).

2. • Thirty-one percent of STTR respondents indicated that they had stopped work altogether.
   • A further 57 percent reported that they had slowed their work.
3. Concerns about funding gaps are among the most frequently highlighted issues in the open-ended responses solicited in the 2014 Survey.¹⁰⁹
4. While NIH does permit companies to “work at their own risk” between SBIR/STTR Phase I and II, this is insufficient to address the challenges facing many small companies:
   • Small companies may not have the resources to pay their staff well in advance of NIH payments.
   • Some survey respondents did not know about this opportunity, as indicated by the fact that they asked for policy changes to provide this in the future.
5. NIH does not have a gap funding program, such as those offered by some components at the DoD.¹¹⁰
6. NIH is funding more Fast Track awards, which can solve the issue for those projects.¹¹¹

VI. STTR

1. STTR is meeting the program objectives defined in the Small Business Administration’s Policy Guidance for STTR.

1. STTR is stimulating technological innovation, as evidenced by the substantial knowledge effects identified in Chapter 5 and the relevant case studies referenced in Chapter 7.
2. STTR fosters cooperative R&D between universities and other research organizations and industry.
   • Seventy-eight percent of STTR survey respondents report that the PI was a faculty member of the partnering research institution.¹¹²
   • Only 1 percent of STTR awards had no research institution linkage, compared to 35 percent for SBIR respondents.¹¹³

_______________

¹⁰⁹See section on “Funding Gaps and Award Timelines” in Chapter 2 (Program Management).

¹¹⁰National Research Council, SBIR at the Department of Defense, Washington, DC: The National Academies Press, 2014, p. 180.

¹¹¹See section on “Fast Track” in Chapter 4 (Awards).

¹¹²See Table 5-20.

¹¹³See Table 5-20.

2. • Casestudy companies indicated that STTR had helped to bridge the gap between research labs and commercial activities (e.g., Stratatech).
3. STTR is meeting the objective of supporting the commercialization of federally funded technologies.
4. STTR survey respondents report an identical rate of reaching the market with their products compared to SBIR, at 49 percent of responding projects,¹¹⁴
5. Fewer STTR respondents report no additional investment in the technology aside from program funds (11 percent against 19 percent for SBIR).¹¹⁵
6. STTR projects report receiving additional investment from venture firms at essentially the same rate as SBIR respondents.¹¹⁶

2. The STTR program at NIH is administered as an adjunct to the much larger SBIR program.

1. NIH staff discussions confirm that the agency runs both SBIR and STTR as a single program, with minor differences in participation rules.
2. Solicitations for STTR and SBIR are announced jointly.

3. Outcomes from STTR are broadly similar to those from SBIR.

1. Participation rates for research institution staff are broadly similar for SBIR and STTR (with one important exception discussed below).¹¹⁷
2. Outcomes for commercialization and for knowledge effects show minimal differences between SBIR and STTR.¹¹⁸

4. Companies in some cases do utilize STTR differently from SBIR.

1. STTR rules permit PIs to work less than 51 percent time on the funded project. SBIR does not. As a result, PIs who wish to retain a half-time position or more at a research institution find STTR a helpful option.
2. STTR also permits a larger share of the award to be subcontracted to the research institution. Companies sometimes find this useful when they need to utilize specialized equipment or skill sets.

_______________

¹¹⁴See Table 5-7.

¹¹⁵See Table 5-12.

¹¹⁶See Table 5-13.

¹¹⁷See Table 5-20.

¹¹⁸See “Commercialization” and “Knowledge Effect” sections in Chapter 5 (Outcomes).

RECOMMENDATIONS

Although the NIH SBIR/STTR programs generate substantially positive outcomes, the committee has identified a series of recommendations to improve its processes and outcomes. The order of these recommendations reflects the relative emphasis of the committee. The first set of recommendations address the challenge of drawing more women- and minority-owned companies into the SBIR and STTR programs. The second set of recommendations focuses on ways to improve the commercialization of SBIR/STTR projects. The final three sets of recommendations address how NIH can improve the operation of their SBIR and STTR programs. They examine the Phase IIB and other funding mechanisms beyond Phase I/I; ways to improve the monitoring, assessment and reporting on the programs; and overall changes in management practices to improve program operations.

I. Addressing Underserved Populations

NIH should immediately examine past and current efforts to address the Congressional mandate to foster the participation of underserved populations in the SBIR/STTR programs, examine and report on best practices, develop an outreach and education program aimed at expanding participation of under-served populations, create benchmarks and metrics to relate the impact of such activities.

1. Quotas are not recommended. It is not recommended that NIH develop quotas for inclusion of selected populations into the SBIR/STTR programs, because of the potential problems that this might entail, such as raising issues of fairness and lack of transparencies with the selection process. At the same time, it is important that steps be taken to improve the current situation.¹¹⁹

2. NIH should develop new benchmarks and metrics.¹²⁰

1. Improve participation metrics: The SBIR/STTR Program Office should work to improve metrics for benchmarking the participation of underserved populations, developing and publishing benchmarks based on a defensible analysis of existing data.
2. Disaggregate benchmarks: Measures of the participation of socially disadvantaged groups must be disaggregated by race or ethnicity, and attention focused on the congressional intent to support “minority” participation. The current SBA definition of “socially and economically disadvantaged” is not sufficient to meet this objective.

_______________

¹¹⁹See Chapter 6.

¹²⁰See Finding II-A.

3. Customize benchmarks: Points of reference should be developed separately (though perhaps drawing on a shared methodology) for women and minorities. Benchmarks should address key questions that would include the following metrics, all of which should include both absolute levels and trends over time:
4. • Shares of applications from companies owned by women and minorities.
   • Shares of applications with woman and minority principal investigators.
   • Shares of Phase I awards to companies owned by women and minorities.
   • Shares of Phase I awards with woman and minority principal investigators.
   • Shares of Phase II awards to companies owned by women and minorities.
   • Shares of Phase II awards with woman and minority principal investigators. The field of degree or other STEM area classification of the principal investigator.
   • The participation rates of women and minorities on review panels, reflecting the fact that panel participation has been cited by some as a path that led them to company formation and further SBIR involvement including application to the program.
5. Track related program operations: Metrics should also track related program operations including outreach efforts (See below).

3. NIH should develop an outreach and education program focused on expanding participation of underserved populations.¹²¹

1. This will require the provision of agency resources and senior staff time and should be a high priority for the program, because the existing efforts are not sufficient. NIH will need to make concerted efforts in this area.
2. Develop enhanced outreach strategy: NIH should develop a coherent and systematic outreach strategy that provides for cost-effective approaches to enhance recruitment of woman- and minority-owned companies, as well as female and minority PIs, developed in conjunction with other stakeholders and with experts in the field. Outreach efforts should aim to expand SBIR/STTR awareness among potential applicants from underserved demographics.

_______________

¹²¹See Finding II-B.

3. Establish a graduate intern program for qualified women and minorities with the goal of providing the interns with first-hand experience in applying to SBIR/STTR. The internship could operate with salaries paid for by NIH, with their participation allowed in CAP and other business training, and with the interns potentially provided as additional resources available to awardees who make the best case.
4. Provide management resources: NIH should provide significant management resources, because improving participation is likely to be both difficult and a long-term effort.
5. Designated staff: NIH should consider designating a senior staffer to work exclusively on improving women and minority participation in order to improve reporting and the deployment of new initiatives in this regard.
6. Add-ons to existing outreach activities are not sufficient. There is no evidence that a panel at the National SBIR conference or at AdvaMed has attracted significant numbers of new participants into the program. Focused and extensive outreach activities will be needed.

4. NIH should review selection procedures and remove any identified biases in the selection process.¹²²

1. Review selection processes: NIH should review internal award and selection data and processes to address questions arising from disparities between Phase I and Phase II awards, and divergent success rates, for selected populations. The goal is to ensure that there are no biases in the selection process that are adversely affecting the selection of women and minorities.
2. Monitor selection processes: NIH should ensure that patterns of applications, awards, and success rates are monitored going forward and are reported out annually.

II. Improving Commercialization Outcomes

The NIH SBIR/STTR programs are focused on commercialization, and findings of this report indicate that it is doing so with considerable success despite the substantial barriers facing the commercialization of biomedical research. However, it is worth considering possible improvements.

_______________

¹²²See Finding II-A.

1. NIH should continue to address challenges that conducting clinical trials pose for the commercialization of SBIR/STTR technologies.¹²³

NIH should provide improved support for awardees in meeting the challenges in funding clinical trials:

1. NIH should consider options for supporting companies in their approach to clinical trials more effectively. While preliminary evidence suggests that Phase IIB is working to support companies through clinical trials, NIH should consider whether adjustments are warranted.¹²⁴
2. NIH should provide improved support for awardees in dealing with the FDA.¹²⁵
   • NIH should seek to provide ongoing expert consulting to awardees in relation to FDA requirements.
   • NIH should provide more and better detailed briefings on FDA requirements on the NIH SBIR/STTR website.
   • NIH should explore ways in which the NIH SBIR/STTR programs could create better linkages directly to FDA that would benefit awardees.
   • NIH should consider whether a standard briefing on the FDA process should become part of the SBIR/STTR programs for all new awardees.
3. NIH should help awardees to find strategic partners.
   • NIH can leverage its substantial convening power as the premier biomedical research organization worldwide in this effort. The NCI Investor Conferences are a promising initiative that could be expanded to other biomedical subsectors, could be more frequent, and could provide more systematic connection beyond simple conference attendance.
   • NIH should review DoD efforts to build searchable databases of awardees and their activities, for possible adoption. Providing more frequent and better information to potential investors may help to increase the pipeline of projects with funding for clinical trials.
   • NIH should in part refocus its commercialization strategy toward strategic partners: The limited funding for seed and startup projects from U.S. venture capital in general, and the low numbers of NIH firms that report venture capital funding, suggests that NIH should not

_______________

¹²³See Finding I-G.

¹²⁴See also Recommendations III on Phase IIB.

¹²⁵See Finding V-G.

2. NIH should review the effectiveness of its commercialization support and training initiatives.¹²⁶ NIH should be commended for providing commercialization support on a regular basis to both Phase I and Phase II SBIR/STTR awardees. NIH was one of the first agencies to provide this support, and it has been a feature of the program for more than 10 years. Building on this:

1. NIH should consider whether current commercialization support is effective: Evidence for outcomes provided by LARTA is inconclusive. Evidence from participants is mixed. It seems plausible that LARTA is effective in some cases, and that NIH should determine where this program is effective and where it is not.
2. NIH should look to ICs to identify best practices. NIH should view the management initiatives at NCI, NHLBI, and other ICs as offering potential insights into best practices.
3. The NIH Program Office should review initiatives undertaken by ICs regularly to identify apparent successes (such as the I-Corps program) for potential replication and to learn from failures. ICs can be laboratories for NIH as whole, if they are supported, monitored, and evaluated effectively.

III. Phase IIB and Other Funding Mechanisms Beyond Phase I/II

1. NIH should continue to operate the Phase IIB program and consider expanding its size within the context of a more flexible approach.¹²⁷

1. Preliminary data suggest that the Phase IIB program has had a positive impact on the commercialization of SBIR/STTR-funded research.

_______________

¹²⁶See Finding V-G.

¹²⁷See Finding V-C.

2. Respondents to the 2014 Survey support retention of the program.
3. Relatively few projects currently benefit from the program because of its limited use. NIH might therefore consider whether additional funds should be made available for an expansion of Phase IIB support.
4. But Phase IIB should not come to dominate the program financially. Given the costs involved ($3 million per award), NIH should be cautious about the impact of expansion on the availability of funding for other aspects of the SBIR/STTR programs.

2. NIH should consider a more flexible approach to funding Phase IIB awards.¹²⁸

1. A more flexible approach would avoid a “one-size fits all” funding approach.
   • It could also provide more effective support for more demanding projects.
   • As the cost of clinical trials varies substantially by project, it seems unnecessary to constrain support to an exact amount.

3. NCI and NHLBI should consider modifying the criteria that define an acceptable third-party match for Phase IIB purposes. (V-E)

1. Explore use of in-kind contributions: NCI/NHLBI may wish to explore allowing the limited use of some specified in-kind contributions (e.g., the cost of drugs used in the trial) as part of the matching funds.
2. Review other types of matching commitment: The original Phase IIB program was developed at NSF. However, the NSF program is much more flexible with regard to matching funds: for example, sales and company revenues can be counted. NCI/NHLBI should review projects that have been excluded from the program to determine whether different matching criteria would be a better fit.

IV. Improving Monitoring, Evaluation, and Assessment

The development of more careful monitoring and more sophisticated analysis of key variables is necessary to improve program outcomes. Although NIH recognizes the need for better data and is working to develop improved tracking mechanisms, more remains to be done in this area.

_______________

¹²⁸Ibid.

1. NIH should improve current data collection approaches and methodologies.¹²⁹

Data collected through the current process are a good start but are far from sufficient to underpin a data-driven program.

1. NIH should improve data collection and organization:
   • NIH should collect outcomes data and improve program evaluation, management, and outcomes. This data collection effort should address the entire range of congressionally mandated outcomes, not only commercialization, and should be extended to other aspects of the program, including demographic data for applicants and awardees.
   • Data should also be collected about other aspects of the program. For example, NIH should know the extent to which proposals are in reality constrained by topic boundaries. There are no current data on this.
   • Further, NIH should know the frequency with which review panel funding advice is being overruled in different areas; they should know the composition of review panels; they should know the success rates of applicants at the level of gender and ethnicity; they should know the rates of resubmissions and their success rates.
   • NIH should develop a dataset that can provide a basis for longitudinal analysis.
2. NIH should expand tracking of commercialization outcomes:
   • NIH should track commercialization outcomes in ways similar to the now widely accepted methodology developed for the SBIR studies by the Academies.¹³⁰ This approach focuses on multiple metrics in order to provide a deeper and more nuanced basis for analysis.¹³¹
   • Although NIH tracks outcomes for the participants of its Commercialization Assistance Program, this tracking excludes a significant number of projects, and ends well before significant positive outcomes are likely to occur.
   • The data collection effort now under way at SBA may help NIH build a tracking and analysis capability. And NIH is already aware that NIH-specific metrics will need to be captured separately.

_______________

¹²⁹See Finding V-F.

¹³⁰National Research Council, An Assessment of the Small Business Innovation Research Program: Project Methodology, Washington, DC: The National Academies Press, 2004.

¹³¹See Chapter 1 (Introduction) and Appendix A (Methodology).

3. NIH should collect enhanced demographic data.
4. • NIH should take immediate steps to improve its collection of demographic data about applicants and awardees.
   • NIH should extend its collection of the demographics of company ownership to show which of SBA’s socially and economically disadvantaged categories an applicant belongs to. In addition, applicants should be asked the same demographic questions about the principal investigator.
5. NIH should also develop and adopt a more systematic and critical approach to the use of detailed case studies and success stories.
   • Case studies—written by NIH staff or third parties—can describe the roles played by SBIR/STTR awards, the challenges faced by small businesses, insights into needed improvements in process, lessons learned, and other important information not available elsewhere about program impacts.
   • Success stories—provided by the companies—can provide inspiration and promote interest in the program, but should not be regarded as sole evidence of program effectiveness.
6. NIH should take advantage of modern information management and data visualization tools both in its data collection effects, for communication with companies about program activities and operations, and to facilitate networking of program participants.
   • NIH should explore ways to use new technology such as social media to collect more current data. SBIR/STTR companies—like “customers” in other markets—are an important source of information about program strengths and weaknesses. This knowledge is currently not systematically included in internal program evaluation by NIH’s SBIR/STTR programs.
   • NHLBI efforts to use social media may provide a worthwhile template for expanding activities already under way through the NIH Program Office.
7. NIH should develop feedback tools: NIH should develop pathways to provide ongoing feedback from companies about program activities and operations. These should include various electronic communication tools.
8. NIH should improve networking: Similarly, NIH should consider developing mechanisms (like electronic tools) through which recipients can share information about their SBIR/STTR projects, helping them both

2. NIH should improve the utilization of outcomes data. As NIH starts to collect effective outcomes data, it should ensure that these data are systematically employed to guide program management.

1. NIH should develop a plan for data analysis: NIH should seek to develop a more sophisticated approach to analyzing and applying the data that are already collected.
2. NIH should evaluate data to identify factors that tend to encourage successful transitions between Phases, into Phase IIB, and then into full-scale commercialization. Such an approach could identify key issues for program management.
3. NIH should develop metrics to gauge how well it is meeting the congressionally mandated objectives for the SBIR/STTR programs.
4. NIH should undertake regular analysis of data. By collecting more and better data on outcomes and participation, NIH will be positioned to undertake regular analysis—either internally or with third-party help—on key program management issues, such as:
   • What is the longterm impact of commercialization training, partnership programs, and other commercialization supports?
   • Is Phase IIB simply picking successful companies or is it at least, in part, causing companies to be successful?
   • How well do NIH selection processes predict eventual successful projects?
   • How effectively do initiatives like direct to Phase II, ICorps, and the NCI/NHLBI management models improve outcomes?
5. NIH should recognize the impacts of data collection and analysis. In some cases, simply measuring something more closely can provoke needed action. Closely tracking the participation of women and minorities could help assure a fair process and surface problem issues early, when they can be most easily corrected.

3. NIH should prepare an SBIR/STTR Annual Report to the NIH Director and Congress.

1. New annual report: Imposing new reporting burdens on the NIH SBIR/STTR programs is not without cost, but an annual report to Congress could improve transparency and provide a coherent point of discussion for stakeholders.

2. Although the precise details should be left to the agency, NIH should consider including the following areas of program operations:
3. • Program Inputs: This relates to budget and related resources put into the program’s front end.
   • Program Outputs: This includes initiatives developed, outreach activities, competitions/solicitations held, applications/proposals received, awards and contracts made.
   • Program Results:
     • Early outcomes: This includes progress measures such as attraction of additional funding by funded companies, formation of partnerships, early sales, patents, publications, and licensing agreements.
     • Intermediate outcomes. This includes resulting company growth in sales, employment, and knowledge benefits through the citation of patents and publication.
     • Long-term impacts. This includes measures of the economic return on investment, improvements in national innovation capacity, gains in strength of small businesses attributed to the programs, and growth in the numbers and percentage of women and minority businesses comprising the SBIR/STTR client base.
   • Qualitative review, based on improved use of case studies, as well as success and failure stories and social media.
   • Impact assessment, focused on the extent to which NIH meets congressional objectives for the program.
   • Summary conclusions, including prospective views on program activities and improvements for the coming year.
4. The new Annual Report should replace all existing reporting required from the program.

V. Improving Program Management

The following recommendations are designed to improve program operations in ways that should enhance the program’s ability to address some or all of its objectives.

1. NIH should improve its application review system.¹³²

1. In consultation with an expert in this process, NIH should convene a high-level task force to improve the consideration of commercial potential in the selection process for SBIR/STTR applications.
   • The task force should include the Director of CSR, other CSR staff, the SBIR/STTR Program Manager, the Director of the Office of Extramural Research (OER), and selected staff from ICs recommended by the SBIR/STTR Program Manager.
   • The task force should provide a report within 6 months to the Director of NIH with an assessment of the SBIR/STTR review process and recommendations for improvement.
2. Some of the changes that should be considered by the task force include:
   • Guarantee of commercial expertise for all SBIR/STTR proposal reviews (especially Phase II). Every reviewed proposal should receive expert commercialization assessment. Commercialization reviews should be made based on published selection criteria that are designed to draw out the applicant companies’ commercial thinking and planning. The effect of contracting on commercialization and company growth should also be analyzed.
   • More agile approaches to review that would streamline the process sufficiently to provide timely debriefs for resubmission at the next deadline.
   • Better prereview briefing for all panelists to ensure that they fully understand the SBIR/STTR programs. In particular, to ensure that they understand that the full commercialization pathway is considered part of “innovation” by NIH.
   • Exploring opportunities for a more interactive process whereby companies have an opportunity to provide a brief further explanation or rebuttal to reviewer comments during the process. If feasible,

_______________

¹³²See Finding V-A.

2. NIH should address the funding gap between Phase I and II awards.¹³³ Despite some efforts, it is apparent from the case studies and survey responses that funding gaps between the NIH Phase I and Phase II awards are a problem for small businesses. Although recognizing that some delays are unavoidable—for example, Congress has at times not provided definitive budgets until well into the fiscal year—NIH should take steps to support a smooth transition.

1. NIH should address the funding gap between Phase I and Phase II awards. Although data from the NIH Annual Report to SBA are not definitive on this point, there is evidence to suggest that significant improvements could be made.
2. NIH should improve awareness of the “work at own risk” process. Some survey respondents recommended that something similar be implemented, suggesting that they do not know how this works at NIH.
3. NIH should streamline review so that debriefs can be provided more rapidly. NIH is aware of this problem and is working to address it, but insufficient progress has been made to date.
4. NIH should encourage more firms to apply for Fast Track. Some survey respondents indicated that they thought success in applying for Fast Track was essentially impossible, but the awards data suggest otherwise.
5. NIH should make it clear that firms are able to resubmit for Phase I if they fail to get a Fast Track.
6. NIH should also consider additional ways to provide financial support during funding gaps. Such support might for example be available to top scoring Phase II proposals as a supplement to their Phase I award.

_______________

¹³³See Finding V-H.

C. NIH should track and evaluate new program management initiatives.

1. Use of contracts:¹³⁴ The shift toward contracts at NCI reflects the new management approach adopted there. Although understandable, we believe that the NIH SBIR/STTR programs should remain primarily grants-based (i.e., investigator initiated) and that the use of contracts should be limited. NCI should track the new approach carefully and should seek to determine whether using contracts generates more or less positive outcomes.
2. Investigator initiated research: SBIR/STTR funding at NIH is now more closely targeted at IC priorities, in contrast to the traditional model of investigator-initiated research. NIH should assess whether targeting generates improved or less favorable outcomes for the SBIR/STTR companies and programs overall, and also to what extent awards are still being made to investigator-initiated projects. As part of this assessment, NIH should gather data on the extent to which the traditional model of investigator-initiated research is under pressure in the NIH SBIR/STTR programs.

_______________

¹³⁴See Finding V-D.