SBIR/STTR at the National Institutes of Health (2015) / Chapter Skim
Currently Skimming:
5 Quantitative Outcomes
5 Quantitative Outcomes
Pages 140-184
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 140... ...
; however, data from these sources are not yet available. Thus, the analysis of outcomes in this report is based primarily on the 2014 Survey by the Academies, which tried to survey all SBIR and STTR Phase II awardees for the period FY2001-2010 inclusive.2 A detailed description of the methodology underlying the 2014 survey is provided in Appendix A of this report (see Box 5-1)
|
From page 141... ...
COMMERICIALIZATION As with our other reports on the SBIR program, we have adopted a broad view of commercialization, taking it to include additional investments from outside the SBIR/STTR programs as well as sales and licensing revenues. In addition, given the long time to market required for many life sciences technologies, we have been careful to include a range of benchmarks and metrics, having determined that no single metric can appropriately capture such a broad concept.
|
From page 142... ...
Overall, just less than one-half of projects reported some sales or licensing revenues, and a further 25 percent expected sales in the future. These data are similar to those generated by the previous survey of NIH SBIR-only awardees by the National Research Council (NRC)
|
From page 143... ...
Although the United States has historically been a leader in venture capital and angel investment, these are not the only or even the primary sources of additional investment funding for NIH SBIR/STTR projects. Overall, more than 80 percent of respondents indicated that their project received additional investment in the technology related to the surveyed project.7 As 6 See National Research Council, An Assessment of the SBIR Program, Washington, DC: The National Academies Press, 2008.
|
From page 144... ...
These data highlight funding challenges for these companies because the cost of Phase III clinical trials has recently been estimated at $26,000 per patient.8 Phase III trials can require the enrollment of more than 1,000 patients.9 Of the 470 respondents that reported additional funding, 44 percent was from private-sector sources, 9 percent was from venture capital sources, and 14 percent was from angel and other private equity investors. Twenty-one percent reported strategic investments from partners, which is especially important in the context 8 Jon Hess, "Clinical Operations: Accelerating Trials, Allocating Resources and Measuring Performance," Cutting Edge Information, October 12, 2014.
|
From page 145... ...
SOURCE: 2014 Survey, Question 36. TABLE 5-1 Additional Funding Received by Funding Mechanism and Amount Percentage of Responses NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees None ($0)
|
From page 146... ...
Survey responses indicate that this was true for 46 percent of projects.10 For these companies, the road to successful conclusion of clinical trials is very challenging (Table 5-3)
|
From page 147... ...
Nine percent of projects had completed clinical trials (21 percent of Phase IIB projects)
|
From page 148... ...
Figure 5-5 shows that of the 21 Phase IIB responses, one-third thought the funding was not sufficient even to complete preparation for clinical trials, and a further 29 percent thought it was sufficient only for completion of those preparations. Five percent (one respondent)
|
From page 149... ...
. NOTE: N=226 (projects requiring FDA approval)
|
From page 150... ...
As shown in Table 5-19 in the annex section of this chapter, two-thirds of Phase IIB responses reported publishing at least 3 articles and 21 percent reported publishing at least 10. 13 2014 Survey, Question 39.1.2.
|
From page 151... ...
TABLE 5-5 Number of Patents Received Related to Surveyed Project Percentage of Responses NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees 0 47.0 47.2 46.3 25.9 1 23.7 23.0 27.5 22.2 2 12.0 12.1 11.3 7.4 3 or 4 8.2 7.6 11.3 22.2 5 to 9 5.2 5.9 1.3 14.8 10 or more 4.0 4.3 2.5 7.4 1 or more 53 52.8 53.8 74.1 Mean 1.63 1.71 1.23 3.11 Median 1 1 1 2 BASE: TOTAL RESPONDENTS 502 422 80 27 ANSWERING QUESTION SOURCE: 2014 Survey, Question 38.1.
|
From page 152... ...
. COUNTERFACTUALS Because there is no available matched set of companies that did not receive SBIR/STTR Phase II funding at precisely the point in time that surveyed companies did receive funding, it is not possible to develop an appropriate control group against which to measure impacts (see discussion of the Academies efforts to do so in Appendix A)
|
From page 153... ...
or adjunct faculty 38.7 38.2 41.1 35.7 member(s) worked on this project in a role other than PI Graduate students worked on this project 22.3 21.3 27.8 21.4 The technology for this project was 16.0 14.0 26.7 17.9 licensed from an RI The technology for this project was 20.2 17.1 36.7 17.9 originally developed at an RI by one of the participants in this project An RI was a subcontractor on this 37.4 35.5 47.8 53.6 project None of the above 29.7 35.1 1.1 32.1 BASE: TOTAL RESPONDENTS 569 479 90 28 ANSWERING QUESTION SOURCE: 2014 Survey, Question 71.
|
From page 154... ...
effort, while about 34 percent indicated that they were funding more than one-half.16 This picture is reinforced by data on sources of company revenues. Thirty-four percent of responding companies reported zero SBIR/STTR revenues, while about 27 percent reported receiving more than onehalf of the company's revenues from SBIR/STTR.17 The survey also asked about the overall impact of SBIR/STTR on the company.
|
From page 155... ...
QUANTITATIVE OUTCOMES 155 PosiƟve (35%)
|
From page 156... ...
This Annex provides addi tional details of the commercial outcomes of the NIH SBIR/STTR programs, as well as quantitative outcome measures related to stimulating technological innovation. SOURCES OF DATA Although NIH was an early adopter of survey-driven outcomes research, the agency has not thereafter led the way on tracking outcomes.
|
From page 157... ...
21 Phase IIB is an award made to some firms at the end of Phase II. It supports those working to complete clinical trials.
|
From page 158... ...
Although we have already cautioned against overuse of this metric -- warnings that are applicable to the wide range of metrics a dopted for use in the current assessment -- sales and revenues are still important considerations.23 22 Foran overview of the commercialization metrics and survey used in this study, see Appendix A 23 Similarwarnings can be found in the 2009 report on the NIH SBIR program -- National Research Council, An Assessment of the SBIR Program at the National Institutes of Health, 2009, 81.
|
From page 159... ...
for the 2014 Survey. The 2005 Survey of SBIR companies found that 24 percent of respondent companies had no sales and expected none, 19 percent had no sales but expected sales in the future, and 57 percent had already generated sales from the surveyed project.24 Amount of sales and licensing revenues.
|
From page 160... ...
TABLE 5-8 Distribution of Total Sales Dollars, by Range STTR SBIR Phase IIB Under $100,000 34.1 40.6 23.1 $100,000-$499,999 22.0 22.6 23.1 $500,000-$999,999 17.1 10.4 $1,000,000-$4,999,999 14.6 14.2 23.1 $5,000,000-$9,999,999 6.1 7.7 $10,000,000-$19,999,999 2.4 2.3 $20,000,000-$49,999,999 2.4 1.9 7.7 $50,000,000 or more 1.9 15.4 BASE: ANY SALES RESULTING 38 212 13 FROM THE PROJECT NOTE: See Pie Chart in Figure 5-2. SOURCE: 2014 Survey, Question 34.
|
From page 161... ...
TABLE 5-10 Number of Employees at Time of Award Percentage of Companies Responding NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees 0 0.5 0.3 1.6 1 3.7 3.1 6.8 2 7.1 6.1 12.5 5.7 3 or 4 19.7 19.5 20.8 5 to 9 29 27.7 35.9 35.4 10 to 19 19.2 20.1 14.1 24.9 20 to 49 12.5 14 4.4 12.4 50 to 99 3.6 3.7 3.1 8.6 100 or more 4.8 5.5 0.8 12.9 Mean 19 21 10 33 Median 7 8 5 12 BASE: TOTAL RESPONDENTS 418 354 64 17 ANSWERING QUESTION NOTE: Answers from individual respondents were aggregated and averaged for each company, and company responses are reported above. SOURCE: 2014 Survey, Question 14.1.
|
From page 162... ...
27 There has also been interest in the sources of additional funding for high-tech innovation. The United States has historically been a leader in venture capital and angel investment.
|
From page 163... ...
One percent reported receiving $50 million or more in additional funding; 9 percent reported receiving $5 million or more. Given that the cost of Phase 3 clinical trials has recently been estimated at $26,000 per patient29 and that can require the enrollment of more than 1,000 patients,30 the funding challenge for SBIR/STTR companies is immediately apparent.
|
From page 164... ...
. TABLE 5-13 Distribution of All Reported Additional Investment Funding by Source of Funds Percentage of Responses NIH SBIR STTR Phase IIB Source of Funding Total Awardees Awardees Awardees Non-SBIR/STTR Federal Funds 25.2 21.9 41.4 28.6 Private Investment: U.S.
|
From page 165... ...
Phase IIB respondents tended to see the training as less valuable, perhaps because they were already farther down the 31 2014 Survey, Question 49.
|
From page 166... ...
Forty-three percent of respondant companies reported that their company had at least one full-time marketing staff.32 Conclusions: Commercialization at the Company Level Evidence from the 2014 Survey provides useful insight into the commercialization record of SBIR/STTR companies at NIH, on a number of dimensions. The data confirm that a substantial percentage of projects do indeed commercialize through sales of products or services and/or through the receipt of additional development funding.
|
From page 167... ...
Many Phase II projects are not yet ready for the marketplace at the end of the award period, especially given the need for regulatory compliance.34 Three- quarters of all respondents reported receiving less than $1 million in additional funding. One percent reported receiving $50 million or more in additional funding; 9 percent reported receiving $5 million or more.
|
From page 168... ...
Overall, about two-thirds of companies (and more than 80 percent of Phase IIB recipients) reported the award at least one patent related to any SBIR/STTR-funded technology; 13 percent reported at least 10 related patents (see Table 5-17)
|
From page 169... ...
QUANTITATIVE OUTCOMES 169 TABLE 5-17 Number of Patents Related to All Company SBIR/STTR Awards Percentage of Companies Responding NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees 0 33.6 30.8 48.8 18.3 1 14.0 16.0 3.2 12.2 2 10.2 9.1 16.2 3 8.3 8.6 6.9 25.4 4 5.8 5.0 10.3 9.1 5 to 9 15.1 16.2 9.0 10.7 10 or more 12.9 14.2 5.6 24.4 1 or more 66.4 69.2 51.2 81.7 Mean 4.74 5.22 2.11 5.40 Median 2.00 2.00 1.00 3.00 BASE: TOTAL COMPANIES 409 347 63 16 ANSWERING QUESTION SOURCE: 2014 Survey, Question 11. TABLE 5-18 Patents Awarded Related to Surveyed Project Percentage of Responses NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees 0 47.0 47.2 46.3 25.9 1 23.7 23.0 27.5 22.2 2 12.0 12.1 11.3 7.4 3 or 4 8.2 7.6 11.3 22.2 5 to 9 5.2 5.9 1.3 14.8 10 or more 4.0 4.3 2.5 7.4 1 or more 53 52.8 53.8 74.1 Mean 1.63 1.71 1.23 3.11 Median 1 1 1 2 BASE: TOTAL RESPONDENTS 502 422 80 27 ANSWERING QUESTION SOURCE: 2014 Survey, Question 38.1.
|
From page 170... ...
Overall 42 percent reported publishing three or more related papers. The median number of publications for Phase IIB respondents was five.
|
From page 171... ...
Although far from a perfect metric, we believe these data provide a preliminary indication of the connections between specific universities, university systems, and the NIH SBIR/STTR programs. Finally, for 85 percent of companies in the sample, at least one founder had an academic background (see Table 5-22)
|
From page 172... ...
through the patent system. About two-thirds of companies reported receipt of at least one patent based on their work under SBIR/STTR contracts, while 53 percent reported receipt of at least one patent related to the surveyed project only specifically.
|
From page 173... ...
worked on this project in a role other than PI Graduate students worked on this project 22.3 21.3 27.8 21.4 The technology for this project was 16.0 14.0 26.7 17.9 licensed from an RI The technology for this project was 20.2 17.1 36.7 17.9 originally developed at an RI by one of the participants in this project An RI was a subcontractor on this 37.4 35.5 47.8 53.6 project None of the above 29.7 35.1 1.1 32.1 BASE: TOTAL RESPONDENTS 569 479 90 28 ANSWERING QUESTION NOTE: Responses do not sum to 100 percent because respondents could select more than one answer. SOURCE: 2014 Survey, Question 71.
|
From page 174... ...
174 SBIR/STTR AT THE NATIONAL INSTITUTES OF HEALTH TABLE 5-21 University Participants Mentioned by Four or More Respondents Research Institution Number of Mentions University of Michigan 14 Duke University 10 Johns Hopkins University 8 Massachusetts Institute of Technology 7 University of North Carolina Chapel Hill 7 University of Pittsburgh 7 University of Virginia 7 Indiana University 6 Pennsylvania State University 6 University of Florida 6 University of Illinois Chicago 6 University of Massachusetts Medical School 6 University of Utah 6 Vanderbilt University 6 Children's Hospital Boston 5 MD Anderson Cancer Center 5 Medical University of South Carolina 5 Oregon Health & Science University 5 Texas A&M University 5 UC San Francisco 5 University of Arizona 5 University of Kentucky 5 Case Western Reserve University 4 Cornell University 4 Dartmouth College 4 Harvard University 4 Mayo Clinic 4 University of California, Berkeley 4 University of Connecticut 4 University of Louisville 4 University of Minnesota 4 University of New Mexico 4 University of Pennsylvania 4 Washington University of St Louis 4 SOURCE: 2014 Survey, Question 60.
|
From page 175... ...
Seventeen percent of respondent companies were founded because of the SBIR/ STTR programs, and a further 27 percent were formed in part because of the program (see Table 5-24)
|
From page 176... ...
Thirty-four percent of companies reported zero SBIR/STTR revenues, while 27 percent reported receiving greater than one-half of revenues from SBIR/STTR. Three percent were entirely dependent on SBIR/STTR (see Table 5-26)
|
From page 177... ...
QUANTITATIVE OUTCOMES 177 TABLE 5-25 Percentage of R&D Effort Funded by SBIR/STTR Percentage of Company Responses NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees 0% 31.5 32.9 24.4 35.4 1-10% 10.5 10.5 10.4 25.8 11-25% 12.8 13.1 11.2 1.0 26-50% 11.5 11.0 14.2 1.4 51-75% 14.7 14.1 18.1 19.1 76-100% 18.9 18.4 21.7 17.2 Mean 33 32 38 29 Median 18 18 38 5 BASE: TOTAL COMPANIES 421 355 66 17 ANSWERING QUESTION SOURCE: 2014 Survey, Question 10. TABLE 5-26 Percentage of Company Revenues from SBIR/STTR Percentage of Responding Companies NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees 0% 34.0 35.8 24.4 46.9 1-10% 13.9 14.0 13.2 28.2 11-25% 12.2 11.3 16.5 2.9 26-50% 12.8 13.1 10.9 51-75% 9.9 8.7 16.0 7.7 76-99% 14.0 14.4 12.2 11.5 100% 3.3 2.6 6.9 2.9 Mean 29.0 28.0 35.0 20.0 Median 18 18 18 5 BASE: TOTAL COMPANIES 409 344 66 17 ANSWERING QUESTION SOURCE: 2014 Survey, Question 9.
|
From page 178... ...
These results indicate an overwhelmingly positive impact. Overall, 97 percent of SBIR/STTR respondents reported a positive effect, and 62 percent reported a transformative effect.
|
From page 179... ...
QUANTITATIVE OUTCOMES 179 TABLE 5-28 Prior SBIR/STTR or STTR Phase II Awards Related to the Surveyed Project Percent of Responses NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees 0 23.9 25.0 17.9 11.5 1 46.0 43.9 57.1 38.5 2 18.9 19.8 14.3 34.6 3 or 4 8.1 8.3 7.1 15.4 5 to 9 1.9 1.8 2.4 10 or more 1.1 1.1 1.2 1 or more 76.1 75.0 82.1 88.5 Mean 1.34 1.35 1.32 1.54 Median 1 1 1 1.5 BASE: TOTAL RESPONDENTS 528 444 84 26 ANSWERING QUESTION SOURCE: 2014 Survey, Question 39.2. TABLE 5-29 Long-Term Effects on Recipient Companies Percentage of Responses NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees Positive, highly positive, or transformative 96.5 96.5 96.7 92.9 effect Highly positive or transformative effect 61.8 62.7 56.7 71.4 Positive effect 34.7 33.8 40.0 21.4 No effect 3.2 3.1 3.3 7.1 Negative, highly negative, or disastrous 0.4 0.4 effect Negative effect 0.2 0.2 Highly negative or disastrous effect 0.2 0.2 BASE: TOTAL RESPONDENTS 570 480 90 28 ANSWERING QUESTION SOURCE: 2014 Survey, Question 57.
|
From page 180... ...
• Supported company formation • Provided first dollars • Funded areas where venture capital and other funders were not interested • Supported development of critical company infrastructure • Opened doors to potential partners • Helped address niche markets too small for major players/funders • Funded technology development • Enabled projects with high levels of technical risk and high potential return • Supported adaptation of technologies to new uses, markets, and industry sectors • Funded development of core technology • Diversified expertise, allowed hiring of specialists • Gave companies immediate credibility • Funded researchers to enter business full time • Transformed company culture to become more market driven • Created new companies and kept companies in business (that would not exist without SBIR/STTR funding) • Helped increase the company's knowledge base applied to later projects • Expanded the scope and scale of R&D capabilities • Supported technology development that led to spin-off companies From these responses it is clear that small innovative companies are highly sensitive to the impact of outside factors.
|
From page 181... ...
TABLE 5-30 Distribution of Responses by Sector Phase Percent of Responses NIH SBIR STTR Phase IIB Total Awardees Awardees Awardees Aerospace and Defense 2.6 2.9 1.1 6.9 Aerospace 0.2 0.2 Defense-specific products and 2.6 2.9 1.1 6.9 services Energy and the Environment 2.5 2.9 Renewable energy production (solar, 0.2 0.2 wind, geothermal, bio-energy, wave) Energy storage and distribution 0.7 0.8 Energy efficiency 0.3 0.4 Other energy or environmental 1.6 1.9 products and services continued
|
From page 182... ...
Research tools 25.8 25.2 29 10.3 Education materials 8.9 8.5 10.8 Other medical products and services 3.9 4.3 2.2 Other (please specify) 6.9 8 1.1 3.4 BASE: TOTAL RESPONDENTS 608 515 93 29 ANSWERING QUESTION NOTE: Responses do not sum to 100 percent because respondents could select more than one answer.
|
From page 183... ...
In particular, the survey asked Phase II recipients whether the project would have been undertaken absent SBIR/STTR funding and whether the scope and timing would have been affected. Responses are summarized in Table 5-31.
|
From page 184... ...
184 SBIR/STTR AT THE NATIONAL INSTITUTES OF HEALTH TABLE 5-31 Project Undertaken in the Absence of this SBIR/STTR Award Project Go-Ahead Absent Award Percentage of Responses Yes 6.7 Definitely yes 1.7 Probably yes 5.0 Uncertain 19.0 Probably not 40.8 Definitely not 33.5 100.0 N= 179 SOURCE: 2014 Survey, Question 24. Overall, these views indicate that SBIR/STTR funding was important not only for the go/no-go decision but also for the eventual shape and indeed likely impact of the project.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.