National Academies Press: OpenBook

SBIR at the National Science Foundation (2015)

Chapter: 7 Findings and Recommendations

« Previous: 6 Insights from Case Studies and Extended Survey Responses
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×

7

Findings and Recommendations

The findings and recommendations in this chapter reflect the congressional objectives for the SBIR program, as reiterated in the recent program reauthorization and in the subsequent SBA policy Directive that guides program implementation at all agencies.1 Section 1c of the Directive states program objectives as follows:

The statutory purpose of the SBIR Program is to strengthen the role of innovative small business concerns (SBCs) in Federally-funded research or research and development (R/R&D). Specific program purposes are to: (1) Stimulate technological innovation; (2) use small business to meet Federal R/R&D needs; (3) foster and encourage participation by socially and economically disadvantaged small businesses (SDBs), and by women-owned small businesses (WOSBs), in technological innovation; and (4) increase private sector commercialization of innovations derived from Federal R/R&D, thereby increasing competition, productivity and economic growth.2

The findings below review the extent to which each of these program objectives is being addressed at NSF, as well as examine some specific aspects of program management.

FINDINGS

The Small Business Innovation Research (SBIR) program at the National Science Foundation is having a very positive overall impact. It is meeting three

___________________

1See Box 1-2 and the discussion in Chapter 1 of the committee’s task.

2SBA SBIR Policy Directive, October 18, 2012, p. 3.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×

of the four legislative objectives of the program; we find that more needs to be done to “foster and encourage participation by socially and economically disadvantaged small businesses (SDBs), and by women-owned small businesses (WOSBs), in technological innovation.” Even so, the program is supporting the birth and growth of small innovative companies, which indirectly impacts all of the congressional objectives.

I. Commercialization

Each agency has its own priorities for the program. At NSF, the overwhelming emphasis has been on commercialization, which here means that projects are commercially successful in private sector markets.

  1. SBIR projects at NSF commercialize at a substantial rate.3
    • Substantial Sales Reported: About 70 percent of Phase II respondents reported sales, based on responses to the 2011 Survey.4 This represents a slight increase from rates reported in the previous Academies report.
    • Sales Anticipated: An additional 19 percent of Phase II respondents reported that they anticipate future sales.5
    • Scale of Commercialization: Despite the high rates of commercialization, the scale of commercialization was limited: no projects surveyed reported aggregate sales of $50 million or more, and only 6 projects (out of 114) reported project-related sales of $10-50 million.6
  2. Academies survey data show that NSF SBIR projects are primarily commercializing in the domestic private sector.
    • Just under 60 percent of respondents with sales reported revenues from domestic private sector customers.7
    • About 15 percent reported export customers.8
    • About 16 percent of responses identified customers in the public sector (primarily defense and defense contractors.)9
  3. Further investment in SBIR is additional evidence of commercial activity. Subsequent investment provides further evidence that SBIR projects are expected to generate significant commercial value even if they have not yet reached the market. Academies survey data show that

___________________

3NSF declined to share the results of its own internal commercialization tracking system on confidentiality grounds.

4See Table 4-1.

5See Table 4-1.

6See Table 4-2.

7See Table 4-3.

8See Table 4-3.

9See Table 4-3.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
    • More than 60 percent of Phase II survey respondents reported additional investment funding.10
    • The most likely source of additional funding for Phase II recipients (other than their own company and personal funds) was non-SBIR federal funding, which is funding provided by from federal agencies from other budgets outside the SBIR programs (39 percent of responses), closely followed by other companies (32 percent), private equity/angels (28 percent). About 15 percent reported venture capital (VC) funding.11
  1. SBIR is associated with job growth. With respect to direct job growth, Academies survey data indicate that the median size of firms with NSF Phase II awards grew from 6 employees at the time of award to 10 employees at the time of survey.12
  2. SBIR funding makes an important difference to project outcomes. SBIR funding makes the difference in determining project initiation, scope, and timing. Academies survey data show that
    • 68 percent of Phase II respondents reported that the project probably or definitely would not have proceeded without SBIR funding.13
    • 75 percent of those who would likely have proceeded anyway reported that the project would have been narrower in scope.14
    • About two thirds of those who would likely have proceeded anyway reported that the project would have been delayed by at least one year.15
  3. Commercialization and venture funding
    • NSF has focused on an expected pattern of product development and commercialization that is modeled on that for projects funded by venture capitalists (VCs).
    • This model is not an appropriate model for NSF SBIR project commercialization for several reasons:
      • VCs seek projects that meet a number of narrow criteria, including timeline to market exit, size of opportunity, amount of funding required, capabilities of the management team, and industry sector. The vast majority of NSF SBIR projects do not and will not meet these criteria as they are typically specified by VCs.
      • NSF appears to make a point of funding very small companies at or soon after startup. The median size of Phase II company at

___________________

10See Table 4-6.

11See Table 4-8.

12See Tables 4-4 and 4-5.

13See Table 4-19.

14See Table 4-20.

15See Table 4-21.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
  • time of award is 6 employees.16 These companies are generally too small to interest VCs.

  • Data from the Academies survey indicates that 63 percent of Phase II respondents received additional investments, but of these only 15 percent received funding from VCs.17

II. Fostering the Participation of Women and Other Underserved Groups in the SBIR Program

  1. Current outcomes data show that this objective has not been met.
    • Levels of participation by underserved groups are low and declining.
      • Data from NSF indicate that approximately 10 percent of SBIR Phase I awards go to Minority-Owned Small Businesses (MOSBs). These figures have declined over the past decade, as has the MOSB share of Phase II awards.18 MOSBs also show lower success rates for both Phase I and Phase II applications than non-MOSB applicants, with the success for Phase I MOSB applicants being lower in every year, and success rate of Phase II MOSBs averaging 12 percentage points lower over the period.19
      • NSF data show that about 13 percent of SBIR Phase I awards were to Woman-Owned Small Businesses (WOSBs) and make up just over 10 percent of Phase II awards.20 The higher share for WOSBs compared to MOSBs is based on the large numbers of awards won by 3 well established WOSBs.
      • NSF does not maintain data on woman and minority Principal Investigators (PIs). Data from the Academies survey indicates that these numbers are also low and not rising.
      • The Academies survey indicated that 13 percent of PIs were female.21
      • These discouraging results correspond to similar poor results across many STEM education fields in the United States.
    • NSF has no separate data on African-American-, Hispanic-, or Native American-owned small businesses.
      • The Academies survey, however, indicated that black- and Hispanic-owned small businesses are themselves a very small

___________________

16See Table 4-4.

17See Tables 4-6 and 4-8.

18See Figure 3-10 for percentages of SBIR Phase I awards going to MOSB, and Figure 3-24 for percentages of SBIR Phase II awards going to MOSB.

19See Figure 3-8 for Phase I MOSB comparative success rates for applications receiving awards, and Figure 3-22 for Phase II MOSB comparative success rates.

20See Figure 3-15 for percentages of SBIR Phase I awards going to WOSB, and Figure 3-28 for percentage of SBIR Phase II awards going to WOSB.

21See Chapter 2, “Evidence provided by Academies Survey Data,” and 2011 Survey, Question 14A.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
      • share of MOSBs overall. Black-owned small businesses accounted for approximately 1 percent of all respondents; Hispanic-owned firms, about 3 percent.22

      • In the Academies survey, companies reported that 17 percent of Phase I PIs and 15 percent of Phase II PIs were minority. However, further analysis indicates that less than 1 percent overall were African American, with 1.5 percent in Phase I and none in Phase II. Less than 3 percent overall were Hispanic, with 3.9 percent in Phase I and 1.1 percent in Phase II. There were no Native Americans in Phase I or Phase II.23
    • There are further questions raised by the data regarding the shares of Phase I and Phase II awards accounted for by WOSB.
      • The Academies survey data show that WOSBs accounted for 17.1 percent of Phase I companies but only 11.3 percent of Phase II companies responding to the Academies survey.24
  1. NSF’s efforts to “foster and encourage” the participation of woman-owned and minority-owned small businesses have not been effective.
    1. No substantial program has been implemented for outreach to these communities.
      • No examples or other evidence was found of persistent efforts by NSF to reach out to WOSB.
      • The NSF SBIR/STTR Advisory Committee also raised this issue in 2012.
    2. NSF does not report on or sufficiently track participation by WOSB and MOSB.
      • NSF does not provide an annual report that covers either data on participation or efforts to foster and encourage participation.
      • No evidence was found that NSF tracks MOSBs at a level sufficient to meet congressional intent.
      • No evidence was found that NSF tracks the participation of woman and minority principal investigators in the program.
  2. There appears to have been insufficient effort made at NSF to understand why these rates remain low. Concerted effort would be needed to determine what practical steps can be taken to improve participation and hence both meet congressional objectives of the program and expand the pool of qualified applicants and capabilities.25

___________________

22Calculated from data reported in the text of Chapter 2, just prior to Table 2-9.

23Calculated from data reported in Table 2-7 and accompanying text.

24See Table 2-10.

25A discussion of conclusion about woman and minority participation and NSF’s limited efforts to address the issue is provided in more detail in Chapter 2, “Conclusions: Woman and Minority Participation in the NSF SBIR Program.”

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×

III. Stimulating Technological Innovation and Meeting Agency Mission Needs

NSF’s agency mission is focused on the advancement of science.26 Thus for the NSF SBIR program, the twin objectives to use small business to meet federal R/R&D needs and to stimulate technological innovation are close complements and are therefore discussed together in this section.

  1. Meeting federal R/R&D needs: The SBIR program at NSF supports the development and adoption of technological innovations. Selection of both topics and individual projects for funding reflects a strong focus on technological innovation.27
    • Focused Topic Selection: Program Directors strongly assert that topic selection is driven by efforts to enhance and support innovation, and topics undergo a review process within the agency toward that end.
    • Solicitations are not Exclusive: NSF does not formally require that projects specifically match the topics described in the solicitation. Under the terms of the solicitation, non-matching applications can also be accepted, in an effort to ensure that potentially important innovations are not excluded by the topic structure. However, NSF does not track whether applications match the solicitation topics, there is no evidence to confirm or refute if solicitations are nonexclusive.
  2. Emphasis on Technological Innovation: Scoring for individual projects is weighted toward technological innovation: NSF merit criteria emphasize the importance of technical innovation, and selection decisions are—according to Program Directors who make recommendations for those decisions—also heavily weighted toward the support of potentially transformational technologies. SBIR continues to connect companies and universities. Survey data indicate that NSF SBIR projects continue to utilize universities in a variety of ways (excluding the even closer connection through Small Business Technology Transfer [STTR]):
    • Faculty and Student Participation: Just under 60 percent of Phase II survey respondents reported a link to a university. In about 35 per-

___________________

26“The National Science Foundation (NSF) is an independent federal agency created by Congress in 1950 “to promote the progress of science; to advance the national health, prosperity, and welfare; to secure the national defense…” See National Science Foundation, “NSF—At A Glance,” accessed on June 2, 2014. A more extended description from the same source states that goals are “to advance the frontiers of knowledge, cultivate a world-class, broadly inclusive science and engineering workforce and expand the scientific literacy of all citizens, build the nation’s research capability through investments in advanced instrumentation and facilities, and support excellence in science and engineering research and education through a capable and responsive organization.”

27See Chapter 2 for a discussion of NSF’s topic selection and funding decisions.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
    • cent of responses, university faculty worked on the project (not as PI); 30 percent employed graduate students; and in almost a quarter of Phase II projects reporting, universities were subcontractors.28 These figures are broadly similar to those reported from the Academies’ 2005 survey.

    • Project Partners: 114 different universities were identified by survey respondents as project partners; 13 were mentioned by more than 3 respondents.29
    • Academic Founders: More than 80 percent of NSF Phase II SBIR companies responding to the survey reported at least one academic founder, and about a quarter reported that the most recent prior employment of the founder was at a university.30
  1. Academies survey data show that SBIR projects generate substantial knowledge-based outputs such as patents and peer reviewed publications.
    • Patents: Patenting remains an important component of knowledge diffusion (and protection).
      • More than 70 percent of the Academies’ Phase II survey respondents reported filing at least one patent related to the surveyed project.
      • 23 percent reported filing five or more related patents.
    • Publications: Publication of peer reviewed articles remains the primary currency of scientific discourse, and despite the need to protect ideas in the commercial environment of small businesses, the Academies’ Phase II Survey shows that SBIR firms continue to participate deeply in scientific publication.
      • About 80 percent of surveyed Phase II projects reported at least one resulting peer reviewed publication.31
      • 32 percent reported more than three publications resulting from the surveyed project.32
      • Many of the companies interviewed for case studies made a point of indicating that they take a great deal of pride in the number of peer reviewed publications developed by their scientists and engineers, both within and outside of the SBIR program.
  2. SBIR funds projects with social benefits that may not be attractive to commercial sources of funding.33

___________________

28See Table 4-17.

29See Table 4-18.

30See Table 4-28 and Chapter 4 text, just before “Conclusions: Knowledge Effects.”

31See Table 4-16.

32See Table 4-16.

33See Chapter 6, “Funding Otherwise Un-fundable Projects.”

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
  • Projects with Social Benefits: NSF SBIR funds some projects that are socially desirable and market oriented but that are unlikely to generate the high returns need to attract venture-type funding. For example, NSF SBIR-funded projects supporting assistive technologies for the blind are now in use worldwide (for a detailed analysis of socially valuable projects with limited markets, see Touch Graphics case study34).
  • Projects with Long Timelines: NSF also funds some projects with timelines that are too long for venture funders, some of which may turn out to be “platform technologies” with multiple applications (for example, new materials or battery technologies.)

IV. Fostering Innovative Companies

  1. Company formation and early funding: Many SBIR companies surveyed by the Academies report reported that SBIR funding was instrumental in the founding of the company. The formation of new innovative companies is a positive outcome for the program.
    • 45 percent of Phase II respondents to the Academies survey said that the company was founded entirely or in part because of the SBIR program.35
    • For some companies included among the case studies, SBIR funding permitted the shift from an exploratory to a professional operation.36
  2. De-risking subsequent investment and new markets
    • Mitigating Risks: One important impact of the NSF SBIR program is to provide funding that reduces the risk for subsequent investors. Risk mitigation often leads to the leveraging of SBIR funding while retaining the power of markets to make final decisions about funding.37
    • Matching Funds: The NSF Phase IIB program lowers risks by providing matching funds to companies with external investors or other approved sources of funding.38
    • Early Stage: More widely, many respondents to the Academies survey and a number of case study interviewees said that NSF SBIR funding was provided at a stage when the project was sim-

___________________

34See Appendix E: Touch Graphics Case Study.

35See Table 4-29.

36See “Validation Effects” in Chapter 6, and Box 6-7.

37See section on “Risk” in Chapter 6, and survey results for risk reduction in Box 6-4.

38See discussion in Chapter 5 of requirements and effects of matching funding for Phase IIB projects, and tables 5-2 and 5-3 that show a 3:1 ratio of matching investments to NSF funding over a dozen years.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
    • ply too risky for commercial sources of funding. Once the project proceeded further, risk was lower and additional funding could be acquired.39

    • Support for Technology Development: NSF SBIR funding supports technology development, which can be supported through commercial funding further downstream. SBIR is particularly important for funding proof of concept for new technologies.40
    • Validation: NSF SBIR funding has itself been important validation for companies seeking further investments, according to case study interviews and survey responses. The strength of the selection process and growing understanding of SBIR among investors appears to be strengthening this effect.41
    • NSF funding can support company efforts to enter new markets.42
    • Exploit technology platforms: In some cases, companies use SBIR funding to build off existing platform technologies specifically to enter new markets. This platform-driven approach is used by a number of the interviewed companies. (See Chapter 6.)
    • Strategic corrections: Innovative companies must often make mid-course corrections. NSF funding has—according to Academies survey recipients—helped a number of surveyed companies successfully make what are often difficult changes that are hard to fund.
  1. Long term positive effect on SBIR awardee companies. Commercialization in the long run requires sustainable companies, and SBIR has supported the development of an ecosystem of small innovative companies in the United States.43 The Academies survey provided SBIR companies with the opportunity both to report the overall impact of SBIR on the company, and to identify specific kinds of impacts.
    • 35 percent of Phase II winning recipients indicated that the NSF SBIR program had a “transformative” effect on their company. Another 54 percent said that it had a “substantial positive long-term effect.”44
    • Of the 179 detailed comments received in the Academies survey, none reported negative effects. Widely differing kinds of impacts were reported, as summarized in Box 7-1.45

___________________

39See “Company Formation and Very Early-stage Funding” in Chapter 6, and Box 6-2.

40See “Funding Core Technology Development” in Chapter 6, and Box 6-5.

41See “Validation Effects” in Chapter 6, and Box 6-7.

42See “Expansion into New Markets” in Chapter 6.

43See “Capacity Building—Human Capital” in Chapter 6, and Box 6-8.

44See Table 4-23.

45See “Key Aspects of SBIR-Driven Transformation” in Chapter 4, as well as Chapter 6 and cases in Appendix E.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×

BOX 7-1
Different Ways in Which SBIR Awards Helped to Transform Companies

  • Provided first dollars.
  • Funded areas where venture capital and other funders were not interested.
  • Provided funding during downturns in the business cycle.
  • Opened doors to many potential stakeholders in specific technologies, including agencies, prime contractors, investors, suppliers, subcontractors, and universities.
  • Helped address niche markets too small for major players/funders.
  • Supported adaptation of technologies to new uses, markets, and industry sectors.
  • Funded technology development.
  • Enabled projects with high levels of technical risk.
  • Reduced technological risk.
  • Diversified expertise and allowed hiring of specialists.
  • Attracted and developed young researchers.
  • Redirected company activities to new opportunities.
  • Reduced costs.
  • Helped address needs that require high tech at low volume and relatively low cost.
  • Gave companies added credibility because SBIR research is seen as peer reviewed.
  • Funded researchers to enter business full time.
  • Transformed company culture to become more market driven.
  • Drove researchers to focus on technology transition.
  • Created new companies and kept companies in business (that would not exist without SBIR funding).
  • Supported feasibility testing for high-risk/high-payoff projects.
  • Supported projects with longer time horizons/long sales cycles.
  • Provided the basis for spin-off companies.
  • Funded proof of concept.
  • Encouraged R&D companies to transition into manufacturing.
  • Funded disruptive technologies.
  • Provided significant mentoring especially for new businesses.
  • Stimulated international collaboration.

SOURCE: Analysis of company responses to 2011 Survey. For each bullet multiple responses indicated its existence and importance for surveyed projects and firms.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
  1. Limiting Company dependence on SBIR
    • Limits on Multiple Awards: The NSF program tightly limits the capacity of firms to garner multiple awards.46
      • Firms are permitted to apply for only two awards per year.47
      • The most prolific firms received fewer than 10 SBIR Phase II awards over a period of 10 years.48
      • Interviews with Program Directors indicate a preference for spreading Phase II awards more widely.49
    • NSF firms are not dependent on SBIR awards: Less than 20 percent of companies responding to the 2011 Survey report that SBIR accounts for more than half of current revenues;50 however, a considerable number of surveyed firms reported in textual responses that SBIR is the most important source of funding prior to reaching the market.51
    • SBIR Innovation is nonlinear: Most projects at most companies do not proceed directly from Phase I to Phase II to commercialization.52
      • About 85 percent of Phase II survey respondents reported at least one additional SBIR Phase II award related to the surveyed project.53
      • About one third reported at least two additional related Phase II awards.54
      • As noted above, more than 60 percent of Phase II respondents report additional investment funding related to the project subsequent to the SBIR award.55

V. The Phase IIB Program

The NSF Phase IIB program supports the accelerated commercialization of SBIR-funded research through the provision of matching NSF funds provided companies can generate additional investment funding of their own. The evidence shows that Phase IIB projects tend to commercialize more than Phase II-only projects. In part, this reflects the rigorous selection process—NSF selects Phase IIB projects specifically for their commercial potential, so improved commercial

___________________

46Previous Academies studies discussed the question of multiple SBIR awards to companies. The 2009 report concluded that “mills” are not a significant problem.

47National Science Foundation, June 2014 SBIR program Solicitation, NSF-13-599, p. 1.

48See Chapter 3, Annex Table 3-6.

49See text in Chapter 3 at the end of the section, “New Companies in the SBIR Program.”

50See Table 4-36.

51See Chapter 6 and case studies in Appendix E.

52See Chapter 4’s section, “Prior Use of the SBIR Program.”

53See Table 4-38.

54See Table 4-38.

55See Table 4-6.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×

outcomes should be expected. Moreover, evidence from surveyed recipients and from case studies strongly suggests that the program does have a catalyzing effect, attracting additional funding, and that it does make a positive difference to company activities and outcomes.56

  1. The Academies survey of Phase IIB projects reports increased commercialization and larger revenues.
    Phase IIB program attracts third-party investors, thereby accelerating and enhancing the commercialization of SBIR-funded technologies:
    • Phase IIB projects reach the market at a high rate—81 percent of surveyed Phase II projects reported that they had achieved some sales.57
    • Products in use. Just under half of the respondents to the Phase IIB survey report that their products, processes, or services are in use today.58
    • Projects with large revenues. Almost a quarter of Phase IIB projects surveyed reported project-based revenues of $3 million or more percent. Median reported sales were $650,000.59
    • Important role of Phase IIB. Eighty percent of respondents to the Phase IIB survey indicated that Phase IIB has accelerated commercialization of their product or services.60 This is an important point because the program by design focuses on firms with maximum commercial potential. Confirmation from firms about the role and impact of the program is therefore highly relevant.
  2. The Academies survey of Phase IIB projects reports employment effects and opportunities for female PI’s.
    • Employment growth. Respondents to the Phase IIB survey reported employment growth from a median of 8 employees at the time of the award to 17 at the time of the survey.61
    • More opportunities for female principal investigators. Phase IIB respondents reported more female principal investigators than Phase II projects—18 percent versus 7 percent.62
  3. The Academies survey of Phase IIB projects reports that by requiring matching funds, the Phase IIB program provides incentives for firms to acquire additional investment.

___________________

56See Chapter 5 for a description of the Phase IIB program and its effects.

57See Table 5-8. See also Table A-5 for information on the Phase IIB survey responses. A total of 281 survey responses were received, reflecting an effective response rate of 48.3 percent.

58See Table 5-7.

59See Table 5-9.

60See Table 5-27.

61See Tables 5-11 and 5-12.

62See Table 5-6.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
  • Phase IIB projects leverage NSF funding.
    • Respondents to the Phase IIB survey reported a mean of $651,000 in additional matching funds.63
    • Sixty-two percent of Phase IIB respondents reported at least $500,000 in additional funding, and 36 percent reported receiving at least $1 million.64
    • Phase IIB respondents reported funding from a range of sources: 23 percent of their matching funds from sales, 21 percent from another U.S. company, and 16 percent from U.S. angel investment. Nine percent came from venture capital investors.65
    • The requirement for matching funds works to screen out projects that are less able to attract additional investment funding. About two thirds of the Phase II projects that did not apply for Phase IIB funding reported that they could not raise the required matching funds.66
  • Attracting New Investors: Some Phase IIB projects successfully attracted new investors, in some cases attracting substantial additional funding.
    • Thirty percent of Phase IIB projects reported that that their Phase IIB matching funding came from a new financial partner or investor.67
    • While sales were the single largest source of matching funds, three quarters of responses indicated that funds were acquired from financial, technical, or commercial partners with whom they had a long-term relationship.68
    • Sixty-one percent of respondents indicated that the new funding would not have been acquired without Phase IIB.69
  • Providing Direct Incentives for Firms: Phase IIB provides direct incentives for firms, and imposes reasonable administrative burdens.
    • More than half of Phase IIB respondents indicated that the Phase IIB program provided a substantial incentive for or drove the process for seeking external funding.70
    • Forty-one percent of projects reported the total effort required to apply for Phase IIB as “no additional effort needed except

___________________

63There were no statistically significant differences between survey responses from Phase IIB respondents and agency data from the NSF awards database. See Table 5-23.

64See Table 5-23.

65See Table 5-24.

66See Table 5-17.

67See Table 5-26.

68See Table 5-26.

69See Table 5-29.

70See Table 5-30.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
  1. paperwork” or “less than 2 weeks FTE for company staff.”71 About one third of respondents believed the application process was easier than for comparable Federal R&D applications, while nineteen percent thought it more difficult.72

  2. Companies report positive views of Phase IIB impacts.
    Companies had remarkably positive views of the impact of Phase IIB funding on both their projects and their companies as a whole.
    • Worth Applying: Ninety-eight percent of Phase IIB respondents believed that the funding they received was worth the effort involved in applying—even though more than a fifth had spent at least two months of senior management time on the application.73
    • Long-term Impact: Ninety-six percent of Phase IIB respondents indicated that Phase IIB had a positive long-term impact on the company. Nearly two-thirds stated that the program had a substantial positive impact, and nearly one in ten responded that the program had had a “transformative effect” on the company.74
    • Broad Support: Comments from respondents indicate that Phase IIB supports a wide range of activities tied to a considerable variety of commercialization strategies and approaches.
  3. Other Aspects of the Phase IIB program.
    • Strong support for companies from NSF program officers. Survey respondents and interviewees both had generally strongly positive views on the role of NSF program officers in the application process (and indeed during the course of the award—see Chapter 2 and Appendix E).
  4. Improving Phase IIB. The study draws attention to potential areas of improvement for the Phase IIB program. These include the following:
    • Confusion about matching funds criteria. There was considerable confusion even among recipients about the criteria for identifying appropriate matching funds. Many respondents identified funding sources that were in fact not acceptable as matching funds; and many failed to identify equity funding and sales, both of which are permitted.75
    • Matching funds criteria. Aside from the confusion about criteria noted above, almost half of Phase IIB awardees indicated that NSF matching fund criteria were too stringent in excluding in kind con-

___________________

71See Table 5-19.

72See Table 5-20.

73See Table 5-34.

74See Table 5-28.

75See Table 5-21.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
  • tributions. However, stringent criteria are a strength of the program, in the view of the Committee.76

  • Commercialization focus. While the commercial focus of the NSF SBIR program is widely appreciated by survey respondents and interviewees (as reflected in comments in Chapter 6 and Appendix E) a number of respondents to the Phase IIB survey suggested that NSF should review the balance between rapid commercialization and powerful innovation, as the current balance might end up excluding potentially important projects.

VI. Program Management

  1. The NSF Management Model
    • Key Role of Program Directors: The NSF model relies on a small cadre of highly qualified Program Directors who have strong technical and commercial backgrounds to provide guidance and support to companies that apply for SBIR grants and that seek to commercialize SBIR technologies. Evidence from the Phase II survey and from case studies suggests that in many cases the relationship between the Program Director and the company is strong and that the model works as anticipated.77
    • There is evidence from several sources that the Program Directors face a heavy workload.78
      • Program Directors interviewed for this study indicated that they are responsible for about 30 Phase II awards at any one time. Given this workload, it is not clear how much time they are able to devote to any one company.
      • Phase II Recipient survey responses indicate that in some cases companies believe that the workload for Program Directors is too great for them to provide appropriate support.
  2. Best Practices
    • Effective Program Directors: Companies strongly appreciate the guidance offered by program officers in most cases, and in interviews suggest that they benefit considerably from the high quality of the NSF program directors.79
    • Limits on submissions per company: These limits are an appropriate mechanism for ensuring that the number of submissions is limited while at the same time not excluding any specific project.

___________________

76See Table 5-37.

77See “Role and Effectiveness of Program Directors” and “Qualifications of PDs” in Chapter 2 and Box 2-6.

78See Tables 2-11, 2-12, 2-13, 2-14, 2-15, 2-16, 2-17, and 2-18, and related text in Chapter 2.

79See Box 2-6 in Chapter 2.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
    • Companies decide which projects to submit, reducing the number of applications that program staff must review and ensuring that if a project is funded, it will be one that was at or near the top of the company’s priority list.80

    • Inclusive Topic Deployment: The broad topics offered by NSF—in conjunction with the increase in the number of solicitations by topic to two per year—indicate that the agency is working hard to ensure that promising projects are not excluded unnecessarily. NSF program officers indicate that they believe any technically and commercially worthwhile project could be funded under the published solicitation guidelines.81 Some companies, however, disagree.
    • The Phase IIB Program: This program has been an important initiative at NSF, and has since been adopted in different forms at other agencies.82
    • Partnerships: NSF is to be commended for exploring a range of partnerships and mechanisms through which to support the commercialization of funded projects.
    • i-Corps, an NSF program that offers entrepreneurship training (primarily for academics), seeks to ensure that some key steps in the development of a sustainable company are taken at an early stage in the product development cycle.83
    • The Innovation Accelerator, a private organization partnering with NSF to facilitate the commercialization efforts of high-technology small businesses funded by grants from the NSF SBIR program, is also potentially valuable, although its reach may have to be expanded beyond the information technology sector.
  1. Key Concerns
    • Data Tracking, Management, and Use: There are broad challenges in tracking commercialization, at both the company and project levels. Companies move in and out of the program, and tracking is harder once they have left. More generally, commercialization may come years after an award, and may involve multiple awards plus considerable additional funding. All this makes it difficult to assert that any specific outcome “results from” an SBIR award. But there are also specific challenges with existing tracking tools.
    • Recognition of NSF efforts to track outcomes. NSF’s existing effort to track outcomes is commendable. It predates similar efforts at other agencies by a number of years, and by seeking information

___________________

80See, for example, Chapter 3, last paragraph before “Demographics of Phase I Winning Companies.”

81See “Topic Selection” in Chapter 2, and Box 2-9.

82For a description of the Phase IIB Program at NSF, see Chapter 5.

83See I Corps program highlights in Chapter 2, Box 2-3.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
  • at set times after the end of each award provides the agency with longitudinal data.

  • Limits to existing approach: The current effort is limited, as the collection mechanism may be insufficiently standardized (ad hoc interviews) and also relies entirely on self-reported data with no cross checks involved. It is also focused almost entirely on only one of the congressional objectives—commercialization.
  • Need for Additional Tools: Additional data collection tools that reflect the changing information environment have not been deployed—NSF does not use additional methods of data collection beyond the flow of awards and applications data and the annual telephone survey.84 Provisions of a web-based survey tool for use by companies periodically to update their progress metrics may facilitate company reporting with reduced burden.
  • Need for a Broader Focus: The focus of outcomes data collection is almost entirely on commercialization that, as a result, misses opportunities to provide detailed feedback about program operations as well as other congressionally-mandated outcomes.
  • Need for an Information Management System: There is no single information management system through which agency staff can visualize the complete range of relevant program metrics and data.
  • Uses of the data are limited. Interviews with program officials suggest that the data collected through the internal exercise are not systematically used to improve program management. Given that the agency has collected commercialization data for a number of years, it would seem worthwhile for the agency to explore more deeply potential causal variables related to commercial success and to use data in program management.
  • Fixed Award Sizes: Like most other agencies NSF provides award funding that is largely fixed in size: prior to reauthorization, Phase II was $500,000 plus additional Phase IIB funding for some companies. Phase II funding is now a maximum of $1 million, and most awards will likely be made at that level. This fixed approach does not seem best designed to ensure that each project receives the appropriate amount of funding.
  • High Rate of Failed Applications: Some see the rate of failed applications to be burdensome for both applicants and agency staff. (At best 1 in 6 Phase I applications and less than 50 percent of Phase II

___________________

84It appears that this may be in part a result of compliance with the Paperwork Reduction Act, which limits agency tracking capabilities.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×

applications are successful for any given solicitation.)85 These success rates impose costs on both companies and the agency: 5 out of 6 Phase I applicants are not funded so their application effort is largely wasted; and agency reviewers must spend considerable time and effort on projects that are not funded. On the other hand, public R&D programs have traditionally encouraged companies to participate and have developed review strategies for dealing with the larger numbers and varied quality of proposals that invariably result.

RECOMMENDATIONS

While the NSF SBIR program generates substantially positive outcomes, the committee has identified a series of recommendations to improve its processes and outcomes.

I. Addressing Underserved Populations

NSF should immediately examine past and current efforts to address the clear congressional mandate to foster the participation of underserved populations in the SBIR program, examine and report on best practices and create benchmarks to relate the impact of such activities.

  1. Quotas are not necessary. NSF should not develop quotas for the inclusion of selected populations into the SBIR program. Such an approach is not necessary to meet congressional intent, and is likely to reduce program effectiveness.
  2. NSF should develop new benchmarks and metrics.86
    • Improve participation metrics: The SBIR/STTR program office should work with the NSF indicators group to develop much improved metrics for benchmarking the participation of underserved populations, developing and publishing clear benchmarks based on a defensible analysis of existing data.
    • Disaggregate Benchmarks: Measures of the participation of socially disadvantaged groups must be disaggregated by race or ethnicity, and attention focused on the clear congressional intent to support “minority” participation. We do not believe a focus on the current SBA definition of “socially and economically disadvantaged” is sufficient to meet this objective.

___________________

85But others see the NSF Phase I and Phase II success rates as not low relative to the acceptance rates of other competitive R&D funding programs. Similarly, some see failed applications as a waste of company and reviewer time, while others see knowledge benefits and capacity building in company efforts to develop proposals for the SBIR program even if they are not successful in obtaining an SBIR grant. Formulated plans may go on to generate benefits in other ways.

86See Finding II-A and II-C.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
    • Customize Benchmarks: Points of reference should be developed separately (though perhaps drawing on a shared methodology) for women and minorities. These benchmarks should be shared with other SBIR agencies. Benchmarks should address key questions that would include the following metrics, all of which should include both absolute levels and trends over time:
      • Shares of applications from companies owned by women and minorities
      • Shares of applications with woman and minority principal investigators
      • Shares of Phase I awards to companies owned by women and minorities
      • Shares of Phase I awards with woman and minority principal investigators
      • Shares of Phase II awards to companies owned by women and minorities
      • Shares of Phase II awards with woman and minority principal investigators
      • Shares of Phase IIB awards to woman and minority owned firms
      • Shares of Phase IIB awards with woman and minority principal investigators
    • Track Related Program Operations: Metrics should also track related program operations including outreach efforts. (See below.)
  1. NSF should undertake an investigation to understand better why its efforts to date to expand the participation of under-served populations have been largely unsuccessful.87
    • Develop Outreach Strategy: Based on this investigation, NSF should develop a coherent and systematic outreach strategy that provides for cost-effective approaches to enhance recruitment of both woman- and minority-owned companies and female and minority PI’s, developed in conjunction with other stakeholders and with experts in the field, including relevant programs at the state level. Outreach efforts should aim to expand SBIR awareness among potential applicants from underserved demographics.
    • Integrate Outreach Effort: NSF should ensure that outreach to selected populations is an integral part of its overall outreach.
    • Provide Management Resources: NSF should provide significant management resources given that improving participation is likely to be both difficult and a long term effort.

___________________

87See Finding II-B.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
    • Designate Staff: NSF should consider designating a senior staffer to work exclusively on improving women and minority participation in order to identify strategic initiatives, implement them, and provide consistent and timely reporting on their impacts.
  1. NSF should review internal award and selection data and processes to identify and address disparities between Phase I and Phase II awards to selected populations. The goal is to ensure that there are no biases in the selection process that are adversely affecting the selection of women and minorities.88
    • NSF should carefully review patterns of success rates for WOSB and MOSB to identify potential disparities.
    • As part of the review, NSF should investigate the possibility of selection-bias in the review process and take actions to make sure that it is avoided—including ensuring that review panels are reasonably diverse.

II. Commercialization

The NSF SBIR program is tightly focused on commercialization that is based on a venture capital model. However, it is worth considering some possible adjustments to the current approach.

  • Broaden Perspective to include strategic partners:89 The limited funding for seed and startup projects from U.S. venture capital in general, and the low numbers of NSF firms that report venture capital funding, suggests that NSF should not focus too tightly on commercialization models that rest on venture capital funding. Many alternatives exist—for example, a deeper focus on finding strategic partners. The VC-focused commercialization model narrows the program in several ways—by limiting the timeframe viewed as appropriate for commercialization, and also by anticipating certain levels of commercial scale needed to attract VC-type funding. We recommend that NSF review its conceptual approach to commercialization with a view to ensuring that different paths to commercial success are fully included, such as angel funding and strategic investments by other companies. In addition, NSF should explore newer alternatives—for example, open source models or use of equity crowdfunding. This is to some degree implicit in the I-Corps model, and NSF should ensure that its grantees are aware of alternative funding paths.

___________________

88See Finding II-A, points 2 and 3.

89See Finding I-F.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×

III. Phase IIB

  1. NSF should continue to operate the Phase IIB program.90
    • The program has had a positive impact on the commercialization of SBIR funded research.
    • Respondents strongly support retention of the program.
  2. NSF could consider expanding the size of the Phase IIB program.91
    • The strongly positive impact suggests that there may be additional projects that could benefit from Phase IIB funding.
    • But NSF should be cautious about the impact of expansion on other aspects of the program. A majority of Phase IIB respondents would not support the addition of Phase IIB awards at the cost of fewer Phase I or Phase II awards.
  3. NSF should not increase the dollars per Phase IIB maximum award if that reduces the number of Phase IIB awards.92
    • The program appears to successfully meet its mission at current award levels. Two-thirds of respondents agreed that the amount of funding was about right.
    • NSF should be cautious about reducing the number of Phase IIB awards in order to increase the size of each award. The analysis identified no evidence to suggest that this would be an improvement to the program or that recipients would support such an adjustment.
    • Survey responses indicated that while some supported such an adjustment, more than twice as many did not favor a trade-off of larger awards for fewer awards.
  4. NSF should clarify the criteria under which funding qualifies as an acceptable match for Phase IIB purposes.93
    • Survey and interview data indicated confusion among recipients (and presumably potential applicants) about acceptable criteria, and there are inconsistencies between different statements on the NSF Phase IIB web page.
  5. Matching funds requirements.94
    • Explore use of In-Kind Contributions: NSF may wish to explore allowing the limited use of some specified in-kind contributions as part or all of the matching funds. Nearly half of survey respondents agreed. Perhaps NSF might opt to allow in-kind contributions in “exceptional circumstances,” offering a limited but potentially important degree of flexibility.

___________________

90See Finding V-A.

91See Finding V-A.

92See Finding V-C.

93See Finding V-F

94See Finding V-F.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
  • Review Matching Requirement: Similarly, NSF may wish to review the requirement that matching funds be developed during the timeline of the Phase II. Some respondents indicated that this excludes funding developed during Phase I, for example. It also implies a simple linear progression toward market, which may also tend to exclude projects otherwise appropriate for funding.

IV. Improving Monitoring, Evaluation, and Assessment

The development of more careful monitoring and more sophisticated analysis of key variables are necessary to improve program outcomes. The NSF deserves considerable credit for its early efforts to track outcomes, but more remains to be done in this area. Newly available administrative funding can improve this area of program operations.

  1. Improving current data collection approaches and methodologies.95 Data collected through the current process are a good start but are far from sufficient to underpin a data driven program.
    • Improve Data Collection and Organization:
      • NSF should more systematically collect a range of quantitative data and standardize key questions to improve program evaluation, management, and outcomes.
      • This data collection effort should address the entire range of congressionally mandated outcomes, not just commercialization.
      • NSF should develop a dataset that can provide a basis for longitudinal analysis.
    • Track Commercialization Outcomes:
      • NSF should track commercialization outcomes in ways similar to the now widely-accepted methodology developed for the Academies studies. This approach focuses on multiple metrics in order to provide a deeper and more nuanced basis for analysis. (See methodology discussion in Chapter 1 and Appendix A).
      • Although NSF already tracks outcomes for several years after the end of an award, this period should be extended to more fully assess long term impacts as these may not become fully discernible during a more limited period of review.
    • Manage Information:
      • NSF should explore the development of an integrated information management system to improve the management of its SBIR program.
      • NSF is now developing a database of Phase IIB projects for which it is to be commended. However, program management

___________________

95See Findings II-A and VI-C.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
    • would be on a firmer basis if managers had readily at hand information about company participation in all of the program’s activities, including Phase IIB, various supplements, I-Corps, and the Innovation Accelerator.

    • Collect Demographic Data:
      • NSF should take immediate steps to improve its collection of demographic data, using the Fastlane electronic submission program.
      • NSF should extend its collection of the demographics of company ownership to show which of the SBA’s socially and economically disadvantaged categories an applicant belongs to. In addition, applicants should be asked the same demographic questions about the principal investigator.
    • Collect qualitative outcomes data:
      • NSF should also develop and adopt a more systematic and critical approach to the use of detailed case studies and success stories. Success stories can provide inspiration lessons learned, and important information not available elsewhere about program impacts. Case studies can inform the roles played by SBIR awards, the challenges faced by small businesses, insights into needed improvements in process, lessons learned, and other important information not available elsewhere about program impacts.
  1. Use available commercial tools to gather more current data on a long-term basis. While we recommend utilizing modern information management and data visualization tools, we stress that effective products are widely available on the commercial market.96
    • Use New Technologies: NSF should explore ways to use new technology such as social media to collect more current data. SBIR companies—like “customers” in other markets—are an important source of information about program strengths and weaknesses. This knowledge is currently not systematically included in internal program evaluation by NSF’s SBIR program.
    • Develop Feedback Tools: NSF should develop pathways to provide ongoing feedback from companies about program activities and operations. These could include various electronic communication tools.
    • Improve Networking: Similarly, NSF should consider developing mechanisms—like electronic tools—through which recipients can share information about their SBIR projects, helping them both to find technical or marketing or investment partners and to navigate

___________________

96 See Finding VI-C.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
  1. the often-complex regulatory and technical environment of NSF programs.

  2. Improved Data Utilization. NSF has collected data on outcomes for some years, but it is not clear whether these data are systematically employed to guide program management.97
    • Develop a Plan for Data Analysis: NSF should seek to develop a more sophisticated approach to analyzing and applying the data that are already collected. For example, there appear to be no plans in place within NSF to use the data collected over a number of years to evaluate factors that tend to encourage successful transitions between Phases, into Phase IIB, and then into full scale commercialization. Such an approach could identify key issues for program management.
    • Undertake Regular Analysis of Data: By collecting more and better data on outcomes and participation, NSF will be positioned to undertake regular analysis—either internally or with third-party help—on key program management issues, such as
      • What is the long term impact of partnership programs and other commercialization supports?
      • Is Phase IIB simply picking successful companies or is it at least, in part, causing companies to be successful?
      • How well do NSF selection processes predict eventual successful projects?
    • Recognize Impacts of Data Collection and Analysis: In some cases, simply measuring something more closely will likely provoke action. Closely tracking the participation of women and minorities could help assure a fair process and surface problem issues early, when they can be most easily corrected. For example, systematically tracking woman and minority participation would have surfaced the issue of differential Phase I and Phase II success rates identified earlier.
  3. Annual Report to the National Science Board and Congress: The SBIR/STTR program should provide a comprehensive annual report to the National Science Board and Congress and the public on its operations. While the precise details should be left to the agency we would recommend that it consider including six areas of program operations:98
    • Inputs, including aggregate current and longitudinal data on numbers of applications and awards, broken out by relevant subgroups (such as demographics, state)

___________________

97 See Finding VI-C.

98 See Findings II-B and VI-C.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
    • Program initiatives, which would provide NSF with an opportunity to describe the numerous and impressive initiatives undertaken by the program
    • Aggregated outputs drawn from an improved tracking, outcomes collections, and analysis process
    • Qualitative review, based on improved use of case studies, as well as success stories and social media
    • Impact assessment, focused on the extent to which NSF meets congressional objectives for the program. Impact assessment should be contracted out to ensure objectivity.
    • Summary conclusions, including prospective views on program activities and improvements for the coming year.

    While caution should be employed when imposing new reporting burdens on the NSF SBIR program, implementation of an improved information management system would provide a cost- and time-effective basis on which to provide better reporting on the program. An annual report would provide much improved transparency and would provide a coherent point of discussion for other stakeholders. It would effectively replace the existing report to SBA which is of limited utility for NSF or other stakeholders, and could be organized to meet some of the new reporting requirements imposed under reauthorization.

  1. Additional and better data collection tools and analysis consume staff time and resources. NSF should ensure that its staff is incentivized, and that sufficient funding is available for this purpose.99

V. Improving Program Management

Recommendations in this section are designed to improve program operations in ways that should enhance the program’s ability to address some or all of its objectives.

  1. NSF should seek to reduce the burden on program directors.100
    • Review Program Director Workloads: NSF should consider whether the current workload is appropriate. The NSF model calls for a mentoring relationship between Phase II winners and Program directors, and for a closer than normal relationship between Phase I winners and Program directors. This is hard to sustain given current levels of activity with each program director responsible for about 30 Phase II awards and perhaps twice that many Phase I awards.
    • Consider Ways to Make the NSF Model Work Better: While Providing Individual Attention to Projects and Not Overburdening PDs:

___________________

99 See Finding VI-C.

100 See Finding VI-A.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
    • For example, consider adding part-time support staff or an additional PD to make the workloads more manageable, such as a PD to manage proposals in topics outside the announced solicitation.

    • Consider Virtual Meetings: NSF should also consider whether the current approach to identifying candidates to be recommended for awards could be streamlined as suggested by the NSF SBIR/STTR Advisory Committee through the use of more virtual meetings—both for review panels and for review of Phase IIB candidates where currently all have to go through an in-person interview process.
  1. NSF should consider ways to improve the average quality of applications.101
    • Encourage firms to submit higher quality proposals: NSF’s use of a pre-selection White Paper approach, which provides guidance and feedback to prospective applicants, appears to be partially effective in improving quality while reducing the number of applications, as is the 2 per company limit. NSF should look to find additional ways to attract fewer and higher quality applications.
    • Eliminate use of applicant success rates to validate program quality. NSF should at a minimum not use low success rates to validate program quality, as this in part simply reflects the receipt of weak proposals. Screening out more weak proposals before application would raise success rates for the remaining applications, while maintaining or even improving the quality of the awardees.
  2. NSF should consider the adoption of further mechanisms to reduce funding gaps between Phase I and Phase II.102
    Some of the mechanisms used at other agencies include a Fast Track process for merging Phase I/Phase II applications; use of agency options as bridge funding; early identification of projects that are likely to be funded. NSF could explore these and other options.
  3. NSF should investigate how open in its solicitations are in practice. It should track the number of proposals by topic and determine the percentage that fit the announced solicitation topic(s). It should explore whether NSF approach to topics is sufficient to keep up with evolving opportunities. The relationship between topics and the number of applications should also be subject to ongoing review.

___________________

101 See Finding VI-C.

102 See Finding VI-C.

Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 175
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 176
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 177
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 178
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 179
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 180
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 181
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 182
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 183
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 184
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 185
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 186
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 187
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 188
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 189
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 190
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 191
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 192
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 193
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 194
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 195
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 196
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 197
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 198
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 199
Suggested Citation:"7 Findings and Recommendations." National Academies of Sciences, Engineering, and Medicine. 2015. SBIR at the National Science Foundation. Washington, DC: The National Academies Press. doi: 10.17226/18944.
×
Page 200
Next: Appendixes »
SBIR at the National Science Foundation Get This Book
×
Buy Paperback | $75.00 Buy Ebook | $59.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Small Business Innovation Research (SBIR) program is one of the largest examples of U.S. public-private partnerships, and was established in 1982 to encourage small businesses to develop new processes and products and to provide quality research in support of the U.S. government’s many missions. The U.S. Congress tasked the National Research Council with undertaking a comprehensive study of how the SBIR program has stimulated technological innovation and used small businesses to meet federal research and development needs, and with recommending further improvements to the program. In the first round of this study, an ad hoc committee prepared a series of reports from 2004 to 2009 on the SBIR program at the five agencies responsible for 96 percent of the program’s operations -- including the National Science Foundation (NSF). Building on the outcomes from the first round, this second round presents the committee’s second review of the NSF SBIR program’s operations.

Public-private partnerships like SBIR are particularly important since today's knowledge economy is driven in large part by the nation's capacity to innovate. One of the defining features of the U.S. economy is a high level of entrepreneurial activity. Entrepreneurs in the United States see opportunities and are willing and able to assume risk to bring new welfare-enhancing, wealth-generating technologies to the market. Yet, although discoveries in areas such as genomics, bioinformatics, and nanotechnology present new opportunities, converting these discoveries into innovations for the market involves substantial challenges. The American capacity for innovation can be strengthened by addressing the challenges faced by entrepreneurs.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!