SBIR/STTR at the National Institutes of Health (2015)

1

Introduction

Small businesses are an important driver of innovation and economic growth in the United States.¹ Despite the challenges of changing global environments and the impacts of the 2008 financial crisis and subsequent recession, innovative small businesses continue to develop and commercialize new products for the market, improving the health and welfare of Americans while strengthening the nation’s security and competitiveness.²

Created in 1982 through the Small Business Innovation Development Act,³ the Small Business Innovation Research (SBIR) program remains the nation’s largest innovation program for small businesses. The SBIR program offers competitive awards to support the development and commercialization of innovative

_______________

¹See Z. Acs and D. Audretsch, “Innovation in Large and Small Firms: An Empirical Analysis,” The American Economic Review, 78(4):678-690, 1988. See also Z. Acs and D. Audretsch, Innovation and Small Firms, Cambridge, MA: The MIT Press, 1991; E. Stam and K. Wennberg, “The Roles of R&D in New Firm Growth,” Small Business Economics, 33:77-89, 2009; E. Fischer and A.R. Reuber, “Support for Rapid-Growth Firms: A Comparison of the Views of Founders, Government Policymakers, and Private Sector Resource Providers,” Journal of Small Business Management, 41(4):346-365, 2003; M. Henrekson and D. Johansson, “Competencies and Institutions Fostering High-Growth Firms,” Foundations and Trends in Entrepreneurship, 5(1):1-80, 2009.

²See D. Archibugi, A. Filippetti, and M. Frenz, “Economic Crisis and Innovation: Is Destruction Prevailing over Accumulation?” Research Policy, 42(2):303-314, 2013. The authors show that “the 2008 economic crisis severely reduced the short-term willingness of firms to invest in innovation” and also that it “led to a concentration of innovative activities within a small group of fast growing new firms and those firms already highly innovative before the crisis.” They conclude that “the companies in pursuit of more explorative strategies towards new product and market developments are those to cope better with the crisis.”

³Small Business Innovation Development Act of 1982, P.L. 97-219, July 22, 1982.

technologies by small private-sector businesses.⁴ At the same time, the program provides government agencies with technical and scientific solutions that address their various missions.

Seeking to bridge the gap between basic science and commercialization of resulting innovations, the Small Business Technology Transfer (STTR) program, created in 1992 by the Small Business Research and Development Enhancement Act of 1992,⁵ seeks to expand joint venture opportunities for small businesses and nonprofit research institutions. Under the STTR program, a small business receiving an award must collaborate formally with a research institution.

The SBIR/STTR programs consist of three phases:

• Phase I provides limited funding (up to $100,000 prior to the 2011 reauthorization and up to $150,000 thereafter) for feasibility studies.
• Phase II provides more substantial funding for further research and development (typically up to $750,000 prior to 2012 and $1 million after the 2011 reauthorization).⁶
• Phase III involves commercialization without providing access to any additional SBIR/STTR funding, although funding from other federal government accounts is permitted.

The SBIR program has four congressionally mandated goals: (1) stimulate technological innovation, (2) use small business to meet federal research and development (R&D) needs, (3) foster and encourage participation by minority and disadvantaged persons in technological innovation, and (4) increase private-sector commercialization derived from federal research and development.⁷ The goals for the STTR program are to (1) stimulate technological innovation, (2) foster technology transfer through cooperative R&D between small businesses and research institutions, and (3) increase private-sector commercialization of innovations derived from federal R&D.⁸ Each of the research agencies has sought to pursue

_______________

⁴SBIR awards can be made as grants or as contracts. Grants do not require the awardee to provide an agreed deliverable (for contracts this is often a prototype at the end of Phase II). Contracts are also governed by federal contracting regulations, which are considerably more onerous from the small business perspective. Historically, all Department of Defense (DoD) and National Aeronautics and Space Administration (NASA) awards have been contracts; all National Science Foundation (NSF) and most National Institutes of Health (NIH) awards have been grants, and the Department of Energy (DoE) has used both vehicles.

⁵Small Business Research and Development Enhancement Act, P.L. 102-564, S. 2941, Oct. 28, 1992.

⁶All resource and time constraints imposed by the program are somewhat flexible and are addressed by different agencies in different ways. For example, NIH and to a much lesser degree DoD have provided awards that are much larger than the standard amounts, and NIH has a tradition of offering no-cost extensions to allow for completion of work on an extended timeline.

⁷Small Business Innovation Development Act of 1982, P.L. 97-219, S. 881, July 22, 1982.

⁸Small Business Administration, “About STTR,” https://www.sbir.gov/about/about-sttr, accessed July 9, 2015.

these goals in administering its SBIR/STTR programs, utilizing the administrative flexibility built into the SBIR program to address its unique mission needs.⁹

Although the SBIR and STTR programs have similar objectives, they differ, according to the National Institutes of Health (NIH), “in two major ways related to the Program Director (PD)/Principal Investigator (PI) and non-profit research partner. Under SBIR, the PD/PI must be primarily employed with the small business concern at the time of award and for the duration of the project period, unless a waiver is granted by the NIH. Under the STTR Program, primary employment is not stipulated, so the PD/PI may be primarily employed by either the small business concern or the collaborating non-profit research institution at the time of award and for the duration of the project period.”¹⁰

The STTR program also differs from the SBIR program in that it requires that the small business concern formally collaborate with a nonprofit research institution. Research partnerships are permitted under the SBIR program, but the partnering research institution can complete no more than one-third of the Phase I work and no more than one-half of the Phase II work. In contrast, “Under STTR, the small business must perform at least 40 percent of the work and the research institution must perform at least 30 percent. The remaining 30 percent may be . . . [completed by] the small business concern, the collaborating non-profit research institution, or an additional third party.”¹¹

Over time, through a series of reauthorizations, SBIR/STTR legislation has required federal agencies with extramural R&D budgets in excess of $100 million to set aside a growing share of their budgets for the SBIR program and those with extramural R&D budgets in excess of $1 billion to set aside a growing share of their budgets for the STTR program (see Table 1-2). By FY2012, the 11 federal agencies, listed in Table 1-1, that administer SBIR/STTR programs were disbursing $2.4 billion dollars a year.¹² As shown in Figure 1-1, five agencies administer more than 96 percent of SBIR/STTR funds: Department of Defense (DoD), Department of Health and Human Services (HHS; including particularly NIH), National Aeronautics and Space Administration (NASA), National Science Foundation (NSF), and Department of Energy (DoE). Aggregate award amounts for the five largest agencies for FY2015 are provided in Table 1-2.

In December 2011, Congress reauthorized the SBIR/STTR programs for an additional 6 years,¹³ with a number of important modifications. Many of these modifications—for example, changes in standard award size—were consistent

_______________

⁹The committee commended this flexibility in its 2008 assessment of the SBIR program. See Finding C, National Research Council, An Assessment of the SBIR Program, Washington DC: The National Academies Press, 2008, p. 59.

¹⁰See https://sbir.nih.gov/about/critical, accessed on July 9, 2015.

¹¹Ibid.

¹²Small Business Association (SBA), SBIR/STTR annual report, http://www.sbir.gov/, accessed July 2015. FY2012 is the most recent year for which SBA publishes comparative data across agencies.

¹³Section 5137 of P.L. 112-81.

TABLE 1-1 Agencies Currently Participating in the SBIR and STTR Programs

SOURCE: Small Business Administration (SBA).

FIGURE 1-1 Percentage of total SBIR/STTR funding by agency, FY2012.
SOURCE: SBA, SBIR/STTR annual report, http://www.sbir.gov, accessed June 4, 2015.

with or followed recommendations made in a 2008 National Research Council (NRC)¹⁴ report on the SBIR program, a study mandated as a part of the program’s

_______________

¹⁴Effective July 1, 2015, the institution is called the National Academies of Sciences, Engineering, and Medicine. References in this report to the National Research Council or NRC are used in an historic context identifying programs prior to July 1.

TABLE 1-2 SBIR/STTR Funding by the Five Principal Funding Agencies, FY 2015

SOURCE: SBA awards database, https://www.sbir.gov/sbirsearch/award/all, accessed July 16, 2015.

2000 reauthorization.¹⁵ The 2011 reauthorization also called for further studies by the Academies.¹⁶

The first-round assessment resulted in 11 reports including the 2008 report cited above (see Box 1-1 for the list of reports). In a follow-up to the first round, NIH requested from the Academies an assessment focused on operational questions in order to identify further improvements to the program.

This introduction provides general context for the analysis of the program developments and transitions described in the remainder of the report. The first section provides an overview of the history of the SBIR/STTR programs across the federal government. This is followed by a summary of the major changes mandated through the 2011 reauthorization and the subsequent Small Business Administration (SBA) Policy Directive; a review of the programs’ advantages and limitations, in particular the challenges faced by entrepreneurs using (and seeking to use) the program and by agency officials running it; and a summary of the technical challenges facing this assessment and recommended solutions to those challenges.

PROGRAM HISTORY AND STRUCTURE¹⁷

During the 1980s, the perceived decline in U.S. competitiveness due to Japanese industrial growth in sectors traditionally dominated by U.S. firms—autos, steel, and semiconductors—led to concerns about future economic growth in the

_______________

¹⁵National Research Council, An Assessment of the SBIR Program. The National Research Council’s first-round assessment of the SBIR program was mandated in the SBIR Reauthorization Act of 2000, P.L. 106-554, Appendix I-H.R. 5667, Section 108.

¹⁶The National Defense Reauthorization Act for Fiscal Year 2012, P.L. 112-81, Section 5137.

¹⁷Parts of this section are based on the Academies’ previous report on the NIH SBIR program, An Assessment of the SBIR Program at the National Institutes of Health, Washington, DC: The National Academies Press, 2009.

United States.¹⁸ A key concern was the perceived failure of American industry “to translate its research prowess into commercial advantage.”¹⁹ Although the United States enjoyed dominance in basic research—much of which was feder-

_______________

¹⁸See J. Alic, “Evaluating competitiveness at the Office of Technology Assessment,” Technology in Society, 9(1):1-17, 1987, for a review of how these issues emerged and evolved within the context of a series of analyses at a Congressional agency.

¹⁹D.C. Mowery, “America’s industrial resurgence (?): An overview,” in D.C. Mowery, ed., U.S. Industry in 2000: Studies in Competitive Performance, Washington, DC: National Academy Press, 1999, p. 1. Other studies highlighting poor economic performance in the 1980s include M.L. Dertouzos et al., Made in America: The MIT Commission on Industrial Productivity, Cambridge, MA: The MIT Press, 1989; and O. Eckstein, DRI Report on U.S. Manufacturing Industries, New York: McGraw Hill, 1984.

ally funded—applying this research to the development of innovative products and technologies remained a challenge. As the great corporate laboratories of the post-war period were buffeted by change, new models such as the cooperative model utilized by some Japanese keiretsu seemed to offer greater sources of dynamism and more competitive firms.

At the same time, new evidence emerged to indicate that small businesses were an increasingly important source of both innovation and job creation.²⁰ This evidence reinforced recommendations from federal commissions dating back to the 1960s, that federal R&D funding should provide more support for innovative small businesses (which was opposed by traditional recipients of government R&D funding).²¹

Early-stage financial support to innovative technology-based small businesses for developing high-risk technologies with commercial promise was first advanced by Roland Tibbetts at NSF. In 1976, Mr. Tibbetts advocated shifting some NSF funding for this purpose. NSF adopted this initiative first, and after a period of analysis and discussion, the Reagan administration supported an expansion of this initiative across the federal government. Congress then passed the Small Business Innovation Research Development Act of 1982, which established the SBIR program.

Initially, the SBIR program required agencies with extramural R&D budgets in excess of $100 million²² to set aside 0.2 percent of their funds for SBIR. Program funding totaled $45 million in the program’s first year of operation (1983). Over the next 6 years, the set-aside grew to 1.25 percent.²³

SBIR Reauthorizations of 1992 and 2000

The SBIR program approached reauthorization in 1992 amidst continued worries about the ability of U.S. firms to commercialize inventions. (See Box 1-2.) Finding that “U.S. technological performance is challenged less in the creation of new technologies than in their commercialization and adoption,” the Academies

_______________

²⁰For an alternate view, see S.J. Davis, J. Haltiwanger, and S. Schuh, Small Business and Job Creation: Dissecting the Myth and Reassessing the Facts, Working Paper No. 4492, Cambridge, MA: National Bureau of Economic Research, 1993. Evaluating the empirical basis for conventional claims about the job-creating prowess of small businesses, the authors find inter alia that conventional wisdom about the job-creating prowess of small business rests on misleading interpretations of the data.

²¹For an overview of the origins and history of the SBIR program, see G. Brown and J. Turner, “The federal role in small business research,” Issues in Science and Technology, Summer 1999, pp. 51-58.

²²That is, those agencies spending more than $100 million on research conducted outside agency labs.

²³Additional information regarding SBIR’s legislative history can be accessed from the Library of Congress. See http://thomas.loc.gov/cgi-bin/bdquery/z?d097:SN00881:@@@L.

recommended an increase in SBIR funding as a means to improve the economy’s ability to adopt and commercialize new technologies.²⁴

The Small Business Research and Development Enhancement Act (P.L. 102-564) reauthorized the SBIR program until September 30, 2000, and doubled the set-aside rate to 2.5 percent. The legislation also more strongly emphasized the need for commercialization of SBIR-funded technologies.²⁵ Legislative language explicitly highlighted commercial potential as a criterion for awarding SBIR contracts and grants.

At the same time, Congress expanded the SBIR program’s purposes to “emphasize the program’s goal of increasing private sector commercialization developed through federal research and development and to improve the federal government’s dissemination of information concerning the small business innovation, particularly with regard to woman-owned business concerns and by socially and economically disadvantaged small business concerns.”²⁶

The Small Business Reauthorization Act of 2000 (P.L. 106-554) extended the SBIR program until September 30, 2008. It also called for an NRC assessment of

_______________

²⁴See National Research Council, The Government Role in Civilian Technology: Building a New Alliance, Washington, DC: National Academy Press, 1992, p. 29.

²⁵Small Business Research and Development Enhancement Act, P.L. 102-564, S. 2941, October 28, 1992. See also R. Archibald and D. Finifter, “Evaluation of the Department of Defense Small Business Innovation Research program and the Fast Track Initiative: A balanced approach,” in National Research Council, The Small Business Innovation Research Program: An Assessment of the Department of Defense Fast Track Initiative, Washington, DC: National Academy Press, 2000, pp. 211-250.

²⁶Small Business Research and Development Enhancement Act, P.L. 102-564, S. 2941, October 28, 1992.

the program’s broader impacts, including those on employment, health, national security, and national competitiveness.²⁷

STTR Reauthorizations

Established by the Small Business Technology Transfer Act of 1992 (P.L. 102-564, Title II), the STTR program was reauthorized until the year 2001 by the Small Business Reauthorization Act of 1997 (P.L. 105-135) and reauthorized again until September 30, 2009, by the Small Business Technology Transfer Program Reauthorization Act of 2001 (P.L. 107-50).

As explained below, the SBIR/STTR Reauthorization Act of 2011 included a number of changes to the SBIR/STTR programs, including increases in the set-asides over the next 6 years and expanded eligibility for STTR awardees to take part in technical assistance programs.

The 2011 SBIR/STTR Reauthorization

The anticipated 2008 reauthorization was delayed in large part by a disagreement between long-time program participants and their advocates in the small business community and proponents of expanded access for venture-backed firms, particularly in biotechnology where proponents argued that the standard path to commercial success includes venture funding at some point.²⁸ Other issues were also difficult to resolve, but the conflict over participation of venture-backed companies dominated the process²⁹ following an administrative decision to exclude these firms more systematically.³⁰

After a much extended discussion, passage of the National Defense Act of December 2011 reauthorized the SBIR/STTR programs through FY2017.³¹ The new law maintained much of the core structure of both programs but made some

_______________

²⁷The current assessment is congruent with the Government Performance and Results Act (GPRA) of 1993: http://govinfo.library.unt.edu/npr/library/misc/s20.html. As characterized by the Government Accountability Office (GAO), GPRA seeks to shift the focus of government decision making and accountability away from a preoccupation with the activities that are undertaken—such as grants dispensed or inspections made—to the results of those activities. See http://www.gao.gov/new.items/gpra/gpra.htm.

²⁸For a review of the issues, see National Research Council, Venture Funding and the NIH SBIR Program, Washington DC: National Academies Press, 2009. See also D.C. Specht, “Recent SBIR extension debate reveals venture capital influence,” Procurement Law, 45:1, 2009.

²⁹W.H. Schacht, “The Small Business Innovation Research (SBIR) program: Reauthorization efforts,” Congressional Research Service, Library of Congress, 2008.

³⁰A. Bouchie, “Increasing number of companies found ineligible for SBIR funding,” Nature Biotechnology, 21(10):1121-1122, 2003.

³¹SBIR/STTR Reauthorization Act of 2011, P.L. 112-81, December 31, 2011.

important changes, which were to be implemented via the SBA’s subsequent Policy Guidance.³²

The eventual compromise on the venture funding issue allowed (but did not require) agencies to award up to 25 percent of their SBIR grants or contracts (at NIH, DoE, and NSF) or 15 percent (at the other awarding agencies) to firms that benefit from private, venture capital investment. It is too early in the implementation process to gauge the impact of this change.

The reauthorization made changes to the SBIR program that were recommended in prior Academies reports.³³ These included the following:

• Increased award size limits
• Expanded program size
• Enhanced agency flexibility—for example, for Phase I awardees from other agencies to be eligible for Phase II awards or to provide an additional Phase II award
• Improved incentives for the utilization of SBIR technologies in agency acquisition programs
• Explicit requirements for better connecting prime contractors with SBIR awardees³⁴
• Substantial emphasis on developing a more data-driven culture, which has led to several major reforms, including the following:
  • adding numerous areas of expanded reporting
  • extending the Academies’ evaluation
  • adding further evaluation, such as by the Government Accountability Office and Comptroller General
  • tasking the SBA with creating a unified platform for the collection of data from agencies with SBIR/STTR agencies
• Expanded management resources (through provisions permitting use of up to 3 percent of program funds for [defined] management purposes)
• Expanded commercialization support (through provisions providing companies with direct access to commercialization support funding and through approval of the approaches piloted in Commercialization Pilot Programs)
• Options for agencies to add flexibility by developing other pilot programs—for example, to allow awardees to skip Phase I and apply for a Phase II award directly or for NIH to support a new Phase 0 pilot program

_______________

³²See SBA post, S. Greene, “Implementing the SBIR and STTR Reauthorizations: Our Plan of Attack,” 02/21/2012. http://www.sbir.gov/news/implementing-sbir-and-sttr-reauthorization-our-plan-attack, February 21, 2012.

³³See Appendix B for a list of the major changes to the SBIR program resulting from the 2011 Reauthorization Act.

³⁴Prime Contractors, who have primary contracts for a project, are often interested in subcontracting with small businesses.

The reauthorization also made changes that were not mentioned in previous reports of the Academies. These included the following:

• Expansion of the STTR program
• Limitations on agency flexibility—particularly in the provision of larger awards
• Introduction of commercialization benchmarks for companies, which must be met if companies are to remain in the program. These benchmarks are to be established by each agency.

Other clauses of the legislation affect operational issues, such as the definition of specific terms (such as “Phase III”), continued and expanded evaluation by the Academies, mandated reports from the Comptroller General on combating fraud and abuse within the SBIR program, and protection of small firms’ intellectual property within the program.

PREVIOUS RESEARCH ON SBIR

Studies pre-dating the Academies’ first-round assessment in 2002–2009, most notably by the Government Accountability Office and the SBA, focused only on specific aspects or components of the SBIR/STTR programs.³⁵ In addition, prior to the first-round assessment, there had been few internal assessments of agency SBIR/STTR programs. The academic literature on SBIR was also limited,³⁶ except for an assessment in the 1990s by Joshua Lerner of the Harvard Business School who found “that SBIR awardees grew significantly faster than a matched set of firms over a ten-year period.”³⁷

To help fill this assessment gap, the NRC’s Committee for Government-Industry Partnerships for the Development of New Technologies (GIP, which preceded the NRC’s first-round congressionally mandated study of the SBIR program) convened a workshop in 1998 to discuss the SBIR program’s history

_______________

³⁵An important step in the evaluation of the program has been to identify existing evaluations of the program. These include U.S. Government Accounting Office, Federal Research: Small Business Innovation Research Shows Success But Can Be Strengthened, Washington, DC: U.S. General Accounting Office, 1992; and U.S. Government Accounting Office, Evaluation of Small Business Innovation Can Be Strengthened, Washington, DC: U.S. General Accounting Office, 1999. There is also a 1999 unpublished SBA study on the commercialization of SBIR Phase II awards from 1983 to 1993 among non-DoD agencies.

³⁶Early examples of evaluations of the SBIR program include S. Myers, R. L. Stern, and M. L. Rorke, A Study of the Small Business Innovation Research Program, Lake Forest, IL: Mohawk Research Corporation, 1983; and Price Waterhouse, Survey of Small High-tech Businesses Shows Federal SBIR Awards Spurring Job Growth, Commercial Sales, Washington, DC: Small Business High Technology Institute, 1985.

³⁷See J. Lerner, “The government as venture capitalist: The long-run effects of the SBIR program,” Journal of Business, 72(3), 1999.

and rationale, review existing research, and identify areas for further research and program improvements.³⁸ In addition, in its report on the SBIR Fast Track Initiative at the Department of Defense, the GIP committee found that the SBIR program contributed to mission goals by funding “valuable innovative projects.”³⁹ It concluded that a significant number of these projects would not have been undertaken absent SBIR funding⁴⁰ and that DoD’s Fast Track Initiative encouraged the commercialization of new technologies⁴¹ and the entry of new firms into the program.⁴² The GIP committee also found that the SBIR program improved both the development and utilization of human capital and the diffusion of technological knowledge.⁴³ Case studies provided some evidence that the knowledge and human capital generated by the SBIR program have positive economic value, which spills over into other firms through the movement of people and ideas.⁴⁴ Furthermore, by validating promising new technologies, SBIR awards encourage further private-sector investment in an award-winning firm’s technology.⁴⁵

ROUND ONE ASSESSMENT OF THE SBIR PROGRAM

The 2000 SBIR reauthorization mandated that the NRC complete a comprehensive assessment of the SBIR program.⁴⁶ The assessment of the SBIR programs at DoD, NIH, NASA, NSF, and DoE began in 2002 and was conducted in three steps. As a first step, the committee authoring this study developed a research methodology⁴⁷ and gathered information about the program by convening workshops where officials at the relevant federal agencies described their program operations, challenges, and accomplishments. These meetings highlighted the important differences in agency goals, practices, and evaluations. They also served to describe the evaluation challenges that arise from the diversity in program objectives and practices.⁴⁸

The committee implemented the research methodology during the second step. As set out in the methodology, multiple data collection modalities were deployed. This included the first large-scale survey of SBIR award recipients. Case

_______________

³⁸See National Research Council, The Small Business Innovation Research Program: Challenges and Opportunities, Washington, DC: National Academy Press, 1999.

³⁹National Research Council, An Assessment of the DoD SBIR Fast Track Initiative, 32.

⁴⁰Ibid., 32.

⁴¹Ibid., 33.

⁴²Ibid., 34.

⁴³Ibid., 33.

⁴⁴Ibid., 33.

⁴⁵Ibid., 33.

⁴⁶SBIR Reauthorization Act of 2000, P.L. 106-554, Appendix I-H.R. 5667, Section 108.

⁴⁷National Research Council, An Assessment of the Small Business Innovation Research Program: Project Methodology, Washington, DC: The National Academies Press, 2004.

⁴⁸Adapted from National Research Council, SBIR: Program Diversity and Assessment Challenges, op. cit.

studies were also developed on a wide variety of SBIR firms. The committee then evaluated the results and developed the findings and recommendations presented for improving the effectiveness of the SBIR program.

During the third step, the committee reported on the program through a series of publications in 2008-2010: five individual volumes on the major funding agencies and an additional overview volume titled An Assessment of the SBIR Program.⁴⁹ Together, these reports provided the first detailed and comprehensive review of the SBIR program and, as noted above, served as an important input into SBIR reauthorization prior to December 2011 (see Box 1-1).

CURRENT, ROUND TWO STUDY: CHALLENGES AND OPPORTUNITIES

The first-round study of the SBIR program found that the program was, overall, “sound in concept and effective in practice.”⁵⁰ Furthermore, in its review of the NIH SBIR program, the committee concluded: “The NIH SBIR program is making significant progress in achieving the congressional goals for the program”⁵¹ [emphasis in original]. The current study, described in the Statement of Task in Box 1-3, provides a second snapshot to measure the program’s progress against its legislative goals.

This volume partially addresses this Statement of Task. It is supplemented by a number of workshops and other publications from the Committee on Capitalizing on Science, Technology, and Innovation: An Assessment of the Small Business Innovation Research Program—Phase II. For example, workshops were convened on the participation of women and minorities in the SBIR/STTR programs (February 2013), the evolving role of university participation in the programs (February 2014), the relationship between state innovation programs and the SBIR program (October 2014), the STTR program (May 2015), and the economics of entrepreneurship in relation to the SBIR program (June 2015). The committee published a report on Innovation, Diversity, and Success in the SBIR/STTR Programs (2015), based on the 2013 workshop.

The current volume updates the Academies’ 2009 assessment of the NIH SBIR program, by refreshing the data, providing new descriptions of recent programs and developments, and providing fresh company case studies. Guided by this Statement of Task, the committee has sought answers to questions such as the following:

_______________

⁴⁹National Research Council, An Assessment of the SBIR Program.

⁵⁰Ibid., 54.

⁵¹National Research Council, An Assessment of the Small Business Innovation Research Program at the National Institutes of Health, 19.

• Are there initiatives and programs within NIH that have made a significant difference to outcomes and in particular to agency take-up of SBIR-funded technologies?
• Can they be replicated and expanded?
• What are the main barriers to meeting Congressional objectives more fully?
• What program adjustments would better support commercialization?

• Are there tools that would expand utilization by woman and minorityowned firms and participation by female and minority principal investigators?
• Can links with universities be improved?
• Are there aspects of the program that make it less attractive to small firms? Could they be addressed?
• What can be done to expand access in underserved states while maintaining the competitive character of the program?
• Can the program generate better data on both process and outcomes and use those data to fine-tune program management?

STUDY METHODOLOGY

The SBIR/STTR programs are unique in terms of scale and mission focus. In addition, the evidence suggests that there are no truly comparable programs in the United States, and those in other countries operate in such different ways that their relevance is limited.⁵² Thus, it is difficult to identify programs comparable to SBIR/STTR against which to benchmark their results.

Assessing the SBIR/STTR programs at NIH is challenging for other reasons as well. Unlike DoD and NASA, SBIR/STTR awards at NIH are not primarily designed to generate tools and capabilities for agency use. They are instead explicitly designed to generate technologies that will be adopted outside the agency, primarily in the private sector. Thus success cannot be measured internally by commercialization of projects sold to the agency.

The NIH SBIR/STTR programs are also highly decentralized. Although the SBIR/STTR program office within the Office of Extramural Programs sets policy and provides critical cross-agency communication flows, as well as links the program to outside stakeholders, award funding is determined by each Institute or Center (IC) separately. ICs take different views of the program and use different approaches to program management. Therefore, generalizations about the NIH SBIR/STTR programs must be made with care. Indeed, approximately 24 separate programs are run by the various NIH ICs.

Focus on Legislative Objectives

This volume—and this study—do not seek to provide a comprehensive review of the value of the SBIR/STTR programs, in particular measured against other possible uses of federal funding. Such a review is beyond the study’s scope. Rather, the work is focused on assessing the extent to which the NIH SBIR/STTR programs have met their congressionally mandated objectives, determining in

_______________

⁵²See National Academies of Sciences, Engineering, and Medicine, Workshop on “Learning from Each Other: U.S. European Perspectives on Small Business Innovation Programs,” Washington, DC, March 19, 2015.

particular whether recent initiatives have improved program outcomes, and providing recommendations for further program improvements.⁵³

Thus, as in the first-round study, this second-round study will “not consider whether or not SBIR should exist”—Congress and the President have already decided affirmatively on this question, most recently in the 2011 reauthorization of the program.⁵⁴ Rather, this study is charged with “providing assessment-based findings of the benefits and costs of SBIR . . . to improve public understanding of the program, as well as recommendations to improve the program’s effectiveness.” Also as in the first-round study, this study will “not seek to compare the value of one area with other areas; this task is the prerogative of the Congress and the Administration acting through the agencies. Instead, the study is concerned with the effective review of each area.”⁵⁵

Defining Commercialization

Commercialization offers practical and definitional challenges. As described in Chapter 5, several different definitions of commercialization can be used when discussing the SBIR/STTR programs. In fact, it is important to use more than one simple definition. For example, the percentage of funded projects that reach the marketplace is not the only measure of commercial success.

In the private sector, commercial success over the long term requires profitability. However, in the short term, the path to successful commercialization can involve many different aspects of commercial activity, from product rollout to licensing to patenting to acquisition. Even during new product rollout, companies often do not generate immediate profits. This report uses multiple metrics to address the question of commercialization (see Chapter 5).

Quantitative Assessment Methods

More practically, several issues relate to the application of quantitative assessment methods, including decisions about which kinds of program participants should be targeted for survey deployment, the number of responses that are appropriate, selection bias, nonresponse bias, the design and implementation of survey questionnaires, and the level of statistical evidence required for drawing conclusions in this case. These and other issues were discussed at a workshop

_______________

⁵³These limited objectives are consistent with the methodology developed by the committee. See National Research Council, An Assessment of the Small Business Innovation Research Program: Project Methodology.

⁵⁴National Defense Authorization Act of 2012 (NDAA), HR.1540, Title LI.

⁵⁵National Research Council, An Assessment of the Small Business Innovation Research Program: Project Methodology.

and summarized in a 2004 report.⁵⁶ Also, as noted above, a peer-reviewed report on the study methodology completed by the first-round committee provided the baseline for the initial study and for follow-on studies—including this one.⁵⁷

Survey Development

For the current study, a survey of SBIR and STTR award recipients was developed and deployed, a necessity given the absence of quantitative outcomes data at NIH. The survey was based closely on previous surveys, particularly the 2005 Survey that focused exclusively on SBIR, but nonetheless included significant improvements.⁵⁸ The description of the survey and improvements in methodology, including a discussion of the survey outreach and response, are documented in Appendix A. Most notably, the survey development made an ambitious but ultimately unsuccessful effort to develop a comparison group to provide context and a benchmark for analyzing the results (this effort is also discussed in Appendix A).

The survey delved more deeply into the demographics of the program. It also included questions about the role of agency liaisons who deal with contract operations and thereby provide a link between individual projects and NIH. Furthermore, it provided unique opportunities to collect qualitative opinions on the program and recommendations for improvement from award recipients.

It was the intention of the 2014 Survey to send a questionnaire to every principal investigator (PI) who received a Phase II award from NIH during fiscal years 2001-2010. The preliminary population prior to contact was 3,375. Of these, 1,723 were determined to be not contactable at the SBIR/STTR company listed in the NIH awards database.⁵⁹ The remaining 1,652 awards constitute the effective population for this study. From this group, 726 responses were received, for a preliminary population response rate of 21.5 percent and an effective population response rate of 43.9 percent. PIs of more than one awarded project were asked to complete a maximum of two questionnaires.

Appendix A provides a detailed discussion of the issues related to quantitative methodologies, as well as a review of potential biases. As a result of the relatively small response rate, there are significant limitations on the conclusions that can be drawn from this quantitative assessment, which is reflected in the wording of findings and recommendations (Chapter 8). At the same time, drawing on

_______________

⁵⁶National Research Council, The Small Business Innovation Research Program: Program Diversity and Assessment Challenges, National Academies Press, Washington DC, 2004.

⁵⁷National Research Council, An Assessment of the Small Business Innovation Research Program: Project Methodology.

⁵⁸The survey carried out as part of this study was administered in 2014, and the survey completed as part of the Academies’ first-round assessment of the SBIR program was administered in 2005. In this volume all survey references are to the 2014 survey unless noted otherwise.

⁵⁹See Appendix A for a description of the survey effort.

quantitative analysis is a crucial component of the overall study, reflective of the need to identify and assess outcomes that are found only by querying individual projects and participating companies.

A Complement of Approaches

Partly because of these limitations, the 2004 methodology report stressed the importance of utilizing a complement of research modalities, an approach that has been adopted here.⁶⁰ Although quantitative assessment represents the bedrock of our research and provides insights and evidence that could not be generated through any other modality, it is, in and of itself, insufficient to address the multiple questions posed in this analysis. Consequently, we undertook a series of additional activities:

• Case studies. We conducted in-depth case studies of 20 NIH SBIR recipients. These companies were geographically and demographically diverse (by sex and race), funded by different NIH ICs, focused on different kinds of technologies, and at different stages of the company lifecycle. Lessons from the case studies are described in Chapter 7, and the case studies themselves are included as Appendix E.
• Workshops. We conducted workshops, including workshops to discuss the participation of women and minorities and the role of universities in the SBIR/STTR programs,⁶¹ to allow stakeholders, agency staff, and academic experts to provide insights into program operations, as well as to identify issues that need to be addressed.
• Analysis of agency data. As appropriate, we analyzed and included data from NIH that cover various aspects of SBIR/STTR activities.
• Open-ended responses from SBIR/STTR recipients. For the first time, we collected textual responses in the survey. More than 450 recipients provided narrative comments. These comments are addressed in Chapter 7.
• Agency consultations. We engaged in discussions with agency staff at several of the Centers about the operation of their programs and the challenges they face.

_______________

⁶⁰National Research Council, An Assessment of the Small Business Innovation Research Program: Project Methodology.

⁶¹Workshops convened by the committee as part of the overall analysis include NASA Small Business Innovation Research Program Assessment: Second Phase Analysis, January 28, 2010; Early-Stage Capital in the United States: Moving Research Across the Valley of Death and the Role of SBIR, April 16, 2010; Early-Stage Capital for Innovation—SBIR: Beyond Phase II, January 27, 2011; NASA’s SBIR Community: Opportunities and Challenges, June 21, 2011; Innovation, Diversity, and Success in the SBIR/STTR Programs, February 7, 2013; Commercializing University Research: The Role of SBIR and STTR, February 5, 2014; SBIR/STTR & the Role of State Programs, October 7, 2014; The Small Business Technology Transfer Program, May 1, 2015, and the Economics of Entrepreneurship, June 29, 2015. Each of these workshops was held in Washington, DC.

• Literature review. Since the start of our research in this area, a number of academic and policy papers have been published that address various aspects of the SBIR/STTR programs, many drawing from the survey and other data made available by our reviews. In addition, other organizations—such as the Government Accountability Office—have reviewed specific parts of the SBIR/STTR programs. The committee has incorporated references to this work, where useful, into its analysis.

Data Sources and Limitations

Multiple research modalities are especially important because limitations still exist in the data collected for the SBIR/STTR programs. As described in Chapter 5, the survey deployed in the past by NIH to identify SBIR outcomes is no longer current. Furthermore, NIH thus does not maintain a comprehensive dataset on award outcomes and cannot provide data about the take-up of technologies funded by the SBIR/STTR programs. NIH is however making efforts to address this issue (see Chapter 5).

The lack of current outcomes data from NIH means that the current survey provides the only available quantitative data on SBIR/STTR outcomes and processes at NIH.

Cooperation with NIH

In general, we received substantial cooperation from NIH and its ICs. Agency staff and researchers deployed by the committee engaged in numerous discussions, and NIH provided data, papers, and presentations.

In short, within the limitations described, the study utilizes a complement of tools to ensure that a wide spectrum of perspectives and expertise is reflected in the findings and recommendations. Appendix A provides an overview of the methodological approaches, data sources, and survey tools used in this study.

ORGANIZATION OF THE REPORT

The analysis and conclusions are organized as follows. Chapter 2 provides a review of program operations, describing the program in some detail and addressing a range of issues related to program management. Chapter 3 describes and analyzes agency initiatives that have been developed and implemented over the past 8 to 10 years, including the role of awards larger than SBA guidelines (approved with an SBA waiver). Chapter 4 reviews NIH data concerning applications and awards, drawing out demographic and geographic differences as well as previous experience with the program. Chapter 5 provides a quantitative assessment of the program, based primarily on the 2014 Survey in the absence of data from NIH or other sources. Chapter 6 addresses the congressional man-

date to foster the participation of women and minorities, utilizing data from NIH and from the 2014 Survey. Chapter 7 draws on company case studies and on the textual responses from survey respondents to provide a qualitative picture of program operations, issues, and possible solutions. Chapter 8 provides the findings and recommendations from the study.

The report’s appendixes provide additional information. Appendix A sets out an overview of the methodological approaches, data sources, and survey tools used in this assessment. Appendix B describes key changes to the SBIR program from the 2011 reauthorization. Appendix C reproduces the 2014 Survey instrument. Appendix D lists the research institutions involved in NIH SBIR/STTR awards. Appendix E presents the case studies of selected firms with NIH awards. Appendix F provides a glossary of acronyms used, and Appendix G provides a list of references.