Consensus Study Report
NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This activity was supported by a contract between the National Academy of Sciences and the National Science Foundation (49100419C0006). Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-70027-6
International Standard Book Number-10: 0-309-70027-2
Digital Object Identifier: https://doi.org/10.17226/26884
This publication is available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2023 by the National Academy of Sciences. National Academies of Sciences, Engineering, and Medicine and National Academies Press and the graphical logos for each are all trademarks of the National Academy of Sciences. All rights reserved.
Printed in the United States of America.
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2023. Review of the SBIR and STTR Programs at the National Science Foundation. Washington, DC: The National Academies Press. https://doi.org/10.17226/26884.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.
Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.
Rapid Expert Consultations published by the National Academies of Sciences, Engineering, and Medicine are authored by subject-matter experts on narrowly focused topics that can be supported by a body of evidence. The discussions contained in rapid expert consultations are considered those of the authors and do not contain policy recommendations. Rapid expert consultations are reviewed by the institution before release.
For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.
INNOVATION RESEARCH AND SMALL BUSINESS TECHNOLOGY TRANSFER PROGRAMS AT THE NATIONAL SCIENCE FOUNDATION
MARYANN P. FELDMAN (Co-Chair), Watts Endowed Professor of Public Affairs, Arizona State University
SCOTT STERN (Co-Chair), David Sarnoff Professor of Management, Massachusetts Institute of Technology
YAEL HOCHBERG, Ralph S. O’Connor Professor in Entrepreneurship–Finance, Rice University
AMOL M. JOSHI, Associate Professor of Strategic Management, Wake Forest University
RIITTA KATILA, W.M. Keck Sr. Professor of Management Science, and Research Director of the Stanford Technology Ventures Program, Stanford University
LAUREN LANAHAN, Inman Research Scholar and Associate Professor of Management, University of Oregon
MATT MARX, Bruce F. Failing, Sr., Chair in Entrepreneurship, Cornell University
ALEXANDER OETTL, Associate Professor of Strategy & Innovation, Georgia Institute of Technology
WINSLOW SARGEANT, Senior Advisor for Globalization and Head of Capital Markets, Genaesis, LLC, and Managing Director, S&T, LLC
STEPHANIE S. SHIPP, Research Professor and Interim Director, Social and Decision Analytics Division, Biocomplexity Institute, University of Virginia
TIMOTHY SIMCOE, Professor of Strategy & Innovation, Boston University
Study Staff
GAIL E. COHEN, Study Director
MEGHAN ANGE-STARK, Associate Program Officer (through July 2021)
SOPHIE BILLINGE, Senior Program Assistant (through June 2022)
DAVID DIERKSHEIDE, Program Officer
Consultant
EVAN E. JOHNSON
BOARD ON SCIENCE, TECHNOLOGY, AND ECONOMIC POLICY
ADAM B. JAFFE (Chair), Brandeis University
NOËL BAKHTIAN, Bezos Earth Fund
BRENDA J. DIETRICH (NAE), Cornell University
BRIAN G. HUGHES, HBN Shoe, LLC
PAULA E. STEPHAN, Georgia State University
SCOTT STERN, Massachusetts Institute of Technology
JOHN C. WALL (NAE), Cummins, Inc. (Retired)
JOHN L. ANDERSON (NAE), Ex Officio Member, National Academy of Engineering
VICTOR J. DZAU (NAM), Ex Officio Member, National Academy of Medicine
MARCIA McNUTT (NAS/NAE), Ex Officio Member, National Academy of Sciences
Staff
GAIL COHEN, Senior Director
DAVID DIERKSHEIDE, Program Officer
GRETE GANSAUER, Christine Mirzayan Science and Technology Fellow
CLARA SAVAGE, Financial Officer
Reviewers
This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following individuals for their review of this report:
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by STEVEN BATTEL, Battel Engineering. He was responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.
This page intentionally left blank.
Contents
ESTABLISHMENT OF THE SBIR AND STTR PROGRAMS: BACKGROUND AND CONTEXT
THEORETICAL SUPPORT FOR THE SBIR AND STTR PROGRAMS
CONFLICTING PROGRAMMATIC GOALS
THE NATIONAL ACADEMIES STUDY MANDATE
STUDY METHODOLOGY AND LIMITATIONS
THE NATIONAL SCIENCE FOUNDATION: WHERE DISCOVERIES BEGIN
SURVEY OF THE EMPIRICAL LITERATURE
BROAD CHALLENGES FACING SBIR/STTR ASSESSMENT
APPROACH TO DEALING WITH EVALUATION CHALLENGES
NSF’S SBIR/STTR OFFICE AND TOPIC SOLICITATION
NSF SBIR/STTR PROGRAM DIRECTORS
OUTREACH BY PROGRAM DIRECTORS AND EFFORTS TO IMPROVE DIVERSITY
ANNEX 3-1: QUESTIONS FOR PROGRAM DIRECTOR INTERVIEWS
4 The Landscape of NSF SBIR/STTR Awardees
5 The Impact of the NSF SBIR/STTR Programs: Outcomes of NSF SBIR and STTR Awardees
ASSESSING CAUSAL IMPACTS OF NSF SBIR/STTR AWARDS: ESTABLISHING A COMPARISON GROUP
INNOVATION AND COMMERCIALIZATION OUTCOMES
Boxes, Figures, and Tables
BOXES
S-2 Complete List of Recommendations
3-1 Participation in NSF’s Beat the Odds Boot Camp
3-2 Participation in the NSF Commercialization Programs
FIGURES
3-2 Number of NSF SBIR/STTR Phase I applications by year, fiscal years 2002–2021
4-1 NSF SBIR/STTR applications from woman-owned small businesses (WOSBs)
4-6 Share of SBIR/STTR Phase I awards to first-time recipients, fiscal years 2000–2021
4-7 Share of NSF SBIR/STTR awards to principal investigators with prior NSF grants
5-1 Primary industry group of firms receiving venture capital funding, 1983–2019
5-2 Demographics of venture capital recipients (January 2000–June 2022)
TABLES
1-1 Federal Agencies Participating in the SBIR and STTR Programs
3-1 NSF SBIR/STTR Timeline Description
4-2 Average Firm Age and Number of Employees for NSF SBIR and STTR Awardees, Fiscal Years 2011–2021
4-3 Agency Shares of First-time and Repeat SBIR/STTR Award Recipients, Fiscal Years 1985–2020
4-5 Previous NSF Grants to NSF SBIR/STTR Awardees, Fiscal Years 2000–2021
4-6 Top 10 States for NSF Awards, Fiscal Years 2012–2021
4-7 Characteristics of NSF SBIR–Awarded Firms versus NSF STTR–Awarded Firms, 2001–2019
4-8 Cumulative Awards for NSF SBIR– and STTR–Awarded Firms at Ages 5 and 10
4-9 Characteristics of Research Partners for NSF STTR–Awarded Firms
4-10 Share of NSF SBIR/STTR Awardees That Had Previously Participated in Innovation Corps (I-Corps)
5-1 SBIR/STTR Awardees and Control Group: Differences in Advancing Science
5-2 Differences in Patenting between NSF SBIR/STTR Awardees and the Control Group
5-4 Distribution of Venture Capital Funding for NSF SBIR/STTR Awardees and the Control Group
5-5 Progress toward Liquidity Events
5-1.1 Patents, Publications, and Commercial Products of Awardees on the NSF Showcase Website
Preface
This report is the result of a request by Congress for an assessment of the Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs at each of the principal agencies that conduct or fund research and development activities across the federal government. The SBIR program has become the largest and most comprehensive public research and development funding program for small business research in the United States, and indeed has been emulated by other countries. An underlying tenet of the SBIR program, and the related STTR program, is that small and young firms are an important source of new ideas that provide the basis for technological innovation, productivity increases, and subsequent economic growth. Predicated on the observation that it is difficult for small and young firms to find financial support for their ideas, the SBIR/STTR programs have become known as America’s Seed Fund. Yet this characterization captures only one dimension of the legislative objectives and operation of the programs. By involving qualified small businesses in the nation’s research and development efforts, SBIR/STTR awards stimulate the development of innovative technologies, help move research closer to the market, and address the needs of citizens underserved because of limited market incentives. Equally important, the SBIR/STTR programs aim to help federal agencies fulfill their missions and objectives and provide a pathway for firms owned by women and socially or economically disadvantaged persons.
Specifically, this report focuses on the operation and performance of the SBIR/STTR programs at the National Science Foundation (NSF). Most notably, the committee convened by the National Academies of Sciences, Engineering, and Medicine to carry out this study undertook a detailed assessment of the process by which SBIR and STTR awards are made at NSF, a survey of the landscape of awards that have been granted (broken down by geography, gender, and socioeconomic characteristics, as well as research topic), and a detailed quantitative analysis examining the innovation and commercialization outcomes of firms participating in the programs (compared against firms that did not receive an award and NSF-funded researchers that did not participate in the programs). Collectively these analyses, documented in this report, are intended to offer a
more comprehensive and precise assessment of the SBIR and STTR programs than has been provided in previous studies of this agency carried out by the National Academies. Here we highlight three broad thematic conclusions and make one plea in the name of subsequent studies.
First, in contrast to mission-oriented federal research agencies, NSF is granted a broad mission to promote the progress of science; advance the national health, prosperity and welfare; and secure the national defense. Accordingly, NSF funds a broad research portfolio across a wide range of scientific fields, including foundational research. However, simply funding research does not guarantee translation into commercial products and services. To address the challenge of how to leverage the nation’s broad research effort in the process of innovation, NSF introduced the SBIR program in 1977. At its core, the SBIR and STTR programs focus specifically on offering financial support for small firms seeking to commercialize innovative products and services to enhance U.S. prosperity, competitiveness, and social progress. Both by statute and as reflected in its operation, the SBIR program is a “small business innovation research” program. Consistent with earlier studies and research documenting successful commercialization outcomes from the NSF SBIR/STTR programs, this report documents that these programs continue to be associated with funding firms who are associated with innovation in the broad national interest, and specifically that recipients attract follow-on private capital that enables the commercialization of new technology. Indeed, it is this continued record of impact that has led the NSF SBIR/STTR programs to serve as a model, both within the federal government, which expanded the SBIR program to other large federal research agencies, and around the world for countries and regions seeking to enhance the process by which research is ultimately translated into innovation.
Second, perhaps reflecting its pioneering legacy and position within the United States national innovation system, the NSF SBIR/STTR programs have been associated with both a high level of experimentation and adaptation, as well as features that have allowed the programs to operate successfully over an extended period of time. For example, the NSF SBIR/STTR programs are centralized in a single office (now included within the newly created Directorate on Technology, Innovation and Partnerships [TIP]), which has allowed the programs to build up significant expertise and capability in meeting the programs’ mission, as well as reducing the redundancies and administrative burdens that might be associated with a less focused administrative structure. Moreover, this centralized structure has focused on distinctive practices of hiring (relative to other NSF program officers), with an emphasis on staff with experience and interest in accelerating innovation in young and small businesses. As well, this organizational structure has allowed the programs to be more proactive in undertaking experiments and being responsive and adaptive to the changing needs of the U.S. innovation system. For example, over the past decade, the SBIR/STTR programs have undertaken proactive steps to provide early-stage feedback to prospective applicants, and worked to enable the growth of mentoring and education programs such as NSF I-Corps.
The effectiveness of the American innovation system depends on a productive interplay among publicly funded research and the critical role played by the private sector (including both large and small and both young and old companies). Within that system, the NSF SBIR/STTR programs provide a critical and dedicated channel through which small and young firms are able to contribute in a meaningful and sustained way to research and innovation.
Over the past decade the NSF SBIR/STTR programs have focused on prioritizing applications by firms (usually younger firms) without prior experience in the programs. In other words, while the formal scope of the programs is the funding of “small” firms, the NSF programs focus on ensuring that young innovative firms can get early-stage capital that can accelerate innovation and follow-on funding and commercialization. While this focus seems like a good match for NSF, we believe that its success at NSF does not necessarily imply that such a focus should be mandated (or even encouraged) for other federal research agencies. Many of the mission-oriented research agencies leverage the SBIR/STTR programs to directly serve their agency missions, and it may be the case that firms with experience in those domain areas may be the most appropriate recipients of funding in those narrower domains. Indeed, this report documents that many firms that first receive funding from the NSF SBIR/STTR programs subsequently receive follow-on support from other federal research agencies.
Finally, with these ongoing evaluations of the SBIR/STTR programs and the launch of the new TIP Directorate, there is a need for a framework and a system that tracks the progress of these investments. Our study has demonstrated the use of publicly available data to track firm development. It is also important to recognize that research is conducted by individual principal investigators working in firms. Tracking these individuals over time provides greater understanding of knowledge spillovers and the circuitous path that are important for the translation of research into innovation.
ACKNOWLEDGMENTS
We would like to express our appreciation for insights, information, experiences, and perspectives provided by invited speakers during the conduct of this study. We recognize the considerable efforts made by NSF, especially Ben Schrag, who was responsive and generous with his time in answering the committee’s questions. The committee also wants to thank Evan Johnson, principal consultant, for invaluable contributions of research and technical assistance in the preparation of this report. Finally, we would particularly like to recognize the leadership of Gail Cohen and the contributions of the National Academies staff, especially Meghan Ange-Stark, David Dierksheide, and Sophie Billinge.
Maryann P. Feldman, Co-Chair
Scott Stern, Co-Chair
Committee on the Review of the Small Business Innovation Research and Small Business Technology Transfer Programs at the National Science Foundation
