8

Findings and Recommendations

The findings and recommendations in this chapter address the Congressional objectives for the Small Business Innovation Research (SBIR) program, as reiterated in the recent program reauthorization and in the subsequent Small Business Administration (SBA) Policy Directive that guides program implementation at all agencies. Section 1c of the Directive lists the program goals as follows:

This chapter reviews the extent to which each of these program goals is being addressed by the National Aeronautics and Space Administration (NASA). We also address some aspects of program management. However,

__________________

¹ Small Business Administration, Small Business Innovation Research (SBIR) Program Policy Directive, February 24, 2014.

prior to this analysis, we describe how the limited outcomes data available at NASA limited the scope of the assessment.

REVIEWING THE EVIDENCE

In assessing the NASA SBIR program, we found that quantitative data on the program outcomes was limited. Outcomes data for agency SBIR programs come from the funded companies and the funding agency. While there are major challenges in gathering and evaluating this data—described in Chapter 5 (Quantitative Outcomes)—most SBIR agencies have made significant efforts to acquire the quantitative data that permits evaluation and subsequently, in the ideal, more effective management of the program. The other four agencies studied by the Academies² in recent years—Department of Defense (DoD), National Institutes of Health (NIH), National Science Foundation (NSF), and Department of Energy (DoE)—have made efforts, with varying degree and success, to acquire relevant outcomes data and to use those data to develop internal assessments of the program. Until recently, this was not the case at NASA.

There are three paths for collection of company data: (1) utilization of a reporting system such as the Company Commercialization Record (CCR) database at DoD, where companies are required to report on outcomes from previous projects; (2) surveys of past awardees for the same purpose; and (3) contracts analysis which for agencies that acquire the end product of SBIR awards can indicate which technologies were further utilized within the agency.

The DoD’s CCR database requires all companies to update outcomes for all previous SBIR awards (at DoD and elsewhere) every time they seek new funding. NSF and DoE track outcomes; NIH has done so sporadically. For both types of data acquisition, the goal is to determine, on a systematic basis, what happened after SBIR funding was provided. DoD (including in particular some departments such as Navy) has also made a substantial effort to identify SBIR Phase III contracts within DoD by analyzing data from the Federal Data Procurement System (FPDS) database.

Since 2012, NASA has started to collect data via an agency database where companies are encouraged but not required to update their information; NASA staff tell us that there are no program-related sanctions imposed on those failing to provide this information.

The new NASA tracking system appears to be well-designed to capture important elements of the use of SBIR technologies within NASA.³ This is potentially valuable as these elements are not as effectively captured by the DoD

__________________

² Effective July 1, 2015, the institution is called the National Academies of Sciences, Engineering, and Medicine. References in this report to the National Research Council or NRC are used in an historic context identifying programs prior to July 1.

³ See discussion of the NASA SBIR Electronic Handbook in Chapter 3 (Initiatives).

CCR. It is, however, still too early to determine whether the data collected will be sufficiently comprehensive to permit effective evaluation and analysis.

Qualitative information, including “success stories,” supplements quantitative data. NASA does collect case studies of success stories, but most are company-provided with limited agency corroboration. It is therefore difficult to determine which cases reflect important accomplishments from the agency point of view.

Sources of Findings

The committee’s findings are based on a complement of quantitative and qualitative tools including a survey, case studies of award recipients, agency data, public workshops, and agency meetings. The methodology is described in Chapter 1 and Appendix A of this report. In reviewing the findings below, it is important to note that the Academies’ 2011 Survey—hereafter referred to as the 2011 Survey—was sent to every principal investigator (PI) who won a Phase II award from NASA, FY1998-2007 (not the registered company points of contact [POC] for each company.⁴ Each PI was asked to complete a maximum of two questionnaires, which as a result excludes some awards from the survey. The preliminary population was developed by taking the original set of SBIR Phase II awards made by NASA during the study period and eliminating on a random basis awards to PIs who received more than two awards (to limit the burden on respondents). The resulting preliminary population was 1,131 awards. PIs for 641 of these awards were determined to be not contactable at the SBIR company listed in the NASA awards database. The remaining 490 awards constitute the effective population for this study. From the effective population, we received 179 responses. As a result, the response rate in relation to the preliminary population was 15.8 percent and in relation to the effective population response was 36.5 percent.

The absence of usable quantitative outcomes data from NASA limits the conclusions that can be drawn from this assessment. Although the 2011 Survey provides quantitative data on NASA outcomes agency-wide, the number of responses is too limited to permit definitive conclusions.⁵ Similarly, although the limited data provided by NASA and that provided by DoD on NASA projects recorded in the DoD Company Commercialization Record database are helpful, neither is comprehensive.

Given the size of the survey population and response rates and overall potential sources of survey bias, the following findings and recommendations rely more heavily on company case studies, discussions with agency staff, and other documentation than we would have preferred. The committee’s findings are accordingly qualified.

__________________

⁴ Because there is a time lag in commercialization for new technologies, the survey did not include more recent awards than 2007. See Box A-1 for a discussion of this commercialization lag.

⁵ See Appendix A for details on survey methodology.

STUDY FINDINGS

Although more and better data would improve the grounding for these findings, it is our judgment that the NASA SBIR program is encouraging the expansion of technical knowledge. And although the limited data available from the 2011 Survey indicates limited infusion of SBIR technologies into NASA Mission Directorates for awards made in FY1998-2007, the program has since then become increasingly aligned with NASA Mission Directorate needs. NASA SBIR projects commercialize at a level similar to that of comparable SBIR programs at DoD, although the small size of the NASA market limits the scale of commercialization. However, with regard to the third program objective, we conclude that the NASA SBIR program is not adequately fostering and encouraging participation by women and minorities and socially and economically disadvantaged small businesses.

It is our view that SBIR works best when the agency’s leadership recognizes the strategic potential of the SBIR program and leverages it to help realize NASA’s mission needs. Although in large measure the results of the NASA SBIR program appear to be positive, NASA has not developed a coherent place for SBIR as a valued part of its strategic plans for addressing its mission. It is telling that there is no section on SBIR in the 2014 NASA Strategic Plan, and only a handful of mentions in passing.⁶ While other agencies have in recent years come to see that SBIR can, if utilized well, provide substantial value to the agency, it is hard to escape the conclusion that NASA has not fully embraced the possibility of treating SBIR as an opportunity (rather than a tax on its extramural research budget).

Moreover, we believe that the NASA SBIR program could be reconfigured to address a series of further opportunities: These include pilot initiatives to harness fast moving innovative small companies, to support the commercialization of space (e.g., the commercial use of space satellite navigation systems), and to integrate better with the Department of Defense, the closest analogous agency that also has mission needs in Space. Seizing these opportunities requires that NASA generate data and metrics that can guide and enhance the SBIR program’s performance.

The findings are organized in terms of the four legislative objectives of the SBIR program plus findings on the management of the program. The summary below provides a guide to the more detailed description to follow.

Summary of Findings

1. Commercialization
   1. NASA SBIR projects commercialize at a substantial rate.

__________________

⁶ NASA, Strategic Plan 2014. Accessed on August 14, 2015, at https://www.nasa.gov/sites/default/files/files/FY2014_NASA_SP_508c.pdf.

2. A plurality of NASA SBIR projects has achieved some private sector commercial success.
3. Subsequent investment provides further evidence that NASA SBIR projects generate potential commercial value even if they have not yet reached the market.
4. NASA SBIR projects are associated with modest job growth.
5. NASA SBIR funding makes a substantial difference in determining project initiation, scope, and timing.
6. Some NASA SBIR companies report significant commercial outcomes.

2. Meeting the Mission Needs of the Agency
   1. The lack of comprehensive quantitative data concerning the agency uptake of SBIR-funded technologies prevented effective determination of the program’s impact within NASA.
   2. Responses to the 2011 Survey shows limited uptake of SBIR projects within NASA.
   3. There is qualitative evidence on uptake of SBIR-funded technologies within NASA.
   4. Recent changes in program management, which has increased the alignment between SBIR and the Mission Directorates, may increase the uptake of SBIR technologies within NASA for awards made after FY2007.
3. Fostering the Participation of Women and Other Under-represented Groups in the NASA SBIR Program
   1. The levels of participation by minority-owned and woman-owned firms in the NASA SBIR program are low and in some areas falling.
   2. NASA has not made sustained efforts to “foster and encourage” the participation of WOSBs and MOSBs.
   3. NASA does not report on or sufficiently track participation by WOSBs and MOSBs.
   4. NASA has not engaged sufficiently with the challenge of encouraging women and minority participation in SBIR.
4. Enhancing Science and Technology
   1. The SBIR program at NASA supports the development and adoption of technological innovations. However there is growing misalignment between the enhancement of science and technology and the demands of meeting specific agency mission needs.
   2. The NASA SBIR program continues to connect companies and universities.

3. NASA SBIR projects generate substantial knowledge-based outputs such as patents and peer reviewed publications.

5. Fostering Innovative Companies
   1. The NASA SBIR program fosters the formation of innovative small companies.
   2. NASA SBIR awards lower the risks of innovation and helps small businesses enter new markets.
   3. The NASA SBIR program has supported the development of small innovative companies in the United States.
   4. The NASA SBIR program limits small company dependence on government grants.
6. Program Management
   1. NASA’s SBIR program is not sufficiently driven by metrics.
   2. Many of NASA’s commercialization initiatives are potentially promising but are too recent to provide an outcome assessment.
   3. NASA’s monitoring and evaluation of the SBIR program is insufficient.
   4. Some NASA program management practices do not reflect best practices.

I. Commercialization

Each agency has its own priorities for the SBIR program. At NASA, the overwhelming emphasis has been on the adoption of SBIR-funded technologies for use within the agency in support of its mission. This mission support overlaps to a substantial degree with the commercialization objective of the SBIR program: projects adopted for use within NASA are also provided with downstream NASA or other federal contracts and are therefore, in our view, successfully commercialized.

However, the focus on agency mission has important implications for the extent of commercialization. At NIH and NSF, commercial success is achieved in most cases outside the agency and hence is measured in terms of standard economic outputs such as sales, revenues, and company growth. At DoD, the acquisitions market is large enough that companies can become successful by serving that market alone: defense contractors can grow to become very large, and contracts in the tens or hundreds of millions of dollars are possible.

At NASA, however, contracts within the agency for SBIR-funded technologies tend to be relatively small. There is rarely a need for thousands of a particular item; it is more common for a technology requirement to be for a particular instrument or a component for a larger system of which, at best, a

small number will be built. For example, the market for the SBIR-funded batteries that are designed to power the Mars Rover is limited. Indeed, NASA’s own newsletter notes the need to find outside markets: “NASA technology needs are more likely to be met if they can be engineered to overlap significantly with commercial or Department of Defense (DoD) needs.”⁷ Of course, even tightly targeted mission-oriented projects can and do produce technologies with sometimes significant non-NASA market potential. Most notably, a NASA SBIR contract to develop technology for autonomous rendezvous and docking of space vehicles to service satellites was later adapted to track and compensate for eye movements in now commonplace laser surgery for vision correction.⁸

In the main, however, it appears that many NASA SBIR companies are affected by the small size of the NASA marketplace and sometimes very long lags as technology matures and large scale programs evolve toward completion. In some ways, they also suffer from the NASA SBIR program’s focus on NASA’s specialized needs.

Such companies would seldom be able to grow rapidly and become substantial commercial entities with hundreds of employees—as has happened to a number of SBIR companies working primarily within DoD. The market within NASA is not large enough to support this kind of development. Discussions with companies and agency staff as well as survey responses indicate that small companies, whose mission is to work on space technologies for NASA, typically remain small in size and hence are unlikely to generate huge commercial outcomes from their projects. (NASA SBIR companies averaged 10 employees at the time of the surveyed award, and 15 at the time of the 2011 Survey). More research is required to determine the scale and impact of this phenomenon.

That said, there remains a distinction between projects that generated sales or further investment and resources, especially from within NASA, and those that did not. In some cases, spinoffs from the technologies led to contracts and successes outside NASA. Some company case studies reveal that the NASA contracts base was not sufficient to support the company’s vision and that they had successfully gone outside NASA into the commercial marketplace or DoD (a number of companies working within NASA also acquire contracts from DoD).

Within this broad context, we make the following findings:

__________________

⁷ NASA, The Concept, vol. 3, no. 2, Spring 2012, p.3.

⁸ This technology has been cited in the White House Tibbitts Awards. See https://www.sbir.gov/sites/default/files/tibbetts_2013_book_print_version.pdf.

1. NASA SBIR projects commercialize at a substantial rate.⁹
   1. Forty-six percent of respondents to the 2011 Survey reported some sales.¹⁰
   2. An additional 26 percent reported that they anticipate future sales.¹¹
   3. However, the scale of commercialization was limited: no projects reported aggregate sales of $20 million or more, and 1 percent of projects reported project-related sales of $10 million to <$20 million.¹²
2. A plurality of NASA SBIR projects has achieved some private-sector commercial success. Among surveyed projects reporting sales—
   1. An average of 35 percent of project sales were to domestic private-sector customers.¹³
   2. An average of 9 percent of project sales were to export customers.¹⁴
3. Subsequent investment provides further evidence that NASA SBIR projects generate potential commercial value even if they have not yet reached the market.
   1. Sixty-five percent of survey respondents reported receiving additional investment funding in the surveyed project.¹⁵
   2. For those projects receiving funding, non-SBIR-STTR federal funding was overwhelmingly the most likely source of additional funding (71 percent of reported funding). No other source provided more than 10 percent of funding. Two percent of reported funding came from U.S. venture capital (VC).¹⁶

__________________

⁹ NASA does not maintain data on commercialization rates across the study period (the 2011 Survey covered awards made in FY 1998-2007 inclusive). The data in this section are drawn from the 2011 Survey, which generated 179 responses from Phase II SBIR awardees. Unless otherwise noted, all percentage responses are therefore percentages of those 179 responses, which is 36.5 percent of the effective population of awards and 15.8 percent of the preliminary population of awards. See Appendix A for a description of the survey methodology.

¹⁰ See Table 5-2.

¹¹ See Table 5-2.

¹² See Table 5-3.

¹³ See Table 5-4.

¹⁴ See Table 5-4.

¹⁵ 2011 Survey, Question 33.

¹⁶ See Table 5-7.

4. NASA SBIR projects are associated with modest job growth.
   1. Respondents reported that the median size of firms with NASA Phase II awards grew from 10 employees at the time of award to 15 employees at the time of survey.¹⁷
   2. SBIR firms with NASA awards are small, substantially smaller than at DoD. Despite large percentage increases over time, the average SBIR firm at NASA starts from a small base.¹⁸
5. NASA SBIR funding makes a substantial difference in determining project initiation, scope, and timing:
   1. Seventy-five percent of respondents reported that their NASA project probably or definitely would not have proceeded without SBIR funding.¹⁹
   2. Among those anticipating that the project would have been initiated in the absence of SBIR funding, about two-thirds reported that the project would have been delayed by at least 1 year, and 67 percent reported that the project would have been narrower in scope.²⁰
6. Some NASA SBIR companies report significant commercial outcomes.
   1. Although company case studies do not provide a basis for quantitative assessment, they do provide examples of companies that have become commercially successful and sustainable based on their work for NASA via the SBIR program.
   2. There is an important strategic difference between companies that are started to address a commercial need (and use SBIR as a way to fund their R&D and product development) versus those that start specifically to pursue an announced SBIR topic. The latter sort of company is typically much less successful in achieving meaningful commercialization while the former is more likely to pursue truly dual-use investigations and, eventually, achieve their commercialization goals.

__________________

¹⁷ See Table 5-5. Although these survey data cover Phase II awards made from FY1998 to FY 2007 and firm growth can vary according to length of time since award was made, data from firms with older awards may also be biased toward surviving firms.

¹⁸ See Table 5-5. N=170 respondents. See also Tables 3-4 and 3-5 in National Research Council, SBIR at the Department of Defense, Washington, DC: The National Academies Press, 2014.

¹⁹ See Table 5-1.

²⁰ As this question was asked only of the small number of companies who would have proceeded even absent funding, these numbers should be treated with caution; they are indicative only. N=12. See 2011 Survey, Questions 25 and 26.

II. Meeting the Mission Needs of the Agency

NASA’s current mission is to “Drive advances in science, technology, aeronautics, and space exploration to enhance knowledge, education, innovation, economic vitality, and stewardship of Earth.”²¹

In general, the NASA SBIR program has focused on developing technologies that can meet the agency’s own mission needs. Discussions with agency staff indicate that the agency’s primary metric for program success is the deployment of SBIR-funded technologies on NASA missions.

Case studies and NASA success stories show that SBIR has provided important support for some missions. The question is whether this is happening at a sufficient rate. NASA has not provided comprehensive quantitative data against which to measure success on this core metric. NASA has not effectively tracked Phase III contracts stemming from SBIR awards or systematically tracked the utilization of SBIR-funded technologies at NASA Field Centers or on NASA missions. However, the new tracking database does lay the groundwork for such tracking in the future, and this is a positive step that has the potential to aid future assessments of the program.

1. The lack of comprehensive quantitative data concerning the agency uptake of SBIR-funded technologies prevented effective determination of the program’s impact within NASA.
   1. NASA was unable to provide comprehensive data on follow-on contracts after Phase II. The new data collection mechanism may provide better data in the future.
   2. The absence of comprehensive data on NASA Phase III contracts limits any conclusions on NASA uptake of SBIR technologies.
   3. NASA has not developed any alternative mechanisms for measuring impact (e.g., surveys either of companies or of NASA acquisitions staff).
2. Responses to the 2011 Survey shows limited uptake of SBIR projects within NASA.²²
   1. Among surveyed projects with some sales, an average of 17 percent of reported sales were to NASA. An average of 14 percent of reported project sales were to DoD. An average of 2 percent were to NASA prime contractors, although as there is considerable overlap between NASA primes and DOD primes, not all of these projects are likely to

__________________

²¹ NASA Strategic Plan FY2014, p. 2.

²² See Table 5-4.

have been for NASA. This means that an average of about 20 percent of reported project sales were to NASA or NASA primes.²³

2. More positively, of the two thirds of respondents that reported further investment in the project beyond Phase II, 71 percent of reported funding was non-SBIR federal funding.²⁴
3. Firms working for both NASA and DoD can create sustainable markets for their products.²⁵ However, NASA does not have in place systematic efforts to connect with DoD projects or programs, for example by taking advantage of opportunities to publicize NASA projects to DoD services or for NASA to acquire DoD SBIR-funded technologies.

3. There is qualitative evidence on uptake of SBIR-funded technologies within NASA.
   1. Company case studies provide examples of technologies that were used on NASA missions or that made important contributions to NASA operations.²⁶ NASA has also published a regular newsletter covering success stories for its SBIR program. A review of “success stories” collected by NASA also shows that in a number of cases SBIR-funded technologies did provide important technologies to NASA and were integral to NASA missions.
   2. Evidence from SBIR-funded companies indicates that some important technologies were developed with the help of SBIR funding and that many of these technologies would not have been developed without NASA SBIR funding because funding sources were not available.²⁷
   3. Discussions with NASA staff indicate that in some cases SBIR filled significant gaps in NASA technology plans, often using less money and taking less time than traditional contracts through the NASA prime contractors.
4. Recent changes in program management, which has increased the alignment between SBIR and the Mission Directorates, have the potential to increase the uptake of SBIR technologies within NASA for awards made after FY2007 (the last year covered by the 2011 Survey).
   1. Mission Directorate staff are now deeply involved in the development of SBIR topics

__________________

²³ See Table 5-4.

²⁴ See Table 5-7. N=116 respondents.

²⁵ See Table 5-4.

²⁶ See also the discussion in Chapter 7 (Insights).

²⁷ For example, see the Honeybee case study in Appendix E (Case Studies).

2. Mission Directorate staff also now have a much greater role in award selection.

As noted under Finding IV, this may have a negative effect on NASA support of technologies that do not have potential for infusion into NASA programs in the short term.

III. Fostering the Participation of Women and Other Under-represented Groups in the NASA SBIR Program

NASA has not effectively addressed the mandate to foster the participation of women and other under-served populations. Current outcomes and activities by NASA are not sufficient to meet the SBIR program objective of fostering and encouraging the participation by minority and disadvantaged persons in technological innovation.

1. The levels of participation by minority-owned and woman-owned firms in the NASA SBIR program are low and in some areas falling.
   1. Data from NASA indicate that approximately 8 percent of Phase I awards in FY2014 went to Woman-owned Small Businesses (WOSBs).²⁸ Approximately an equal share went to Minority-owned Small Businesses (MOSBs).
   2. The percentage share of Phase I awards to MOSB declined steadily during the study period, while the percentage share of Phase I awards to WOSBs were largely flat, excluding FY2014.²⁹
   3. Phase I success rates (awards/applications) for MOSBs were lower than those for non-MOSBs every year since FY2005. In FY2014 the gap was more than 20 percentage points.³⁰
   4. Phase I success rates for WOSB were lower than those for non-WOSBs in all the years studied. The gap was largest in FY2014, about 20 percentage points.³¹
   5. MOSB shares of Phase II awards fell substantially: MOSB firms received 19 Phase II awards in FY2009 and 5 in FY2014.³² Their share declined by over half after FY2006.³³
   6. The WOSB share of Phase II awards declined to below 8 percent in FY2011 and FY2012.
   7. MOSB Phase II success rates in every year FY2005-2012 were lower than those for non-MOSB firms Overall, MOSB success rates were 13 percentage points lower than those for non-MOSBs.³⁴

__________________

²⁸ See Figure 6-7.

²⁹ See Figure 6-7.

³⁰ See Figure 6-9.

³¹ See Figure 6-5.

³² See Figure 6-18.

³³ See Figure 6-18.

8. Phase II success rates for WOSBs were lower than those for non-WOSBs in every year of the study period (FY2005-2012) except for FY2005.³⁵
9. WOSB shares are in part supported by the presence of three especially successful WOSB companies which accounted for nearly one-quarter of all WOSB Phase I awards during this period. There were no comparable MOSBs.³⁶

2. NASA has not made sustained efforts to “foster and encourage” the participation of WOSBs and MOSBs.
   1. NASA did not provide evidence of implementing any substantial program for outreach to these communities, and there is little evidence of any sustained activity focused on this objective. The NASA SBIR program participates in one workshop annually, organized by another component within NASA. No efforts have been made by the NASA SBIR Program to reach out to WOSBs.
   2. There appears to be some evidence of outreach activity at NASA Field Centers (notably Johnson Space Center), but the NASA SBIR Office provided no data on these activities or their outcomes.
3. NASA does not report on or sufficiently track participation by WOSBs and MOSBs.
   1. NASA does not sufficiently track participation by MOSBs. NASA’s annual report to SBA provides some data on participation but provides very limited information on efforts to foster and encourage participation, especially at the different Field Centers.
   2. NASA provided no separate data on Black-owned and Hispanic-owned small businesses or on minority or female principal investigators (PIs). Responses to the 2011 Survey indicate that Black-owned and Hispanic-owned small businesses are themselves a very small share of MOSBs overall.³⁷
   3. NASA did not share any data it may have on woman and minority PIs in their program. The 2011 Survey reveals that about 11 percent of Phase II respondents reported a minority PI and 5 percent a female PI.³⁸

__________________

³⁴Figure 6-17.

³⁵ See Figure 6-12.

³⁶Tables 6-6, 6-7, 6-8, and 6-9.

³⁷ See 2011 Survey, Question 19, and section on “Minority Company Ownership” in Chapter 6 (Participation of Women and Minorities).

³⁸Tables 6-1 and 6-4 (N=177 respondents).

Further analysis of the Survey data indicates that there were two Black PIs in the overall responding population, and four Hispanic PIs.³⁹

4. NASA has not engaged sufficiently with the challenge of encouraging women and minority participation in SBIR.
   1. NASA did not provide evidence that it has reviewed the role of women and minorities within the SBIR program.
   2. There has been no concerted effort to determine what could be done—within budget constraints—to improve participation and therefore both meet Congressional objectives for the program and expand the pool of qualified applicants and capabilities.

IV. Enhancing Science and Technology

NASA undertakes many scientific missions, its outreach to the public focuses on scientific accomplishments, and it retains a strong educational mission (indeed, for a time the SBIR program was located within NASA’s education directorate.) Evidence suggests that the NASA SBIR program is providing support for the development of new technologies related to NASA’s missions and enhancing science and technology more broadly, as is summarized in the following items A-C.

1. The SBIR program at NASA supports the development and adoption of technological innovations.⁴⁰ However there is growing misalignment between the enhancement of science and technology and the demands of meeting specific agency mission needs.⁴¹
   1. Selection of topics and individual projects for funding maintains a strong focus on technological innovation.
   2. Topic selection is closely aligned with the technical needs of NASA Mission Directorates, which have effective veto power over topic and sub-topic statements.
   3. Some SBIR companies indicated that the NASA SBIR program has shifted away from scientific inquiry toward more tightly defined contract research for the agency. In their view, this potentially reduces opportunities for breakthrough innovation.⁴²
   4. Review scoring for individual proposals is now increasingly weighted toward meeting agency mission needs. Peer review is essentially

__________________

³⁹ See Table 6-3 and section on “Minority Principal Investigators” in Chapter 6 (Participation of Women and Minorities).

⁴⁰ See Chapter 2 (Program Management).

⁴¹ See Appendix E (Case Studies).

⁴² See Chapter 5 (Quantitative Outcomes).

advisory, while final decisions are made by Mission Directorates based on their needs and priorities.⁴³

2. The NASA SBIR program continues to connect companies and universities.
   1. NASA SBIR program continues to connect companies and academic institutions in a variety of ways. 2011 Survey data indicate that NASA SBIR projects continue to utilize universities (in addition to the even closer connection through the Small Business Technology Transfer (STTR) program, which was not included in the survey data).
   2. Just over 30 percent of respondents reported a link to a university. About 21 percent of respondents reported that a research institution was a subcontractor; about 15 percent, reported that university faculty worked on the project (not as PI); and 14 percent reported employing graduate students.⁴⁴
   3. Survey respondents identified 75 different universities as project partners; 21 were mentioned by two or more respondents. Universities with the most mentions were cited by four respondents.⁴⁵
   4. More than 60 percent of responding companies reported at least one academic founder, and about 30 percent reported that the most recent prior employment of a founder was at a university.⁴⁶
3. NASA SBIR projects generate substantial knowledge-based outputs such as patents and peer reviewed publications. These are widely recognized metrics for the creation of technical knowledge. Based primarily on data from the 2011 Survey:
   1. Patenting remains an important component of knowledge diffusion (and protection).
      • Forty-five percent of respondents reported receiving at least one patent related to the surveyed project.⁴⁷

__________________

⁴³ Discussions with NASA SBIR managers. See section on “Selection” in Chapter 2 (Program Management).

⁴⁴ See Table 5-13; N=177. These figures are similar to those reported for DoD. See National Research Council, SBIR at the Department of Defense, Washington DC: The National Academies Press, 2014, Chapter 3, University connections.

⁴⁵ See Table 5-14 and Appendix D (List of Universities). These numbers are close to those reported for the DoD SBIR program. See National Research Council, SBIR at the Department of Defense, 2014 op. cit.

⁴⁶ See Tables 5-15 and 5-16.

⁴⁷ See Table 5-11.

• Fourteen percent of responding companies reported receiving 10 or more patents for all SBIR funded technologies.⁴⁸

2. Publication of peer-reviewed articles remains the primary currency of scientific discourse, and despite the need to protect ideas in the commercial environment of small businesses, NASA SBIR firms continue to participate deeply in scientific publication.
   • More than 80 percent of respondents reported at least one resulting peer-reviewed publication related to the surveyed project.⁴⁹ This is much higher than the 40 percent figure found in the 2005 Survey.
   • Thirty-one percent of respondents reported more than three publications resulting from the surveyed project.⁵⁰
   • Some of the case studies indicate that companies take pride in the number of peer-reviewed publications developed by their scientists and engineers, both within and outside of the NASA SBIR program. Papers are often prominently posted on company websites.⁵¹

V. Fostering Innovative Companies

The case studies and 2011 Survey show that the NASA SBIR program supports for the foundation and growth of innovative companies.

1. The NASA SBIR program fosters the formation of innovative small companies.
   1. Forty percent of respondents said that the company was founded entirely or in part because of the SBIR program.⁵²
2. NASA SBIR awards lower the risks of innovation and helps small businesses enter new markets.
   1. The NASA SBIR program provides important seed funding.⁵³

__________________

⁴⁸ See Table 5-10 (N=64 responding companies).

⁴⁹ See Table 5-12, (N=129 respondents).

⁵⁰ See Table 5-12.

⁵¹ See Appendix E (Case Studies).

⁵² See section on “Quantitative Survey Evidence that NASA Stimulated Technological Innovation” in Chapter 5 (Quantitative Outcomes) and 2011 Survey Question 6 (N=73 companies). Survey question refers to SBIR program overall, not necessarily just NASA.

⁵³ See section on “Quantitative Survey Evidence that NASA Stimulated Technological Innovation” in Chapter 5 (Quantitative Outcomes), sections on “Company Formation and Very Early-Stage Funding” and “Funding Otherwise Unfundable Projects” in Chapter 7 (Insights), and Appendix E (Case Studies).

• Open-ended responses to the 2011 Survey, as well as a number of case studies show that the program provided funding at a stage when the project was too risky for other investors.
• NASA SBIR funding supports technology development, which can be supported through additional commercial and government funding further downstream.

2. NASA funding can support company efforts to enter new markets.⁵⁴
   • In some cases, companies use SBIR funding to build on existing platform technologies specifically to enter new markets. This platform-driven approach is used by a number of the companies profiled in the case studies.⁵⁵
   • Innovative companies must often make mid-course corrections. According to respondents, NASA funding has helped a number of surveyed companies successfully make what are often difficult changes that are hard to fund.

2. The NASA SBIR program has supported the development of small innovative companies in the United States.
   1. The 2011 Survey provided a sample of SBIR companies with the opportunity to report the overall impact on the company of the awards about which they were surveyed and to identify specific kinds of impacts.
      • Twenty-five percent of responding companies indicated that the funding had a “transformative” effect on their company. Another 56 percent said that it had a “substantial positive long term effect.”⁵⁶
      • Textual responses revealed a wide range of impacts, summarized in Box 8-1.
3. The NASA SBIR program limits small company dependence on government grants.⁵⁷
   1. The NASA program does not provide large numbers of awards to individual companies.

__________________

⁵⁴ See Chapter 7 (Insights).

⁵⁵ See Appendix E (Case Studies).

⁵⁶ See Table 5-20.

⁵⁷ The evidence here echoes previous studies by the Academies, which discussed the question of multiple SBIR awards to companies. The 2009 report concluded that multiple awards to companies was not a significant problem. See National Research Council, An Assessment of the Small Business Innovation Research Program, The National Academies Press: Washington DC, 2008.

• Firms may apply for only 10 awards per year.
• The most successful firm received 27 SBIR Phase II awards during the period FY2005-2012.⁵⁸
• Many NASA program participants are not dependent on SBIR awards.
• More than one-quarter of responding companies received no SBIR funding in the most recent fiscal year (at the time of the survey).⁵⁹
• Less than one-quarter of respondents reported that SBIR accounted for more than 50 percent of company revenues for the most recent fiscal year (at the time of the survey).⁶⁰

2. Small innovative companies rarely develop linearly, directly from idea to R&D to commercialization/development to sales with a single SBIR award. Notably—
   • More than three quarters of Phase II survey respondents reported at least one prior SBIR or STTR Phase I award related to the surveyed project.⁶¹

__________________

⁵⁸ See Table 4-6.

⁵⁹ See Table 5-18 (N=72 responding companies). Question 9 of the 2011 Survey asked “What percentage of the company’s revenues during its most recent completed fiscal year was Federal SBIR funding (Phase I and/or Phase II)?”

⁶⁰ See Table 5-18 (N=72 responding companies).

⁶¹ See Table 5-19 (N=154 respondents).

• Thirty-nine percent reported at least two additional related awards.

VI. Program Management

1. NASA’s SBIR program is not sufficiently driven by metrics.
   1. NASA lacks sufficient evidence on the operations of its SBIR program: In general, there is insufficient evidence to support definitive conclusions about operation of the NASA SBIR program; this insufficiency of evidence is in itself troubling.
      • NASA has not provided data that could show the extent to which the program is systematically meeting its objectives, and how this has changed over time.⁶²
      • NASA does not have interim milestones to suggest that this is the case.
   2. NASA’s Technology Infusion Strategy is potentially promising, but needs effective metrics.
      • NASA’s use of Technology Infusion Managers (TIMs) at every Field Center is a promising effort to link technologies developed through the SBIR program with the needs of NASA program managers.⁶³
      • TIMs operate quite independently and as a result use different tools and approaches. TIMs are also focused primarily on the NASA programs that run at their Field Center. This can limit their effectiveness as agents of commercialization and technology transfer.⁶⁴
      • However, NASA does not have metrics and reviews in place to determine best practices among TIMs. Hence we have no evidence on which to determine whether the program is fully effective.⁶⁵
   3. The limits placed by NASA on the number of SBIR submissions per company are effective: Limits on submissions per company are an appropriate mechanism for ensuring that the number of submissions is limited while at the same time not excluding any specific project. The

__________________

⁶² See Chapter 5 (Quantitative Outcomes).

⁶³ See Chapter 2: (Program Management).

⁶⁴ See Chapter 2: (Program Management).

⁶⁵ See Chapter 2: (Program Management).

limit of 10 submissions per solicitation does provide some practical limitation on a small number of companies.⁶⁶

4. SBIR-funded research is aligning with the specific needs of the Mission Directorates: The Mission Directorates are playing a growing role within the NASA SBIR program because they are directing the technical aspects of the program more closely. This reflects pressure to align funded research with the specific needs of the Mission Directorates.⁶⁷
5. NASA’s once a year SBIR solicitation is not adequate:
   • The NASA SBIR program continues to offer one solicitation annually. As technology moves more rapidly, this is increasingly problematic, and other major agencies have recently increased the number of solicitations per year.
   • However the long cycles of NASA programs and space technology more generally means that an annual solicitation cycle will be justified after further NASA review.
6. The Phase I to Phase II transition hurts firms and does not serve NASA:
   • According to NASA’s FY 2012 annual report to the SBA, the time lag per project between the end of Phase I funding and the beginning of Phase II funding averaged 233 days.⁶⁸
   • This lag is potentially disastrous for small firms that do not have other suitable projects to fund staff during the gap period.⁶⁹
   • This lag also raises the risk of loss of effort and the inability of the firm to help meet NASA’s mission requirements in a timely and cost effective manner.
7. NASA currently makes it difficult for SBIR companies to connect directly to technical contacts at NASA. Like DoD, NASA enforces communications restrictions based on Federal Acquisition Regulations during the solicitation period. Unlike DoD, it does not offer a pre-solicitation window through which such connections can be made.⁷⁰ This absence is counterproductive, especially as the NASA charter allows the agency to share to the maximum extent practicable the results of its R&D.

__________________

⁶⁶ See Chapter 4 (SBIR Awards).

⁶⁷ See Chapter 2 (Program Management).

⁶⁸ See section on “Funding Gaps” in Chapter 2 (Program Management).

⁶⁹ See section on “Funding Gaps” in Chapter 2 (Program Management).

⁷⁰ See section on “Access to Program Staff During Solicitation Period” in Chapter 2 (Program Management).

2. Many of NASA’s commercialization initiatives are potentially promising but are too recent to provide an outcome assessment.
   1. NASA has initiated a number of programs to support commercialization of NASA Phase II projects.⁷¹
      • The Phase II-E program provides additional bridge funding for selected Phase II projects. However, it is not clear—and NASA has not determined—whether this modest additional funding is sufficient to affect outcomes.
      • The Phase II-X program is more ambitious and can help in de-risking investments for NASA programs, thus encouraging more uptakes within NASA.
      • Select Topics: The even more recent introduction of Select Topics and the proposed effort to introduce a Commercialization Readiness Program are likewise too recent to permit substantive analysis, although the Select Topics initiative seems to be an appropriate effort to add funding to high-priority areas.
   2. NASBO: The NASA Alliance for Small Business Opportunity (NASBO) incubator initiative, discussed in the 2009 National Research Council report on the NASA SBIR program, has ended with little to show in terms of outputs.⁷²
   3. TecFusion^TM: NASA supported development of a TecFusion^TM program to link large and small businesses, although this program was not focused exclusively on SBIR. The program has changed in recent years and now focuses primarily on providing NASA technology managers with improved links to small companies. It is a small program—five managers have received services according to NASA’s contractor.⁷³
   4. Agency Support: NASA does not provide any agency-level support for companies that are focused on commercialization of SBIR projects. It has not in the past funded efforts that involve third-party commercialization support companies. The NASA SBIR program office relies primarily on the agency’s Technology Transfer Program to help NASA SBIR companies find opportunities outside NASA, although this program is not limited to SBIR. NASA does not have data

__________________

⁷¹ See Chapter 3 (Initiatives).

⁷² National Research Council, An Assessment of the SBIR Program at the National Aeronautics and Space Administration, Washington, DC: The National Academies Press, 2009, Chapter 5.

⁷³ See section on “Technology Infusion Managers” in Chapter 2 (Program Management).

available about its effectiveness either in general or in particular with reference to SBIR.⁷⁴

3. NASA’s monitoring and evaluation of the SBIR program is insufficient.

   We have noted in Chapter 1 and in Appendix A that there are broad challenges in tracking commercialization, at both the company and project levels. Companies move in and out of the program, and tracking is harder once they have left. More generally, commercialization can come many years after an award and involve multiple awards plus considerable additional funding. All this makes it difficult to assert that any specific outcome “results from” an SBIR award. But there are also specific challenges with existing tracking tools.

   1. While NASA has initiated a tracking system focused on commercialization starting in 2012, participation is key.⁷⁵
      • The basic data template for NASA’s tracking system is good and includes, in particular, detailed information on technology uptake within NASA.
      • The primary challenge in tracking commercialization and agency uptake of SBIR technologies through the new tool will be to ensure that company participation reaches sufficient levels to generate the comprehensive analysis needed for effective use as a management tool.
   2. NASA does not track other important program outcomes. NASA does not currently have in place data collection and tracking systems that address all congressional objectives. Tracking for knowledge effects is essentially absent, and there are significant weaknesses in tracking of participation by women and minorities and uptake of SBIR technologies within the agency.⁷⁶
   3. NASA has not provided metrics against which program improvement could be measured.⁷⁷ Outcomes potentially provide important signals for program management about the effectiveness of different kinds of topics, different staff, different outreach strategies, etc. These signals are not extensively used for SBIR program management.
   4. NASA has not provided any documentation that suggests that the Program Office uses data collected via the electronic handbook to

__________________

⁷⁴ See section on “Technology Infusion Managers” in Chapter 2 (Program Management).

⁷⁵ See section on “Electronic Handbook (EHB)” in Chapter 3 (Initiatives).

⁷⁶ See Chapter 2: (Program Management) and Chapter 3 (Initatives).

⁷⁷ See Chapter 2: (Program Management).

provide strategic management for the program, beyond operational matters related to the processing of applications and awards.

5. NASA has not sufficiently exploited the opportunities presented by the existence of multiple NASA Field Centers to operate and assess pilot projects. For example, NASA has not provided any data about utilization or outcomes from the Mentor Protégé Program.

4. Some NASA program management practices do not reflect best practices.⁷⁸
   1. NASA’s SBIR contracts management is unnecessarily rigid.
      • Discussions with agency staff confirm information from company case studies that NASA contracts management is quite rigid. For example, NASA does not provide no-cost extensions for Phase I, which may prevent the best possible use of NASA research resources and compresses the actual research conducted under Phase I into a tight timeline.
      • Agency staff also confirmed that NASA SBIR contracts pay out at a set dollar value per month for the life of the contract. This means that not only do companies have no incentive to accelerate promising research, but also the payment schedule actively prevents them from doing so.
   2. Funding gaps between Phase I and Phase II have not been effectively addressed.
      • NASA’s FY2012 annual report to SBA found that the average gap between the last day of Phase I funding and the first day of Phase II funding was 233 days. This is well beyond the SBA benchmark of 180 days.
      • Despite the large gap between Phases, no gap funding is available to provide a bridge between Phase I and Phase II, unlike some other SBIR programs.
        • NASA itself has no gap funding program, such as the Phase I options available for many DoD contracts.
        • Firms cannot begin work at their own risk. There is no mechanism for paying companies for work completed during the gap, if and when, a Phase II contract is eventually issued. This is again unlike the practice at some other SBIR agencies.

__________________

⁷⁸ See Chapter 2: (Program Management).

3. NASA’s reporting on the SBIR program remains weak.
   • As noted above, there is no quantitative reporting on program outcomes, and qualitative reporting is limited in quality.
   • The report provided to SBA meets SBA requirements, but essentially provides only a basic justification for the program. NASA’s annual SBIR-STTR reports do not cover new initiatives effectively and do not address all congressional objectives (indeed they do not address outcomes at all).
4. The number of applications for SBIR support has been declining.
   • While all agencies have seen declines in applications in recent years, NASA’s decline has been substantial: about 50 percent decline in Phase I applications across the FY2005-2014 time frame.⁷⁹
   • This is potentially of significance to the program especially if high-quality companies are concluding that the effort to apply is simply not worth the reward.⁸⁰
   • NASA has not provided a hypothesis to explain the declining numbers (e.g. explanations related to application difficulties, narrower topics, declines in the number of space or aeronautics oriented small companies) and does not have in place any strategy for increasing the number of applicants aside from participation in outreach efforts developed by SBA.

RECOMMENDATIONS

SBIR presents NASA with an important opportunity to advance its mission, work with fast moving innovative small companies, support the commercialization of space, and integrate better with related missions of other agencies. The following recommendations, which are organized in terms of four sets of leading actions needed to improve the SBIR program at NASA, can help improve outcomes. A detailed description follows the summary of key points below.

Summary of Recommendations

1. Furthering a Culture of Monitoring, Evaluation, and Assessment Predicated on Enhanced Information Flows

__________________

⁷⁹ See section on “SBIR Phase I: Applications and Awards” in Chapter 4 (Awards).

⁸⁰ See section on “SBIR Phase I: Applications and Awards” in Chapter 4 (Awards).

1. The NASA SBIR program should improve current data collection approaches and methodologies.
2. NASA should use new commercially available tools to gather more current data on its SBIR program.
3. NASA should better use the data it collects on the SBIR program. These data should be utilized to systematically guide program management.
4. NASA should improve its reporting on the SBIR program.

2. Addressing Under-represented Populations
   1. Quotas are not necessary.
   2. NASA should develop new and improved metrics.
   3. NASA should develop outreach and education programs focused on expanding participation of under-represented populations.
   4. NASA should share best practices.
3. Commercialization
   1. NASA should improve support for the commercialization of SBIR technologies.
4. Improving Program Management
   1. NASA should improve the application process.
   2. NASA should adopt more flexible contracting practices to encourage firm participation in the program.

I. Furthering a Culture of Monitoring, Evaluation, and Assessment Predicated on Enhanced Information Flows

The lack of comprehensive and granular outcomes data prevents development of quantitative analysis which would allow the agency to determine the extent to which is it meeting Congressional mandates and could also allow NASA to identify strengths and weaknesses of its SBIR program and to adjust it accordingly.

1. The NASA SBIR program should improve current data collection approaches and methodologies.⁸¹ NASA should make it a top priority to develop and implement appropriate metrics for assessing program outcomes.

__________________

⁸¹ See Finding VI-C.

1. NASA should expand data collection. NASA’s data collection efforts should expand substantially to address the entire range of congressionally mandated outcomes, not only commercialization. In particular, NASA should ensure that data related to Congressional goals for the participation of women and minorities are fully captured and that uptake of SBIR technologies within NASA is fully captured.⁸²
2. NASA should track the adoption of technologies within NASA. NASA should use the new tools embedded within NASA’s electronic program management tool, the Electronic HandBook (EHB), to ensure that the uptake of SBIR technologies within NASA is comprehensively and consistently tracked, and reported in a timely manner. Such tracking must be comprehensive, but care must be given to ensure that it is not burdensome on small businesses.⁸³
3. NASA should adopt available data collection methodologies. This includes tracking of Phase III contracts; a company commercialization/outcomes record similar to the DoD CCR database (tuned to NASA’s needs); and regular surveys of companies.⁸⁴
4. NASA should adopt best practices in program metrics. NASA should review reports by the Academies on other SBIR programs as well as other assessments (e.g. from Government Accountability Office) to develop a set of metrics for program assessment and evaluation. This process should be completed rapidly.
5. NASA should adopt multiple metrics for commercialization. NASA should address commercialization in ways similar to the now widely accepted methodology developed for the Academies’ SBIR studies—that is, utilizing multiple metrics.⁸⁵ These additional metrics will provide a deeper and more nuanced basis for further analysis.
6. NASA should improve its collection of demographic data. Demographics of company ownership should be extended to show the specific SBA-defined socially and economically disadvantaged category for each minority applicant. Applicants should be asked, on a voluntary basis, to complete the same demographic questions about the PI as well as the company’s owners in the course of their application.⁸⁶
7. NASA should more systematically develop qualitative outcomes data. Qualitative data provide important insights, and NASA should develop and adopt a more systematic approach for the use of success stories. Success stories can provide inspiration, lessons learned, and important information not available elsewhere about program impacts.⁸⁷

__________________

⁸² See Finding III-C.

⁸³ See Finding II-A.

⁸⁴ See Finding IV-C.

⁸⁵ See methodology discussions in Chapter 1 (introduction) and Appendix A (Methodology).

⁸⁶ See Finding III-C.

⁸⁷ See Finding IV-D.

2. NASA should use new commercially available tools to gather more current data on its SBIR program.⁸⁸
   1. NASA should develop pathways to provide ongoing feedback from companies about program activities and operations. These could include various electronic communication tools. SBIR companies and agency staff who act as technical points of contact—like “customers” in other markets—could be important sources of information about program strengths and weaknesses.⁸⁹
   2. Similarly, NASA should develop mechanisms to share information about NASA SBIR projects. Electronic tools should be used through which recipients can share information about NASA SBIR projects, helping them to find technical or marketing partners and to navigate the often-complex environment of NASA programs.⁹⁰
   3. NASA should draw on and adopt best practices from its Field Offices. The NASA SBIR Program Office should identify, evaluate and possibly adopt and adapt program management tools that have already at least in part been developed at Field Centers, such as Langley Research Center.⁹¹
3. NASA should better use the data it collects on the SBIR program.⁹²These data should be utilized to systematically guide program management.
   1. NASA should analyze the data to identify potential factors associated with successful transition first to Phase III and then to adoption by NASA Mission Directorates.⁹³ By collecting more and better outcomes and participation data, as well as participant feedback, NASA will be better positioned to undertake regular analysis of key program management issues, such as the following:
      • What partnership programs and other commercialization supports encourage transition beyond Phase II?
      • How successfully do NASA selection processes predict eventual successful projects? How could these be improved?
      • What Field Center activities and initiatives are associated with better outcomes? Can these be developed into best practices and transferred across the agency?

__________________

⁸⁸ See Finding VI-C.

⁸⁹ See Finding VI-C.

⁹⁰ See Finding II-B.

⁹¹ See Finding VI-D and the discussion in Chapter 2 (Program Management).

⁹² See Finding VI-C.

⁹³ See Finding II-A.

• To what extent is NASA connecting with relevant programs within DoD?

4. NASA should improve its reporting on the SBIR program.⁹⁴

Although caution should be employed when imposing new reporting burdens on the NASA SBIR program, implementation of an improved data collection and information management system would provide a cost- and time-effective basis on which to provide better reporting on the program. The annual report recommended below would provide much improved transparency and a coherent point of discussion for other stakeholders. This annual report would effectively replace the existing report to SBA, which is of limited utility for NASA or other stakeholders.

1. The NASA SBIR program should provide a comprehensive annual report to Congress and the public on its operations.⁹⁵
2. Although the precise details should be left to the agency, NASA should consider including five areas of program operations:⁹⁶
   • Inputs, including aggregate current and longitudinal data on numbers of applications and awards, broken out by relevant subgroups (such as demographics, region).
   • Program management and initiatives, which would provide an opportunity to describe any initiative undertaken by the program as well as how the agency is addressing known issues such as the Phase I-Phase II gap.
   • Aggregated outcomes drawn from any data collected on outcomes and subsequent analysis.
   • Improved qualitative outcomes information drawing on case studies, better quality success stories, and social media.
   • Summary conclusions about the extent to which NASA is meeting Congressional objectives for the program and plans for the coming year to do so more effectively.
3. The proposed annual report to Congress should replace all other reporting requirements imposed on the program by Congress and the SBA.

__________________

⁹⁴ See Finding VI-D.

⁹⁵ See Finding VI-C.

⁹⁶ See Finding VI-C.

II. Addressing Under-represented Populations

NASA should immediately enhance efforts to address the Congressional mandate to foster the participation of under-represented populations in the SBIR program.⁹⁷

1. Quotas are not necessary.⁹⁸
   1. While NASA should strive to increase participation of underrepresented populations in the SBIR program, it should not develop quotas for that purpose.
2. NASA should develop new and improved metrics.⁹⁹
   1. The NASA SBIR-STTR Program Office should develop metrics for the participation of under-served populations in the program. Simply counting awards and applications will not be sufficient: it will be important to develop appropriate metrics in the context of the pool of potential applicants, and we recommend that NASA work closely with experts from the science and engineering indicators group at NSF to develop the more sophisticated metrics required.¹⁰⁰
   2. NASA should ensure that analysis of minority participation fully disaggregates the participation of socially disadvantaged groups by ethnicity, and that attention is focused on the clear Congressional intent to support “minority” participation. The current SBA definition of “socially and economically disadvantaged” is not in any way sufficient to meet this objective.¹⁰¹
   3. NASA’s analysis should address key questions, which would include levels and trends for the following metrics:¹⁰²
      • Shares of applications from companies owned by women and minorities
      • Shares of applications with woman and minority PIs
      • Shares of Phase I awards
      • Shares of Phase II awards
   4. Metrics should also track related program operations, including outreach efforts and initiatives at the program and Field Center levels

__________________

⁹⁷ See Finding III-B.

⁹⁸ See Box 6-3.

⁹⁹ See Finding III-C.

¹⁰⁰ See Finding III-D.

¹⁰¹ See Finding III-B.

¹⁰² See Finding III-C.

to enhance commercialization and NASA uptake of SBIR technologies (see below).¹⁰³

3. NASA should develop outreach and education programs focused on expanding participation of under-represented populations.¹⁰⁴ This will require the provision of agency resources and senior staff time and should be a high priority for the program. NASA will need to make concerted efforts in this area.
   1. NASA should develop a coherent and systematic outreach strategy that provides for cost-effective approaches to enhanced recruitment, developed in conjunction with other stakeholders and with experts in the field. This can in part build on existing efforts at some Field Centers, notably those at the Johnson Space Center as well as other efforts to enhance diversity at NASA.¹⁰⁵
   2. NASA should ensure that outreach to selected populations is an integral part of its overall program outreach activities. These should be defined as women and minorities who are also qualified participants in the SBIR program. Piggybacking on other NASA activities for general outreach is not sufficient.¹⁰⁶
   3. NASA should review internal award and selection data and processes to identify and then understand and potentially address disparities between success rates and award levels for disadvantaged and not disadvantaged populations.¹⁰⁷
   4. NASA should provide significant management resources to address the participation of under-represented populations because expanding their participation is likely to be a difficult and long-term effort. Specifically, NASA should designate a senior staffer to work exclusively on participation issues, providing for both improved reporting and the deployment of new initiatives laid out in the new strategy identified in 1) above.¹⁰⁸
4. NASA should share best practices.
   1. NASA SBIR and STTR program representatives should meet regularly with other SBIR/STTR agencies to share best practices and strategies to increase participation of under-represented populations.

__________________

¹⁰³ See Finding III-B and III-D.

¹⁰⁴ See Finding III-B.

¹⁰⁵ See Finding III-D.

¹⁰⁶ See Finding III-D

¹⁰⁷ See Finding III-C.

¹⁰⁸ See Finding III-B.

III. Commercialization

The NASA SBIR program has focused primarily on uptake within NASA, and has appropriately placed priority on agency utilization of SBIR-funded technologies. However, the agency has recognized that there are limits to the size and scale of commercialization within NASA; commercialization outside NASA remains an important objective for the program.

1. NASA should improve support for the commercialization of SBIR technologies.¹⁰⁹
   1. Reviewing technology infusion activities. The activities of TIMs should be reviewed in light of new data to be collected. Currently, it is difficult to determine whether they are effective providers of support services, particularly for the critical role of connecting companies to markets outside NASA.¹¹⁰
   2. Identifying and applying best practices. Potentially, different approaches adopted by various NASA Field Centers could provide valuable data on more and less effective commercialization support strategies. Such analysis would of course require better data and a commitment to this kind of analysis and subsequent follow-up.¹¹¹
   3. Leveraging existing programs and opportunities. For example NASA should explore more systematic ways to connect with DoD SBIR commercialization efforts, particularly given the significant overlap between NASA and DoD revealed by the 2011 Survey and the DoD CCR database. The innovative program to utilize existing suppliers (Boeing) and a new entrant (Space-X) to replace the original NASA-run shuttle program to resupply the space station is an example of this kind of initiative.¹¹²
      • NASA should seek ways to connect directly to the Air Force SBIR program, and to relevant activities of others at DoD (e.g. MDA, DARPA, and Navy). For example, NASA could seek to participate in the annual Navy Opportunity Forum, in which some Centers have participated on a pilot basis in the past, or Air Force efforts in this area.
      • NASA should seek ways to capitalize on the overlap between DoD primes and NASA primes, with a view to helping companies bring NASA funded technologies to the wider DoD market.

__________________

¹⁰⁹ See Finding VI-A and Finding VI-B.

¹¹⁰ See Finding VI-A.

¹¹¹ See Finding VI-D.

¹¹² See the discussion under “Reviewing the Evidence” in this chapter.

4. NASA should explore ways to connect the SBIR program to the emerging commercial space sector.¹¹³
   • Given the clear overlap of technological interests, NASA should on a pilot basis develop topics that are of special interest not only to NASA Mission Directorates but the commercial space sector as well.
   • NASA should also better connect with state and local entrepreneurship programs and venture capital and angel groups to help drive commercially minded companies into the program.

IV. Improving Program Management

Recommendations in this section are designed to improve program operations in ways that should enhance the program’s ability to address some or all legislative objectives.

1. NASA should improve the application process.¹¹⁴
   1. NASA should improve connections with applicants prior to application. The current rigid exclusion of contacts between NASA technical staff and company scientists once a solicitation is published is counterproductive. It generates proposals that do not adequately address NASA needs and potentially excludes others that could be submitted if more information was available. NASA should review and potentially adopt DoD’s pre-release period, which is used to provide precisely the connection between agency technical staff and company researchers that NASA currently prohibits.¹¹⁵
   2. NASA should also review and consider for adoption a “white paper” process such as those in use at DoE and NSF. White papers provide a structured opportunity for companies to present possible approaches to NASA prior to full submission. Using this approach can reduce the application burden on companies and NASA reviewers, while improving the average quality of applications.¹¹⁶
   3. NASA should not use low success rates to validate program quality. These variables are not in principle related (if more weak proposals were screened out before application, then overall success rates would go up but so would the quality of funded projects). However, very high success rates especially for Phase I are a warning sign that the program

__________________

¹¹³ See the discussion under “Study Findings.”

¹¹⁴ See Finding VI-A and Finding VI-D.

¹¹⁵ See Finding VI-D-4.

¹¹⁶ See Finding VI-D-4.

is not attracting sufficient applications and in particular sufficient high-quality applications.

4. The declining numbers of applications noted in the Awards chapter are a potential concern. Applications are down more than 50 percent between 2005 and 2013. NASA should contact companies that have dropped out of the program to determine why they have done so.¹¹⁷

2. NASA should adopt more flexible contracting practices to encourage firm participating in the program.¹¹⁸
   1. NASA should reduce unnecessary rigidities in its contracting procedures:¹¹⁹
      • NASA should provide more flexibility in terms of payment schedules. It is not in the agency’s interest to insist that work be spread out over the full two years of a Phase II project it if could be accomplished faster with more flexible application of the same resources. The NASA SBIR-STTR Program Office should explore adoption of contracting procedures that are already available within NASA under the Space Act to accelerate payments appropriately.
      • NASA should consider permitting no-cost extensions to Phase I and Phase II awards. High-quality research cannot always be completed exactly on schedule, and the short timeframe for Phase I awards as well as the need to conclude the project to prepare for immediate filing of Phase II applications (see below) force companies to focus only on work that they know can easily be accomplished within a few months. NASA already takes a long time to initiate funding for Phase II awards; some of the gap could be utilized to permit companies to better complete funding Phase I research.
      • NASA should consider loosening the very tight timeline between the end of Phase I and the deadline for Phase II submission (currently 2 weeks). This timeline seems unnecessarily tight and likely reduces the quality of Phase II applications to no purpose (given the long gap between Phase I and Phase II; see B.2).

__________________

¹¹⁷ See Finding VI-D-4.

¹¹⁸ See Finding VI-D.

¹¹⁹ See Finding VI-D-1.

2. NASA should review and address the overly long gap between Phase I and Phase II.¹²⁰
   • NASA should substantially reduce the size of the Phase I-Phase II gap. 233 days is not an acceptable gap for small companies to face without funding, especially for worthy projects that are in the end funded.
   • NASA should also consider ways to provide bridge funding during this period, in line with practice at some other agencies. It might also explore options such as Fast Track at NIH that combines Phase I and Phase II applications.
   • NASA should permit companies to work at their own risk after Phase II submission, being prepared to pay for this work if a Phase II award is eventually made. This accelerates work at no risk or additional cost to NASA.

__________________

¹²⁰ See Finding VI-D-2.