An Assessment of the SBIR Program at the National Aeronautics and Space Administration (2009) / Chapter Skim
Currently Skimming:
4 SBIR Program Outcomes
4 SBIR Program Outcomes
Pages 55-104
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 55... ...
Assessing program outcomes against these four objectives entails numerous See the SBIR Reauthorization Act of 2000 (H.R. 5667 -- Section 108)
|
From page 56... ...
Comparison with Other SBIR Programs Comparing the NASA SBIR program with those at other agencies is superficially more useful. However, as discussed in Chapter 1 of this volume, the SBIR programs at each of the agencies are shaped by the different agency missions.
|
From page 57... ...
• The NRC Phase I Survey covers projects that failed to proceed beyond Phase I • The NRC Project Manager Survey.
|
From page 58... ...
While the NRC Phase II Survey broke new ground in data collection, the amount of sales made -- and indeed the number of projects that gener ate sales -- are inevitably undercounted in a snapshot survey taken at a single point in time. Based on successive data sets collected from NIH SBIR award recipients, it is estimated that total sales from all responding projects will likely be on the order of 50
|
From page 59... ...
For the NRC Phase II Survey, of the 534 NASA firms that could be contacted out of a sample size of 779, 181 responded, representing a 34 percent response rate. The NRC Phase I Survey captured 9 percent of the 3,363 awards made by NASA over the period of 1992 to 2001.
|
From page 60... ...
In time, NASA recognized that this compromise was not successful. On one hand, the NASA's mission realities limited the commercial potential of NASA SBIR projects.
|
From page 61... ...
Finifter in "Evaluation of the Department of Defense Small Business Innovation Research Program and the Fast Track Initiative: A Balanced Approach" in National Research Council, SBIR: An Assessment of the Department of Defense Fast Track Initiative, Charles W Wessner, ed., Washington, DC: National Academy Press, 2000.
|
From page 62... ...
This means that for a significant number of the more recent SBIR projects, commercialization is still in process, and sales -- often substantial sales -- will be made in the future (see the "snapshot effect" discussed in Box 4-2)
|
From page 63... ...
As noted above, Phase III activities within NASA are a primary form of commercialization for NASA SBIR projects. These activities are considered in the mission support section (Section 4.3)
|
From page 64... ...
A few awards generate substantial results, and a small number bring in large revenues.20 Of the 74 NASA SBIR Phase II projects reporting sales greater than $0, average sales per project were $1,154,156. About 40 percent of the total sales dollars were due to the two NASA projects responding to the survey that had received $5,000,000 or more in sales.
|
From page 65... ...
SOURCE: NRC Phase II Survey. Based on responses to Phase II Survey questions 1a, NASA Figure 4-1.eps 1b, 3a, and 3b.
|
From page 66... ...
. According to NASA staff, full commercialization of NASA SBIR projects usually occurs only 7 years after completion of the Phase II award.22 22 Interview with Carl Ray, SBIR Program Director, November 12, 2007.
|
From page 67... ...
About 66 percent of the NRC Phase II Survey respondents at NASA with no sales still expected sales in the future (14 percent of all projects responding) .23 Sales by Sector The NRC Phase II Survey asked respondents to identify the customer base for the products.
|
From page 68... ...
These data have important strengths and weaknesses. The NRC Phase II Survey was sent to all firms with SBIR Phase II awards from 1992 2002.
|
From page 69... ...
Thus, the average sales data generated by surveys reflects average sales to date. Using some simple analytic techniques, it is possible to estimate that on average, the NRC Phase II Survey (and other similar surveys such as the NIH Phase II Survey)
|
From page 70... ...
On average, NASA SBIR 24 NRC Phase II Survey, Question 4. 25 See TiNi Alloy case study in Appendix E
|
From page 71... ...
SOURCE: NRC Phase II Survey, Question 23. projects received almost $800,000 from non-SBIR sources, with over half of survey respondents (51.6 percent)
|
From page 72... ...
Companies and Investors Interactions with 0 0 1 3 0 0 0 0 Foreign Companies and Investors SOURCE: NRC Phase II Survey, Question 12. investors rather than sales abroad.
|
From page 73... ...
SBIR Impact on Further Investment The NRC Phase II Survey sought additional information about the impact of the SBIR program on company efforts to attract third party funding -- the "halo effect" mentioned by some interviewees, who suggested that an SBIR award acted as form of validation for external inventors.29 The fact that 60 percent of NASA SBIR respondents reported no outside funding, and that none at all received venture funding, suggests that receiving a NASA Phase II SBIR award may have only a limited effect in improving the likelihood of external funding for these recipients.30 4.2.4 Small Company Participation and Employment Effects Growing employment is another indicator of commercial success. It also provides evidence that the program is supporting small business.
|
From page 74... ...
Among NRC Phase II Survey respondents, about just over a quarter had between one and five employees at the time of award. A majority (64 percent)
|
From page 75... ...
The NRC Phase II Survey explored whether SBIR awardees had finalized agreements or ongoing negotiations on various companylevel activities. This information is summarized in Table 4-6.
|
From page 76... ...
76 SBIR AT THE NATIONAL AERONAUTICS AND SPACE ADMINISTRATION BOX 4-3 Detecting Toxins at a Distance: The Case of Intelligent Optical Systemsa Intelligent Optical Systems (IOS) has developed a system for using the entire length of a specially-designed fiber-optic cable as a sensor for the detection of toxins and other agents.
|
From page 77... ...
The NRC Phase II survey respondents indicated that 46 percent of NASA SBIR projects had reached the market. These conclusions broadly align with views from within the agency: according to the NRC Project Manager Survey, about 35 percent of projects are believed to have commercialized.
|
From page 78... ...
Better utilization of the NASA SBIR program by the agency must be matched by improved and timelier tracking of outcomes. 4.3 AGENCY MISSION Agency missions vary substantially by agency -- indeed, each agency has a unique mission by design.
|
From page 79... ...
At NASA, the structure of the SBIR solicitation, and the technical taxonomies used to define topics and subtopics, parallel the structure and taxonomies of the Mission Directorate technology program roadmaps. This supports alignment between SBIR projects and agency mission.
|
From page 80... ...
NASA has now also formalized the previous practice of providing additional SBIR awards to centers that provide matching funds -- a clear sign that the proposed project is a high priority item. The new program is called Phase IIe.
|
From page 81... ...
However, the stories themselves, while illustrative of the power of the program to help develop new technologies, are of variable quality. The NRC Phase II Survey data suggest that NASA SBIR has found limited take-up within the agency.
|
From page 82... ...
The differences in means between the score for the sample of SBIR and non-SBIR projects are given in Table 4-9. For the total sample of NASA SBIR projects, the SBIR projects were on average somewhat lower.40 Linkage to Research Mission The NRC Project Manager Survey also addressed the usefulness of the research for the research unit/office, and in particular the extent of linkages between the SBIR research project and other research conducted by the agency.
|
From page 83... ...
SOURCE: NRC Project Manager Survey. Comparative Value of SBIR Projects Another way to look at the value of SBIR-funded research from the agency's perspective is to consider the utility of money spent on SBIR projects compared with money spent on other R&D.
|
From page 84... ...
This response helps to confirm the view that SBIR in general receives high quality applications, and that these are more applications that could be funded than there is funding available. TABLE 4-12 Comparative Dollar Value of Projects Number of Responses Percent Other Other Dollar Value Agencies NASA Agencies NASA SBIR project had fewer benefits for your 119 26 27.6 31.7 agency's mission than the average dollar spent on other contracts you sponsor SBIR project had more benefits for your 128 15 29.7 18.3 agency's mission than the average dollar spent on other research contracts you sponsor Same Benefits 184 41 42.7 50.0 TOTAL SAMPLE SIZE 431 82 100.0 100.0 SOURCE: NRC Project Manager Survey.
|
From page 85... ...
In this context, "ownership" means that the project manager had a potential stake in the project as demonstrated either by being involved in defining the topic or being involved with the recipient firm before the Phase I proposal. Table 4-14 shows that managers with ownership had a much higher opinion of their SBIR projects than those with more of a connection to their projects.
|
From page 86... ...
. However, the proceedings from the NRC Phase III Conference and other agency interviews at NASA and DoD indicate that the timing of that involvement is important.41 About 40 percent of NASA respondents reported that they became involved with the SBIR project before Phase I -- i.e., during the topic development stage.
|
From page 87... ...
Evidence from the NRC Project Manager Survey: Conclusions The NASA SBIR project managers in our sample appear to be, for the most part, engaged with the SBIR program. Many were involved with their projects early and often.
|
From page 88... ...
mission, and the majority of respondents reported that they had more high quality SBIR proposals than they could fund. NASA SBIR project managers seem uninvolved in Phase III activity.
|
From page 89... ...
4.3.3 Conclusions: Agency Mission The case studies and data from the NRC Project Manager Survey both show that NASA has in the past successfully used SBIR to develop technologies that were critically important for some NASA missions. However, NASA itself determined that the existing approach, driven largely from the bottom up by the needs and interests of specific centers, was not working well enough.
|
From page 90... ...
Mode Wing Leading Edge UltraWIS SBIR EWIS Extreme 2005 Standards-based Low Power Impact Integration IDAA STTR DIDS SBIR Detection FIGURE 4-5 Technology Development Tree for Wireless Data acquisition. SOURCE: John Saiz, Johnson Space Center, Houston, TX.
|
From page 91... ...
Woman-owned Firms 30 Minority-owned Firms Either Percent of All Phase I Awards 25 20 15 10 5 0 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 Year NASA Figure 4-6.eps FIGURE 4-6 NASA SBIR Phase I awards, by demographic group, 1992-2005. SOURCE: National Aeronautics and Space Administration.
|
From page 92... ...
Project-level Impacts One way of measuring SBIR's impact is to ask awardees whether their projects would have been implemented without SBIR program funding. Data in Figure 4-8 from the NRC Phase II Survey strongly suggest that SBIR provides funding that plays a determinant role to most of the projects that receive it.
|
From page 93... ...
. NASA Figure 4-8.eps SOURCE: NRC Phase II Survey, Question 13.
|
From page 94... ...
Examples of these firm types can be seen in the case studies of SBIR awardees. Further research in this area may help to establish better, how the NASA SBIR program supports these different kinds of businesses, and businesses at different stages of development.
|
From page 95... ...
No projects generated more than three applications, and only one received three or more patents. 4.5.2 Scientific Publications Publication in peer-reviewed journals and conference proceedings are a standard method for disseminating scientific knowledge.
|
From page 96... ...
Tracking Knowledge Dissemination by Citation Analysis Citation studies have been used extensively to show the transfer of knowledge from federally funded projects to others outside the walls of the funded TABLE 4-19 Publications from NASA SBIR Phase II Awards Number of Publications Number of Responses 0 97 1 20 2 19 3 6 4 4 5 8 8 2 10 3 12 1 30 1 Total 161 SOURCE: NRC Phase II Survey, Question 18.
|
From page 97... ...
Both publication and patent citation analysis could be used to demonstrate and track knowledge dissemination from NASA SBIR projects to others. 4.5.3 Licensing Licensing agreements depend on the protection of the intellectual property.
|
From page 98... ...
The NRC Phase II Survey provided insight about the kinds of partnerships being formed by SBIR recipients. As shown in Table 4-21, partnerships for R&D, for marketing and distribution, with customers, and for manufacturing were found to be formed by these awardees.
|
From page 99... ...
Table 4-22 indicates the extent to which each type of university involvement occurred in the sample Phase II projects. The NRC Phase II Survey results show that the NASA SBIR plays some role in moving research concepts out of the university.
|
From page 100... ...
TABLE 4-23 In Executing the Phase II Award Was There Involvement by University Faculty, Graduate Students, and/or University-developed Technologies -- (N=161) Response Percent of Respondents Yes 29 No 71 SOURCE: NRC Phase II Survey, Question 31.
|
From page 101... ...
• The NRC Phase I survey also suggested that technical risk among NASA projects was high. Of the Phase I projects that did not get a follow-on Phase II award, a leading reason was technical barriers.
|
From page 102... ...
Other indicators of commercialization, such as licensing activities, marketing partnerships, and access to and utilization of further investments from both private and public sources, all confirm that while returns are highly skewed, and the results in general are positive, firms operating within the NASA SBIR program face significant 54 National Research Council, An Assessment of the Small Business Innovation Research Program -- Project Methodology, op.
|
From page 103... ...
• Agency staff in general indicate that SBIR awards are of comparable quality to other NASA research projects. This analysis also indicates that early involvement with and "ownership" of SBIR projects by NASA technical staff is an important factor in the successful utilization of SBIR for agency purposes.
|
From page 104... ...
It is therefore appropriate to conclude that the NASA SBIR program is meeting all four of the congressional objectives.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.