SBIR at NASA (2016) / Chapter Skim
Currently Skimming:
Appendix A: Overview of Methodological Approaches, Data Sources, and Survey To
Appendix A: Overview of Methodological Approaches, Data Sources, and Survey To
Pages 207-223
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 207... ...
2 National Research Council, An Assessment of the Small Business Innovation Research Program: Project Methodology, Washington, DC: The National Academies Press, 2004. 3 National Defense Authorization Act of 2012 (NDAA)
|
From page 208... ...
ESTABLISHING A METHODOLOGY The methodology utilized in this second-round study of the SBIR program builds on the methodology established by the committee that completed the first-round study. Publication of the 2004 Methodology The committee that undertook the first-round study and the agencies under study formally acknowledged the difficulties involved in assessing SBIR programs.
|
From page 209... ...
[Table APP A-1] illustrates some key assessment parameters and related measures to be considered in this study." 6 TOOLS UTILIZED IN THE CURRENT SBIR STUDY Quantitative and qualitative tools being utilized in the current study of the SBIR program include the following: • Case studies.
|
From page 210... ...
SOURCE: National Research Council, An Assessment of the Small Business Innovation Research Program: Project Methodology, Washington, DC: The National Academies Press, 2004, Table 1, p.
|
From page 211... ...
Possible meanings and elements include the following: • issuance of a certified Phase III contract by NASA directly to the small firm; 7 See Chapter 5 (Quantitative Outcomes) for related analysis of commercialization in the SBIR program.
|
From page 212... ...
However, like DoD, and perhaps to an even greater extent, FPDS does not capture Phase III contracts effectively. A recent GAO study concluded that "…comprehensive and reliable technology transition data for SBIR projects are not collected.
|
From page 213... ...
In addition, because designation as a Phase III contract carries with it significant data rights for the small business, there may also be incentives for contracting officers, who do not want to share these rights, to avoid this designation, according to company interviewees. Although the Navy SBIR program has made a concerted effort by devoting dedicated staff time to improving the quantity and quality of Phase III contracts captured by FPDS, this effort is an outlier, and at NASA no such effort has been undertaken.
|
From page 214... ...
Some data from NASA's Electronic Handbook was ultimately made available for this study, but the data was too incomplete to be utilized; however, the EHB does hold substantial promise in eventually helping NASA to address concerns about data collection. OVERVIEW OF THE SURVEY Our analysis of the SBIR program at NASA makes extensive use of case studies, interviews, and other qualitative methods of assessment.
|
From page 215... ...
However, we 9 The survey conducted as part of the current, second-round assessment of the SBIR program is referred to below as the "2011 Survey" or simply the "survey." In general, throughout the report, any survey references are understood to be to the 2011 Survey unless specifically noted otherwise. 10 Delays at NIH and DoE in contracting with the Academies combined with the need to complete work contracted with DoD NSF and NASA led the Committee to proceed with the survey at three agencies only.
|
From page 216... ...
demonstrates that the probability of obtaining research project information by survey decreases for less recently funded projects, and it increases the greater the award amount.b Nearly 75 percent of Phase II responses to the 2011 Survey (the population for which was awards made FY1998-2007) were for awards received after 2003, largely because winners from more distant years are more difficult to reach: small businesses regularly cease operations, are acquired, merge, or lose staff with knowledge of SBIR awards.
|
From page 217... ...
On the basis of successive datasets collected from NIH SBIR award recipients, it is estimated that total sales from all responding projects will be considerably greater than can be captured in a single survey, because technologies continue to generate revenue after the date of the survey. These positive outcomes are therefore not included in any single survey result.e This underscores the importance of follow-on research based on the now-established survey methodology.
|
From page 218... ...
Strict adherence to a revealed preference paradigm could lead to misguided policy conclusions because the paradigm assumes that all policy choices are known and understood at the time that an individual or firm reveals its preferences and that all relevant markets for such preferences are operational.
|
From page 219... ...
Reaching back to awards made in 1998, while ensuring comparability, generated few results from older awards. Determining the Survey Population Following the precedent set by both the original GAO study and the first-round study of the SBIR program, we differentiated between the total population of awards, the preliminary survey target population of awards, and the effective population of awards for this study.
|
From page 220... ...
This process of excluding awards either because they did not fit the protocol agreed upon by the committee or because the agencies did not provide sufficient or current contact information, reduced the total award list provided by the three agencies to a preliminary survey population for all three agencies of
|
From page 221... ...
Secondary Filters to Identify Recipients with Active Contact Information: Identifying the Effective Population This preliminary population still included many awards for which the PI contact information appeared complete, but for which the PIs were no longer associated with the contact information provided and hence effectively unreachable. This is not surprising given that there is considerable turnover in both the existence of and the personnel working at small businesses and that the survey reaches back 13 years to awards made in FY1998.
|
From page 222... ...
Table A-3 shows the response rates at NASA, based on both the preliminary study population and the effective study population after all adjustments Table A-3 shows the response rates at NASA, based on both the preliminary study population and the effective study population. 12 Many surveys of entrepreneurial firms have low response rates.
|
From page 223... ...
These are technology-based companies at an early stage of company development, which have the demonstrated capacity to undertake challenging technical research and to provide evidence that they are potentially successful commercializers. Given that the operations of the SBIR program are defined in legislation and limited by the Policy Guidance provided by SBA, randomly assigned control groups were not a possible alternative.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.