SBIR/STTR at the National Institutes of Health (2015) / Chapter Skim
Currently Skimming:
Appendix A: Overview of Methodological Approaches, Data Sources, and Survey Tools
Appendix A: Overview of Methodological Approaches, Data Sources, and Survey Tools
Pages 275-289
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 275... ...
References in this report to the National Research Council or NRC are used in an historic context identifying programs prior to July 1. 2 National Research Council, An Assessment of the Small Business Innovation Research Program: Project Methodology, Washington, DC: The National Academies Press, 2004.
|
From page 276... ...
. By providing awards to SBCs for cooperative R/R&D efforts with Research Institutions, the STTR Program assists the small business and research communities by commercializing innovative technologies."5 The SBIR/STTR programs, on the basis of highly competitive solicitations, provide modest initial funding for selected Phase I projects (up to $150,000)
|
From page 277... ...
Consequently, this document suggests a broad frame work for methodological approaches that can serve to guide the research team when evaluating each particular agency in terms of the four criteria stated above. Table A-1 illustrates some key assessment parameters and related measures to be considered in this study."8 The tools identified in Table A-1 include many of those used by the committee conducting the first-round study of the SBIR/STTR programs.
|
From page 278... ...
Measures Peer-review Sales, follow-up Patent counts Innovative products scores, funding, other and other resulting from publication commercial IP/employment SBIR/STTR work counts, citation activities growth, number analysis of new technology firms Tools Case studies, Phase II surveys, Phase I and Phase Program manager agency program manager II surveys, case surveys, case program discussions, case studies, study of studies, agency studies, study studies, study of repeat winners program studies, of repeat repeat winners study of repeat winners, winners bibliometric analysis Key Difficulty of Skew of returns; Measures of actual Major interagency Research measuring significant success and failure differences in use Challenges quality and interagency and at the project of SBIR/STTR of identifying inter-industry and firm levels; to meet agency proper differences relationship of missions reference federal and state group programs in this context NOTE: Supplementary tools may be developed and used as needed. In addition, since publication of the methodology report, this committee has determined that data on outcomes from Phase I awards are of limited relevance.
|
From page 279... ...
Taken together with committee deliberations and the expertise brought to bear by the individual committee members, these tools provide the primary inputs into the analysis. For the first-round study and for the current study, multiple research methodologies feed into every finding and recommendation.
|
From page 280... ...
Challenges in Tracking Commercialization Despite substantial efforts at NIH, described below, significant challenges r emain in tracking commercialization outcomes for the NIH SBIR/STTR programs. These include the following: • Data limitations.
|
From page 281... ...
requirements for clinical trials prior to commercialization. SURVEY ANALYSIS Traditional modes of assessing the NIH SBIR/STTR programs include case studies, interviews with program staff, review of documents, and other qualitative methods of assessment.
|
From page 282... ...
While there is evidence of bias for project performance count variables such as the number of publications or patents associated with a publicly-subsidized project, there is also evidence that there may not be a response bias for commercialization measures.c • on-respondent bias. Very limited information is available about SBIR/STTR N award recipients: company name, location, and contact information for the PI and the company point of contact, agency name, and date of award (data on woman and minority ownership are not considered reliable)
|
From page 283... ...
Although the 2005 Survey broke new ground in data C collection, the amount of sales made -- and indeed the number of projects that generated sales -- are inevitably undercounted in a snapshot survey taken at a single point in time. On the basis of successive data sets collected from NIH SBIR/STTR award recipients, it is clear that total sales from all respond ing projects will be considerably greater than can be captured in a single survey.f This underscores the importance of follow-on research based on the now-established survey methodology.
|
From page 284... ...
As noted later in the discussion, the use of a control group was found infeasible for comparing Phase II and Phase I recipients, but feasible for comparing Phase IIB and Phase II recipients. Grunwald Associates LLC was contracted to administer a survey to award recipients.
|
From page 285... ...
The aim in setting the overall time frame at 10 years was to reduce the impact of difficulties in generating information about older awards, because some companies and PIs may no longer be in place and memories fade over time. Determining the Survey Population Following the precedent set by both the original GAO study and the first round of Academies analysis, we differentiate between the total population of SBIR/STTR recipients, the preliminary survey target population, and the effective population for this study, which is the population of respondents that were reachable.
|
From page 286... ...
Initial Filters for Potential Recipients Determining the effective study population required the following steps: • acquisition of data from the sponsoring agencies -- NIH and DoE -- overing c record-level lists of award recipients; • elimination of records that did not fit the protocol agreed upon by the committee -- namely, a maximum of two questionnaires per PI (in cases
|
From page 287... ...
Secondary Filters to Identify Recipients with Active Contact Information This nominal population still included many potential respondents whose contact information was complete but who were no longer associated with the contact information provided and hence effectively unreachable. This is unsurprising given that small businesses experience considerable turnover in personnel and that the survey reaches back to awards made in FY2001.
|
From page 288... ...
Given that the operations of the SBIR/STTR programs are defined in legislation and limited by TABLE A-3 2014 Survey Response Rates at NIH Preliminary population 3,375 Not contactable 1,723 Effective population 1,652 Responses 726 Surveys as Percentage of Awards Contacted 43.9 Surveys as Percentage of Sample 21.5 SOURCE: 2014 Survey.
|
From page 289... ...
NIH Responses and Respondents Table A-4 shows NIH SBIR/STTR responses by year of award. The survey primarily reached companies that were still in business: overall, 94 percent of respondents indicated that the companies were still in business.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.