An Assessment of the SBIR Program at the National Aeronautics and Space Administration (2009) / Chapter Skim
Currently Skimming:
Appendix D: NRC Project Manager Survey
Appendix D: NRC Project Manager Survey
Pages 198-217
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 198... ...
One means of gauging the performance of the SBIR program in light of these criteria is to obtain assessments by agency project managers who manage contracted SBIR programs. The survey findings reported here represent the first systematic effort to obtain such data from SBIR project managers.
|
From page 199... ...
General Background The sample of project managers was first based on the original database provided by Peter Cahill of BRTRC of 11,684 SBIR Phase II projects in the five SBIR agencies (the three listed above plus NSF and NIH, which award grants, not contracts, and hence do not have contract officers as such)
|
From page 200... ...
In Question 2 of the questionnaire, the respondent was asked if he or she was involved in generating the topic that led to this particular Phase II award. Table App-D-1 provides the response to this question for the total sample of project managers and results broken down by agency.
|
From page 201... ...
, and agency-to-agency. Question 10 of the survey asked about the project managers' role(s)
|
From page 202... ...
Overall, project managers seem to have a relatively minor role in Phase III funding for the agencies and projects studied. This is an important finding since project managers would be knowledgeable natural advocates for the project within the agency.
|
From page 203... ...
First, we use Questions 4 and 5 to arrive at a rating of the quality of SBIR Phase II projects from our sample as seen by the project managers. Question 4 asks the respondent to, on a scale of 1 to 10, where 10 represents the best research ever produced in their research unit/office or for their research unit/office and 1 represents the worst research ever produced in their research unit/office or for their research unit/office, rate the quality of the particular SBIR contract listed.
|
From page 204... ...
Using a t-test, these differences are statistically significantly different from zero. The average quality score is slightly lower for SBIR projects than for non-SBIR research projects (although the differences are statistically significant)
|
From page 205... ...
The next question we raised in the survey dealt more with usefulness of the research for the research unit/office. In particular, we asked Question 6 -- Has the TABLE App-D-9 Distribution of Scores: SBIR Quality and Average NonSBIR Project Quality (DoD)
|
From page 206... ...
The responses to this question varied across agencies somewhat. Of those project managers who responded "No, this project was a separate project, and the knowledge generated by this SBIR contract has had no impact on the other TABLE App-D-11 Distribution of Scores: SBIR Quality and Average NonSBIR Project Quality (NASA)
|
From page 207... ...
However, the response was only 25 percent from DoD project managers, 50 percent from DoE project managers, and 44 percent from NASA project managers. The low percentage response for DoD is indicates relatively high linkage and usefulness and this is probably not a surprise for an agency that is so mission driven.
|
From page 208... ...
Information on another dimension of quality and usefulness of the SBIR program was sought through Question 8 -- "In comparison to a dollar spent in your research unit/office on other R&D projects, how did a dollar spent on this SBIR project rank" is shown in Table App-D-13. For the total sample around 28 percent responded that the particular SBIR project had fewer benefits for their agency's mission than the average dollar spend on other contracts they sponsor.
|
From page 209... ...
The project managers who responded the same benefits amounted to around 44 percent for the total sample, ranging from around 40 percent for DoD respondents to 52 and 50 percent for DoE and NASA respondents respectively. Overall the conclusion is that over 71 percent of the SBIR projects had at least as high benefits for the mission of the agency as the average dollar spent on non-SBIR projects.
|
From page 210... ...
Same as average 44.50% 41.80% c. Less than average 25.83% 36.07% "ownership" as those project managers who had a potential stake in the project as demonstrated either by being involved in defining the topic or if they were involved with the firm before the Phase I proposal.
|
From page 211... ...
d. Yes, but blind alley 8 (14.54%)
|
From page 212... ...
How Project Managers Relate to Phase III One final dimension of outcome of an SBIR project that was covered by the project manager survey relates to Phase III funding. Question 11 probed this issue as follows: "This SBIR project received Phase III funding from your agency in TABLE App-D-20 Does SBIR Project Have Non-commercial/Intrinsic Use (Perhaps in Research)
|
From page 213... ...
52 (64.20%) 81 the form of: Further non-SBIR R&D funding; Direct procurement of the product of this SBIR Procurement through incorporation of the result of this project into a system; No Phase III from agency; Unknown." Results of this question are shown in Table App-D-25.
|
From page 214... ...
Most agreed that there is a relative abundance of fundable SBIR proposals that do not get funded. Surprisingly, SBIR project managers seem to have little involvement in what is already a relatively low rate of Phase III activity.
|
From page 215... ...
• Before the Phase I proposal • After the Phase I proposal but before the Phase II proposal • After the Phase II proposal, but before the Phase II project was completed • After the Phase II project was completed 4. a 1 to 10 scale, where 10 represents the best research ever produced in On your research unit/office or for your research unit/office and 1 represents the worst research ever produced in your research unit/office or for your research unit/office, rate the quality of the research in this particular SBIR contract.
|
From page 216... ...
11. This SBIR project received Phase III funding from your agency in the form of: • Further non-SBIR R&D funding • Direct procurement of the product of this SBIR • Procurement through incorporation of the result of this project into a system • No Phase III from agency • Unknown 12.
|
From page 217... ...
(Optional) Annex B1 QUESTION 15. If you have any comments that you think would help us evaluate the effectiveness of the SBIR Program in producing useful research for your agency, please feel free to comment.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.