National Academies Press: OpenBook
« Previous: 1. Introduction
Suggested Citation:"2. Overview of the Study Process." National Research Council. 2004. An Assessment of the Small Business Innovation Research Program: Project Methodology. Washington, DC: The National Academies Press. doi: 10.17226/11097.
Suggested Citation:"2. Overview of the Study Process." National Research Council. 2004. An Assessment of the Small Business Innovation Research Program: Project Methodology. Washington, DC: The National Academies Press. doi: 10.17226/11097.

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2. An Overview of the Study Process Following its approval of the broad study parameters of the study in October 2002, the Committee set out an overall roadmap to guide the research process. Tasks included are the development a set of operational definitions, the identification of detailed metrics, the review existing data sources, and the development of primary research methodologies. Closely interrelated, these tasks will be addressed iteratively. (These iterative tasks are represented in the box within Figure 1.) Following completion of field research, the Committee will conduct its analysis and assessment and will issue its findings and recommendations. Figure 1 NRC SBIR Study: The Logic of Analysis Congressional requirements Project Objectives Operational definitions Potential metrics Existing data sources Iterative Primary processes research Assessm ent and analysis Conclusions and recomm endations 9

The elements of this multi-step process are detailed below: 1. Agree on initial guidelines. These initial guidelines are based on the legislation, the Memorandum of Understanding, and contracts. 2. Clarify objectives. What central questions must the study answer? What other interesting but optional questions should be addressed? What questions will specifically not be considered? This is discussed further in Section 3 of this chapter. 3. Develop operational definitions: For example, while Congress has mandated that the study address the extent to which SBIR supports the agencies’ missions, the Committee needs to develop operational definitions of “support” and “agency mission,” in collaboration with agency managers responsible for program operations. This is a necessary step before developing the relevant metrics. This is discussed further in Section 4 of this chapter. 4. Identify metrics for addressing study objectives. The Committee will determine extent of commercialization fostered by SBIR—measured in terms of products procured by agencies, commercial sales, licensing revenue, or other metrics. This is discussed further in Section 5 of this chapter. 5. Identify data sources. Implementation of agreed metrics requires data. A wide mix of data sources will be used, so the availability of existing data and the feasibility of collecting needed data by different methods will also condition the selection of metrics, and the choice of study methods. The existence or absence of specific methodologies and data sets will undoubtedly lead to the modification, adoption, or elimination of specific metrics and methods. This is discussed further in Section 6 of this chapter. 6. Develop primary research methodologies. The study’s primary research components will include interviews, surveys, and case studies to supplement existing data. Control groups and counterfactual approaches will be used where feasible and appropriate to isolate the effects of the SBIR program. Other evaluation methods may also be used on a limited basis as needed to address questions not effectively addressed by the principal methods. This is discussed further in Section 7 of this report. 7. Complete Phase I. Phase I of the NRC study will be formally completed once a set of methodologies is developed and documented, is approved by the Committee, and passes successfully through the Academy’s peer review process. 8. Implement the research program (NRC Study Phase II). The variety of tasks involved in implementing the research program is previewed in Annex I of this report. 9. Prepare agency-specific reports. Results from the research program will be presented in five agency- specific reports—one for each of the agencies. Where appropriate, agency-specific findings and recommendations will be formulated by the relevant study subcommittee for review and approval by the full Committee. 10. Prepare overview report. A separate summary report, buttressed by the relevant commissioned work and bringing together the findings of the individual agency reports, along with general recommendations, will be produced for distribution. This final report will also draw out, as appropriate, the contrasts and similarities among the agencies in the way they administer SBIR. It will follow the approval procedure outlined above. 11. Organize public meetings to review and discuss findings. Following report review, findings and recommendations will be presented publicly for information, review, and comment. 12. Submit reports to Congress. 13. Disseminate findings broadly. 10

Next: 3. Clarifying Study Objectives »
An Assessment of the Small Business Innovation Research Program: Project Methodology Get This Book
Buy Paperback | $43.00 Buy Ebook | $34.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In response to a Congressional mandate, the National Research Council conducted a review of the SBIR program at the five federal agencies with SBIR programs with budgets in excess of $100 million (DOD, NIH, NASA, DOE, and NSF). The project was designed to answer questions of program operation and effectiveness, including the quality of the research projects being conducted under the SBIR program, the commercialization of the research, and the program's contribution to accomplishing agency missions. This report describes the proposed methodology for the project, identifying how the following tasks will be carried out: 1) collecting and analyzing agency databases and studies; 2) surveying firms and agencies; 3) conducting case studies organized around a common template; and 4) reviewing and analyzing survey and case study results and program accomplishments. Given the heterogeneity of goals and procedures across the five agencies involved, a broad spectrum of evaluative approaches is recommended.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook,'s online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!