National Academies Press: OpenBook

An Assessment of the Small Business Innovation Research Program: Project Methodology (2004)

Chapter:Annex J Template for Individual Agency Reports

« Previous: Annex I Tasks to Further Develop and Implement the Methodology
Suggested Citation:"Annex J Template for Individual Agency Reports." National Research Council. 2004. An Assessment of the Small Business Innovation Research Program: Project Methodology. Washington, DC: The National Academies Press. doi: 10.17226/11097.
×
Page81
Suggested Citation:"Annex J Template for Individual Agency Reports." National Research Council. 2004. An Assessment of the Small Business Innovation Research Program: Project Methodology. Washington, DC: The National Academies Press. doi: 10.17226/11097.
×
Page82
Suggested Citation:"Annex J Template for Individual Agency Reports." National Research Council. 2004. An Assessment of the Small Business Innovation Research Program: Project Methodology. Washington, DC: The National Academies Press. doi: 10.17226/11097.
×
Page83
Suggested Citation:"Annex J Template for Individual Agency Reports." National Research Council. 2004. An Assessment of the Small Business Innovation Research Program: Project Methodology. Washington, DC: The National Academies Press. doi: 10.17226/11097.
×
Page84
Suggested Citation:"Annex J Template for Individual Agency Reports." National Research Council. 2004. An Assessment of the Small Business Innovation Research Program: Project Methodology. Washington, DC: The National Academies Press. doi: 10.17226/11097.
×
Page85

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Annex J: Template for Individual Agency Reports98 Introduction Program history at the agency Basic demographics of the agency program – number of awards, $ awarded, trends over past 20 years [drawing on program manager survey] Organization – who runs what, under what offices, etc. Short description – number of competitions annually, use of topics etc. General methodology problems and approaches* Comparability Measurement Bias Multiple methodology approach, plus brief description of major instruments and approaches Success stories box Outcomes at the Agency Commercialization Strategies and Outcomes Difficulties of evaluating* Research methodology P2 survey data for the agency Selected case study data for the agency Agency-initiated analysis and data Conclusions Mission support Short intro on agency differences, and procuring vs. non-procuring agencies* Identification of specific agency as procuring/non-procuring Research methodology 98 Note: starred (*) items are expected to be identical across all agency reports. 81

Agency interviews [within SBIR program and at more senior levels] Procuring agencies only: Agency contracts analysis (DoD, NASA, parts of DoE) Conclusions Small business support Problems of measuring additionality, and impact* Research methodology* If available, comparative impact study for specific agency (funded vs. unfounded applicants to SBIR) Basic demographics of agency support for small business Small business research $ as share of agency R&D spending; SBIR share of all agency R&D spending on small businesses Conclusions Knowledge base Largely indirect character of knowledge expansion in the commercial environment; other “knowledge base” issues; comparability issues* Research methodology* Patents and trademarks Licensing Citation analysis Equity sales [why is this here?] Partnerships Indirect path: indications, not measurement* Conclusions Program management Topic development and selection Sources for topic ideas Agency-driven vs. investigator driven approach to topics Topic decision-making 82

Outreach (before successful application) Agency outreach objectives (e.g. reaching low-application states and regions; educating very likely applicants.) Outreach programs that the agency runs or participates in (including brief description) Agency support for P1 applicants Agency outreach benchmarks and metrics Grant selection Description of selection processes for P1 and P2 Peer review panels – membership, selection, qualifications Fairness – any issues and changes in procedures to meet problems; (P2 winner and non-winner views from surveys) [make sure there is a “fairness” question in the P2 survey] Scoring procedures Role of program manager Re-submission procedures and outcomes Training (after successful application) Training programs for agency P1 and P2 grantees Benchmarks used to evaluate effectiveness Take-up rates and projections Constraints Phase 2+ program Description of agency’s P2+ program Use of matching funds Application and selection procedures Role of program manager Funding “gaps.” P1 – P2 gap. Average size, drivers. Options for addressing. After P2. Data on subsequent agency funding/take-up of technologies. Agency perceptions of its role. Bridge funding programs (after P2). Program description and objective Limitations (e.g. matching funding requirements) What is the average extent of the gap What is the primary cause of the gap 83

Does the agency think there is a problem Is the agency finding ways to address the problem Reporting requirements Reports submitted to the agency by SBIR winners Report utilization, and utility to the agency Evaluation and assessment Annual and intermittent agency evaluations of its SBIR program Operational benchmarks for the agency SBIR program Evaluators (internal and external) Annual evaluation and assessment budget, and funding sources Flexibility Program managers discretion (project selection, project size, program management) Program manager tools (e.g. shifting funding from one sub-agency to another; adjusting scoring criteria; changing the size of awards) Agency manager perceptions of constraints Size Formal and effective limits on size and duration of awards Distribution of P1 and P2 awards within those limits Availability of additional funding. Amounts and distributions. Online capabilities and plans Components of the grant application, management, and reporting process NOT yet online Plans and timelines for putting these remaining elements online Barriers obstructing implementation of these plans Administrative funding Funding of program administration Control of administration budget Administration budget as percent of agency SBIR funding Evaluation and assessment funding Recommendations Appendices 84

Appendix A. Agency data about its program (incl. demographic data) Appendix B. Agency P2 survey results Appendix C. Agency P1 survey results Appendix D. Selected case studies Appendix E. Selected agency reports and data 85

Next: 1. DRAFT PHASE I SURVEY »
An Assessment of the Small Business Innovation Research Program: Project Methodology Get This Book
×
Buy Paperback | $43.00 Buy Ebook | $34.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

In response to a Congressional mandate, the National Research Council conducted a review of the SBIR program at the five federal agencies with SBIR programs with budgets in excess of $100 million (DOD, NIH, NASA, DOE, and NSF). The project was designed to answer questions of program operation and effectiveness, including the quality of the research projects being conducted under the SBIR program, the commercialization of the research, and the program's contribution to accomplishing agency missions. This report describes the proposed methodology for the project, identifying how the following tasks will be carried out: 1) collecting and analyzing agency databases and studies; 2) surveying firms and agencies; 3) conducting case studies organized around a common template; and 4) reviewing and analyzing survey and case study results and program accomplishments. Given the heterogeneity of goals and procedures across the five agencies involved, a broad spectrum of evaluative approaches is recommended.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!