National Academies Press: OpenBook

A Review of the Environmental Protection Agency's Science to Achieve Results Research Program (2017)

Chapter: Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program

Get This Book
Visit NAP.edu/10766 to get more information about this book, to buy it in print, or to download it as a free PDF.
« Previous: Appendix A: Biographical Information of the Committee on the Review of EPA's Science to Achieve Results Research Grants Program
Page 79 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel

Appendix B

Summary of the Previous Reviews of the Science to Achieve Results Program

Page 80 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel

TABLE B-1 Summary of Reviews of the STAR Program

Review Type of Review Brief summary EPA’s Responses
U.S. Government Accountability Office (U.S. Congress). 2000. Environmental Research: STAR Grants Focus on Agency Priorities, but Management Enhancements Are Possible. Washington, DC. Programmatic The review recommended that:
  • EPA develops program criteria to evaluate the “effectiveness” of each type of STAR grant. EPA to improve “communication and dissemination” of STAR research results within and outside EPA among many of its public and private stakeholders.
  • EPA’s Science Advisory Board (SAB) agreed to review the effectiveness of the results of STAR research by individual research area. EPA contracted with the National Academy of Sciences (NAS) to evaluate the STAR program on a broad scale and then by specific issues.
  • EPA has: (1) developed a system to collect, review, and post abstracts and project reports; (2) established a process to improve the likelihood of timely submission of final reports.
EPA’s Science Advisory Board and the Office of Research and Development’s Board of Scientific Councilors. 2000. A Joint SAB/BOSC Report: Review of the Science to Achieve Results (STAR). Washington, DC: EPA. Scientific and programmatic review This review gave EPA an “overall favorable assessment,” but recommended that:
  • EPA improve “information transfer.”
  • EPA provide additional information in RFAs on research goals and objectives; “relevancy criteria”, and accelerate availability of STAR results.
  • EPA include the ten regional offices into all facets of the STAR process.
  • The STAR program periodically review its research portfolio for its utility and applicability to EPA’s mission.
  • The STAR program expand its partnership with other federal and international agencies as well as private foundations.
  • EPA to look into the evaluation of STAR program results by outside “qualified, highly respected, and independent organization.”
  • No response provided
EPA Office of Inspector General Report, 2003. STAR Fellowship Program Needs to Place Emphasis on Measuring Results. Programmatic review Here, the focus was narrow and on measuring the results of STAR’s fellowship program. EPA was asked to:
  • Maintain necessary data on fellowship applicants and recipients, including collection, maintenance, and review of “demographic composition of the STAR applicant pool and
  • Agency agreed to establish Performance Measures, tracking the fellows after completion of the program, EPA prepared an action plan, including action officials and due dates for each recommendation. However, EPA did not
Page 81 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel

  • the fellows selected, and to adjust outreach efforts accordingly”.

  • Improve internal evaluation, performance measures, and applicant data.

agree with comments on diversity and need for outreach. ORD agreed to conduct internal evaluations, establish performance measures, and collect data.
Office of Research and Development’s Board of Scientific Councilors. 2009. Review of ORD’s National Center for Environmental Research (NCER) Letter Report. Programmatic review Here, the focus was narrow and on measuring the results of EPA’s greater research opportunities (GRO) fellowship program.
  • BOSC recommended that EPA consider eliminating the GRO fellowship programs (graduate and undergraduate).
  • EPA responded by eliminating the GRO graduate program but continuing the undergraduate GRO fellowships.
EPA’s Science Advisory Board and the Office of Research and Development’s Board of Scientific Councilors. 2011. Office of Research and Development (ORD) New Strategic Research Directions. Washington, DC. Programmatic review This report reviewed ORD as a whole following ORD’s 2010 reorganization. It recommended that ORD incorporate the principles of sustainability into the six newly named research areas and that social, behavioral, and decision sciences be emphasized within ORD/support ORD’s technical and innovation goals.
  • No responses specific to the STAR program
EPA’s Science Advisory Board and the Office of Research and Development’s Board of Scientific Councilors. 2012. Implementation of ORD Strategic Research Plans. Programmatic Review This report reviewed ORD as a whole, and recommended that ORD:
  • Provide a comprehensive mapping of project to goals for the six new program areas.
  • Balance immediate program needs and emerging issues through a “structured approach”
  • No responses specific to the STAR program
U.S. Environmental Protection Agency. 2013. EPA Needs to Improve STAR Grant Oversight. Edited by Office of the Inspector General. Washington, DC. Programmatic Review Reviewed the STAR grant oversight process and recommend that EPA:
  • Improve training of project officers to improve baseline monitoring, ensure that reports are accurately completed, enforcing the terms and conditions that allow funds to be withheld, if reports are missing/late.
  • Improve accounting of costs and budgets as well as improve project officer’s understanding of research misconduct reporting requirements.
  • ORD will provide all STAR grant project officers with training on the performance of baseline monitoring.
  • ORD drafted standard operating procedures for steps to be taken when annual progress reports are late, including when to pursue withholding grant funds
  • ORD was to revise its guidance pertaining to annual report reviews and publications with proper acknowledgements and disclaimers. ORD was to include a condition that payments may be withheld when reports are missing or late.
Page 82 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel
Review Type of Review Brief summary EPA’s Responses
(EPA’s Science Advisory Board and the Office of Research and Development’s Board of Scientific Councilors 2016) Programmatic Review This review was of EPA ORD’s entire research program. Recommendations include that EPA should:
  • Develop measures of success for outputs and outcomes for each national program.
  • Further develop and enhance efforts in research synthesis and translation. The Agency might benefit from identifying or training the appropriate people best suited to synthesize or translate research work and provides rewards and incentives for doing so as translational work does not necessarily lend itself to peer-reviewed publications, yet the benefits for policy makers and the public can be substantial.
  • Continue to nurture and expand cross-program and transdisciplinary integration to increase efficiencies and synergies.
  • Maintain alignment between research that is focused on short-term goals and long term objectives.
U.S. Environmental Protection Agency. OIG 2016. EPA Offices Are Aware of the Agency’s Science to Achieve Results Program, but Challenges Remain in Measuring an Internally Communicating Research Results That Advance the Agency’s Mission. Edited by Office of the Inspector General. Washington, DC. Programmatic Review This review recommended that EPA:
  1. ORD NCER should develop and implement SOPs to improve internal communications under ORD’s new matrix structure of STAR grant research results to EPA program and regional offices. The procedures should
    1. Ensure that the STAR grant public website is up to date.
    2. Revise the NCER Project Officer Manual (or develop a more dynamic tool) for communicating grant results.
    3. Clarify and define roles and responsibilities for communicating research results.
  2. Create procedures for developing RFAs to ensure program office input is considered in the RFA development process.
  1. Develop and implement procedures to improve communications with the EPA’s program offices regarding STAR research results.
    1. NCER will establish an SOP to assure that STAR grant updates are provided in a timely manner, as well as a method for identifying missing reports.
    2. NCER will coordinate and work with all involved staff leads (NCER, ORD, NPD) including communications, MIs, and POs to identify best practices to fulfill needs for communicating grant results and developing an SOP for grant research results communications
Page 83 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel
  1. Create procedures for conducting the relevancy reviews to ensure that program office input is more consistently and transparently considered in the grant selection process (to the extent permitted by the FGCAA and EPA Order 5700.1).
  2. Develop goals and objectives for the STAR program 5. Establish a means to capture and report out on how completed STAR grants have met their performance goals and provide incidental research support to program offices.
  1. Working with ORD NPDs, NCER will update the current written SOP to formalize the current standard practice of RFA development that includes program and regional office input and assistance.
  2. NCER is finalizing a written SOP for its relevancy reviews that includes information regarding how relevancy review information is to be incorporated into the grant selection process. The SOP will provide guidance on information to be routinely shared with reviewers including limitations on the use of grants per FGCAA and EPA Order 5700.1, as well as explaining how relevance review results will be incorporated into the grant selection process.
  3. NCER will clarify the goals and objectives for the STAR grants that can be consistently used for various audiences.
  4. NCER, in collaboration with the NPDs, will establish a new SOP (including a communications plan) for documenting ORD L/C/O, program office, and regional office participation in the identification of RFA topics (and funding decisions) to assist EPA in advancing its mission; how individual grants are expected to fulfill the purpose of the RFA; and ultimately presenting how the funded grants met the RFA and individual grant performance measures.
Page 79 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel
Page 79
Page 80 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel
Page 80
Page 81 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel
Page 81
Page 82 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel
Page 82
Page 83 Cite
Suggested Citation:"Appendix B: Summary of the Previous Reviews of the Science to Achieve Results Program." National Academies of Sciences, Engineering, and Medicine. 2017. A Review of the Environmental Protection Agency's Science to Achieve Results Research Program. Washington, DC: The National Academies Press. doi: 10.17226/24757.
×
Save
Cancel
Page 83
Next: Appendix C: Assignment of Science to Achieve Results Request for Applications to Scientific Domains »
A Review of the Environmental Protection Agency's Science to Achieve Results Research Program Get This Book
×
A Review of the Environmental Protection Agency's Science to Achieve Results Research Program
Buy Paperback | $60.00 Buy Ebook | $48.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Environmental research has driven landmark improvements that led to the protection of human and ecosystem health. Recognizing the value of knowledge generated by environmental research and the ingenuity within academic and nonprofit institutions, the US Environmental Protection Agency (EPA) created a program known as Science to Achieve Results, or STAR, in 1995. STAR is EPA’s primary competitive extramural grants program.

A Review of the Environmental Protection Agency’s Science to Achieve Results Research Program assesses the program’s scientific merit, public benefits, and overall contributions in the context of other relevant research and recommends ways to enhance those aspects of the program. This report also considers the conclusions and recommendations of a prior National Research Council review of the STAR program (2003), the STAR program’s research priorities in light of the nation’s environmental challenges, and the effects of recent STAR funding trends on obtaining scientific information needed to protect public health and the environment.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!