National Academies Press: OpenBook
« Previous: Appendix D: Acknowledgments
Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×

Appendix E

References

Bartlett, A.G. 2016. Evaluating relative success of donor-funded collaborative research projects. Research Evaluation 25(4):405-415. https://doi.org/10.1093/reseval/rvw009.

Black, N. 2001. Evidence based policy: Proceed with care. British Medical Journal (Clinical research ed.) 323(7307):275-279. https://doi.org/10.1136/bmj.323.7307.275.

BOEM (Bureau of Ocean Energy Management). 2020a. Environmental Studies Program Strategic Framework. https://www.boem.gov/sites/default/files/documents/about-boem/ESP-StrategicFramework-Final-FY20.pdf.

BOEM. 2020b. America’s Offshore Critical Mineral Resources. https://www.boem.gov/sites/default/files/documents/marine-minerals/Critical-Mineral-State.pdf.

Booth, A. 2004. Formulating answerable questions. In Evidence-Based Practice for Information Professional’s: A Handbook. Edited by A. Booth and A. Brice. London, UK: Facet Publishing.

Booth, A., A. Sutton, and D. Papaioannou. 2016. Systematic Approaches to a Successful Literature Review. 2nd edition. SAGE Publications Ltd.

Bornmann, L., R. Mutz, C. Neuhaus, and H.-D. Daniel. 2008. Citation counts for research evaluation: Standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics 8:93-102. https://doi.org/10.3354/esep00084.

Bridle, H., A. Vrieling, M. Cardillo, Y. Araya, and L. Hinojosa. 2013. Preparing for an interdisciplinary future: A perspective from early-career researchers. Futures 53:22-32. https://doi.org/10.1016/j.futures.2013.09.003.

Brownson, R.C., J.F. Chriqui, and K.A. Stamatakis. 2009. Understanding evidence-based public health policy. American Journal of Public Health 99(9):1576-1583. https://doi.org/10.2105/AJPH.2008.156224.

Carpenter, S.R., E.V. Armbrust, P.W. Arzberger, F.S. Chapin III, J.J. Elser, E.J. Hackett, A.R. Ives, P.M. Kareiva, M.A. Leibold, P. Lundberg, M. Mangel, N. Merchant, W.W. Murdoch, M.A. Palmer, D.P.C. Peters, S.T.A. Pickett, K.K. Smith, D.H. Wall, and A.S. Zimmerman. 2009. Accelerate synthesis in ecology and environmental sciences. BioScience 59(8):699-701. https://doi.org/10.1525/bio.2009.59.8.11.

Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×

CDC (Centers for Disease Control and Prevention). 2011. Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation. https://www.cdc.gov/eval/guide/CDCEvalManual.pdf.

Coglianese, C. 2012. Measuring regulatory performance: Evaluating the impact of regulation and regulatory policy. Expert paper no. 1. Organisation for Economic Co-operation and Development. Paris, France. https://www.oecd.org/gov/regulatory-policy/1_coglianese%20web.pdf.

Cooke, A., D. Smith, and A. Booth. 2012. Beyond PICO: The SPIDER tool for qualitative evidence synthesis. Qualitative Health Research 22(10):1435-1443.

Dale, B.G. 2003. TQM: An overview. Managing Quality. 4th edition. Edited by B.G. Dale. Malden, MA: Blackwell Publishing, Limited.

Davies, K.S. 2011. Formulating the evidence based practice question: A review of the frameworks. Evidence Based Library and Information Practice 6(2):75-80.

Davis, S., J. Roberts, J. Thomas, and H. Shinn. 2019. A Bibliometric Analysis of Articles by NOAA-Affiliated and Funded Authors Published During Fiscal Year 2018. https://library.noaa.gov/Portals/1/Bibliometrics/FY2018%20Annual%20Report%20-%20Author%20and%20Funded.pdf?ver=2019-09-13-151836-963.

Denyer, D., and D. Tranfield. 2009. Producing a systematic review. In The Sage Handbook of Organizational Research Methods. Edited by D.A. Buchanan and A. Bryman. London, UK: Sage Publishers.

DOI (U.S. Department of the Interior). 2019. 2020–2021 Annual Performance Plan and 2019 Report (APP&R). https://www.doi.gov/sites/doi.gov/files/doi-fy19-appr-08302021-.pdf.

DOI. 2021. U.S. Department of the Interior Environmental Justice Annual Implementation Report Fiscal Year 2020. https://www.doi.gov/sites/doi.gov/files/fy2020-ej-annual-report-aug-2021-508c_0.pdf.

Drew, C.H., K.G. Pettibone, F.O. Finch III, D. Giles, and P. Jordan. 2016. Automated research impact assessment: A new bibliometrics approach. Scientometrics 106(3):987-1005. https://doi.org/10.1007/s11192-015-1828-7.

EPA (U.S. Environmental Protection Agency). 2011. Improving Our Regulations: Final Plan for Periodic Retrospective Reviews of Existing Regulations. https://obamawhitehouse.archives.gov/sites/default/files/other/2011-regulatory-action-plans/environmentalprotectionagencyregulatoryreformplanaugust2011.pdf.

EPA. 2015. Guidance on Considering Environmental Justice During Development of Regulatory Actions. https://www.epa.gov/sites/default/files/2015-06/documents/considering-ej-in-rulemakingguide-final.pdf.

EPA. 2020. Air and Energy Strategic Research Action Plan 2019–2022. https://www.epa.gov/research/air-and-energy-strategic-research-action-plan-2019-2022.

EPA. 2021. Learn About Environmental Justice. https://www.epa.gov/environmentaljustice/learnabout-environmental-justice.

ERDC (U.S. Army Engineering Research and Development Center). 2020. Power of ERDC: 2020–2030 Strategy. https://www.erdc.usace.army.mil/Media/Publication-Notices/Article/2604551/the-power-of-erdc-erdc-20202030-strategy.

Eysenbach, G. 2006. Citation advantage of open access articles. PLoS Biology 4:e157. https://doi.org/10.1371/journal.pbio.0040157.

FDA (U.S. Food and Drug Administration). 2018. CERSI Research Impact Metrics. https://www.fda.gov/science-research/advancing-regulatory-science/cersi-research-impact-metrics.

Ferguson, D.B., J. Rice, and C.A. Woodhouse. 2014. Linking Environmental Research and Practice: Lessons from the Integration of Climate Science and Water Management in the Western United States. Tucson, AZ: University of Arizona, Climate Assessment for the Southwest.

GAO (U.S. Government Accountability Office). 2011. Performance Measurement and Evaluation: Definition and Relationships. https://www.gao.gov/assets/gao-11-646sp.pdf.

Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×

Gargouri, Y., C. Hajjem, V. Larivière, Y. Gingras, L. Carr, T. Brody, and S. Harnad. 2010. Self-selected or mandated, open access increases citation impact for higher quality research. PLoS One 5:e13636. https://doi.org/10.1371/journal.pone.0013636.

Hahn, T., P. Olsson, C. Folke, and K. Johansson. 2006. Trust-building, knowledge generation and organizational innovations: The role of a bridging organization for adaptive comanagement of a wetland landscape around Kristianstad, Sweden. Human Ecology 34(4):573-592. https://doi.org/10.1007/s10745-006-9035-z.

Hajjem, C., S. Harnad, and Y. Gingras. 2006. Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. arXiv:cs/0606079.

Henry, A.D., and T. Dietz. 2011. Information, networks, and the complexity of trust in commons governance. International Journal of the Commons 5(2):188. http://doi.org/10.18352/ijc.312.

Hicks, D., H. Tomizawa, Y. Saitoh, and S. Kobayashi. 2004. Bibliometric techniques in the evaluation of federally funded research in the United States. Research Evaluation 13(2):78-86. https://doi.org/10.3152/147154404781776446.

Hong, L., and S.E. Page. 2004. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proceedings of the National Academy of Sciences of the United States of America 101(46):16385-16389. https://doi.org/10.1073/pnas.0403723101.

Horwitz, S.K., and I.B. Horwitz. 2007. The effects of team diversity on team outcomes: A meta-analytic review of team demography. Journal of Management 33:987-1015. https://doi.org/10.1177/0149206307308587.

Howell, E., and A. Yemane. 2006. An assessment of evaluation designs. American Journal of Evaluation 27:219-236.

Hutchins, B.I., X. Yuan, J.M. Anderson, and G.M. Santangelo. 2016. Relative citation ratio (RCR): A new metric that uses citation rates to measure influence at the article level. PLoS Biology 14(9):e1002541. https://doi.org/10.1371/journal.pbio.1002541.

Industrial Economics, Inc. 2020. Evaluating Connections: BOEM’s Environmental Studies and Assessments Evaluation Methodology. https://www.boem.gov/sites/default/files/documents/environment/Evaluating%20Connections%20Methodology%20Report%20-%20Final_0.pdf.

Jacobs, K. 2002. Connecting Science, Policy, and Decision-Making: A Handbook for Researchers and Science Agencies. Silver Spring: MD: Office of Global Programs, Oceanic and Atmospheric Research, National Oceanic and Atmospheric Administration.

Kiker, G.A., T.S. Bridges, A. Varghese, T.P. Seager, and I. Linkov. 2009. Application of multicriteria decision analysis in environmental decision making. Integrated Environmental Assessment and Management 1(2):95-108. https://doi.org/10.1897/IEAM_2004a-015.1.

Koricheva, J., J. Gurevitch, and K. Mengersen. 2013. Handbook of Meta-analysis in Ecology and Evolution. Princeton, NJ: Princeton University Press.

Makkar, S., S. Brennan, T. Turner, A. Williamson, S. Redman, and S. Green. 2016. The development of SAGE: A tool to evaluate how policymakers’ engage with and use research in health policy-making. Research Evaluation 25(3):315-328. https://doi.org/10.1093/reseval/rvv044.

Mascia, M.B., and M. Mills. 2018. When conservation goes viral: The diffusion of innovative biodiversity conservation policies and practices. Conservation Letters 11(3):e12442. https://doi.org/10.1111/conl.12442.

Matt, G.E., and T.D. Cook. 2009. Threats to the validity of generalized inferences. In The Handbook of Research Synthesis and Meta-analysis. Edited by H. Cooper, L.V. Hedges, and J.C. Valentine. New York: Russell Sage Foundation.

McKinsey & Company. 2020. Diversity Wins: How Inclusion Matters. https://www.mckinsey.com/~/media/mckinsey/featured%20insights/diversity%20and%20inclusion/diversity%20wins%20how%20inclusion%20matters/diversity-wins-how-inclusion-matters-vf.pdf.

Meagher, L.R., and U. Martin. 2017. Slightly dirty maths: The richly textured mechanisms of impact. Research Evaluation 26(1):15-27. https://doi.org/10.1093/reseval/rvw024.

Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×

Medin, D.L., and C.D. Lee. 2012. Diversity makes better science. Association for Psychological Science-APS. https://www.psychologicalscience.org/observer/diversity-makes-better-science.

Morton, S. 2015. Progressing research impact assessment: A “contributions” approach. Research Evaluation 24(4):405-419. https://doi.org/10.1093/reseval/rvv016.

NOAA (National Oceanic and Atmospheric Administration). 2020a. Diversity and Inclusion Strategic Plan Fiscal Years 2020–2024. https://www.noaa.gov/sites/default/files/legacy/document/2020/Dec/NOAA%202020-2024%20Diversity%20and%20Inclusion%20Strategic%20Plan.pdf.

NOAA. 2020b. NOAA Research and Development Vision Areas: 2020–2026. https://repository.library.noaa.gov/view/noaa/24933.

NOAA. 2021a. 2020 NOAA Science Report. https://sciencecouncil.noaa.gov/Portals/0/2020%20NOAA%20Science%20Report%20-%20FinalApproved%20-%20508Compliance-IRsubmission.pdf?ver=2021-04-22-100336-233.

NOAA. 2021b. NOAA Climate and Fisheries Initiative Fact Sheet. https://www.fisheries.noaa.gov/resource/document/noaa-climate-and-fisheries-initiative-fact-sheet.

NRC (National Research Council). 1998. Systems Analysis and Systems Engineering in Environmental Remediation Programs at the Department of Energy Hanford Site. Washington, DC: National Academy Press. https://doi.org/10.17226/6224.

NSF (National Science Foundation). 2018. Building the Future: Investing in Discovery and Innovation—NSF strategic plan for fiscal years (FY) 2018–2022. National Science Foundation.

Nutley, S.M., I. Walter, and H.T.O. Davies. 2007. Using Evidence: How Research Can Inform Public Services. Bristol, UK: Policy Press.

Oreskes, N. 2019. Why Trust Science? Princeton, NJ: Princeton University Press.

Parkin, R.T. 2004. Communications with research participants and communities: Foundations for best practices. Journal of Exposure Science & Environmental Epidemiology 14(7):516-523. https://doi.org/10.1038/sj.jea.7500393.

Penfield, T., M.J. Baker, R. Scoble, and M.C. Wykes. 2014. Assessment, evaluations, and definitions of research impact: A review. Research Evaluation 23(1):21-32. https://doi.org/10.1093/reseval/rvt021.

Pew Research Center. 2020. Trust in Medical Scientists Has Grown in U.S., but Mainly Among Democrats. https://www.pewresearch.org/science/2020/05/21/trust-in-medical-scientists-hasgrown-in-u-s-but-mainly-among-democrats.

Pew Research Center. 2021. Public Trust in Government: 1958–2021. https://www.pewresearch.org/politics/2021/05/17/public-trust-in-government-1958-2021.

Phillips, K., and C. O’Reilly. 1998. Demography and diversity in organizations: A review of 40 years of research. Research in Organizational Behavior 20:77-140.

Richardson, W.S., M.C. Wilson, J. Nishikawa, and R.S.A. Hayward. 1995. The well-built clinical question: A key to evidence-based decisions. ACP Journal Club 123(3):A12-A13.

Rofougaran, N.L., and H.A. Karl. 2005. San Francisquito Creek—the Problem of Science in Environmental Disputes. Reston, VA: U.S. Geological Survey. https://pubs.usgs.gov/pp/pp1710/pp1710.pdf.

Rogers, E.M. 1962. Diffusion of Innovations. New York: Free Press of Glencoe.

Rossi, P.H., H.E. Freeman, and M.W. Lipsey. 1999. Evaluation: A Systematic Approach. 6th edition. Thousand Oaks, CA: SAGE Publications.

Rossi, P.H., M.W. Lipsey, and G.T. Henry. 2018. Evaluation: A Systematic Approach. 8th edition. Thousand Oaks, CA: SAGE Publications.

Rovai, A.P. 2003. A practical framework for evaluating online distance education programs. Internet and Higher Education 6(2):109-124.

Samuel, G.N., and G.E. Derrick. 2015. Societal impact evaluation: Exploring evaluator perceptions of the characterization of impact under the Ref2014. Research Evaluation 24(3): 229-241. https://doi.org/10.1093/reseval/rvv007.

Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×

Scoble, R., K. Dickson, J. Fisher, and S. Hanney 2010. Research Impact Evaluation, a Wider Context: Findings from a Research Impact Pilot. https://bura.brunel.ac.uk/handle/2438/4149.

Senge, P.M. 2014. The Fifth Discipline Fieldbook: Strategies and Tools for Building a Learning Organization. New York, NY: Currency, Doubleday.

Susskind, L., P. Field, G. Smith, T. Schenk, and M. Matsuura. 2016. Joint fact-finding: Process and practice. In Joint Fact Finding in Urban Planning and Environmental Disputes. Edited by M. Matsuura and T. Schenk. New York, NY: Routledge

Takeuchi, H., and J. Quelch. 1983. Quality is more than making a good product. Harvard Business Review 61(4):139-145.

Thompson, S.A., R.L. Stephenson, G.A. Rose, and S.D. Paul. 2019. Collaborative fisheries research: The Canadian Fisheries Research Network experience. Canadian Journal of Fisheries and Aquatic Sciences 76(5):671-681. https://doi.org/10.1139/cjfas-2018-0450.

Vidas, H., B. Hugman, A. Chikkatur, and B. Venkatesh. 2012. Analysis of the Costs and Benefits of CO2 Sequestration on the U.S. Outer Continental Shelf. Herndon, VA: Bureau of Ocean Energy Management, U.S. Department of the Interior. OCS Study BOEM 2012-100.

Wang, X., C. Liu, W. Mao, and Z. Fang. 2015. The open access advantage considering citation, article usage and social media attention. Scientometrics 103:555-564. https://doi.org/10.1007/s11192-015-1547-0.

Worthen, B.R., J.R. Sanders, and J.L. Fitzpatrick. 1997. Program Evaluation: Alternative Approaches and Practical Guidelines. White Plains, NY: Longman Publishers.

Zhan, S., N. Bendapudi, and Y.-y. Hong. 2015. Re-examining diversity as a double-edged sword for innovation process. Journal of Organizational Behavior 36(7):1026-1049. https://doi.org/10.1002/job.2027.

Zimmerman, L.A., M.G. Feldman, D.S. Benoit, M.J. Carron, N.M. Daanreuther, K.H. Fillingham, J.C. Gibeaut, J.L. Petit, J.B. Ritchie, R.R. Rossi, S.H. Sempier, J.K. Shaw, J.L. Swanseen, C.A. Wilson, C.J. Yanoff, and R.R. Colwell. 2021. From disaster to understanding: Formation and accomplishments of the Gulf of Mexico Research Initiative. Oceanography 34:16-29. https://doi.org/10.5670/oceanog.2021.114.

Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×
Page 62
Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×
Page 63
Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×
Page 64
Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×
Page 65
Suggested Citation:"Appendix E: References." National Academies of Sciences, Engineering, and Medicine. 2022. Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management. Washington, DC: The National Academies Press. doi: 10.17226/26368.
×
Page 66
Attributes of a First-in-Class Environmental Program: A Letter Report Prepared for the Bureau of Ocean Energy Management Get This Book
×
Buy Paperback | $40.00 Buy Ebook | $32.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Bureau of Ocean Energy Management (BOEM) manages the energy and mineral resources on the outer continental shelf. BOEM's environmental program, by producing environmental studies and conducting environmental assessments, ensures that environmental protection is a critical element of BOEM's decision making. This report addresses BOEM's aspirations to conduct a first-in-class science program within their Environmental Studies Program (ESP).

This report describes attributes identified by the committee of a first-in-class, use-inspired, management-oriented science program (in this case, BOEM's ESP and its connection to the broader BOEM environmental program). The report recommends that BOEM develop procedures and conduct regular evaluations to assess whether and how well its environmental program meets the attributes of a first-in-class program and identify areas for improvement. It also outlines a framework for conducting such an evaluation. The report contains guidance and examples drawn from a workshop series with BOEM's peer agencies and other science programs, as well as other information gathering efforts. The guidance contained in this report is offered to BOEM as a starting point for developing more detailed processes for evaluating and improving its program.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!