Appendix E
References
Bartlett, A.G. 2016. Evaluating relative success of donor-funded collaborative research projects. Research Evaluation 25(4):405-415. https://doi.org/10.1093/reseval/rvw009.
Black, N. 2001. Evidence based policy: Proceed with care. British Medical Journal (Clinical research ed.) 323(7307):275-279. https://doi.org/10.1136/bmj.323.7307.275.
BOEM (Bureau of Ocean Energy Management). 2020a. Environmental Studies Program Strategic Framework. https://www.boem.gov/sites/default/files/documents/about-boem/ESP-StrategicFramework-Final-FY20.pdf.
BOEM. 2020b. America’s Offshore Critical Mineral Resources. https://www.boem.gov/sites/default/files/documents/marine-minerals/Critical-Mineral-State.pdf.
Booth, A. 2004. Formulating answerable questions. In Evidence-Based Practice for Information Professional’s: A Handbook. Edited by A. Booth and A. Brice. London, UK: Facet Publishing.
Booth, A., A. Sutton, and D. Papaioannou. 2016. Systematic Approaches to a Successful Literature Review. 2nd edition. SAGE Publications Ltd.
Bornmann, L., R. Mutz, C. Neuhaus, and H.-D. Daniel. 2008. Citation counts for research evaluation: Standards of good practice for analyzing bibliometric data and presenting and interpreting results. Ethics in Science and Environmental Politics 8:93-102. https://doi.org/10.3354/esep00084.
Bridle, H., A. Vrieling, M. Cardillo, Y. Araya, and L. Hinojosa. 2013. Preparing for an interdisciplinary future: A perspective from early-career researchers. Futures 53:22-32. https://doi.org/10.1016/j.futures.2013.09.003.
Brownson, R.C., J.F. Chriqui, and K.A. Stamatakis. 2009. Understanding evidence-based public health policy. American Journal of Public Health 99(9):1576-1583. https://doi.org/10.2105/AJPH.2008.156224.
Carpenter, S.R., E.V. Armbrust, P.W. Arzberger, F.S. Chapin III, J.J. Elser, E.J. Hackett, A.R. Ives, P.M. Kareiva, M.A. Leibold, P. Lundberg, M. Mangel, N. Merchant, W.W. Murdoch, M.A. Palmer, D.P.C. Peters, S.T.A. Pickett, K.K. Smith, D.H. Wall, and A.S. Zimmerman. 2009. Accelerate synthesis in ecology and environmental sciences. BioScience 59(8):699-701. https://doi.org/10.1525/bio.2009.59.8.11.
CDC (Centers for Disease Control and Prevention). 2011. Introduction to Program Evaluation for Public Health Programs: A Self-Study Guide. Atlanta, GA: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Office of the Director, Office of Strategy and Innovation. https://www.cdc.gov/eval/guide/CDCEvalManual.pdf.
Coglianese, C. 2012. Measuring regulatory performance: Evaluating the impact of regulation and regulatory policy. Expert paper no. 1. Organisation for Economic Co-operation and Development. Paris, France. https://www.oecd.org/gov/regulatory-policy/1_coglianese%20web.pdf.
Cooke, A., D. Smith, and A. Booth. 2012. Beyond PICO: The SPIDER tool for qualitative evidence synthesis. Qualitative Health Research 22(10):1435-1443.
Dale, B.G. 2003. TQM: An overview. Managing Quality. 4th edition. Edited by B.G. Dale. Malden, MA: Blackwell Publishing, Limited.
Davies, K.S. 2011. Formulating the evidence based practice question: A review of the frameworks. Evidence Based Library and Information Practice 6(2):75-80.
Davis, S., J. Roberts, J. Thomas, and H. Shinn. 2019. A Bibliometric Analysis of Articles by NOAA-Affiliated and Funded Authors Published During Fiscal Year 2018. https://library.noaa.gov/Portals/1/Bibliometrics/FY2018%20Annual%20Report%20-%20Author%20and%20Funded.pdf?ver=2019-09-13-151836-963.
Denyer, D., and D. Tranfield. 2009. Producing a systematic review. In The Sage Handbook of Organizational Research Methods. Edited by D.A. Buchanan and A. Bryman. London, UK: Sage Publishers.
DOI (U.S. Department of the Interior). 2019. 2020–2021 Annual Performance Plan and 2019 Report (APP&R). https://www.doi.gov/sites/doi.gov/files/doi-fy19-appr-08302021-.pdf.
DOI. 2021. U.S. Department of the Interior Environmental Justice Annual Implementation Report Fiscal Year 2020. https://www.doi.gov/sites/doi.gov/files/fy2020-ej-annual-report-aug-2021-508c_0.pdf.
Drew, C.H., K.G. Pettibone, F.O. Finch III, D. Giles, and P. Jordan. 2016. Automated research impact assessment: A new bibliometrics approach. Scientometrics 106(3):987-1005. https://doi.org/10.1007/s11192-015-1828-7.
EPA (U.S. Environmental Protection Agency). 2011. Improving Our Regulations: Final Plan for Periodic Retrospective Reviews of Existing Regulations. https://obamawhitehouse.archives.gov/sites/default/files/other/2011-regulatory-action-plans/environmentalprotectionagencyregulatoryreformplanaugust2011.pdf.
EPA. 2015. Guidance on Considering Environmental Justice During Development of Regulatory Actions. https://www.epa.gov/sites/default/files/2015-06/documents/considering-ej-in-rulemakingguide-final.pdf.
EPA. 2020. Air and Energy Strategic Research Action Plan 2019–2022. https://www.epa.gov/research/air-and-energy-strategic-research-action-plan-2019-2022.
EPA. 2021. Learn About Environmental Justice. https://www.epa.gov/environmentaljustice/learnabout-environmental-justice.
ERDC (U.S. Army Engineering Research and Development Center). 2020. Power of ERDC: 2020–2030 Strategy. https://www.erdc.usace.army.mil/Media/Publication-Notices/Article/2604551/the-power-of-erdc-erdc-20202030-strategy.
Eysenbach, G. 2006. Citation advantage of open access articles. PLoS Biology 4:e157. https://doi.org/10.1371/journal.pbio.0040157.
FDA (U.S. Food and Drug Administration). 2018. CERSI Research Impact Metrics. https://www.fda.gov/science-research/advancing-regulatory-science/cersi-research-impact-metrics.
Ferguson, D.B., J. Rice, and C.A. Woodhouse. 2014. Linking Environmental Research and Practice: Lessons from the Integration of Climate Science and Water Management in the Western United States. Tucson, AZ: University of Arizona, Climate Assessment for the Southwest.
GAO (U.S. Government Accountability Office). 2011. Performance Measurement and Evaluation: Definition and Relationships. https://www.gao.gov/assets/gao-11-646sp.pdf.
Gargouri, Y., C. Hajjem, V. Larivière, Y. Gingras, L. Carr, T. Brody, and S. Harnad. 2010. Self-selected or mandated, open access increases citation impact for higher quality research. PLoS One 5:e13636. https://doi.org/10.1371/journal.pone.0013636.
Hahn, T., P. Olsson, C. Folke, and K. Johansson. 2006. Trust-building, knowledge generation and organizational innovations: The role of a bridging organization for adaptive comanagement of a wetland landscape around Kristianstad, Sweden. Human Ecology 34(4):573-592. https://doi.org/10.1007/s10745-006-9035-z.
Hajjem, C., S. Harnad, and Y. Gingras. 2006. Ten-year cross-disciplinary comparison of the growth of open access and how it increases research citation impact. arXiv:cs/0606079.
Henry, A.D., and T. Dietz. 2011. Information, networks, and the complexity of trust in commons governance. International Journal of the Commons 5(2):188. http://doi.org/10.18352/ijc.312.
Hicks, D., H. Tomizawa, Y. Saitoh, and S. Kobayashi. 2004. Bibliometric techniques in the evaluation of federally funded research in the United States. Research Evaluation 13(2):78-86. https://doi.org/10.3152/147154404781776446.
Hong, L., and S.E. Page. 2004. Groups of diverse problem solvers can outperform groups of high-ability problem solvers. Proceedings of the National Academy of Sciences of the United States of America 101(46):16385-16389. https://doi.org/10.1073/pnas.0403723101.
Horwitz, S.K., and I.B. Horwitz. 2007. The effects of team diversity on team outcomes: A meta-analytic review of team demography. Journal of Management 33:987-1015. https://doi.org/10.1177/0149206307308587.
Howell, E., and A. Yemane. 2006. An assessment of evaluation designs. American Journal of Evaluation 27:219-236.
Hutchins, B.I., X. Yuan, J.M. Anderson, and G.M. Santangelo. 2016. Relative citation ratio (RCR): A new metric that uses citation rates to measure influence at the article level. PLoS Biology 14(9):e1002541. https://doi.org/10.1371/journal.pbio.1002541.
Industrial Economics, Inc. 2020. Evaluating Connections: BOEM’s Environmental Studies and Assessments Evaluation Methodology. https://www.boem.gov/sites/default/files/documents/environment/Evaluating%20Connections%20Methodology%20Report%20-%20Final_0.pdf.
Jacobs, K. 2002. Connecting Science, Policy, and Decision-Making: A Handbook for Researchers and Science Agencies. Silver Spring: MD: Office of Global Programs, Oceanic and Atmospheric Research, National Oceanic and Atmospheric Administration.
Kiker, G.A., T.S. Bridges, A. Varghese, T.P. Seager, and I. Linkov. 2009. Application of multicriteria decision analysis in environmental decision making. Integrated Environmental Assessment and Management 1(2):95-108. https://doi.org/10.1897/IEAM_2004a-015.1.
Koricheva, J., J. Gurevitch, and K. Mengersen. 2013. Handbook of Meta-analysis in Ecology and Evolution. Princeton, NJ: Princeton University Press.
Makkar, S., S. Brennan, T. Turner, A. Williamson, S. Redman, and S. Green. 2016. The development of SAGE: A tool to evaluate how policymakers’ engage with and use research in health policy-making. Research Evaluation 25(3):315-328. https://doi.org/10.1093/reseval/rvv044.
Mascia, M.B., and M. Mills. 2018. When conservation goes viral: The diffusion of innovative biodiversity conservation policies and practices. Conservation Letters 11(3):e12442. https://doi.org/10.1111/conl.12442.
Matt, G.E., and T.D. Cook. 2009. Threats to the validity of generalized inferences. In The Handbook of Research Synthesis and Meta-analysis. Edited by H. Cooper, L.V. Hedges, and J.C. Valentine. New York: Russell Sage Foundation.
McKinsey & Company. 2020. Diversity Wins: How Inclusion Matters. https://www.mckinsey.com/~/media/mckinsey/featured%20insights/diversity%20and%20inclusion/diversity%20wins%20how%20inclusion%20matters/diversity-wins-how-inclusion-matters-vf.pdf.
Meagher, L.R., and U. Martin. 2017. Slightly dirty maths: The richly textured mechanisms of impact. Research Evaluation 26(1):15-27. https://doi.org/10.1093/reseval/rvw024.
Medin, D.L., and C.D. Lee. 2012. Diversity makes better science. Association for Psychological Science-APS. https://www.psychologicalscience.org/observer/diversity-makes-better-science.
Morton, S. 2015. Progressing research impact assessment: A “contributions” approach. Research Evaluation 24(4):405-419. https://doi.org/10.1093/reseval/rvv016.
NOAA (National Oceanic and Atmospheric Administration). 2020a. Diversity and Inclusion Strategic Plan Fiscal Years 2020–2024. https://www.noaa.gov/sites/default/files/legacy/document/2020/Dec/NOAA%202020-2024%20Diversity%20and%20Inclusion%20Strategic%20Plan.pdf.
NOAA. 2020b. NOAA Research and Development Vision Areas: 2020–2026. https://repository.library.noaa.gov/view/noaa/24933.
NOAA. 2021a. 2020 NOAA Science Report. https://sciencecouncil.noaa.gov/Portals/0/2020%20NOAA%20Science%20Report%20-%20FinalApproved%20-%20508Compliance-IRsubmission.pdf?ver=2021-04-22-100336-233.
NOAA. 2021b. NOAA Climate and Fisheries Initiative Fact Sheet. https://www.fisheries.noaa.gov/resource/document/noaa-climate-and-fisheries-initiative-fact-sheet.
NRC (National Research Council). 1998. Systems Analysis and Systems Engineering in Environmental Remediation Programs at the Department of Energy Hanford Site. Washington, DC: National Academy Press. https://doi.org/10.17226/6224.
NSF (National Science Foundation). 2018. Building the Future: Investing in Discovery and Innovation—NSF strategic plan for fiscal years (FY) 2018–2022. National Science Foundation.
Nutley, S.M., I. Walter, and H.T.O. Davies. 2007. Using Evidence: How Research Can Inform Public Services. Bristol, UK: Policy Press.
Oreskes, N. 2019. Why Trust Science? Princeton, NJ: Princeton University Press.
Parkin, R.T. 2004. Communications with research participants and communities: Foundations for best practices. Journal of Exposure Science & Environmental Epidemiology 14(7):516-523. https://doi.org/10.1038/sj.jea.7500393.
Penfield, T., M.J. Baker, R. Scoble, and M.C. Wykes. 2014. Assessment, evaluations, and definitions of research impact: A review. Research Evaluation 23(1):21-32. https://doi.org/10.1093/reseval/rvt021.
Pew Research Center. 2020. Trust in Medical Scientists Has Grown in U.S., but Mainly Among Democrats. https://www.pewresearch.org/science/2020/05/21/trust-in-medical-scientists-hasgrown-in-u-s-but-mainly-among-democrats.
Pew Research Center. 2021. Public Trust in Government: 1958–2021. https://www.pewresearch.org/politics/2021/05/17/public-trust-in-government-1958-2021.
Phillips, K., and C. O’Reilly. 1998. Demography and diversity in organizations: A review of 40 years of research. Research in Organizational Behavior 20:77-140.
Richardson, W.S., M.C. Wilson, J. Nishikawa, and R.S.A. Hayward. 1995. The well-built clinical question: A key to evidence-based decisions. ACP Journal Club 123(3):A12-A13.
Rofougaran, N.L., and H.A. Karl. 2005. San Francisquito Creek—the Problem of Science in Environmental Disputes. Reston, VA: U.S. Geological Survey. https://pubs.usgs.gov/pp/pp1710/pp1710.pdf.
Rogers, E.M. 1962. Diffusion of Innovations. New York: Free Press of Glencoe.
Rossi, P.H., H.E. Freeman, and M.W. Lipsey. 1999. Evaluation: A Systematic Approach. 6th edition. Thousand Oaks, CA: SAGE Publications.
Rossi, P.H., M.W. Lipsey, and G.T. Henry. 2018. Evaluation: A Systematic Approach. 8th edition. Thousand Oaks, CA: SAGE Publications.
Rovai, A.P. 2003. A practical framework for evaluating online distance education programs. Internet and Higher Education 6(2):109-124.
Samuel, G.N., and G.E. Derrick. 2015. Societal impact evaluation: Exploring evaluator perceptions of the characterization of impact under the Ref2014. Research Evaluation 24(3): 229-241. https://doi.org/10.1093/reseval/rvv007.
Scoble, R., K. Dickson, J. Fisher, and S. Hanney 2010. Research Impact Evaluation, a Wider Context: Findings from a Research Impact Pilot. https://bura.brunel.ac.uk/handle/2438/4149.
Senge, P.M. 2014. The Fifth Discipline Fieldbook: Strategies and Tools for Building a Learning Organization. New York, NY: Currency, Doubleday.
Susskind, L., P. Field, G. Smith, T. Schenk, and M. Matsuura. 2016. Joint fact-finding: Process and practice. In Joint Fact Finding in Urban Planning and Environmental Disputes. Edited by M. Matsuura and T. Schenk. New York, NY: Routledge
Takeuchi, H., and J. Quelch. 1983. Quality is more than making a good product. Harvard Business Review 61(4):139-145.
Thompson, S.A., R.L. Stephenson, G.A. Rose, and S.D. Paul. 2019. Collaborative fisheries research: The Canadian Fisheries Research Network experience. Canadian Journal of Fisheries and Aquatic Sciences 76(5):671-681. https://doi.org/10.1139/cjfas-2018-0450.
Vidas, H., B. Hugman, A. Chikkatur, and B. Venkatesh. 2012. Analysis of the Costs and Benefits of CO2 Sequestration on the U.S. Outer Continental Shelf. Herndon, VA: Bureau of Ocean Energy Management, U.S. Department of the Interior. OCS Study BOEM 2012-100.
Wang, X., C. Liu, W. Mao, and Z. Fang. 2015. The open access advantage considering citation, article usage and social media attention. Scientometrics 103:555-564. https://doi.org/10.1007/s11192-015-1547-0.
Worthen, B.R., J.R. Sanders, and J.L. Fitzpatrick. 1997. Program Evaluation: Alternative Approaches and Practical Guidelines. White Plains, NY: Longman Publishers.
Zhan, S., N. Bendapudi, and Y.-y. Hong. 2015. Re-examining diversity as a double-edged sword for innovation process. Journal of Organizational Behavior 36(7):1026-1049. https://doi.org/10.1002/job.2027.
Zimmerman, L.A., M.G. Feldman, D.S. Benoit, M.J. Carron, N.M. Daanreuther, K.H. Fillingham, J.C. Gibeaut, J.L. Petit, J.B. Ritchie, R.R. Rossi, S.H. Sempier, J.K. Shaw, J.L. Swanseen, C.A. Wilson, C.J. Yanoff, and R.R. Colwell. 2021. From disaster to understanding: Formation and accomplishments of the Gulf of Mexico Research Initiative. Oceanography 34:16-29. https://doi.org/10.5670/oceanog.2021.114.