Developing a Toolkit for Fostering
OPEN SCIENCE PRACTICES
PROCEEDINGS OF A WORKSHOP
Thomas Arrison, Jennifer Saunders,
and Emi Kameyama, Rapporteurs
Committee on Developing a Toolkit for Fostering
Open Science Practices: A Workshop
Board on Research Data and Information
Policy and Global Affairs
THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This workshop activity was supported by the Arcadia Fund under award number 4186, Open Research Funders Group, Open Society Foundations under award number OR2018-45885, Robert Wood Johnson Foundation under award number 78673, and the Wellcome Trust. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-09361-3
International Standard Book Number-10: 0-309-09361-9
Digital Object Identifier: https://doi.org/10.17226/26308
Additional copies of this publication are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2021 by the National Academy of Sciences.
This work is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). To view a copy of the license, visit https://creativecommons.org/licenses/by/4.0/.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2021. Developing a Toolkit for Fostering Open Science Practices: Proceedings of a Workshop. Washington, DC: The National Academies Press. https://doi.org/10.17226/26308.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.
Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.
For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.
PLANNING COMMITTEE ON DEVELOPING A TOOLKIT FOR FOSTERING OPEN SCIENCE PRACTICES
Loretta Parham (Chair), Chief Executive Officer and Library Director, Robert W. Woodruff Library, Atlanta University Center
Stuart Buck, Former Vice President of Research, Arnold Ventures
Sarah Nusser, Professor, Department of Statistics, Iowa State University
Roger Wakimoto, Vice Chancellor for Research, University of California, Los Angeles
Project Staff
Thomas Arrison, Director, Board on Research Data and Information
George Strawn, Scholar, Board on Research Data and Information
Ester Sztein, Deputy Director, Board on Research Data and Information
Emi Kameyama, Program Officer, Board on Research Data and Information
Consultants
Jennifer Saunders, Consultant
Greg Tananbaum, Director, Open Research Funders Group
ROUNDTABLE ON ALIGNING INCENTIVES FOR OPEN SCIENCE
Thomas Kalil (Co-Chair), Chief Innovation Officer, Schmidt Futures
Keith Yamamoto (NAS/NAM) (Co-Chair), Vice Chancellor for Science Policy and Strategy, University of California, San Francisco
Elizabeth Albro, Commissioner, National Center for Education Research, U.S. Department of Education*
Danny Anderson, President, Trinity University
Roslyn Clark Artis, President, Benedict College
Chris Bourg, Director of Libraries, Massachusetts Institute of Technology
Courtney Brown, Vice President for Strategic Impact, Lumina Foundation*
Anne-Marie Coriat, Head, UK and Europe Research Landscape, Wellcome Trust*
Michael Crow, President, Arizona State University
Mark Cullen (NAM), Director, Center for Population Health Sciences; Senior Associate Dean for Research, School of Medicine, Stanford University
Ronald Daniels, President, Johns Hopkins University
Tashni-Ann Dubroy, Executive Vice President and Chief Operations Officer, Howard University
Susan Fitzpatrick, President, James S. McDonnell Foundation*
Maryrose Franko, Executive Director, Health Research Alliance*
Nicholas Gibson, Senior Program Officer, Human Sciences, John Templeton Foundation*
Daniel Goroff, Vice President and Program Director, Alfred P. Sloan Foundation*
Heide Hackmann, Chief Executive Officer, International Science Council*
Randolph Hall, Vice President for Research and Professor, University of Southern California
Robert Hanisch, Director, Office of Data and Informatics, National Institute of Standards and Technology*
___________________
* Denotes ex officio member.
Patricia Hswe, Program Officer for Scholarly Communications, Andrew W. Mellon Foundation*
Adam Jones, Program Officer, Science Program, Gordon and Betty Moore Foundation*
Renu Khator, President, University of Houston
Richard McCullough, Vice Provost for Research, Harvard University
Peter McPherson, President, Association of Public and Land-grant Universities*
Ross Mounce, Director of Open Access Programs, Arcadia Fund*
Sarah Nusser, Professor, Department of Statistics, Iowa State University
Loretta Parham, Chief Executive Officer and Director, Robert W. Woodruff Library, Atlanta University Center
Heather Pierce, Senior Director, Science Policy and Regulatory Counsel, Association of American Medical Colleges*
Dawid Potgieter, Senior Program Officer, Templeton World Charity Foundation*
Brian Quinn, Assistant Vice President, Research-Evaluation-Learning, Robert Wood Johnson Foundation*
Robert Robbins, President, University of Arizona
Jerry Sheehan, Deputy Director, National Library of Medicine, National Institutes of Health*
Bodo Stern, Chief of Strategic Initiatives, Howard Hughes Medical Institute*
Barbara Snyder, President, Association of American Universities*
Geeta Swamy, Associate Vice President for Research, Duke University; Vice Dean for Scientific Integrity, Duke University School of Medicine
Shirley Tilghman (NAS/NAM), President Emerita, Princeton University
Alan Tomkins, Deputy Director, Social, Behavioral and Economic Sciences, National Science Foundation*
Roger Wakimoto, Vice Chancellor for Research, University of California, Los Angeles
Thomas Wang, Chair, Open Science Committee, American Heart Association; Director, Division of Cardiovascular Medicine, Vanderbilt University Medical Center*
Jennifer Weisman, Director and Chief of Staff, Global Health Division, Bill & Melinda Gates Foundation*
Richard Wilder, General Counsel and Director of Business Development, Coalition for Epidemic Preparedness Innovations*
Duncan Wingham, Executive Chair, Natural Environment Research Council, United Kingdom Research and Innovation*
Open Research Funders Group Secretariat
Heather Joseph, Executive Director, SPARC
Greg Tananbaum, Director, Open Research Funders Group
Staff
Thomas Arrison, Director, Board on Research Data and Information
George Strawn, Scholar, Board on Research Data and Information
Ester Sztein, Deputy Director, Board on Research Data and Information
Emi Kameyama, Program Officer, Board on Research Data and Information
Olivia Torbert, Senior Program Assistant, Board on Research Data and Information
BOARD ON RESEARCH AND DATA INFORMATION
Sarah Nusser (Chair), Professor, Department of Statistics, Iowa State University
Amy Brand, Director, MIT Press
Stuart Feldman, Chief Scientist, Schmidt Futures
Ian Foster, Arthur Holly Compton Distinguished Service Professor of Computer Science, The University of Chicago; Director, Data Science and Learning Division, Argonne National Laboratory
Ramanathan Guha, Google Fellow, Google
Sallie Ann Keller (NAE), Division Director, Social and Decision Analytics and Distinguished Professor in Biocomplexity, University of Virginia
Mary Lee Kennedy, Executive Director, Association of Research Libraries
Barend Mons, Chair, BioSemantics, Leiden University Medical Center
Michael Stebbins, President, Science Advisors, LLC
Bonnie Carroll, Retired Founder and Strategic Consultant, Information International Associates, Inc.; CODATA Secretary General*
John Hildebrand (NAS), Regents Professor of Neuroscience, University of Arizona; NAS International Secretary*
Staff
Thomas Arrison, Director, Board on Research Data and Information
George Strawn, Scholar, Board on Research Data and Information
Ester Sztein, Deputy Director, Board on Research Data and Information
Emi Kameyama, Program Officer, Board on Research Data and Information
Olivia Torbert, Senior Program Assistant, Board on Research Data and Information
___________________
* Denotes ex officio member.
This page intentionally left blank.
Preface and Acknowledgments
The National Academies of Sciences, Engineering, and Medicine’s Roundtable on Aligning Incentives for Open Science convenes critical stakeholders to discuss the effectiveness of current incentives for adopting open science practices, current barriers of all types, and ways to move forward to optimally align reward structures and institutional values. To increase the contribution of open science to producing better science, the roundtable aims to improve coordination among stakeholders and increase awareness of current and future efforts in the broader scientific community. At its first meeting in February 2019, the roundtable developed an initial work plan and set of priorities. On September 20, 2019, the roundtable organized a public symposium, and the resulting publication, Advancing Open Science Practices: Stakeholder Perspectives on Incentives and Disincentives: Proceedings of a Workshop—in Brief, was released in 2020. The third meeting of the roundtable was held on February 27, 2020. Six working groups, created following the inaugural meeting, are driving the work of the roundtable to define specific issues or problems related to open science and discuss possible actions.
An important focus of roundtable discussions and working group activities to date has been the need for information and other resources that could be used by researchers, research institutions, research funders, professional societies, and other stakeholders interested in fostering open science practices. For example, universities that are considering adopting new approaches to encourage open science practices might benefit from
example cover letters or essays introducing open science-related issues that could be adapted for use in their communities. Research institutions and research funders could benefit from examples of language that signals their interest in open science activities and could be utilized in grant applications or job postings. Good practices primers that point to existing open science-related policies and approaches could be useful to a range of stakeholders. A compilation of the various research outputs that stakeholders might consider in developing approaches that foster open science could be useful in cross-disciplinary discussions, since different disciplines generate different types of outputs.
On November 5, 2020, the Roundtable on Aligning Incentives for Open Science organized a virtual public workshop in conjunction with its fourth meeting on November 6. The workshop explored the information and resource needs of researchers, research institutions, research funders, professional societies, and other stakeholders interested in fostering open science practices. Workshop participants discussed approaches to meeting those needs, such as development of a toolkit that could be used by various groups of stakeholders. The workshop included presentations of commissioned papers that describe and provide examples of draft elements of a toolkit that have been developed by members of roundtable working groups (see Appendix C). The workshop and these proceedings were made possible by financial support from the Arcadia Fund, the Open Research Funders Group, the Open Society Foundations, the Robert Wood Johnson Foundation, and the Wellcome Trust. Over its first three years, the roundtable has also received support from Arnold Ventures, Schmidt Futures, the Leona M. and Harry B. Helmsley Charitable Trust, and the National Library of Medicine.
This Proceedings of a Workshop was prepared by the workshop rapporteurs as a factual summary of what was presented and discussed at the workshop. The planning committee’s role was limited to planning and convening the workshop. The statements made are those of the rapporteurs and do not necessarily represent positions of the workshop participants as a whole, the planning committee, or the National Academies of Sciences, Engineering, and Medicine. We wish to extend sincere thanks to all the members of the planning committee for their contributions in scoping, developing, and carrying out this project.
This proceedings has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise, in accordance with procedures approved by the National Academies of Sciences, Engineering,
and Medicine. The purpose of this independent review is to provide candid and critical comments to assist the institution in making its published report as sound as possible and ensure the document meets institutional standards for quality and objectivity. The review comments and draft manuscript remain confidential to protect the integrity of the process. We wish to thank the following individuals for their review of this proceedings: Daniel Himmelstein, University of Pennsylvania; Veronique Kiermer, Public Library of Science; Alexa McCray, Harvard Medical School; and Keith Webster, Carnegie Mellon University. Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the content of the proceedings, nor did they see the final draft before its release. The review of this proceedings was overseen by Mary Lee Kennedy, Association of Research Libraries. Appointed by the National Academies, she was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this proceedings rests entirely with the rapporteurs and the National Academies.
Thomas Arrison, Director
Board on Research Data and Information
National Academies of Sciences, Engineering, and Medicine
This page intentionally left blank.
This page intentionally left blank.