1
Introduction1
Sharing knowledge is what drives scientific progress—each new advance or innovation in biomedical research builds on previous observations. However, for experimental findings to be broadly accepted as credible by the scientific community, they must be verified by other researchers. An essential step is for researchers to report their findings in a manner that is understandable to others in the scientific community and provide sufficient information for others to validate the original results and build on them.
In recent years, concern has been growing over a number of studies that have failed to replicate previous results and evidence from larger meta-analyses, which have pointed to the lack of reproducibility in biomedical research (e.g., Bik et al., 2016, 2018; Ioannidis, 2005; Landis et al., 2012; Sena et al., 2010). In response, funders, publishers, and other key stakeholders have recognized the need to encourage and enhance transparent reporting of preclinical research findings across the biomedical research life cycle (e.g., Kiley et al., 2017; McNutt, 2014; Nosek et al.,
___________________
1 This workshop was organized by an independent planning committee whose role was limited to identification of topics and speakers. This Proceedings of a Workshop was prepared by the rapporteurs as a factual summary of the presentations and discussion that took place at the workshop. Statements, recommendations, and opinions expressed are those of individual presenters and participants and are not endorsed or verified by the National Academies of Sciences, Engineering, and Medicine, and they should not be construed as reflecting any group consensus.
2015; the Center for Open Science;2 policies at the National Institutes of Health [NIH],3 the Gates Foundation,4 or the Wellcome Trust;5 or broadly accepted policy statements such as in the case of public health emergencies6). On September 25 and 26, 2019, the National Academies of Sciences, Engineering, and Medicine (the National Academies) hosted a public workshop in Washington, DC, to discuss the current state of transparency in the reporting of preclinical biomedical research and to explore opportunities for harmonizing reporting guidelines across journals and funding agencies. The workshop primarily focused on transparent reporting in preclinical research, but also considered lessons learned and best practices from clinical research reporting. The agenda for the workshop, titled Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting, was developed by an independent planning committee; the workshop Statement of Task is available in Box 1-1.7 The workshop was convened jointly by the Forum on Drug Discovery, Development, and Translation; the Forum on Neuroscience and Nervous System Disorders; the National Cancer Policy Forum; and the Roundtable on Genomics and Precision Health, and was sponsored by the Cell Press, The Lancet, the National Institutes of Health, and Nature Research.
This workshop builds on recent consensus reports by the National Academies, including Reproducibility and Replicability in Science (NASEM, 2019, discussed further in Chapter 2), Open Science by Design: Realizing a Vision for 21st Century Research (NASEM, 2018), and Fostering Integrity in Research (NASEM, 2017). Selected guidelines and checklists for transparent reporting were also discussed and brief background information was provided to participants to facilitate the discussions (available in Appendix B).
___________________
2 See https://cos.io (accessed January 13, 2020).
3 See https://grants.nih.gov/policy/reproducibility/index.htm (accessed January 10, 2020).
4 Open access policy available at https://www.gatesfoundation.org/How-We-Work/General-Information/Open-Access-Policy (accessed January 13, 2020).
5 Open research policies available at https://wellcome.ac.uk/what-we-do/our-work/open-research (accessed January 13, 2020).
6 For more information, see https://wellcome.ac.uk/press-release/statement-data-sharing-public-health-emergencies (accessed February 10, 2020).
7 The workshop agenda can be found in Appendix C. Archived webcast videos and speakers’ presentations are available on the National Academies website. See http://nationalacademies.org/hmd/Activities/Research/DrugForum/2019-Sept-25.aspx (accessed November 20, 2019).
ORGANIZATION OF THE PROCEEDINGS
This Proceedings of a Workshop summarizes the presentations and discussions that took place at the workshop. As background for the workshop discussions, an overview of the findings and recommendations of the most recent National Academies consensus study related to reproducibility in scientific research, Reproducibility and Replicability in Science, was presented by Harvey Fineberg, president of the Gordon and Betty Moore Foundation and chair of both the workshop and the consensus study
(Chapter 2). Participants were also addressed by National Academy of Sciences (NAS) President Marcia McNutt, who delivered a keynote talk on sustaining public trust in science (Chapter 2). The first workshop panel focused on the current culture of science as it relates to transparency in proposing and reporting preclinical biomedical research. Researchers representing different roles in the science ecosystem shared their perspectives on the incentives, disincentives, challenges, and opportunities associated with transparent reporting and replicability in science (Chapter 3). The second panel considered lessons learned and best practices for increased transparency from the field of clinical research that could be applied to improving transparent reporting of preclinical studies (Chapter 4). In the third panel session, speakers explored the practical application and effectiveness of checklists and guidelines for enhancing transparent reporting of biomedical research (Chapter 5). In the final panel session, speakers representing different stakeholders discussed opportunities for action to harmonize guidelines and develop a common set of minimal reporting standards (Chapter 6). The last session of the workshop provided an opportunity for all participants to engage in small group discussions to apply what they had learned throughout the workshop and consider what actions researchers, publishers, institutions, and funders could take to improve transparent reporting of biomedical research (Chapter 7). Points of interest were shared on Twitter throughout the workshop by participants using the hashtag #reproducibilityinscience.8
___________________
8 The Twitter discussion that took place on September 25 and 26 in association with the workshop can be viewed at https://twitter.com/hashtag/ReproducibilityInScience (accessed November 20, 2019).