National Academies Press: OpenBook

Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop (2020)

Chapter: Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting

« Previous: Appendix A: References
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×

Appendix B

Background Discussion Document: Selected Guidelines for Transparent Reporting

SELECTED GUIDELINES FOR TRANSPARENT REPORTING

The purpose of this document is to help inform workshop discussions on improving the harmonization of guidelines for transparent reporting across journals and funding agencies so that biomedical researchers propose and report data in a consistent manner. This discussion document was compiled based on criteria described in the Transparency and Openness Promotion (TOP) guidelines;1 the National Academies of Sciences, Engineering, and Medicine report on Reproducibility and Replicability in Science;2 and the National Institutes of Health (NIH) policy on Enhancing Reproducibility through Rigor and Transparency.3 Each of the guidelines summarized below has a different scope and purpose.

Rather than a comprehensive comparison of the several guidelines, this document indicates the criteria related to transparent reporting that are covered by the various guidelines. It is intended as a background for the workshop discussion.

___________________

1 Available at https://cos.io/top (accessed January 12, 2020).

2 Available at https://www.nap.edu/catalog/25303/reproducibility-and-replicability-in-science (accessed January 12, 2020).

3 Available at https://grants.nih.gov/policy/reproducibility/index.htm (accessed January 12, 2020).

Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×

BRIEF DESCRIPTIONS OF GUIDELINES SUMMARIZED

  • The Animal Research: Reporting In Vivo Experiments (ARRIVE) guidelines recommend criteria for the reporting of primary research using animals. The guidelines were based on the Consolidated Standards of Reporting Trials (CONSORT) guidelines, but cover diverse study types. They were developed by the National Centre for the Replacement, Refinement and Reduction of Animals in Research (NC3Rs), and were first published in PLOS Biology in 2010. The ARRIVE guidelines are currently being revised; a preprint of the revised ARRIVE guidelines and the accompanying Explanation and Elaboration document are available on BioRxiv, respectively at https://www.biorxiv.org/content/10.1101/703181v1 and at https://www.biorxiv.org/content/10.1101/703355v1.
  • The CONSORT statement recommends information to include when reporting a randomized trial; was developed by an international group of trialists, methodologists, and medical journal editors; and was first published in JAMA in 1996 and last revised in 2010 and published in multiple journals. For more information, see http://www.consort-statement.org.
  • The DRAFT materials, design, analysis, and reporting (MDAR) checklist for authors represents a generic set of minimum requirements applicable to all reporting studies in the life sciences for the explicit purpose of increasing transparent reporting and reproducibility, developed by the MDAR working group specifically seeking common reporting points from across multiple journals. The checklist is being pilot tested by volunteer journals, and therefore has not been published at the time of this writing. The statement of task is available at https://osf.io/preprints/metaarxiv/9sm4x.
  • NIH policies:
    • “Data Sharing” policy, developed by NIH, to encourage data generated with NIH funding to be “made as widely and freely available as possible while safeguarding the privacy of participants, and protecting confidential and proprietary data.” The policy requires a data sharing plan for final research data generated on grants of $500,000 or more, and was implemented in 2003. For more information, see https://grants.nih.gov/grants/policy/data_sharing.
    • “Dissemination of NIH-Funded Clinical Trial Information” policy, developed by NIH and implemented in 2017, mandates that all clinical trials funded in part or in whole by NIH must be registered at ClinicalTrials.gov. The policy also requires that summary results be posted to ClinicalTrials.gov. For more infor
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
  • The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement recommends an evidence-based minimum set of reporting elements for systematic reviews and meta-analyses; was developed by an international group; and was first published in 2009 in multiple journals (PLOS Medicine, Annals of Internal Medicine, BMJ, Journal of Clinical Epidemiology, and Open Medicine). For more information, see http://www.prismastatement.org.
  • The Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement, developed by an international collaboration of trialists, methodologists, journal editors, and ethicists, recommends minimum content to include in clinical trial protocols, from study enrollment through closeout, first published in the Annals of Internal Medicine and BMJ in 2013. For more information, see https://www.spirit-statement.org.
  • The TOP guidelines describe eight modular standards of transparency that journals can select from to introduce policy and best practices at their publication; was developed by the Center for Open Science with input from journals, funders, and professional societies; and was first published in Science in 2015. See https://cos.io/top for information about TOP, including a summary table and links to complete policy language.
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Legend
Criterion covered: Dark boxes
Criterion partially covered: Gray boxes
Criterion not covered: White boxes
ARRIVE
Primary Research in Animals
CONSORT
Randomized Trials
STUDY CHARACTERISTICS
Code availability2
Data availability2,3
Data citation2,3
STUDY METHODS
Analytical methods: attrition, statistical precision, statistical power2,3
Plan for analytical decisions/preregistration2,3
Animal use/sex as a biologic variable4
Details of in-laboratory study replication3
Details of study methods, computation, and associated parameters2,3
Ethics2,5
Information on computational environment (e.g., operating system, library dependencies)3
Materials availability discussed2,3
Material authentication required4
Methods and protocols2,3,4
Sample definition2
RESULTS AND DISCUSSION
Adherence to community standards2
Discussion of uncertainty3
Discussion on generality constraints3
Discuss/assess rigor of prior research4
Dual use research of concern*

1. NIH policies represented include Data Sharing Policy, Dissemination of NIH-Funded Clinical Trial Information, and Rigor and Transparency.

2. Based on criteria described in the Transparency and Openness Promotion (TOP) guidelines, https://cos.io/top.

3. Based on criteria described in the National Academies report Reproducibility and Replicability in Science.

Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
DRAFT MDAR
Compilation of Multiple Life Science Journals
NIH Policies1
Preclinical and Clinical Research
PRISMA
Meta-analyses
SPIRIT
Clinical Trial Protocols
TOP Guidelines
Modular Publishing Standards
 
         
         
         
 
         
         
         
         
         
         
         
         
         
         
         
 
         
         
         
         
         

4. Based on criteria described in the NIH policy on Enhancing Reproducibility through Rigor and Transparency.

5. Covered by multiple NIH policies and offices, including Human Subjects Research policies and the NIH Office of Laboratory Animal Welfare.

* Covered by U.S. government Policy for Institutional Oversight of Life Sciences Dual Use Research of Concern.

Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×

This page intentionally left blank.

Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 105
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 106
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 107
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 108
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 109
Suggested Citation:"Appendix B: Background Discussion Document: Selected Guidelines for Transparent Reporting." National Academies of Sciences, Engineering, and Medicine. 2020. Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/25627.
×
Page 110
Next: Appendix C: Workshop Agenda »
Enhancing Scientific Reproducibility in Biomedical Research Through Transparent Reporting: Proceedings of a Workshop Get This Book
×
Buy Paperback | $58.00 Buy Ebook | $46.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Sharing knowledge is what drives scientific progress - each new advance or innovation in biomedical research builds on previous observations. However, for experimental findings to be broadly accepted as credible by the scientific community, they must be verified by other researchers. An essential step is for researchers to report their findings in a manner that is understandable to others in the scientific community and provide sufficient information for others to validate the original results and build on them. In recent years, concern has been growing over a number of studies that have failed to replicate previous results and evidence from larger meta-analyses, which have pointed to the lack of reproducibility in biomedical research.

On September 25 and 26, 2019, the National Academies of Science, Engineering, and Medicine hosted a public workshop in Washington, DC, to discuss the current state of transparency in the reporting of preclinical biomedical research and to explore opportunities for harmonizing reporting guidelines across journals and funding agencies. Convened jointly by the Forum on Drug Discovery, Development, and Translation; the Forum on Neuroscience and Nervous System Disorders; the National Cancer Policy Forum; and the Roundtable on Genomics and Precision Health, the workshop primarily focused on transparent reporting in preclinical research, but also considered lessons learned and best practices from clinical research reporting. This publication summarizes the presentation and discussion of the workshop.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!