Skip to main content

Currently Skimming:

2 Transparency and Trust
Pages 5-16

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 5...
... and former editor-in-chief of Science, delivered a keynote address to participants on signaling "indicators of trust" to the scientific community and the public. OVERVIEW OF REPRODUCIBILITY AND REPLICABILITY IN SCIENCE Harvey Fineberg, President, Gordon and Betty Moore Foundation The National Academies consensus study report Reproducibility and Replicability in Science was sponsored by the National Science Foundation.1 The committee assessed research and data reproducibility issues with a focus on topics that cross disciplines.
From page 6...
... Furthermore, the report states that neither reproducibility nor replicability alone can ensure the reliability of scientific knowledge. Reproducibility A challenge for computational reproducibility across scientific disciplines is that many reports of studies do not include sufficient information to allow another researcher to reproduce the original computa
From page 7...
... . The committee identified several obstacles to reproducibility, Fineberg reported, including inadequate recordkeeping, reporting that lacks critical elements, obsolete digital artifacts, errors in the attempts to reproduce the findings of others, and cultural barriers.3 The committee also noted that improving computational reproducibility is challenging because experiments are complex and involve multiple steps that must be systematically documented and reported.
From page 8...
... Thomas Curran, executive director and chief scientific officer of Children's Mercy, Kansas City, observed that errors in replicability can lead to new discoveries. He shared an example in which the majority of published research describing a new cancer drug target was wrong due to the use of an inappropriate model, but further research revealed the applicability of the target to rare cancers and led to new drugs that might not otherwise have been developed.
From page 9...
... In closing, Fineberg summarized that data sharing and transparent reporting should be an expectation of the scientific community. Barriers to the persistent availability of the digital artifacts and reagents needed for reproducibility and replicability include costs, lack of infrastructure, the culture of science, and weak incentives, he said.
From page 10...
... Journals and conference organizers are encouraged to: •  and implement desired standards of reproducibility and replicability Set and make this one of their priorities, such as deciding which level they wish to achieve for each Transparency and Openness Promotion guide line and working towards that goal; •  Adopt policies to reduce the likelihood of non-replicability, such as con sidering incentives or requirements for research materials transparency, design, and analysis plan transparency, enhanced review of statistical methods, study or analysis plan preregistration, and replication studies; and •  Require as a review criterion that all research reports include a thoughtful discussion of the uncertainty in measurements and conclusions." RECOMMENDATION 6-9: "Funders should require a thoughtful discussion in grant applications of how uncertainties will be evaluated, along with any relevant issues regarding replicability and computational reproducibility. Funders should introduce review of reproducibility and replicability guidelines and activities into their merit-review criteria, as a low-cost way to enhance both." NOTE: Recommendation numbers refer to the numbering scheme used in the report.
From page 11...
... The lead author of that paper then became involved and sought to retract the paper or correct it. McNutt read quotes from The Chronicle Review article, which suggested that the editor-in-chief of Criminology, who happened to be a university colleague of the associate professor, was highly resistant to a retraction despite the concerns raised by two coauthors of the paper.
From page 12...
... The norms of science promote these qualities, she continued, but scientists often do not "clearly signal" to others in the scientific community or to the public that these norms have been upheld, or when appropriate, that they have been violated. A benefit of communicating the adherence to scientific norms is that it reinforces those norms in the community.
From page 13...
... McNutt suggested that one possibility could be to develop an ongoing National Academies forum to share best practices for integrity, trust, and transparency and coordinate action across stakeholder groups at the research enterprise level. She noted that different stakeholder groups have created their own entities, such as the Committee on
From page 14...
... She suggested that reviewer training could be discussed at scientific society meetings, and that students, early career researchers, and senior investigators could all provide feedback on what would be most helpful for such reviewer training. Communicating Corrections in the Literature Shai Silberberg, director for research quality at the National Institute of Neurological Disorders and Stroke (NINDS)
From page 15...
... He added that comments on the proposal were gathered at a workshop attended by representatives from major journals, COPE, the National Library of Medicine, and others. Deborah Sweet, vice president of editorial at Cell Press, noted the difficulty in getting authors to retract papers.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.