TY - BOOK AU - National Academies of Sciences, Engineering, and Medicine TI - Fostering Integrity in Research SN - DO - 10.17226/21896 PY - 2017 UR - https://nap.nationalacademies.org/catalog/21896/fostering-integrity-in-research PB - The National Academies Press CY - Washington, DC LA - English KW - Policy for Science and Technology AB - The integrity of knowledge that emerges from research is based on individual and collective adherence to core values of objectivity, honesty, openness, fairness, accountability, and stewardship. Integrity in science means that the organizations in which research is conducted encourage those involved to exemplify these values in every step of the research process. Understanding the dynamics that support – or distort – practices that uphold the integrity of research by all participants ensures that the research enterprise advances knowledge. The 1992 report Responsible Science: Ensuring the Integrity of the Research Process evaluated issues related to scientific responsibility and the conduct of research. It provided a valuable service in describing and analyzing a very complicated set of issues, and has served as a crucial basis for thinking about research integrity for more than two decades. However, as experience has accumulated with various forms of research misconduct, detrimental research practices, and other forms of misconduct, as subsequent empirical research has revealed more about the nature of scientific misconduct, and because technological and social changes have altered the environment in which science is conducted, it is clear that the framework established more than two decades ago needs to be updated. Responsible Science served as a valuable benchmark to set the context for this most recent analysis and to help guide the committee's thought process. Fostering Integrity in Research identifies best practices in research and recommends practical options for discouraging and addressing research misconduct and detrimental research practices. ER - TY - BOOK AU - National Academies of Sciences, Engineering, and Medicine A2 - Michael L. Cohen TI - Methods to Foster Transparency and Reproducibility of Federal Statistics: Proceedings of a Workshop SN - DO - 10.17226/25305 PY - 2019 UR - https://nap.nationalacademies.org/catalog/25305/methods-to-foster-transparency-and-reproducibility-of-federal-statistics-proceedings PB - The National Academies Press CY - Washington, DC LA - English KW - Behavioral and Social Sciences AB - In 2014 the National Science Foundation (NSF) provided support to the National Academies of Sciences, Engineering, and Medicine for a series of Forums on Open Science in response to a government-wide directive to support increased public access to the results of research funded by the federal government. However, the breadth of the work resulting from the series precluded a focus on any specific topic or discussion about how to improve public access. Thus, the main goal of the Workshop on Transparency and Reproducibility in Federal Statistics was to develop some understanding of what principles and practices are, or would be, supportive of making federal statistics more understandable and reviewable, both by agency staff and the public. This publication summarizes the presentations and discussions from the workshop. ER - TY - BOOK AU - National Academies of Sciences, Engineering, and Medicine TI - Reproducibility Issues in Research with Animals and Animal Models: Workshop in Brief DO - 10.17226/21835 PY - 2015 UR - https://nap.nationalacademies.org/catalog/21835/reproducibility-issues-in-research-with-animals-and-animal-models-workshop PB - The National Academies Press CY - Washington, DC LA - English KW - Biology and Life Sciences AB - In June 2014, the Roundtable on Science and Welfare in Laboratory Animal Use organized a public workshop to discuss fundamental aspects of experimental design of research using animals and animal models, aimed at improving reproducibility. At this workshop, researchers from around the world explored the many facets of animal-based research that could contribute to irreproducible results, including perspectives on improving experimental planning, design, and execution; the importance of reporting all methodological details; and efforts to establish harmonization principles of reporting on the care and use of animals in research studies. ER - TY - BOOK AU - National Academies of Sciences, Engineering, and Medicine A2 - Michelle Schwalbe TI - Statistical Challenges in Assessing and Fostering the Reproducibility of Scientific Results: Summary of a Workshop SN - DO - 10.17226/21915 PY - 2016 UR - https://nap.nationalacademies.org/catalog/21915/statistical-challenges-in-assessing-and-fostering-the-reproducibility-of-scientific-results PB - The National Academies Press CY - Washington, DC LA - English KW - Math, Chemistry, and Physics KW - Surveys and Statistics AB - Questions about the reproducibility of scientific research have been raised in numerous settings and have gained visibility through several high-profile journal and popular press articles. Quantitative issues contributing to reproducibility challenges have been considered (including improper data measurement and analysis, inadequate statistical expertise, and incomplete data, among others), but there is no clear consensus on how best to approach or to minimize these problems. A lack of reproducibility of scientific results has created some distrust in scientific findings among the general public, scientists, funding agencies, and industries. While studies fail for a variety of reasons, many factors contribute to the lack of perfect reproducibility, including insufficient training in experimental design, misaligned incentives for publication and the implications for university tenure, intentional manipulation, poor data management and analysis, and inadequate instances of statistical inference. The workshop summarized in this report was designed not to address the social and experimental challenges but instead to focus on the latter issues of improper data management and analysis, inadequate statistical expertise, incomplete data, and difficulties applying sound statistic inference to the available data. Many efforts have emerged over recent years to draw attention to and improve reproducibility of scientific work. This report uniquely focuses on the statistical perspective of three issues: the extent of reproducibility, the causes of reproducibility failures, and the potential remedies for these failures. ER - TY - BOOK AU - National Academies of Sciences, Engineering, and Medicine TI - Open Science by Design: Realizing a Vision for 21st Century Research SN - DO - 10.17226/25116 PY - 2018 UR - https://nap.nationalacademies.org/catalog/25116/open-science-by-design-realizing-a-vision-for-21st-century PB - The National Academies Press CY - Washington, DC LA - English KW - Policy for Science and Technology AB - Openness and sharing of information are fundamental to the progress of science and to the effective functioning of the research enterprise. The advent of scientific journals in the 17th century helped power the Scientific Revolution by allowing researchers to communicate across time and space, using the technologies of that era to generate reliable knowledge more quickly and efficiently. Harnessing today's stunning, ongoing advances in information technologies, the global research enterprise and its stakeholders are moving toward a new open science ecosystem. Open science aims to ensure the free availability and usability of scholarly publications, the data that result from scholarly research, and the methodologies, including code or algorithms, that were used to generate those data. Open Science by Design is aimed at overcoming barriers and moving toward open science as the default approach across the research enterprise. This report explores specific examples of open science and discusses a range of challenges, focusing on stakeholder perspectives. It is meant to provide guidance to the research enterprise and its stakeholders as they build strategies for achieving open science and take the next steps. ER - TY - BOOK AU - National Academies of Sciences, Engineering, and Medicine TI - Reproducibility and Replicability in Science SN - DO - 10.17226/25303 PY - 2019 UR - https://nap.nationalacademies.org/catalog/25303/reproducibility-and-replicability-in-science PB - The National Academies Press CY - Washington, DC LA - English KW - Surveys and Statistics KW - Policy for Science and Technology AB - One of the pathways by which the scientific community confirms the validity of a new scientific discovery is by repeating the research that produced it. When a scientific effort fails to independently confirm the computations or results of a previous study, some fear that it may be a symptom of a lack of rigor in science, while others argue that such an observed inconsistency can be an important precursor to new discovery. Concerns about reproducibility and replicability have been expressed in both scientific and popular media. As these concerns came to light, Congress requested that the National Academies of Sciences, Engineering, and Medicine conduct a study to assess the extent of issues related to reproducibility and replicability and to offer recommendations for improving rigor and transparency in scientific research. Reproducibility and Replicability in Science defines reproducibility and replicability and examines the factors that may lead to non-reproducibility and non-replicability in research. Unlike the typical expectation of reproducibility between two computations, expectations about replicability are more nuanced, and in some cases a lack of replicability can aid the process of scientific discovery. This report provides recommendations to researchers, academic institutions, journals, and funders on steps they can take to improve reproducibility and replicability in science. ER -