Summary

The scientific research enterprise is a cornerstone of modern society. In the United States alone, the public and private sectors invest hundreds of billions of dollars and countless hours of highly skilled labor into the generation, validation, and dissemination of new knowledge every year. This investment delivers enormous benefits to society in the form of better health, enhanced understanding of the natural world, and new technologies that boost economic growth and improve life in myriad ways.

The integrity of knowledge that emerges from research is based on individual and collective adherence to core values of objectivity, honesty, openness, fairness, accountability, and stewardship. When researchers commit research misconduct or engage in other behavior that clearly damages research—what this report terms detrimental research practices—they stray from the norms and appropriate practices of science. Yet the research process itself, including its system of incentives, goes beyond the actions of individual researchers. Integrity in research means that the organizations in which research is conducted encourage those involved to exemplify these values in every step of the research process: planning, proposing, performing, and reporting their work; reviewing proposals and work by others; training the next generation of researchers; and maintaining effective stewardship of the scholarly record.

The research enterprise is a complex system that includes universities and other research institutions that educate, employ, and train researchers; the federal, foundation, and industrial sponsors of research; science, engineering, technology, and medical journal and book publishers; and scientific societies. These organizations can act in ways that either support or undermine the integrity of research.

For example, research institutions may—or may not—create and maintain

research environments that support integrity, including the policies and capabilities needed to respond responsibly to allegations of research misconduct. Science, engineering, technology, and medical journal and book publishers may provide high levels of rigor in review of manuscripts, or they may put pressure on prospective authors to add citations to manuscripts to improve a journal’s score on a bibliometric indicator. Fields and disciplines may take on as a community the task of defining and upholding necessary standards in areas such as data sharing, or they may fail to do so and, in effect, tolerate detrimental research practices.

Evidence accumulated over the past several decades, and particularly the past several years, provides strong support for the proposition that failing to define and respond strongly to research misconduct and detrimental research practices constitutes a significant threat to the research enterprise. This evidence is discussed mainly in Chapter 5 and Chapter 7. Highly visible research misconduct cases continue to appear regularly around the world. Appendix D describes several case studies of the multilayered challenges facing the U.S. research enterprise in fostering research integrity. Addressing threats to this integrity requires a contemporary understanding of the research system and challenges to the integrity of that system.

Concerns about scientific research that have emerged in the scientific and general media over the past several years reinforce the need to rethink and reconsider the strategies used to support integrity in research environments, including those used to prevent and address research misconduct and detrimental research practices. A growing body of evidence indicates that substantial percentages of published results in some fields are not reproducible; this lack of reproducibility appears to have many causes, ranging from essential aspects of the research process or differences in procedures to research misconduct or detrimental research practices. There also has been a remarkable increase in the number of retractions of journal articles, with analyses showing that a significant percentage of these retractions are due to research misconduct (Fang et al., 2012; Grieneisen and Zhang, 2012; Steen et al., 2013). The increase in retractions does not necessarily indicate that the incidence of misconduct is also increasing; other factors such as more vigilant scrutiny by the community and retractions becoming a more common practice among journals may be contributing factors. New forms of detrimental research practices are also appearing, such as “predatory” journals that do little or no editorial review or quality control of papers while also charging authors substantial fees and predatory conferences that charge researchers to speak at conferences that subsequently are canceled.

The research environment continues to change in significant ways that affect efforts to foster research integrity. Longstanding trends include growth in the size and scope of the research enterprise, the expansion of regulatory requirements, and an increased emphasis on industry sponsorship and entrepreneurial research. In addition, several important newer trends have emerged, including the pervasive and growing importance of information technology in research, the global-

ization of research, the increasing relevance of knowledge generated in certain fields to policy issues and political debates, and a pervasive media environment that can help generate and spread findings and controversies. These changes have led to important shifts in the institutions that support and underlie the research enterprise, such as science, engineering, technology, and medical publishing.

In assessing the trends and phenomena discussed above, along with possible new approaches, this report does not conclude that the research enterprise is broken. However, the research enterprise faces serious challenges in creating the appropriate conditions to foster and sustain the highest standards of integrity. To meet these challenges, deliberate steps must be taken to strengthen the self-correcting mechanisms that are an implicit part of research. The recommendations presented below are intended as a start to this process.

BACKGROUND OF THE STUDY

Several decades ago, prompted by a series of high-profile cases where data fabrication, falsification, and plagiarism were alleged and investigated, the U.S. research enterprise began to institute new approaches aimed at strengthening the capacity of researchers and research institutions to foster integrity and to address research misconduct. These approaches included the development of training materials and programs in the responsible conduct of research and formal federal oversight of research misconduct investigations affecting federally funded work.

As part of these efforts, the Committee on Science, Engineering, and Public Policy (COSEPUP) of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine formed a panel to undertake a major study of issues related to scientific responsibility and the conduct of research. Completed in 1992, Responsible Science: Ensuring the Integrity of the Research Process recommended steps for reinforcing responsible research practices (NAS-NAE-IOM, 1992). The report also developed a framework to distinguish three categories of behaviors that can compromise the integrity of the research process. Misconduct in science was defined as “fabrication, falsification, or plagiarism in proposing, performing, or reporting research.” Questionable research practices were defined as “actions that violate traditional values of the research enterprise and that may be detrimental to the research process.” Other misconduct was defined as “forms of unacceptable behavior that are clearly not unique to the conduct of science, although they may occur in the laboratory or research environment.” The unified federal policy adopted in 2000 uses a definition of research misconduct that largely reflects the recommendations of the COSEPUP panel.

Several years ago, COSEPUP commissioned a new committee to prepare a second edition of Responsible Science. In undertaking this effort, it became clear to the committee that changes in the research environment and the extent of the current challenges posed by research misconduct and other detrimental research

practices that clearly damage research required the development of a substantially new report.

RECOMMENDATIONS

While reaffirming the central recommendation from Responsible Science that formally places the primary responsibility for strengthening the responsible conduct of research on individual researchers and research institutions, the committee also believes that the integrity of research depends on creating and maintaining a system and environment for research in which institutional arrangements, practices, policies, educational programs, and incentive structures support responsible conduct.

The committee also endorses the definition of research misconduct recommended in Responsible Science, while recommending refinements and harmonization of the definition and its use. The committee believes that many of the practices that have been categorized up to now as questionable research practices, such as the misleading use of statistics that falls short of falsification and failure to retain sufficient research data, should be recognized as detrimental to research. Detrimental research practices also should be understood to include irresponsible or abusive actions taken by research institutions and journals in addition to the actions of individual researchers.

Lack of attention to or tolerance of detrimental research practices by stakeholders makes it difficult to expose misconduct, wastes human and financial resources, impairs the overall quality of research, and diminishes public trust in science. In addition, weaknesses in the system for identifying, investigating, and addressing research misconduct—most notably unevenness in the policies and capabilities of research institutions and science, engineering, technology, and medical journal and book publishers—create barriers to uncovering misconduct and taking appropriate action. Similarly, in industry-performed or industry-sponsored research, pressures associated with regulatory approvals or commercial release may create disincentives for full data transparency or biases that favor conclusions of safety and efficacy. Finally, changes in the research environment such as technological advances and globalization are making it more difficult and complex for all stakeholders in the enterprise to update and ensure adherence to best practices.

The checklists presented in Chapter 9 should form the basis of strategies to refine and implement best practices by researchers, research institutions, research sponsors, journals, and societies.

The key responsibilities for research institutions fall into four areas. The first is creating and sustaining a research culture that fosters integrity and encourages adherence to best practices. This includes maintaining education and training efforts that support a culture of integrity, consistent with the current state of knowledge (see Recommendation Ten).

A second task is monitoring the integrity of research environments. Such monitoring is critical to further advance understanding of how institutional structure, context, and incentives interact to buttress or detract from research integrity. Research organizations have an obligation to assess, monitor, and work to implement improvements to their research environments. Where institution-wide assessments identify units with particularly strong integrity environments, they should be examined and their practices should be disseminated and emulated.

The third institutional responsibility is ensuring that research institutions sustain the capacity needed to effectively investigate and address allegations of research misconduct. No institution can be expected to prevent all lapses in research integrity, but all should ensure that when problems in the conduct of research are alleged or identified there is a prompt, effective, and documented response to the allegation.

A fourth responsibility is ensuring that senior institutional leaders, including the president, other senior executives, and faculty leaders, are guiding and actively engaged in the preceding three tasks. When institutional leaders are accessible and knowledgeable about institutional capacity to address allegations of misconduct, they are in a position to help keep people and processes on track when specific allegations arise. Should later events call into question the rigor of an institutional response to allegations of misconduct in research, top institutional leadership should be expected, as a matter of course, to examine the shortcomings of the process and share lessons learned with the larger community of scholars. Institutional leaders should also reiterate the importance of critical standards such as appropriate authorship practices, data sharing, and complete reporting of results.

Those who raise concerns about the integrity of research, often referred to as whistleblowers, can play a critical role in supporting best practices in research and in uncovering research misconduct. Individuals closest to the research are in the best position to identify and correct problems as early as possible and can be expected to play this role for the foreseeable future. Inadequate responses to expressed concerns have constituted a critical point of failure in many cases of misconduct where investigations were delayed or sidetracked. Those who raise concerns are often the most vulnerable participants in the system, typically holding little institutional power or status. Research institutions and federal agencies should understand the implicit bias that exists against those who in good faith raise fact-based concerns about the integrity of research.

While various groups, institutions, and individuals are doing valuable work to foster and promote research integrity in the United States, no permanent organizational focus for efforts to foster research integrity at a national level currently exists. The Research Integrity Advisory Board recommended by the committee would bring a unified focus to understanding and addressing challenges across all disciplines and sectors.

The RIAB could facilitate the exchange of information regarding approaches to assessing and creating environments of the highest integrity and to the handling of allegations of misconduct and investigations. It could provide advice, support, encouragement, and, where helpful, advocacy on what needs to be done by research institutions, science, engineering, technology, and medical journal and book publishers, and other stakeholders in the research enterprise to promote research integrity. The RIAB will have no direct role in investigations, regulation, or accreditation. Rather, it will serve as a neutral resource based in the research enterprise that helps the enterprise respond to ongoing and future changes.

Authorship practices are a fundamental component of the research enterprise’s operation, and observance of good practices is a key factor in ensuring research integrity. Authorship crucially designates who bears responsibility for the work. Clarifying authorship responsibility is also critical in cases of error or allegations of misconduct.

The current situation, in which authorship practices and conventions are largely left to individual institutions and journals, is increasingly problematic. Greater clarity at the disciplinary level about the significant intellectual contributions that merit authorship, the roles that do not merit authorship, the significance of author order, and the responsibilities of a primary or corresponding author would be very helpful in facilitating appropriate decisions and practices in laboratories and collaborations. Universal condemnation (i.e., by all disciplines) of gift or honorary authorship, coercive authorship, and ghost authorship would also contribute to changing the culture of research environments where these practices are still accepted.

In many fields and disciplines, current standards for transparency are not adequately supporting reproducibility and the ability to build on previous work. However, the research enterprise has begun to take important steps. Some journals have begun to implement requirements that authors make the data and computer code required to regenerate the published results available upon request. Many universities and funding agencies have created online repositories to support the dissemination of digital data. Current data practices vary significantly by field and discipline, and making certain types of data broadly accessible presents special challenges. The successful development and implementation of new standards and requirements will depend upon sufficient investments in necessary human and physical infrastructure.

Preparing data and code for release can be expensive and time consuming. Researchers are currently rewarded for manuscript publication, but the professional rewards for preparing data and code for publication are minimal. The resources to support the endeavor are often limited, and the feasibility and time required depend on the type of research data and how those data were collected.

Journals should update their publication requirements to include access to data and code needed to replicate results in the manuscript. These data and codes can be deposited at any repository that can reasonably guarantee a persistent URL, to be provided in the text of the published paper. To facilitate the reuse of scientific code and data, these objects should be shared in a way that maximizes access while respecting scientific norms such as attribution.

Today, several initiatives exist to encourage and promote reproducibility in research. As routine reporting of negative results and statistical tests becomes the standard for all fields, research spending will become more productive, with more knowledge generated per dollar of research investment. Changing the culture of research and publication so that reporting negative results is required will depend on a persistent effort on the part of disciplines, sponsors, and journals.

While understanding of the causes and incidence of research misconduct and detrimental research practices has increased, critical knowledge gaps remain. For example, official statistics on findings of research misconduct may represent a lower bound on incidence, with survey data pointing to a significantly higher incidence of misconduct, but no reliable estimate of incidence or trends exists. Also, detrimental research practices are more widespread and may ultimately be more damaging to the research enterprise than research misconduct, which points to the need to address challenges to research integrity more broadly. In addition, trends in some indicators—such as declining success rates for grant applications, and an increasing ratio of PhD production to available faculty positions—raise the possibility that both local organizational environments and the broader structural arrangements of research are moving in directions that might threaten research integrity. Additional theoretically grounded research with subsequent testing in practice is warranted to more completely inform efforts to improve research environments and incentive structures.

Formal education and training in the responsible conduct of research (RCR) can play an important role in fostering integrity and strengthening research environments. Evidence developed to date indicates that much remains to be learned about the approaches that are most effective. RCR education can serve as a key element in strategies to promote integrity, but perhaps not as the primary means of addressing research misconduct and detrimental research practices in the short term. Evidence-based assessment and improvement of RCR education programs are needed, with the focus expanded to include the social and institutional environment for research. RCR education should engage not only junior researchers but also senior researchers and industrial researchers.

While the committee was tasked with considering the issue of research integrity regarding U.S.-based institutions and U.S. policies, a good deal of research now takes place in other countries and across national boundaries. Given that research misconduct, detrimental research practices, and the need to foster research integrity are challenges facing all countries that fund and perform research, the global research enterprise will benefit from the knowledge gained from examining research practices globally.