Synopsis: Responsible conduct of research (RCR) education programs have become more common in recent years, partly as a result of policy changes such as the National Science Foundation’s mandate that students supported by NSF research grants receive RCR education. Knowledge as to how best to provide such education is still developing. RCR educators are seeking to understand and articulate the most appropriate goals for such education, the most effective methods to be used, and the best formats in which to provide training. Improved assessments of the effects of RCR education can help develop approaches that will support both researchers and the broader institutional climate in which research takes place. RCR education can be a significant component in improving research integrity, and it will be most effective when undertaken as one of a broader set of strategies that encourage responsible conduct and discourage research misconduct and detrimental research practices. This chapter references a paper prepared for the project by Michael D. Mumford, which is included as Appendix C of this report.
This report emphasizes that the system of research and the environments in which research is conducted should both be addressed because each strongly affects how individual researchers behave. Those who enter science and engineering learn, one way or another, about the norms and practices of the research enterprise. They are socialized into research environments and must understand something about these environments to succeed in their careers. Too often, the socialization or training they receive is often ad hoc, on the job, and not intentionally provided. Responsible conduct of research (RCR) education is important because the careers of all researchers can be significantly affected by lack of attention to responsible research practices. Thus, it is equally important in industrial, governmental, and academic settings, as well as in both private and public institutions.
One way of framing RCR education is as an intervention to improve the ethical conduct of investigators (see Appendix C). However, this framing can suggest that RCR education is a response to a problem and that it is external to research. An alternative, more appropriate, and likely more effective approach is to think
of RCR education as an integral part of research because RCR education aims to ensure that the knowledge, skills, and awareness essential to responsible research are intentionally, explicitly, and accurately conveyed.
Framing responsible research as the norm is both the best frame for RCR education and an essential objective of that education. RCR education targeted to individuals is designed to influence the way they understand the research enterprise and how they make decisions. In targeting individuals, the committee hopes RCR education affects attitudes and actions in ways that ultimately influence the research environment. Additionally, RCR education within an institution can create conduits for communicating and fostering a more positive institutional climate. Open discussion of ethical issues can contribute to collective openness within an institution (Anderson, 2007).
If RCR education is to be seen as more than an intervention, integration of such education into the research endeavor is key. Nominally, this includes instruction in the norms and practices of research across many and varied disciplines. Ideally, RCR education should be incorporated into the socialization and training students experience on the job, whether in the laboratory or in the myriad other locations where researchers do their work.
The National Institutes of Health began requiring RCR education in 1989 and continues to expand and refine those requirements (NIH, 2009). In 2007 the America COMPETES Act mandated that all trainees funded by the National Science Foundation (NSF) receive RCR training as well (NSF, 2009). In 1997, NSF instituted a broader impacts criterion for the evaluation of NSF proposals, which required researchers who submit proposals to NSF to address the broad impacts of their research on society. It can be argued this was a move in the direction of expanded RCR education, but the 2007 legislation made the requirement explicit.
NSF formally established its RCR requirement in 2009, which it explained as follows:
[E]ducation in RCR is considered essential in the preparation of future scientists and engineers. The COMPETES Act focuses public attention on the importance of the national research community’s enduring commitment and broader efforts to provide RCR training as an integral part of the preparation and long-term professional development of current and future generations of scientists and engineers.
The National Institutes of Health and NSF requirements have been a major impetus for the expansion of RCR educational activities. The objectives, goals, and benefits of RCR education provide additional incentives for this expansion.
In thinking about the aims of RCR education, it is helpful to distinguish objectives, goals, and benefits. Objectives are the broad aims of RCR education; they are what RCR education seeks to achieve in the long term and as part of a diverse set of activities. Achievement of objectives may not be measurable within a particular course or activity. For example, it may not be possible to determine whether or to what extent a particular course or course module has contributed to the objective of reducing the incidence of research misconduct.
In contrast with objectives, goals are narrower in scope and more specific. Goals might be measured in the assessment of a particular activity. For example, a goal might be to ensure that researchers are aware of codes of conduct. Goals are related to objectives in the sense that they may be adopted because of their contribution to a broader objective. For example, a course might have the goal of improving ethical decision making. This goal in turn contributes to the broader objective to ensure the integrity of research.
In addition to objectives and goals, RCR education may provide benefits not identified as an objective or a goal. For example, improving the retention of researchers who might otherwise have left the field due to disappointment in the practiced norms of research is an advantage of RCR education, but may not be a specified objective or goal. Objectives, goals, and advantages overlap and are not always easy to distinguish.
Among the major objectives identified in the literature on RCR education are the following:
- Ensuring and improving the integrity of research; promoting good behavior and quality research conduct;
- Preventing bad behavior; decreasing research misconduct;
- Making trainees aware of the expectations about research conduct within the research enterprise and as articulated in various federal, state, institutional, and professional laws, policies, and practices that exist;
- Making practitioners and trainees aware of the uncertainty of some norms and standards in research practices due to such factors as changes in the technology used in research and the globalization of research;
- Promoting and achieving public trust in science and engineering;
- Managing the impact of research on the world beyond the lab, including society and the environment.
Goals for Educational Activities
These broad objectives have been formulated into more concrete goals to be achieved by particular forms of RCR education. For example, Michael Davis (Davis and Feinerman, 2010) identifies four goals for RCR education: ethical sensitivity (being able to recognize ethical issues), ethical knowledge, ethical judgment, and ethical commitment. The report of a 2008 workshop organized by the National Academy of Engineering and sponsored by NSF describes a set of skills to be developed in RCR education as follows (NAE, 2009):
- Recognizing and defining ethical issues;
- Identifying relevant stakeholders and sociotechnical systems;
- Collecting relevant data about the stakeholders and systems;
- Understanding stakeholder perspectives;
- Identifying value conflicts;
- Constructing viable alternative courses of action or solutions and identifying constraints;
- Assessing alternatives in terms of consequences, public defensibility, and institutional barriers;
- Engaging in reasoned dialogue or negotiations;
- Revising options, plans, or actions.
Each of the skills listed is an activity that contributes to ethical decision making. For example, to behave responsibly one has to recognize that a situation poses an ethical problem; then, if one is to act in the situation, it is important to identify the relevant stakeholders, construct and assess alternative courses of action, and so on.
For many RCR educators, decision making, and specifically ethical decision making, should be the primary focus of RCR education. Kalichman has succinctly argued for this; he identifies three possible objectives for RCR education: decreased research misconduct, increased responsible conduct of research, and improvements in ethical decision making (Kalichman, 2012). He rejects the first two as unknowable and focuses on ethical decision making.
As can be seen from the preceding, the challenge of RCR education derives in part from the nature of ethics teaching, which involves conveying knowledge, developing skills, shaping attitudes, and affecting behavior.
As a formal undertaking, RCR education is still in the early stages of its development. Experience over the last several decades has provided some basis for going forward, but the state of knowledge in the field is far from mature.
A particular focus within RCR education has been the assessment of its effects, but assessment of education in ethics is a relatively new field. The chal-
lenges of assessing ethics education are intertwined with the challenges of identifying and specifying the objectives and goals of such education.
In his review of assessments of the effectiveness of RCR training, Mumford concludes that the evidence indicates weak but positive effects (see Appendix C). Restricting his evaluation to improvements in ethical decision making, Kalichman characterizes the evidence as equivocal at best (Kalichman, 2012).
In the interaction between assessment, objectives, and goals, a “chicken-and-egg” problem may arise. One of the purposes of assessment is to help identify the most effective approaches to take in RCR education. At the same time, useful assessment depends on identifying and articulating measurable objectives and goals. In other words, specifying appropriate objectives and goals for RCR education is critical to assessment, yet assessment informs the selection of appropriate objectives and goals. Hence, a point of caution is appropriate here. In the interplay between assessment and RCR education, assessment should follow, not lead. One standard criticism of assessment is captured in the phrase “measures become targets.” The concern is that assessment will take forms or produce results that have too strong an influence on the structure or content of RCR education. RCR educators may teach or train to the assessment tool rather than continue to reflect on what are the important objectives, measurable or not.
One of the challenges in identifying what constitutes strong RCR education programs is the varied approach to assessment of RCR education, which in part arises from diverse perspectives on the educational goals of RCR courses. Achievement of the broad objectives and ancillary benefits described earlier can be even more difficult to assess, since many accrue over the long term and require large populations to demonstrate statistical significance (such as effects on the incidence of misconduct or rates of retention in science).
Despite the challenges of assessment, the Project for Scholarly Integrity at the Council of Graduate Schools and discussion at the Ethics Education in Science and Engineering Workshop at the National Academy of Engineering both support the assertion that assessment is critical to successful and sustainable RCR programs (CGS, 2012; NAE, 2009). Kalichman describes four key goals of RCR education that could be assessed: (1) increases in knowledge of issues and practices, (2) increases in skills related to ethical decision making and conflict management, (3) improved attitudes toward open communication and respect of issues, and (4) improvements in behavior and choices (Kalichman, 2012). Mumford describes key goals of RCR education as improvements in ethical decision making, perceptions of ethical climate, and knowledge (Appendix C). Mumford goes on to describe assessment measures of RCR education as measures of performance (such as decision making in ethics cases), knowledge (such as the results of an exam on human subjects regulation), climate (such as the extent to which individuals endorse ethical behaviors), products (such as self-reflection exercises), or organizational outcomes (such as a drop in the incidence of ethical violations) (Appendix C). Most assessment efforts for RCR education
have focused on improvements in ethical decision making (Antes et al., 2010; Bebeau, 2002; Mumford et al., 2008; Pimple, 2001; Schmaling and Blume, 2009) and/or knowledge (Elliott and Stern, 1996; Pimple, 2001; Schmaling and Blume, 2009). While more difficult to assess, some have attempted to assess behavioral choices (Anderson et al., 2007a; Wester et al., 2008).
How RCR education is assessed depends on the goals of the educational activity, and the goals in turn depend on the form the education takes. For example, a 1-hour module on data sharing or conflicts of interest will have narrower and different goals than would a full-semester course or a guest lecture series.
A number of formats have been adopted to provide RCR training for science and engineering trainees, including graduate students, postdoctoral trainees, and undergraduate students. These include stand-alone courses (DuBois et al., 2008; Elliott and Stern, 1996; Kalichman and Plemmons, 2007; Plemmons et al., 2006; Powell et al., 2007; Schmaling and Blume, 2009), seminar/workshop series that are either concentrated within a short period (such as an ethics week) or spread across a longer term (Antes et al., 2009; Clarkeburn et al., 2002; Ferrer-Negron et al., 2009; Fischer and Zigmond, 2001), ethics across the curriculum approaches that embed ethics materials into science and engineering coursework (Antes et al., 2009; Canary and Herkert, 2012; Davis and Riley, 2008; Frugoli, 2002; Smith et al., 2007), web-based training modules (Braunschweiger and Goodman, 2007; DuBois et al., 2008; Sieber, 2005), hybrid programs that use combinations of these approaches (Canary and Herkert, 2012), and laboratory-based interactions (Canary and Herkert, 2012).
Studies of the relative efficacy of these different approaches remain limited, but some modest positive results have been found for most approaches (see Appendix C; see also Antes et al., 2009; Elliott and Stern, 1996; Ferrer-Negron et al., 2009). However, one study has also found some negative effects of RCR education, particularly when students internalize the ideas that decisions regarding ethical issues have the potential to derail careers and that other researchers are unethical (Antes et al., 2010). Another found negative effects of RCR training but positive effects with RCR mentorship (Anderson et al., 2007a). Others have found a lack of change in assessment measures (Kalichman and Friedman, 1992).
Antes has argued that separate courses and seminars are more successful in ethical decision making than embedded programs (Antes et al., 2009). Others argue that RCR education content embedded in disciplinary or methods courses can also be successful (Davis and Riley, 2008). Web-based training has received the most criticism for possible lack of effectiveness, particularly when it is a pass/fail endeavor with little interpersonal interaction (NAE, 2009). Teaching RCR in a purely online format raises issues that surround online education more generally,
such as the difficulty of interacting with an instructor personally. In addition, the goals of teaching ethics, especially the skill, attitude, and behavior components, may not be as amenable to an online format as other material. Online formats tend to focus on knowledge and can be limited in their ability to teach ethical decision making and conflict management skills, which many identify as critical components of RCR education (Appendix C; see also Kalichman and Plemmons, 2007). However, the effectiveness of such web-based training, and indeed any training approach, appears to depend on what is included and how exactly students are engaged in the materials (Antes et al., 2009). Mumford (Appendix C) and Antes et al. (2009) both point to RCR programs that involve active and cooperative formats as being more effective in developing ethical decision-making processes, and these formats can be difficult to achieve online. Canary and Herkert (2012) found the strongest efficacy in RCR programs that take a hybrid approach.
While these studies are interesting, solid research on the efficacy of these different delivery methods is scarce for several reasons, including the lack of a standard approach to assessment, a lack of agreement on the goals of RCR education, and the challenges of conducting such educational research. In addition, RCR education is profoundly affected by the context in which that education exists (NAE, 2009). As such, efficacy may need to be studied within the context of the institution and the research field.
Within these RCR curricular approaches, educational methods can also vary widely. These include lecture, discussion of professional codes, expert panels, case-based discussions (Antes et al., 2009; Bebeau, 1995; DuBois et al., 2008), presentation and discussion of moral exemplars (Harris, 2008), role playing (Brummel et al., 2010; Seiler et al., 2011; Strohmetz, 1992), ethics issues embedded in science and engineering problems, and service learning (Fitch, 2004; Pritchard, 2000). Methods that encourage interaction have generally been found to be more successful (Antes et al., 2009; NAE, 2009). However, the relative effectiveness of these methods has yet to be examined fully, and such an examination may again be limited by the lack of standardization of RCR goals and assessment methods.
Beyond improved moral reasoning, the topics in research practice that should be covered in an RCR educational program have received only limited discussion. Such topics can include issues of credit in authorship and intellectual property (including issues of plagiarism), appropriate treatment of human and/or animal subjects, issues of conflict of interest, appropriate data management (including issues of fabrication and falsification), issues in the peer review process, mentoring and employment relationships, and societal impacts of research. In addition, specific disciplines can have discipline-specific topics such as the relationship between research and clinical practice in medicine or design and manufacturing issues in engineering. Conflict management techniques and processes (including both interpersonal communication skills and knowledge and understanding of organizational and institutional dynamics and structures) have been suggested
as an important component of RCR education (Gunsalus, 1998a; Kalichman and Plemmons, 2007). Topics in RCR education also are not static. For example, emerging issues include the use and accessibility of computer code in research and the proper application of statistical methods to large datasets.
The topics discussed in this report could provide guidance on topics appropriate for inclusion in RCR education. In particular, reducing detrimental research practices and improving best practices benefit from open and active discussion among scientists. Awareness of environmental effects on individual choices could support stronger individual decision making.
Resources for RCR Education
Since the publication of Responsible Science in 1992, a number of resources for RCR education have been created. Despite the inherent challenges with online education, many institutions have begun to use online training resources. The most used in the United States currently is the Collaborative Institutional Training Initiative program. This program has provided web-based training to thousands of institutions in 40 countries since 2000. Another example is the online content provided by Epigeum, a British education provider that offers courses on research ethics. A number of institutions have collected a range of online resources, and the Office of Research Integrity has produced an online video to address some elements of content.
Additional online repositories and collaborative environments for RCR educational materials include the website of the National Academy of Engineering’s Online Ethics Center for Science and Engineering (www.onlineethics.org), the NSF-funded National Ethics Center (Ethics CORE) (nationalethicscenter.org), the Resources for Research Ethics Education website (research-ethics.net) sponsored by the University of California, San Diego, and the Committee on Publication Ethics’ eLearning course on publication ethics for editors and publishers (COPE, 2017). Journals in the field include Science and Engineering Ethics (http://link.springer.com/journal/11948), which regularly publishes articles on research ethics and the teaching of research ethics, and Research Ethics (http://journals.sagepub.com/home/rea), which is sponsored by the Association for Research Ethics and is devoted to ethical research in human beings. Publishers have also developed RCR education materials; the majority of these materials specifically address publishing ethics. Wiley has recently released a second edition of Best Practice Guidelines on Publication Ethics: A Publisher’s Perspective (Graf et al., 2014), Elsevier provides a Publishing Ethics Resource Kit (https://www.elsevier.com/editors/perk), and BioMed Central provides resources on the “Publication Ethics” page of its website (https://www.biomedcentral.com/getpublished/writing-resources/publication-ethics).
In 2009, the National Academies published the third edition of its widely used reference, On Being a Scientist: A Guide to Responsible Research Conduct,
which is intended to supplement research ethics lessons provided by institutions, research mentors, and supervisors (NAS-NAE-IOM, 2009b). Among other things, the guide discusses treatment of data, research misconduct, authorship credit, and conflicts of interest. Hundreds of thousands of print and electronic copies of the guide have been distributed since the first edition was released in 1988.
In 2016 the InterAcademy Partnership released Doing Global Science: A Guide to Responsible Conduct in the Global Research Enterprise, intended for use in education and training contexts on a global basis (IAP, 2016). In addition, other textbooks, compilations of case studies, and other written materials are available (e.g., Penslar, 1995).
As mentioned before, RCR training is most effective when it is one element in a comprehensive approach to improve an institution’s system of research. If a comprehensive approach is not taken, aspects of the broader system may undermine the effectiveness of RCR education. For example, when faculty members and administrators or managers express a lack of enthusiasm or even disdain for RCR training, students and postdoctoral researchers get the message and may come to see RCR training as a regulatory burden or even believe that responsible conduct is not important to their research or careers. Research environments can convey this message in a number of subtle and unintended ways. For example, having RCR materials presented in the classroom only as the last lecture in a full-semester course or as a guest lecture on a day the professor will be absent may be interpreted as reflecting the professor’s lack of interest in the material.
Limitations on instructional time and the demands of research can limit the amount of RCR education that can be provided. For example, institutional pressures on research productivity have been shown to have negative effects on responsible conduct, and such pressures can also affect the willingness of research mentors to allow time for RCR education (Anderson et al., 2007b).
Although all the participants involved in research are important in creating an environment that is conducive to the responsible conduct of research, two categories of participants are especially important—institutional leaders and mentors. As Chapter 6 describes, institutional leadership and climate can be either a support or a barrier to effective RCR education. The Council of Graduate Schools’ project on scholarly integrity has recommended engaging the leadership of institutions as a critical part of any sustainable and effective RCR program. Beyond supporting RCR educational programs within the institution, institutions should be looking more broadly at educational and other activities that encourage research integrity. For example, in Reason’s 2000 BMJ article, a suggestion was made for institutions to move from blame-and-shame methods for dealing with misconduct to reporting and feedback. Such a reporting-and-feedback dynamic
might be created and nurtured through the RCR educational process as an element of a broader institutional program.
Mentors have particularly important roles because they advise aspiring researchers and because young researchers look to them as role models. The words and actions of a research mentor can both positively and negatively impact ethical behaviors and potentially support or undermine RCR educational efforts (Anderson et al., 2007a; Antes et al., 2010; Wright et al., 2008). Thus, to have an effective RCR environment, mentors must understand the effects of their behavior on young researchers and must be held accountable for conveying the importance of responsible conduct to their trainees. Ideally, mentors, other research scientists, and institutional leaders all actively participate in RCR discussions, since everyone involved in such discussions can benefit from open and honest discourse regarding best practices and detrimental research practices.
RCR education cannot be considered a total solution to the problem of ensuring responsible conduct. Rather, it should be seen as one component in a comprehensive approach that includes improving mentorship and institutional climate.
RCR education can continue to develop through the identification of a strong set of educational goals, the development of new educational tools, and the refinement of assessments. It also can be expanded to include not just trainees but research mentors, principal investigators, and institutional leaders in discussions of research ethics. Such involvement will contribute to a positive institutional climate and a greater collective openness.
In particular, since research mentors are so influential in the development of ethical behavior, RCR educational efforts should examine ways to use this relationship more productively to foster responsible conduct.