National Academies Press: OpenBook
« Previous: 5 Online Ethics Center
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

6
RCR Training Exemplars

Over the course of two of the workshop’s sessions, seven speakers described examples of RCR training programs developed by various institutions and chosen to present at the workshop by the planning committee. A call for nominations yielded 25 submissions describing programs, which were reviewed by the planning committee and chosen based on innovativeness, adaptability to other institutions, and an evaluation component. The committee also chose programs with a range of disciplines and locations. Each speaker provided a 5-minute overview, followed by a 15-minute question and answer session. In addition, each speaker recorded a 15-minute presentation with more details about their program.

YOUNG SCIENTISTS NETWORK-ACADEMY OF SCIENCES-MALAYSIA

When De-Ming Chau (Universiti Putra Malaysia) returned to Malaysia eight years ago after completing his graduate training in the United States, there was no institutional RCR training at Malaysia’s universities. Today, he said, there is a widely available national RCR program supported by the Young Scientists Network, some Malaysian universities have incorporated RCR training for graduate students and faculty, and the Ministry of Higher Education is developing a mechanism to mandate RCR training in Malaysia.

The Malaysian RCR program owes its existence to the members of the Young Scientists Network, which is part of the Academy of Sciences-Malaysia. This group of young scientists, said Chau, had the same vision of creating more awareness of RCR in their country. They took a bottom-up approach that received strong support from various organizations, including the US National Academies, and from senior professors in Malaysia who are members of the Academy of Sciences-Malaysia.

The program has been active for three years and uses active learning activities to create awareness about RCR through workshops, seminars, and classes for early career researchers and students in universities. Chau and his colleagues have developed content that they contextualized for the Malaysian audience using programs originating in the United States, Australia, and the United Kingdom.

Discussion

Kalichman began the discussion by asking Chau if there was resistance from institutions, faculty, or students to introducing RCR training, and if so, what he and his colleagues did to overcome that resistance. Chau replied that one of the blessings of being young was that he and his colleagues just charged ahead, ignoring the messages that RCR training was a waste of time and resources and overcoming the reluctance to engage in RCR by finding the right partners to talk to reluctant individuals. For example, they recruited senior faculty to speak with the minister

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

for higher education to garner support for this effort. At the institutional level, young scientists who are now advancing in their careers are becoming leaders in RCR at their institutions and are taking the lead in convincing the top management at their universities to promote RCR. Students, it turns out, are enthusiastic supporters of RCR and enjoy the active learning pedagogy, and although Chau noted that one challenge has been getting them to speak up in classes, he added that setting the right tone and providing language to use when bringing up RCR issues has brought students out of their comfort zone and made them more willing to talk about various issues in RCR.

When asked how they structure their RCR sessions, Chau responded that the sessions typically range from a half-day to three days, depending on the topic, and use case studies, role-playing, brainstorming, sticky notes, and formative assessment.

Chau, responding to a question about whether the bottom-up approach has a different character or impact than the more top-down approach of the United States, said that he thinks the bottom-up approach has a great deal of impact because the majority the participants in RCR training are early- to mid-career researchers and students. “I think there is a sense of relatability when they know that the program is run by young scientists themselves,” he explained. Having the speakers be young scientists, too, creates a more open environment that encourages the participants to open up and engage more easily than if senior professors were speaking or running the program, he added. In terms of accountability, Chau said that the bottom-up approach also promotes accountability among the young scientists in a way that might not occur between a senior scientist and a junior scientist.

An attendee asked if Chau had any advice for adapting international programs to another context. He replied that drawing from international programs is important because many of them have years of experience developing content. Adaptations he and his colleagues made were as simple as changing the names in case studies to make it easier for Malaysian researchers to relate to them. He did note that many of the scenarios in these case studies were developed for biomedical researchers, so those were altered to relate to researchers in other fields, and that they needed to change some of the wording to reflect local language usage and be more sensitive to some specific features of Malaysian culture.

As a final question, Kalichman asked Chau to describe what he believes would be an ideal approach to assess the effects of his program. Chau replied that they use pre- and post-assessment, mainly for knowledge. Identifying any shifts in attitude toward RCR will occur over the next three to four years as more people participate in the workshops.

UNIVERSITY OF VIRGINIA DEPARTMENT OF ANTHROPOLOGY

In their presentation, Lise Dobrin and Ida Hoequist (University of Virginia) described an RCR program designed to meet the needs of anthropologists, whose research often has potential emotional, psychological, and social consequences. The program, the Field Work Ethics and Ethnographic Writing Workshop, is designed to address RCR in a way that is meaningful for her scholarly community. Dobrin explained that it addresses disciplinary diversity across archeologists, linguists, and anthropologists and incorporates the various methods used by these disciplines (i.e., fieldwork, speech analysis, embedding in communities).

All this, said Dobrin, makes the idea of topical coverage unrealistic and suggests an emergent type of RCR. In fact, she objects to the term “training,” and instead believes that a reasonable goal is engaging students in reasoning about ethical problems, exploring the problems

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

and potential solutions, and reaching out to others for help. “The more difficult the situation, the more you need helping thinking it through,” she said, “and that is something we can practice doing together.”

The workshop series follows guidance from the NIH to hold four 2-hour sessions a year. One session focuses on the role of Institutional Review Boards (IRBs) and how to collaborate with them and manage the process, while the other three sessions are open-ended and have covered a wide range of topics. She noted that the program satisfies regulatory requirements, but more importantly, fulfills the desire students have to process real scenarios they might encounter. In fact, like the Malaysian program, this program has a bottom-up component in that graduate students participate in choosing topics, inviting faculty, organizing the sessions, and acting as moderators. Her role, she added, is more one of advising and encouraging the students, and occasionally, steering the discussion back to RCR. She noted that faculty come as presenters, but they often come to other sessions because they find the topic interesting and want to hear their colleagues talk about it.

Dobrin said it never occurred to her to assess what the students are taking away from the sessions and that she will now start to think about doing assessments. For her, the success of the program is indicated by compliance with the program and the fact that students get involved in it far more than just signing in for compliance purposes. She noted that if she polled the students, they would not think they are doing RCR. In fact, the program is fully integrated into the life of the department, and being involved in organizing the sessions is just part of their job.

Regarding assessment, Hoequist, who has served as one of the early directors of the workshop, said that the IRB module includes a test to determine if the students have understood and integrated those principles into their thinking. They noted, though, that they have never had a subsequent conversation with any other student about that module. In contrast, the fieldwork, ethics, and ethnographic writing workshops generate a great deal of conversation both before and after the sessions. To them, this sustained engagement around these topics is a great measure of success for ethical conduct as researchers.

Discussion

Asked to say more about why she objects to the term RCR or ethics training, Dobrin replied that she does not believe that training conveys the mission, so perhaps engagement or grappling might be a better term because that is what ethics is about. “It is not about finding the right answer on a test,” she said. “It is navigating a difficult situation in which there is probably more than one right course of action or maybe no right course of action.”

Frances Ligler (North Carolina State University and the University of North Carolina at Chapel Hill), in her role as session moderator, asked if the program has continued in virtual mode during the pandemic and how that has affected the program. Dobrin replied that the program did continue online, and that the topics were adapted to fit an online format. For example, the students were concerned about how to plan and conduct fieldwork during a pandemic when travel is not allowed and were therefore thinking about archival research. They planned a session about ethics and archival research that Dobrin said she found very exciting.

Remarking that one of the things that made their program exciting and engaging for students was to focus on topics directly related to the anthropology field, Ligler asked Dobrin and Hoequist how they would advise researchers in other fields to go about planning their programs. Dobrin replied that every field knows its own problems—economists, for example, are concerned with financial conflicts of interest, while anthropologists are not—and so it requires

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

an insider’s perspective to select the right topics that are relevant and will engage people in that particular field. Hoequist added that the success of the program owes to the fact that the people participating in it are the same people who are designing it and selecting the topics with which they are grappling. “The model is to make it small scale, talk about what is important to you right now, and make the people are working together talk about it together,” said Hoequist.

Ligler asked if the program reaches anyone outside of those who participated in the workshops. Hoequist replied that it definitely does through discussions with graduate students and faculty in other departments. They noted that students leave the workshops excited to talk about what they discussed with anyone who is interested, and that the workshops give students an arena to talk to faculty on equal footing. Dobrin said she feels the workshops do open lines of communication between students and faculty and that she has had students come to her to talk about issues they are facing.

NATIONAL CENTER FOR PROFESSIONAL AND RESEARCH ETHICS (NCPRE)

In her presentation, C. K. Gunsalus (University of Illinois Urbana-Champaign) discussed the program she and her colleagues are building called Labs that Work for Everyone,24 one premised on an understanding that creating and sustaining a culture of excellence involves thinking less about what work is done and more on how the work is done. The cornerstones of the program, she said, are rigor, reproducibility, meaningful inclusion, and integrity. “Ethical professional conduct requires attention to the things we do every day, our behaviors and interactions, as well as learning about profession responsible conduct research issues, so we have approached this as better science via leadership development,” said Gunsalus.

The program is a leadership development curriculum aimed at Howard Hughes Medical Institute (HHMI) lab members that focuses on providing tools and perspectives to be able to deal with colleagues directly and appropriately in groups, because research is often a team-based endeavor. As such, the program aims to develop the skills that go into working and effectively resolving differences and disputes involving professional issues.

To do so, the program takes an evidence-based and values-based approach that is emotionally engaging and built around professional development. It is set in real laboratory environments and shows nuanced RCR dilemmas on which reasonable people can disagree. Gunsalus explained that because the program is evidence based, it uses relatable lab-based content developed through focus groups, interviews, and interactions with researchers working in real laboratories. Given that relatable context supports intrinsic motivation, Gunsalus said, the program features engaging stories to increase attention to and retention of the information.

Discussion

Edward Hackett (Arizona State University) began the discussion by asking how to establish sufficient counter-pressure to keep people on the right path, given the intense pressure on faculty and students to perform. Gunsalus replied that she believes, and that there is evidence, that most people most of the time want to do the right thing. For her, the issue is that there will always be a small number of people who will misbehave and some other system for dealing with

___________________

24 https://labsthatwork.web.illinois.edu/

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

them may be necessary. For the majority of people who seek to do the right thing most of the time, her concern is creating create a system that provides them with information, support, reinforcement, tools, concepts, and ways they can push back against those pressures and deal with them effectively, as well as tools for working within the system to change it. “I see parts of this program as subversive because if we empower enough lab members with the same tools, concepts, vocabulary, and approach, at some point we cannot ignore them,” said Gunsalus. She added that she is interested in creating healthy laboratory environments, given that the lab is the smallest unit of health at a research institution. “The idea is that ethical precepts do not end at the lab walls, but should stretch across the campus,” she said.

Hackett then asked Gunsalus if the open science movement is affecting how she thinks about RCR. She replied that this is a generational conflict, that early career researchers are much more interested and committed to open science than the previous generation that benefited from a different system. The challenge is to provide “the tools to talk about those issues and resolve conflicts in a professional approach where people can have differences in a healthy way in a healthy lab climate” and that deals with the very real power differentials that exist in a lab. She noted that in the biomedical world, the National Institutes of Health has committed to open publishing and her program has built that into all of its storylines and professional development activities.

An attendee asked how Gunsalus chooses the videos to use, to which she replied that she and her colleagues created the two feature films used in the modules. One film, A Tale of Two Labs, shows “a troubled collaboration between a chemistry and a biology lab; so we start with how the chemists and biologists ‘other’ each other,” about their work. The module also includes the topics of communication and othering individuals in terms of gender, race, and school. The topics for discussion are rooted in real stories that she and her colleagues have heard in multiple interviews with people who have encountered these situations.

Kalichman asked how Gunsalus explicitly addresses the fact that the concepts participants will learn and hear may be very inconsistent with what they hear from the faculty around them. She reiterated that she sees this program as subversive because if it equips enough people with the same skills and concepts for raising hard problems, faculty will eventually confront a situation where they have to decide to change what they are doing to meet the students where they are and address the issues that concern them.

In response to a question about how the program measures transformative change and if it measures inclusiveness, Gunsalus replied that she has funding to develop an extension of an existing survey on organizational research integrity climates that will also measure civility, psychological safety, harassment, and inclusion in laboratories. She and her colleagues will then conduct a randomized trial where they will administer the modified survey instrument to labs that have had the intervention and others that have not. They are also seeking to conduct surveillance of labs on a routine basis to see what kind of environments exist in these labs.

Asked if the goal of transformative change addresses power differentials, Gunsalus replied that it does not and is meant to help people think about values, identification, and articulation, and about who they are, what they stand for, what difference do they want to make, and why they are pursuing their chosen careers and then equip them with the skills that matches their career as it unfolds. Regarding power differentials, they exist, and it is imperative to acknowledge them and recognize that “the elephant in the room is you have more power than I do, and I am really concerned with something you've asked me to do,” she said. The goal is to

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

provide ways that someone at the low end of the power scale can effectively deal with those issues. “We call our work relentlessly practical,” she added.

Gunsalus noted that one of the storylines addresses the issue of collaborating with people who come from different international backgrounds and who may approach thinking about what they are doing differently. She also pointed out how important it is to change the incentive system that rewards faculty to include professional development.

INSTITUTIONAL RE-ENGINEERING OF ETHICAL DISCOURSE IN STEM

Erica Baranski (California State University, East Bay) introduced the Institutional Re-Engineering of Ethical Discourse in STEM (IREDS)25 program that she and Plemmons managed at the University of California, Riverside. The goal of IREDS is to make engaging and meaningful discussions about ethical practices in research a daily part of research in STEM laboratories. Toward that end, the program promotes the idea that a culture of ethics and communication is an integral facet of productive and good research, and gives research teams a structured and facilitated opportunity to intentionally and explicitly discuss the ethical dimensions of specific practices or projects in their labs. The training uses online collaboration tools that lab members use to learn norms and communication strategies to discuss ethics and RCR within their laboratory and discipline.

Baranski noted that the IREDS program conducted a randomized, controlled trial to evaluate the effectiveness of the interventions designed to facilitate discourse about ethical practices and improve attitudes and behavior related to RCR. This trial invited 458 STEM labs at the University of California, Riverside and the Scripps Research Institute; 34 labs and 184 lab members joined the project. IREDS staff conducted pre- and post-surveys that covered objective lab practices, authorship decision practices, data management practices, and ethical research practices.

The intervention curriculum was delivered using a peer-engaged, decentralized approach in which graduate students from the enrolled labs worked with the team to select a project around which to build the training. The training consisted of two central elements. One was a demonstration of the OSF platform, which is a free digital system designed to enhance communication, transparency, and reproducibility in the sciences.26 The other element was a discussion about project-based ethics training using a peer-mentor structure and group discussion. Results of the trial, said Baranski, demonstrated that participants in the program, compared to those in the control labs, were more likely to change their behavior regarding authorship policies, change their views of ethical research practices, agree that ethics discourse is relevant to their scientific work, agree that the discourse in their labs was respectful and open to diverse views, and agree that discussions in the lab were more constructive in the sense that disagreement over research were reduced.

Baranski explained that those who participated in the training were also significantly more likely to understand the importance of and rationale for having authorship policies but did not significantly improve their understanding of data management policies. Participants also reported that they were more likely to archive their data and preserve replication materials underlying their research. In conclusion, she said, the training met its goals for fostering a better

___________________

25 The program is described in this article: https://www.pnas.org/doi/abs/10.1073/pnas.1917848117.

26 Additional information is available at https://osf.io/.

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

climate and deliberative communication in STEM labs compared to those who did not receive the training. Moreover, the training had a lasting effect based on observations made four-and-a-half months after the intervention. She noted that these results do not indicate what would have happened if the training had been compulsory across the entire institution, and added that a modified IREDS approach has been developed by the National Center for Principled Leadership and Research Ethics for scientists at the HHMI.

Discussion

Berne, as moderator, asked Baranski and Plemmons if they observed the nature of discourse in labs before beginning the training, to which Baranski replied that she and Plemmons did go into each lab over the course of the study, though not before the training began, and there were significant differences across labs and that the principal investigators led the discourse for the most part. However, as the training proceeded, discourse became more relaxed and inclusive of more members of the lab. Plemmons added that there was a parallel study conducted by an anthropology professor and graduate student in which they conducted in-depth interviews with some of the participants, including the principal investigator and lab members, and observed the labs as the researchers conducted their work. The findings from that study are forthcoming, and Plemmons hopes the results will contextualize some of their findings.

Kalichman asked if Baranski and Plemmons had any thoughts on why the intervention did not produce a significant increase in data management policies, but it did for authorship policies. Baranski replied that she believes it is because there is more nuance to data management that might be more specific to a laboratory, whereas authorship policy is more straightforward. Plemmons added that there was also a sense that principal investigators believe they have data management nailed down, given that it is part of being a scientist, and so they did not think they needed a plan to follow through. There was also some resistance to using the OSF platform, she noted, and it was a struggle to assure them that they were not trying to change their data management style to use that platform. The amended IREDS training has made changes to account for that resistance.

One attendee appreciated the idea of focusing on discourse and group deliberation given that people learn to talk about topics in ways they pick up from those around them. Baranski replied that this is one of IREDS’s strengths in that it provides labs a blueprint for how to have these conversations and creates momentum for having them. It also allows undergraduates and graduate students to confront these issues with their principal investigator with the confidence that they are doing so in a safe place where everyone has an equal voice.

Berne asked if there was any resistance to the training, and Baranski replied that some principal investigators were closed off and had the attitude that the IREDS team was “trying to shake up the lab dynamics and tell them what to do.” Once they realized that the training was meant to facilitate conversation built around existing practices in their lab, the discourse improved.

DATA STEWARDSHIP AT DELFT UNIVERSITY OF TECHNOLOGY

Yan Wang (Delft University of Technology) explained in her opening remarks that each of the eight STEM departments at her institution has it’s own disciplinary data management expert who helps researchers in their department with their data management. This team of data

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

stewards27 also helps the institution increase research transparency and improve RCR overall by influencing researchers’ behavior through culture change. Each data steward is a full-time member of the faculty, and Wang coordinates their efforts though the university’s central library, which funded the data stewards’ time in the beginning of the program, although costs are shifting to the individual departments.

The data stewards, said Wang, are the go-to individuals for researchers at all levels who have questions about collecting, processing, sharing, and managing their research data. Frequent questions include those about privacy issues and ethical issues when researchers are collecting personal data or when human test subjects participate in the research, as well as about how to share and publish data and how to settle authorship issues. She noted that the data stewards take a proactive, bottom-up approach that builds trust with the research community rather than dictating procedures (Cruz et al. 2019). This has led, over the past three to four years, to students and faculty seeing the data stewards as an integral and trusted part of research teams. In fact, researchers are now calling on data stewards for help planning their project, documenting and budgeting for data management, and building their digital skills. The data stewards also provide advice on how researchers can be better rewarded for good data management practices.

Wang described the research data management ecosystem at her institution. While researchers are core stakeholders in this ecosystem, the data stewards connect them with other teams at the university, including legal, ethics, and privacy experts; information and communication technology experts who support the university’s technical infrastructure, troubleshoot technology problems, and provide data management tools and platforms; grant officers, and the central library. “There is no clear divide between researchers and support staff,” said Wang. “We help each other to achieve better behavior in doing science.”

Concluding her remarks, Wang said that every department at Delft University of Technology, with support from their data stewards, has now approved its own research data management policy. The key to the program’s success, she added, was to engage with the researchers, focus on understanding their data management needs and developing solutions to fit them, expanding the conversation to include other aspects about their RCR behavior, and showing them how good data management practices, including open-access publishing, can come with professional rewards.

Discussion

Chavonda Jacobs-Young (Agricultural Research Service, U.S. Department of Agriculture) opened the discussion by noting that a 2013 presidential executive order required taxpayer-funded research data be available to the public. Since then, her agency has been working hard to develop the infrastructure to support that effort, and she applauded Wang’s institution for the way it has approached this challenge. In particular, Jacobs-Young liked that the data stewards have succeeded in getting researchers to include data management as part of the initial planning for a project, rather than as an afterthought. She then pointed out that her agency has perhaps 100 years of data in its archives and asked Wang if her institution had an opportunity for retrospective actions around previously existing data. Wang replied that this is a critical issue, particularly regarding orphan data for which there is no clear owner anymore. Currently, she and her colleagues are trying to ascertain who owns various tranches of data and determine whether

___________________

27 More information about the program is at https://www.tudelft.nl/en/library/research-data-management/r/support/data-stewardship.

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

there are any privacy or sensitivity issues regarding those data. Based on the results of those efforts, the next step will be to decide how to manage those data.

Wang responded to a question about the most useful elements to successfully change research by saying that the first step is to help researchers understand that all of their research data, regardless of form, are valuable and encourage them to take better care of those data. The second step is to incentivize and reward researchers for doing that work, particularly the laboratory assistants or software engineers who are often left off the list of authors. Asked to discuss the elements that define successful culture change in research, Wang replied that the mindset of both researchers and institutional management are key elements for success. She noted that changing culture at the management level is harder than at the researcher level.

Wang also said that the kinds of data-related issues that faculty members and graduate students face are similar, but at least in The Netherlands, there are more restrictions for graduate students because they are not paid staff, which raises issues about data ownership. Because faculty are employees of the institution, there are defined policies and principles regarding their data management practices.

CRITICAL THINKING AND RESEARCH ETHICS IN BIOMEDICINE AND BEYOND

In 2017, Gundula Bosch (Johns Hopkins Bloomberg School of Public Health) and her colleagues began to reform graduate student science education in light of what she calls R3: rigor, reproducibility, and responsibility.28 She suggested that efforts to improve the scientific enterprise include changes to graduate education, and the program was inspired by role models of science (e.g., Albert Einstein, Marie Curie, Charles Darwin) to focus on taking broader views of science, specializing less, and emphasizing the philosophical foundations of science and interdisciplinary science education. The resulting program does this by including rigor, reproducibility, and responsibility in all aspects of how faculty teach science by emphasizing the components of scientific accuracy—logic, research design, error analysis, intellectual honesty, and quantitative reasoning using probability and statistics—in a formal training program (Bosch and Casadevall, 2017; Casadevall and Fang, 2016).

The program itself is based on six core competencies: theory and practice of science, scientific error analysis, practical ethics, communication, methods and innovation, and quantitative reasoning. Bosch stressed that these are not individual courses, but core competencies that are the focus of several courses. The theory and practice of science competency includes a strong focus on the history of science, and it introduces the basics of epistemology, logic, ethics, and methods. Every course the program offers includes a focus on scientific error analysis and practical ethics, she added.

Bosch then discussed a recent NSF-funded study that evaluated a module on questionable research practices that was incorporated into the program’s interdisciplinary RCR course taken each year by more than 400 students from three divisions at Johns Hopkins (arts and sciences, public health, and engineering). The course’s aim is to help students recognize and prevent questionable research practices in science and engineering. She noted that while much of the RCR literature tends to focus on either stellar research practices or outright misconduct,

___________________

28 More information about the R3ISE program is at https://www.jhsph.edu/departments/w-harry-feinstone-department-of-molecular-microbiology-and-immunology/academics-and-degree-programs/R3-PhD-program/index.html.

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

questionable research practices are not so easy to define and differ across disciplines. Such practices include:

  • Hypothesis myopia, or looking only for data to support but not falsify a hypothesis,
  • Portraying patterns in data that are not present,
  • Asymmetric attention, or meticulously retesting data not expected but accepting the data that is expected without retesting it, and
  • Storytelling in a way that modifies the hypotheses or accommodates all the data.

The module starts by teaching students the fundamentals about the basic concepts of errors in different disciplines; students then apply those lessons to real world cases drawn from Retraction Watch and suggest ways to address those cases. The students also apply these lessons in a detective game in which they analyze an anonymized journal article from Retraction Watch and justify their reasoning in a debate. The capstone project has the students evaluate a publicly available preprint manuscript and write a critical review to the authors. Bosch and her colleagues have evaluated the program by asking students about how they are applying the skills they learn in the classroom to their laboratory and fieldwork. In the future, they hope to see how the students apply these lessons to their publications and presentations, how they engage in interdisciplinary projects, and via a survey of employers, how program graduates apply this knowledge in their professional lives.

Bosch noted in closing that since 2015, the R3 program has established a global network of partner institutions that are collaborating on Ph.D. reform projects that foster rigor, reproducibility, and responsibility. She also said that there was some concern among her colleagues that this program would increase graduate students’ time to complete their degrees. In fact, the opposite appears may be occurring. “We think that when our students have the opportunity to think about science from its first principles and have structured training in logic and critical thinking that they make fewer mistakes and more informed decisions that reduces the time to complete their degrees,” said Bosch.

Discussion

Kalichman noted that historically, principal investigators conducted this type of training and he asked Bosch why there was a need for a formal program. She replied that conversations with colleagues have suggested that principal investigators over the years have become more focused on teaching specialized subject matter and are spending less time on teaching the foundations of science. This program is meant to support principal investigators by providing a formal approach to teaching critical thinking to students before they enter the lab.

Asked to discuss some challenges and lessons learned from her experience with adapting this model to so many different international settings, Bosch replied that there have not been many challenges because the program is built on teaching science from first principles and those are the same regardless of discipline or setting. If there was a challenge, it was getting faculty to accept that this was important and needed. However, students were enthusiastic and now faculty are supportive, she added.

When asked if she was willing to share her educational materials, Bosch replied that anyone who joins the network has complete access to all materials, and eventually will make all of the materials available on an open-access platform.

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

NYU-UNIVERSITY OF GHANA RESEARCH INTEGRITY TRAINING PROGRAM

Kyle Ferguson (New York University Grossman School of Medicine) discussed the two specific goals of the program.29 The first aim is to produce a core group of expert researchers who have mastery of research ethics and integrity who will go on to lead and mentor international research teams, teach bioethics, review research protocols, and develop institutional and national research ethics policies. Currently, said Ferguson, the program is training 30 fellows, six of whom will pursue a master’s degree in bioethics at New York University. Those six fellows will then be eligible to play a role as faculty and achieve the program’s second aim, which is to establish a master’s degree program in bioethics at the University of Ghana School of Public Health with a specialization in research ethics. The second aim focuses on developing a curriculum for that program and engaging in faculty development.

Ferguson said the initial goal of the fellowship program was to enroll 24 fellows, but “we took on more when we were astounded by the interest in the program and the excellence of those who were applying to it,” he said. The 30 fellows—10 in each of three cohorts, the first of which started in the 2019-2020 academic year—take three intensive courses on the history and philosophy of research ethics, research integrity, and developing a collaborative research output where the research is different from the type of research the fellows do in their own scholarship. He noted that one promising aspect of the research integrity course, which does a deep dive on nuanced issues, is that it makes the case that establishing a culture of research integrity is essential for achieving the twin aims of protecting research participants and building trust in the research process.

The program starts with four weeks of Collaborative Institutional Training Initiative (CITI) program training to give the fellows a baseline or common ground on ethics. Next comes the 10-week history and philosophy of research ethics course that examines classic research ethics issues and provides some training in developing philosophical arguments and writing skills. Ferguson noted that while most of the fellows find that type of writing boring, it is a necessary skill for bioethics professionals. Faculty at the University of Ghana teach the last four modules of this course, which provides the fellows with an idea of how researchers express these issues and values in a local context. One deliverable from this course is an essay in which the fellows develop a philosophical argument for a position regarding a research ethics issue specific to the Ghanaian context.

The research integrity course follows over the next 10 weeks. Here, the fellows do deep dives on research integrity; questionable research misconduct; questionable research practices; research governance; research bias and incentives; data acquisition, management, and sharing; ethical issues in international research collaborations; peer review; and publication ethics. While taking this course, the fellows develop a research proposal where they envision a project that could shed light on how these issues are expressed in the Ghanaian context.

The final course in the sequence, developing a collaborative research output, pairs fellows in order to develop a pilot study proposal. The fellows work with mentors, and once they complete the course they engage in a six-month research project based on their pilot study

___________________

29 More information on the program is available at https://med.nyu.edu/departments-institutes/population-health/divisions-sections-centers/medical-ethics/education/nyu-university-ghana-research-integrity-training-program.

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

proposal. These research projects overlap with a three-month practicum experience that embeds them in an IRB, research ethics committee, or a research integrity office.

One unique feature of this program, said Ferguson, is that it not only focuses on research integrity training for individuals, but also on institutional and national research integrity policies and research governance. Those system or institution level mechanisms are what provide an ethical scaffold for producing knowledge and structuring incentives. He described a previous project that reviewed the research regulatory environment and ethics apparatus and assessed the perceived adequacy of the current institutional practices, opportunities, and incentives for promoting research integrity in Ghana (Laar et al. 2020). Currently, the fellows’ collaborative research projects provide a more detailed map of that landscape with projects such as:

  • Conflicts of interest in public health and nutrition: A content analysis of Ghanaian newspapers and policy documents
  • Assessment of the operational characteristics of Research Ethics Committees in Ghana
  • By which means shall Ethics Review Committees be funded in Ghana? Views from stakeholders
  • Biobanking, genomics research, and data analyses: Ethical considerations in Ghana
  • Infection disease research governance structures and guidelines in Ghana
  • Research misconduct: Perspectives of medical students at University of Cape Coast, Ghana
  • Guidelines to improve seeking informed consent among persons with mental illness in Ghana
  • Capacity enhancement of IRBs in Ghana
  • The role of research funders in promoting ethical international research collaborations
  • Ethical and policy implications of feedback of genetic findings to research participants in Africa: An analysis of the H3Africa guideline

The program has also established a one-year master’s degree program in bioethics at the University of Ghana—the country’s first graduate program in bioethics—after receiving approval in early 2021. The first application cycle began in the summer of 2021, with the first cohort scheduled to enroll for the 2021-2022 academic year. Going forward, the goal is to focus on strengthening that program to ensure that it will have a long and successful future.

Discussion

Rita Colwell (University of Maryland at College Park and Johns Hopkins University Bloomberg School of Public Health) opened the discussion by asking Ferguson if there are any topics that seem to most interest the students. He replied that in the first course, the students show a great deal of interest in developing guidelines for obtaining informed consent from vulnerable populations. In the research integrity course, there is a strong interest in authorship guidelines and other issues.

Colwell then asked Ferguson to elaborate on the program’s efforts to restructure the research integrity system in Ghana. He explained that Ghana does not have a national policy or legislation on research integrity, so local institutions develop their own policies. In many cases, those institutions defer to the policies of countries such as the United States, where they might have collaborations established or from which they receive funding. The program focuses on assessing those policies and practices in the Ghanaian context, occasionally authoring them for the first time. Ferguson noted that the program recruited and enrolled students from various

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

institutions across Ghana with the goal of having a nationwide reach. While the master’s degree program is centered at the University of Ghana, it is admitting students from across the country as well as neighboring nations. He added that although the COVID-19 pandemic affected plans to bring more students from Ghana to work in NYU laboratories and foster collaborations, the program’s second cohort is engaging in collaborative research projects within the department of public health.

Asked how the program prepares instructors for mutual cultural sensitivity and humility, Ferguson replied that the program instructors have built a good collaborative relationship with their Ghanaian colleagues. In addition, Ghana has real ownership of this project; Ghanaian faculty teach a number of the modules author research integrity policies. This approach, he added, grew out of mutual respect between the American and Ghanaian collaborators.

Responding to a question about future plans, Ferguson said the program is in the fourth year of its five-year grant30 and is preparing a renewal proposal to expand the program to accept fellows from neighboring English-speaking countries such as Liberia, Sierra Leone, The Gambia, and Nigeria.

BREAKOUT DISCUSSIONS

Once presentations on RCR exemplars were complete, the workshop participants joined breakout groups to discuss the most promising programmatic elements they heard in those presentations, how they might implement those in their work, the barriers they anticipate when implementing those elements, the outcomes they will measure in assessing their efforts, and other topics.

Several groups discussed assessment and its challenges. For example, the R3 project measures time to completion of the Ph.D. and number of publications, has the trainees evaluate the course in terms of how well it achieved its objectives, and also conducts more traditional assessments of learning. Another group discussed the notion of having continuous assessments of the students’ performance and the style and effectiveness of the communication and outreach methods used by training leaders, noting that most programs currently use quantitative measures, but much of the work would be better assessed using qualitative measures, which are harder to implement.

However, groups noted that many of the goals of all RCR programs are long term and somewhat ethereal, which creates a challenge for assessment. One group discussed specific and measurable sub goals as a way to improve assessment in relation to those longer-term goals. Another group discussed increasing formative qualitative assessment using focus groups.

Another theme discussed in the breakout sessions was challenges, such as overcoming resistance and getting support and engagement from faculty and other gatekeepers who are leery of participating in an ethics projects or acknowledging that international graduate students in STEM might have different cultural understandings of ethics. Discussions also focused on the resources, expertise, and time involved to create and sustain high-quality RCR programs, including necessary resources for continued practitioner education, finding faculty members who are willing to become instructors, and engaging potential participants.

Several groups also described strategies for successful implementation, such as winning trust by including faculty and practitioners in the development process, pointing out mutual

___________________

30 Information about the grant is available at https://reporter.nih.gov/project-details/10331848.

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

benefits of participating in the program, breaking down complex content into manageable portions related to practice, or bringing up relevant issues when the timing is right. One group noted that the educational research literature has shown that small efforts can have big effects, which suggests that program leaders find the leverage points and use them to amend their programs. In that same vein, keeping goals small and manageable can be a crucial factor in developing a successful program, and storytelling across stakeholder groups can engage students and faculty.

Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 21
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 22
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 23
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 24
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 25
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 26
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 27
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 28
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 29
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 30
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 31
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 32
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 33
Suggested Citation:"6 RCR Training Exemplars." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 34
Next: 7 Creative Approaches to RCR »
Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop Get This Book
×
 Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop
Buy Ebook | $14.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The responsible conduct of research (RCR) is foundational to good science and engineering. However, there are several serious detrimental effects of research not conducted ethically and responsibly. Thus, federal legislation mandates that all National Science Foundation (NSF) grant recipients provide adequate training for undergraduate students, graduate students, and postdoctoral researchers about the Responsible Conduct of Research, and the National Institutes of Health (NIH) further specifies that topics such as conflict of interest, mentor/mentee responsibilities and relationships, collaborative research, and safe laboratory practices should be included in all instruction in RCR.

To identify and disseminate information about exemplary RCR education practices and programs, the National Academy of Engineering convened a virtual workshop series October 6, 7, 20, and 21, 2021. This workshop provided a forum for interdisciplinary discussions of effective strategies for building an RCR culture in institutions along with potential improvements and further research in the area. It also highlighted selected exemplar RCR programs for administrators and faculty who are charged with RCR education and compliance at the institutional level. This publication summarizes the presentation and discussion of the workshop.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!