National Academies Press: OpenBook
« Previous: 1 Introduction
Suggested Citation:"2 The Dual Challenge for RCR: Minds and Systems." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

2
The Dual Challenge for RCR: Minds and Systems

Brian Nosek (Center for Open Science, University of Virginia), began his talk11 stating that researchers’ RCR is constrained by two things: their own their self-perception of control over their own behavior and the social context that also influences individual behavior. Given those two constraints, effective RCR should attend to systems level solutions, not just rely on knowledge and good intentions of researchers.

Humans use information about the world around them to construct a perception of that world to make behavioral decisions. However, said Nosek, reality and the perception of reality constructed by individuals are not the same thing, because incoming information is shaped by one’s prior experiences as well as cognitive shortcuts that can lead a person away from their intentions, values, and goals. The reason for this, he explained, is that humans do not always recognize that they are using these shortcuts to make sense of the world around them.

“We do not get to decide how we perceive the world, as we have a variety of automatic, unconscious processes that provide that experience after it has been filtered through lots and lots of information,” said Nosek, adding “much of our general reasoning is subject to those same kinds of constraints and biases that influence how we understand and consume information and understand each other” in ways that may not accurately reflect reality. As an example of how context affects perception, Nosek showed a visual illusion developed in 1995 by Edward Adelson.12

Individuals also exist in a system of universities, publishers, funders, and societies that shape what is valued, rewarded, and incentivized in a research career. The system shapes what work is valued, rewarded, and incentivized for researchers to progress in their careers. This leads to the one core issue for RCR, said Nosek: the incentives for an individual’s success focus on getting their research published, not on getting it right.

Most researchers, he said, go into science because they want to do good work, solve problems, and make discoveries, not to write research papers and grants. The reality, however, is that authoring research papers and grants is how a researcher maintains and thrives in their career, and those activities are not necessarily reinforced by the same behaviors in that positive results and novel findings are more publishable and more likely to secure funding than negative results and replicating prior findings. In other words, incentives for individual success are focused on getting published, not responsible research conduct (Nosek et al. 2012). Because publication is the currency of reward, behaviors can change that lead to questionable research practices, selective reporting of results, a lack of transparency and sharing, and failure to

___________________

11 The presentation slides are available at https://osf.io/ynb7v.

12 Illustrated by an animation here: https://michaelbach.de/ot/lum-adelsonCheckShadow/. Slides 4 and 5 of Nosek’s presentation show the illusion without animation.

Suggested Citation:"2 The Dual Challenge for RCR: Minds and Systems." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

reproduce results that reduces the credibility of the findings, sometimes without the researcher even understanding that they are doing something wrong. Consequently, the credibility of the published literature suffers, self-correcting processes do not function, and valuable resources are wasted.

BARRIERS TO RCR

Nosek then discussed the barriers to RCR and enabling the social system to reward researchers for ethical behaviors while avoiding the biasing influences that might change how they approach their research and enable others to analyze their work. One barrier is a lack of accountability (Lerner and Tetlock 1999) given that the academic community at large does not see what occurs in a given laboratory, just the resulting publication. Addressing this barrier would expand the notion of what researchers share when they publish their research to include more information on the protocol, methodology, materials the researchers used to execute their work, and the data the experiments generated. Making this information available creates an environment where self-correction is possible, skeptical responses can address more specifically potential issues, and others can more easily reproduce and extend findings.

A second barrier is motivated reasoning, in which humans “can rationalize any kind of behavior that is in [my] self-interest as being the right thing to do,” said Nosek. Two examples of this motivated reasoning are confirmation bias and hindsight bias (Kunda 1990). Confirmation bias leads a researcher to orient their future research to confirm their initial finding rather than challenging it. “That is a natural response and hard to avoid,” said Nosek. Hindsight bias occurs when a researcher observes something unexpected, and once they have seen that they can make perfect sense of it and develop a story as if they expected that result all along, even if the result might have occurred by accident. One approach to reducing motivated reasoning is preregistration, a process in which the researcher states in advance the study’s goals and hypotheses, including what the researcher expected to find. At the conclusion of the research, they explain what they discovered, and even for an exploratory study in a new research topic, making hypotheses explicit prior to conducting the research can then help calibrate confidence in findings the researcher reports after the fact. In fact, research has shown that preregistration can reduce motivated reasoning and address the challenges of the bias toward positive results (Scheel et al. 2021).

Perceived norms represent a third barrier to RCR. Nosek cited a study that asked U.S. scientists if they endorsed transparency over secrecy, self-skepticism over dogmatism, and a focus on quality rather than quantity (Anderson et al. 2007). Over 90 percent of the respondents said they agreed with those core values and that they live by those norms. However, when asked if others in their field behave according to those norms, between 60 and 75 percent of the respondents said either that others in their field do not or that the culture does not reinforce those values and norms of science. “The challenge is that while the norm already exists in people’s values, it is not manifest in the culture,” said Nosek, which he said is the definition of a dysfunctional culture in which everyone has one set of values and almost nobody sees that the culture aligns with those values.

The easiest way to change that situation, he said, is to make visible those desired behaviors and increase the visibility of people behaving in ways that are valued. “That can very rapidly shift the norms in a community,” said Nosek. Journals, for example, can implement this change by adding badges to papers for which the researchers have shared their data and materials

Suggested Citation:"2 The Dual Challenge for RCR: Minds and Systems." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

or preregistered their study so that readers can see others behaving according to the desired norms (Kidwell et al. 2016). He noted that when the journal Psychological Science implemented badges for this behavior in 2014, the proportion of articles that had open data soared to 40 percent over the next 18 months and now stands at more than 80 percent.

The final barrier to RCR is that all researchers have a lot of work and many competing demands on their time, which Nosek said is an underappreciated but huge barrier to adopting rigor, transparency, and integrity and improving behaviors. The key here is to meet researchers where they are, to provide them with tools and resources that are concrete, actionable, and integrated with their current workflow. “There is a need to adapt how we conduct interventions for RCR so that they are embedded in the context in which researchers are doing their work,” said Nosek.

DISCUSSION

When asked how bias against publishing null results affects RCR, Nosek said that a 1975 study estimated that approximately 35 percent of published findings were unlikely to be reproducible and more likely to be false positive results as a consequence of only publishing statistically significant results (Greenwald 1975). Preregistration with a guarantee that the journal would publish the resulting paper even if the results did not support the hypothesis is one approach to addressing this problem. Preregistration can work with exploratory research if the investigators are more explicit about when a study will be exploratory rather than confirmatory. One problematic aspect of today’s research culture is that calling a study exploratory can undermine its value, which Nosek said is counterproductive given how important exploratory research is to the advancement of science. This attitude incentivizes researchers to write papers as if they had a theoretical context in mind to explain the results, which dresses up exploratory research as a confirmatory finding.

David Allison (Indiana University Bloomington School of Public Health) raised an issue that he has confronted when publishing data to accompany a paper, which is that people look for mistakes and then accuse him of making that mistake intentionally. To him, it seems that attitude is a disincentive to publishing data along with a paper. Nosek acknowledged that this is a risk, but he sees it as a temporary one that will vanish when sharing data becomes the norm those who do not share data risk their own credibility. One step Allison takes is to not allow his group to submit a research paper until a professional statistician in his group examines all the data and reproduces every number in every line of text and every figure in every table. “We have never yet, not once in dozens of papers, failed to detect an error that the first analyst made,” said Allison. “The errors tend to be small, but they are still errors.” Nosek finished by saying that all of these efforts take work and resources by the research groups and those groups that do not have the resources will just have to accept error spotting by the community as a good thing, not an attack.

Suggested Citation:"2 The Dual Challenge for RCR: Minds and Systems." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×

This page intentionally left blank.

Suggested Citation:"2 The Dual Challenge for RCR: Minds and Systems." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 5
Suggested Citation:"2 The Dual Challenge for RCR: Minds and Systems." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 6
Suggested Citation:"2 The Dual Challenge for RCR: Minds and Systems." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 7
Suggested Citation:"2 The Dual Challenge for RCR: Minds and Systems." National Academies of Sciences, Engineering, and Medicine. 2023. Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/27085.
×
Page 8
Next: 3 Reproducibility and Data Reuse »
Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop Get This Book
×
 Promising Practices and Innovative Programs in the Responsible Conduct of Research: Proceedings of a Workshop
Buy Ebook | $14.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The responsible conduct of research (RCR) is foundational to good science and engineering. However, there are several serious detrimental effects of research not conducted ethically and responsibly. Thus, federal legislation mandates that all National Science Foundation (NSF) grant recipients provide adequate training for undergraduate students, graduate students, and postdoctoral researchers about the Responsible Conduct of Research, and the National Institutes of Health (NIH) further specifies that topics such as conflict of interest, mentor/mentee responsibilities and relationships, collaborative research, and safe laboratory practices should be included in all instruction in RCR.

To identify and disseminate information about exemplary RCR education practices and programs, the National Academy of Engineering convened a virtual workshop series October 6, 7, 20, and 21, 2021. This workshop provided a forum for interdisciplinary discussions of effective strategies for building an RCR culture in institutions along with potential improvements and further research in the area. It also highlighted selected exemplar RCR programs for administrators and faculty who are charged with RCR education and compliance at the institutional level. This publication summarizes the presentation and discussion of the workshop.

READ FREE ONLINE

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!