Practical Guidance on Science and Engineering Ethics Education for Instructors and Administrators: Papers and Summary from a Workshop December 12, 2012 (2013)

Institutional Strategies for Effective Research Ethics Education:
A Report from the Council of Graduate Schools

JULIA D. KENT
Director, Global Communication and Best Practices
Council of Graduate Schools (CGS)

Introduction and Background on CGS Initiatives

For nearly a decade the Council of Graduate Schools (CGS), an organization devoted to advancing graduate education and research, has worked with US universities to enhance the preparation of graduate students for the ethical challenges and responsibilities of scholarship and research.¹ This work has responded to CGS member institutions’ desire to effectively prepare graduate students to conduct research responsibly and to ensure the quality of research conducted at their institutions. CGS’s collaborative research with member institutions is also motivated by recognition that the current research environment is creating new challenges for researchers. These include, to name only a few, increasing pressures to produce publications and quantifiable research outputs; the interlinking of research sectors (academic, commercial, government) that may have different expectations about the outcomes of research; and the globalization of research, which requires researchers to navigate different research norms and policies, and to identify situations where norms for research practice may not be transparent (CGS 2012b).

Since 2003 CGS has granted subawards to 22 universities (and worked with an additional 44 affiliate universities and colleges)² to create graduate education programs and resources for the responsible conduct of research (RCR). In 2004–2006, through a contract with the DHHS Office of Research Integrity (ORI), CGS worked with ten institutions to develop and test interventions and assessment strategies for the training of graduate students from the behavioral and biomedical sciences in the responsible conduct of research. The results of this project were published in Graduate Education for the Responsible Conduct of Research (CGS 2006). In 2006–2008, supported by a two-year grant from NSF, CGS worked with eight institutions to develop interdisciplinary programs in research ethics for students in science and engineering (S&E), a project that resulted in Best Practices in Graduate Education for the Responsible Conduct of Research (CGS 2009).

¹ For an overview of CGS initiatives in the areas of research and scholarly integrity, see www.cgsnet.org/scholarly-integrity-and-responsible-conduct-research-rcr.

² Many affiliate universities chose to fund and implement part of their proposed projects. In addition, they joined project activities such as PSI discussions and CGS Annual Meeting and Summer Workshop sessions, and implemented institutional assessment activities using the common assessment tools developed for the project.

Building on the recommendations and lessons learned from these two initiatives and with funding from ORI, CGS launched the Project for Scholarly Integrity (PSI) in 2007.³ The council worked first with an advisory committee and then with six US universities to define and develop a framework for a comprehensive institutional approach to research and scholarly integrity that was then pilot-tested by the six universities. A monograph on the project, Research and Scholarly Integrity in Graduate Education: A Comprehensive Approach, was published along with an online data dashboard (CGS 2012b).

This paper focuses on the institutional strategies used in these projects to develop (1) effective research ethics and RCR education programs on US campuses and (2) resources and tools that may be useful to both administrators and instructors in research ethics programs. Special attention is given to the lessons learned and best practices developed through the Project for Scholarly Integrity, offering successful models for communication and collaboration between graduate schools and other campus leaders and entities such as graduate program directors, college deans, directors of centralized RCR programs, research integrity officers, graduate student organizations, and other stakeholders.

Next, I share the goals of a current CGS project, Modeling Ethics Education in Graduate International Collaborations, funded by NSF’s Ethics Education in Science and Engineering (EESE) program (NSF #1135345). This project may be of particular interest to both instructors and administrators of ethics education programs in science and engineering because it addresses a widely recognized gap in research integrity and ethics training for graduate students: the need to prepare graduate students at US institutions to manage the unique challenges and questions that arise in international research collaborations.

The terms “research integrity,” “RCR,” and “research ethics” are used throughout this paper, each in a specific way. “Research integrity” encompasses a broad range of positive attributes of researchers and institutions that are incorporated in programs, institutional processes, and training methods designed to instill aspirational qualities associated with honesty in research. “RCR” training, an important component of all institutional PSI projects, refers to the Office of Research Integrity’s definitions of the term; however, in cases where RCR training was included in programs designed to promote positive qualities of researchers more broadly, “research integrity” is used. “Research ethics” is a broad term that has a variety of uses. In the Project for Scholarly Integrity, it refers to a definition frequently used by educators—the principles that help people adjudicate and make decisions when values may be in conflict. The CGS monograph on the Project for Scholarly Integrity provides more detailed definitions of these terms in the context of the project (CGS 2012b, pp. xvi–xix).

The Role of the Graduate School in Supporting Research Ethics Education

Graduate schools play an important role in fostering interdisciplinary, cross-campus collaboration in research ethics education. The previous CGS projects present examples of how graduate schools have effectively brought together multiple campus units and program faculty with complementary areas of expertise. Graduate deans have provided strong leadership and support in assessing vulnerabilities, identifying needs, and supporting the faculty-led development of curricula and activities targeted to meet those needs and vulnerabilities.

At many partner institutions, collaboration between graduate schools and research ethics and RCR programs have resulted in a hybrid program design. Resources and activities are distributed

³ Information about the PSI is available online at www.scholarlyintegrity.org/ShowContent.aspx?id=78#.

between centralized sources, on the one hand, and program sources on the other, including coursework and in-lab activities. Such a distributed model of research ethics education provides students with both general and field-specific skills and knowledge, supports sustainability, and furthers campus integration.

The value of this hybrid design is supported by research that calls for closer attention to the role of institutional environments in supporting (or hindering) RCR and research integrity education.. In medical fields, the need to address the institutional systems that foster integrity is well established. In 2002, an influential Institute of Medicine report endorsed an “open-systems model” to conceptualize the dynamic relationship between the different elements of a research organization that contribute to a climate of integrity. The knowledge, attitudes, and behaviors of an institution’s members are strongly tied to distinct aspects of organizational structure, such as mission, goals, and strategies for promoting research integrity, as well as the processes used to support those goals through strong leadership, communication, and socialization of members around this issue (IOM 2002).⁴ Focusing on units of institutional culture in other S&E fields, Melissa Anderson has argued that greater attention must be given to the ways labs and research groups socialize and reinforce the ethical behaviors of their members, from students to senior scientists (Louis et al. 2007).

CGS projects have been successful because they leverage the support of graduate schools, which can promote culture change at the organizational level as well as in departments and programs where research and research training take place. The CGS experience suggests that this comprehensive, integrated approach to research ethics education is the most effective approach in terms of gaining the broad faculty input necessary to ensure relevance and meet student needs.

The Project for Scholarly Integrity

A Framework for Collaborative Action

The PSI was guided by a Framework for Collaborative Action (CGS 2008), developed by a planning committee tasked with identifying core components of an institutional approach to supporting and advancing research integrity in graduate education. The goal of the framework was to support programs that were comprehensive in scope, sustainable, and responsive to a broad range of needs and issues. In the context of the PSI, a “comprehensive” approach went beyond providing training programs that were isolated and/or not reinforced by graduate training in the disciplines.

The planning committee’s five-part framework was intended to provide institutions with the flexibility to develop activities that were well suited to the needs of their graduate communities, while also creating a structure of collaboration that would encourage the exchange of ideas and promising practices among institutions. Each institution that submitted a proposal to CGS for funding developed a plan to:

1.   engage the community in identifying needs,

2.   invite campus stakeholders to reflect on a plan for action,

3.   act on those reflections (put the plan into motion, implement project activities),

4.   communicate to the broader community about activities and their ongoing impacts, and

⁴ Thrush et al. (2007, 2011) have used IOM’s model as the foundation of a climate survey instrument, a pilot version of which was adopted by all awardees for CGS’s current PSI project and administered across all science and engineering fields.

5.   integrate activities to ensure greatest impact and sustainability.

It is important to note that these steps did not always occur in sequence. For example, integration of PSI activities into program activities and existing university resources was a key part of many projects and was planned and executed throughout the projects, not necessarily as a final step.

Through a competitive award process, an external advisory committee chose six institutions to participate in the project as awardees:

•   Columbia University

•   Emory University

•   Michigan State University

•   Pennsylvania State University

•   The University of Alabama at Birmingham

•   The University of Arizona

The next sections highlight some of the successful strategies used by institutions in the first three areas of collaborative action.

Step One: Engage the Community in Identifying Needs

One of the central goals of the PSI project was to promote the recognition that research integrity is a topic that concerns all members of the graduate community—administrators, faculty researchers and supervisors, graduate students, RCR program directors, and beyond. Working in the context of the Framework, institutional awardees engaged a broad range of campus partners for frank discussions of campus needs. Two Framework principles guided their work: finding opportunities to recognize vulnerabilities in the graduate community and rewarding excellence in upholding high standards and value for research integrity.

Some of the most effective strategies for communicating with campus groups about the importance of research integrity reflected serious consideration of the specific goals of graduate education and the interests of graduate students and faculty. For example, (1) project messages were presented in academic, intellectual contexts, such as an invited speaker series or a faculty-led workshop on ethical issues in research, and (2) the relevance of PSI activities was communicated to specific disciplinary units or programs. These strategies helped institutions emphasize that research integrity requires high-level, learned skills and that it directly impacts the quality of research.

Step Two: Invite Campus Stakeholders to Reflect on a Plan of Action

While institutions used different approaches for organizing their campuswide activities, common strategies included appointing a planning or steering committee with a variety of representatives from across campus, appointing a project director, and creating neutral forums for discussion and evaluation.

Also key to this stage of the process was creating partnerships and alliances that could support the goals of the project in an ongoing way or at key points. Graduate deans and project staff used this stage to both reinforce existing relationships and create new ones. Their reach was quite broad, and included

•   coordinators for professional development programs for faculty or students

•   student and postdoctoral associations

•   ethics centers

•   research offices

•   research integrity or compliance offices

•   interdisciplinary research centers, and

•   graduate student associations

These relationships provided a foundation for creating campuswide buy-in for the project and expanding project resources.

Step Three: Put the Plan into Motion (Implement Project Activities)

The implementation phase, focused on educational activities for graduate students, required thoughtful consideration on the part of all institutions. The monograph on the PSI project (CGS 2012b) addresses three concerns taken up by project awardees prior to and during this phase: (1) developing the right curricular content, (2) determining the sequencing of content and pedagogy, and (3) building collaborations to extend the reach of research integrity programs.

One model for curriculum development created by the University of Arizona established a small grants program that invited students to partner with faculty mentors to develop courses and lessons in research integrity. These grants not only provided opportunities for institutions to engage graduate students and faculty but also recognized excellence in research integrity education (awardees were acknowledged in a “Grantees Showcase” and contributed to an online campus repository of RCR resources).

Several other institutions developed required, hybrid programs for research integrity that were led both by faculty in the disciplines and by central research integrity programs. Examples of the ways in which institutions developed and sequenced content are discussed in detail in the CGS monograph (CGS 2012b).

PSI Assessment Activities

PSI awardees administered two assessment tools as part of their institutional projects: a Research Integrity Inventory Survey, which collected data on campus resources and activities, and a prevalidated version of the Survey of Organizational Research Climate (SORC) developed by Lauren Crain, Brian Martinson, and Carol Thrush, working in collaboration with a consortium of awardees composed of three institutions: Michigan State University, Pennsylvania State University, and the University of Wisconsin–Madison (Thrush et al. 2007, 2011). Institutions also worked independently to develop learning assessments appropriate to curricular activities on their campuses.

Findings from the PSI Research Integrity Inventory Survey

The PSI data confirmed findings from the earlier project (CGS 2009): there is a gap between the way program faculty perceive the training they are providing to students in research and scholarly integrity and the training that students say they are receiving.

S&E students receive information through a wide range of activities and resources, such as online training modules, required and elective courses in the program or graduate college, program courses that incorporate research ethics issues (e.g., research methods), orientation programs, workshops, seminars and speaker series. Among individuals responding to the Research Integrity Inventory Survey on behalf of 240 graduate programs or departments, nearly four of every five (78 percent) reported that students in their programs receive information on

research ethics issues from advisors and mentors, whereas 50 percent or fewer reported that students receive this information through other means, such as coursework, workshops, or online and print materials. (When we analyzed data for physical sciences, engineering, and mathematics separately, respondents reported that students were three times more likely to receive information through advising and mentoring than through any other modality.)

In contrast, graduate students participating in focus groups for the project reported that they receive much of the information they need about research ethics and integrity from sources other than their advisors and mentors and that, depending on their supervisor, do not always find that the mentoring relationship provides adequate guidance on issues of research integrity.

In the context of international research, some students describe “trial by fire” situations in which collaborative or field research with international partners or in another country involves challenges that have been unanticipated by students’ research advisors in courses, supervision, or the grant project design.⁵

The PSI findings indicate a need to provide graduate students with multiple touch points for research integrity education. A second and more promising finding is that students are eager to receive this preparation. The institutions that participated in the PSI as awardees and affiliates indicate that students are very interested in research integrity education that is delivered both centrally and in departments, especially when these opportunities are tied to their professional development as researchers.

An in-depth analysis of the Research Integrity Inventory Survey and the climate survey, the methods used in administering them, and the assessment results can be found in Part III of Research and Scholarly Integrity in Graduate Education (CGS 2012b).

PSI Data Dashboard

As a companion to the PSI monograph (CGS 2012b), CGS has developed a Benchmarking Tool that enables member institutions to compare aggregate data collected by CGS from awardee institutions using data from the two surveys.⁶

Modeling Effective Research Ethics Education in Graduate International Collaborations: A Learning Outcomes Approach

CGS is conducting a project that will result in the development of institutional models for preparing graduate students to confront the broad range of ethical issues that typically arise in international S&E research and educational collaborations.

In April 2012, CGS invited US member institutions to submit proposals that address issues of research ethics and research integrity encountered in international S&E research collaborations and exchanges as well as joint or dual degree programs. The selection criteria encouraged these institutions to also address one or both of two priority areas: (1) those faced by graduate students conducting field research in international settings, and (2) those that international graduate students frequently encounter in US programs.

The project takes an innovative, “learning outcomes” approach to supporting the education and development of graduate students. Learning outcomes, a concept developed and refined in a

⁵ For analyses of PSI survey data and semistructured discussions with S&E students and faculty on multiple campuses, see CGS (2012b).

⁶ Information about the dashboard is available online at www.cgsnet.org/benchmarking/best-practices-data/PSIdashboard.

large body of Scholarship of Teaching and Learning (SoTL) research, are explicit statements of generic skills, abilities, and disciplinary competencies that a student is expected to have acquired by successfully completing a course, a program, or other activities including cocurricular experiences.

Through the research and educational activities of this project, CGS will engage faculty, experts, and universities in (1) defining the discrete knowledge, skills, and behaviors that are especially valued in the careers of scientists and engineers in their fields, and (2) using these desired outcomes to develop curricular content, assess student understanding, and improve educational programs. Such an approach will make it possible to address a well-documented gap in understanding of the outcomes of international research experiences. It will also improve knowledge of the effectiveness of research ethics education in an international context.

The project is intended to enhance the US S&E graduate community’s understanding of the effectiveness of different approaches to integrating research ethics education in international collaboration and integrating international issues in research ethics programs. The project will result in three types of resources: (1) five model sets of learning outcomes that identify research ethics skills and abilities for graduate students in international collaboration, addressing issues typical of different disciplinary and international collaborative contexts; (2) at least five case studies that describe how these outcomes are being used to evaluate and enhance both research ethics education and international collaborations at the graduate level; and (3) an online repository of graduate learning outcomes for international collaborations. A CGS template for developing learning outcomes will be shared with US universities and offered for consideration and use to other CGS member institutions, along with a preliminary framework for incorporating a learning outcomes approach into graduate education. Five institutions have been selected to participate in the project: Emory University, Northern Arizona University, the University of Puerto Rico–Rio Piedras, the University of Oklahoma, and Virginia Tech.

Institutions and ethics instructors can find additional resources on the CGS website (under Selected Resources on Research Ethics Education in International Collaborations), organized into six broad categories: Research Ethics Issues in International Collaborations, Research Ethics for US Scholars Abroad, Research Ethics in Graduate Education, Research Ethics Education for International Graduate Students, Integrating and Assessing Research Ethics Education, and Other Resources.

Conclusion

In my overview of the CGS Project for Scholarly Integrity, I have highlighted several key ingredients of characteristics of successful institutional efforts to promote research ethics and RCR education at US institutions.

The first of these is engaging the leadership of institutions to support the goals of individual programs. (This is advice that graduate leaders not only recommended but also, in the context of individual PSI projects, practiced. For example, to gain broader support for their PSI initiatives, graduate deans engaged the office of their university president or the vice president for research.) Seeking the support of the graduate school and other university administrators has proven to be an effective way to generate campuswide support and to construct a thoughtful institutional strategy. A campuswide approach can also be critical to problem solving and overcoming obstacles that one campus program or unit cannot solve on its own, and to finding cost-effective solutions that pool the resources of different groups on campus.

Second, CGS and its project partners learned that assessment is a critical part of any sustainable and effective program in research integrity education. It is particularly important for engaging faculty in the disciplines, whose curricula, research practices, and mentoring habits have great power to reinforce (or undermine) the value of research integrity to their graduate students. Research and Scholarly Integrity in Graduate Education describes several examples of the ways in which institutions used results of the common assessment activities to initiate conversations with faculty about graduate student needs.

Program faculty clearly have an important role to play in any comprehensive approach to research ethics education, and it is important for universities to demonstrate the need for their direct engagement and disciplinary expertise.

References

CGS [Council of Graduate Schools]. 2006. Graduate Education for the Responsible Conduct of Research. Washington.

CGS. 2008. The Project for Scholarly Integrity in graduate education: A framework for collaborative action. Available online at www.cgsnet.org/ckfinder/userfiles/files/PSI_framework_document.pdf.

CGS. 2009. Best Practices in Graduate Education for the Responsible Conduct of Research. Washington.

CGS. 2012a. Modeling Effective Research Ethics Education in Graduate International Collaborations: A Learning Outcomes Approach. Washington.

CGS. 2012b. Research and Scholarly Integrity in Graduate Education: A Comprehensive Approach. Washington.

IOM [Institute of Medicine]. 2002. Integrity in Scientific Research: Creating an Environment That Promotes Responsible Conduct. Washington: National Academies Press.

Louis KS, Holdsworth JM, Anderson MS, Campbell EG. 2007. Becoming a scientist: The effects of work-group size and organizational climate. Journal of Higher Education 78(3):311–336.

Thrush CR, Vander Putten J, Rapp CG, Pearson LC, Berry KS, O’Sullivan PS. 2007. “Content validation of the organization climate for research integrity (OCRI) survey.” Journal of Empirical Research on Human Research Ethics 2(4):35–52.

Thrush CR, Martinson BC, Crain AL, Wells JA. 2011. User’s manual for the Survey of Organizational Research Climate. Available online at https://sites.google.com/site/surveyoforgresearchclimate/.

Note
This paper includes selected portions of the framework paper that guided the development of proposals to the EESE project, Modeling Effective Research Ethics Education: A Learning Outcomes Approach (CGS 2012a), a CGS report prepared for publication by Daniel Denecke and Julia Kent. The author would like to acknowledge the contributions of Daniel Denecke, PI for CGS’s PSI and NSF EESE projects, and Jeffrey Allum, a coauthor of the PSI monograph, in the preparation of this paper.