Fostering Integrity in Research (2017) / Chapter Skim
Currently Skimming:
Appendix C: Assessing the Effectiveness of Responsible Conduct of Research Training: Key Findings and Viable Procedures
Appendix C: Assessing the Effectiveness of Responsible Conduct of Research Training: Key Findings and Viable Procedures
Pages 245-264
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 245... ...
It is concluded that systematic evaluation of educational programs not only allow for the appraisal of instructional effectiveness but also allows for progressive refinement of educational initiatives. Ethics in the sciences and engineering is of concern not only because of its impact on progress in the research enterprise but also because the work of 1 As the committee launched this study, members realized that questions related to the effectiveness of Responsible Conduct of Research education programs and how they might be improved were an essential part of the study task.
|
From page 246... ...
Attempts to answer these questions and improve RCR instruction must ultimately be based on systematic program evaluation efforts. Accordingly, our intent in the present effort is to examine the evaluation of RCR educational programs to both determine what we know about the effectiveness of instruction and how we might go about improving RCR instruction.
|
From page 247... ...
As a result, the measures used to appraise the effects of one instructional program may not be identical to the measures used to appraise the effects of another instructional program. Although a variety of measures may be used to appraise the effects of RCR instruction, it is critical the measures employed evidence adequate reliability and validity (Messick, 1995)
|
From page 248... ...
In the following section we will consider each of these issues in the context of RCR training. The principle use of evaluation data is determining whether the RCR instructional program did result in change on the measures being used to appraise program effects.
|
From page 249... ...
The performance measures used in evaluation of ethics instruction do not focus on real-world ethical performance or breaches in ethical conduct in part because of the frequency of such events and in part because of ethical concerns attached to measuring such events. Rather, to assess performance, low-fidelity simulation measures are used (Motowidlo et al., 1990)
|
From page 250... ...
In the case of RCR training evaluation, this domain may reflect ethical knowledge in general, ethical knowledge applying to a particu lar field, or ethical knowledge specifically provided in training. These differing frameworks for generating knowledge items result in differences in the generality of the conclusions flowing from evaluation studies.
|
From page 251... ...
A final approach that might be used to appraise the effectiveness of RCR instruction may be found in organizational outcomes. For example, a drop in the number of ethics cases brought to university officials following introduction of an RCR program represents one such measure.
|
From page 252... ...
EVALUATION OF RCR TRAINING Meta-Analyses Although a variety of measures are available for evaluation of RCR training, systematic evaluation of the effectiveness of instruction has been sporadic. Some programs have been evaluated while others have not.
|
From page 253... ...
By the same token it should be recognized that these studies have focused on performance criteria. Although use of performance criteria is desirable, it should be recognized that these findings do not speak to other criteria, knowledge, climate, and organizational outcomes that might be used to evaluate the effectiveness of RCR training.
|
From page 254... ...
Thus although RCR training has value, its value may not always be maintained when instruction becomes institutionalized. This finding points to the importance of ongoing evaluation of the effectiveness of RCR instruction.
|
From page 255... ...
They found this instruction resulted in Cohen's Δ between .49 and 1.82 across decisions involving data management, study conduct, professional practices, and business practices. The average effect size was .91.
|
From page 256... ...
Thus ethics education apparently resulted in the acquisition of stronger mental models -- stronger mental models which were maintained over a 6-month period and were evident on transfer tasks. In addition to improvements in ethical decision making and ethical decisionmaking strategies, both performance measures, and improvements in the mental models used to understand ethical problems -- improvements maintained over a 6-month period on transfer tasks -- evaluation of this professional ethical instructional program has also considered student reactions.
|
From page 257... ...
Third, the people reporting these incidents of misconduct are doctoral students who have completed the professional ethics/responsible conduct of research education program. Taken as a whole, the sense-making RCR education program appears effective with respect to performance, mental models, reactions, and organizational outcome evaluation criteria.
|
From page 258... ...
All these studies examined merits of different approaches to the presentation of case material in RCR instruction. All these studies presented one or two cases describing complex ethical issues where breaches in ethical conduct occurred.
|
From page 259... ...
Any attempt to answer this question must bear in mind the issue "work with respect to what." RCR education programs might be evaluated with respect to changes in ethical decision-making performance, knowledge of ethics, the mental models people employ to understand ethical issues, perceptions of ethical climate, the products people produce, reactions to instruction, and organizational outcomes. Prior evaluation efforts have focused primarily, almost exclusively, on ethical decision-making performance (Antes et al., 2009)
|
From page 260... ...
Nonetheless, the evaluation data gathered for this program are noteworthy not only because they indicate that RCR training can work but that viable RCR training is most likely to be developed when courses are designed to take into account the findings obtained in earlier evaluation studies. Moreover, evaluation may be embedded in instructional programs as an ongoing element of instruction (e.g., Thiel et al.
|
From page 261... ...
. Mental models: An alternative evaluation of a sensemaking approach to ethics instruction.
|
From page 262... ...
. A new approach to assessing ethical conduct in scientific work.
|
From page 263... ...
. Effects of alternative scenarios and structured outcome evaluation on case-based ethics instruction.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.