3

Building a Stronger Learning Health Professions Education System with Implementation Science

Objective: To lay a foundation for understanding implementation science (IS) in anticipation of small-group activities that apply IS to health professions education (HPE), and to hear from IS health professions researchers about opportunities and challenges in applying IS in HPE.

This chapter is divided into four sections. Raechel Soicher’s presentation is covered in the first two sections, where she describes implementation science in the unique space of health professions education and the current state of evidence for specific pedagogy used in the health professions, on which implementation science research would be built. In the third section, health professional educators who use, or have used, implementation science offer their reflections on its use as part of a panel discussion. The chapter closes with the panelists’ responses to audience questions.

IMPLEMENTATION SCIENCE IN HEALTH PROFESSIONS EDUCATION (HPE)

Many educators, said Soicher, are subject matter experts in the area they teach, but they are not experts in pedagogy. While there is research on effective practices in teaching, there is not widespread uptake of these practices among educators. This gap between research and practice, she said, is similar to the gap between medical research and routine clinical practice; there is an oft stated statistic that it takes 17 years for a research finding to make its way into clinical practice (Morris et al., 2011). Implementation science can help close this gap by identifying best practices and studying

ways to successfully implement them in new contexts. Soicher noted that one silver lining of the COVID-19 pandemic has been an increased attention on the methods and strategies of teaching, owing to the need to quickly pivot and innovate.

An Example of Implementation Science in HPE

To illuminate the process of implementation science, Soicher shared an example from her own work implementing a utility value intervention at Oregon State University. The utility value intervention, she explained, is a brief essay writing assignment in which students are asked to make explicit connections between course content and their own lives. This intervention has both a theoretical basis (Eccles, 1983; Eccles and Wigfield, 2020) as well as evidence of effectiveness in the laboratory and the classroom. The theory is based on the idea that achievement is influenced by two components: the student’s expectation of performance and the subjective value that a student assigns to the task. The intervention was implemented in introductory psychology courses, which include roughly 3,500 students from diverse backgrounds. Students were sent the assignment through the campus learning management system, and instructors graded the assignment based on completion without reading the answers. In addition to the essay, students were asked to fill out a brief survey that collected data on implementation outcomes including acceptability, appropriateness, and feasibility of the intervention (i.e., the essay). The instructors for each class participated in semistructured interviews to collect further data.

Using the Consolidated Framework for Implementation Research (Damschroder and Hagedorn 2009; Damschroder et al., 2011), Soicher evaluated the implementation of the intervention in four domains: characteristics of the intervention, internal support for the intervention, the implementation process, and individuals’ knowledge or beliefs about the intervention.¹ Students largely found the intervention acceptable, appropriate, and feasible, said Soicher. Facilitators and barriers of the intervention were identified. Facilitators included the participation of the director and coordinator of the program, and barriers included the instructors’ desire for more communication about implementing the intervention and some instructors not implementing it as directed. The intervention took place in the winter and spring of 2020, so the rapid shift to remote classes because of COVID-19 was another significant and unexpected barrier.

___________________

¹ A fifth domain from the Consolidated Framework for Implementation Research is the external networks or policies that support or inhibit the implementation; Soicher did not examine this in her research.

Soicher also used the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework (Glasgow et al., 1999, 2019) to evaluate the intervention; she chose to examine three of the five elements: reach, adoption, and implementation. This evaluation (Figure 3-1) gave explicit details on who participated and why some did not (e.g., instructor forgot to use, students declined).

CURRENT EVIDENCE ON TEACHING PRACTICES IN HPE

There are a number of pedagogical innovations that could be implemented in HPE, said Soicher, but many are not supported with a robust evidence base. Soicher examined the literature on the science of learning

in HPE and shared what she learned. Gooding et al. (2007) identified a number of important elements in HPE, including

• an understanding of information processing (e.g., creating presentations that do not surpass cognitive load thresholds),
• the use of retrieval practice as a learning tool (e.g., quizzes),
• simulation and case-based studies,
• facilitating reflection on learning, and
• considering the social context in which learning occurs.

Rochmawati and Wiechula (2010) examined education strategies to develop clinical reasoning skills. Their strategies included problem-based learning, integrative curriculum, reflection, and concept mapping.

Evidence-Based Teaching Practices

Based on the literature, Soicher identified three specific teaching practices on which evidence has been collected, and she shared the state of the evidence for each. She emphasized, however, that clear or sufficient evidence for the effectiveness of any of these practices is not yet available. The three practices she discussed were flipped learning, problem-based learning, and reflection.

Flipped Learning

The idea behind flipped learning is for students to spend class time engaged in active learning, whether working on a problem or in small-group discussion. The key to flipped learning, Soicher humorously noted, is to “never tell anyone” because there are a lot of misconceptions about it. Research has demonstrated several benefits of flipped learning, including:

• It promotes student engagement, increases student satisfaction, improves student performance, and improves clinical outcomes (Nursing, as cited in Njie-Carr et al., 2017).
• It increases course, quiz, and final exam grades (Pharmacy, as cited in Rotellar and Cain, 2016).

• Students tend to prefer the in-class focus of hands-on, problem-solving activities (as cited in Rotellar and Cain, 2016).

There are a number of factors to consider when implementing a flipped classroom model, said Soicher. Students and faculty need to be prepared, and misconceptions or barriers need to be addressed. For example, she said one study found it took faculty considerably more time to prepare for a flipped learning class than a traditional class. Pre- and in-class activities and type of content need to be operationalized, and the class size may affect the type of strategies that can be used. Importantly, she said, there is a need to determine which aspects of flipped learning would have to be maintained for effectiveness and which elements can be adapted. Soicher pointed workshop participants toward two resources identifying best practices for flipped learning: Nine Design Principles of Flipped Classrooms (Kim et al., 2014), and Recommendations for Implementing a Flipped Classroom (Rotellar and Cain, 2016).

Problem-Based Learning

In problem-based learning, there is an emphasis on a real-world scenario that students are working on together, and critical thinking and free discussion are encouraged, said Soicher. Evidence suggests that problem-based learning is associated with:

• increased self-directed study skills, recall, and transfer or application of knowledge (as cited in Tavakol and Reicherter, 2003),
• greater motivation to attend class,
• expressed satisfaction with the course (for students and faculty), and
• better clinical knowledge and skills (as cited in Telang, 2014).

Tavakol and Reicherter (2003) described a five-step process for problem-based learning, said Soicher. First, the facilitator presents a realistic clinical situation to a small group of students. Next, the students discuss the problem, in the context of their prior knowledge or experience, and identify learning goals. Third, the facilitator guides critical analysis of the case, and fourth, students seek out additional resources to help with the case. The final step is for students to come back in a second session to present their new information and a potential solution to the problem. Telang (2014) also described some best practices for problem-based learning, said Soicher, including that it be constructive, collaborative, contextual, and self-directed; that problems be complex and open-ended, with no single correct answer; and that facilitators act as guides rather than experts.

Implementation of problem-based learning involves many considerations similar to those for flipped learning, Soicher said. Educators may be hesitant to adopt this model for several reasons (Creedy and Hand, 1994; Doring et al., 1995; Haith-Cooper, 2000; Khumalo and Gwele, 2000), possibly because educators themselves are not confident in their abilities owing to a lack of specific training in the modality. Additionally, there may be questions about whether problem-based learning should be used as one of many teaching methods or integrated into the curriculum (Andrews and Jones, 1996), and students’ perceptions of the method may lead to educators’ hesitation in using the model.

Reflection

Reflection can be defined as “active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it,” said Soicher (Dewey, 1933, p. 9). The use of reflection in the classroom has been associated with deeper learning and positive learning experiences (Sobral, 2000), improved understanding of context and reappreciating the value of health professions education (Glaze, 2001), and transformative learning in decision making (McLeod et al., 2015). McLeod et al. (2015) identified a number of best practices for the use of reflection in learning:

• Use written reflection, such as journaling, learning contract, and a personal portfolio.
• Pair written reflection with dialogue or discussion.
• Reflective activities should occur in the context of real-life practice.
• Train staff in skilled mentoring of reflection.
• Provide timely, personalized feedback on reflection.
• Provide repeated reflection throughout a curriculum.

Before implementation, Soicher said it is important to consider several issues. How might the curriculum need to be shifted in order to incorporate reflection? What support networks need to be in place for staff to support one another and students (Nicholl and Higgins, 2004)? What pressures is HPE facing from licensing bodies requiring evidence of reflection?

INTERPROFESSIONAL REFLECTIONS

In this session of the workshop, three panelists were asked to share their thoughts on using implementation science in HPE. Gail M. Jensen, vice provost for learning and assessment, professor of physical therapy, School of Pharmacy and Health Professions, Creighton University, began by saying

that one big challenge is helping educators gain a deep understanding of learning science and how to put it into practice. Far too much time is spent on content and procedural knowledge rather than on cognitive and structural knowledge. The goal is to “prepare an expert learner, not just dump knowledge into their head.” Another challenge, added Natalie Douglas, professor, communication sciences and disorders, at Central Michigan University, is a lack of attention to the effect of the organizational context and climate in health professions practice.

She explained that HPE classrooms often focus on preparing students to perform a certain evidence-based practice but do not prepare them to be cognizant of how the practice might be affected by factors outside themselves. For example, said Douglas, an intern may discover that a certain practice they learned in the classroom is infeasible or not appropriate for their internship setting. Hannah Wilson, assistant professor, nutrition, dietetics and exercise science at Concordia College in Minnesota, introduced the issue of data collection, noting that some educators may think that collecting data on implementation outcomes is too time consuming. She then reiterated an earlier comment by Soicher, saying many educators are already collecting or could easily collect these types of data. A course evaluation is a good example where validated questions could be included about the acceptability of a teaching strategy. Wilson added that there are many things educators are already doing that can help them get over the mental barrier of engaging in implementation research. Soicher concurred and then shared her own experience with implementing the utility value intervention: the intervention assignment was given to students via the learning management system, just like any other assignment, and students were given the survey to complete upon turning in the assignment.

One thing that implementation science offers is the ability to collaborate across classrooms and across institutions, said Jensen. The structure of implementation science facilitates the identification and dissemination of best teaching practices; the implementation of new best practices is needed because “we teach the way we’re taught” unless there is a pedagogical intervention. Soicher added that implementation science also offers a terminology and a structure for systematically measuring the variables that educators are interested in knowing more about. Implementation science gave Soicher a validated, established way to answer the questions she said she already had in her mind about her students and her class.

Wilson encouraged workshop participants to “dip their toes” into implementation science. There is so little implementation science being done in HPE, she noted, that any effort is useful to expand the literature and help other educators. Soicher remarked on a tendency in education to believe that “all the research has been done” on a practice and that it is ready for implementation. However, she said, there is often a lack of rich

detail about exactly how the practice was implemented, and these details are necessary for others to implement the practice themselves. She urged educators to publish details about the work they are doing—whether in a journal, on a preprint server, or in a white paper—so other people can find it and learn from it.

Panelists then welcomed questions from workshop participants.

Is institutional review board (IRB) approval needed in order to use student evaluation data for research?

The answer will depend on the specific IRB, but Soicher said that at her institution, the consent form indicates that data will be used for publication. Brooks added that as this type of research becomes more prevalent and IRBs become more familiar with it, the process should become simpler and more straightforward.

Are there funding opportunities for non-U.S. based investigators?

The majority of funding opportunities tend to be specific to the country, and they may use other terminology, such as improvement science or knowledge translation, said Douglas. She added that the Global Implementation Society works to engage countries outside of the United States and Europe.

What other resources are available to help us “dip our toes” into implementation science?

The Society for Implementation Research Collaboration has a members-only assessment database, which provides information about reliability and validity of measures for implementation outcomes, said Soicher.

Is implementation science relevant to clinical rotations, or is it only applicable to the didactic content of HPE?

“There’s space for implementation science everywhere,” said Soicher. In the clinical setting, for example, it could be useful for students to record and reflect upon the organizational context and culture.

Which implementation science framework is a good starting point?

Soicher said that the Consolidated Framework for Implementation Research is the most comprehensive list of questions about what to consider for implementation.² The authors (Damschroder and Hagedorn 2009; Damschroder et al., 2011) are explicit that researchers should not try to measure everything all at once, but should instead select the questions that are most relevant to implementation in the local context.

___________________

² More information on implementation research can be found at https://cfirguide.org/.

How can we bridge the conversations between implementation science and creating inclusive learning environments?

Most of the evidence-based practices in education make room for inclusive teaching by their nature, said Soicher. For example, a flipped classroom model makes content available to students to digest at their own pace, and problem-based learning can make room for hearing diverse voices. However, she noted, faculty development is essential in order to realize the promise of these strategies. If an instructor is trained in inclusive teaching and facilitating problem-based learning, underrepresented students are more likely to have a voice. In addition, some practices—such as simulation-based learning—may need to be adjusted so it is accessible to all students.

This page intentionally left blank.