Skip to main content

Currently Skimming:

4 The Challenges of Assessing the Impact of Integration in Higher Education on Students
Pages 83-94

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 83...
... As the committee approached its task to examine "the evidence behind the assertion that educational programs that mutually integrate learning experiences in the humanities and arts with science, technology, engineering, mathematics, and medicine (STEMM) lead to improved educational and career outcomes for undergraduate and graduate students," committee members found it necessary to first examine the nature and meaning of "evidence," and how different kinds of evidence (e.g., qualitative, quantitative, narrative, anecdotal observation, etc.)
From page 84...
... We discuss the challenges of generalizing the "evidence of improved educational and career outcomes" when different stakeholders have different interpretations of positive educational outcomes, assess outcomes in different ways, and use different pedagogical structures to approach integrative teaching and learning. We conclude that it is appropriate and necessary to consider multiple forms of evidence when considering the impact of an educational experience on a student, and that approaches to evaluating the impact of courses and programs that integrate the humanities, arts, and STEMM will necessarily be diverse and should be aligned with the specific learning goals of the course or program in its own institutional context.
From page 85...
... Furthermore, many clinical trials in medicine begin only after an "n of 1" observation suggests to a biomedical researcher that a unique, unexpected, and possibly useful phenomenon has revealed itself. An example is John Fewster's observation that a farmer who had been infected with cowpox was subsequently protected against smallpox (an example of anecdotal evidence in the form of clinical observation)
From page 86...
... The committee expects that integrated educational experiences will have multiple impacts on students, some of which lend themselves to quantitative or qualitative approaches to data collection and analysis, whereas others defy traditional measures of impact. For example, can the impact of a work of art or a musical performance always be sufficiently described in words or numbers?
From page 87...
... The desired and measurable learning outcome in the math course might be the ability to fundamentally understand and logically solve the equation, while in the English course it may be the capacity to engage in a close reading and informed interpretation of a literary text. Understanding the impact of a course that integrates mathematics and English could be confounded by the challenge of developing an assessment tool that can adequately compare two very different approaches to education or discipline-specific conceptions of desirable learning outcomes.
From page 88...
... These designs often, but not always, include the following features: longitudinal approaches, comparison groups, theoretically validated and empirically derived measures of student learning, and samples adequately robust to address the questions asked. Using these design elements as evidence of empirical trustworthiness is a strategy consistent with that offered in the largest and most cited literature synthesis in higher education: How College Affects Students, Volume 3: 21st Century Evidence That Higher Education Works (Mayhew et al., 2016)
From page 89...
... The challenge for faculty and scholars who strive to evaluate integrative learning experiences will be to develop frameworks that permit them to evaluate the student learning outcomes they value and hope to provide, rather than those that are easy to measure. This process begins with the faculty member or evaluator asking a series of questions: What are the expected student learning outcomes from this integrative course or program?
From page 90...
... Because of its emphasis on learning outcomes and evaluation, backward design can serve as a means to figure out the optimal combination of quantitative and qualitative approaches for the evalu ation of a given integrative program. First introduced by Wiggins and McTighe in 1998, this approach provides a "roadmap" for instructors who want to focus on the learning goals and objectives, gather the right kind of evidence to demonstrate mastery of those objectives, and create educational experiences to support those learning objectives.
From page 91...
... To support those efforts and bolster evidence-based evaluative practices, the committee suggests a research program to generate and collect robust forms of the following types of evidence in evaluating the impact of integrative educational experiences: • Qualitative, longitudinal testimony from students, teachers, and administrators on the impact of integrative programs and courses • Quantitative, controlled, longitudinal data on the impact of inte gration on students' grades, attitudes, and competencies, retention and graduation rates, college major, employment status, salary, civic engagement, and satisfaction in life and career • Narrative case studies that offer an in-depth description and analy sis of the nature of integrated programs and courses, the learning goals of the students taking the course, and the goals of the profes sor teaching the course or program • Detailed descriptions of the curricula and pedagogies of integrative courses and programs • Portfolios of student work, performances, or exhibitions Currently, only limited evidence from each of these categories is available. While this is unfortunate, it is not surprising.
From page 92...
... Given that the limited evidence available is promising and suggests positive learning and career outcomes for students, the committee is urging a new nationwide effort to develop and fund the research needed to establish appropriate protocols for the collection of the kinds of robust and multifaceted evidence that the broader educational community can accept and embrace. Institutions of higher education that have implemented integrative models, or that plan to do so, might consider working with faculty and higher education researchers to evaluate courses and programs in ways that could address a number of outstanding research questions.
From page 93...
... • Can a successful integrative program or course at one college or university be implemented at another such that it results in the same kinds of learning outcomes? To what extent can the positive impacts of integrative efforts be achieved independently of the idiosyncratic professors and administrators who implement such courses and programs?


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.