Chapter 3
Improving Quality

During the workshop, participants offered specific suggestions for improving quality. Keynote speaker Carol Schneider (Association of American Colleges & Universities) said, “We’re not starting fresh and trying to define quality,” noting that higher education should pay attention to what educators and employers have been saying about this topic for some time. In her view, higher education already has (1) clarity on the goals for student learning, (2) research showing that high-impact practices for engaged learning are effective, (3) a way to capture that information and examine the situation on specific campuses (made possible by National Survey of Student Engagement), and (4) tools for mapping the goals onto the curriculum and thereby supporting continuous improvement. Schneider believes that the “definition of quality is the least of our challenges” and that more imperative is the need for a broader policy discussion that incorporates what is known about the needs of employers, students, and society overall into the strategies that institutions can implement to meet those needs.

INSTITUTIONAL ATTRIBUTES AND EFFORTS

Workshop participants offered a number of suggestions for how institutions—individually and regionally—could improve the quality of the educations they provide. Several participants spoke about the need for strong leadership, both from a school’s administration and from faculty champions, stressing that ongoing conversations about quality should become the norm on college and university campuses.

Affordability and Inclusion

Several participants discussed metrics based on an institution’s mission. James Kvaal (the White House) suggested that an institution’s affordability should be included in measures of quality. He explained that institutions that serve students from all backgrounds are “very important for the future of our society,” precisely because they produce a large number of graduates in an affordable way. Several participants mentioned that inclusion—the quality of an education

provided for students from underrepresented groups—should be measured at every type of institution. One participant suggested that an institution needs both the capacity to provide quality education to diverse students as well as the resources to deliver on that capacity. Carlos Castillo-Chavez (Arizona State University) spoke highly of Cornell University’s engineering school’s inclusion of students from diverse backgrounds, efforts that Castillo-Chavez connected to the school’s creation of an office of diversity and recruitment of prominent engineering faculty to direct it.

Network Effects

Some participants mentioned the network effects made possible within and outside of an institution as a signal of quality. For example, Paul LeBlanc (Southern New Hampshire University) discussed the value of a university’s music school as extending well beyond music majors. James Grossman (American Historical Association) spoke similarly, highlighting the value of ensuring that students from diverse disciplines are in close proximity of one another—for example, the sciences and humanities. Participants cited an institution’s relationships with alumni and community networks as also important, including Aprille Ericsson (NASA) and Scott Ralls (Northern Virginia Community College). Ericsson asked, “How do we maintain those relationships? How do we develop them and allow them to replicate throughout the university environment?”

A Student-Centered Approach

Several participants suggested that higher education institutions must adopt a more student-centered approach to improve the quality of the education experience. For example, Castillo-Chavez noted that it is difficult for individual faculty or an organization to excel in research while also providing a high-quality education to undergraduates: “How do we change the university so that there are good researchers that are student-centered, that are supported, encouraged, and rewarded by the institution?”

Alexander McCormick (Indiana University) highlighted the contrast between the elaborate procedures required for research involving human subjects, including certification of knowledge and ethical responsibilities, and the absence of processes that oversee the quality of new undergraduate courses. He called it an “interesting irony” that essentially no standards or systems exist for review course content and experiences.

Use of Technology to Customize Education to the End User

Several participants mentioned the growing role of technology in U.S. culture, which can be used to customize experiences around the end user but contrasts with higher education’s traditional structure. Individual participants noted how

students largely adhere to the pathways created by institutions. Sally Johnstone (Western Governors University) asked whether institutions should play the role of curator of learning experiences from which students select, where disaggregated education and learning experiences are curated around individual student needs. Under this scenario, the unit of quality measurement might be the student him- or herself. She encouraged institutions to provide a “responsive education.” Some participants view students as grazers (wanting the freedom to pick and choose) while others view students, especially adult learners pursing job-ready educational experiences, as looking for a clear roadmap to their desired education outcomes.

Use of Tools Designed to Improve Quality

Schneider discussed the Degree Qualifications Profile (DQP), a framework for mapping a curriculum that invites students to integrate their learning across fields and apply their learning to complex problems. Based on a recognition that the major is not enough, the DQP helps to ensure that students have ample opportunities to do real-world problem-solving and draw on perspectives from multiple disciplines—combining experiential with academic learning. The DQP, in addition to folding together those practices known to support student learning, pays specific attention to civic and global learning. Schneider also presented a matrix for mapping the DQP onto the curriculum. The matrix guides faculty in designing student assignments and mapping their programs onto the DQP, thus assisting students in achieving quality learning.

Schneider described a multi-state proof-of-concept study on 2- and 4-year college students’ achievement levels in selected essential learning outcomes. Faculty members across many institutions were trained to use the VALUE rubrics, and faculty from other institutions assessed their students’ learning in the areas of critical thinking, communication, and quantitative reasoning. Significant numbers of students made gains, but the picture was mixed. She highlighted competencies for which students showed the weakest performance, which included their use of evidence and communication skills.

Schneider concluded her discussion of employers’ expectations and tools for improving quality (the DQP and VALUE rubrics) by asking how institutions can be held accountable for providing students with the learning outcomes required for success in today’s world. Schneider noted, “Using the VALUE rubrics and using students’ work is only a piece of it, because just assessing the work is only going to get us more evidence of under-achievement.” She continued, “We need to hold ourselves accountable to thinking in new ways about quality.” Step one is clearly defining learning outcomes, which many institutions have done. Step two is mapping these outcomes onto program offerings: “How many institutions have faculty who are prepared to work with students in an inquiry mode of learning” utilizing high-impact practices? “How many faculty can tell you where in their program people are working on ethical learning or collaborative problem solving, and how frequently this is done?”

APPROACHES IN THE CLASSROOM

Instruction Aligned with Research on Learning

A number of participants described actions to ensure alignment of faculty’s teaching methods with how people learn, as revealed by research. Schneider outlined several high-impact educational practices that faculty are implementing. These practices included first-year seminars that emphasize critical inquiry, intensive writing, and collaborative learning; learning communities; writing-intensive courses; collaborative projects; diversity/global learning; community-based learning; internships; and capstone courses. She highlighted the finding that these high-impact practices, while positive for all students, have an especially strong effect on the outcomes of students who “traditionally are starting with more strikes against them,” such as students from communities of color as well as students with lower SAT or ACT scores.¹

Linda Slakey (Association of American Universities STEM initiative and Association of American Colleges and Universities) noted that alignment of what is known about student learning with current instruction methods is undergoing rapid change. Several participants, including Roy Swift (Workcred), Cliff Adelman (Institute for Higher Education Policy), and Ralls, advocated for the use of methods that connect what students know with what they can do—that is, understanding and measuring not only the acquisition of knowledge and skills but also the application of knowledge and skills to real-world problems and challenges. Emily Slack (Education and Labor Committee, U.S. House of Representatives) and other participants highlighted the value of approaches such as competency-based education.

A number of participants stressed the importance of institutions addressing broad-based competencies as well as using practices germane to student learning that are more domain-specific. Other elements of quality cited by participants included an institution’s ability to (1) meet the needs of people with different expectations; (2) teach students to think critically, communicate their ideas, and apply their knowledge in the field; (3) help students apply knowledge to other occupations or fields; and (4) address students’ overall well-being.

Hands-on Learning

McCormick described the Integrated Concentration in Science program at the University of Massachusetts at Amherst, in which students in STEM fields study a concentration in addition to their major. The program’s goal is to engage students in real-world problems selected by the students themselves. He

___________________

¹ Hart Research Associates. (2015). Falling Short: College Learning and Career Success. Washington, D.C.: AAC&U., and; Finley, Ashley, and Tia Brown McNair. (2013). Assessing Underserved Students’ Engagement in High-Impact Practices. Washington, D.C.: AAC&U.

described the transformation of students as they become more comfortable tackling open-ended, real-world problems, in a team setting, beginning in their first year. Ericsson spoke about the value of hands-on learning in STEM education from an employer’s perspective, saying, “We [NASA] build things. … Students actually getting out there and making something, a product: that is what I would like to see on their resumes.”

Adaptive Learning

LeBlanc encouraged the group to consider the importance of technology during its discussions, citing advances in data analytics, learning science, and immersive learning environments that can inform understanding about what students know and can do. Marco Molinaro (University of California, Davis) agreed that adaptive learning can be valuable, especially in the remedial or introductory arena and, at the other end of the spectrum, in very specific, highly detailed areas such as flight or medical simulators. To take advantage of adaptive learning, he believes higher education must accept that a model developed at one institution will be valid in other institutions—that is, institutions should adopt broadly available products rather than invest in the development of tools only for their use.

Slack believed that integrating adaptive assessment of students’ progress with course material, which alters a student’s pace to fit his or her learning, could work well for some students, especially adult learners. In her view, adaptive assessment could potentially revolutionize thinking about what constitutes quality in higher education and student learning.

Instruction Aligned to Employer Needs

On behalf of Adam Enbar of Flatiron School (who was unable to attend the workshop), LeBlanc described the school’s strategy for achieving high levels of quality. Flatiron runs 12- to 15-week “boot camps” that provide students with intensive training in Web development, an area with high market demand in the current economy. A projected 1.4 million positions (with an average starting salary of $76,000) will become available in the Web development/IT workforce in the next 5 years, and traditional higher education will produce only about 400,000 graduates. Results obtained by an external auditing firm verify the claims the school makes for their graduates. Flatiron’s graduates are receiving a quality education according to several measures, including technical Web development skills, team work, and other professional skills.

THE CHANGING ROLE OF FACULTY

Workshop participants emphasized the importance of faculty leadership in improving the quality of education within the classroom. A considerable amount of discussion focused on faculty roles and how they are changing or, in the view

of some participants, should change. Participants advocated for new ways to differentiate traditional faculty roles.

Faculty as Curators

Johnstone discussed how the current transformation of higher education relies heavily on technologies that were not available when the majority of higher education institutions were founded. She encouraged institutions to create teams of subject-matter experts and researchers who work with learning scientists and technology experts to create optimal learning environments. Individual participants, including Paul Courant (University of Michigan) and LeBlanc, suggested that the role of faculty and institutions may be shifting to one of “curating” information and learning environments. Schneider strongly disagreed with the notion that faculty’s job is to curate. She asserted instead that “the intellectual talent that we have invested in and cultivated through our universities has been a driver for everything that’s been productive about this country’s standing in the world in the economy and creativity, even in democracy.”

“Unbundling” the Faculty Roles

LeBlanc affirmed the critical role of faculty and shared his perspective that, although the displacement of faculty through the use of adjunct faculty has been long under way, faculty’s role is now being “unbundled”: “What are those roles that we ask faculty to do in other ways, to have those functions done more effectively and [perhaps] less expensively?” He suggested that the optimal configuration of faculty roles depends on the educational context. For example, on a residential 4-year campus where 18-year-olds are having “the coming of age experience,” it might be best to have more full-time faculty who can focus on student engagement and mentoring. In contrast, faculty roles might be different at an institution with a large online program serving adults “who have had about all the coming of age they can handle” and who want to acquire very specific knowledge and/or skills to prepare for a specific career or job. He encouraged the group to focus less on the issues arising from the increasing use of adjunct faculty per se and more on the functions that faculty provide and how these might be fulfilled. He compared an undergraduate education to the manufacture of a car: a single worker could build a car, bringing to bear the skills needed for every task, or a car could be built by people with expertise in sub-tasks. A car built by one person, he said, is too expensive; likewise “making education affordable is going to increasingly require the unbundling of those roles that are very expensive to maintain.”

Other participants, such as David Dill (the University of North Carolina), highlighted issues with the current unbundling of the faculty role that arise from the use of part-time instructors, that is, the lack of integration of part-time instructors’ courses into academic programs and into an institution’s quality

framework, and the relative absence of informal interactions with tenured faculty, which has historically been a key element of the coordination and integration of programs.

Regarding the next generation of the professoriate, Jay Labov (National Academies of Sciences, Engineering, and Medicine) encouraged the audience to consider efforts to improve the quality of undergraduate education by promoting modifications to graduate education, because most undergraduate professors have graduate degrees. “In many ways, the future of undergraduate education depends upon the future of graduate education,” he said.

Supporting Faculty Efforts to Improve Student Learning

Denise Simmons (Virginia Tech) described a need to provide faculty with resources to assist them in making positive changes to their courses and pedagogy. Resources that might be readily available on campuses, for example, instructional designers, could support faculty who both recognize the need to modify their courses and are willing to make those changes.

INSTRUCTION OUTSIDE OF THE CLASSROOM

Several participants spoke about the need for a quality undergraduate education to include connecting students with real-world problems, including providing them with experiences in organizations and businesses in the community. Ericsson, Andy MacCracken (National Campus Leadership Council), Elsa Núñez (Eastern Connecticut University), and others noted the importance of compensating students for their time devoted to off-campus learning experiences, which is a significant consideration for some students from underrepresented minority groups and others who struggle to pay for their educations. MacCracken asked “How do we make sure that students who are already having a hard time paying for this education are getting the experiences that we know will give them a better sense of quality and a better platform to be successful later on in the workplace?” He discussed the potential use of the federal work-study program to connect students with relevant, real-world work experience, which he called “a tremendous resource that is vastly underutilized and under resourced.” He explained that a part of the federal program now allows universities to have host sites off campus and described communities that are working with industry to invest in the development of programs similar to federal work-study programs.

Several participants advocated for a stronger relationships between higher education and industry. Swift believes that higher education should acknowledge that learning occurs in many environments and needs to bring industry’s real-world challenges to the classroom. He cited a recent meeting hosted by the Business-Higher Education Forum around data science and data analytics that examined the competencies needed by students need in these areas. Participants at that meeting stated that students who encounter industry’s

problems in the classroom are able to function in the workplace much more quickly.

ELEMENTS OF A QUALITY EDUCATION THAT ACTIVELY SUPPORT UNDERREPRESENTED MINORITIES

Several participants described elements of a quality education necessary to ensure the inclusion of all students. Schneider discussed the positive effect of research-based, high-impact practices on the experiences of underrepresented minorities. Ericsson argued for the necessity of paid internships, noting, for example, that many bright students at Historically Black Colleges and Universities (HBCUs) struggle to maintain a high grade point average (GPA) because they must work part-time while pursuing their studies. If their GPA suffers, then they are less likely to secure a higher-paying job or a job with a minimum GPA requirement (such as those at NASA) upon graduation. Ericsson acknowledged the value of the Pathways program, which is mandated across federal agencies, but noted that it does not allow the full volume of students to utilize those internship opportunities because of the hiring bottleneck it has created.² Castillo-Chavez cited Cornell University’s successful efforts to increase diversity in its engineering program through the creation of an office of diversity in engineering. Castillo-Chavez attributed the program’s success in part to the leadership of very distinguished faculty, who would not have accepted the positions without Cornell’s guarantee that they would be empowered to make significant change. He highlighted the importance of inclusion of strong leadership at the university president’s level as well.

A ROLE FOR THE ACCREDITATION PROCESS

Several participants believe that the United States’ accreditation process can play a significant role in improving the quality of undergraduate education. Slack stated that the accreditation system is very valuable and that she thought the attention paid to it during the current reauthorization cycle will assist in shifting the process away from an inputs-driven model to one that considers student learning outcomes in light of an institution’s mission. She recommended that “we make sure that we are strengthening the accreditation process, not dismantling it,” adding that “accreditors, because they are peers, know about quality and about student learning and have a much better perspective on that than some other outside folks that could be in charge of quality.” Kvaal voiced the concern that the accreditation process is “too tough and too loose at the same time.” For example, an institution’s accreditation does not ensure that it provides quality; however, students rely on accreditation as a seal of approval of

___________________

² See https://www.opm.gov/policy-data-oversight/hiring-information/students-recent-graduates/.

institutional quality. On the other hand, Kvaal believes that concerns that accreditation prevents innovation are legitimate.

ROLES OF PROFESSIONAL ORGANIZATIONS

Several participants discussed how professional organizations play a role in adoption by faculty of teaching strategies known to connect with how students learn. One participant, a member of the Accreditation Board for Engineering and Technology (ABET), suggested that the role of professional organizations in the accreditation process in their own disciplines could be strengthened. For example, the involvement of professional organizations in the accreditation of electrical and computer engineering programs has helped to define student outcomes that are more practical in the private sector.

Another participant, a member of the American Society for Mechanical Engineers, highlighted a role for professional societies in program design. She described experiences with students who completed substantial hands-on projects in high school and entered engineering programs in universities with great enthusiasm. However, after spending the first 2 years not making anything, they leave the major. The American Society for Mechanical Engineers is working with ABET to explore integration of more project-based learning into the first 2 years of the major course of study.

Participants in one small-group discussion acknowledged the presence of James Grossman, Executive Director of the American Historical Association, at the workshop, applauding his leadership and noting the value of the association’s involvement in “tuning,” a process that utilizes the DQP in a discipline-specific way. Participants in another small-group discussion mentioned a similar project by the American Association for the Advancement of Science, identifying this initiative as an opportunity for collaboration.

This page intentionally left blank.