9

Conclusions and Recommendations

Practitioners designing or improving undergraduate research experiences (UREs) can build on the experiences of colleagues and learn from the increasingly robust literature about UREs and the considerable body of evidence about how students learn. The questions practitioners ask themselves during the design process should include questions about the goals of the campus, program, faculty, and students. Other factors to consider when designing a URE include the issues raised in the conceptual framework for learning and instruction, the available resources, how the program or experience will be evaluated or studied, and how to design the program from the outset to incorporate these considerations, as well as how to build in opportunities to improve the experience over time in light of new evidence. (Some of these topics are addressed in Chapter 8.)

Colleges and universities that offer or wish to offer UREs to their students should undertake baseline evaluations of their current offerings and create plans to develop a culture of improvement in which faculty are supported in their efforts to continuously refine UREs based on the evidence currently available and evidence that they and others generate in the future. While much of the evidence to date is descriptive, it forms a body of knowledge that can be used to identify research questions about UREs, both those designed around the apprenticeship model and those designed using the more recent course-based undergraduate research experience (CURE) model. Internships and other avenues by which undergraduates do research provide many of the same sorts of experiences but are not well studied. In any case, it is clear that students value these experiences; that many faculty do as well; and that they contribute to broadening participation in science,

technology, engineering, and mathematics (STEM) education and careers. The findings from the research literature reported in Chapter 4 provide guidance to those designing both opportunities to improve practical and academic skills and opportunities for students to “try out” a professional role of interest.

Little research has been done that provides answers to mechanistic questions about how UREs work. Additional studies are needed to know which features of UREs are most important for positive outcomes with which students and to gain information about other questions of this type. This additional research is needed to better understand and compare different strategies for UREs designed for a diversity of students, mentors, and institutions. Therefore, the committee recommends steps that could increase the quantity and quality of evidence available in the future and makes recommendations for how faculty, departments, and institutions might approach decisions about UREs using currently available information. Multiple detailed recommendations about the kinds of research that might be useful are provided in the research agenda in Chapter 7.

In addition to the specific research recommended in Chapter 7, in this chapter the committee provides a series of interrelated conclusions and recommendations related to UREs for the STEM disciplines and intended to highlight the issues of primary importance to administrators, URE program designers, mentors to URE students, funders of UREs, those leading the departments and institutions offering UREs, and those conducting research about UREs. These conclusions and recommendations are based on the expert views of the committee and informed by their review of the available research, the papers commissioned for this report, and input from presenters during committee meetings. Table 9-1 defines categories of these URE “actors,” gives examples of specific roles included in each category, specifies key URE actions for which that category is responsible, and lists the conclusions and recommendations the committee views as most relevant to that actor category.

RESEARCH ON URES

Conclusion 1: The current and emerging landscape of what constitutes UREs is diverse and complex. Students can engage in STEM-based undergraduate research in many different ways, across a variety of settings, and along a continuum that extends and expands upon learning opportunities in other educational settings. The following characteristics define UREs. Due to the variation in the types of UREs, not all experiences include all of the following characteristics in the same way; experiences vary in how much a particular characteristic is emphasized.

TABLE 9-1 Audiences for Committee’s Conclusions and Recommendations

• They engage students in research practices including the ability to argue from evidence.
• They aim to generate novel information with an emphasis on discovery and innovation or to determine whether recent preliminary results can be replicated.
• They focus on significant, relevant problems of interest to STEM researchers and, in some cases, a broader community (e.g., civic engagement).
• They emphasize and expect collaboration and teamwork.
• They involve iterative refinement of experimental design, experimental questions, or data obtained.
• They allow students to master specific research techniques.
• They help students engage in reflection about the problems being investigated and the work being undertaken to address those problems.
• They require communication of results, either through publication or presentations in various STEM venues.
• They are structured and guided by a mentor, with students assuming increasing ownership of some aspects of the project over time.

UREs are generally designed to add value to STEM offerings by promoting an understanding of the ways that knowledge is generated in STEM fields and to extend student learning beyond what happens in the small group work of an inquiry-based course. UREs add value by enabling students to understand and contribute to the research questions that are driving the field for one or more STEM topics or to grapple with design challenges of interest to professionals. They help students understand what it means to be a STEM researcher in a way that would be difficult to convey in a lecture course or even in an inquiry-based learning setting. As participants in a URE, students can learn by engaging in planning, experimentation, evaluation, interpretation, and communication of data and other results in light of what is already known about the question of interest. They can pose relevant questions that can be solved only through investigative or design efforts—individually or in teams—and attempt to answer these questions despite the challenges, setbacks, and ambiguity of the process and the results obtained.

The diversity of UREs reflects the reality that different STEM disciplines operate from varying traditions, expectations, and constraints (e.g., lab safety issues) in providing opportunities for undergraduates to engage in research. In addition, individual institutions and departments have cultures that promote research participation to various degrees and at different stages in students’ academic careers. Some programs emphasize design and problem solving in addition to discovery. UREs in different disciplines can

take many forms (e.g., apprentice-style, course-based, internships, project-based), but the definitional characteristics described above are similar across different STEM fields.

Furthermore, students in today’s university landscape may have opportunities to engage with many different types of UREs throughout their education, including involvement in a formal program (which could include mentoring, tutoring, research, and seminars about research), an apprentice-style URE under the guidance of an individual or team of faculty members, an internship, or enrolling in one or more CUREs or in a consortium- or project-based program.

Conclusion 2: Research on the efficacy of UREs is still in the early stages of development compared with other interventions to improve undergraduate STEM education.

• The types of UREs are diverse, and their goals are even more diverse. Questions and methodologies used to investigate the roles and effectiveness of UREs in achieving those goals are similarly diverse.
• Most of the studies of UREs to date are descriptive case studies or use correlational designs. Many of these studies report positive outcomes from engagement in a URE.
• Only a small number of studies have employed research designs that can support inferences about causation. Most of these studies find evidence for a causal relationship between URE participation and subsequent persistence in STEM. More studies are needed to provide evidence that participation in UREs is a causal factor in a range of desired student outcomes.

Taking the entire body of evidence into account, the committee concludes that the published peer-reviewed literature to date suggests that participation in a URE is beneficial for students.

As discussed in the report’s Introduction (see Chapter 1) and in the research agenda (see Chapter 7), the committee considered descriptive, causal, and mechanistic questions in our reading of the literature on UREs. Scientific approaches to answering descriptive, causal, and mechanistic questions require deciding what to look for, determining how to examine it, and knowing appropriate ways to score or quantify the effect.

Descriptive questions ask what is happening without making claims as to why it is happening—that is, without making claims as to whether the research experience caused these changes. A descriptive statement about UREs only claims that certain changes occurred during or after the time the students were engaged in undergraduate research. Descriptive studies

cannot determine whether any benefits observed were caused by participation in the URE.

Causal questions seek to discover whether a specific intervention leads to a specific outcome, other things being equal. To address such questions, causal evidence can be generated from a comparison of carefully selected groups that do and do not experience UREs. The groups can be made roughly equivalent by random assignment (ensuring that URE and non-URE groups are the same on average as the sample size increases) or by controlling for an exhaustive set of characteristics and experiences that might render the groups different prior to the URE. Other quasi-experimental strategies can also be used. Simply comparing students who enroll in a URE with students who do not is not adequate for determining causality because there may be selection bias. For example, students already interested in STEM are more likely to seek out such opportunities and more likely to be selected for such programs. Instead the investigator would have to compare future enrollment patterns (or other measures) between closely matched students, some of whom enrolled in a URE and some of whom did not. Controlling for selection bias to enable an inference about causation can pose significant challenges.

Questions of mechanism or of process also can be explored to understand why a causal intervention leads to the observed effect. Perhaps the URE enhances a student’s confidence in her ability to succeed in her chosen field or deepens her commitment to the field by exposing her to the joy of discovery. Through these pathways that act on the participant’s purposive behavior, the URE enhances the likelihood that she persists in STEM. The question for the researcher then becomes what research design would provide support for this hypothesis of mechanism over other candidate explanations for why the URE is a causal factor in STEM persistence.

The committee has examined the literature and finds a rich descriptive foundation for testable hypotheses about the effects of UREs on student outcomes. These studies are encouraging; a few of them have generated evidence that a URE can be a positive causal factor in the progression and persistence of STEM students. The weight of the evidence has been descriptive; it relies primarily on self-reports of short-term gains by students who chose to participate in UREs and does not include direct measures of changes in the students’ knowledge, skills, or other measures of success across comparable groups of students who did and did not participate in UREs.

While acknowledging the scarcity of strong causal evidence on the benefits of UREs, the committee takes seriously the weight of the descriptive evidence. Many of the published studies of UREs show that students who participate report a range of benefits, such as increased understanding of the research process, encouragement to persist in STEM, and support that helps them sustain their identity as researchers and continue with their

plans to enroll in a graduate program in STEM (see Chapter 4). These are effective starting points for causal studies.

Conclusion 3: Studies focused on students from historically underrepresented groups indicate that participation in UREs improves their persistence in STEM and helps to validate their disciplinary identity.

Various UREs have been specifically designed to increase the number of historically underrepresented students who go on to become STEM majors and ultimately STEM professionals. While many UREs offer one or more supplemental opportunities to support students’ academic or social success, such as mentoring, tutoring, summer bridge programs, career or graduate school workshops, and research-oriented seminars, those designed for underrepresented students appear to emphasize such features as integral and integrated components of the program. In particular, studies of undergraduate research programs targeting underrepresented minority students have begun to document positive outcomes such as degree completion and persistence in interest in STEM careers (Byars-Winston et al., 2015; Chemers et al., 2011; Jones et al., 2010; Nagda et al., 1998; Schultz et al., 2011). Most of these studies collected data on apprentice-style UREs, in which the undergraduate becomes a functioning member of a research group along with the graduate students, postdoctoral fellows, and mentor.

Recommendation 1: Researchers with expertise in education research should conduct well-designed studies in collaboration with URE program directors to improve the evidence base about the processes and effects of UREs. This research should address how the various components of UREs may benefit students. It should also include additional causal evidence for the individual and additive effects of outcomes from student participation in different types of UREs. Not all UREs need be designed to undertake this type of research, but it would be very useful to have some UREs that are designed to facilitate these efforts to improve the evidence base.

As the focus on UREs has grown, so have questions about their implementation. Many articles have been published describing specific UREs (see Chapter 2). Large amounts of research have also been undertaken to explore more generally how students learn, and the resulting body of evidence has led to the development and adoption of “active learning” strategies and experiences. If a student in a URE has an opportunity to, for example, analyze new data or to reformulate a hypothesis in light of the student’s analysis, this activity fits into the category that is described as active learning. Surveys of student participants and unpublished evaluations pro-

vide additional information about UREs but do not establish causation or determine the mechanism(s). Consequently, little is currently known about the mechanisms of precisely how UREs work and which aspects of UREs are most powerful. Important components that have been reported include student ownership of the URE project, time to tackle a question iteratively, and opportunities to report and defend one’s conclusions (Hanauer and Dolan, 2014; Thiry et al., 2011).

There are many unanswered questions and opportunities for further research into the role and mechanism of UREs. Attention to research design as UREs are planned is important; more carefully designed studies are needed to understand the ways that UREs influence a student’s education and to evaluate the outcomes that have been reported for URE participants. Appropriate studies, which include matched samples or similar controls, would facilitate research on the ways that UREs benefit students, enabling both education researchers and implementers of UREs to determine optimal features for program design and giving the community a more robust understanding of how UREs work.

See the research agenda (Chapter 7) for specific recommendations about research topics and approaches.

Recommendation 2: Funders should provide appropriate resources to support the design, implementation, and analysis of some URE programs that are specifically designed to enable detailed research establishing the effects on participant outcomes and on other variables of interest such as the consequences for mentors or institutions.

Not all UREs need to be the subject of extensive study. In many cases, a straightforward evaluation is adequate to determine whether the URE is meeting its goals. However, to achieve more widespread improvement in both the types and quality of the UREs offered in the future, additional evidence about the possible causal effects and mechanisms of action of UREs needs to be systematically collected and disseminated. This includes a better understanding of the implementation differences for a variety of institutions (e.g., community colleges, primarily undergraduate institutions, research universities) to ensure that the desired outcomes can translate across settings. Increasing the evidence about precisely how UREs work and which aspects of UREs are most powerful will require careful attention to study design during planning for the UREs.

Not all UREs need to be designed to achieve this goal; many can provide opportunities to students by relying on pre-existing knowledge and iterative improvement as that knowledge base grows. However, for the knowledge base to grow, funders must provide resources for some URE designers and social science researchers to undertake thoughtful and well-planned studies

on causal and mechanistic issues. This will maximize the chances for the creation and dissemination of information that can lead to the development of sustainable and effective UREs. These studies can result from a partnership formed as the URE is designed and funded, or evaluators and social scientists could identify promising and/or effective existing programs and then raise funds on their own to support the study of those programs to answer the questions of interest. In deciding upon the UREs that are chosen for these extensive studies, it will be important to consider whether, collectively, they are representative of UREs in general. For example, large and small UREs at large and small schools targeted at both introductory and advanced students and topics should be studied.

CONSTRUCTION OF URES

Conclusion 4: The committee was unable to find evidence that URE designers are taking full advantage of the information available in the education literature on strategies for designing, implementing, and evaluating learning experiences. STEM faculty members do not generally receive training in interpreting or conducting education research. Partnerships between those with expertise in education research and those with expertise in implementing UREs are one way to strengthen the application of evidence on what works in planning and implementing UREs.

As discussed in Chapters 3 and 4, there is an extensive body of literature on pedagogy and how people learn; helping STEM faculty to access the existing literature and incorporate those concepts as they design UREs could improve student experiences. New studies that specifically focus on UREs may provide more targeted information that could be used to design, implement, sustain, or scale up UREs and facilitate iterative improvements. Information about the features of UREs that elicit particular outcomes or best serve certain populations of students should be considered when implementing a new instantiation of an existing model of a URE or improving upon an existing URE model.

Conclusion 5: Evaluations of UREs are often conducted to inform program providers and funders; however, they may not be accessible to others. While these evaluations are not designed to be research studies and often have small sample sizes, they may contain information that could be useful to those initiating new URE programs and those refining UREs. Increasing access to these evaluations and to the accumulated experience of the program providers may enable URE designers and implementers to build upon knowledge gained from earlier UREs.

As discussed in Chapter 1, the committee searched for evaluations of URE programs in several different ways but was not able to locate many published evaluations to study. Although some evaluations were found in the literature, the committee could not determine a way to systematically examine the program evaluations that have been prepared. The National Science Foundation and other funders generally require grant recipients to submit evaluation data, but that information is not currently aggregated and shared publicly, even for programs that are using a common evaluation tool.¹

Therefore, while program evaluation likely serves a useful role in providing descriptive data about a program for the institutions and funders supporting the program, much of the summative evaluation work that has been done to date adds relatively little to the broader knowledge base and overall conversations around undergraduate research. Some of the challenges of evaluation include budget and sample size constraints.

Similarly, it is difficult for designers of UREs to benefit systematically from the work of others who have designed and run UREs in the past because of the lack of an easy and consistent mechanism for collecting, analyzing, and sharing data. If these evaluations were more accessible they might be beneficial to others designing and evaluating UREs by helping them to gather ideas and inspiration from the experiences of others. A few such stories are provided in this report, and others can be found among the many resources offered by the Council on Undergraduate Research² and on other websites such as CUREnet.³

Recommendation 3: Designers of UREs should base their design decisions on sound evidence. Consultations with education and social science researchers may be helpful as designers analyze the literature and make decisions on the creation or improvement of UREs. Professional development materials should be created and made available to faculty. Educational and disciplinary societies should consider how they can provide resources and connections to those working on UREs.

Faculty and other organizers of UREs can use the expanding body of scholarship as they design or improve the programs and experiences offered to their students. URE designers will need to make decisions about how to adapt approaches reported in the literature to make the programs they develop more suitable to their own expertise, student population(s), and available resources. Disciplinary societies and other national groups, such as those focused on improving pedagogy, can play important roles in

___________________

¹ Personal knowledge of Janet Branchaw, member of the Committee on Strengthening Research Experiences for Undergraduate STEM Students.

² See www.cur.org [November 2016].

³ See (curenet.cns.utexas.edu) [November 2016].

bringing these issues to the forefront through events at their national and regional meetings and through publications in their journals and newsletters. They can develop repositories for various kinds of resources appropriate for their members who are designing and implementing UREs. The ability to travel to conferences and to access and discuss resources created by other individuals and groups is a crucial aspect of support (see Recommendations 7 and 8 for further discussion).

See Chapter 8 for specific questions to consider when one is designing or implementing UREs.

CURRENT OFFERINGS

Conclusion 6: Data at the institutional, state, or national levels on the number and type of UREs offered, or who participates in UREs overall or at specific types of institutions, have not been collected systematically. Although the committee found that some individual institutions track at least some of this type of information, we were unable to determine how common it is to do so or what specific information is most often gathered.

There is no one central database or repository that catalogs UREs at institutions of higher education, the nature of the research experiences they provide, or the relevant demographics (student, departmental, and institutional). The lack of comprehensive data makes it difficult to know how many students participate in UREs; where UREs are offered; and if there are gaps in access to UREs across different institutional types, disciplines, or groups of students. One of the challenges of describing the undergraduate research landscape is that students do not have to be enrolled in a formal program to have a research experience. Informal experiences, for example a work-study job, are typically not well documented. Another challenge is that some students participate in CUREs or other research experiences (such as internships) that are not necessarily labeled as such. Institutional administrators may be unaware of CUREs that are already part of their curriculum. (For example, establishment of CUREs may be under the purview of a faculty curriculum committee and may not be recognized as a distinct program.) Student participation in UREs may occur at their home institution or elsewhere during the summer. Therefore, it is very difficult for a science department, and likely any other STEM department, to know what percentage of their graduating majors have had a research experience, let alone to gather such information on students who left the major.⁴

___________________

⁴ This point was made by Marco Molinaro, University of California, Davis, in a presentation to the Committee on Strengthening Research Experience for Undergraduate STEM Students, September 16, 2015.

Conclusion 7: While data are lacking on the precise number of students engaged in UREs, there is some evidence of a recent growth in course-based undergraduate research experiences (CUREs), which engage a cohort of students in a research project as part of a formal academic experience.

There has been an increase in the number of grants and the dollar amount spent on CUREs over the past decade (see Chapter 3). CUREs can be particularly useful in scaling UREs to reach a much larger population of students (Bangera and Brownell, 2014). By using a familiar mechanism—enrollment in a course—a CURE can provide a more comfortable route for students unfamiliar with research to gain their first experience. CUREs also can provide such experiences to students with diverse backgrounds, especially if an institution or department mandates participation sometime during a student’s matriculation. Establishing CUREs may be more cost-effective at schools with little on-site research activity. However, designing a CURE is a new and time-consuming challenge for many faculty members. Connecting to nationally organized research networks can provide faculty with helpful resources for the development of a CURE based around their own research or a local community need, or these networks can link interested faculty to an ongoing collaborative project. Collaborative projects can provide shared curriculum, faculty professional development and community, and other advantages when starting or expanding a URE program. See the discussion in the report from a convocation on Integrating Discovery-based Research into the Undergraduate Curriculum (National Academies of Sciences, Engineering, and Medicine, 2015).

Recommendation 4: Institutions should collect data on student participation in UREs to inform their planning and to look for opportunities to improve quality and access.

Better tracking of student participation could lead to better assessment of outcomes and improved quality of experience. Such metrics could be useful for both prospective students and campus planners. An integrated institutional system for research opportunities could facilitate the creation of tiered research experiences that allow students to progress in skills and responsibility and create support structures for students, providing, for example, seminars in communications, safety, and ethics for undergraduate researchers. Institutions could also use these data to measure the impact of UREs on student outcomes, such as student success rates in introductory courses, retention in STEM degree programs, and completion of STEM degrees.

While individual institutions may choose to collect additional information depending on their goals and resources, relevant student demographics

and the following design elements would provide baseline data. At a minimum, such data should include

• Type of URE;
• Each student’s discipline;
• Duration of the experience;
• Hours spent per week;
• When the student began the URE (e.g., first year, capstone);
• Compensation status (e.g., paid, unpaid, credit); and
• Location and format (e.g., on home campus, on another campus, internship, co-op).

National aggregation of some of the student participation variables collected by various campuses might be considered by funders. The existing Integrated Postsecondary Education Data System database, organized by the National Center for Education Statistics at the U.S. Department of Education, may be a suitable repository for certain aspects of this information.

Recommendation 5: Administrators and faculty at all types of colleges and universities should continually and holistically evaluate the range of UREs that they offer. As part of this process, institutions should:

• Consider how best to leverage available resources (including off-campus experiences available to students and current or potential networks or partnerships that the institution may form) when offering UREs so that they align with their institution’s mission and priorities;
• Consider whether current UREs are both accessible and welcoming to students from various subpopulations across campus (e.g., historically underrepresented students, first generation college students, those with disabilities, non-STEM majors, prospective kindergarten-through-12th-grade teachers); and
• Gather and analyze data on the types of UREs offered and the students who participate, making this information widely available to the campus community and using it to make evidence-based decisions about improving opportunities for URE participation. This may entail devising or implementing systems for tracking relevant data (see Conclusion 4).

Resources available for starting, maintaining, and expanding UREs vary from campus to campus. At some campuses, UREs are a central focus and many resources are devoted to them. At other institutions—for example, many community colleges—UREs are seen as extra, and new resources may be required to ensure availability of courses and facilities. Resource-

constrained institutions may need to focus more on ensuring that students are aware of potential UREs that already exist on campus and elsewhere in near proximity to campus. All institutional discussions about UREs must consider both the financial resources and physical resources (e.g., laboratories, field stations, engineering design studios) required, while remembering that faculty time is a crucial resource. The incentives and disincentives for faculty to spend time on UREs are significant. Those institutions with an explicit mission to promote undergraduate research may provide more recognition and rewards to departments and faculty than those with another focus. The culture of the institution with respect to innovation in pedagogy and support for faculty development also can have a major influence on the extent to which UREs are introduced or improved.

Access to UREs may vary across campus and by department, and participation in UREs may vary across student groups. It is important for campuses to consider the factors that may facilitate or discourage students from participation in UREs. Inconsistent procedures or a faculty preference for students with high grades or previous research experience may limit options for some student populations.

UREs often grow based on the initiative of individual faculty members and other personnel, and an institution may not have complete or even rudimentary knowledge of all of the opportunities available or whether there are gaps or inconsistencies in its offerings. A uniform method for tracking the UREs available on a given campus would be useful to students and would provide a starting point for analyzing the options. Tracking might consist of notations in course listings and, where feasible, on student transcripts. Analysis might consider the types of UREs offered, the resources available to each type of URE, and variations within or between various disciplines and programs. Attention to whether all students or groups of students have appropriate access to UREs would foster consideration of how to best allocate resources and programming on individual campuses, in order to focus resources and opportunities where they are most needed.

MENTORING

Conclusion 8: The quality of mentoring can make a substantial difference in a student’s experiences with research. However, professional development in how to be a good mentor is not available to many faculty or other prospective mentors (e.g., graduate students, postdoctoral fellows).

Engagement in quality mentored research experiences has been linked to self-reported gains in research skills and productivity as well as retention in STEM (see Chapter 5). Quality mentoring in UREs has been shown

to increase persistence in STEM for historically underrepresented students (Hernandez et al., 2016). In addition, poor mentoring during UREs has been shown to decrease retention of students (Hernandez et al., 2016).

More general research on good mentoring in the STEM environment has been positively associated with self-reported gains in identity as a STEM researcher, a sense of belonging, and confidence to function as a STEM researcher (Byars-Winston et al., 2015; Chemers et al., 2011; Pfund et al., 2016; Thiry et al., 2011). The frequency and quality of mentee-mentor interactions has been associated with students’ reports of persistence in STEM, with mentoring directly or indirectly improving both grades and persistence in college. For students from historically underrepresented ethnic/racial groups, quality mentoring has been associated with self-reported enhanced recruitment into graduate school and research-related career pathways (Byars-Winston et al., 2015). Therefore, it is important to ensure that faculty and mentors receive the proper development of mentoring skills.

Recommendation 6: Administrators and faculty at colleges and universities should ensure that all who mentor undergraduates in research experiences (this includes faculty, instructors, postdoctoral fellows, graduate students, and undergraduates serving as peer mentors) have access to appropriate professional development opportunities to help them grow and succeed in this role.

Although many organizations recognize effective mentors (e.g., the National Science Foundation’s Presidential Awards for Excellence in Science, Mathematics, and Engineering Mentoring), there currently are no standard criteria for selecting, evaluating, or recognizing mentors specifically for UREs. In addition, there are no requirements that mentors meet some minimum level of competency before engaging in mentoring or participate in professional development to obtain a baseline of knowledge and skills in mentoring, including cultural competence in mentoring diverse groups of students. Traditionally, the only experience required for being a mentor is having been mentored, regardless of whether the experience was negative or positive (Handelsman et al., 2005; Pfund et al., 2015). Explicit consideration of how the relationships are formed, supported, and evaluated can improve mentor-mentee relationships. To ensure that the mentors associated with a URE are prepared appropriately, thereby increasing the chances of a positive experience for both mentors and mentees, all prospective mentors should prepare for their role. Available resources include the Entering Mentoring course (see Pfund et al., 2015) and the book Successful STEM Mentoring Initiative for Underrepresented Students (Packard, 2016).

A person who is an ineffective mentor for one student might be inspiring for another, and the setting in which the mentoring takes place (e.g., a CURE or apprentice-style URE, a laboratory or field-research environment) may also influence mentor effectiveness. Thus, there should be some mechanism for monitoring such relationships during the URE, or there should be opportunity for a student who is unhappy with the relationship to seek other mentors. Indeed, cultivating a team of mentors with different experiences and expertise may be the best strategy for any student. A parallel volume to the Entering Mentoring curriculum mentioned above, Entering Research Facilitator’s Manual (Branchaw et al., 2010), is designed to help students with their research mentor-mentee relationships and to coach them on building teams of mentors to guide them. As mentioned in Chapter 5, the Entering Research curriculum also contains information designed to support a group of students as they go through their first apprentice-style research experience, each working in separate research groups and also meeting together as a cohort focused on learning about research.

PRIORITIES FOR THE FUTURE

Conclusion 9: The unique assets, resources, priorities, and constraints of the department and institution, in addition to those of individual mentors, impact the goals and structures of UREs. Schools across the country are showing considerable creativity in using unique resources, repurposing current assets, and leveraging student enthusiasm to increase research opportunities for their students.

Given current calls for UREs and the growing conversation about their benefits, an increasing number of two- and four-year colleges and universities are increasing their efforts to support undergraduate research. Departments, institutions, and individual faculty members influence the precise nature of UREs in multiple ways and at multiple levels. The physical resources available, including laboratories, field stations, and engineering design studios and testing facilities, make a difference, as does the ability to access resources in the surrounding community (including other parts of the campus). Institutions with an explicit mission to promote undergraduate research may provide more time, resources (e.g., financial, support personnel, space, equipment), and recognition and rewards to departments and faculty in support of UREs than do institutions without that mission. The culture of the institution with respect to innovation in pedagogy and support for faculty development also affects the extent to which UREs are introduced or improved.

Development of UREs requires significant time and effort. Whether or not faculty attempt to implement UREs can depend on whether departmental

or institutional reward and recognition systems compensate for or even recognize the time required to initiate and implement them. The availability of national consortia can help to alleviate many of the time and logistical problems but not those obstacles associated with recognition and resources.

It will be harder for faculty to find the time to develop UREs at institutions where they are required to teach many courses per semester, although in some circumstances faculty can teach CUREs that also advance their own research (Shortlidge et al., 2016). Faculty at community colleges generally have the heaviest teaching expectations, little or no expectations or incentives to maintain a research program, limited access to lab or design space or to scientific and engineering journals, and few resources to undertake any kind of a research program. These constraints may limit the extent to which UREs can be offered to the approximately 40 percent of U.S. undergraduates who are enrolled in the nation’s community colleges (which collectively also serve the highest percentage of the nation’s underrepresented students).⁵

Recommendation 7: Administrators and faculty at all types of colleges and universities should work together within and, where feasible, across institutions to create a culture that supports the development of evidence-based, iterative, and continuous refinement of UREs, in an effort to improve student learning outcomes and overall academic success. This should include the development, evaluation, and revision of policies and practices designed to create a culture supportive of the participation of faculty and other mentors in effective UREs. Policies should consider pedagogy, professional development, cross-cultural awareness, hiring practices, compensation, promotion (incentives, rewards), and the tenure process.

Colleges and universities that would like to expand or improve the UREs offered to their students should consider the campus culture and climate and the incentives that affect faculty choices. Those campuses that cultivate an environment supportive of the iterative and continuous refinement of UREs and that offer incentives for evaluation and evidence-based improvement of UREs seem more likely to sustain successful programs. Faculty and others who develop and implement UREs need support to be able to evaluate their courses or programs and to analyze evidence to make decisions about URE design. This kind of support may be fostered by expanding the mission of on-campus centers for learning and teaching to focus more on UREs or by providing incentives for URE developers from the natural sciences and engineering to collaborate with colleagues in the social sciences or colleges of education with expertise in designing studies

___________________

⁵ See http://nces.ed.gov/programs/coe/indicator_cha.asp [November 2016].

involving human subjects. Supporting closer communication between URE developers and the members of the campus Institutional Review Board may help projects to move forward more seamlessly. Interdepartmental and intercampus connections (especially those between two- and four-year institutions) can be valuable for linking faculty with the appropriate resources, colleagues, and diverse student populations. Faculty who have been active in professional development on how students learn in the classroom may have valuable experiences and expertise to share.

The refinement or expansion of UREs should build on evidence from data on student participation, pedagogy, and outcomes, which are integral components of the original design. As UREs are validated and refined, institutions should make efforts to facilitate connections among different departments and disciplines, including the creation of multidisciplinary UREs. Student engagement in learning in general, and with UREs more specifically, depends largely on the culture of the department and the institution and on whether students see their surroundings as inclusive and energetic places to learn and thrive. A study that examined the relationship between campus missions and the five benchmarks for effective educational practice (measured by the National Survey of Student Engagement) showed that different programs, policies, and approaches may work better, depending on the institution’s mission (Kezar and Kinzie, 2006).

The Council on Undergraduate Research (2012) document Characteristics of Excellence in Undergraduate Research outlines several best practices for UREs based on the apprenticeship model (see Chapter 8). That document is not the result of a detailed analysis of the evidence but is based on the extensive experiences and expertise of the council’s members. It suggests that undergraduate research should be a normal part of the undergraduate experience regardless of the type of institution. It also identifies changes necessary to include UREs as part of the curriculum and culture changes necessary to support curricular reform, co-curricular activities, and modifications to the incentives and rewards for faculty to engage with undergraduate research. In addition, professional development opportunities specifically designed to help improve the pedagogical and mentoring skills of instructional staff in using evidence-based practices can be important for a supportive learning culture.

Recommendation 8: Administrators and faculty at all types of colleges and universities should work to develop strong and sustainable partnerships within and between institutions and with educational and professional societies for the purpose of sharing resources to facilitate the creation of sustainable URE programs.

Networks of faculty, institutions, regionally and nationally coordinated URE initiatives, professional societies, and funders should be strengthened

to facilitate the exchange of evidence and experience related to UREs. These networks could build on the existing work of professional societies that assist faculty with pedagogy. They can help provide a venue for considering the policy context and larger implications of increasing the number, size, and scope of UREs. Such networks also can provide a more robust infrastructure, to improve the sustainability and expansion of URE opportunities. The sharing of human, financial, scientific, and technical resources can strengthen the broad implementation of effective, high-quality, and more cost-efficient UREs. It may be especially important for community colleges and minority-serving institutions to engage in partnerships in order to expand the opportunities for undergraduates (both transfer and technical students) to participate in diverse UREs (see discussion in National Academies of Sciences, Engineering, and Medicine, 2015, and Elgin et al., 2016). Consortia can facilitate the sharing of resources across disciplines and departments within the same institution or at different institutions, organizations, and agencies. Consortia that employ research methodologies in common can share curriculum, research data collected, and common assessment tools, lessening the time burden for individual faculty and providing a large pool of students from which to assess the efficacy of individual programs.

Changes in the funding climate can have substantial impacts on the types of programs that exist, iterative refinement of programs, and whether and how programs might be expanded to broaden participation by more undergraduates. For those institutions that have not yet established URE programs or are at the beginning phases of establishing one, mechanisms for achieving success and sustainability may include increased institutional ownership of programs of undergraduate research, development of a broad range of programs of different types and funding structures, formation of undergraduate research offices or repurposing some of the responsibilities and activities of those which already exist, and engagement in community promotion and dissemination of student accomplishments (e.g., student symposia, support for undergraduate student travel to give presentations at professional meetings).

Over time, institutions must develop robust plans for ensuring the long-term sustained funding of high-quality UREs. Those plans should include assuming that more fiscal responsibility for sustaining such efforts will be borne by the home institution as external support for such efforts decreases and ultimately ends. Building UREs into the curriculum and structure of a department’s courses and other programs, and thus its funding model, can help with sustainability. Partnerships with nonprofit organizations and industry, as well as seeking funding from diverse agencies, can also facilitate programmatic sustainability, especially if the UREs they fund can also support the mission and programs of the funders (e.g., through research internships or through CUREs that focus on community-

based research questions and challenges). Partnerships among institutions also may have greater potential to study and evaluate student outcomes from URE participation across broader demographic groups and to reduce overall costs through the sharing of administrative or other resources (such as libraries, microscopes, etc.).

REFERENCES

Pfund, C., Byars-Winston, A., Branchaw, J.L., Hurtado, S., and Eagan, M.K. (2016). Defining attributes and metrics of effective research mentoring relationships. AIDS and Behavior, 20, 238-248.

Schultz, P.W., Hernandez, P.R., Woodcock, A., Estrada, M., Chance, R.C., Aguilar, M., and Serpe, R.T. (2011). Patching the pipeline reducing educational disparities in the sciences through minority training programs. Educational Evaluation and Policy Analysis, 33(1), 95-114.

Shortlidge, E.E., Bangera, G., and Brownell, S.E. (2016). Faculty perspectives on developing and teaching course-based undergraduate research experiences. BioScience, 66(1), 54-62.

Thiry, H., Laursen, S.L., and Hunter, A.B. (2011). What experiences help students become scientists? A comparative study of research and other sources of personal and professional gains for STEM undergraduates. Journal of Higher Education, 82(4), 358-389.

This page intentionally left blank.