4

Challenges and Limitations that Arise When Evaluating Sexual Harassment Prevention Efforts

Throughout the workshop, participants discussed the challenges and limitations in developing and implementing evaluation for sexual harassment prevention programs. This chapter summarizes several of the issues and barriers identified.

Several participants discussed challenges in evaluating prevention efforts that are unique to sexual harassment. Clemencia Cosentino, National Science Foundation, underscored the importance of developing a deep understanding of the unique challenges facing evaluation of sexual harassment prevention efforts to ensure the development of rigorous methods that result in useful, actionable findings. She noted that Eden King, Rice University, had stressed that selection bias, emotional reactivity, confidentiality, legal considerations, biases, and more make this work extremely challenging. Other speakers had recognized that evaluation is complicated by the fact that sexual harassment often goes unreported. Despite these issues, participants reinforced how important it is to evaluate these programs.

Kelley Bonner, National Oceanic and Atmospheric Administration, was one of several participants who noted that a lack of knowledge, time, and experience with evaluation, including survey development, pose barriers to developing and implementing evaluation programs. Robert Brinkerhoff, Western Michigan University, reiterated this point, noting that evaluation does not have to be challenging. Brinkerhoff suggested that programs just begin to collect data to answer questions about how well, for example, a sexual harassment prevention training is being used, when it is being used,

and why. He added that training programs often get predictable results, for example, some participants will apply their learning outcomes, and some will not; thus, an evaluation of who has been reached can help to determine how to increase the effectiveness of the program. In essence, examining the failure of an intervention can be just as important as evaluating its success. Brinkerhoff noted, “Evaluation and program efforts can mature as you go.” Kurt Kraiger, University of Memphis, stated that programs should “start by doing something.”

Several participants noted that a common barrier is a lack of communication, particularly with leaders, around the need for culture change and how inclusivity can and should be nurtured. As described further in Chapter 5, Karen Stubaus, Rutgers University, said that there is a need to engage leadership on these issues, including identifying which evaluative measures to use to make the best case. Readiness for change evaluation is also a challenge, she added. She also noted that engaging leadership can be complicated by the constant change that is occurring within institutions of higher education. While change can be challenging for any organization, it is particularly difficult for institutions of higher education given that many of those in leadership positions come from faculty ranks and have not participated in leadership training, further complicating and lengthening the change process. Other participants reflected on challenges, including a lack of a long-term commitment to these efforts, continuity of evaluation and funding, and an institutional structure to support the work, noting these could be addressed with stronger leadership and support.

Several participants noted that current efforts to evaluate sexual harassment prevention efforts often do not address issues of intersectionality and identity-based discrimination. As described more fully in Chapter 5, Jennifer M. Gómez, Wayne State University, said often there is significant attention paid to developing a strong sexual harassment prevention program, while diversity, equity, and inclusion (DEI) are considered an afterthought. This is a challenge for those who are designing and implementing evaluation

programs and want to ensure that the programs are reaching those most likely to be targets of harassment. Participants added that evaluating programs through an intersectional lens may surface an additional challenge—the “small N-problem,” in which there are only a few people of color, as faculty or students, who are the focus of the program and evaluation but may not be able to share candidly in an anonymous way. King noted that examining intersections, particularly in smaller institutions where there are very few people of color, can make these groups further vulnerable to labeling or harassment.¹

It was noted by several participants that limited resources serve as a barrier for institutions evaluating their sexual harassment prevention programs. To address this, organizations facing the resource challenges (including community colleges and institutions financially affected by the COVID-19 pandemic) might consider developing partnerships as part of the solution, stated one participant. Regional partnerships and partnerships across systems of higher education can be helpful to leverage resources. Participants also suggested other resources to support organizations with limited budgets for evaluation, including the National Sexual Violence Resource Center (NSVRC) evaluation toolkit² and the Strategic Prevention Solutions Resources.³

___________________

¹ Privacy is an important consideration when evaluating the experiences of underrepresented populations, especially people of color, sexual and gender diverse individuals, and those with disabilities, who experience higher rates of harassment than their white, cisgender, and heterosexual colleagues (Basile et al., 2016; Cortina, 2004; Lombardi et al., 2002; Settles et al., 2008).

² Available: https://www.nsvrc.org/prevention/evaluation-toolkit.

³ Available: https://www.strategicpreventionsolutions.com/digitaldownloads.

This page intentionally left blank.