Appendix C
Improving Representativeness in Clinical Trials and Research: Facilitators to Recruitment and Retention of Underrepresented Groups
*Franchesca Arias, Ph.D. (1), (2), (3); *Nicole Rogus-Pulia, Ph.D., C.C.C.-S.L.P. (4), (5); and Amy J. H. Kind, M.D., Ph.D. (4), (5), (6)
Affiliations:
- Aging Brain Center, Hinda and Arthur Marcus Institute for Aging Research at the Hebrew SeniorLife, Boston, Massachusetts
- Department of Neurology, Beth Israel Deaconess Medical Center, Boston, Massachusetts
- Harvard Medical School, Boston, Massachusetts
- Department of Medicine, Division of Geriatrics and Gerontology, University of Wisconsin-Madison, School of Medicine and Public Health, Madison, Wisconsin
- Madison VA Geriatrics Research Education and Clinical Center, Middleton VA Hospital, Madison, Wisconsin
- Center for Health Disparities Research, University of Wisconsin–Madison, Madison, Wisconsin
*Contributed equally as first authors.
ABSTRACT
Objective: Clinical trials are essential for determining safety and efficacy of health-related interventions as well as informing future research and funding priorities in the United States. However, recruitment and retention challenges result in underrepresentation of diverse groups in clinical trials, which limits understanding of disease mechanisms as well as generalizability of findings. The purpose of this study was to elucidate facilitators to recruitment and retention strategies of underrepresented groups, based on a representative sample of published clinical trials with successful inclusion.
Method: A mixed-methods approach was employed to accomplish these aims. Research teams with experience recruiting underrepresented groups were invited to participate in individual comprehensive interviews. Twenty interviews were completed that focused on understanding facilitators to recruitment and retention into clinical trials. To identify studies appropriate for in-depth qualitative interviews, we first conducted a systematic review of published clinical trials available on PubMed between 2001 and 2021 across the top six diseases leading to mortality in the United States (heart disease, cancer, chronic lower respiratory disease, stroke, Alzheimer’s disease, and diabetes). From these trials, we randomly selected 162 trials stratified by disease and geographic location. We then benchmarked the number of study participants by race, ethnicity, and sex against the local (single site) or national (multisite) data as reported by the American Community Survey Demographic and Housing Estimates years 2010 to 2019. Study teams for those trials that met diversity criteria (50 percent or greater recruitment in at least one category) were invited to participate in the interviews.
Results: Of the 162 randomly selected trials, 142 met diversity criteria following benchmarking. Incomplete reporting of sample characteristics was observed in the majority of studies; however, 96 percent of trials reported the sex of their study participants. Of the trials that achieved success in recruiting representative samples, less than 33 percent reported information about ethnicity and less than 66 percent of trials included a robust breakdown of the racial representation. Of the 142 study teams invited, 20 participated in the interviews. Results from qualitative analysis interview transcripts revealed eight main themes with associated subthemes: (1) starting with intention and agency to achieve representativeness; (2) establishing a foundation of trust with participants and community; (3) anticipating and removing barriers to study participation; (4) adopting a flexible approach to recruitment and data collection; (5) building a robust network by identifying all relevant stakeholders; (6) navigating scientific, professional peer, and social expectations; (7) optimizing study team to ensure alignment with research goals; and (8) attaining resources and support to achieve representativeness.
Discussion: While issues of representativeness in research have been at the forefront of science in the past 10 years, additional efforts are necessary to systematically assess and comprehensively report social and cultural characteristics of cohorts in peer-reviewed publications of clinical trials. While intentionality drives current scientific efforts to understand how diseases affect persons from diverse groups, this work remains underfunded and undervalued. A call to action that involves providing resources, expanding the definition of stakeholders, integrating community-based stakeholders as equitable partners, and involving national funding organizations, academic institutions, and the scientific community is necessary to meaningfully advance work in this area.
Keywords: recruitment, retention, minority, representation, clinical trial
INTRODUCTION
Clinical trials provide the most robust evidence to document the efficacy and safety of pharmacological and non-pharmacological interventions. Moreover, clinical trial evidence is often used to identify areas for future research as well as to guide funding priorities and allocation of resources. However, recruitment challenges often hinder the utility and generalizability of clinical trials. Recent data from the National Institutes of Health (NIH) indicate that less than 20 percent of clinical trials in the United States meet their recruitment targets, and up to 80 percent of these studies require extensions as a result of low enrollment (Clinical Trials Arena, 2012). Recruitment challenges are even more pronounced when considering representation of diverse groups.
Representation in clinical trials is particularly important in the context of the changing U.S. demographics. By 2045, it is anticipated that nearly half of the U.S. population will self-identify as ethnoracially diverse (Census, 2018). Persons who self-identify as Black/African American, American Indians/Alaska Native, Asian/Asian American, Native Hawaiian, or Hispanics/Latino(a) are more likely to be poor and underinsured. Moreover, persons from these and other historically underrepresented groups experience increased disease burden from common conditions such as heart disease, diabetes, asthma, obstructive pulmonary disease, stroke, obesity, and liver disease (Carratala and Maxwell, 2020) For example, persons who self-identify as lesbian, gay, bisexual, transgender, or queer (LGBTQ+) have unique health experiences and are disproportionately affected by mental health conditions and sexually transmitted diseases (SAMHSA, 2012). There is substantial underrepresentation of diverse groups in clinical research. Data published by the Food and Drug Administration (FDA) suggest that participants enrolled in clinical trials for pharmacological interventions overwhelmingly self-identified as non-Hispanic white (81 percent), with 4 percent Black/African American and 12 percent Asian/Asian American. Ethnicity is inconsistently reported. When available, Hispanic/Latina(o) represent about 11 percent of patients enrolled in
pharmacological trials (Duma et al., 2018; FDA, 2020; Frew et al., 2014; Gong et al., 2019; Khan et al., 2020; McCarthy-Keith et al., 2010; Rosende-Roca et al., 2021; Vitale et al., 2016). These numbers do not reflect the current U.S. population, in which 14 percent of persons in the United States identify as Black/African American, 7 percent as Asian/Asian American, 2 percent as American Indian, and 18 percent as Hispanic/Latino(a) (Census, 2019). Lack of adequate representation threatens the integrity of science. Interventions evaluated on a subset of the population and under circumscribed settings may not realistically generalize to other groups and settings (Haidich and Ioannidis, 2001). For certain medical conditions (e.g., asthma, heart failure, cancer), drug response profiles may differ based on ethnoracial factors (Jamerson and DeQuattro, 1996; Tay et al., 2020); however, there is not currently enough representation of low-income and non-white persons in drug trials to determine whether social or biological factors are associated with differential responses to drugs (Odierna and Bero, 2009). Finally, lack of representation and limited reporting on the social and contextual factors influencing disease trajectories may interfere with replicability of findings (Glasgow et al., 2018) and our ability to identify mechanisms underlying diseases (Ix et al., 2008).
The NIH has implemented initiatives designed to foster the inclusion of underrepresented groups in NIH-supported clinical research trials (NIH, 2001). Similarly, the FDA implemented reporting requirements and issued a recommendation for sponsors of clinical trials to increase enrollment of underrepresented populations (FDA, 2014). Nevertheless, underrepresentation of diverse groups in clinical trials persists (Nazha et al., 2019). Engaging in strategies that will ultimately increase recruitment and retention of underrepresented groups in clinical trials will result in diverse samples that more appropriately reflect the population who will ultimately utilize, and who stands to benefit from, the intervention. Importantly, increased representativeness in clinical trials assures the efficacy and safety of treatments in these diverse subgroups.
The purpose of this study was to characterize current efforts on representativeness in clinical research and to systematically assess effective recruitment and retention strategies. Finally, we hope to recommend strategies that can be used by scientists to diversify clinical trial participant populations.
METHODS
We developed a novel Systematic Randomized Qualitative Assessment (SRQA) methodology that incorporated both quantitative and qualitative techniques to answer our question in a balanced and inclusive manner. Since our goal was to elucidate facilitators for recruitment and retention strategies of underrepresented groups, we first needed to identify studies with successful recruitment. Thus, we performed a systematic search of clinical trials published in PubMed between 2001 and 2021. We aimed to identify U.S.-based clinical trials that suc-
cessfully recruited historically underrepresented groups as assessed by objective population-representative benchmarking criteria (described below). Figure C-1 offers an overview of this process according to the 2012 PRISMA guideline for systematic reviews (Moher et al., 2012).
Inclusion and Exclusion Criteria
The following inclusion criteria were applied to the studies identified from the systematic review: (1) manuscripts written in English, (2) recruitment conducted only in the United States, (3) publication in a peer-reviewed journal, (4) inclusion of adult participants (age 18 years and over), and (5) recruitment completed by time of publication. Secondary analyses were included if prospective data were collected, and the new data were available to complete benchmarking, or if original data were available for benchmarking.
First, we prioritized our search in clinically relevant areas. As such, we focused our search on clinical trials addressing the top six causes of mortality in the United States according to the Centers for Disease Control and Prevention (CDC): heart disease, cancer, chronic lower respiratory disease, stroke, Alzheimer’s disease, and diabetes. Unintentional injuries and intentional self-harm were excluded to maintain a focus on medical conditions (CDC, 2020). Our exclusion criteria were (1) non-interventional studies, (2) dissertations, (3) non-human studies, (4) case studies, and (5) meta-analyses (see Figure C-1).
Given that representation of diverse groups differs across U.S. regions (Frey, 2019), our methodology sought to ensure that all areas of the United States were represented in the search. To this end, we stratified our search by using geographic filters in PubMed based on the nine U.S. Census Divisions: New England, Middle Atlantic, East North Central, West North Central, South Atlantic, East South Central, West South Central, Mountain, and Pacific (Lewison, 1997). These filters were based on the address of the corresponding author for the published trial and were added to search terms for each disease category. Given that PubMed is the only available search engine that allows for this type of custom geographic filter specific to U.S.-based trials, other search engines were not included (see Appendix C-1 for search terms that were applied).
Once the study was selected, we performed full text review to confirm the study recruitment location, and any discrepancies were resolved. To avoid penalizing research studies located in regions with low representation of target groups, studies were benchmarked to data from the American Community Survey Demographic and Housing Estimates years 2010 to 2019 collected by the U.S. Census Bureau using the region of recruitment. County-level data were used for single-site studies. Multisite studies conducted within the same state were benchmarked using state-level data. Studies that included sites dispersed across multiple states were benchmarked using national-level data.
Our goal in this stage was to identify a group of trials, stratified by disease condition and geographic division, which met our prespecified definition of diverse recruitment. In total, three trials from each of the nine regions for each of the six disease categories were to be identified (3 × 9 × 6 = 162). To accomplish this, we first stratified the trials into six CDC disease categories and then subdivided the strata into the nine U.S. Census divisions. Next, we randomly selected studies from each division by disease cell and applied our inclusion and exclusion criteria until an eligible trial was identified. We repeated this process until we reached the target number of three eligible trials per disease condition by geographic division (n = 162) (see Table C-1).
Data Extraction
Once the final set of 162 eligible trials was identified, data extraction was carried out independently by four reviewers using an extraction form specifically designed for the purpose (see Appendix C-2). The form was pilot tested for feasibility and reliability on five sample trials prior to use. Extraction was completed in duplicate for each trial, and any discrepancies between reviewers were resolved through consensus. Data were extracted on clinical trial study design, study setting, study inclusion and exclusion criteria, and interventions. In addition, information about the characteristics of the included study sample (e.g., age, race, ethnicity, sex/gender, gender identity, sexual orientation, disability status) was extracted, with a special focus on the demographics for underrepresented groups. Underrepresented groups were identified according to the fundamental causes of disparities listed in the National Institute on Aging (NIA) Health Disparities Framework (Hill et al., 2015).
Variable Definitions
Representativeness of the cohort was reported using a multilevel approach.
Level 1: NIH-Mandated Diversity Variables
These diversity variables were defined as characteristics currently required for reporting by the NIH in the targeted enrollment tables that are mandatory for every clinical trial since March 1994 (Taylor, 2008). These variables include the number and proportion of participants recruited by sex, ethnicity, and race with data presented according to well-established classifications (Riley et al., 2018). Detailed information about ethnic/racial groups was abstracted when available, as follows: (1) for Hispanic/Latino(a) participants, we reported whether a cohort was from the Caribbean, Central America, North America, or South America; (2) for Asian and Pacific Islander participants, we reported whether the cohort was predominantly from Eastern Asia, South Central Asia, Southeastern Asia, or Western Asia; (3) for American Indian or Alaska Native participants, we reported
whether the cohort was predominantly from the Great Plains Area, Alaska Area, Albuquerque Area, Bemidji Area, Billings Area, California Area, Nashville Area, Navajo Area, Oklahoma Area, Phoenix Area, Portland Area, or the Tucson Area; (4) for African American or Black participants, we reported whether the cohort was predominantly from African origin, Caribbean, and Other origin.
Level 2: Intersectionality
Information about participants’ intersectionality, or the interconnectedness among social groups and how these intersections confer unique privilege or vulnerabilities, was collected if these characteristics were described in the published manuscript (e.g., multirace; LGBTQ+ participant from racially underrepresented groups) (Crenshaw, 2017). Intersectionality of study participants (sex + race + ethnicity) was recorded in a binary form (characterized or not characterized).
Level 3: Other Underrepresented Groups
Recruitment of individuals from other underrepresented groups (80 years and older, disability, gender identity, and sexual orientation) was also recorded in binary form (characterized or not characterized for each diversity category).
Outcomes/Benchmarking
The primary outcome of interest was the diversity achieved in each study cohort as compared to census data for the specific region, time frame, and specific diversity variable of interest. The goal of our rubric was to assess representativeness for each trial, taking into consideration the characteristics of the population available to those researchers in their communities. As such, to avoid penalizing research studies located in regions with low representation of target groups, studies were benchmarked to data from the American Community Survey Demographic and Housing Estimates years 2010 to 2019 (U.S. Census Bureau data). For single-site studies, county-level data were used. For multisite studies, state-level data or national-level data were used, depending on whether sites were within the same state or dispersed across the United States. “Successful” studies were those that recruited 50 percent or greater of the proportional target population in their region (i.e., 50 percent of the county-level base rate, the state-level base rate, or the national-level base rate) for the targeted demographic characteristic (sex, ethnicity, or race). This process was independently assessed by two reviewers.
Qualitative Study: In-Depth Interviews
The next step was to conduct in-depth qualitative interviews to provide a comprehensive assessment of their recruitment strategies, with particular interest
in practical examples of successful and innovative strategies. To this end, the corresponding author on each benchmarked trial with “successful” recruitment of diverse groups was invited to participate (or to designate an appropriate recruitment staff member to participate) in these interviews. We contacted 142 corresponding authors for studies that met the inclusion criteria via email. Interview requests were submitted in batches of 30. All eligible trials were contacted at least twice. Of these, 40 investigators responded, 5 of whom declined participation. The most common reasons for declining participation included lack of involvement in study recruitment or retention and lack of time for the interviews. Scientists from 20 studies ultimately agreed to participate in the interviews (see Figure C-1).
Our interview guide (see Appendix C-3), which focused on facilitators to recruitment and retention of diverse groups, was pilot tested and iteratively refined during the first four interviews. Interviews were conducted with individuals using open-ended questions, such as “Tell us about the strategies you implemented to enhance recruitment of underrepresented populations in your study.” Specific probes asked such questions as “Did you engage stakeholders in your study? If yes, at what point in the process were they involved?” The Model Framework of Multilevel Factors Affecting Decision to Participate in Clinical Trial, which was proposed by Ford et al. (2013), guided the development of our clinical interview and general approach to the qualitative portion of this project.
To ensure rigorous data collection, all interviews were conducted by video, audio-recorded, and transcribed verbatim with all identifying names or other personal health information removed. Interviews lasted 45 minutes on average, and we conducted interviews until adequate saturation was reached for thematic content analysis (no new concepts detected for at least three interviews).
Qualitative data were iteratively coded, sorted, and compared using thematic analysis by two raters (Boyatzis, 1998). An initial codebook was developed during piloting, and two raters separately identified tentative themes and subthemes using data from the pilot interviews by conducting line-by-line coding using NVivo 11 software (QSR International). After reconciling differences in the inductive codes, we updated the coding scheme. Knowledge gained from each interview was incorporated into subsequent sessions to refine questions and explore salient themes. During coding, inter-rater reliability was assessed and any discrepancies were resolved through consensus. To ensure the rigor of the analytic approach, we implemented several processes, including peer debriefing, independent and collaborative coding, refinement of themes by examining supporting and contradictory cases, and documentation of a decisional audit trail (Frey, 2019).
This study was reviewed by the institutional review boards (IRB) of the National Academies of Sciences, Engineering, and Medicine, the University of Wisconsin–Madison, and Beth Israel Deaconess Medical Center/Harvard Medical School, and was determined to be exempt from IRB review.
RESULTS
Quantitative Categorization of the Diversity of Study Cohorts
A total of 131,028 clinical trials were identified using the search strategy in Appendix C-1 across all six disease categories and nine geographic divisions. Numbers within each category of disease and geographic region are displayed in Table C-1.
A total of 1,279 clinical trials were randomly selected, stratified by disease category and geographic region. Careful review of abstracts and, when available, trial information on ClinicalTrials.gov, was performed applying inclusion and exclusion criteria to identify 162 eligible trials (three trials for each of the six CDC disease categories across the nine U.S. Census regions). More than 1,100 trials were excluded (see Figure C-1 for exclusion reasons).
TABLE C-1 Number of Trials in Each Disease Category by U.S. Census Geographic Regions and Divisions
| U.S. Census Regions | U.S. Census Divisions | Heart Disease | Cancer | Chronic Lower Respiratory Disease | Stroke | Alzheimer’s Disease | Diabetes |
|---|---|---|---|---|---|---|---|
| 1 - Northeast | 1. New England | 3,836 | 1,350 | 434 | 337 | 141 | 5,429 |
| 2. Middle Atlantic | 5,410 | 2,103 | 580 | 422 | 176 | 6,349 | |
| 2 - Midwest | 3. East North Central | 4,266 | 1,474 | 429 | 347 | 151 | 5,888 |
| 4. West North Central | 2,848 | 838 | 288 | 221 | 85 | 3,272 | |
| 3 - South | 5. South Atlantic | 12,161 | 4,311 | 1,643 | 972 | 292 | 19,306 |
| 6. East South Central | 2,176 | 587 | 180 | 161 | 27 | 3,056 | |
| 7. West South Central | 4,144 | 1,851 | 400 | 346 | 104 | 6,008 | |
| 4 - West | 8. Mountain | 2,277 | 1,009 | 405 | 190 | 126 | 4,327 |
| 9. Pacific | 5,425 | 2,264 | 688 | 451 | 243 | 9,224 |
Met Diversity Criteria
Study teams met diversity criteria if they recruited 50 percent or greater in at least one category: race, ethnicity, and sex relative to county (single site) or state/national (multisite) data as reported by the American Community Survey Demographic and Housing Estimates years 2010 to 2019.
Did Not Participate in Interview
Researchers that met diversity criteria were contacted sequentially in batches of 30. All eligible researchers were contacted twice within a 3-week period.
Benchmarked Trials
All 162 eligible trials were benchmarked in each of the NIH-mandated diversity categories (sex, race, and ethnicity) against local county-level U.S. Census
data (for single-site study) and state- and national-level data (see Appendix C-4). Of the 162 eligible trials, 142 trials (88 percent) met the preestablished diversity criteria of 50 percent or greater recruitment in at least one of the three NIH-mandated categories (sex, race, and ethnicity). Of those, 53 percent met criteria for recruiting female participants only, and about 1 percent did not describe their sample in terms of sex; 64 percent of the trials that met the preestablished diversity criteria did not report details on the ethnicity of their participants; and 3 percent of trials met criteria for one of the four racial groups recorded by the U.S. Census (African American or Black [n = 4], American Indian or Alaska Native [n = 1], Native Hawaiian or Pacific Islander [n = 0], or Asian [n = 0]). One trial met criteria for recruiting multiple races (n = 1). About 46 percent of trials did not report information on the racial breakdown of their sample, and many reported race as “White” and “Other than White.” Thirty-two percent of trials met criteria for sex and race; 1 percent met criteria for sex and ethnicity; 1 percent met criteria for race and ethnicity; and 7 percent met criteria for all three categories.
Regarding intersectionality, only 5 percent of the trials reported whether participants in their sample self-identified as belonging to more than one underrepresented group. Similarly, only 7 percent of studies reported information on the disability status of their participants. Self-reported gender identity and/or sexual orientation were not included in any of the trials reviewed.
Qualitive Assessments of Trials that Succeeded in Recruiting a Diverse Cohort
A total of 20 study teams and 22 participants were included in the qualitative portion of the study (18 single and 2 two-person interviews). The average age of participants was 49.8 (13.9) [M(SD)] with a range of 27 to 73. Overall, 74 percent self-identified as female, and no participant self-identified as nonbinary. Participants self-identified as Hispanic/Latina(o) (11 percent), non-Hispanic white (53 percent), Black/African American (11 percent), and Asian (26 percent). Native American/Alaska Native or Native Hawaiian/Pacific Islander scientists were not represented in this study.
In the thematic analysis, eight major themes emerged that were broadly related to representativeness in clinical research: (1) starting with intention and agency to achieve representativeness; (2) establishing a foundation of trust with participants and community; (3) anticipating and removing barriers to study participation; (4) adopting a flexible approach to recruitment and data collection; (5) building a robust network by identifying all relevant stakeholders; (6) navigating scientific, professional peer, and social expectations; (7) optimizing study team to ensure alignment with research goals; and (8) attaining resources and support to achieve representativeness. Major themes and subthemes are listed and described below. Additional details and quotes are provided in Tables C-2a to C-2h.
Starting with Intention and Agency to Achieve Representativeness
Participants described the work with diverse communities as resource, time, and labor intensive. For example, “It’s a lot of work and a lot of time and it takes years. . . . We’ve been working with the same community partners now for 12, 13 years. They see us all the time.” They explained that the intention must be based in the reality that a multistage process is required to achieve representativeness. That is, contact with community members begins long before recruitment and extends long after research support ends, and resources ought to continue to effect changes in communites long after the study ends. Participants emphasized that collaboration with community members specific to recruitment and retention strategies occurs across different stages of the study. For example, “I think some of the principles that are laid out for stakeholder engagement basically involve them in the design of the study, the conception of the study, what questions you’re asking, as well as in how you’re doing, the recruitment, who you’re recruiting, what your materials are, and then what the study involves, like kind of soup to nuts kind of thing. And so I try to do that as much as I can.”
Greater than 80 percent of the participants reported that being intentional about having representation of historically underrepresented groups was instrumental to their success. Intrinsic motivation stems from a personal commitment to promote equity and erradicate health disparities, ethical and professional values fueling the desire to elucidate the biological and contextual mechanisms driving the effectiveness of an intervention, and a genuine scientific curiosity to understand disease in different patient groups. For example, “We don’t know how they respond to different interventions. We just don’t know what the differences are, we don’t have nearly enough data,” and “I need to tell you about a comment from one of the Black leaders . . . when we were discussing the . . . drug . . . that has only seven percent African-American and then one Native American in the entire study. . . . This is disrespecting the Black body in the same way as slavery. You’re not respecting people. You’re the ones that sit at these tables where we are not, how dare you all put out a drug that everyone can’t use as if we’re not dying from this disease.”
Frequent subthemes were the perception that clinical trials provided access to innovative treatments. For example, “It’s absolutely important in terms of behavioral interventions and how you implement [with] certain people or not if you don’t have access to the things that people of high social economic status take for granted. If you don’t have that kind of access, then you’re not going to be as able to implement any intervention, especially behavioral ones that require changes in lifestyle, taking time out of your day and stuff like that.” Under these themes, the view that research is an endeavor largely funded by taxpayers that should benefit taxpayers across all socioeconomic levels was also evident. For example, when I go to the talks, I say, this is your . . . these are your tax dollars at work. This is your money. You need to benefit from this too. You and your friends and family, let’s get everybody on board.” Extrinsic motivation came from external factors, including requirements by funding agencies, parameters imposed by the environ-
ment (e.g., need to recruit from a given state or setting), and factors driven by the characteristics of the diseases, such as base rates.
Establishing a Foundation of Trust with Participants and the Community at Large
The idea of building and maintaining trust with both study participants and their larger communities was reported by 100 percent of the participants. History of abuse by researchers, experiences with other research groups that approached underrepresented communities for the purposes of a study and did not remain engaged, and beliefs that research is not beneficial to the community were cited as barriers to establishing trust with persons from diverse communities. For 85 percent of the participants, the development of trust requires a long-term commitment by the principal investigators, study team, and local institutions that benefit from this research. Respondents described the necessity of building trust over time through consistent engagement in the community, developing meaningful relationships with study participants, and giving to the community without the expectation of anything in return. Many participants emphasized that while trust has to be built over time, trust can be broken with individuals and communities in an instant. For example, “There’s such trust building, that . . . takes awhile. And if one person drops and doesn’t keep the trust, then I’m not going to be able to most likely get back that location again.” An approach to community partnership that is truly equitable and not hierarchical in nature (15 percent) was suggested as a way to mitigate distrust in these communities and to most effectively leverage resources for truly meaningful and translatable work. For example:
I think that is the goal to get to full equity with the community partner, writing the grants and getting the money and sharing everything from the ground up to the study. I think we’re still unequal with academic partners. So doing a grant writing, getting the funding and working with community partners and giving them funding from the grant. So I think there’s still this hierarchy. Unfortunately, we’re trying to break those down. We’re trying to get to parity as much as possible. And that’s just going to take time and it’s going to take investment.
According to 20 percent of the participants, in addition to facililtating recruitment, establishing relationships with community leaders provided opportunities to understand the needs of the community in order to build trust over time.
Anticipating and Removing Barriers to Study Participation
Included here are aspects of clinical trial participation that may influence accessibility to research for members of underrepresented communities. High-priority subthemes focused on barriers to participation and removal of these barriers. Systemic barriers included complex consent language, lengthy research visits, research activities that place undue burden on participants (e.g., requiring them to miss work), or issues related to physical access (e.g., driving many hours to participate in person) that are unique to the research enterprise. Other barriers were sociocultural and revolved around the goodness of fit between the
participant’s values and characteristics and that of the research staff and research materials. Seventy percent of participants also discussed the importance of recognizing heterogeneity within cultural groups. They explained that taking an individualized approach, without compromising the science, may allow researchers to acknowledge individual experiences. Other solutions include collaboration with interpreters to provide services to non-English-speaking prospective participants and/or providing options for in-home or remote visits to overcome lingustic and physical access barriers, respectively.
Adopting a Flexible Approach to Recruitment and Data Collection
Seventy-five percent of the participants endorsed the importance of flexibility for the successful recruitment and retention of diverse groups. Participants frequently described recruitment strategies evolving as studies progressed. Recruitment techniques were incorporated or abandoned in response to study needs, and changes were guided by community representatives and relevant stakeholders. This adaptability extended beyond recruitment. For example, flexibility at the time of data collection was reported as necessary to retain participants (50 percent), particularly those with limited resources or constraints on their time due to competing demands (e.g., childcare).
Building a Robust Network by Identifying All Relevant Stakeholders
Eighty percent of the participants discussed the importance of identifying all stakeholders, highlighting the major role they played in informing study design and driving recruitment and retention of diverse participants. Who is considered a stakeholder, and their level of involvement, varied based on cultural preferences of the prospective participants, the condition being studied, and the nature of the research study. The term stakeholder was defined broadly to include caregivers, family members, friends, clinical providers and administrators, community advocates, peers, religious leaders, and political figures. Developing relationships with caregivers and family members was identified as instrumental to recruitment and retention of underrepresented groups. For example, “I realized that not talking to caregivers was a pretty big misstep in our original trial. If you have these populations that are vulnerable enough to have caregivers and other people who are already kind of with them maybe consider including them as part of the trial and obviously with patient consent, sort of incorporating it.” Community advisory boards and other strategies for eliciting commiunity expertise were crucial to protocol development and study execution emerged in the context of this discussion. Finally, conceptualizing study participants as partners in research was an important component that required openness by the study team to learn from the participants’ experiences.
Navigating Scientific, Professional Peer, and Societal Expectations
In the context of this theme, the participants described challenges related to acknowledging scientific and societal expectations while striving to maintain
scientific rigor. Many participants perceived that efforts to promote representativeness, and decisions made to support these efforts, are not entirely appreciated by peers and organizations responsible for making funding decisions and/or budget decisions. Twenty-five percent of the participants described how creative strategies designed to engage communities that have traditionally been excluded from research are evaluated relative to more traditional strategies, which tend to be rigid. Thus, researchers are encouraged to use traditional approaches to retention and recruitment, which may be burdensome for prospective participants with multiple vulnerabilities, and may result in less participant diversity.
Another subtheme revolved around the incongruence between current emphasis on recruitment and retention of diverse participants and the consistent underfunding of researchers applying for grants to conduct this work. For example, “It seems that there’s a real incongruence where the NIH is saying disparities work, disparities work, disparities work, and then you put it in and reviewers don’t acknowledge the disparities aspect. They are fixated on errors in your approach or concerns about your theoretical model, and so it does seem that there is an incongruence in the way that the funding source of NIH wants to value efforts to recruit and retain these folks and then the way that it’s reviewed. So that is an issue.” Participants emphasized that efforts to be intentional and plan ahead to prepare for additional costs related to this work are undermined due to budget constraints. Seventy percent of the participants suggested that funding agencies, as well as those responsible for approving proposals and distributing budgets, should be required to gain competencies in nontraditional methodological research approaches.
Optimizing Study Team to Ensure Alignment with Research Goals
All of the participants described the composition of the study team as an important component of representative research. Study staff interact with potential study participants and are instrumental in recruitment and retention success. Diverse study teams were generally described as being helpful to recruitment given congruence between staff and potential participants, and this congruence was described in different ways depending upon the focus of the study (e.g., age, sex, race, ethnicity). Of note, 25 percent of participants added that cultural and linguistic congruence with the target population was not enough. That is, commitment to the study and its outcomes were as important when working with diverse communities. Retaining study staff over time was emphasized as very important to recruitment and retention success; however, this was also described as a challenge given issues with staff salaries.
Attaining Resources and Support to Achieve Representativeness
A variety of resources are needed to accomplish the goal of a representative sample. Eighty percent of the participants considered time and money as ultimately the most instrumental material resources necessary to conduct this kind
of work successfully. With this in mind, funding support for these recruitment efforts was a main focus for specific funding announcements focused on underrepresented groups, expanded budgets for teams attempting to recruit and retain these groups, and flexibility within budgets to allow for deeper engagement of community partners. In addtition to funding, participants emphasized education of researchers and supports in the form of professional networks and institutional resources. Finally, material support for community organizations so that they can build infrastructure also emerged as part of this theme. In particular, resources that could assist these organizations in building the foundation for research would bolster these efforts for successful partnerships.
DISCUSSION
Employing our unique SRQA mixed-methods approach for this study, we examined facilitators to recruitment and retention in clinical trials. We conducted 20 in-depth qualitative interviews with researchers from U.S.-based studies who succeeded at recruiting diverse samples. In addition, we examined reporting practices of cultural and demographic sample characteristics in 162 randomly selected U.S.-based clinical trials published between 2001 and 2021. This approach provided the opportunity to learn about the unique and innovative techniques being incorporated by study teams nationwide, regardless of study size and/or national recognition.
This qualitative work sought to elucidate novel recruitment and retention strategies incorporated by researchers who have been successful at achieving representativeness in their cohorts. Several themes emerged that revolved around having intentionality and agency, building trust, recognizing heterogeneity, adopting a flexible approach to recruitment and data collection, and appreciating stakeholders. The characteristics of the study staff figured predominantly in the discussions, and involving and retaining experienced study staff was identified as a key ingredient to success by most of the participants. Barriers to recruiting and retaining experienced staff included low pay, job insecurity, and devaluing of their expertise based on lack of formal training. The findings suggest that equitable distribution of resources must extend to all levels of the research with underrepresented groups, community organizations, community advocates, caregivers, participants, research staff, and principal investigators. A genuine commitment to recognizing and respecting the contribution of each stakeholder is needed for success and sustainability.
Overall, most of the participants emphasized that expedient time frames and budget restraints inherent in existing funding mechanisms through the NIH and other agencies (e.g., the R01 mechanism) are not adequate to support research that seeks to include underrepresented groups. To develop the infrastructure necessary to support these efforts, flexible funding mechanisms that allow for inclusion of community partners will be essential. Additionally, specific funding
announcements with a focus on supporting efforts to recruit and retain underrepresented groups will be needed going forward. In the context of institutional support, many participants called for academic health centers, which play a major role in employment of community members, maintaining the health of communities, and providing outreach to communities across the country, to build relationships with diverse communities that would, over time, allow for systems-level barriers to be addressed and trust to be established. These results suggest that financial support must precede additional calls to increase representativeness.
All of the participants expressed enthusiasm for the increased focus on representativeness by professional institutions, funding agencies, and scientists. They reported that shedding light on these issues is instrumental for science to remain relevant and consistent with its fiduciary duties to taxpayers who ultimately fund, at least in part, these efforts. The study participants believe that while most scientists value representativeness in research, many professional peers, who review grants and make decisions about funding priorities, are not receptive to research proposals demonstrating flexibility and adaptability. As such, resources in the form of training, support, and increasing representation of scientists from diverse cultural and academic backgrounds in review panels and positions of power is instrumental to this effort.
These results align with findings from prior research focused on participant-reported barriers and facilitators to enrollment in clinical trials (Ejiogu et al., 2011; Ford et al., 2013; George et al., 2014; Gilmore-Bykovskyi et al., 2019). The model proposed by Ford et al. (2013), which was informed by Hispanic/Latina(o) and Black/African American adults, organizes the sources of racial and ethnic disparities in recruitment in clinical trials across three major areas: (1) characteristics of study processes, (2) characteristics of health researchers, and (3) preferences and attitudes of community members and potential trial participants toward clinical trials. This study extends beyond these three areas by capturing systems-level issues related to the characteristics and values of academic institutions, the need for increased funding support for these efforts, and recognition of the importance of representativeness in clinical trials as a social justice issue. In a recent systematic review, George et al. (2014) identified shared and distinct facilitators and barriers to participation in research among persons from diverse backgrounds. Overall, cultural congruence between the study and the target community, benefits to participation, altruism, and convenience were listed as major facilitators to recruitment. In addition to cultural congruence (i.e., how good of a fit between the participants’ linguistic, racial, and ethnic background and that of the research staff), many of the participants reported that successful recruiters valued research and appeared uniquely invested in understanding the experiences of those living with a given condition. According to George et al. (2014), participants from diverse ethnic and racial groups reported distinct barriers to participation. Many of the participants in our study cautioned that, even when techniques
are implemented a priori to address these barriers, flexibility is instrumental to this work, as heterogeneity exists between and within different groups.
Across most of the randomly selected studies, incomplete reporting of sample characteristics was observed. While 90 percent of the trials (n = 162) reported information on the sex/gender of their sample, none described the sexual orientation of their study participants. Furthermore, less than 50 percent, 30 percent, and 20 percent of the studies reported whether their participants self-identified as African American, Asian and Asian Americans, and American Indian or Alaska Native, respectively (see Appendix C-4). Information about the representation of Native Hawaiian or Pacific Islander was reported by less than 10 percent of the trials. Most of the trials included in this study (90 percent) did not report information about the socioeconomic characteristics, disability status, and living arrangements (i.e., homeless or not) of their participants. Issues around intersectionality, or whether participants reported belonging to more than one group that has been historically marginalized, were not explicitly reported.
While 88 percent randomly selected trials achieved success in recruiting a representative sample, 53 percent of these were determined to meet criteria based solely on recruiting females. Several participants indicated that inclusion of women in their studies was due more to factors such as age (Graaf et al., 2018), disease (Duma et al., 2018), and the nature of the research trial (Vitale et al., 2017) than to intentional approaches to recruiting women. Regardless of the reason for adequate representation of women in these trials, this remains an important finding and focus for future work. Although women live longer than men, they report an increased number of years living with functional limitations (Zunzunegui et al., 2015), and older women score significantly lower on most indicators of subjective well-being and mental health than their male peers.
Less than 66 percent of the trials included a robust breakdown of the racial representation in the sample in ways that supported benchmarking (i.e., an “Other” category was used to describe anything other than Non-Hispanic white). These results suggest that despite consistent efforts to raise awareness about the importance of recruiting representative samples, including recommendations by the NIH and FDA to consider fundamental causes of disparities in research, improvement in this area is imperative. Calls to recruit representative samples must be accompanied by clear guidelines on how to comprehensively collect such characteristics. Moreover, consistent reporting of comprehensive sample characteristics may require explicit recommendations from journals as well as national institutions and organizations.
Several limitations merit mention. First, the definition of “success” focused on meeting U.S. Census targets based on regional proportions of diverse populations. However, chronic conditions are disproportionately distributed across different cultural groups. In an effort to account for variability in base rates, we selected trials across six disease types in the United States. Future researchers
should examine these findings in the context of an expanded disease criteria that includes suicidality, substance use disorders, and other neurodegenerative conditions. Second, regional representation for a given trial was determined using geographical filters included only in PubMed. Additionally, the corresponding author’s information, which is used to populate regional information in PubMed, does not necessarily reflect the location of recruitment. This represented a challenge to our explicit intention to benchmark to county, state, and national data. To the extent possible, our research team confirmed whether recruitment site was different from the corresponding author’s site, and studies were benchmarked using U.S. Census data from where the study was conducted. While thematic saturation was met for this specific cohort of researchers, we acknowledge that it is possible additional themes could emerge if time allowed for inclusion of an expanded cohort of more diverse study team members.
CONCLUSIONS
This study highlights several strategies to promote representativeness in clinical trials and provide practical and innovative recommendations for relevant stakeholders in the field: peers, journals, and funding agencies. Ultimately, efforts to improve representativeness must involve provision of financial resources for research teams, material and social support for community advocates and organizations, and education about the relevance of these efforts to scientists, community members, and allied professionals. Priority funding should anchor research activities on representativeness, with community stakeholders at the forefront of every consideration.
Acknowledgments
The authors would like to acknowledge the participants who gave so generously of their time and expertise. We would also like to acknowledge the contributions of the National Academies of Sciences, Engineering, and Medicine committee. We are also grateful to Michael Davenport, Mary Hitchcock, Nicholas Picanso, Payton Sheridan, Nicole Razdolsky, Vanessa Cannaday, Alexis Micale, Rodolfo Pena Chavez, Ryan Burdick, Celia Deckelman, Andrea Gilmore-Bykovskyi, and Michael Pulia for their assistance with this project.
REFERENCES
1. Haidich, AB, Ioannidis, JP. Patterns of patient enrollment in randomized controlled trials. Journal of clinical epidemiology. 2001;54(9). doi: 10.1016/s0895-4356(01)00353-5. PubMed PMID: 11520646.
2. Clinical Trials AReNA Clinical Trial Delays: America’s Patient Recruitment Dilemma 2012 [cited 2019 December 5]. Available from: https://wwwclinicaltrialsarenacom/analysis/featureclinical-trial-patient-recruitment/.
3. Census (U.S. Census Bureau). Hispanic Population to Reach 111 Million by 2060 2018 [cited 2021 July 30]. Available from: https://www.census.gov/library/visualizations/2018/comm/his-panic-projected-pop.html.
4. Carratala, S, Maxwell, C. Health Disparities by Race and Ethnicity2020. Available from: https://cdn.americanprogress.org/content/uploads/2020/05/06130714/HealthRace-factsheet.pdf?_ga=2.194956205.1806611089.1627327183-1860413079.1627327183.
5. SAMHS (Substance Abuse and Mental Health Services Administration). Top Health Issues for LGBT Populations Information & Resource Kit. Rockville, MD: Substance Abuse and Mental Health Services Administration; 2012.
6. FDA (U.S. Food and Drug Administration). Enhancing the diversity of clinical trial populations—eligibility criteria, enrollment practices, and trial designs guidance for industry: Center for Biologics Evaluation and Research Center for Drug Evaluation and Research; 2020 [cited 2021 July 30].
7. Duma, N, Aguilera, J Vera, Paludo, J, Haddox, CL, Velez, M Gonzalez, Wang, Y, Leventakos, K, Hubbard, JM, Mansfield, AS, Go, RS, Adjei, AA. Representation of minorities and women in oncology clinical trials: Review of the past 14 years. Journal of oncology practice. 2018;14(1). doi: 10.1200/JOP.2017.025288. PubMed PMID: 29099678.
8. Gong, IY, Tan, NS, Ali, SH, Lebovic, G, Mamdani, M, Goodman, SG, Ko, DT, Laupacis, A, Yan, AT. Temporal Trends of Women Enrollment in Major Cardiovascular Randomized Clinical Trials. The Canadian journal of cardiology. 2019;35(5). doi: 10.1016/j.cjca.2019.01.010. PubMed PMID: 31030866.
9. McCarthy-Keith, D, Nurudeen, S, Armstrong, A, Levens, E, Nieman, LK. Recruitment and retention of women for clinical leiomyoma trials. Contemporary clinical trials. 2010;31(1). doi: 10.1016/j.cct.2009.09.007. PubMed PMID: 19788933.
10. Vitale, C, Rosano, G, Fini, M. Are elderly and women under-represented in cardiovascular clinical trials? Implication for treatment. Wiener klinische Wochenschrift. 2016;128(Suppl 7). doi: 10.1007/s00508-016-1082-x. PubMed PMID: 27655475.
11. Frew, PM, Saint-Victor, DS, Isaacs, MB, Kim, S, Swamy, GK, Sheffield, JS, Edwards, KM, Villafana, T, Kamagate, O, Ault, K. Recruitment and retention of pregnant women into clinical research trials: an overview of challenges, facilitators, and best practices. Clinical infectious diseases : an official publication of the Infectious Diseases Society of America. 2014;59 Suppl 7(Suppl 7). doi: 10.1093/cid/ciu726. PubMed PMID: 25425718.
12. Rosende-Roca, M, Abdelnour, C, Esteban, E, Tartari, JP, Alarcon, E, Martínez-Atienza, J, González-Pérez, A, Sáez, ME, Lafuente, A, Buendía, M, Pancho, A, Aguilera, N, Ibarria, M, Diego, S, Jofresa, S, Hernández, I, López, R, Gurruchaga, MJ, Tárraga, L, Valero, S, Ruiz, A, Marquié, M, Boada, M. The role of sex and gender in the selection of Alzheimer patients for clinical trial pre-screening. Alzheimer’s research & therapy. 2021;13(1). doi: 10.1186/s13195-021-00833-4. PubMed PMID: 33952308.
13. Khan, MS, Shahid, I, Siddiqi, TJ, Khan, SU, Warraich, HJ, Greene, SJ, Butler, J, Michos, ED. Ten-year trends in enrollment of women and minorities in pivotal trials supporting recent US Food and Drug Administration approval of novel cardiometabolic drugs. Journal of the American heart association. 2020;9(11):e015594. Epub 20200519. doi: 10.1161/JAHA.119.015594. PubMed PMID: 32427023; PMCID: PMC7428976.
14. Census (U.S. Census Bureau). The American Community Survey 2005-2019. 2019.
15. Haidich, AB, Ioannidis, JP. Patterns of patient enrollment in randomized controlled trials. Journal of clinical epidemiology. 2001;54(9):877-83. doi: 10.1016/s0895-4356(01)00353-5. PubMed PMID: 11520646.
16. Tay, TR, Pham, J, Hew, M. Addressing the impact of ethnicity on asthma care. Current Opinion in Allergy and Clinical Immunology. 2020;20(3):274-81. doi: 10.1097/ACI.0000000000000609. PubMed PMID: 31850922.
17. Jamerson, K, DeQuattro, V. The impact of ethnicity on response to antihypertensive therapy. The American Journal of Medicine. 1996;101(3A):22S-32S. doi: 10.1016/s0002-9343(96)00265-3. PubMed PMID: 8876472.
18. Odierna, DH, Bero, LA. Systematic reviews reveal unrepresentative evidence for the development of drug formularies for poor and nonwhite populations. Journal of Clinical Epidemiology. 2009;62(12):1268-78. Epub 20090417. doi: 10.1016/j.jclinepi.2009.01.009. PubMed PMID: 19375890.
19. Glasgow, RE, Huebschmann, AG, Brownson, RC. Expanding the CONSORT Figure: Increasing Transparency in Reporting on External Validity. The American Journal of Medicine. 2018;55(3):422-30. Epub 20180720. doi: 10.1016/j.amepre.2018.04.044. PubMed PMID: 30033029.
20. Ix, J. H., Allison, M. A., Denenberg, J. O., Cushman, M., Criqui, M. H. Novel cardiovascular risk factors do not completely explain the higher prevalence of peripheral arterial disease among African Americans. The San Diego Population Study. Journal of the American College of Cardiology. 2008;51(24):2347-54. doi: 10.1016/j.jacc.2008.03.022. PubMed PMID: 18549921.
21. NIH (National Institutes of Health). NIH Policy and Guidelines on The Inclusion of Women and Minorities as Subjects in Clinical Research 2001 [cited 2021 July 30]. Available from: https://grants.nih.gov/policy/inclusion/women-and-minorities/guidelines.htm.
22. FDA (U.S. Food and Drug Administration). Establishing a list of qualifying pathogens under the Food and Drug Administration Safety and Innovation Act. Final rule. Federal register. 2014;79(108). PubMed PMID: 24908687.
23. Nazha, B, Mishra, M, Pentz, R, Owonikoko, TK. Enrollment of Racial Minorities in Clinical Trials: Old Problem Assumes New Urgency in the Age of Immunotherapy. American Society of Clinical Oncology Educational Book. American Society of Clinical Oncology Annual Meeting. 2019;39. doi: 10.1200/EDBK_100021. PubMed PMID: 31099618.
24. Moher, D, Hopewell, S, Schulz, KF, Montori, V, Gøtzsche, PC, Devereaux, PJ, Elbourne, D, Egger, M, Altman, DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. International journal of surgery (London, England). 2012;10(1). doi: 10.1016/j.ijsu.2011.10.001. PubMed PMID: 22036893.
25. CDC (U.S. Centers for Disease Control and Prevention). Leading Causes of Death 2020 [updated 2021-03-01T08:25:51Z]. Available from: https://www.cdc.gov/nchs/fastats/leadingcauses-of-death.htm.
26. Frey, WH. Six Maps That Reveal America’s Expanding Racial Diversity Washington, DC: Brookings Institution; 2019 [cited 2021 July 30]. Available from: https://www.brookings.edu/research/americas-racial-diversity-in-six-maps/.
27. Lewison, DM. Perhaps the most widely used regional classification system is one developed by the U.S. Census Bureau. Prentice Hall; 1997.
28. Hill, CV, Pérez-Stable, EJ, Anderson, NA, Bernard, MA. The National Institute on Aging Health Disparities Research Framework. Ethnicity & disease. 2015;25(3). doi: 10.18865/ed.25.3.245. PubMed PMID: 26675362.
29. Taylor, HA. Implementation of NIH inclusion guidelines: survey of NIH study section members. Clinical Trials. 2008;5(2):140-6. doi: 10.1177/1740774508089457. PubMed PMID: 18375652; PMCID: PMC2861770.
30. Riley, WT, Riddle, M, Lauer, M. NIH policies on experimental studies with humans. Nature Human Behaviour. 2018;2(2):103-6. Epub 20171122. doi: 10.1038/s41562-017-0265-4. PubMed PMID: 30662956; PMCID: PMC6338435.
31. Crenshaw, KW. On intersectionality: Essential writings: The New Press; 2017.
32. Ford, ME, Siminoff, LA, Pickelsimer, E, Mainous, AG, Smith, DW, Diaz, VA, Soderstrom, LH, Jefferson, MH, Tilley, BC. Unequal burden of disease, unequal participation in clinical trials: solutions from African American and Latino community members. Health Soc Work. 2013;38(1):29-38. doi: 10.1093/hsw/hlt001. PubMed PMID: 23539894; PMCID: PMC3943359.
33. Boyatzis, RE. Transforming qualitative information: Thematic analysis and code development. SAGE; 1998.
34. George, S, Duran, N, Norris, K. A systematic review of barriers and facilitators to minority research participation among African Americans, Latinos, Asian Americans, and Pacific Islanders. American Journal of Public Health. 2014;104(2):e16-31. Epub 20131212. doi: 10.2105/AJPH.2013.301706. PubMed PMID: 24328648; PMCID: PMC3935672.
35. Gilmore-Bykovskyi, AL, Jin, Y, Gleason, C, Flowers-Benton, S, Block, LM, Dilworth-Anderson, P, Barnes, LL, Shah, M, Zuelsdorff, M. Recruitment and retention of underrepresented populations in Alzheimer’s disease research: A systematic review. Alzheimer’s and dementia (New York, N. Y.). 2019;5:751-70. Epub 20191119. doi: 10.1016/j.trci.2019.09.018. PubMed PMID: 31921966; PMCID: PMC6944728.
36. Ejiogu, N, Norbeck, JH, Mason, MA, Cromwell, BC, Zonderman, AB, Evans, MK. Recruitment and retention strategies for minority or poor clinical research participants: lessons from the Healthy Aging in Neighborhoods of Diversity across the Life Span study. Gerontologist. 2011;51 Suppl 1:S33-45. doi: 10.1093/geront/gnr027. PubMed PMID: 21565817; PMCID: PMC3092978.
37. Graaf, R van der, Zande, ISE van der, Ruijter, HM den, Oudijk, MA, Delden, JJM van, Rengerink, K Oude, Groenwold, RHH. Fair inclusion of pregnant women in clinical trials: an integrated scientific and ethical approach. Trials. 2018;19(1):78. Epub 20180129. doi: 10.1186/s13063-017-2402-9. PubMed PMID: 29378652; PMCID: PMC5789693.
38. Vitale, C, Fini, M, Spoletini, I, Lainscak, M, Seferovic, P, Rosano, GM. Under-representation of elderly and women in clinical trials. International Journal of Cardiology. 2017;232:216-21. Epub 20170110. doi: 10.1016/j.ijcard.2017.01.018. PubMed PMID: 28111054.
39. Zunzunegui, MV, Alvarado, BE, Guerra, R, Gómez, JF, Ylli, A, Guralnik, JM, Group, Imias Research. The mobility gap between older men and women: the embodiment of gender. Archives of Gerontology and Geriatrics. 2015;61(2):140-8. Epub 20150617. doi: 10.1016/j.arch-ger.2015.06.005. PubMed PMID: 26113021.
TABLE C-2a Starting with Intention and Agency to Achieve Representativeness
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Importance of intentional approach | “And so if you want to be inclusive, you need to then think about how many from that population you want to enroll and begin to work towards that goal. That’s number one. So I think that goes into the framework of intentionality, right? We need to be intentional. We want to do this and want to be intentional about doing it. . . . I believe very strongly that many times we want to do this as an afterthought. So we didn’t go into the study saying that we want to enroll this many African American, Latino people, but [should] make that as part of the initial goal.” | Male, Study Investigator |
| “But the number one principle I do think is intentionality. You have to want to do it because expediency will kick in that you need to close the study in one year and you want to get those patients enrolled. But I do think if you start to plan from the beginning to have an inclusive group, that’s important.” | Male, Study Investigator | |
| Motivation to pursue representative sample | “I mean, NIH, when you fill out the RPPR and you fill out your little diversity table, it’s always hard to put in those zeros.” | Female, Study Investigator |
| “I think the mandate… was so hard. But when we had to get underrepresented groups three out of every ten, we did it and otherwise we don’t . . . it just feels really good, working hard to get underrepresented groups.” | Female, Study Investigator | |
| “[I] try to make the sample representative of where I am at the time, doing the work and where I am, there’s a high percentage of LatinX. We need them in the work… to understand what’s going on with them.” | Female, Study Investigator | |
| “So women are somewhat easier to reach in that way than men who are traditionally a little bit harder but not with this ethnic group. It was like we won’t know anything about women’s health if we don’t have more women in the study, meaning equal numbers of men and women. . . . So it’s like you got to get invested enough.” | Female, Study Investigator | |
| “It was motivated by having a representative sample and understanding how so many individuals are not represented, I just think that’s really important. And I just think as the scientist, well, we have to do that. It’s part of who we are and it’s why, as I said, it’s hard but it’s so important.” | Female, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Budget planning with recruitment in mind | “So it’s really budgeting for time and effort of people who are not typically thought of in grants. But those are like most of my grants, like all of the funding goes externally like subcontracts to different partners.” | Female, Study Investigator |
| “When he was writing, he’s like ‘Send me the budget for the recruitment, for the outreach.’ Amazing, I mean I put in there, I can tell you my budget was transportation to events and food for events, and he was like ‘Great, thank you.’ No questions asked, submit it.” | Female, Study Coordinator | |
| Proactive study design | “We said we were going to design the study to do subgroup analyses to look at if the intervention was effective and in historically underrepresented groups. And that was important for us.” | Female, Study Investigator |
| “And so we designed with and for people at risk for the worse outcomes . . . And so the way you design for that, a lot of people say, well, it’s only among Black women that we’re testing this intervention and that’s great. I don’t have a problem with that at all. But the way I have done it is to say we’re going to ensure that there are enough Black women in the sample that we can do a subgroup test to make sure that the effect size that they see is on par with the effect size that non-Black women see to try to understand if our interventions are exacerbating disparities.” | Female, Study Investigator | |
| “The other thing that I think is really important is designing and piloting study materials, whether it’s the study intervention or the consent form or the recruitment process, for underrepresented groups, and then they will play fine for the other groups.” | Female, Study Investigator | |
| “So the whole grant was written around community engagement, mixed methods, both quantitative and qualitative, really understanding segmented assimilation and new ways of thinking about immigrant health and how to really quantify where the gaps are, what the barriers are, and how we can improve health.” | Female, Study Investigator | |
| “I think I would look pretty closely at the inclusion/exclusion criteria. I guess of the top of my head common inclusion/exclusion that would apply or would be different based on your gender or race would be a lot of rules around pregnancy and having to use certain methods of birth control. . . . I think that can deter some patients. And, on the flip side, if you’re a male enrolling into a clinical trial, you don’t have to have any sort of birth control.” | Female, Study Coordinator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Combining recruitment approaches to optimize enrollment of diverse groups | “We were purposeful in our recruitment strategies, we used EHR data to prioritize folks that were either from a minority racial or ethnic group or had an indicator for insurance that they may be uninsured or underinsured. And we prioritized recruitment of those groups.” | Female, Study Investigator |
| “We tried other methods, like using electronic health records using self-identified ethnicity that is collected by health systems data to reach out to this particular demographic that we really wanted to recruit. And those letters were just terrible in terms of getting any yield . . . it’s horrendous because it’s just not meaningful.” | Female, Study Investigator | |
| “So basically how recruitment happened is based on individuals who were Medicare eligible, who lives in these neighborhoods . . . and so they were selected based on this random recruitment effort to find people in each catchment area based on the census in the neighborhood. And that was the goal to get this representativeness. So the beginning of this study, there is actually a lot more White people . . . But then that started to switch as the composition of the neighborhood switched.” | Female, Study Coordinator | |
| “So I think in some sense the clinics did that for us, like if this is a clinic that largely serves the homeless population downtown and we partner with that clinic, we don’t need to do a lot of extra stuff to reach those patients. So making sure those clinics were priorities for us and we did adjust a lot of our approach in working with the clinic.” | Female, Study Investigator | |
| “So we partnered with community organizations, faith based groups, leaders in those vulnerable communities that have traditionally been left out of research and just did a lot of outreach activities, both in person and in terms of health fairs and other community venues.” | Female, Study Investigator | |
| “So we’re using probability sampling in terms of knowing the demographics from the last five years of the American community survey in these geographic locations, knowing the sex and age distribution and education distribution in these communities…So we’re not doing all community engaged methods. We have to put some kind of boundaries around that so the validity of our data is still there and will not be questioned.” | Female, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Designing marketing strategies to reach underrepresented groups | “So we used a lot of different community activities as well as a radio and ethnic radio station ads and interview with radio stations that are to reach these populations, some TV spots as well. A lot of faith based organizations helped us with providing us a little bit of advertising in a way but letting people know about the study.” | Female, Study Investigator |
| “The venues that we were allowed to use marketing in front of ethnic markets and restaurants and in other community settings. And a lot of that was actually quite successful in getting the word out and getting people interested.” | Female, Study Investigator | |
| Reciprocity with Study participants and community | “I also think it’s important to share how this study is going. We put together a tipsheet because when you’re doing this study it’s like a black hole. So I’m in this trial and I have no idea what’s happening. So I think getting an idea of how this study is going, regular information shared with people in the trial about the progress of the study is important.” | Male, Study Investigator |
| “We actually suggested providing other ancillary services, educational materials. So you’re in a study, you know that these other health issues are related to X, Y, Z, so I think a lot of people select into those preferences and provides useful information for them to engage them in science without compromising their study goals.” | Male, Study Investigator | |
| “One of the key challenges that we face is downstream care. Sometimes our goals are undermined by word of mouth, so someone gets a positive test and they can’t do anything else with it and they say ‘Oh yeah, I couldn’t get that colonoscopy, they don’t care about me’ so I think that sort of downstream continuum of care, a cascade of care that is needed for the clinical trial needs to be provided. And to me that is best practice.” | Male, Study Investigator | |
| “I think we need to develop a process by which we have relationships with people. It is through that ongoing feedback to the community or participants over time.” | Male, Study Investigator | |
| “Community engagement is important for the downstream care after clinical trials—it is critically important and should not be ignored because I do think that, when we fail to do that, it is in the care process. A clinical trial undermines efforts to build trust and growth in science.” | Male, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “And so we do try to give back. We don’t just recruit, we always try to give back to the community. I think that’s really important if you want to have a relationship with the community, you don’t just take. Whatever that community is, we try to teach you, we go to health fairs, we try to give something back.” | Female, Study Investigator | |
| “Tokens of appreciation, we were constantly giving feedback to the clinic about how many people we were recruiting, and then we gave feedback on our results and things we were finding and publishing.” | Female, Study Investigator | |
| “That’s who we’re recruiting are staff members and people interacting with participants from the community. So that’s another really big way that we’re invested. It’s that reciprocal relationship.” | Female, Study Coordinator |
TABLE C-2b Establishing a Foundation of Trust with Participants and Community
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Importance of building trust with community and prior issues of distrust | “And the reason that I don’t think it worked well [sending letters] is that there’s no trust. There’s a lot of mistrust in getting a letter from a random person, even though it has a university letterhead on it. You don’t know anything about the person or the research or if you’re undocumented or don’t speak the language, if you’ve never been exposed to research, what the point is for research, there’s many layers of trust that cannot be broached with an invitational letter and brochure.” | Female, Study Investigator |
| “I trust [her] but do I trust the system? Do I trust the hospital? . . . I have some case studies and that actually comes from you showing them what you are doing with the data and what is being not only done but not done. Are you giving feedback individually? Are you giving feedback as a whole? It’s the community, how is it being used to further policies?” | Female, Study Coordinator | |
| “So I would give a talk and try to sit with people. And we had food afterwards usually, so we could all just sit and talk casually. But they’re telling me, over and over again, there’s just a lot of distrust in the medical community and I get it, I understand why.” | Female, Study Investigator | |
| “This one community that I’m thinking about has been a little historically suspicious because of bad experiences they’ve endured of medical research and perhaps academic medical research and so sending out a single notice is not going to be sufficient in order to have meaningful recruitment of these groups. It’s really going to start with building relationships of trust and then later availing those groups of opportunities.” | Male, Study Investigator | |
| “So I think it is the relationship and trust to me is the key. Once trust is established, people will do things that I believe are coming from you and you better keep that promise.” | Male, Study Investigator | |
| Dedication to true engagement with community regardless of study enrollment | “And so I think that’s a way to cause [distrust], you have these studies where people are meeting that requirement and they’re not treating the community very respectfully often or they just don’t know how. And it’s maybe, it’s usually unintentional but it’s a consequence.” | Female, Study Coordinator |
| “I don’t know if this is tested anywhere is this idea of helicopter history. So we’re coming to do our research, we’re done, and then you’re never seen again. Then the next time a research study is done, you can see it and it’s done and you never see it again. I think we need to develop a process by which we have relationships with people.” | Male, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “It’s really hard work in terms of it takes a lot of energy and time investment. You have to really stay connected with the community or you cannot just go in and out. I mean, that’s a commitment, right?” | Female, Study Investigator | |
| “We participate in community events that the clinics did. So if they did a diabetes day or a health fair, we were there with our table and we didn’t really recruit people from that but it was just sort of part of being in the community we helped with. . . . Resources that the clinics have put into place, we were able to participate in, and that helped us as well.” | Female, Study Investigator | |
| “So the key is as a study team we need to also be doing community outreach and service to actual caregivers and [patients] to make that connection.” | Female, Study Investigator | |
| “Having staff available to go to those satellite sites as needed I think is a good strategy to maybe improve recruitment of specific populations.” | Female, Study Coordinator | |
| “We were trying to be visible and physically present as often as possible within the clinic and also really so that the staff were very familiar with us, they saw us. So we wanted to be present and we didn’t want to disappear once we start the study or the data collection. I think that’s really important, wherever you could be, whenever you could be physical, and let them know that you are still here.” | Female, Study Investigator | |
| “I think and it may not be that the people in the clinic are not necessarily are participants but I think I’m talking about it’s more a long-term strategy that is not just for your study. But I do want to maintain this relationship and I want to continue to recruit diverse patients and families in our study that I have to be present. I think the study team has to be present in many different ways.” | Female, Study Investigator | |
| Bring research to the community | “So really embedding our staff in the communities, completely doing all of the outreach and all of the clinical exams in the community setting, making it as less clinical as possible, making every attempt to reach people where they live, work, pray, and play.” | Female, Study Investigator |
| “Right now, the series that we’re going to start, we do them in the area that we recruit from. So I also like to detach that from the institution because we have institutional events maybe here because it’s easy. But I’d rather find places for me to host events that are outside of the institution because those are more safe. I think those are safe spaces.” | Female, Study Coordinator | |
| “We have to provide the best care and the best trials where people live and minimize the disruption that they face. And only until we do would we see sustained improvement in access to different clinical trials.” | Male, Study Coordinator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Developing lasting relationships with study participants | “[We are] trying to be much more centered on that person and their individual needs.” | Female, Study Investigator |
| “And the women are probably just overwhelmed with other work that they have to do in the home with childcare, with employment, with finances. So making it as easy as possible for women. We have had our coordinators actually go to the woman’s house, picked them up and come with them either on public transportation or a shared ride or whatever, just kind of building more of a relationship. So they feel like they know this person. They feel safe with this person.” | Female, Study Investigator | |
| “We send them birthday cards and holiday cards. We do all the obvious things that many, many cohorts do. But we try to always put more of a personal touch. So the coordinator that they know signs the cards with their name so that they know the signature of the coordinators. They know that she really did that. On Monday, when they call they want to talk to the coordinator. They know her. They trust her.” | Female, Study Investigator | |
| “Yeah, incentives, we paid them. And then establishing that personal connection with them because they were letting us into their homes with these video recorders and things. So I would talk to them on the phone each week. And sometimes these conversations would last 15 minutes, sometimes they would last 2 hours. Where we would just chat about ‘How’s it going?’ I really tried to get to know them on a personal level.” | Female, Study Investigator | |
| “Just sitting in the church cafeteria or wherever we eat and just sitting and breaking bread with people. That’s just a traditional time for people to maybe gain a little bit of trust. And somebody would ask me a question that they didn’t want to ask in front of everybody. So I’d go at this table with three people and we’d talk and it was just so much more intimate. And I think they felt listened to and asking questions that they didn’t want to ask in front of everybody, I think. And by the end, I knew their names. They knew my name and I did get a lot of calls.” | Female, Study Investigator | |
| “The project coordinator and the recruitment coordinator were two different people. And it actually helps . . . because then I would become the person who can be neutral and because I’m not calling to recruit you. I’m actually telling you about the study. And I’m going to listen to you. And you’re going to decide based on what I’m saying whether you want to participate or not . . . because you’re now somebody who is more likely to be trusted because you’re just there to explain and you can maintain a relationship.” | Female, Study Coordinator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “What I did find is that sometimes it’s too much. You have to make yourself available but also like, just lean back, because I do find that unfortunately this population of elders, there’s a lot of people trying to get them. So I have found [with] . . . this community, less is more. We don’t send cards, I know a lot of people do but the birthday carts that to them, believe it or not, . . . feels a little like, oh, it’s just random. Like everybody, like another thing…doesn’t cut it. What we do is . . . we find anything that’s awesome about them. They could be a painter. They could play guitar. They could be clay sculptors. Whatever it is, we find out. I have a section where I write that little tidbit about them and we’ll make sure when we see them again or talk to them or I call them and I’ll say something. Something’s interesting, there’s something personal about it. Personalized, no cookie cutter response that’s done to everyone.” | Female, Study Coordinator | |
| “That’s why we don’t force, everything’s voluntary. . . . They can withdraw at any time. So we make sure that they instill that in anything that we do, no forcing answering questions. Their well-being is first, the study goes second. And then it just always comes first with us because we just put them first. So they put the study first.” | Female, Study Coordinator | |
| “I mean we do provide incentives for follow-up interviews. I think that certainly helps. But I think once you have established a relationship through your first baseline visit.” | Female, Study Investigator | |
| Maintaining a favorable study reputation with the community | “And so that’s why we’re hoping that the use of these community outreach events that we did and the people who care about us and see a face to a name and you hear about it from other trusted leaders and champions in the community and that would give us an air of legitimacy in reaching out.” | Female, Study Investigator |
| “It starts conversations with friends, conversations when we’re not present, which is in the kitchens and the dining rooms. So then we find a way for our findings and what we do to become part of their everyday conversations. So when we started doing that, then we started getting people who not only were aware of studies but we started getting people who knew of other people who were in research studies but had no idea that’s what it was.” | Female, Study Coordinator | |
| “But how do I make sure that the same institution that they work for also doesn’t screw me over for your study. So that’s just to tell you how the instances, even within the research community, impacts other studies indirectly, maybe not for the long run but it does for now.” | Female, Study Coordinator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Developing lasting equitable relationships with community partners | “If we activated communities enough to buy into the concept of research and they are advocating for research, then all of this will kind of change, right?” | Male, Study Investigator |
| “We try to go to the community like beauty shops, barber shops. Those are really good we found to post things because it’s pretty small. And we can sometimes, if they want to, they can post something right up on the mirror at the beauty shop. So we do a lot of churches and go to church events. And recruit there.” | Female, Study Investigator | |
| “When they need something, like they’re having a health fair and has nothing to do with the research we’re doing, they need to reach out to me. And my academic institution has plenty of people who can volunteer at their health fair and can be there and can partner and do things if they need help with. So it’s not only about us and the research we bring to them but it’s about they want to get this health care done. Got that kind of relationship.” | Female, Study Investigator | |
| “I’m also going to organizations that have budgets or . . . do these events within the community. And I’ll just circle in . . . finding those organization that are not necessarily mirroring what you’re doing but finding connections because everything has a connection, right?. . . Although there will be limited funding and strategies but I think there’s always a connection with what you do within any community. There’s a way for you to connect and then bring your message.” | Female, Study Coordinator | |
| “The other resources were there was a consortium . . . here that all of the . . . clinics would meet once a month. And so that gave us opportunities to be present, understand initiatives that are going on, present the study, present findings from the study. . . . If it wasn’t for that infrastructure, it would have been much harder to sort of roll out a new . . . clinic to build new relationships.” | Female, Study Investigator | |
| “It is a very slow process. So we started with one clinic that we had done a lot of formative work and we partnered with them for some time.” | Female, Study Investigator | |
| “It’s a two-way street. I don’t just go to them when I have a study. And I can’t expect them to be open and ready to help me with every study and I’m not truly there for them. So it’s not only me, but it’s like having this kind of relationship that is enduring and takes time to build. And it’s not a trivial commitment. It’s a real long-term commitment. And so we built these relationships with our community partners for now more than a decade and have been and those relationships come with both give and take of information.“ | Female, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “Our academic partners have been working with these community organizations and actually have community health workers who worked with them on other projects. So it’s easy to take them from one project to another until they have this track record. And it works really nicely for them because they have built in trust already.” | Female, Study Investigator | |
| “And I have maintained relationships with people for years without them actually joining the study. And one of those people, she is a community activist advocate. And people thought that this person has been studied for years. And I’m like, no, she just joined two years ago. They’re like, ‘What?’ I’m like ‘Yeah because it just took that long.’ It may have to for whatever reason but it took that long for her to say yes.” | Female, Study Coordinator | |
| “We have mature community engagement programs . . . I choose to hold a community partnership. Recruiting within that partnership is easy because you have trust build over many years.” | Male, Study Investigator | |
| “I think that is the goal to get to full equity with the community partner, writing grants and getting the money and sharing everything from the ground up to the study. I think we’re still unequal with academic partners doing grant writing, getting funding, and working with community partners and giving them funding from the grant. So I think there’s still this hierarchy, unfortunately, we’re trying to break those down. We’re trying to get to as much parity as possible and that’s just going to take time and it’s going to take investment.” | Female, Study Investigator |
TABLE C-2c Anticipating and Removing Barriers to Study Participation
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Increasing physical access | “In my view, it involves some remote access to trials . . . so that not everything needs to be face-to-face. If you put transportation between you and a trial, it falls down [in the] engagement of patients” | Male, Study Investigator |
| “Paying for shared rides, Uber or Lyft to make it easy and convenient for people to come instead of having to pay for public transportation.” | Female, Study Investigator | |
| “So that’s one of the ways, a lot of people as they get older and more frail, they don’t want to travel into the clinic appointments and do all these tests. So we go into their home doing what we can in the home and getting some measures rather than all of the measures.” | Female, Study Investigator | |
| “Their coming into the clinic like three days a week to get . . . lab samples and that is a lot of driving, that’s a lot of time to . . . have to take off work, or have to take away from family. And not all patients are privileged enough to be able to take time off and come to the center every day.” | Female, Study Coordinator | |
| “And so travel to centers . . . it’s a big barrier . . . so assisting in transportation centers is important if that’s required. Remote monitoring is important because I think why bring people back just to check that they’re ok when it can be done remotely.” | Male, Study Investigator | |
| Increasing linguistic access | “If you want to get folks involved in your research and they happen to be part of that community, boy, you better have people on your team that have language skills related to that and certainly you better have your documents professionally translated into those languages.” | Male, Study Investigator |
| “So there were two language translations that were required in order to do our study . . . if you don’t have those materials prepared and you don’t anticipate the need to have those materials a priori, it sort of becomes a self-fulfilling prophecy in that you’re not going to accrue well or at all in those populations.” | Male, Study Investigator | |
| Ensuring study team accessibility | “We provided a helpline where people could call and just leave a voicemail. And there was no threat that anybody was going to answer that they would have to speak to. So they would just call the voicemail and say ‘Hey, my cell phone number changes. Here it is.’ or ‘Hey, I’m moving. Here’s my new address.’ And we got a lot of sort of contact change information from that helpline.” | Female, Study Investigator |
| Recognizing within-group heterogeneity and tailoring approaches | “I think it would be much more important if you did include the actual ethnic groups that you’re asking about. You can’t generalize what I said about all the studies we’ve done to other groups. . . . None of these lessons learned naturally translate to other groups.” | Female, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “So it’s about adjusting and making changes based on the observations of people who are having one on one contact with the cohort within the community.” | Female, Study Coordinator | |
| “So again, I can tell you how it works here. It doesn’t necessarily mean that it’s going to transfer to another community… Maybe that is a strategy, to look for diversity even within the groups and see what works for each group and take the time to do that and do exploratory findings.” | Female, Study Coordinator | |
| “There was no cultural tailoring at all. There was a ton of individualized tailoring. The intervention itself is highly individually tailored. And so we just developed personalized approaches to everyone. And, in doing that, we didn’t have to put people into categories to try to tailor to them.” | Female, Study Investigator | |
| Adapting study materials and consent process | “You have to spell it out. . . . And how do you explain this to persons who don’t have a background on just simple science? Let’s say because these people have low literacy or didn’t go to school for many years. Well it just takes time. . . . I tend to have conversations with them . . . it may make sense to me but I may not be explaining it well and what does this mean to you?’ And then with that feedback, give some suggestions back to the PIs and then we make those changes to the consent form.” | Female, Study Coordinator |
| “Consent process was long. It was actually very well written but I can imagine people would say, well, I’m not understanding this concept, even if you translated it into another language people could not read. And then they will not allow you to read it to people. So you have a process that becomes very difficult. So I think the consent process is harder than it needs to be. And being accessible, in my view, is some of the ways that we’ve sought to [overcome] that.” | Male, Study Investigator | |
| “And so in our part of the country, persons don’t always have the best education . . . some have like third grade education. And so we just try to use the simplest words . . . we try to use . . . very few direct words. And we spend hours, we revise and revise to get it to that point.” | Female, Study Investigator | |
| “Our materials do have diverse people on them, all kinds of LatinX and Asian and African American because I think that’s important. I mean, it’s so important. What you see is that ‘I don’t see me there’.” | Female, Study Investigator | |
| “If a patient is deaf or blind, just having those resources available in our center when needed so we are not limiting our recruitment of ‘disrepresented’ or misrepresented patients in any way.” | Female, Study Coordinator |
TABLE C-2d Adopting a Flexible Approach to Recruitment and Data Collection
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Adapting recruitment approach to address low enrollment | “We applied an agile process, we constantly looked at data, we were constantly saying “Is it possible to get the sample we want from this clinic? Is it our processes that are the problem? Is it the patient pool that’s the problem?’ And through doing that we engaged with the clinicians there . . . saying help us crack this nut. And they were like ‘You know, you’re not going to get it there. You need to be looking here’.” | Female, Study Investigator |
| “Now in subsequent waves of recruitment, we’ve used a lot more community engagement.” | Female, Study Investigator | |
| Adapting study protocols as needed | “I guess one of the models that I do a little bit differently than some of my colleagues. . . . I meet with the entire team once a week and I also meet one on one with people. . . . I got to hear them saying ‘This part of the protocol is making people sort of turn off.’ And so I was like, ok well, then let’s revise that part of the protocol.” | Female, Study Investigator |
| “So one clinic didn’t have space for us to meet one on one with people. And so we would try to figure out, is there a safe way? Like, for instance, is it safe for a research assistant to meet at the mission at the homeless shelter next to the clinic? Is that a good idea or not? So we did a lot of tailoring to our processes based on the clinic requirements and restrictions.” | Female, Study Investigator | |
| “We allowed women to bring their spouse to the visit and then allowed the spouse to have a limited exam. And we didn’t use the data for the spouse. So we got her approval to have a limited exam visit for the spouse to be included because the spouse would bring the women to the exam and the woman is a participant. It helped us retain women in the study.” | Female, Study Investigator | |
| Flexibility in data collection procedures | “We have to meet with them weekends and whenever we can. Yeah, that’s one of the criteria for being able to work on my project, just be available . . . So almost everyone has some degree of expectation of flex schedule.” | Female, Study Investigator |
| “So we had to be very flexible in how we collect the data. We ultimately ended up giving people multiple data collection options, so we tried to enroll everyone and do baseline data collection in person for folks, for literacy reasons, for understanding comprehension and for trust building. And then after that, they could meet us in person or in the clinic. They could meet us in person in our research offices. They could do it online via REDCap. They could do it via phone with a research assistant. They could be mailed a paper survey. And similarly they could go in for a . . . test at a clinic or they could do a mailing kit.” | Female, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “I get pushback from people on other studies that I’m not a PI when I describe that we use all these different methods for data collection. People react to that and I disagree. I think data are far better than no data because, when we have no data, it’s biased in systematic ways. When you have data from multiple sources, it’s just a little bit more variable. And all that does is make it more difficult for you to detect, well potentially we don’t know for sure, an effect but it doesn’t systematically bias the effect that you’re going to find. And so I just think that it’s important to offer as many forms of data collection as you can to increase retention, even if you have some measurement error.” | Female, Study Investigator |
TABLE C-2e Building a Robust Network by Identifying All Relevant Stakeholders
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Caregiver and family involvement in study helps participation | “Having a family member . . . or someone who helps them with their day-to day tasks, that was extremely important for patients, and, perhaps, if they didn’t have that in their life, it would have been difficult for them to enroll and complete this study.” | Female, Study Coordinator |
| “A lot of times patients rely on family member, or friends, or other people in their lives to get them to appointments for this study.” | Female, Study Coordinator | |
| Participants as partners in research | “And also revising things not just on our feedback but their feedback. So, for example, we had administered a discrimination index questionnaire. So one of the causes or reasons for discrimination did not include gender. I think it was gender orientation. It was one participant who was like ‘You’re not including this.’ So, we gave feedback to the PI who had, she’s the one who had put the question in. And Dr. X had a very long conversation with this participant about how this was not capturing a reason why it was discrimination and, guess what, we went and revised that just because one participant said it. So it’s so important that they’re listened to, that questions are being asked, but they could be revised because it’s listening to their experience.” | Female, Study Coordinator |
| “I also think X does a really good job of, when they’re in the study, making them feel like they’re part of the study . . . like with the feedback and really taking it in. And they really feel like, ok, you’re not just here like with your ivory hat on and telling me what to do. I actually feel like I’m a part of this study and I’ve been here for twenty six years doing this, like I feel like this is my family.” | Female, Study Coordinator | |
| “So older adults, the participants, were definitely the key primary stakeholders.” | Female, Study Investigator | |
| “But I think that actually giving people the opportunity to give their feedback, turning them into sort of active participants in their own intervention of things, I think there’s a lot of power to that.” | Male, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Staff as partners in recruitment | “And so one of the things that really helps is that research assistants who are on the ground going into clinics got to hear me think through scientific decisions and say, ‘I don’t think that’s going to work in this clinic. Here’s why.’ Or I got to hear them saying, ‘this part of the protocol is making people sort of turn off.’ And so then let’s revise that part of the protocol. So it was way more of a free flow of information from the boots on the ground people to the decision maker for study design. And through that process they understand a lot of why we were doing things a certain way. And I understood when that way wasn’t working and could make changes to it.” | Female, Study Investigator |
| “You have a team where people’s, my, input is heard, which is . . . not common, right? Just because I don’t have a PhD background but I am well-versed in what I do in recruitment.” | Female, Study Coordinator | |
| “So again, it’s about adjusting and making changes based on the observations of people who are having one on one contact with the cohort within the community.” | Female, Study Coordinator | |
| Community members inform research procedures | “Get the input of those who are actually working within the communities. . . I think you will come up with a lot of different ways how . . . to diversity their cohort.” | Female, Study Coordinator |
| “So we have community advisory boards that are built very early in the process and each site has a different community advisory board because the issues that come up with each geographic location are very different and the communities to serve are very different. . . . We try to get a good representation of age and gender and different types of work and the experience in the community.” | Female, Study Investigator | |
| “You would go to the community . . . and say ‘I have an idea for research. I’d like your opinion on what the community might feel about this. Am I trying to get too many people? What would I need to establish a relationship? How can I help you to help me hire out of the community so that they can have people that are easily accessible to ask questions?” | Female, Study Coordinator |
TABLE C-2f Navigating Scientific, Professional Peer, and Social Expectations
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Inadequate understanding of recruitment and retention challenges among proposal reviewers and funders | “A barrier to that is the . . . misinformed notion that you have to be powered for an interaction term and that’s a real problem in review because it’s not actually what you want to know. You don’t want to know if the intervention was significantly differently effective. You want to know, was it effective in minority groups? Was it effective in low SES groups? And so I think getting that communication to reviewers is going to be important for this work moving forward.” | Female, Study Investigator |
| “I think heavier weight in review. Instructions for reviewers that heavier weight should be given to plans for these types of recruitment efforts, that things like measurement bias should be downweighted and it seems that there’s a real incongruence where the NIH is saying disparities work, disparities work, and then you put it in and reviewers don’t acknowledge the disparities aspect. They are fixated on errors in your approach or concerns about your theoretical model and it does seem that there is an incongruence in the way that the funding source of NIH wants to value efforts to recruit and retain these folks and the way it’s reviewed. So that is an issue.” | Female, Study Investigator | |
| “The funding agency, if anything, it’s been a barrier because of the reviewers we’ve gotten. They don’t understand the significance and we have to basically turn some results to have them. But we’ve had a lot of biased reviews. Each report that we’ve written that has been funded has taken multiple, multiple tries. . . . So it’s not very easy. We’re pushing, it’s an uphill battle every single time.” | Female, Study Investigator | |
| “They have to be mindful of it. They have to have representative review panels. Panels cannot be a single demographic, mostly white male groups. You have to have representation of the communities that are represented in the US on your panels. You have to educate people about the need for this.” | Female, Study Investigator | |
| “And that means that the panels that review the research have to be educated and representative and cannot be biased the way they currently are.” | Female, Study Investigator | |
| “I do think that more people from the NIH need to come to low income areas, urban areas, and do home visits for like, spend a day visiting families . . . because when you start visiting families and start seeing what’s really the issue, it’s hard to ignore it.” | Female, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Shift from historical perspective on clinical trial recruitment to focus on diversity | “I think if I compare my thought process X years ago when we started the study and now, my single thing I would point to that’s different is my awareness of underrepresented groups has been much heightened. And I don’t think that I have to go through all the reasons why that is because I think you know what those reasons are anybody who’s sentient and keeping track of the current events of the day would have some idea why that would be. And so, because my awareness has been increased, I don’t think that’s peculiar to me being a scientist as much as it is just being a member of the human race in the cities in the US in the current times.” | Male, Study Investigator |
| “I think it’s more of like the scientific rigor about recruitment, but I think this cohort in itself is unique because it was diverse before it had to be diverse. So now you have the whole NIH diversity initiatives and I can tell you that if someone is doing it for those reasons, it is going to show because it’s already showing in many other studies.” | Female, Study Coordinator | |
| “They are specifically asking about how many patients of a specific ethnicity or race you think you could recruit at your site. So a few years ago they would never get as specific with that. They would just ask total patients that you think you could recruit. But now they are specifically focusing on improving clinical trial diversity.” | Female, Study Coordinator |
TABLE C-2g Optimizing Study Team to Ensure Alignment with Research Goals
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| PI investment in supporting and training study team and leading by example | “I guess one of the models that I do a little bit differently than some of my colleagues is that I imagine a lot of people that run trials as PIs meet with their coordinator and their coordinator meets with the team. I don’t work like that. I meet with the entire team once a week and I also meet one on one with people. But the coordinator doesn’t like to filter the information up to me or down through her.” | Female, Study Investigator |
| “I do vividly remember training them in how did you inform consent in a conversational way and like looking the person in the eye and not having to read every word. And how to do teach back in a very casual way. . . . And so that process was important as well, especially when they’re not in one research office, they’re out in the community.” | Female, Study Investigator | |
| “We keep track of people are doing that, we have accountability checks. So if somebody hasn’t been following up with their folds for retention, they sort of appear on this slide in front of a team and it’s like ‘Do better’ and we’ll move on with our lives.” | Female, Study Investigator | |
| “I do like staff building events. Like I have people over at my house. We do a lot of . . . family building events like gifts, but not because I want to just retain them, because I really do care for them as human beings. They’re part of my work family. And so we’re on texts together and we know each other now. And it’s been a long-term relationship that we built in years, trying to drop out all of the formality from day one and to make this like a group effort. And so we’re in this together.” | Female, Study Investigator | |
| “I felt the obligation that I needed to lead by example. So, to be honest with you, that maybe that was a prideful reason for why I did what I did. But that was the motivating factor. Day after day, week after week, year after year, the study was, I’ve got to set the example . . . I mean, if you’re going to be a leader, you need to be a leader in all aspects of things.” | Male, Study Investigator | |
| “So sometimes trying to create that kind of environment on the team where that’s a topic of discussion. It’s prioritized as something that’s important to us. It’s kind of like that atmosphere type of thing.” | Female, Study Investigator | |
| Expressing trust and appreciation of staff | “I also do a lot of work with my team to teach them that my priority is them and then the participant and then the data.” | Female, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “So having the same staff at our site, we’ve had the same staff for 11 years now and are so thankful and grateful. And we’ve done everything to retain the staff . . . because they’re the face of the study.” | Female, Study Investigator | |
| “Well you had mentioned salary in the context of health care coordinators. I’m not sure my challenges with the turnover of the study’s personnel were related to salary as much as it was related to embracing them, making them feel like they were part of the team, help recognizing their important contributions.” | Male, Study Investigator | |
| “Like all the people of color on my team are making the least amount of money and all the white people were making the most amount of money. And so then it’s all those things I realized over the years that we just don’t place the same value on, like being an amazing recruiter and an amazing person for retention.” | Female, Study Investigator | |
| “But I feel like the staff are the experts when it comes to the patients. I mean, I really do rely on the staff. And there are also, again all of their staff are awesome and great at explaining things.” | Female, Study Investigator | |
| Consistent and experienced staff | “We try to have the same person reach out to them to collect data. . . . It helps to have a consistent person, like, ‘Gee, you know, XXX just called me again. I know XXX. I recognize XXX. That is nice. I like XXX. And I’m more likely to pay attention.’ So I think a consistent individual to follow through on multiple contacts without . . . making yourself just a nuisance. And I think those are important.” | Female, Study Investigator |
| “Although it’s hard to achieve, it’s best not to have a rotating door study coordinators but to try to have the same study coordinator because I can tell you my patients developed relationships with me, my study coordinator, with my nurse, with these positions being fixed during the study period.” | Male, Study Investigator | |
| “But I do think it depends on the level of experience of the staff you have, right? If you’re someone who is very, very new, it would be a colossal mistake to put them to do a study like this, because if we think about issues of trust, of science, you can break that trust in many different ways. And one of those ways is to have a bad experience and participate in research.” | Male, Study Investigator |
TABLE C-2h Attaining Resources and Support to Accomplish a Representativeness
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| Inclusiveness should be a national priority | “Inclusiveness in research should be a national priority. Again, for the reasons that not only do we need science that provides us data relevant to our population so we’re not just studying while men and using the information to treat black men or black women or vice versa. That to me is really, really important and should be a national priority.” | Male, Study Investigator |
| “But even with the clinical and the pharmaceutical companies, they are going to benefit because they’re going to prescribe these medications to everybody. And does it work for everybody? I don’t know. Do we know? And they don’t need to. They don’t. There’s no pressure on them to find out.” | Female, Study Investigator | |
| Funding needs to be increased for studies that prioritize inclusiveness | “You have to have specifically motivated program announcements . . . towards communities that have been traditionally left out.” | Female, Study Investigator |
| “So you couldn’t have done this study on a typical R01 and the biggest reason for that is the extra cost.” | Female, Study Investigator | |
| “I think that it would be good for efforts to recruit and retain these folks, to have potential additional budgeting so like it’s a $500,000 grant but you’re going to recruit over 40 percent folks with lower socioeconomic status, than there’s an extra $50,000 a year for direct costs to support those efforts. I think we have to put our money where our mouth is, and I don’t see that is happening. Especially because what is happening is that you’re being held to task a lot more as a clinical trialist, you’re held to task a lot more for hitting your recruitment targets. And so an acknowledgment that I can easily hit those recruitment targets with the wrong people. But I cannot easily hit them if I’m being really intentional about this, so we need some sort of incentive to balance that.” | Female, Study Investigator | |
| “You have to include the community partners and the community organizations staffing and what they need. You have to include a timeline that also takes into consideration because it does take much longer to establish and start up a study in the community setting because they have other priorities. They have a lot of other stuff going on. . . . It’s not realistic to expect it to be something that fits the model of what’s done in the academic ivory towers. So, yea, it’s really budgeting for time and effort of people who are not typically thought of in grants. But those are like most of my grants, like all of the funding goes externally . . . like subcontracts to different partners.” | Female, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “Research funding has been so disproportionately low in these communities compared to the population size and how we need to rewrite these rules. We need to put directed funding into these types of research.” | Female, Study Investigator | |
| “We really did want at least a partial rural sample because that’s probably who needs telehealth the most. And so we did have the funds to do that driving. We had, it was a large R01 trial. So, I mean our budget was hardy and we were able to have a research coordinator, which was me, do the driving and I was just part of the job . . . That as the work time of driving that was certainly part of that 40 hour work week.” | Female, Study Investigator | |
| “The budgeting is really critical. You mention that because if you don’t have enough money to hire a recruiter, you’ve just kept your budget for personnel really tiny in your head. So I see this in a lot of studies I review. So there’s many people who are Co-Investigators on these projects and they’re all 10% effort. But you have very few people on the ground doing the work. You’re not going to be successful, certainly with this more difficult group to recruit, because it takes a lot of time.” | Female, Study Investigator | |
| “We need more personnel than expected. We have to hire more RAs than we thought. And it wasn’t really expensive. And I know since then NIH has done some efforts to make the R01 four years. And it wouldn’t have been possible in that scenario. So I do think financially finances are a big thing.” | Female, Study Investigator | |
| “I guess I would just say that I think it’s a combination of both a site-level responsibility and also a sponsor-level responsibility. I think it would be much harder to accomplish and increase diversity in clinical trials if both parties weren’t doing everything they could to try and improve diversity.” | Female, Study Coordinator | |
| “I think everything comes back to money and time. You know, I think the key is having time to make those connections, having time to reach these communities. But time equals money. And so I think it’s just understanding that it’s a priority. And so I do think particularly with NIH, each year they cut our budget and then we have to reallocate time and figure out how that’s going to work and all that. But it’s nice to be able to have a cushion for that infrastructure to be able to do these things.” | Female, Study Investigator | |
| “But, unfortunately, as you know, a lot of our budgets don’t allow us to do it. We’re barely making it through the study and we’re like, oh my goodness. But I do think funding agencies need to pay attention to . . . how we have that sort of close relationship right with the participant.” | Male, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “Let’s say I budget myself for 20% on the time on a grant knowing that 5% of that time I’m going to be using that for outreach. Then when, each year, I can build that into my grant when I write that. But then when they cut my budget the next year, then I go down to 15% because that’s what it’s budgeted for. The university gives us a raise but NIH says no raises and so then your time has to go down. And so even if I plan ahead and I build in this time and it keeps getting less and less, then it’s hard to figure out where that comes from.” | Female, Study Investigator | |
| “I would say they could look to fund protocols or projects that are written by people of color, not white physicians at large academic institutions. So look to fund more diverse PI populations.” | Female, Study Coordinator | |
| “I think that for me one thing that would help is if funding agencies provided additional funding, there was an option for supplemental funding, for example, to bring on sites that could enhance the diversity of the population.” | Female, Study Investigator | |
| “You have grants that are specifically for underrepresented minorities and their application process is different. . . . . NIH just had these transformational applied research grants, [they] were the first time that they ever had them. And so I think something like that is getting there but it’s still too academic. . . . We are scientists. We want to ask research questions but the reality is it misses this really large group of very smart people doing really good work and we’re missing it. We’re not funding it.” | Female, Study Investigator | |
| Provide material resources to community organizations | “It’s also bringing resources to them. So we’re having grants to fund them and their staff and the work they do and the other programs that they care about building and working on. So having, working that into grants, it’s really important.” | Female, Study Investigator |
| “I think that they actually have to provide financial incentives for organizations, actually nursing staff, as well as the organizations to care about these issues. I mean, really facilities do care about their operational and management issues. Those are key priorities for the resources, not environment at all, even though they may have a great future. And there is some movement to create, and I’m talking about typically financial or physical material. . . . So resources, concrete resources, financial incentives to our facilities that actually care. That give up more of a diverse representation . . . . I think that there has to be some kind of flexibility in terms of how we can use the fund to actually motivate and engage facilities and providers. They are huge gatekeepers.” | Female, Study Investigator |
| Subthemes | Selected Quotes | Gender/Role |
|---|---|---|
| “Also look to fund research centers . . . in more rural populations, or in cities that have a larger Black population, larger Hispanic population. Yea, I would say throw money at the diverse population.” | Female, Study Coordinator | |
| Creating education resources and networks for study teams at federal and institutional level | “I think education, I mean recruiting more leaders and then education for those leaders, all in the spectrum in health care.” | Female, Study Investigator |
| “This original trial we started recruiting back in XXXX. And, since then, I have really seen an increase in recruitment strategies at national presentations we go to. There’s been a lot more preconference workshops on it and just presentations and general symposiums. . . . So I think education has certainly helped. I think that’s been a good first step.” | Female, Study Investigator | |
| “I think that I attend diversity trainings as voluntary once a month and we have forums and different speakers and it’s just opened my eyes up so much and made me very sensitive to how important all this is.” | Female, Study Investigator | |
| “I’ve always thought that our universities, bit as it really is, why isn’t there an office? I mean, there’s an office of diversity, which as you know, they organize all those panels that I go to and people with disabilities, it’s all different. So it gives you this whole picture of all different kinds. I just don’t understand why and I tried to talk to somebody here and she was not helpful at all, but it should be. How about a group of underrepresented people that are paid to do this, to have connections in the community so that you trust her . . . Somebody that already has these connections so that they can help with cancer research, recruitment, or Alzheimer’s or anything else. I mean, why isn’t it? Seems like a good way to go and I just feel like universities haven’t really done that. These little committees I go to, people say, oh, you might be able to call this but there’s just, they don’t seem to have a real super interest in it.” | Female, Study Investigator | |
| “Put them in consortiums of them together and then try to see what we find in this community. And why can’t they do that match? . . . There’s a lot of people studying Asian here and we’re all disconnected. Big no-no, NIH should know everyone with their funding for Asian studies. Why don’t they also say, ‘Hey guys, work together. We’re giving you millions of dollars each to do this. What are you all doing together?’” | Female, Study Coordinator | |
| “I do think that federal government and funding agencies need to invest money into looking for best practices. Not just in one place, with best practices that can be scalable to various settings because, yes, we need to enroll trials in high diverse areas but we need to also get people in on diverse areas to learn how to be more effective at recruiting diverse patients in the clinical trial.” | Male, Study Investigator |
This page intentionally left blank.
