3

BCOR Findings

In this chapter, the committee presents its findings related to the activities conducted by the Building Communities of Recovery (BCOR) grantees and the outcomes of these activities. The findings are organized by the framework developed by the committee as described in Chapter 2, including grantee activities, partnerships and network building, environmental and structural change, and outcomes at the individual and community levels. It begins with a summary of the program purpose and a brief description of the information sources reviewed and their respective limitations. Committee conclusions about all four Comprehensive Addiction and Recovery Act (CARA) programs are presented in Chapter 7.

PROGRAM PURPOSE

Title III, Treatment and Recovery, Section 302 of CARA authorized the U.S Department of Health and Human Services (HHS) to award grants for supporting the development and expansion of recovery services. BCOR is intended to organize community resources and bolster support systems for substance use disorders (SUDs). Specifically, the grant’s aim is to support “the development, enhancement, expansion, and delivery of recovery support services (RSS) as well as promotion of and education about recovery.”¹

___________________

¹ Though BCOR is a new grant program, the committee notes that recovery support services are not a new construct for the Substance Abuse and Mental Health Services Administration (SAMHSA) funding; the agency has previously funded a program that shares similar goals, called the Recovery Community Services Program. For a recent Funding Opportunity Announcement, see https://www.samhsa.gov/grants/grant-announcements/ti-20-002 (accessed January 31, 2023).

BCOR grantees are registered 501(c)(3)s and—to ensure appropriate representation—must be led and governed by individuals within the local communities of recovery (SAMHSA, 2017a). The Substance Abuse and Mental Health Services Administration (SAMHSA) funded 8 grantees in fiscal year (FY) 2017 and an additional 18 grantees in FY2018 (SAMHSA, 2017g, 2018f).² Grantees received funding for a maximum of 3 years (SAMHSA, 2017a), although two grantees received no-cost extensions and one relinquished the grant.³ A complete list of grantees can be found in Appendix A.⁴

INFORMATION SOURCES AND LIMITATIONS

As discussed in Chapter 2, the committee reviewed information from three different sources.⁵ The committee received aggregate data from SAMHSA about client enrollment and outcomes submitted by BCOR grantees through the Government Performance and Results Act (GPRA) reporting system—a more detailed description of the content of the reporting system and its limitations is included in Chapter 2. The committee also reviewed grantee progress reports. The committee reached out to grantees directly and requested comprehensive progress reports or most recent progress reports, as available. Five BCOR grantees submitted these reports. SAMHSA then submitted redacted progress reports from the remaining grantees. In order to supplement the information it received from SAMHSA, the committee subcontracted with NORC to conduct interviews with a sample of grantees. The report is reprinted in its entirety in Appendix B, and the limitations are discussed in greater depth there. It should be noted that many of the findings were grouped by Center for Substance Abuse Treatment (CSAT) versus Center for Substance Abuse Prevention (CSAP) grantees, which meant that it was not always possible to comment on the extent to which findings pertained specifically to BCOR grantees, as opposed to State Pilot Grant Program for Treatment for Pregnant and Postpartum Women (PPW-PLT) grantees.

In this chapter, the committee has also noted relevant literature about the effectiveness of interventions similar to those grantees implemented.

___________________

²Box A-1 in Appendix A lists the grantees of the BCOR program.

³ Personal communication, email with Tanya Geiger, SAMHSA, September 9, 2022.

⁴ Text in this section is reprinted from the second report in this series (NASEM, 2021).

⁵ All information supplied by SAMHSA is available on request through the National Academies Public Access File, see https://www.nationalacademies.org/our-work/review-of-specificprograms-in-the-comprehensive-addiction-and-recovery-act (accessed March 6, 2023). Information used directly in the report are publicly available in an online appendix on the National Academies Press website as “additional resources” to the published report, which can be found at https://nap.edu/26831 (accessed March 9, 2023).

However, because the committee often did not have enough information about grantee contexts, it also cannot directly compare grantee efforts to this outside literature or directly extrapolate inferences about grantee effectiveness. Nevertheless, it notes some instances in which grantee plans were rooted in interventions that have potential to be effective.

As described in Chapter 2, the committee reviewed the information from all three sources and organized the information according to the framework discussed in Chapter 2. The findings are described below. Conclusions are presented in Chapter 7.

FINDINGS: GRANTEE ACTIVITIES

BCOR grantees engaged in a range of activities, including building partnerships and systems, as well as providing services for people with SUD, services to professionals in the SUD field, services to the broader community, and government outreach.

Accomplishments

Nearly all grantees described accomplishments of their activities, despite facing barriers (described in the following section). Numerous types of activities were reported within and across grantees in their progress reports, with some exemplar accomplishments highlighted. These activities are summarized here. The information provided in the progress reports was generally descriptive, and rarely gave a sense of the reach (e.g., numbers of clients or professionals served, attendees at events). NORC’s interviews were able to bring more clarity to some activities.

Partnership Building

Many grantees aimed to build or improve their partnerships with organizations in their communities, counties, or even states. In the NORC interviews, when asked to comment on the successes of the grant program, many CARA grantees across the four programs highlighted building and strengthening partnerships; the report stated that this was especially true of BCOR grantees. In progress reports, most BCOR grantees reported connections with multiple key stakeholders for supporting recovery. A breadth of relationships was described, which often included health care (e.g., emergency departments, health centers), police departments, corrections (e.g., jails), SUD treatment programs, other recovery-focused organizations, services providers (e.g., legal, vocational, social services) and governmental agencies. At least one engaged with an organization serving veterans, another with an innovative policing program (LEAD), and another with area

churches. Often this relationship-building was seen by grantees as foundational to improving knowledge about SUD and recovery, reducing stigma, and creating opportunities for future endeavors.

In many cases there were exchanges of services (e.g., training and referrals), described in the following section on “Partnerships and Networks.” Some partnerships were informal and some were formalized through agreements. Grantees also engaged in community organizations and meetings to make themselves known, highlight recovery and reduce stigma, and set the stage for future partnerships.

Building Internal Capacity

Efforts to build internal capacity were frequently highlighted. Grantees who pivoted to virtual services during the pandemic frequently emphasized the long-term benefit for their clients. Others focused on building their technology systems and improved physical infrastructure (e.g., new locations). Adding staff as part of the grant, and training staff as described below, was viewed as essential capacity-building to enable the grantees to provide services to their clients and engage with the community. Some acquired external funding to support ongoing services, or made other sustainability efforts, to ensure that their internal capacity built through the BCOR funding could be maintained.

Many grantees highlighted leadership training and other workforce development as key to building and sustaining their services. Some mentioned helping their staff obtain certifications as well.

Services for People with SUD

All of the data sources spoke to the efforts grantees took to provide services for people with SUD.⁶ Few details about these services or their models were made available to the committee, by SAMHSA, to evaluate.

According to the grantee progress reports, peer services, recovery coaching, and recovery support services were offered by the majority of grantees, although not necessarily all of these by the same grantee. In some cases, grantees focused on specific populations (e.g., women with children, Latinx, LGBTQ+, HIV+, family members).

Some grantees provided wraparound or ancillary services to meet the needs of their clients in recovery, including clinical case management, social

___________________

⁶ The committee notes that not all BCOR grantees provided those services directly (e.g., some trained peer support counselors who would later provide such services), which contributed to difficulties using the GPRA data tool. Please refer to the section on “Individual-Level Outcomes” in this chapter for a more detailed discussion.

supports, educational/vocational supports, and, during the height of the pandemic, even meals and transportation; such services have been demonstrated to support recovery (Eddie et al., 2019; McLellan et al., 1999; Scott and Dennis, 2009; Stewart et al., 2021). One grantee developed a program to partner with organizations and businesses for hiring and support of people in recovery and reducing stigma; maintaining employment has been demonstrated to support recovery (Sherba et al., 2018).

In the NORC interviews, many grantees reported that the CARA grants allowed them to expand their target populations or geographic service areas, which allowed them to increase the number of clients eligible for services.

In progress reports, grantees noted that the switch to virtual or hybrid services in some cases also increased their reach in terms of target population and geographic service area. This point was echoed in the NORC interviews, with one grantee commenting that the switch to virtual services also brought cost savings. In other cases, as highlighted in the section on “Barriers” in this chapter, it was identified as a detriment since in-person contact is quite important for relationship building during recovery.

More general health and wellness activities were deemed important to recovery and offered by some grantees. These included support groups, educational sessions, and even a conference. Social connectedness was another area of focus, which overlaps in part with community events such as recovery rallies or walks.

Data on the provision of selected services were obtained as part of the standardized GPRA reporting on participating clients [B3a].⁷ The committee has highlighted some of the activities most relevant to the BCOR grant goals, and the percentage of clients that received them, for which we have data (henceforth called “reporting clients”). Before presenting these data, the committee notes that they had several key limitations.⁸ In particular, the committee only had data for those who completed a discharge GPRA interview (about one-third of the enrolled clients), not all clients in the program; the committee could not tell what were expected or appropriate services for any given client or grantee; and the data do not reflect how many times each client received a given service. Finally, the committee does not have information on exactly the definitions that grantees used for these services, or their evidence base for the specific services provided, and, due to the anonymization of grantees by SAMHSA, cannot comment on the

___________________

⁷ Citation refers the source of the material from that submitted by SAMHSA and included in the online appendix. Indicates the program and Excel citation (BCOR [B], workbook [3], and worksheet [a]).

⁸ For a more detailed discussion of the limitations of the GPRA tool in general, refer to Chapter 2 or to the section in this chapter on “Individual-Level Outcomes.”

suitability of their particular service models to the populations they served. However, these data suggest that most clients reflected in the GPRA dataset were receiving the basic recovery support services, and likely a range of other services including formal treatment, self-help, and needed supports and referrals. Selected services provided by BCOR grantees include:

• Core BCOR services
  • Assessment: 87 percent of reporting clients
  • Case management: 90 percent of reporting clients
  • Recovery support services: 79 percent of reporting clients
• Other recovery support services
  • Treatment or recovery planning: 54 percent of reporting clients
  • Relapse⁹ prevention: 35 percent of reporting clients
  • Recovery coaching: 37 percent of reporting clients
  • Peer coaching or mentoring: 26 percent of reporting clients
  • Other peer to peer recovery support services: 9 percent of reporting clients
• Formal treatment
  • Group counseling: 69 percent of reporting clients
  • Individual counseling: 14 percent of reporting clients
• Informal treatment
  • Self-help and other support groups: 24 percent of reporting clients
  • Spiritual support: 23 percent of reporting clients

The committee notes that assessment, case management, and recovery support services are the core activities expected under the header of recovery support services that the BCOR grant aimed to fund, and are generally evidence-based (Bassuk et al., 2016; Eddie et al., 2019; Humphreys and McLellan, 2011; McLellan et al., 1999). The data provided by SAMHSA do not allow an examination of whether these three core services were complements or substitutes, but it is reasonable to assume that every reporting client received at least one. In addition to those core services, the majority of reporting clients (84%) participated in alcohol- and drug-free social activities, an essential element for building the connectedness that is key to recovery. Information and referral highlighted the importance of

___________________

⁹ This is the term used for this type of service in the GPRA tool, and is often the term used in clinical settings. In this report, the committee uses the phrase “relapse or return to use.” This choice was informed by two factors: first, recognition that “relapse” can be perceived as stigmatizing, and second, recognition that use of “relapse” alone can imply an all-or-nothing view that any departure from abstinence constitutes a “failed” recovery (for a lengthier discussion, see Miller, 2015); a harm reduction lens applied to SUDs recognizes that this is not the case.

knowledge to support recovery and additional services, and it was received by 27 percent of reporting clients [B3a]. Additionally, housing support is an important ancillary service to increase the likelihood of staying in recovery (Kirst et al., 2015; Reif et al., 2014b), and it was received by 17 percent of reporting clients [B3a].

Summary of Reporting Client Characteristics

The committee received intake data from 3,084 clients [B1a], which reflects data from less than three-quarters (19 of 26) BCOR grantees. The committee notes, additionally, that it is unclear exactly who received intake interviews—that is, whether the individuals were receiving wraparound services from grantees, whether they were peer counselors receiving training, or whether they were community members who participated in a training.

Among those clients with intake data, 59 percent identified as male, 40.5 percent identified as female, and 0.4 percent identified as transgender. In terms of race, 56.1 percent identified as white, 12.1 percent identified as Black, and 27.2 percent as “none of the above.” Hispanic/Latino clients represented 14.7 percent of the intake interviews [B2b]. The committee cannot comment on whether these demographic breakdowns are representative of grantee communities or represent steps toward addressing disparities in access to services, because it does not know which grantees are reflected in the GPRA data SAMHSA shared, and because the intake data are aggregated across all grantees. As such, the committee could not compare these demographics to those of the grantees’ communities or catchment areas.

At the time of intake, 74.4 percent of clients reported having zero substance use in the prior 30 days [B2c]. This relatively low rate of use is not surprising for the BCOR program, which is focused on recovery, as opposed to treatment programs, where substance use at intake is very common.

Services to Professionals in the SUD Field

Recovery coach training was a foundational activity for many grantees that was supported by the BCOR funding. Peer recovery coaching is generally evidence based (Eddie et al., 2019; White et al., 2007). In most cases, this served as professional development for people who were already providing peer recovery services. In some cases, people who were allies, family, or friends of people with SUD were also welcome to be trained. Many examples of recovery coach training were described. At least one grantee used CARA funds to offer scholarships to the people seeking training. This is one place where the COVID-19 pandemic required a pivot, with grantees generally offering training virtually (versus eliminating this activity). Grantees who did so commented on their ability to reach a broader

pool of trainees and generally increase accessibility. In NORC interviews, grantees identified hiring people with their own lived experience with SUD as a general facilitator of many grantee activities and to the organizations more broadly; this was a common activity among BCOR grantees.

Services to the Broader Community

Many grantees engaged in efforts to reduce stigma and build knowledge about recovery throughout the broader community. To do so, many grantees built advocacy networks. As one grantee described, “a well-organized, local, independent advocacy movement that reduces stigma and promotes and serves as a liaison for adequate resources and aftercare for people with substance abuse issues” is invaluable. Some grantees trained participants in advocacy approaches, and some created speakers bureaus of people in recovery who could share their experiences and knowledge with the community and in specific settings such as public meetings.

Participation in community events was mentioned by many grantees as another strategy for reducing stigma and building public awareness. Efforts included recovery-specific events and engaging in broader community events. One mentioned providing event space for other community organizations as a way to reduce stigma and model leadership. Some grantees created recovery-focused groups in the community, such as a monthly open meeting for anyone interested in recovery support services or learning lunches.

Several grantees participated in the development of resources of value to the broader community. One collaborated with others in the state to create a statewide public resource for finding SUD treatment and recovery services, with an emphasis on “recovery assets.” Another created an instant online access to an up-to-date local list of recovery support meetings, recovery supports including clinical treatment, and ancillary supports important to recovery. At least one grantee distributed naloxone kits.

Several grantees had an intentional focus on engaging with governmental leaders and agencies. For some, this was about contributing to legislative efforts. For others, it was about being prepared for licensure, certification, and similar governmental requirements. Some grantees were in states or communities that had recovery represented within governmental agencies, such as an office of recovery-oriented systems of care, with which they were engaged.

Barriers

In both grantee progress reports and in the NORC interviews, most grantees noted barriers that affected their ability to conduct their planned activities.

The COVID-19 Pandemic

In particular, the COVID-19 pandemic had a major impact. Though both cohorts of grantees began their work prior to the beginning of the pandemic, the FY2017 cohort was funded through September 29, 2020, and the FY2018 cohort was funded through September 29, 2021; in both cases, their implementation periods overlapped with the pandemic. The two grantees that received no-cost extensions continued to implement their programs further into the pandemic. Examples of COVID-related challenges highlighted in progress reports include sites or partners shutting down services or moving to virtual services. For many grantees, this limited the services they were able to provide to people with SUD, professionals in the field, and the broader community, starting in March 2020. For some grantees, the adaptations became successes in their ability to reach more participants through virtual activities, or in their ability to offer social and other supports to their clientele that were increasingly important during the pandemic. At the same time, grantees noted that transitions to virtual services and social distancing were a detriment since in-person contact is quite important for relationship building during recovery. NORC interviews echoed the fact that this was a challenge for BCOR grantees, given that recovery programs are about personal connection.

In both progress reports and the NORC interviews, grantees highlighted that the pandemic also was associated with an increase in SUD and overdose, even in areas that had seen prior decreases, creating additional burden for these grantee service agencies. In NORC interviews, grantees noted that this trend created stress both for increased demand for services and potential return to use by clients and peer staff.

Interviewed grantees in the NORC report also mentioned overall staff fatigue due to operating during the pandemic, and difficulties partnering with their health care partners (e.g., emergency departments, federally qualified health centers) that were overburdened.

Staffing

Staffing was another key barrier. In some progress reports, grantees reported that staff turnover was frequent; in part, grantees attributed this to the fact that as people were trained (e.g., through staff development or certifications) they became more valuable employees not just to the grantee but also to other organizations providing similar services in the area willing and/or able to pay them more. Several grantees had transitions at higher staff levels (e.g., project director), which greatly affected implementation of the proposed activities. In one case, the lack of a project director led the grantee to relinquish the CARA grant altogether and cease activities.

Several grantee progress reports noted that the transition from client (i.e., a person in recovery receiving recovery support services) to paraprofessional (i.e., peer or recovery coach) can be challenging. Staff who are in recovery themselves are also at risk of returning to substance use, particularly when they become heavily engaged in clients’ own lives. Some of the peer recovery coaching staff had difficulty creating boundaries or ensuring their own supports to maintain their recovery, a concern that is common across the peer recovery field, and is not unique to BCOR (Felton et al., 2023; White et al., 2007). Those BCOR grantees that provided additional support resources to these staff members (e.g., holding support groups for these paraprofessionals, trainings on how to maintain boundaries with clients) may have been more successful at reducing return to use among the peer support paraprofessionals.

Systemic and Policy Barriers

Another type of barrier identified in grantee progress reports was related to the broader, structural environment. The grantees cited as barriers to various activities the low availability of adequate government funding to support recovery activities, difficult processes to license recovery homes, and low willingness of other organizations to partner with recovery organizations.

For client engagement and services provision to individuals with SUD, some grantees in the NORC interviews cited stigma around harm reduction and medications to treat OUD as a barrier to client engagement. Grantees providing treatment or recovery services (the CSAT programs) also highlighted the lack of infrastructure in their catchment areas for other wraparound services that would support recovery (e.g., transportation, housing), and medications for opioid use disorder (MOUD) access. Some grantees in the NORC report also cited lack of internet as a barrier to client participation, especially during the COVID-19 pandemic and in rural areas; however, it is unknown whether this finding applied specifically to BCOR grantees.

SAMHSA and Grant Management

Several barriers were related to the funding agency’s management of the grant itself. In the NORC interviews, CSAT grantees highlighted funding delays, which affected ability to start up the grantee activities in a timely manner, inconsistent or nonexistent communication with SAMHSA Government Program Officers, short implementation periods, and the challenge of the funds matching requirement for the BCOR program specifically.

Additionally, in the NORC interviews, CSAT grantees raised concerns related to reporting and GPRA data collection. They highlighted that some

questions were stigmatizing or irrelevant to their targeted populations, which were either individuals in recovery or recovery coaches in training. In progress reports, some grantees reported intentionally slipping on standard procedures (e.g., GPRA required at intake as part of SAMHSA funding) in order to “gain trust” of their clients. The tool was especially irrelevant for recovery coaches in training (a finding from NORC interviews that was specific to the BCOR grantees), but grantees were still expected to conduct interviews; follow-up was especially difficult for attendees of trainings and events. In part, this reflects the belief by some grantees that the GPRA was not an ideal tool, especially for activities focused on recruiting and training recovery coaches. The suitability of the GPRA tool for the BCOR program is discussed in more detail in the “Individual-Level Outcomes” section of this chapter.

FINDINGS: PARTNERS AND NETWORK

A required activity of CARA BCOR grantees was to “support the development, expansion and enhancement of community and statewide recovery support services.” An allowable activity included a priority on developing sustainable infrastructure through partnerships with community and state organizations.

Types of Partnerships

As highlighted in progress reports and NORC interviews, grantees partnered with diverse entities ranging from the more common partnerships with law enforcement, faith-based and other community-based organizations, treatment providers, social services agencies, and health care organizations, to unique partnerships with doula services and legislative representatives.

Partnerships between grantees and existing treatment programs within their geographical region could allow for synergy between people with a cultural awareness of recovery. These types of alliances may be more straightforward as a shared clinical language already exists. For example, one grantee appeared to be an existing, similar program that used grant funds to (1) expand existing recovery support services (specifically based in Wellness Recovery Action Plan approach) throughout the county in collaboration with 10 publicly funded drug treatment providers and (2) increase participants’ access to and use of recovery services and supports. This alliance demonstrated sustained commitment to recovery and maintaining and improving individuals’ quality of life.

Several grantees partnered in the employment support space. They partnered with organizations providing services such as supportive employ-

ment, job training, and education to employers about the value of hiring people in recovery. One grantee partnered with the state Department of Workforce Development, where it was listed as a vendor service provider to support rehabilitation for individuals with disabilities.

In the NORC interviews, one BCOR grantee stated that they felt having a SAMHSA grant lent their organization credibility that facilitated partnership building.

Focus on Sustainability

Collaboration across disciplines and sectors is vital to a growing a community of recovery. In this way, sustainability of partnerships can support success.

Grantees tried to find creative ways to partner that would promote sustainability of recovery services. Grantees reported that creating statewide consortiums to share ideas, outcome data, and resources could increase number of people served and improve fidelity to models of care. Grantees who included infrastructure development of shared databases, streamlining program certification efforts, and creating consortiums of collaborative entities allowed for traction in funding and trackable outcome data. One grantee was able to “unite existing local recovery supports to build and sustain a coordinated recovery-oriented system of care” through these mechanisms. Another spent the first year of their grant building a qualified peer recovery coach workforce, then worked with policy makers in the state Medicaid office to set standardized training for reimbursement, implemented this, and succeeded in becoming one of the only state-certified training resources for recovery organizations billing to Medicaid.

Some grantees faced challenges in creating strategies for sustaining partnerships. Outcomes regarding how many people were offered employment support services were trackable at intake and were reported out at grant completion, but determining mechanisms to support sustainability were not noted. Teaming with a state entity that has an internal tracking mechanism in place, such as the Department of Workforce Development example provided above, may be a way to increase likelihood of sustainability.

Challenges to Partnerships

Despite these successes, grantees noted a number of challenges that complicated efforts to build partnerships.

Challenges to BCOR partnership work, similarly to grantee activities discussed in the first section, were most frequently a result of the COVID-19 pandemic; grantees highlighted such challenges in progress reports and in the NORC report. In-person services and work needed to quickly pivot to

virtual platforms. Training that required a clinical component of training and education was put on hold until treatment programs and organizations reopened their doors to the general public. Hiring and training new staff in a virtual, uncertain fiscal market slowed any growth.

Several grantees highlighted that the stigma that accompanies building services for people living with SUDs at times made it difficult to engage new partners. A sense of “not in my backyard” infiltrated some grantees’ work toward growing and strengthening partnerships. In other cases, grantees noted a lack of understanding about why recovery support resources in the community were needed instead of just treatment (in that communities did not always understand the chronic, relapsing nature of the condition).

Additionally, staffing shortages faced by teams both inside grantee and partner organizations, worsened by pandemic challenges, made some potential partners unable to engage during the timeline of the grant.

Some grantees suggested that teaming with law enforcement made the treatment environment less of a supportive environment for clients, and furthered the narrative that people in recovery have more contact with law enforcement for punitive reasons, despite the grantees’ goal of improving knowledge and reducing stigma across law enforcement.

FINDINGS: STRUCTURAL AND ENVIRONMENTAL CHANGE

This section focuses on grantee activities aimed at strengthening and maintaining the intervention well past the end of the grant, for example, ensuring programming sustainability or advocating for policies with eventual payoffs for the recovery community. As such, these activities might not directly impact the populations targeted by the grant activities but rather constitute systems or structural changes.

The committee notes that it was not always possible to discern the extent to which efforts with potential environmental/structural impacts were implemented, let alone produced expected effects. Thus, in this section, the committee identifies “potential” environmental/structural changes occurring during the period of grantee activity.

Accomplishments

Outreach to Policy-Making Bodies

Based on the information provided in the reports, grantees varied in their outreach (whether done at all, who it was directed to) and the focus of their work with policy makers. Policy makers targeted ranged from local and county agencies to state agencies. While for some grantees the focus of their outreach was on facilitating certification or credentialing

of RSS providers, some grantees devoted substantial energy to developing and strengthening relationships with policy makers to sensitize and educate them and pave the way for BCOR-promoting policies. For example, a grantee routinely shared their activities and results with legislative leaders:

Another developed a statewide network of peer advocates who engaged in advocacy through writing letters, inviting elected officials to their programming, visiting elected officials at the Capitol, and regularly providing testimony during budget committee hearings. Yet another grantee invited elected officials to attend advocacy events and provided testimony before the legislature. The outcomes of such efforts were not reported and may require a longer time frame to evaluate. However, the Committee agrees that policy changes could support efforts similar to these.

Efforts to Secure Funding Past the End of the Grant

Many grantees reported various efforts to secure additional funding both concurrent with the grant, and in order to ensure the viability of grant-funded activities past the end of the grant. This was echoed in the NORC interviews, in which several grantees across the CARA programs reported they were “applying for additional federal, state and/or local funding to maintain internal supports (e.g., staff) and processes (e.g., trainings).” According to progress reports, some grantees took initiative to learn about ways to generate additional revenue streams, implemented revenue-generating operations (e.g., thrift shops) or joined income-generating programs (e.g., 340B pharmacy program), or were awarded state or local foundation grants to expand current operations.

Others were able to meet requirements to be able to bill Medicaid and other payers for their services. Notably, two grantees were able to shape state policy around the certification process for peer and non-peer training curriculum, credentialing, and requirements for reimbursement through Medicaid. The ability to bill insurance for peer recovery support services represents an important step toward sustainability, as well as broader implementation and adoption of these services.

A smaller number of grantees were explicit in their efforts to secure funding to ensure that their programming would remain operational following the expiration of the grant. One grantee in the latter group reported

that the county would continue to fund some of its operations, another created a statewide coalition of recovery community organizations (RCOs) that was able to receive state funding to continue to provide RSS after the end of the grant, and another reported having applied and received local grants to enable the continuation of their services.

Efforts to Create Lasting Partnerships

Partnership-building efforts, as described in the previous section, could help improve coordination across services that benefit the recovery population. This could bring benefits in the long-term, like improving care, increasing community knowledge, and decreasing stigma.

Few grantees were explicit in highlighting efforts to ensure lasting partnerships, and SAMHSA reporting materials did not inquire about such efforts. Among the few grantees that did, a grantee reported efforts to work with existing partners toward finding new mutually beneficial partnerships in the county’s local communities.

Technical Assistance, Training, and Capacity-Building

Participation in technical assistance programs can promote sustainability by preparing professionals inside and outside of the organization to deliver evidence-based practices for this population. Few grantees reported engaging in this activity (though the committee notes that this was not required in SAMHSA reporting tools). Among them, one grantee reported participation in SAMHSA’s Bringing Recovery Supports to Scale Technical Assistance Center Strategy (BRSS TACS),¹⁰ while another contracted with Ohio Recovery Housing to gain certification and also utilize their technical assistance and training associated with certification.

Barriers

The nature of these outcomes complicates the identification of barriers, and the data sources did not always provide relevant information. Generally, many of the barriers identified for grantee activities may have impacted efforts presented in this section. In particular, the most commonly reported barriers to grantee activities (COVID-19 pandemic and staffing challenges) are likely to have limited grantees’ ability to invest in sustainability-promoting activities and advocacy activities with policy makers. The systemic and policy barriers highlighted, such as lack of adequate funding for recovery

___________________

¹⁰ For more information, see https://www.samhsa.gov/brss-tacs (accessed February 1, 2023).

services, also likely limited grantees’ ability to devote resources to environmental and structural change efforts.

Additionally, the committee notes that the generally low quality of the evaluation designs (essentially before-after designs), the substantial loss of subjects at follow-up (see section on “Individual-Level Outcomes”), and the low quality of the reported findings may complicate sustainability efforts and advocacy activities with policy makers. However, a few grantees were able to produce good-quality evaluation results, which they shared with local policy makers. One NORC interviewee (though it is unclear whether this was a BCOR grantee) did note that it thought leveraging its CARA data may have helped it to secure other funding.

Last, grantees probably varied in their expertise in applying for additional funding, depth and breadth of their networks, and connections with policy-making bodies. Future grants should include sufficient resources for evaluation activities and technical assistance so that grantees can develop and implement plans for sustainability early in the life of the grants.

FINDINGS: INDIVIDUAL-LEVEL OUTCOMES

It was an explicit objective of Title III, Sec. 302 of CARA to support independent, nonprofit organizations in the development and expansion of recovery services. Therefore, per the reporting requirements of SAMHSA, most of the 27 BCOR grantees provided some type of information that could be used to support the success of the grantees in retaining people in treatment or recovery. Despite SAMHSA providing the data, the committee had several concerns about the representativeness of the data as well as the strength of the evaluation designs used to collect them.

First, the only outcome data that were consistently reported across some of the BCOR grantees were those collected within the GPRA system. Of the 27 grantees, 19 reported some GPRA data, and SAMHSA did not clarify why the other 8 were not included; it was also not clear that the data SAMHSA shared covered all fiscal years for each of the 19 reporting grantees. The grantee progress reports shed light on several reasons for this. First, several of the BCOR grantees only trained recovery coaches (six grantees) or provided wraparound services to people in treatment (three grantees); they did not actually provide recovery services to individual clients, so they did not find the questions in the GPRA reporting tool pertinent for their clients/activities. Second, some grantees that were willing to use the GPRA reporting tool said that they found the intake questions too invasive for individuals participating in their programming (cited by at least two grantees), whether it was for clients or recovery coaches, so they delayed the interview until trust could be built up, which meant several clients never

underwent the intake interviews (as they left before trust could be built). Thus, it is clear that the GPRA intake data the committee received from SAMHSA suffer from self-selection and items that are misaligned with the goals of the BCOR grantees and the people receiving their services.

Another problem with the GPRA outcome data is that even those grantees successful at obtaining intake data had difficulty obtaining followup data on the same participants. As shown in Table 3-1 below, the committee received follow-up outcome data on only two-thirds of those with intake data, across all reporting grantees; the follow-up rates also varied greatly between grantees [B2f]. From the data provided, it cannot be known whether the lack of data at follow-up for those not reporting is due to the participants still in treatment not yet reaching the 6-month point, or if those non-reporting reflect participants dropping out of the grantees’ programming before reaching the 6-month mark. In the NORC report, CSAT grantees reported trying to provide client incentives and gift cards as one strategy to addressing low follow-up rates, though the committee notes that it does not have enough information to confirm whether such strategies increased follow-up rates.

Given all of these selection and measurement issues represented in the GPRA data, the committee determined these data were insufficient for drawing any conclusions regarding the effectiveness of the BCOR program on individual outcomes. The selection bias caused by nonrandom reporting across grantees as well as across participants within reporting grantees makes it impossible to know how generalizable any of the findings would be.

Similarly, the confounds of the GPRA data limited the committee’s ability to analyze the GPRA data on length of stay in treatment [B2a] and on client discharges [B4a]. First, length of stay and discharge were hard to interpret given that, as already raised, some grantees were surveying recovery coaches or individuals receiving wraparound services, rather than surveying individuals who were undergoing treatment, for whom length of stay and discharge might be more relevant. Additionally, the committee received length of stay and discharge data for only 30.7 percent of those clients with intake data; the committee is unsure of whether the missing data are due to clients still being enrolled in the grantees’ programming.

As discussed above, a recurring point made in several final progress reports submitted by grantees was that the GPRA form was not really appropriate for evaluating the individuals going through their respective programming, especially for individuals in recovery or recovery coaches in training. Additionally, a small number of grantees suggested that questions about abstinence in the past 30 days, which is important to understand for people’s intentions and readiness for treatment, was found by staff to be difficult to ask at intake before trust had been built; this is in part because return to use, although common for those in recovery, can be viewed as

TABLE 3-1 Outcomes from Participants for Which Grantees Have Both Intake and Follow-Up Data (N = 2030)

^a Note that for some clients, the 6-month follow-up interview occurred before a discharge interview, and for some, the reverse was true. These outcomes are for the “first” follow-up, so whichever of those two interview types occurred first for a given client.

^b Note that only 65.8% of clients with intake had a follow-up interview; thus, these outcomes may not be representative of all clients.

SOURCE: B2f.

a failure. Researchers in recovery often use “number of days sober” as a measure of current stability in recovery, rather than abstinence in the past 30 days; even this is a simplistic measure of recovery, which SAMHSA defines as “a process of change through which individuals improve their health and wellness, live self-directed lives, and strive to reach their full potential” (SAMHSA, 2012).

The lack of more salient measures of successful recovery on the GPRA forms led several grantees to report additional measures—beyond those reported using the GPRA tools—in their final progress reports, including information on participants’ self-efficacy, recovery capital, and stigma. While the committee agrees that some of these alternative outcomes are in fact important outcomes to measure when evaluating recovery programs, these data also had important limitations. First, there was no consistency in the reporting of specific measures across grantees. The lack of consistency is not

surprising given that grantees were not instructed by SAMHSA to provide such information, and the individual grantees’ goals and activities varied widely. Second, among those that reported some additional measures, few grantees collected these in the systematic fashion required for effectiveness evaluation. Those that did had non-negligible loss of subjects at follow-up and/or no control group.

Without reliable data on which to assess outcomes consistently before and after BCOR program implementation, it is difficult for the committee to say conclusively what these grantees, and the BCOR program overall, were or were not able to accomplish in terms of individual outcomes.

FINDINGS: COMMUNITY-LEVEL OUTCOMES

Unfortunately, the committee found a similar pattern with respect to community-level outcomes. Measurement of community-level outcomes requires identification of those outcomes and good baseline data on them prior to the start of grantee implementation as well as at completion. No such process for identifying the same community outcomes or their assessment consistently across communities was required by SAMHSA.

Nonetheless, it was clear from grantee reports that many pursued three common community-level goals: (1) policy changes that increase the stability, availability of, and retention in high-quality treatment; (2) raising community knowledge about SUD and recovery; and (3) reducing stigma associated with treatment and being in recovery. More details on the activities undertaken in support of these goals can be found in the first section of this chapter’s findings on grantee activities and in the section on “Structural and Environmental Change.”

While no formal evaluation was done enabling the committee to draw firm conclusions of the effectiveness of these grantees in achieving each of these goals, it is clear that many efforts were made by individual grantees in each of these areas. These efforts may have long-run impacts on individual and community outcomes in the future. Community knowledge about SUD and recovery services could be impacted by the consistent reporting of community-wide public education activities, trainings with law enforcement and/or county jails, and partnerships with local medical centers and emergency departments from many of the grantees. Stigma reduction may have been accomplished building new partnerships that were forged with previously skeptical community groups, including law enforcement agencies, workplaces, and health professionals, which subsequently became partners. These efforts have potential to have long-run impacts on individual and community outcomes in the future. As summarized in a quote from NORC’s report, but reiterated by many progress reports, grantees expected that partnership would “change attitudes toward persons in recovery as well as increase access to services” for much longer than the short-term funding.

This page intentionally left blank.