Review of Four CARA Programs and Preparing for Future Evaluations (2023) / Chapter Skim
Currently Skimming:
Pages 163-207
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 163... ...
, the Committee suggested that SAMHSA implement and/or develop additional data-collection tools that assess alcohol and drug use and recovery in a more realistic and comprehensive manner. For the second report, Progress of Four Programs from the Comprehensive Addiction and Recovery Act,4 the Committee gathered data (work plans, evaluation plans, and progress reports)
|
From page 164... ...
Specifically, the interviews gathered information to address the following research objectives: • The goals of implementing grantees, and how they fit within the stated goals of the SAMHSA funded CARA grant programs; • The process of implementing each grantee organization's SAMHSA-funded CARA grant program, and factors contributing to successes and challenges; • How each grantee organization evaluated its SAMHSA-funded CARA grant program, with a specific focus on the types of data collected related to process and outcome metrics; • The intended and unintended impacts of the grantee organization's SAMHSA-funded CARA grant program on the population served, organizations involved, and the broader community, and assess any efforts related to program sustainability; and • Grantee recommendations for future programs and evaluations. Report Purpose and Structure This report presents findings from an analysis of interviews with program administrators at grantee organizations implementing and/or evaluating SAMHSA-funded CARA programs.
|
From page 165... ...
Methodological Approach and Data Collection As previously noted, the study's goal is to supplement information previously reported to SAMHSA on the progress of CARA grant programs. To accomplish this goal, NORC was tasked with conducting individual, semi-structured interviews with program administrators at grantee organizations that had participated in the implementation and/or the evaluation of one of the four SAMHSA-funded CARA grant programs (i.e., FR-CARA, BCOR, PPW-PLT, and OD Tx; see Exhibit 1 for a brief description of each program)
|
From page 166... ...
. NORC received from NASEM a list of grantee 1 For a more detailed description of the required and allowable activities of the CARA grant programs, please see: FR-CARA Funding Opportunity Announcement SP-17-005.
|
From page 167... ...
BCOR grantees included recovery community organizations. Exhibit 2: Population of SAMHSA-funded CARA Grantees, by Program CARA Program Universe of Grantees Selected for Interview Recruitment FR-CARA 48 19 BCOR 26 14 PPW-PLT 6 6 OD Tx 6 6 TOTAL 86 45 Recruitment of Selected Interviewees.
|
From page 168... ...
within the SAMHSA-funded CARA grant program and grantee organization • Involvement in the SAMHSA CARA grant proposal Grant Program Goals • Goals of each SAMHSA-funded CARA grant program • How program goals fit within the broader goals of the grantee organization Grant Implementation Process • Identification of key successes and challenges encountered while FINAL REPORT | SEPTEMBER 2022
|
From page 169... ...
The consent to record was also documented in the interview notes. Video recordings were destroyed immediately following the completion of the interviews, however, any notes, audio recordings, and transcripts used by the NORC study team to develop the final report will be destroyed following the conclusion of the Committee's work on June 30, 2023.
|
From page 170... ...
After coding and analysis were complete, overlapping thematic topics were consolidated and organized by the five topic areas shown in Exhibit 4. Limitations of the Research This study provides a snapshot of the experiences of grantees implementing and evaluating their SAMHSA-funded CARA grants.
|
From page 171... ...
About half of the participants had either developed or written the CARA grant proposal. Regardless of title, tenure, or involvement in the proposal, all interview participants were knowledgeable about their grant program and able to speak to the broad set of discussion topics identified in the initial outreach email (see Appendix A)
|
From page 172... ...
. For further discussion on the impact the SAMHSA-funded CARA grants had on grantees' ability to create new partnerships and sustain existing ones, see the Grant Implementation Process section.
|
From page 173... ...
we're able to offer this training, but then also help them become their own program and have capacity to • Several CSAP grantees noted that the distribute naloxone. So, then it's not just a one and CARA grant allowed them to expand done." – ID 1007 (CSAP)
|
From page 174... ...
Some PPW-PLT and FR-CARA program grantees noted successfully extending the use of their funds beyond what they had originally proposed with approval from their SAMHSA program/project officer (PO)
|
From page 175... ...
While several grantees were successful at reducing stigma within their communities, it remained a challenge for other grantees across the CSAP and CSAT programs. Grantees identified stigma among individuals administering overdose prevention medications (e.g., first FINAL REPORT | SEPTEMBER 2022
|
From page 176... ...
Therefore, the first six months of their grant was focused on educating community members about the many pathways to recovery, including harm reduction. A few CSAP grantees noted the creation of a data dashboard or repository to facilitate data sharing as a success of their CARA grant.
|
From page 177... ...
challenges identified. As noted previously, several grantees successfully worked with their POs to expand the use of CARA grant funds.
|
From page 178... ...
their CARA grant program. One CSAT grantee had a very positive reaction to the web reporting tools and felt the information collected told "a good story about what was going on with each [client]
|
From page 179... ...
Grant Program Impacts and Sustainability Grantees were asked to discuss the perceived impacts of their SAMHSA-funded CARA grant programs and their plans for program sustainability. In general, all grantees believed they were able to achieve at least some of their program goals.
|
From page 180... ...
Some grantees noted that they had not yet used the SAMHSA-funded CARA grant program or corresponding financial or impact assessments to leverage additional supports. Others reported that they were actively applying for or had already received additional federal, state, and/or local funding, including new SAMHSA CARA grant funds.
|
From page 181... ...
additional funds to cover the cost of overdose prevention medications was seen as a bigger hurdle for CSAP grantees. Some grantees are considering changes to their organization and workflow to be more sustainable in the absence of CARA grant funds.
|
From page 182... ...
to include other organizations like local health departments and schools. As noted in the Grant Implementation Process section previously, some grantees successfully expanded the use of CARA grant funds to include additional services like mobile prevention or expanded to other geographic locations like surrounding counties.
|
From page 183... ...
The goal of this work was to support NASEM and the Committee's efforts in making recommendations to Congress about the future of the federal government's response to the opioid crisis. For this study, NORC conducted semi-structured interviews with 22 grantee organizations that participated in the implementation and/or evaluation of one of the four SAMHSA-funded CARA grant programs awarded in FY2017 or FY2018.
|
From page 184... ...
However, it is important to note that in several instances securing SAMHSA funding gave grantees the credibility, clout, and confidence to persist in the face of entrenched stigma. The CARA grant program requirements supported grantees in making considerable inroads in their communities to change attitudes and perceptions about individuals experiencing overdose and/or in recovery.
|
From page 185... ...
. Measuring Success in Substance Use Grant Programs: Outcomes and Metrics for Improvement.
|
From page 186... ...
Please note that the NORC study does not seek to duplicate the information that your organization/agency may have previously provided to SAMHSA. Rather, we are interested in hearing about your organization/agency's insights and experiences with implementing and evaluating your CARA grant program, that can supplement any prior reporting to SAMHSA on the progress of your CARA grant program.
|
From page 187... ...
We look forward to hearing from you! Best, The NORC Study Team Follow-Up Interview Email for PI/co-PI as Respondent Subject Line: Invitation to Participate in a Confidential Discussion for a Study Related to SAMHSA-funded CARA Grant Programs Hello [PI/Co-PI Name]
|
From page 188... ...
Thank you, The NORC Study Team Follow-Up Interview Email for Other Program Team Member Subject Line: Invitation to Participate in a Confidential Discussion for a Study Related to SAMHSA-funded CARA Grant Programs Hello [Study Staff Name] , CC: PI/co-PI The National Academies of Sciences, Engineering, and Medicine (the ‘National Academies')
|
From page 189... ...
All interview responses will be kept confidential and stored separately from contact information, and all such data will be stored on a highly secure and encrypted server at NORC. NORC will not share contact information or interview responses with anyone outside our Study Team.
|
From page 190... ...
As part of their qualitative information gathering, NORC plans to conduct virtual interviews with individuals from approximately 45 grantees under the CARA program. As one such CARA program grantee, we are hoping that your organization/agency would be willing to participate in this NORC study.
|
From page 191... ...
NORC will not share your contact information or interview responses with anyone outside our Study Team. All interview responses will be kept confidential and stored separately from your contact information, and all such data will be stored on a highly secure and encrypted server at NORC.
|
From page 192... ...
NORC will prepare a report of findings for submission to the National Academies in July 2022. In that report, all findings will only represent aggregate opinions across interviewees so that none of the findings will be attributed to individuals, and re-identification of individual respondents would not be possible by either the National Academies or SAMHSA.
|
From page 193... ...
will include any information that identifies you. All interview responses will be kept confidential and stored separately from contact information, and all such data will be stored on a highly secure and encrypted server at NORC.
|
From page 194... ...
grant implementation process, (3) grant evaluation process, (4)
|
From page 195... ...
* Were you working at your organization/agency prior to the inception of the SAMHSA CARA grant program?
|
From page 196... ...
did the goals of your organization/agency change? Section 2: Grant Implementation Process Now we're going to move into the second section of the interview, which focuses on the process of implementing your organization/agency's SAMHSA CARA grant program.
|
From page 197... ...
, etc., to document and manage client needs, care processes, integration with related support services, and outcomes. - Train or develop the workforce to help state staff or community provider employees to identify mental health or substance abuse or provide effective services consistent with the purpose of the grant program.
|
From page 198... ...
a) Did your organization/agency work with an external evaluator to assist with a formal evaluation of your SAMHSA CARA grant program?
|
From page 199... ...
* Does your organization/agency systematically track how your SAMHSA CARA grant program has been impacting your community?
|
From page 200... ...
* Did your organization/agency use the SAMHSA CARA grant program or funding to leverage other supports, for example, additional city, state, national or foundational funding?
|
From page 201... ...
18) In order to help the federal government more effectively prioritize and allocate future funding to help address opioid misuse, overdose, and other substance misuse, can you describe the types or areas of future funding that would be most beneficial to your organization/agency?
|
From page 203... ...
Appendix C Evaluation Types and Data Requirements INTRODUCTION To obtain information useful for policy making through an effective evaluation requires substantial coordination, support, and data sharing among the stakeholders (e.g., the funding agency, grantees, and the evaluator)
|
From page 204... ...
Process evaluations are typically used to assess programs that have been operating for several years and are in the early- to mid-stages of development. After clearly articulating how the program is intended to work and the goals and objectives for each of its targeted populations, a process evaluation addresses the following types of questions: • How well are the core components of the program working?
|
From page 205... ...
Process evaluations can also address systematic hypotheses about process and specific implementation findings. Summative or Outcomes Evaluations Summative (or outcomes)
|
From page 206... ...
While many public programs involve the same terms, such as "naloxone distribution" or "treatment access," the programs actually implemented on the ground can differ dramatically.
|
From page 207... ...
An economic evaluation focused on the effects of providing recovery support, if nonspecific, can include a range of ways in which recovery might be supported including employment support, peer support aids, clean living assistance, and job training, while an economic evaluation focused on the impact of training and using peer support workers would have a narrower set of outcomes and costs to consider. Step 3: Clearly identify the perspective to be taken.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.