Improving Evaluation of Anticrime Programs (2005) / Chapter Skim
Currently Skimming:
3 When Is an Impact Evaluation Appropriate?
3 When Is an Impact Evaluation Appropriate?
Pages 22-33
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 22... ...
A valid and informative impact evaluation, however, cannot necessarily be conducted for every criminal justice program whose effects are of interest to policy makers. Impact evaluation is inherently difficult and depends upon specialized research designs, data collections, and statistical analysis (discussed in more detail in the next chapter)
|
From page 23... ...
It follows that adequate identification of programs that may be significant enough to any one of these groups to be candidates for impact evaluation will require input from informed representatives of that group. Sponsors of evaluation research across the spectrum of criminal justice programs will need input from all these groups if they wish to identify the candidates for impact evaluation likely to be most significant for the field.
|
From page 24... ...
It is the job of impact evaluation to determine effectiveness, which makes this a difficult criterion to apply when selecting programs for impact evaluation. Nonetheless, an informed judgment call about the potential effectiveness of a program can be important for setting evaluation priorities.
|
From page 25... ...
Because it is a difficult task, expert criminal justice professionals, policy makers, and researchers should be employed to review candidate programs, discuss their significance for impact evaluation, and make recommendations about the corresponding priorities. EVALUABILITY OF THE PROGRAM A criminal justice program that is significant in terms of the criteria described above may, nonetheless, be inappropriate for impact evaluation.
|
From page 26... ...
It is commonplace for many medical and mental health programs to develop treatment protocols -- manuals that describe what the treatment is and how it is to be delivered -- but this is not generally the case for criminal justice programs. In such instances, the evaluation research may need to include an observational and descriptive component to characterize the nature of the program under consideration.
|
From page 27... ...
Outcome Data Impact evaluation requires data describing key outcomes, whether drawn from existing sources or collected as part of the evaluation. The most important outcome data are those that relate to the most policyrelevant outcomes, e.g., crime reduction.
|
From page 28... ...
Similar considerations apply when the outcome data are collected from existing records or data archives. Many of the data sets used to study criminal justice policy are not probability samples from the particular populations at which the policy may be aimed (see NRC, 2001)
|
From page 29... ...
Data on individuals' criminal offense records that are kept in various local or regional archives, for instance, are usually not accessible to researchers without a court order or analogous legal authorization. If the relevant authorities are unwilling to provide that authorization, those records become unavailable as a source of outcome data.
|
From page 30... ...
Resources The ability to conduct an adequate impact evaluation of a criminal justice program will clearly depend on the availability of resources. Relevant resources include direct funding as a major component, but also encompass a range of nonmonetary considerations.
|
From page 31... ...
With a set of programs judged significant identified, assessment of how amenable they are to sound impact evaluation research is a different matter. The expertise relevant to this judgment resides mainly with evaluation researchers who have extensive field experience conducting impact evaluations of criminal justice programs.
|
From page 32... ...
· Does the proposal/project director describe key project elements? · Do they describe how the project's primary activities contribute to goals?
|
From page 33... ...
· What would the target population receive in a comparison sample? · What are the shortcomings/gaps in delivering the intervention?
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.