Community Programs to Promote Youth Development (2002) / Chapter Skim
Currently Skimming:
6 Lessons from Experimental Evaluations
6 Lessons from Experimental Evaluations
Pages 147-196
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 147... ...
First, we were interested in whether there is evidence from the most rigorous evaluations available that community programs for youth make a difference in the lives of their participants. Second, we wanted to know whether the programs that had received these types of rigorous evaluations included either the program features outlined in Chapter 4 as potentially important components of community programs for youth or the personal and social assets outlined in Chapter 3 as potentially important targets for these pro147
|
From page 148... ...
As noted, the primary objective of reviewing these meta-analyses was to provide our best assessment of the extent to which community programs have beneficial effects for youth. Our secondary objective was to determine the extent to which these programs incorporate the features of positive developmental settings developed in Part I and whether there is evidence of links between these program features and positive youth outcomes.
|
From page 149... ...
(Primary prevention programs intervene with normal populations to prevent problems from developing. Secondary prevention programs target individuals already at risk or exhibiting problems)
|
From page 150... ...
150 - o o ¢ o Cal Cat o so .= 1 ~4 ¢ U
|
From page 151... ...
151 _ _ t<, ~ o ·— o ~ C - O Al ~ 5 5 ~ ~¢ 5 ~ ~ O sol ~ 0 m^ ~ ~ ~ 0 ~ cd ~ so be · - ·,- by L)
|
From page 155... ...
155 so t~ ~ ~° at e ~ ~ O e ~ ~= an ~ =~ 0 :~= =.=u ·~^ a ~ 0 ~~ to ~= a u ~ ~ ~ ~ ~ E ' E o U
|
From page 161... ...
161 be ~ a ~ ~ c E x E E G E .
|
From page 162... ...
162 .~ o 1 ~4 ¢ u, be .> ~ u: u, _ 0 ~ 0 ~ 0 u, u, u, ~ ¢ ~ _ 0 cot · 0 u: u, o o .
|
From page 163... ...
be .= - ·~ a^ in,, a Id =~ , ^ = O So O ~ So .
|
From page 165... ...
165 a: sit .^ be .
|
From page 166... ...
Environmentcentered programs tried to change either the home setting (through parent education about child development and changing parental attitudes and childrearing techniques when appropriate) or the school setting (often through teacher training in interactive instructional techniques and classroom management skills)
|
From page 167... ...
The indicated prevention programs targeted individuals identified as having early warning signs of mental disorders but not yet meeting diagnostic criteria. Out of 130 total programs, researchers selected 34 using the following criteria: a randomized experimental design or quasi-experimental design with a comparison group; pretest and posttest measures; a written manual specifying the theory and procedures used in the program intervention; a clearly defined sample, with adequate information about their behavior and social characteristics; and some evidence of positive mental health outcomes.
|
From page 168... ...
4. Prevention programs targeting multiple domains (i.e., individual, school, and community)
|
From page 169... ...
Only the Penn Prevention Project found unambiguous evidence of both follow-up and sleeper effects. This selected intervention program targeted children ages 10 to 13 at risk due to elevated depressive symptoms or family conflict.
|
From page 170... ...
The positive effects of this program were maintained at least through the 20-month follow-up assessment. The other two programs, Big Brothers Big Sisters and the School Transition Environmental Project, were 9 to 12 months in length.
|
From page 171... ...
So, do the programs targeting adolescents, warrant the same conclusions reached by Greenberg and colleagues for the full array of programs included in the review? First, do multiyear preventive programs produce longer-term effects than short-term program interventions?
|
From page 172... ...
Violence Prevention Programs The Center for the Study and Prevention of Violence at the University of Colorado at Boulder published a series of Blueprints (Center for the Study and Prevention of Violence, 2000) describing program interventions effective in preventing violence.
|
From page 173... ...
effect on delinquency, drug use, and/or violent behavior; at least one additional site replication with experimental design and demonstrated effects; and evidence that the deterrent effect was sustained for at least one year following treatment. Of the 10 model programs, 6 included youth ages 10 to 18 in their target population: Big Brothers Big Sisters, Quantum Opportunities, the Midwestern Prevention Project, Life Skills Training, Multisystemic Therapy, and the Bullying Prevention Program.
|
From page 174... ...
reviewed evaluations of primary prevention programs designed to reduce sexual risk-taking and teen pregnancy. These programs met the following criteria: published in 1980 or later; experimental or quasi-experimental in design; a minimum sample size of 100 in combined experimental and control groups; targeted 12- to 1 8-year-olds; conducted in the United States or Canada; and measures of program impact on sexual or contraceptive behavior or pregnancy or birth rates.
|
From page 175... ...
In summary, the strongest evidence for program effectiveness using a youth development framework comes from the Teen Outreach Program. Why was this service-learning program effective while the others were not?
|
From page 176... ...
In general, these characteristics and strategies of effective positive youth development programs closely match the personal and social assets and features of positive developmental settings identified in Part I as important for healthy adolescent development. Catalano et al.
|
From page 177... ...
A focus on social or cognitive behavioral skills occurred in 96 percent of the programs; decision-making and self-management skills in 73 percent; coping skills in 62 percent; and refusal-resistance skills in 50 percent. Effective environmental or organizational change programs focused on influencing teacher classroom practices and peer social norms.
|
From page 178... ...
; and (c) prevention programs focused on resistance skills.
|
From page 179... ...
Adventure programs also typically include features of positive developmental settings outlined in Part I Common features of adventure programs include wilderness or backcountry settings, groups of less than 16, mentally and physically challenging experiences, intense social interactions often relating to group problem solving and decision making, trained nondirective leaders, and an average duration of two to four weeks.
|
From page 180... ...
But it might also suggest a scarcity of well-validated measures of personal and social assets and features of positive developmental settings (see Chapter 8 for more discussion of this issue)
|
From page 181... ...
That is, they do not have: an analysis of program theory, quality information about implementation, quality information about effectiveness, quality analysis of why effects did or did not come about, an analysis of the replicability of the program, or an analysis of how important the program is and why. THREE MODEL PROGRAM EVALUATIONS Three programs the Teen Outreach Program, Quantum Opportunities, and Big Brothers Big Sisters stand out among the programs included in the meta-analyses and reviews just summarized for several reasons.
|
From page 182... ...
The program was designed to help adolescents understand and evaluate their future life options and develop life skills and autonomy in a setting that creates strong social ties to adult mentors. To accomplish these goals, three components were designed: supervised community service, classroom-based discussions of service experiences, and classroom-based discussion and activities related to social-developmental tasks of adolescence.
|
From page 183... ...
Strengths and Limitations Some aspects of the evaluation of Teen Outreach Program are exemplary. Initial nonexperimental evaluations of the program (Allen et al., 1990, 1994)
|
From page 184... ...
at 5 sites 8 8 of the 100 program participants; 82 of 100 control participants were interviewed in the follow-up evaluation. $10,600/student for 4 years; $2,650/ student/year Sample: Attrition analysis: Cost: The Quantum Opportunities Program was designed to serve very poor adolescents living in high-risk neighborhoods by providing educa
|
From page 185... ...
It is a community-based, year-round, multiyear, and multilevel youth development program. Each participant was eligible to receive the following experiences each year: · 250 hours of education activities, such as participating in computer-assisted instruction, peer tutoring, etc., with the goal of enhancing basic academic skills; · 250 hours of developmental activities, including participating in cultural enrichment and personal development, acquiring life/ family skills, planning for college or advanced technical/vocational training, and job preparation; and · 250 hours of service activities, such as participating in community service projects, helping with public events, and working as a vol.
|
From page 186... ...
Each of these program design components is compatible with the committee's list of features of positive developmental settings. The multiple components of the program produced multiple positive effects using a strong pre- and postexperimental design.
|
From page 187... ...
Evocation Design and Results An experimental design using random assignment was used to evaluate the Big Brothers Big Sisters program at eight program sites: Phoenix
|
From page 188... ...
Finally they also reported better parental relationships and more parental trust. Strengths and Limitations The Big Brothers Big Sisters program is a very promising program based on the multiple benefits to youth of being mentored by nonfamilial adults.
|
From page 189... ...
Second, outcome effects were reported using relative rather than absolute percentages, thereby possibly inflating effect sizes. In summary, Big Brothers Big Sisters is a very promising community program for youth.
|
From page 190... ...
In addition, many of the programs officially designated mentoring as the central component of the program. The best example of this is Big Brothers Big Sisters.
|
From page 191... ...
Positive Social Norms Even when not explicitly stated as a program goal, all of these programs promoted positive social norms and discouraged norms related to the major problem behaviors of concern in these programs. Programs with a community service component Quantum Opportunities and the Teen Outreach Program best exemplify this feature.
|
From page 192... ...
programs, Quantum Opportunities and the Teen Outreach Program, were successful in involving high school students in community service, life and family skills training, and support in planning for college and jobs. Opportunities for Skill Building Opportunities for skill building were plentiful in the programs reviewed.
|
From page 193... ...
Parents were involved in school activities, and students were exposed to role models in the community through presentations and field trips. Structure and Safety These two characteristics of positive developmental settings from our list appropriate structure and physical and psychological safety received little mention in the program descriptions.
|
From page 194... ...
Community programs for youth, whether they are packaged in teen pregnancy prevention programs, mental health programs, or youth development programs, can facilitate positive outcomes for youth. This review of experimental and quasi-experimental evaluation of community programs for youth revealed that participation is associated with increases in such outcomes as motivation, academic performance, self-esteem, problem-solving abilities, positive health decisions, interpersonal skills, and parent-child relationships, as well as decreases in alcohol and tobacco use, depressive symptoms, weapon-carrying, and violent behavior.
|
From page 195... ...
If such programs are designed from the beginning with a well-articulated theory of change grounded in solid developmental theory and research, we will learn a lot more about what programs might do to facilitate positive youth development and to prevent problems from emerging,
|
From page 196... ...
We turn now to exploring evaluation methodologies in more detail, looking specifically at the role for evaluation in Chapter 7 and data collection in Chapter 8 for the future of these programs. There are particular challenges inherent to evaluating community programs for youth.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.