Youth Employment and Training Programs The YEDPA Years (1985) / Chapter Skim
Currently Skimming:
Appendix C: Implications of the Youth Employment Experience for Improving Applied Research and Evaluation Policy
Appendix C: Implications of the Youth Employment Experience for Improving Applied Research and Evaluation Policy
Pages 231-253
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 231... ...
The purpose of this appendix is to capitalize on hard experience in making that judgment in one arena -- employment and training programs supported by the federal government. The program evaluations reviewed by the Committee on Youth Employment Programs have a variety of implications for evaluation policy.
|
From page 232... ...
review of 200 evaluations issued by the Office of Economic Opportunity before 1968, for example, uncovered no randomized field experiments and only about 25 reports with credible evidence. Professional and institutional guidelines for improving the quality of evaluation designs are readily available.
|
From page 233... ...
How precedent, pilot tests, ethics, and law constrain or enhance feasibility is considered briefly below. Precedent Some opponents of controlled randomized tests maintain that randomized experiments are rarely feasible in field settings.
|
From page 234... ...
For this reason, pilot tests of large-scale field experiments are worth considering. That is, small experiments prior to the main field experiment can provide evidence on feasibility that is more direct than what precedent can offer, can identify problems that otherwise could not be anticipated, and can help to resolve predictable problems before the main effort.
|
From page 235... ...
The presumption has been wrong at times in medical research, e.g., experiments in day care for the chronically ill during the late 1970s. It has also been wrong in educational research, notably in attempts to do randomized field experiments on Head Start preschool programs and in planned variations.
|
From page 236... ...
Nor is there any reference to shortfall in the reports by the Vera Institute on the Alternative Youth Employment Strategies Project. The report on the Opportunities Industrialization Center (OIC)
|
From page 237... ...
Tallmadge-Yuen's Career Intern report suggests that there were, at most, 75 subjects per cohort per site; there is no reference to a power analysis in the final report, and results are mixed. Measures of Program Implementation More orderly, verifiable information on the degree of program implementation needs to be collected.
|
From page 238... ...
Drug trials and other randomized clinical trials in medicine must often accommodate departures from protocol and noncompliance (e.g., Silverman, 1977~.
|
From page 239... ...
experiments. The strategy has been exploited in a few youth employment program evaluations, in research which preceded development of Sesame Street, and in experiments on surgical and health innovation.
|
From page 240... ...
Coupling Experiments to Longitudinal and Panel Studies Randomized experiments ought to be coupled routinely to longitudinal surveys and panel studies. The purposes of this "satellite" policy include calibration of nonrandomized experiments, more generalizable randomized experiments, and better methods for estimating program effects.
|
From page 241... ...
The most dramatic recent empirical evidence stems from Fraker and Maynard's (1985) comparisons of program effects based on randomized experiments and effects based on nonrandomized data, notably the CLMS and the Current Population Survey.
|
From page 242... ...
And applications of well-explored theory, erg., classical mechanics and the conflation of several complex theories or laws operating in a complex environment, may lead to analytic problems that are intractable. And so, it is not uncommon to design randomized experiments to assess changes in chemical production processes, acoustics, and other areas (e.g., Hahn, 1984~.
|
From page 243... ...
So, for example, the best reports tell us what the attrition rate is from programs or from program versus control groups. But many reports do not.
|
From page 244... ...
Attrition Rates The difference between the ultimate target samples of program participants and control-group members is crucial. Estimates of program effect may be inflated, deflated, or remain unchanged, relative to their true value, depending on the magnitude of attrition in the groups.
|
From page 245... ...
Site Selection Many of the field tests of youth employment programs involve multiple sites. The 9 randomized trials reviewed seriously by the committee, for example, involve a randomized experiment in each of 40 sites.
|
From page 246... ...
Boruch, R.F. 1975 Coupling randomized experiments and approximations to experiments in social program evaluation.
|
From page 247... ...
1980 Advanced Education and Training -- Interim Report on the Career Advancement Voucher Demonstration. Youth Knowledge Development Report No.
|
From page 248... ...
15. Federal Judicial Center 1983 Social Experimentation and the Law.
|
From page 249... ...
Lind 1983 A Reevaluation of the Civil Appeals Management Plan. Washington, D.C.: Federal Judicial Center.
|
From page 250... ...
Beverly Hills, Calif.: Sage Publications. B IBLIOGRAPHY Guidelines, Standards, and Related Papers on Appraising the Quality of Social Program and Project Evaluations The guidelines issued by the Evaluation Research Society, the U.S.
|
From page 251... ...
Illustrative, Recent Randomized Field Experiments in Law Enforcement and Corrections, Welfare, Court Procedures, Health Services, and Other Areas For a list of over 300 field experiments run before 1979, see Boruch, McSweeny, Soderstrom (1978~. Law Enforcement and Corrections Berk, R.A., and L.W.
|
From page 252... ...
Department of Agriculture. Randomized Experiments, Planned and Executed in Varying Degrees, on Youth Employment Programs Clark, Phipps, Clark & Harris, Inc.
|
From page 253... ...
1983. Alternative Youth Employment Strategies Project.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.