Skip to main content

Currently Skimming:

Appendix D: Estimates of Effects of Employment and Training Programs Derived from National Longitudinal Surveys and Continuous Longitudinal Manpower Survey
Pages 254-280

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 254...
... , and a special youth sample of the National Longitudinal Survey (NLS, administered by the Center for Human Resource Research of Ohio State University with data collection by the National Opinion Research Center -- NORC)
From page 255...
... limit the extent to which one can take into account the effects of local labor market conditions. Third, these data are not derived from experiments in which subjects were randomly assigned to take part in a program, and hence the resultant estimates of program effectiveness require strong assumptions about the adequacy of the model specification and the matching procedures used to construct synthetic "control" groups.
From page 256...
... Time Period for Evaluation If we assume that recall and other errors in the interview data (and processing and reporting errors in Social Security earnings records) are not too troubling, the CLMS data provide interview data for three years after a young person entered a program.
From page 257...
... The variances for estimates derived from such designs can differ considerably from Thor the CLMS analyses, further problems are posed by deficiencies in the information available from the CPS files used in matching. First, many CPS youths will have been enrolled in CETA programs themselves, but neither the CPS nor the accompanying SSA files records such participation.
From page 258...
... FINDINGS OF STUDIES USING CLMS DATA BASE Three studies compare CETA participants from the fiscal 1976 and fiscal 1977 Continuous Longitudinal Manpower Surveys with comparison groups selected from the March 1976 or March 1977 Current Population Surveys. These studies were conducted by Westat (1984)
From page 259...
... model of earnings (and accounting for "selection biased. Basically, the three studies may be distinguished in the following ways: Westat devoted substantial resources over several years to creating a comparison file with a "cell matching" and weighting technique, but ultimately used a fairly straightforward regression analysis to estimate net impacts.
From page 260...
... The Supported Work study by Mathematica provides similar evidence that the particular procedure used to select a comparison sample is less important than the net-impact estimation model (a fixed-effects estimator led to a more negative result than a linear model of postprogram earnings)
From page 261...
... For example, as a result of cell matching or nearest-neighbor matching, the CPS pool is winnowed from a largely white sample of in-school youths or high school graduates from families above the poverty level to a mixed black/white sample that includes large numbers of high school dropouts from families below the poverty line. The comparison groups also resemble CETA participants in preprogram earnings.
From page 262...
... SRI Findings SRI's analysis differs from Westat's in two key respects: in the selection of the comparison group and in its "sampling frame." SRI's comparison groups were drawn by use of a "nearest-neighbor" matching procedure based on minimizing the "distance" of CLMS participants and selected CPS matches along earnings-related variables. SRI's sampling frame differed from Westat's in the following specific ways: development of calendar year cohorts rather than fiscal year cohorts; SRI
From page 263...
... From their analyses, the SRI authors concluded that most of the differences could be attributed to choices made in the sampling frame and to an updating of 1979 SSA earnings.3 3 Net impacts were minimally sensitive to the estimation model or to the matching technique used.
From page 264...
... , et al., 1984) found, for youths: · Significant earnings losses for young men of all races and no significant impacts for young women; these impacts persist into the second postprogram year; · Significant positive net impacts for young women, particularly minorities in Public Service Employment and on-the-job training and significant negative or insignificant net impacts for all groups in work experience; · Among subgroups, the most negative findings were for white males, the most positive for minority females; · Older youths (22-year-olds)
From page 265...
... The estimates made by both studies indicate relatively modest effects of employment and training programs on the subsequent income, employment status, and educational attainment of the youths who participated in those programs. For CETA programs both studies find negative overall effects of CETA on employment, although PRG reports some positive effects at two years after CETA completion.
From page 266...
... employs a nearest neighbor matching procedures to construct a "matched" control group to be compared with the 1,114 respondents in the NLS youth sample who reported participating in CETA programs. (The matched sample was constructed by selecting respondents from among the 4,608 NLS respondents who reported that they had not participated in a CETA program and who were neither in the military nor had family incomes above 325,000 in 1978, and who did respond to the questions used to construct the eight matching variables.)
From page 267...
... In personal interview surveys, such sample clustering is a practical necessity, since any random draw of subjects in a national survey might require some interviewers to travel hundreds of miles between successive interviews. Because a sample of, say 5,000, may thus consist of 500 randomly selected "blocks," from which 10 individuals were selected, the variances of both univariate and multivariate statistics are not accurately portrayed by the well-known formulas that apply to simple random samples.
From page 268...
... of design effects for eight variables in the Current Population Survey found values of deft that ranged from 1.1 to 1.5. (The variables studied were number of persons in household, number aged 0-17, number in labor force, household income, and, for the "head of household, n income, age, sex, and educational attainment.)
From page 269...
... Frankel's (1971) findings suggest that standard errors for his eight variables would be understated by about 10 to 50 percent if SRS formulas were used (with the larger numbers applying to multiple correlation statistics, the smaller to multiple regression coefficients, and means and simple correlations resting in the middle)
From page 270...
... Family income 1978 Area unemployment rate 1979 Weeks employed 1978 High school dropout 1979 interview date Female Family size Knowledge-of-world-ofwork scale Numerical operations standard score Age at 1979 interview Does not live at home Paragraph comprehension standard score Math knowledge standard score Word knowledge standard score Arithmetic comprehension scale Constant -1640.2 (-7.54)
From page 271...
... estimate a model for the outcome variables of interest. The PRG analysis of the NLS data base differs from CEIS's in its use of a wider range of outcome measures (including earnings, employment, educational, and marital outcomes)
From page 272...
... Estimated net impacts for other outcome variables are also negative or "insignificant." (Note, however, that the t-ratios are likely to be inaccurate since the PRG analysis treated the NLS data as if they had been derived from a simple random sample of the population; see note 5.) The sole positive result shown in this analysis is for education, for which it is estimated that the net impact of CETA was to increase the probability that the youth would remain in {or return to)
From page 273...
... The two exceptions were increased use of drugs among CETA participants (net impact +7.3 percent) and increased likelihood of being married (10.2 percent)
From page 274...
... The Mathematica study used data from a true experimental design that randomly assigned loin addition to the potential bias in the matched control groups, there are two other reasons to question negative conclusions from the CLMS studies. The CPS lacks data on enrollment in CETA on the part of the comparison group and, as a result, positive net impacts may be underestimated since some of the "controls" were actually program participants.
From page 275...
... Comparison of Supported Work participants and the CPS matched sample, however, yielded either insignificant or significantly negative effects. Moreover, the bias apparent in the match sample estimates was even greater using a fixed-effects estimator rather than a basic earnings model.
From page 276...
... shows net impact estimates derived from Mathematica's analyses of Supported Work, together with estimates of overall program impact from the studies by Westat, SRI, and the Urban Institute. Mathematica acknowledges that its Supported Work sample is more severely disadvantaged and therefore more likely to have lower earnings profiles than the typical CETA youth participant.
From page 277...
... However, only 12 percent of Because of such similarities, Mathematica analysts argue that similar biases in estimates of program effectiveness may exist in the net impacts estimated by Westat, SRI, and the Urban Institute, and they conclude that "It is not possible to generate reliable net program impact estimates using ex-post comparison group procedures."
From page 278...
... For the latter problem youths presents a particularly difficult case for any match strategy because preprogram earnings data are either not extant or not reliable indicators of the uncontrolled variables that are of interest to program evaluators. 2 Estimates of the magnitude and direction of the bias in matched-group evaluations are only available for the one youth program (Supported Work)
From page 279...
... Spencer 1971 National Longitudinal Survey of Labor Force Behavior, Youth Survey: Technical Sampling Report. Chicago, Ill.: National Opinion Research Center.
From page 280...
... 1982 Continuous Longitudinal Manpower Survey. Net Impact Report No.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.