Fairness in Employment Testing Validity Generalization, Minority Issues, and the General Aptitude Test Battery (1989) / Chapter Skim
Currently Skimming:
9 Differential Validity and Differential Prediction
9 Differential Validity and Differential Prediction
Pages 172-190
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 172... ...
We are not concerned here with setting. Our investigation is limited to the possibility that the GATB functions differently for different population groups, and specifically that correlations of GATB scores with on-thejob criterion measures may differ by racial or ethnic group or gender, or that predictions of criterion performance from GATB scores may differ for employees on a given job who are of different racial or ethnic status or gender.
|
From page 173... ...
Conversely, it is possible for two groups to have the same prediction equations and the same variability of actual criterion scores about their predicted values, and yet have different validity coefficients. Prediction equations are usually based on a linear regression model and are influenced by means and standard deviations of the test and criterion measure as well as the correlation.
|
From page 174... ...
GROUP DIFFERENCES IN CORRELATIONS David J Synk and David Swarthout compared the validity coefficients obtained for black and for nonminority employees in 113 Specific Aptitude Test Battery validation studies conducted since 1972 for which there were at least 25 people in each of the two groups (U.S.
|
From page 175... ...
Department of Labor made data available to the committee that we used to compute correlations between the job-specific GATB composite and criterion measures. These correlations were computed separately for each job with at least 50 black and 50 nonminority employees with GATB scores and scores on the criterion measure.
|
From page 176... ...
However, the results raise serious questions about the degree of validity of the job family composites for blacks, especially in Job Families IV and V for which the results are based on a sizable number of studies and large samples of black employees. Not only are the average
|
From page 177... ...
It might be noted, however, that the average validities reported here are higher for men and women TABLE 9-3 Weighted Average Job Family Correlations for Male and Female Employees Men Women Job Number of Average Number of Average Family Studies N Validity Studies N Validity IV 51 8,793 .24 37 7,101 .25 V 23 2,365 .20 41 6,262 .22 SOURCE: U.S. Department of Labor.
|
From page 178... ...
Since the available reports comparing validities do not provide direct evidence regarding the possibility of differential prediction, the committee conducted analyses for this report. Data for these analyses were provided by the Northern Test Development Field Center of the U.S.
|
From page 179... ...
Only individuals identified as either black or nonminority were included in the analyses. For each of the 72 Specific Aptitude Test Battery validation studies in the data file that had data for 50 or more black and 50 or more nonminority individuals, the following statistics were computed separately for each group and for the total combined group: the mean and standard deviation of the job family composite test score and criterion measure, the correlation between the composite test score and the criterion measure, the slope and intercept of the regression of the criterion measure on the composite test score, and the standard error of prediction.
|
From page 180... ...
A positive l-ratio indicates that the slope for nonminor~ty employees is greater than the slope for black employees, albeit not necessarily significantly greater. Conversely, a negative l-ratio indicates that the slope for black employees is greater than that for nonminority employees.
|
From page 181... ...
To get a sense of the magnitude of the difference in intercepts, the intercept for black employees was subtracted from the intercept for nonminority employees and the difference was divided by the standard deviation of the criterion scores based on the sample of black employees. The latter step was taken, in part, to account for differences in the criterion scale from one study to another and, in part, to express the difference in a metric that is defined by the spread of the scores for one of the groups.
|
From page 182... ...
studies are included in the distribution, even though 2 of the studies had significant differences in the slopes, suggesting that a pooled, withingroup slope is not entirely appropriate.) As the table shows, the difference is positive more often than it is negative, with a median value roughly equal to one-quarter of the standard deviation of criterion scores for black employees.
|
From page 183... ...
More specifically, at each of three score values on the GATB job family composite, two scores were obtained: the predicted criterion score based on the total-group equation and the predicted criterion score based on the equation for black employees only. The latter predicted value was subtracted from the former and, as before, the difference was divided by the standard deviation of the criterion scores for black employees to take into account between-study differences in the metric of the criterion measure.
|
From page 184... ...
The latter difference is a consequence of the total-group slope typically being slightly greater than the TABLE 9-8 Stem-and-Leaf Chart of Standardized Difference in Predicted Criterion Scores Based on the Total-Group and Black-Only Regression Equations: GATB Composite Score = Black Mean Plus One Standard Deviation Stem Leaf Count .6 133 3 ·5 011 3 .4 2567 4 .3 01235588 8 .2 00123456788999 14 .1 00244567788889 14 .0 1122233345677 13 -.0 0222445667 10 -.1 27 2 -.2 0 -.3 4 1 NOTE: Median = .18.
|
From page 185... ...
Given the low correlation and the substantial difference in mean scores of blacks and whites on the GATE, use of the test for selection of black applicants without taking the applicant's race into account would yield very modest gains in average criterion scores but would have substantial adverse impact. It is within this context that the differential prediction results need to be evaluated.
|
From page 186... ...
The second paper (Ford et al., 1986) represents a preliminary attempt to address the issue of the extent to which race differences in assessments of job performance are the product of meaningful performance differences or the product of rater bias.
|
From page 187... ...
In comparison, the effect sizes for direct performance were .16 for objective measures and .22 for subjective measures. Note that although race differences are smaller when measures of direct performance are used than when cognitive performance measures are used, all measures of on-thejob performance produce much smaller differences in scores between blacks and whites than do predictor tests such as the GATB, on which blacks are typically found to score about one standard deviation below whites.
|
From page 188... ...
The average correlation of the GATB composite with the criterion measure was .12 for black employees and .19 for nonminority employees. Moreover, for a quarter of the studies, the correlation for blacks is .03 or less.
|
From page 189... ...
Employment Service for its local office operations; discusses the claims that have been made for the system; and assesses the evidence available on its implementation from a small number of pilot studies. Chapter 11 discusses the likely effects of widespread adoption of the VG-GATB system on the specific groups involved: employers, job seekers, in particular minority job seekers, people with handicapping conditions, and veterans.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.