Fairness in Employment Testing Validity Generalization, Minority Issues, and the General Aptitude Test Battery (1989) / Chapter Skim
Currently Skimming:
8 GATB Validities
8 GATB Validities
Pages 149-171
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 149... ...
. As part of its study of validity generalization for the GATB, the committee has conducted independent analyses of the existing GATB validity studies.
|
From page 150... ...
It included these and an additional set of 264 studies carried out in the 1970s and 1980s. (The Hunter studies appear as 491 studies in this data tape, because some pairs of studies in the original 515 consisted of validity coefficients for the same set of workers using two different criteria for job performance; these pairs each appear in a single study on the data tape.)
|
From page 151... ...
. TABLE 8-1 Mean and Standard Deviation of the Validity Coefficients, Weighted by Study Sample Sizes, Computed for Each Composite Across the 264 Studies, and Compared with Those Hunter Reported for the Original 515 Studies Scatter diagram to 0oooo oC' x 0 0 0 0 oOo° x of 0 x 0 0° 0° 0 ~o° x °°oOO x GVN SPQ KFM 515 264 515 264 515 264 Mean .25 .21 .25 .17 .25 .13 Standard deviation .15 .11 .15 .11 .17 .12
|
From page 152... ...
The values presented are the percentage of studies falling into each validity category. Clearly, the range of observed validity coefficients is large.
|
From page 153... ...
Sample size Sampling error appears to be the single factor with the largest influence on variance in validity from study to study: removing the influence of sampling error is a major component of any validity generalization analysis. To get an intuitive feel for the effects of sampling error, GVN validities were examined separately for the entire sample, for samples with more than 100 subjects, for samples with more than 200 subjects, and for samples with more than 300 subjects.
|
From page 154... ...
.22 .18 .24 .18 .30 .16 Job Family In both the original 515 studies and the recent 264 studies, validity clearly varies across job families. The mean observed validities (for both the data used by Hunter and the full data set)
|
From page 155... ...
ob Performance Versus Training Success It has frequently been reported in the personnel testing literature that higher validity coefficients are obtained for ability tests when training success rather than job performance is used as the criterion. This makes conceptual sense, as there are probably fewer external factors influencing training success than job performance (e.g., job performance typically covers a longer time period and is probably more heavily influenced by supervision, work-group norms, variation in equipment, family problems, and so on)
|
From page 156... ...
tends to be somewhat lower for older workers, though not enough to require special consideration in validity generalization analysis. This finding does not seem to hold for SPQ and KFM.
|
From page 157... ...
. Breaking studies down by job family and criterion type (performance criterion versus training criterion, the two important moderator variables identified in the earlier analyses)
|
From page 158... ...
GVN Performance Training Male Female Male Female Job Number of Number of Number of Number of Family Mean Studies Mean Studies Mean Studies Mean Studies I .28 21 .49 1 II .23 1 .13 16 III .36 12 .37 2 .32 7 .57 1 IV .24 98 .23 31 .34 38 .38 14 V .21 46 .20 118 .35 2 .37 2 SPQ I .29 21 .40 1 II .22 1 .15 16 III .24 12 .25 2 .19 7 .50 1 IV .23 98 .22 31 .29 38 .29 14 V .25 46 .23 118 .45 2 .40 2 KFM I .18 21 - .03 1 II .29 1 .34 16 - ~ III .13 12 .19 2 .05 7 .48 1 IV .19 98 .23 31 .21 38 .22 14 V .25 46 .30 118 .32 2 .46 2 categories in which comparisons can be made (Job Families IV and V with a performance criterion and Job Family IV with a training criterion) , the results suggest no effect due to sex.
|
From page 159... ...
Thus partial correlations were computed controlling for criterion type (job performance versus training success) and for study type (predictive versus concurrent)
|
From page 160... ...
First, the sample sizes for Families II and III for the job performance criteria are small; thus we focus solely on Families I, IV, and V Second, only for Job Family IV is the sample size adequate to put any confidence in the findings using the training criterion.
|
From page 161... ...
. With the training criterion, only Family IV has a large sample size; GVN validity actually is slightly higher for the new studies, and the drop in SPQ and KFM in validities is smaller than for the performance criteron for the same family.
|
From page 162... ...
Both of these factors have been found to moderate GATB validity. However, since analyses reported here present results within job families and within criterion types, this explanation has been ruled out.
|
From page 163... ...
The chapter ends with our conclusions about the most appropriate estimates of the true validity of the GATB for Employment Service jobs. Correcting Only for Sampling Error In this analysis, variance expected due to sampling error is computed: the variance is a function of the mean observed validity and the mean
|
From page 164... ...
This corrected SD is found by subtracting variance expected due to sampling error from observed variance: this gives a corrected variance, the square root of which is the corrected standard deviation. Thus, within each job family, the mean observed validity estimates the average true validity of the population of jobs in the family, and, provided the population validities are normally distributed, 90 percent of validities can be expected to fall above the point defined by multiplying 1.28 times the corrected standard deviation (1.28 SD units below the mean is the 10th percentile of a normal distribution)
|
From page 165... ...
Thus correcting only for sampling error, one finds evidence of modest validity for the GATB for all job families. Correcting for Criterion Unreliability Ideally, a good reliability estimate would be available for each study, in which case each validity coefficient could be corrected for unreliability.
|
From page 166... ...
Correcting for Range Restriction If the test standard deviation is smaller in the study sample than in the applicant pool, then the validity coefficient for workers will be reduced due to range restriction and will be an underestimate of the true validity of the test for applicants. If the standard deviation for the applicant pool is known, the ratio of study SD to applicant SD is a measure of the degree of range restriction, and the validity coefficient can be corrected to produce the value that would result if the full applicant population had been represented in the study.
|
From page 167... ...
Empirical data on test SDs in applicant pools for a variety of jobs filled through the Employment Service are needed to assess whether Hunter's analysis overcorrects for range restriction. In the absence of applicant pool data, the conservative correction for restriction of range would be simply to apply no correction at all.
|
From page 168... ...
The 264 more recent studies simply produce different empirical findings that is, lower validities than the earlier 515. Optimal Predictors Based on the Recent 264 Studies The corrected correlations in Table 8-14 may be used to develop composite predictors of job performance in the different job families based on the recent 264 studies.
|
From page 169... ...
There are notable differences in the results of GATB validity studies conducted prior to 1972 and the later studies. The mean observed correlation between supervisor ratings and cognitive or perceptual aptitude scores declines to .19, and between supervisor ratings and psychomotor aptitude scores declines to .13.
|
From page 170... ...
This assumption causes the observed correlations to be adjusted upward by 25 percent for small correlations and by 35 percent for observed validities of .50. Restriction-of-range estimates should be based on data from applicants for homogeneous clusters of jobs.
|
From page 171... ...
The concept of validity generalization does not obviate the need for continuing validity studies for different jobs and for the same job at different times.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.