Fairness in Employment Testing Validity Generalization, Minority Issues, and the General Aptitude Test Battery (1989) / Chapter Skim
Currently Skimming:
5 Problematic Features of the GATB: Test Administration, Speedness, and Coachability
5 Problematic Features of the GATB: Test Administration, Speedness, and Coachability
Pages 99-118
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 99... ...
referral system. The difficulties we see range from easily cured problems with the current test administration procedures to some fundamental design features that must be revised if the General Aptitude Test Battery is to take on the ambitious role envisioned in the VG-GATB Referral System.
|
From page 100... ...
and the contents of the GATB subtests has raised additional concerns about the vulnerability of the test battery to guessing. Consider Subtest 1, name comparison, a speeded test of clerical perception.
|
From page 101... ...
Our analysis of individual item functioning demonstrates the potential effects of guessing in increasing GATB subtest scores. Table 5-1 presents a worksheet showing the score increase that could be expected for each of the would-be power tests, i.e., those where speed of work does not seem to be a defensible part of the construct (Subtests 2, 3, 4, and 61.
|
From page 102... ...
, this deficiency will affect the test scores of examinees who follow the instruction most closely. For some subtests, such as the name comparison test, the design of the NCS answer sheet might add a significant psychomotor component to the abilities required to perform well.
|
From page 103... ...
Commonly used aptitude tests rarely, if ever, fit the definition of a pure power test or a pure speed test. Many such tests, including the subtests of the GATB, combine elements of speed of work and quality of work to a largely unknown degree.
|
From page 104... ...
~ \ ° ~ \ \ \ o \ a\ \ ~ .~d o\ \ o\ · \\ a\ ~ \ .\o\ \`o\ .. ~ o\ · ~ White Attempted Black Attempted O White Correct · Black Correct ·~ I 40 49 FIGURE 5-1 Percentages attempting and number of items correct for whites and blacks on Subtest 5, tool matching (speeded)
|
From page 105... ...
o O\ \\\\ \ \ · \ \ \ \ 20 _ 10 _ O I 1 1 ~I I 1 1 1 1 . ~ · ° At\ White Attempted Black Attempted O White Correct · Black Correct I I ~ I t ~ 1 2 3 4 5 6 7 8 910 111213141516171819202122232425 NUMBER OF ITEMS FIGURE 5-2 Percentages attempting and number of items correct for whites and blacks on Subtest 6, arithmetic reasoning (power test)
|
From page 106... ...
Since the subtest is tightly timed, identification of the correct answer bubble from the relatively long list presented on the answer sheet might become a significant component of the skill assessed. One could, by inspection, confidently advance the argument that the subtest measures not only form perception, but also the speed of list processing and skill in decoding complex answer sheet formats.
|
From page 107... ...
Bias is defined as differential validity whereby individuals of equal ability but from different groups have different success rates on test items. To establish that individuals have equal ability, the various item-bias methods rely on total test score (or some transformation of total score)
|
From page 108... ...
Thus these subtests are more than twice as long for the given time limits than is appropriate for power tests. The committee also conducted item-bias analyses using the MantelHaenszel procedure, whereby majority and minority examinees are matched on total score before examining differential performance on individual test items.
|
From page 109... ...
Earlier in the chapter we hypothesized possible strategies of random response to improve test scores. How test-wise are GATE test takers about the advantage of marking uncompleted items when time runs out?
|
From page 110... ...
PRACTICE EFFECTS AND COACHING EFFECTS Because of the speededness of the GATB, the test is very vulnerable to practice effects and coaching. If the test comes to be widely used for referral, USES policy makers must be prepared for the growth of coaching schools of the kind that now provide coaching for the Scholastic Aptitude Test and tests for admission to professional schools.
|
From page 111... ...
Third Quartile Median Quartile FIGURE 5-3 Practice effects when the same test form was used both times. Distributions of estimated effect sizes (initial testing to retesting)
|
From page 112... ...
are expressed in standard deviations of initial aptitude distributions.
|
From page 113... ...
and M A group of 40 instructed subjects showed average gains in mean scores, expressed in units of estimated effect sizes, of 0.94 for K, 0.43 for F
|
From page 114... ...
Although the generalizability of these results to nonretarded examiners is questionable, the potential coachability of the GATB subtests that compose the psychomotor aptitudes is clearly indicated. TEST SECURITY If the Department of Labor decides to continue and expand the VG-GATB Referral System, USES will have to develop new test security procedures like those that surround the Scholastic Aptitude Test, the American College Testing Program, the Armed Services Vocational Aptitude Battery, and other major testing programs.
|
From page 115... ...
In contrast, the major college testing programs develop new forms annually, and the Department of Defense develops three new forms of the Armed Services Vocational Aptitude Battery at about four-year intervals. In addition, test security has not been a primary concern so long as the GATB was used largely as a counseling tool; it appears to be fairly easy for anyone to become a certified GATB user and obtain access to a copy of the test battery.
|
From page 116... ...
The GATB's speededness, its consequent susceptibility to practice effects and coaching, the small number of alternate forms, and low test security in combination present a substantial obstacle to a broad expansion of the VG-GATB Referral System. RECOMMENDATIONS Test Security If the GATB is to be used in a widespread, nationwide testing program, we recommend the adoption of formal test security procedures.
|
From page 117... ...
It looks at the possibility that correlations of GATB scores with on-thejob performance measures differ by racial or ethnic group or gender, and the possibility that predictions of criterion performance from GATB scores differ by group.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.