Performance Assessment for the Workplace Volume I (1991) / Chapter Skim
Currently Skimming:
3. Improving Job Performance Criteria for Selection Tests
3. Improving Job Performance Criteria for Selection Tests
Pages 56-72
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 56... ...
As we recounted in the Overview, the uncertainties introduced by the misnorming of the Armed Services Vocational Aptitude Battery (ASVAB) and the general dismay in Congress and the Department of Defense when it became clear that the resulting inflation of scores meant that, not 20 percent, but closer to 50 percent of those enlisted between 1976 and 1980 were in Category IV (percentile score range 10 to 30)
|
From page 57... ...
And finally, success in technical training requires academic skills that may not be of great importance in the workplace. To the extent that training departs from actual job requirements, either in what it omits or in the additional skills or characteristics it demands, its value for validating selection and classification procedures is lessened.
|
From page 58... ...
Each task in turn can be broken down into a series of steps to be executed, so that job performance consists of a prescribed set of observable acts or behaviors. This definition of a job corresponds with what the Army and Marine Corps call a military occupational specialty (MOS)
|
From page 59... ...
The primary indicator of job performance is individual proficiency on a set of tasks specific to a job tasks that elicit "manifest, observable job behaviors" that can be scored dichotomously (go/no go, pass/fail) as a prescribed series of steps (internal memorandum, October 10, 19834.
|
From page 60... ...
Benchmarks and Surrogates The hands-on test is an assessment procedure honored more in the breach Despite the inherent attractiveness of assessing actual performance in a controlled setting, the enormous developmental expense and the logistical difficulties in administering hands-on tests have meant that the methodology has been largely unrealized until now. The performance rating, despite its many and well-documented frailties, continues to be, by far, the most common criterion measure used to validate employment tests (Landy and Parr, 19831.
|
From page 61... ...
From this vantage point, the crucial question with regard to enlistment standards is not so much whether the ASVAB is a valid selection instrument, but how high the standards need to be to ensure that service personnel can do their assigned jobs competently. The hope was that the job performance data gathered in Phase I of the project would permit development in Phase II of models for setting quality standards that would enable DoD to estimate the minimum cost required to achieve alternative levels of performance and to evaluate the policy trade-offs.
|
From page 62... ...
THE RESEARCH PLAN The long-term goal of military planners is to have a DoD-wide program for collecting job performance data that can be related to recruit capabilities. With the hands-on test as its anchor in reality, the JPM Project sought to develop a comprehensive set of performance measures for a sample of military jobs so that it would be possible to evaluate the relative merits of the various types of measures.
|
From page 63... ...
The jobs for which each Service agreed to develop performance measures are identified below. Army The Army selected the following nine military occupational specialties for study: Infantryman Cannon crewman Tank crewman Radio teletype operator Medical specialist Light wheel vehicle/power generation mechanic Motor transport operator .
|
From page 64... ...
Marine Corps The Marine Corps focused on testing multiple specialties within occupational fields (functionally similar collections of specialties)
|
From page 65... ...
Aircraft maintenance occupational field: Helicopter mechanic, CH-46 Helicopter mechanic, CM-53A/D Helicopter mechanic, U/AH-1 Helicopter mechanic, CH-53E Future plans call for similar data collection efforts of multiple specialties for the data communications maintenance and avionics occupational fields (representing the electronics repair aptitude composite) and the personnel administration and supply administration occupational fields (representing the clerical/administrative aptitude composite)
|
From page 66... ...
In this instance, the test administrator presented a scenario and the subject responded by moving chips representing an infantry platoon. Walk-Through Performance Tests The Air Force developed walk-through performance tests, a novel type of test that combines hands-on measures with interview procedures.
|
From page 67... ...
The Army and the Marine Corps developed job knowledge tests for each job they studied, and the Air Force for half of the specialties. Some individual test items corresponded to a certain task or to a specific step within a task, and others related to more global aspects of the job that could not be assessed in the hands-on mode.
|
From page 68... ...
and manuals (e.g., the Marine Corps Individual Training Standards) that provide detailed information about the tasks within an occupational specialty.
|
From page 69... ...
were asked to rank the 150 tasks for a particular job on the basis of their importance in a European theater combat situation; they were also asked to group the tasks into 8 to 12 clusters based on similarity of procedures or principles. As a basis for ~ Proponent schools provide technical training for each military occupational specialty.
|
From page 70... ...
15 subject matter experts from proponent school: rank tasks by mosVleast important sort tasks into 8-12 clusters based on performance requirements Estimate distribution of 10 soldiers on proficiency scale. T Staff job analysts select 30 knowledge tasks (15 appropriate for hands-on)
|
From page 71... ...
The prototype performance measures were then field tested, after which they were sent to the commander of the appropriate proponent school for approval. SIZE OF THE RESEARCH EFFORT This overview of the JPM Project concludes with a brief discussion of the human and monetary resources that were expended on developing the criterion measures and collecting data.
|
From page 72... ...
In addition to the service personnel who were test subjects, many hundreds of research scientists, test administrators, and base personnel who provided logistical support played a part in the JPM Project. The level of effort is not likely to be duplicated by any other institution, or indeed by the military in the foreseeable future.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.