National Academies Press: OpenBook
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×

Performance Assessment for the Workplace

VOLUME I

Alexandra K. Wigdor and Bert F. Green, Jr., Editors

Committee on the Performance of Military Personnel

Commission on Behavioral and Social Sciences and Education

National Research Council

NATIONAL ACADEMY PRESS
Washington, D.C.
1991

Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×

NOTICE: The project that is the subject of this report was approved by the Governing Board of the National Research Council, whose members are drawn from the councils of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine. The members of the committee responsible for the report were chosen for their special competences and with regard for appropriate balance.

This report has been reviewed by a group other than the authors according to procedures approved by a Report Review Committee consisting of members of the National Academy of Sciences, the National Academy of Engineering, and the Institute of Medicine.

The National Academy of Sciences is a private, nonprofit, self-perpetuating society of distinguished scholars engaged in scientific and engineering research, dedicated to the furtherance of science and technology and to their use for the general welfare. Upon the authority of the charter granted to it by the Congress in 1863, the Academy has a mandate that requires it to advise the federal government on scientific and technical matters. Dr. Frank Press is president of the National Academy of Sciences.

The National Academy of Engineering was established in 1964, under the charter of the National Academy of Sciences, as a parallel organization of outstanding engineers. It is autonomous in its administration and in the selection of its members, sharing with the National Academy of Sciences the responsibility for advising the federal government. The National Academy of Engineering also sponsors engineering programs aimed at meeting national needs, encourages education and research, and recognizes the superior achievements of engineers. Dr. Robert M. White is president of the National Academy of Engineering.

The Institute of Medicine was established in 1970 by the National Academy of Sciences to secure the services of eminent members of appropriate professions in the examination of policy matters pertaining to the health of the public. The Institute acts under the responsibility given to the National Academy of Sciences by its congressional charter to be an adviser to the federal government and, upon its own initiative, to identify issues of medical care, research, and education. Dr. Stuart Bondurant is acting president of the Institute of Medicine.

The National Research Council was organized by the National Academy of Sciences in 1916 to associate the broad community of science and technology with the Academy's purposes of furthering knowledge and advising the federal government. Functioning in accordance with general policies determined by the Academy, the Council has become the principal operating agency of both the National Academy of Sciences and the National Academy of Engineering in providing services to the government, the public, and the scientific and engineering communities. The Council is administered jointly by both Academies and the Institute of Medicine. Dr. Frank Press and Dr. Robert M. White are chairman and vice chairman, respectively, of the National Research Council.

This project was supported by the Office of the Assistant Secretary of Defense (Force Management and Personnel).

Library of Congress Catalog Card No. 91-67187

International Standard Book Number 0-309-04538-X

Additional copies of this report are available from:

National Academy Press
2101 Constitution Avenue N.W. Washington, D.C. 20418

S375

Printed in the United States of America

Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×

COMMITTEE ON THE PERFORMANCE OF MILITARY PERSONNEL

BERT F. GREEN, JR. (Chair),

Department of Psychology, Johns Hopkins University

JERALD G. BACHMAN,

Institute for Social Research, University of Michigan

V. JON BENTZ,

Elmhurst, Ill.

LLOYD BOND,

School of Education, University of North Carolina, Greensboro

RICHARD V.L. COOPER,

Ernst & Young, Chicago, Ill.

RICHARD DANZIG,

Latham & Watkins, Washington, D.C.

FRANK J. LANDY,

Department of Psychology, Pennsylvania State University

ROBERT L. LINN,

School of Education, University of Colorado, Boulder

JOHN W. ROBERTS

(USAF, ret.), San Antonio, Tex.

DONALD B. RUBIN,

Department of Statistics, Harvard University

MADY W. SEGAL,

Department of Sociology, University of Maryland

RICHARD J. SHAVELSON,

Graduate School of Education, University of California, Santa Barbara

H.P. VAN COTT,

Committee on Human Factors, National Research Council

HAROLD WOOL,*

Bethesda, Maryland

ALEXANDRA K. WIGDOR, Study Director

CAROLYN J. SAX, Administrative Assistant

*Member, 1983-1985

Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
This page in the original is blank.
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×

CONTENTS OF VOLUME II

 

 

Implications of Cognitive Psychology for Measuring Job Performance
Robert Glaser, Alan Lesgold, and Sherrie Gott

 

 

 

Work Samples as Measures of Performance
Frederick D. Smith

 

 

 

Measuring Job Competency
Bert F. Green, Jr., and Alexandra K. Wigdor

 

 

 

The Evaluation of Alternative Measures of Job Performance
Linda S. Gottfredson

 

 

 

Range Restriction Adjustments in the Prediction of Military Job Performance
Stephen B. Dunbar and Robert L. Linn

 

 

 

Alternatives to the Validity Coefficient for Reporting the Test-Criterion Relationship
Linda J. Allred

 

 

 

Generalizability of Military Performance Measurements: I. Individual Performance
Richard J. Shavelson

 

 

 

Procedures for Eliciting and Using Judgments of the Value of Observed Behaviors on Military Job Performance Tests
Richard M. Jaeger and Sallie Keller-McNulty

 

 

 

Exploring Strategies for Clustering Military Occupations
Paul R. Sackett

 
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×

Preface

This is the sixth and final report of the Committee on the Performance of Military Personnel, which was established at the request of the Department of Defense (DoD) in 1983 to provide scientific oversight of the Joint-Service Job Performance Measurement/Enlistment Standards (JPM) Project. That pioneering research effort, now 10 years under way, proposed to develop robust measures of performance in entry-level military jobs so that, for the first time, military enlistment standards could be linked to performance on the job. The JPM Project was launched during a time when the transition to a system of voluntary military service was still a matter of public controversy and the issue of whether the Services could attract enough qualified people into the enlisted ranks a central policy concern. Its fruition is coming after years of great recruiting success, at a time when budgetary stringencies have recast the policy debate to address the cost-effectiveness of high-quality enlisted personnel.

Because of the complexity of the JPM Project research goals, as well as the pressing need to establish the credibility of military selection and classification procedures after technical errors in computing enlistment test scores were discovered in 1980, DoD turned to the National Academy of Sciences to provide independent technical review of the research as it progressed.

Created by act of Congress in 1863 as a private, nonprofit corporation, the National Academy of Sciences enjoys a unique position as an official, yet independent, adviser to the federal government. The Academy accomplishes its work through committees of experts established within the National Research Council. It has demonstrated consistently that it can call on

Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×

the very best scientific and technical expertise nationwide. The independence and objectivity of NRC committees are carefully protected: the Academy does not reply to competitive requests for proposals; members of study committees serve without monetary compensation and are carefully screened for possible conflicts of interest; and the reports produced by committees are subject to thoroughgoing review before being transmitted to sponsors.

The Committee on the Performance of Military Personnel was selected in accordance with this tradition of excellence and independence. Housed within the Commission on Behavioral and Social Sciences and Education, its membership was drawn from the scientific and policy fields of greatest relevance to the JPM Project: psychometrics, performance measurement, industrial and organizational psychology, research design and statistics, military sociology, economics, human resource management, and military manpower policy. The committee included both scholars and practitioners—those whose scientific sophistication represents the state of the art, as well as those who understand firsthand the exigencies of real-world applications. General John Roberts (USAF, Ret.), former head of Air Training Command, brought to committee deliberations experience at the highest policy levels with military selection, classification, and training issues. John Bentz offered the insights gleaned from 35 years as director of personnel research for a major nationwide retailing company. Biographies of all committee members and staff appear in Appendix B.

The first phase of the JPM Project entailed the development of measures of performance for a sample of military jobs—jobs like jet engine mechanic, avionics communication specialist, infantryman, and medical technician. The goal was to determine how well the Armed Services Vocational Aptitude Battery (ASVAB), the test that is used throughout the military to assess enlistment eligibility, predicts performance on these high-fidelity measures of job performance. The committee 's advisory activities in the early years centered on evaluation of the research design and the development of measurement instruments (e.g., hands-on tests, written job knowledge tests, supervisor ratings) and issues in hands-on test administration. Later the focus turned to evaluation of the quality of the measures and analysis of the performance data to understand the relationships among the various new performance measures and to ASVAB scores and other input characteristics. (The second phase of the project, now well under way, is exploring the use of cost/performance trade-off models for setting enlistment standards. A new Committee on Military Enlistment Standards has been established by the National Academy of Sciences to advise on that work.)

In order to enhance its understanding of the job performance measurement research and to place the project in proper context, members of the Committee on the Performance of Military Personnel also learned about military entrance processing, entry-level jobs in the military, technical training,

Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×

and the general outlines of how entrance standards are currently set. Committee and staff members made a series of site visits to Army, Navy, Marine Corps, and Air Force bases to see enlisted personnel at work, to talk to their supervisors about the content of entry-level jobs, and to observe the administration of hands-on tests. They experienced at firsthand the operation of military entrance processing stations, visited training facilities, and also made a number of visits to military personnel research laboratories to gather information and discuss the progress of the research.

To facilitate the exchange of ideas, over the years the committee invited JPM Project scientists as well as other experts to explore solutions to specific technical problems in a series of workshops. And, to supplement its own activities, the committee has called on outside experts to prepare background papers on various aspects of the issues involved. A selection of these papers is presented in the companion volume to this report.

This book is a distillation of the experiences and insights and many things learned by the committee over the years since its founding in 1983 and, we hope, conveys at least a modicum of what the Service research scientists learned as well. It represents many years of faithful attention by committee members to the task at hand, frequent delight in the collegial nature of the enterprise, and a lot of sheer hard work. In the course of preparing the report, each of the members took an active role in drafting sections or chapters, providing data, leading discussions, and reading and commenting on successive drafts. Robert Linn bore primary responsibility for drafting the chapter on criterion-related validity. Richard Shavelson made major contributions to several chapters. Frank Landy and Lloyd Bond organized team efforts on the development of performance measures and standardization issues, respectively. John Roberts drafted sections on manpower policy issues.

Our work has benefited over the years from the contributions of many inside and outside the military. We were received with unfailing courtesy and openness at numerous military installations and provided with as much information as we could think to request. We developed a productive working relationship with the Job Performance Measurement Working Group, which coordinates the research effort, under the successive leadership of Col. Charles R. Curran, Lt. Col. Burt Itoga, Lt. Col. Dickie Harris, and Lt. Col. Thomas Ulrich. David J. Armor, former principal deputy to the Assistant Secretary of Defense (Force Management and Personnel) and currently a member of the follow-on Committee on Military Enlistment Standards, stimulated many a good discussion of technical challenges. Our workshops were enlivened by the presentations of many of the military researchers and their contractors, among others Milton H. Maier, Lawrence Hanser, Gerald Laabs, Col. Rodger D. Ballentine, Paul W. Mayberry, John Campbell, Leonard P. Kroeker, and Jerry W. Hedge.

Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×

In addition to the authors of papers that appear in the accompanying volume, Brian K. Waters, Janice H. Laurence, and Wayne J. Camara contributed a separately published paper on personnel enlistment and classification procedures in the U.S. military. Stephen B. Dunbar 's analysis of JPM data forms the basis of the discussion of group-to-group differences in prediction systems presented in Chapter 8. Gail P. Baxter and James R. Valadez of the University of California at Santa Barbara assisted Richard J. Shavelson with background materials.

Carolyn Sax, the committee's administrative assistant, was indispensable in keeping track of everything from errant committee members to errant commas. We could not have managed half so well without her quiet competence. Thanks are due as well to her extremely able predecessor, Diane Goldman, to research associates Hilda Wing and Anne S. Mavor, and to Christine McShane, who has edited all of the committee's reports and in the process become something of an expert on job performance measurement in the military.

A special note of thanks is due to W.S. Sellman, director for accession policy in the Office of the Assistant Secretary of Defense (Force Management and Personnel), who had the tenacity needed to see the JPM Project through to fruition and a great enough respect for science to subject himself to the ministrations of an independent advisory committee.

Bert F. Green, Jr., Chair

Alexandra K. Wigdor, Study Director

Committee on the Performance of Military Personnel

Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×

Performance Assessment for the Workplace

Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
This page in the original is blank.
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R1
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R2
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R3
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R4
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R5
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R6
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R7
Page viii Cite
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R8
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R9
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R10
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R11
Suggested Citation:"Front Matter." National Research Council. 1991. Performance Assessment for the Workplace: Volume I. Washington, DC: The National Academies Press. doi: 10.17226/1862.
×
Page R12
Next: Overview »
Performance Assessment for the Workplace: Volume I Get This Book
×
Buy Paperback | $75.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Although ability testing has been an American preoccupation since the 1920s, comparatively little systematic attention has been paid to understanding and measuring the kinds of human performance that tests are commonly used to predict—such as success at school or work. Now, a sustained, large-scale effort has been made to develop measures that are very close to actual performance on the job. The four military services have carried out an ambitious study, called the Joint-Service Job Performance Measurement/Enlistment Standards (JPM) Project, that brings new sophistication to the measurement of performance in work settings.

Volume 1 analyzes the JPM experience in the context of human resource management policy in the military. Beginning with a historical overview of the criterion problem, it looks closely at substantive and methodological issues in criterion research suggested by the project: the development of performance measures; sampling, logistical, and standardization problems; evaluating the reliability and content representativeness of performance measures; and the relationship between predictor scores and performance measures—valuable information that can also be useful in the civilian workplace.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!