As an all-volunteer service accepting applications from nearly 400,000 potential recruits annually from across the U.S. population, the U.S. military must accurately and efficiently assess the individual capability of each recruit for the purposes of selection, job classification, and unit assignment. In light of this continual challenge, the U.S. Army Research Institute for the Behavioral and Social Sciences (ARI) requested that the National Research Council (NRC) conduct a study to examine the future of military entrance assessments. On April 3-4, 2013, the committee appointed by the NRC to conduct this study, the Committee on Measuring Human Capabilities: Performance Potential of Individuals and Collectives, held a workshop on the topic of “New Directions in Assessing Individuals and Groups.”
In advance of the workshop, participants were invited to read materials provided by ARI to familiarize them with current Army selection practices. The opening pages of the 2011 ARI special report, Select for Success, capture the spirit in which this workshop was planned and hosted:
What if the Army could
… screen out low motivated, low performing applicants?
… screen in highly motivated, high performing applicants?
… better predict APFT [army physical fitness test] success?
… reduce the number of Soldiers who attrit before commitment completion?
… reduce the time and effort units devote to dealing with problem Soldiers?
… select Soldiers who are more satisfied with the Army and their MOS [military occupation specialty]?
… select Soldiers who are more likely to espouse and reflect Army values? (U.S. Army Research Institute for the Behavioral and Social Sciences, 2011, p. 1)
For these reasons and others, it is increasingly important that the U.S. Army leverage effective means to select and assign personnel. To that end, this study in its entirety was designed to investigate cutting-edge research into the measurement of both individual capabilities and group composition in order to identify future research directions that may lead to improved assessment and selection of enlisted personnel for the U.S. Army. The first phase of the study included a workshop to which scientists from a variety of relevant areas were invited to discuss research that bears on the issues at hand. (The statement of task for the study’s first phase is reproduced in Box 1-1.) In the second phase, the committee will consider in more depth the research areas discussed at the workshop, as well as additional areas of research, and will develop consensus conclusions and recommendations to inform future research related to the measurement of individual and group capabilities. The ultimate goal of both the workshop and the committee report is to inform the design of a maximally effective selection and assessment system.
As committee chair Jack Stuster of Anacapa Sciences, Inc., noted in his introductory comments, the goal of the workshop was “to encourage interdisciplinary dialogue on the current and future state-of-the-science in measurement of individual capability and the combination of individual capabilities to create collective capacity to perform.” The workshop emphasized cognitive and noncognitive attributes that can be used in the initial testing and assignment of enlisted personnel. Certain topics were excluded from consideration, including physical attributes and skills, endocrinology or blood chemistry indicators of performance, genetic screening, biographical data as predictors of performance, birth-order effects, work samples, miniature training and evaluation technologies, and mid-career and post-injury assessments. Some of these were excluded because of ethical considerations, while others were excluded because of the practical needs of standardized and mass testing.
The specific workshop topics included
- the evolving goals of candidate testing,
- emerging constructs and theory,
- ethical implications of testing methods,
BOX 1-1
Phase 1 Statement of Task
An ad hoc committee will plan and host a public workshop to encourage interdisciplinary dialogue on the current and future state-of-the-science in measurement of individual capability and the combination of individual capabilities to create collective capacity to perform. The workshop will feature invited presentations and discussions to address the following questions:
- Beyond tests of cognitive ability (e.g., Armed Services Vocational Aptitude Battery [ASVAB]) and personality (e.g., Tailored Adaptive Personality Assessment System [TAPAS]) already in use by the U.S. Army, which cutting-edge areas of research, including those already being investigated by the U.S. Army Research Institute (ARI), may provide new and unique scientifically valid methods for measuring individual capabilities and predicting individual and collective performance?
- Are there recent or emerging theoretical, technological, and/or statistical advances that have enabled new approaches and/or measurement capabilities with respect to the measurement of individual capability and the combination of individual capabilities to create collective capacity to perform?
- Are there neuroscience or psychophysiology advances related specifically to the understanding of individual differences that suggest new ways to approach empirical research and theory development in this area? If so, what are they, who are the researchers, and how might they be applied in ARI’s basic research program?
- ARI would benefit from an interdisciplinary discussion of “gaps” and potential research related specifically to theories of individual differences (cognitive, affective, personality, social or interpersonal skills), testing and measurement methods, test theory, statistical and mathematical modeling of collective/group/team performance, and the combination of individual capabilities to create collective capacity to perform. Which “gaps” exist in the scientific findings? Which “gaps” appear the most promising for future research with potential near-term payoff?
The discussion and major themes that emerge from the workshop will be synthesized in an individually authored workshop summary.
- measuring individual differences and predicting individual performance,
- group composition processes and performance, and
- crosscutting links and research gaps.
By bringing together scientists from a variety of areas of research, the workshop was designed to explore interdisciplinary scientific approaches to individual and group assessments and to identify promising concepts
for further consideration. The overall study is designed to inform the development of near-term basic research programs leading to applied research and specific applications in the long term. Therefore, many of the discussions were more broadly applicable to a wide range of testing and assessment conditions, rather than to the narrowly focused needs of Army assessment and selection.
This publication is a summary and synthesis of the two-day workshop held in fulfillment of phase 1 of the study. Readers of this summary should keep in mind that it is only reporting on the presentations and discussions of individuals at the workshop. In particular, any opinions, conclusions, or recommendations reported here are solely those of the individuals who offered them. The presentations and discussions were limited by the time available for the workshop; see the Appendix for the workshop agenda and list of participants. Potentially important subjects and areas of research that were not covered during the workshop are not included here, and their omission from this summary should not be interpreted as an assessment of their value, but only that time did not allow their inclusion. There was no attempt to reach consensus at the workshop, and any statements that appear in this workshop summary should not be taken as indicative of the ultimate conclusions and recommendations that the committee may include in its final report at the end of the second phase. This summary was prepared by a rapporteur and summarizes views expressed by individual workshop participants. The NRC is responsible only for its overall quality and accuracy as a record of what transpired at the workshop.
U.S. Army Research Institute for the Behavioral and Social Sciences. (2011). Select for Success: A Toolset for Enhancing Soldier Accessioning. Special Report 70. Available: http://www.dtic.mil/cgi-bin/GetTRD?AD=ADA554057 [July 2013].
