Strengthening Data
Science Methods
FOR DEPARTMENT OF DEFENSE
PERSONNEL AND
READINESS MISSIONS
Committee on Strengthening Data Science Methods for
Department of Defense Personnel and Readiness Missions
Committee on Applied and Theoretical Statistics
Board on Mathematical Sciences and Their Applications
Division on Engineering and Physical Sciences
A Report of
THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This project was supported by Contract No. HQ0034-10-D-0003 TO#12 with the U.S. Department of Defense and Grant No. H98230-14-1-0116 from the National Security Agency. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the view of any organization or agency that provided support for the project.
International Standard Book Number 13: 978-0-309-45078-2
International Standard Book Number 10: 0-309-45078-0
Digital Object Identifier: 10.17226/23670
Additional copies of this report are available for sale from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2017 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2017. Strengthening Data Science Methods for Department of Defense Personnel and Readiness Missions. Washington, DC: The National Academies Press. doi:10.17226/23670.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. C. D. Mote, Jr., is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.national-academies.org.
Reports document the evidence-based consensus of an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and committee deliberations. Reports are peer reviewed and are approved by the National Academies of Sciences, Engineering, and Medicine.
Proceedings chronicle the presentations and discussions at a workshop, symposium, or other convening event. The statements and opinions contained in proceedings are those of the participants and have not been endorsed by other participants, the planning committee, or the National Academies of Sciences, Engineering, and Medicine.
For information about other products and activities of the National Academies, please visit nationalacademies.org/whatwedo.
COMMITTEE ON STRENGTHENING DATA SCIENCE METHODS FOR DEPARTMENT OF DEFENSE PERSONNEL AND READINESS MISSIONS
STEPHEN M. ROBINSON, NAE,1 University of Wisconsin, Madison, Chair
PASCALE CARAYON, University of Wisconsin, Madison
DAVID CHU, Institute for Defense Analyses, Alexandria, Virginia
CYNTHIA DWORK, NAS2/NAE, Microsoft Research, Mountain View, California
TERRY P. HARRISON, Pennsylvania State University, University Park
ALAN F. KARR, RTI International, Research Triangle Park, North Carolina
SALLIE KELLER, Social Decision Analytics Laboratory, Biocomplexity Institute of Virginia Tech, Arlington
ALAIR MACLEAN, Washington State University, Vancouver
DAVID MAIER, Portland State University, Oregon
STEPHEN M. POLLOCK, NAE, University of Michigan, Ann Arbor (until June 2015)
PAUL R. SACKETT, University of Minnesota, Minneapolis
MARK S. SQUILLANTE, IBM Research, Yorktown Heights, New York
WILLIAM J. STRICKLAND, HumRRO, Alexandria, Virginia
STEVEN TADELIS, University of California, Berkeley
Staff
MICHELLE K. SCHWALBE, Senior Program Officer, Study Director
SCOTT WEIDMAN, Board Director
CHERIE CHAUVIN, Senior Program Officer
LINDA CASOLA, Senior Program Assistant
RODNEY N. HOWARD, Administrative Assistant
BETH DOLAN, Financial Manager
JONATHAN YANGER, Research Associate
CLAIRE JI, Christine Mirzayan Science and Technology Policy Graduate Fellow
___________________
1 NAE, National Academy of Engineering.
2 NAS, National Academy of Sciences.
COMMITTEE ON APPLIED AND THEORETICAL STATISTICS
CONSTANTINE GATSONIS, Brown University, Providence, Rhode Island, Chair
DEEPAK AGARWAL, LinkedIn, Mountain View, California
KATHERINE BENNETT ENSOR, Rice University, Houston, Texas
MONTSERRAT (MONTSE) FUENTES, North Carolina State University, Raleigh
ALFRED O. HERO III, University of Michigan, Ann Arbor
DAVID M. HIGDON, Social Decision Analytics Laboratory, Biocomplexity Institute of Virginia Tech, Arlington
ROBERT E. KASS, Carnegie Mellon University, Pittsburgh, Pennsylvania
JOHN LAFFERTY, University of Chicago, Illinois
XIHONG LIN, Harvard University, Cambridge, Massachusetts
JOSÉ M.F. MOURA, NAE, Carnegie Mellon University, Pittsburgh, Pennsylvania
SHARON-LISE T. NORMAND, Harvard University, Cambridge, Massachusetts
GIOVANNI PARMIGIANI, Harvard University, Cambridge, Massachusetts
ADRIAN RAFTERY, NAS, University of Washington, Seattle
LANCE WALLER, Emory University, Atlanta, Georgia
EUGENE WONG, NAE, University of California, Berkeley
Staff
MICHELLE K. SCHWALBE, Senior Program Officer
RODNEY N. HOWARD, Administrative Assistant
LINDA CASOLA, Senior Program Assistant
BETH DOLAN, Financial Manager
BOARD ON MATHEMATICAL SCIENCES AND THEIR APPLICATIONS
DONALD G. SAARI, NAS, University of California, Irvine, Chair
DOUGLAS N. ARNOLD, University of Minnesota, Minneapolis
JOHN B. BELL, NAS, Lawrence Berkeley National Laboratory, Berkeley, California
VICKI M. BIER, University of Wisconsin, Madison
JOHN R. BIRGE, NAE, University of Chicago, Illinois
RONALD R. COIFMAN, NAS, Yale University, New Haven, Connecticut
L. ANTHONY COX, JR., NAE, Cox Associates, Denver, Colorado
MARK L. GREEN, University of California, Los Angeles
PATRICIA A. JACOBS, Naval Post Graduate School, Monterey, California
BRYNA KRA, Northwestern University, Evanston, Illinois
JOSEPH A. LANGSAM, Morgan Stanley (retired), Palm Beach Gardens, Florida
SIMON A. LEVIN, NAS, Princeton University, New Jersey
ANDREW W. LO, Massachusetts Institute of Technology, Cambridge
DAVID MAIER, Portland State University, Oregon
WILLIAM A. MASSEY, Princeton University, New Jersey
JUAN C. MEZA, University of California, Merced
FRED S. ROBERTS, Rutgers University, Piscataway, New Jersey
GUILLERMO SAPIRO, Duke University, Durham, North Carolina
CARL P. SIMON, University of Michigan, Ann Arbor
KATEPALLI SREENIVASAN, NAS/NAE, New York University
ELIZABETH A. THOMPSON, NAS, University of Washington, Seattle
Staff
SCOTT WEIDMAN, Board Director
NEAL GLASSMAN, Senior Program Officer
MICHELLE K. SCHWALBE, Senior Program Officer
RODNEY N. HOWARD, Administrative Assistant
BETH DOLAN, Financial Manager
This page intentionally left blank.
Acknowledgments
This report has been reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the institution in making its published report as sound as possible and to ensure that the report meets institutional standards for objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process. We wish to thank the following individuals for their review of this report:
Brenda Dietrich, NAE,1 IBM,
Alexis Fink, Intel,
Christine Fox, Johns Hopkins University Applied Physics Laboratory,
Ronald D. Fricker, Jr., Virginia Polytechnic and State University,
Henry J. Hatch, NAE, U.S. Army (retired),
Carl Hoffmann, Human Capital Management and Performance LLC,
Ronald C. Kessler, NAS2/NAM,3 Harvard Medical School,
Philip Neches, NAE, Teradata,
Stephen M. Pollock, NAE, University of Michigan, and
Michael G. Rumsey, U.S. Army Research Institute (retired).
___________________
1 National Academy of Engineering.
2 National Academy of Sciences.
3 National Academy of Medicine.
Although the reviewers listed above have provided many constructive comments and suggestions, they were not asked to endorse the views presented in the report, nor did they see the final draft of the report before its release. The review of this report was overseen by Robert F. Sproull, University of Massachusetts, Amherst, NAE, who was responsible for making certain that an independent examination of this report was carried out in accordance with institutional procedures and that all review comments were carefully considered. Responsibility for the final content of this report rests entirely with the committee and the institution.
The committee would like to thank the following individuals for providing input to this study:
Joseph Adams, Institute for Defense Analyses,
Amy Alrich, Institute for Defense Analyses,
Jeff Angers, Office of the Assistant Secretary of the Army, Manpower and Reserve Affairs,
Beth Asch, RAND Corporation,
Stephanie Barna, Office of the Under Secretary of Defense (Personnel & Readiness),
Mark Breckenridge, Defense Manpower Data Center,
Iveta Brigis, Google People Operations,
Kelly Buck, Defense Manpower Data Center,
Roll Buller, RAND Corporation,
Samuel Buttery, Naval Postgraduate School,
John Carbone, Office of the Assistant Secretary of the Army, Manpower and Reserve Affairs,
Linda C. Cavalluzzo, Center for Naval Analyses,
Ray Conley, RAND Corporation,
Elia Crisman, Office of the Chief of Naval Operations,
Matthew Davis, Defense Manpower Data Center,
Bruce Doll, Defense Health Agency,
Michael Dominguez, Institute for Defense Analyses,
Colin Doyle, Institute for Defense Analyses,
Mark Engelbaum, U.S. Air Force, Manpower, Personnel and Services,
Alexis Fink, Intel,
Colum Foley, IBM Smarter Workforce,
Christine Fox, Johns Hopkins Applied Physics Laboratory,
Peggy Golfin, Center for Naval Analyses,
Jay Goodwin, U.S. Army Research Institute,
Kristy Gore, RAND Corporation,
David Gregory, Center for Naval Analyses,
Jane Handley, RAND Corporation,
Kim Hepner, RAND Corporation,
Carl Hoffmann, Human Capital Management and Performance, LLC,
Stanley Horowitz, Institute for Defense Analyses,
Mike Housman, Cornerstone OnDemand,
David Hudak, Office of the Secretary of Defense,
Magdi Kamel, Naval Postgraduate School,
David Knapp, RAND Corporation,
Jennifer Kurkoski, Google People Operations,
Lance Landower, RAND Corporation,
Marco Laumanns, IBM Research,
Paul B. Lester, U.S. Army, Research Facilitation Laboratory,
Scott Lutterloh, Department of the Navy, Manpower and Reserve Affairs,
Arun Maiya, Institute for Defense Analyses,
Philip Major, Institute for Defense Analyses,
Bill Mason, Department of Defense, Office of Information,
Sean McDonald, Office of the Chief of Naval Operations,
Laura Miller, RAND Corporation,
Dave Nap, RAND Corporation,
David Nicholls, Institute for Defense Analyses,
Mary Kate Norton, Google People Operations,
Laura Odell, Institute for Defense Analyses,
Peter Olsen, U.S. Air Force, Manpower, Personnel and Services,
Julie Pechacek, Institute for Defense Analyses,
Jeff Petersen, Center for Naval Analyses,
Christopher Rice, Intel,
Susan Rose, Institute for Defense Analyses,
Mark Rosen, Center for Naval Analyses,
Johannes Royset, Naval Postgraduate School,
Jackie Ryan, IBM Smarter Workforce,
Richard Schaeffer, Office of the Assistant Secretary of Defense,
Ed Schmitz, Center for Naval Analyses,
Brandon Shapiro, Institute for Defense Analyses,
Yu-Chu Shen, Naval Postgraduate School,
Max Simkoff, Cornerstone OnDemand,
Don Simrod, Center for Naval Analyses,
Cara Sims, RAND Corporation,
Allison Taylor, Institute for Defense Analyses,
Fred Thompson, Office of the Under Secretary of Defense (Personnel & Readiness),
George Tolis, Institute for Defense Analyses,
Mike Walker, United States Military Academy at West Point,
Kayla Williams, RAND Corporation, and
John D. Winkler, RAND Corporation.
This page intentionally left blank.
4 OVERVIEW OF DATA SCIENCE METHODS
Limitations of Privacy-Preserving Approaches
The Federal Statistical System’s Legal and Governance Policy
6 COMMERCIAL STATE OF THE ART IN HUMAN RESOURCES ANALYTICS
7 IDENTIFYING P&R OPPORTUNITIES AND IMPLEMENTING SOLUTIONS
Improve Data Quality and Sharing
Strengthen P&R Workforce Capabilities in Data Science
B Biographies of the Committee
D Stochastic Models of Uncertainty and Mathematical Optimization Under Uncertainty