Principles and Practices for a
Federal Statistical Agency
Committee on National Statistics
Constance F. Citro, Editor
Division of Behavioral and Social Sciences and Education
A Consensus Study Report of
THE NATIONAL ACADEMIES PRESS
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW Washington, DC 20001
This activity was supported by Grant No. SES-1024012 between the National Academy of Sciences and the National Science Foundation, which provides support for the work of the Committee on National Statistics from a consortium of federal agencies. Also supporting the Committee’s work are a cooperative agreement from the U.S. Department of Agriculture, to which several agencies contribute, and individual agreements with agencies in the U.S. Departments of Commerce, Health and Human Services, Housing and Urban Development, and Treasury. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author(s) and do not necessarily reﬂect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-46167-2
International Standard Book Number-10: 0-309-46167-7
Digital Object Identifier: https://doi.org/10.17226/24810
Additional copies of this publication are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2017 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. (2017). Principles and Practices for a Federal Statistical Agency, Sixth Edition. Washington, DC: The National Academies Press. https://doi.org/10.17226/24810.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. C. D. Mote, Jr., is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.
Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.
For information about other products and activities of the National Academies, please visit nationalacademies.org/whatwedo.
COMMITTEE ON NATIONAL STATISTICS
LAWRENCE D. BROWN (Chair), Department of Statistics, The Wharton School, University of Pennsylvania
FRANCINE BLAU, Department of Economics, Cornell University
MARY ELLEN BOCK, Department of Statistics, Purdue University (emerita)
MICHAEL CHERNEW, Department of Health Care Policy, Harvard Medical School
JANET CURRIE, Department of Economics, Princeton University
DONALD A. DILLMAN, Social and Economic Sciences Research Center, Washington State University
CONSTANTINE GATSONIS, Department of Biostatistics and Center for Statistical Sciences, Brown University
JAMES S. HOUSE, Survey Research Center, Institute for Social Research, University of Michigan, Ann Arbor
THOMAS L. MESENBOURG, U.S. Census Bureau (retired)
SUSAN A. MURPHY, Department of Statistics and Institute for Social Research, University of Michigan
SARAH M. NUSSER, Office of the Vice President for Research and Department of Statistics, Iowa State University
COLM A. O’MUIRCHEARTAIGH, Harris School of Public Policy Studies, University of Chicago
RUTH D. PETERSON, Criminal Justice Research Center and Department of Sociology, Ohio State University (emerita)
ROBERTO RIGOBON, Sloan School of Management, Massachusetts Institute of Technology
EDWARD H. SHORTLIFFE, Department of Biomedical Informatics, Columbia University and Arizona State University
CONSTANCE F. CITRO, Director
BRIAN HARRIS-KOJETIN, Deputy Director
EILEEN LEFURGY, Program Coordinator
THIS SIXTH EDITION OF Principles and Practices for a Federal Statistical Agency (known as P&P) is dedicated to Miron L. Straf, third director (1987–1999) of the Committee on National Statistics (CNSTAT), who originated the concept for this report. He coedited the first (1992) edition with the late Margaret E. Martin, coedited the second, third, and fourth editions with Martin and Constance F. Citro, and coedited the fifth edition with Citro.
Straf joined CNSTAT in 1977 and served as its research director before becoming its director. He contributed to CNSTAT studies on environmental monitoring, sharing research data, cognitive aspects of survey methodology (for which he received the Innovators Award of the American Association for Public Opinion Research), and statistical assessments as evidence in the courts. During his time with CNSTAT, he developed over 50 major studies and over 40 conferences on the application of statistics to public policy. He has served the statistics and social science professions in many other ways, including as president of the American Statistical Association (ASA). He is a fellow of the ASA and the Royal Statistical Society and an elected member of the International Statistical Institute.
Following his directorship of CNSTAT, Straf worked at the National Science Foundation to develop the research priorities for the social, behavioral, and economic sciences. He then served as deputy director of the Division of Behavioral and Social Sciences and Education at the National Research Council, where his projects included studies that produced the reports Using Science as Evidence in Public Policy and Furthering America’s Research Enterprise. He is a visiting scholar in the Social and Decision Analytics Laboratory of the Virginia Bioinformatics Institute of Virginia Tech.
THE COMMITTEE ON NATIONAL STATISTICS (CNSTAT) is a standing unit of the National Academies of Sciences, Engineering, and Medicine, established in 1972 to provide an independent, objective resource for evaluating and improving the work of the decentralized federal statistical system. Under the terms of the 1863 congressional charter to the National Academy of Sciences to provide advice to the government on scientific and technical matters, CNSTAT has assessed a wide range of statistical methods and data sources for information that the public and policy makers need. It has evaluated numerous federal statistical programs, such as censuses and surveys, and addressed a range of statistical policy issues, such as the independence required for statistical agencies to be credible with policy makers, data users and the public, regardless of persuasion or party.
From time to time in its early years, CNSTAT was asked for advice on what constitutes an effective federal statistical agency. When consulted by congressional staff on the matter in the late 1980s and early 1990s during legislative debates over the (unsuccessful) establishment of a Bureau of Environmental Statistics and the (successful) establishment of a Bureau of Transportation Statistics, CNSTAT decided to prepare a document of high-level guidance. The result, Principles and Practices for a Federal Statistical Agency, was published in 1992. It defined and discussed reasons for the establishment of a statistical agency, identified three fundamental principles for an effective statistical agency (relevance to policy, credibility with data users, and trust of data providers), and identified 11 practices to enable a statistical agency to operationalize and adhere to the principles. A separate “commentary” section discussed each principle and practice in greater detail, drawing on relevant CNSTAT reports for illustration.
When it became clear not only that the 1992 document served a useful purpose for the federal statistical community, but also that it needed to
be updated to respond to changes in the political, economic, social, and technological environment for statistical agency work, CNSTAT decided to prepare a new edition with additional examples from its reports. The second edition was released in 2001, and subsequent editions have been released every 4 years to be available to new appointees and others at the beginning of a presidential term of office. Officials in the various cabinet departments and independent agencies that house federal statistical agencies are not always cognizant about what is proper for these agencies to be credible sources of objective, relevant, accurate, and timely statistics. The document is designed to assist them, as well as the statistical agencies’ leadership and staff, to be fully aware of the standards and ideals that are fundamental to the agencies’ work.
Each of the second–sixth editions has included the three original principles. The fourth edition, in light of some threats to independence, elevated statistical agency independence from a practice to a fourth principle. In addition to the four principles, this sixth edition has 13 practices, most of which were in the original 1992 edition; a few others have been added or reworded in subsequent editions based on the conclusions and recommendations in CNSTAT study reports.
This sixth edition has a new format: Part I defines and identifies reasons to establish a statistical agency, brieﬂy relates the U.S. statistical system’s evolution over time, and discusses the value of federal statistics; Part II explicates and comments on each principle; and Part III does the same for each practice. Two appendixes provide updated information on legislation and regulations that govern federal statistics and the organization of the federal statistical system.
CNSTAT thanks the many people who contributed their time and expertise to the preparation of this report. We are most appreciative of their cooperation and assistance.
We are particularly grateful to the CNSTAT staff, including director Constance Citro, along with Daniel Cork, Brian Harris-Kojetin, and Eileen LeFurgy. This edition, like its predecessors, benefited from the editing of Eugenia Grohman of the Division of Behavioral and Social Sciences and Education. In addition, we owe to her the concept for reorganizing the sixth edition to be more accessible to readers. We are also indebted to many others who offered valuable comments and suggestions, too numerous to mention.
This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and
responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following individuals for their review of this report: Mary Jo Hoeksema, Director, Government Affairs, Population Association of America/Association of Population Centers; Felice J. Levine, Executive Director, American Educational Research Association; James P. Lynch, Professor and Chair, Criminology and Criminal Justice Department, University of Maryland; Miron L. Straf, Social and Decision Analytics Laboratory, Virginia Bioinformatics Institute, Virginia Polytechnic Institute and State University, National Capital Region, Arlington, VA; Roger Tourangeau, Vice President and Associate Director, Westat, Inc.; and Katherine K. Wallman, Chief Statistician (retired), Statistical and Science Policy Office, U.S. Office of Management and Budget.
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by Daniel Kasprzyk, Senior Fellow, NORC at the University of Chicago. He was responsible for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.
Finally, we thank the many federal agencies that support the Committee on National Statistics directly and through a grant from the National Science Foundation, a cooperative agreement from the National Agricultural Statistics Service, and several individual contracts. Without their support and their commitment to improving the national statistical system, the committee work that is the basis of this report would not have been possible.
Lawrence D. Brown, Chair
Committee on National Statistics
Definition of a Federal Statistical Agency
Establishment of a Federal Statistical Agency
Brief History of the U.S. Federal Statistical System
II Principles for a Federal Statistical Agency
Principle 1: Relevance to Policy Issues
Principle 2: Credibility among Data Users
Principle 3: Trust among Data Providers
Principle 4: Independence from Political and Other Undue External Inﬂuence
III Practices for a Federal Statistical Agency
Practice 1: A Clearly Defined and Well-Accepted Mission
Practice 2: Necessary Authority to Protect Independence
Practice 3: Use of Multiple Data Sources for Statistics That Meet User Needs
Practice 4: Openness about Sources and Limitations of the Data Provided
Practice 5: Wide Dissemination of Data
Practice 6: Cooperation with Data Users
Practice 7: Respect for the Privacy and Autonomy of Data Providers
Practice 8: Protection of the Confidentiality of Data Providers’ Information
Practice 9: Commitment to Quality and Professional Standards of Practice
Practice 10: An Active Research Program
Practice 11: Professional Advancement of Staff
Practice 12: A Strong Internal and External Evaluation Program
Practice 13: Coordination and Collaboration with Other Statistical Agencies
Appendixes (in online version, at http://www.nap.edu)
A Legislation and Regulations That Govern Federal Statistics
Authority of the U.S. Office of Management and Budget (OMB) over Federal Statistics
The 1980 Paperwork Reduction Act, as Reauthorized and Amended in 1995, and Associated 2006 Implementation Guidance (Guidance on Agency Survey and Statistical Information Collections—Questions and Answers When Designing Surveys for Information Collections (issued 2006)), also the 1942 Federal Reports Act and the 1950 Budget and Accounting Procedures Act
OMB Statistical Policy Documents
Statistical Policy Directive No. 1—Fundamental Responsibilities of Federal Statistical Agencies and Recognized Statistical Units (issued 2014)
Statistical Policy Directive No. 2: Standards and Guidelines for Statistical Surveys (updated and revised 2006) (including 2016 addendum: Standards and Guidelines for Cognitive Interviews)
Statistical Policy Directive No. 3: Compilation, Release, and Evaluation of Principal Federal Economic Indicators (revised 1985)
Statistical Policy Directive No. 4: Release and Dissemination of Statistical Products Produced by Federal Statistical Agencies (issued 2008) (including proposed 2016 addendum: Performance Review)
Statistical Policy Directive No. 14: Definition of Poverty for Statistical Purposes (updated 1978); also, the 2010 Observations from the Interagency Technical Working Group on Developing a Supplemental Poverty Measure
North American Industry Classification System (NAICS)
North American Product Classification System (NAPCS)
Standard Occupational Classification (SOC)
Metropolitan Statistical Areas (MSAs)
Standards for Maintaining, Collecting, and Presenting Federal Data on Race and Ethnicity (originally issued as Statistical Policy Directive No. 15; revised 1997; including potential changes to the standards)
Other OMB Guidance for Federal Statistics
2014 Guidance for Providing and Using Administrative Data for Statistical Purposes (M-14-06)
2015 Improving Statistical Activities through Interagency Collaboration (M-15-15)
Confidentiality and Privacy Protection
1974 Privacy Act (including systems of records notices, SORNs)
1991 Federal Policy for the Protection of Human Subjects, 45 Code of Federal Regulations (CFR) 46, Subpart A (“Common Rule”), as Revised in 2017
1997 Order Providing for the Confidentiality of Statistical Information
Confidential Information Protection and Statistical Efficiency Act of 2002 (CIPSEA) and Associated 2007 Implementation Guidance
E-Government Act of 2002, Section 208, and Associated 2003 Implementation Guidance (including the privacy impact assessments, PIAs, required for federal data collections)
Federal Information Security Management Act of 2002 (FISMA)
2014 Federal Information Technology Acquisition Reform Act (FITARA) and Associated 2015 and 2016 Implementation Guidance
Federal Cybersecurity Enhancement Act of 2015
Information Quality, Peer Review, Performance Evaluation, Scientific Integrity, and Transparency
The Information Quality Act of 2000 and Associated 2002 Implementation Guidance
2002 Federal Statistical Agency Guidelines for Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity of Disseminated Information
2004 OMB Final Information Quality Bulletin for Peer Review
2010 Government Performance and Results Modernization Act
2010 Office of Science and Technology Policy (OSTP) Memorandum on Scientific Integrity (implementing the President’s 2009 Memorandum on Scientific Integrity)
2013 OSTP Memorandum on Increasing Access to the Results of Scientific Research
B Organization of the Federal Statistical System
U.S. Office of Management and Budget
Principal Statistical Agencies
Bureau of Transportation Statistics
Energy Information Administration
National Agricultural Statistics Service
National Center for Education Statistics
National Center for Health Statistics
National Center for Science and Engineering Statistics
Office of Research, Evaluation, and Statistics
Other Recognized Statistical Units
Center for Behavioral Health Statistics and Quality
National Animal Health Monitoring System Program Unit
Medical Expenditure Panel Survey
National Agricultural Workers Survey
List of Figures and Boxes
I.1 Organization of principal federal statistical agencies and other statistical units and programs, by department, 2017.
I.2 Principal statistical agencies, rotating ICSP members, other recognized statistical units, and the Statistical and Science Policy Office, by congressional appropriations committee and parent department/agency.
I.1 Federal Statistics as a Public Good
III.1 Federal Surveys: Brief History and Examples
III.2 Uses of Administrative Records for Federal Statistics: Examples
III.3 Uses of Nontraditional Data for Federal Statistics: Examples