Federal statistical agencies must continually seek to improve and innovate their processes, methods, and statistical products to better measure an ever changing world.
FEDERAL statistical agencies cannot be static. They must continually improve and innovate to be able to create reliable information on new policy questions, to provide objective information in a cost-effective way, and to meet user demands for more timely and granular information for statistical purposes.
Policy needs shift and evolve, and the society and economy that federal statistical agencies seek to measure are also evolving and changing at a rapid pace. To provide relevant information, statistical agencies must attend to changes in policy issues in their domain, identify emerging needs, and work with their data users and stakeholders to identify gaps in the agency portfolio or potential new statistical products that are needed (see Practice 9). One option to address needs for new information is for the agency to create experimental series; doing so allows the agency and its users time to evaluate a new data product without impacting existing data series (see Practice 5).
Recent years have witnessed an explosion of new data sources, some providing more geographic detail and more timely (some in near real-time) information than federal statistical programs. Users have come to expect more, better, and faster data. At the same time, individuals and businesses have been less and less willing to complete federal surveys and provide information to the government (a phenomenon that also affects private sector surveys). Declining response rates have increased agency data collection costs, while federal statistical agency budgets have generally declined in real terms for more than a decade. Thus, agencies
Agencies should engage in regular, periodic reviews of their major data collection programs that consider how to produce relevant, accurate, and timely data in the most cost-effective manner possible, while seeking to maintain comparability in key statistics over time and across geographies (see Practice 6). In ongoing programs for which it would be disruptive to implement improvements on a continuous basis, a common practice is to bundle changes to implement several at the same time. For example, classifications such as the North American Industry Classification System (NAICS) are updated every 5 years and agencies may implement other changes at the same time as this. Agencies should ensure that the intervals between major research and development activities do not become so long that data collection programs deteriorate in quality, relevance, and efficiency (see Practice 6). When changes are made to ongoing data series, agencies should provide information to help users bridge across the old and new series.
An effective statistical agency keeps up to date on developments in theory and practice that may be relevant to its program. Examples of such developments include new techniques for imputing missing data (see, e.g., NRC, 2004a, 2010e) or for combining data from more than one source and estimating error in the resulting statistics (see NASEM, 2017b); new technologies for data collection, processing, and dissemination; new methodologies addressing data confidentiality and disclosure avoidance; new techniques, such as machine learning or artificial intelligence, to analyze and process data; and new kinds of and uses for data about collection processes (paradata) (see, e.g., NRC, 2013a).
Statistical agencies need a robust research program that includes statistical methods, quality assessments, and evaluations of potential new data sources. An effective statistical agency seeks out and carefully evaluates the quality and utility of potential new data sources and methods to harness information that could be useful for statistical purposes. Nontraditional data sources, such as sensor or transactions data, and fuller use of administrative records can potentially contribute to statistical programs by: (1) augmenting information obtained from traditional sources such as surveys; (2) replacing information elements previously obtained from traditional sources; (3) providing earlier estimates that
are later benchmarked with traditional sources; and (4) analyzing information streams to identify needed changes (see Practice 5). Agencies also need the appropriate IT infrastructure to handle alternative data sources. History has repeatedly shown that research conducted within federal statistical agencies on subject areas, methods, and operations can lead to large productivity gains in statistical activities for a relatively low cost (see, e.g., Citro, 2016; NRC, 2010c).
An effective statistical agency has a culture of continual improvement and innovation. All employees, and not just research staff, should be encouraged to seek to innovate and improve their functions within the organization. Staff in production and support areas should seek to improve processes, methods, and cost-effectiveness (see Practice 3). A statistical agency also needs to hire staff with cutting-edge skills and maintain and enhance the skills of its current staff through ongoing training and development opportunities so that it can continually improve and innovate (see Practice 4). To take the greatest advantage of staff with new and improved skills and to better support their operations, statistical agencies should maintain and regularly upgrade their information technology infrastructure.
The decentralized nature of the U.S. federal statistical system can make it difficult for federal statistical agencies to easily learn from each other, but interagency and international collaborations can provide important and useful means for improving statistical programs. Some issues, such as accessing and using new data sources, are common to many statistical agencies and can benefit from collaborative research across organizations (see Practice 7).
Practices That Are Particularly Relevant for Principle 5