Transparency in Statistical Information for the National Center for Science and Engineering Statistics and All Federal Statistical Agencies (2022) / Chapter Skim
Currently Skimming:
Summary
Summary
Pages 1-14
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
: Federal statistical agencies must have credibility with those who use their data and information.
|
From page 2... ...
For our purposes, transparency is the provision of sufficiently detailed documentation of all the processes of producing official estimates. The goal of transparency is to enable consumers of federal statistics to accurately understand and evaluate how estimates are generated.
|
From page 3... ...
and associated computer code -- seeks to determine if the published results based on these stages of the research process can be reproduced. Answering question 3 involves conducting a much broader independent investigation, within the "specific study context" that produced the original official estimates.
|
From page 4... ...
Second, transparency facilitates innovation and improvement in statistical methods. Internal researchers from a statistical agency producing official statistics, as well as external researchers, are in a better position to enhance methods for data collection or estimation if clear and detailed documentation of existing processes and sources for producing official estimates is maintained and made accessible.
|
From page 5... ...
To be in agreement with that notion of reproducibility, statistical agencies would need to retain the code used for all data treatments and estimation steps. If one then used the archived input datasets as inputs into this retained code (in the appropriate computer environment)
|
From page 6... ...
federal statistical agencies and various international statistical offices are to benefit from sharing methods, data, and results with each other, such efforts would be eased by making use of common tools for documentation and exchange. TRANSPARENCY AS A NECESSARY COMPONENT OF FEDERAL STATISTICS The panel reviewed federal legislation, OMB statistical policy directives and memoranda, best practice documents, including Principles and Practices, and other sources to identify what is variously required and recommended for running an effective federal statistical agency.
|
From page 7... ...
Recommendation 3.1: The agencies that produce federal statistics, through the leadership of the Interagency Council on Statistical Policy and the Chief Statistician of the United States, should fully comply with federal record schedules, ensuring that the input datasets that can legally be retained, and official estimates that are produced, are archived in the National Archives and Records Administration. The metadata that accompany such data should also be preserved using broadly accepted metadata standards appropriate to the data at hand.
|
From page 8... ...
Since the federal statistical agencies use software processes to help to collect data, treat data in preparation for use in estimation, and estimate a set of official statistics, these tools should be examined for their utility in supporting greater transparency of official statistics. Recommendation 4.1: Agencies that produce federal statistics, includ ing the National Center for Science and Engineering Statistics, should review and make a priority of adopting modern information technology tools that assist in collaborative software development and documenta tion of workflow and methodology.
|
From page 9... ...
prioritize and emphasize the importance and benefits of federal statistical agency staff engaging in international metadata stan dards and tool development, and (2) organize a discussion among statistical agencies that leads to an effective, coordinated, and account able approach for staff in agencies that produce federal statistics to contribute to international metadata standards and tool development.
|
From page 10... ...
However, more than one program requested users to go to a technical report to find out what data treatments were used to address nonresponse or failed edits. Further, it was not always clear where the input data or resulting official estimates were archived.
|
From page 11... ...
Recommendation 6.5: The National Center for Science and Engineering Statistics' (NCSES) information technology staff and NCSES's program staff should collaborate to standardize the inclusion of language in their contracts and interagency agreements requiring that contractors pro vide machine-actionable metadata and code so that NCSES can meet acceptable standards of transparency about its data products for users and other agencies and achieve consistency in the metadata used across NCSES's statistical programs.
|
From page 12... ...
SYSTEMWIDE BEST PRACTICES The panel tried to assess the degree to which the information currently provided by NCSES and the other federal statistical agencies about their programs and series of official statistics is consistent with the goals of being transparent. Many agencies do not have formal guidelines or rules that their statistical programs can use to decide what information to provide both internally to agency staff and publicly to their user communities.
|
From page 13... ...
Recommendation 7.1: The National Center for Science and Engineer ing Statistics (NCSES) and all agencies that produce federal statistics should, to the fullest extent feasible, document their data collection methods, their data treatments, their estimation methodologies, and assessments of the quality of their official estimates, and they should archive their input datasets and their official estimates to support reproducibility and later reuse, as specified in the tables developed by the panel.
|
From page 14... ...
This entails support for pilot projects, additional training of existing staff, enlisting of assistance from experts through support contracts, and reconfiguring of existing processes. Recommendation 7.3: Agencies that produce federal statistics, in o rder to implement many of the recommended initiatives in this report, should be provided with additional funds to acquire the necessary training and information technology assistance, as well as cover any increased operational costs, to modify current processes to improve documentation and archiving in support of the greater transparency of official statistics.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.