Commitment to Quality and Professional Standards of Practice
A federal statistical agency’s commitment to quality and professional standards is the foundation of its credibility. Such commitment should be deeply embedded in the agency’s culture and reflected through:
- adhering to and implementing OMB standards and guidelines;
- publishing and implementing agency quality standards;
- maintaining quality assurance programs to improve data quality and the processes of compiling, editing, documenting, analyzing, and disseminating data;
- evaluating the quality of the agency’s data (see Practice 6);
- communicating clearly what is known about the validity and accuracy of the agency’s data and the resulting measures of quality (both uncertainty and bias) (see Practice 10);
- documenting concepts, definitions, and data collection methods and possible sources of error in data releases to the public (see Practice 10); and
- developing and maintaining relationships with appropriate professional organizations in statistics and relevant subject-matter areas (see Practice 5).
An effective statistical agency devotes resources to developing and implementing standards for data quality and professional practice. Although a long-standing culture of data quality contributes to professional practice, an agency should document standards through an explicit
process. Having explicit standards, which are regularly reviewed and updated, facilitates the training of new in-house staff and contractors’ staffs. The reviews should include a careful consideration of quality frameworks used by other national statistical organizations as well as international organizations (see Appendix C).
To ensure the quality of its data collection programs and data releases, an effective statistical agency combines formal quality assurance programs with mechanisms and processes for obtaining both inside and outside reviews (see Practice 6). Formal quality assurance programs include such procedures as well-developed methods for detecting outliers and other errors in raw data, as well as methods for identifying errors from editing and other data processing steps. Reviews help ensure data quality by addressing various aspects of an agency’s operations, including the soundness of the data collection and estimation methods and the completeness of the documentation of the methods used and the error properties of the data. For individual reports, formal processes are needed that incorporate review by agency technical experts and, as appropriate, by technical experts in other agencies and outside the government.16
An effective statistical agency keeps up to date on developments that may be relevant to its program—for example, methods for combining data from more than one source and for estimating error in the resulting statistics (see NASEM, 2017b, 2017c, 2017d, 2018a, 2018d, 2019d, 2020a); and new technologies for data collection, processing, and dissemination (see, e.g., NASEM, 2018c, 2019b; NRC, 2013a).
Statistical agencies should be alert to social and economic changes that may call for innovations in the concepts or methods they use (see, e.g., NASEM, 2017a, 2018b, 2019c, 2020a). The need for change often conflicts with the need for comparability with past data series. Agencies have the responsibility to manage this conflict by initiating more relevant series or revising existing series to improve quality, while providing information to compare old and new series.
The best resource for ensuring high-quality data is a strong professional staff, which includes experts in the subject-matter fields covered by the agency’s program, experts in statistical methods and techniques, and experts in data collection, computing and information science, and other operations (see Practice 4). A major function of an agency’s leadership is
16 See Appendix A for a description of OMB guidelines for peer review of scientific information.
to strike a balance among these staff and to promote collaboration, with each group of experts contributing to the work of the others. An effective statistical agency encourages professional staff to participate in relevant professional associations to refresh their skills and knowledge and to develop networks of experts from other statistical agencies, academia, and the private sector (see Practice 4).
An effective statistical agency also has policies and practices to instill the highest possible commitment to professional ethics among its staff. Because knowledge of codes of ethics from professional associations can reinforce this commitment in the agency culture, an effective agency ensures that its staff members are aware of and have access to such statements of professional practice as those of the American Association for Public Opinion Research (2015), the American Statistical Association (2018), and the International Statistical Institute (2010), as well as to the agency’s own policies and practices regarding such matters as the protection of confidentiality, respect for privacy, and standards for data quality. An effective agency endeavors in other ways as well to ensure that its staff are fully cognizant of the ethics that must guide their actions in order for the agency to maintain its credibility as a source of objective, reliable information for use by all (Hartman et al., 2014).
This page intentionally left blank.