Principles and Practices for a Federal Statistical Agency Fifth Edition (2013) / Chapter Skim
Currently Skimming:
Part II: Commentary
Part II: Commentary
Pages 25-78
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 25... ...
Statistical agencies have been established to serve several purposes, including: • to develop new information for an area of public concern (e.g., the Bureau of Labor Statistics [BLS] and the National Center for Health Statistics)
|
From page 26... ...
Such data are usually collected under a pledge of confidentiality. Statistical agencies may collect information from government agencies in which individual reporting units are identified because the data are already public i nformation -- as, for example, in the Census Bureau's program to collect financial and employment information for state and local governments (see National Research Council, 2007a)
|
From page 27... ...
Statistical agencies should carefully consider the advantages and dis advantages of undertaking a program with both statistical and nonstatistical purposes. One potential advantage is that there may be improved consistency and quality when a statistical agency collects information for its own use and that of other parts of its department.
|
From page 28... ...
The work of federal statistical agencies is coordinated through the U.S. Office of Management and Budget (OMB)
|
From page 29... ...
Similarly, the principles and practices may be relevant to statistical units in state and local government agencies, as well as for international statistical agencies. ESTABLISHMENT OF A FEDERAL STATISTICAL AGENCY One of the most important reasons for establishing a statistical agency is to provide information that will allow for an informed citizenry.
|
From page 30... ...
Many federal statistical agencies that can trace their roots back to the 19th or early 20th century, such as the National Center for Education Statistics and the National Center for Health Statistics, were organized in their current form following World War II. Several relatively new agencies have since been established, including the Energy Information Administration, the Bureau of Justice Statistics, and the Bureau of Transportation Statistics.3 In every case, the agency itself, in consultation with users of its in formation, has major responsibility for determining its specific statistical programs and for setting priorities.
|
From page 31... ...
. 5Under the guidance issued for CIPSEA in 2007, OMB has recognized 4 new tatistical s units in addition to the 12 statistical agencies originally recognized (see ppendix A)
|
From page 32... ...
The statistics provide much of the basis on which the government itself is judged. This role places a heavy responsibility on federal statistical agencies for impartiality and objectivity.
|
From page 33... ...
. Many agencies obtain advice from committees that are chartered under the Federal Advisory Committee Act: examples include the Advisory Committee on Agriculture Statistics for the National Agricultural Statistics Service; the Board of Scientific Counselors for the National Center for Health Statistics; the Data Users Advisory Committee and the Technical Advisory Committee for the Bureau of Labor Statistics; and the Scientific Advisory Committee and the National Advisory Committee on Race, Ethnic, and Other Populations for the Census Bureau.
|
From page 34... ...
Links to policies of an agency's parent cabinet department or independent agency that clearly specify the authority that is delegated to the statistical agency also enhance credibility and build trust with users. Principle 3: A federal statistical agency must have the trust of those whose information it obtains.
|
From page 35... ...
Principle 4: A federal statistical agency must be independent from political and other undue external influence in developing, produc ing, and disseminating statistics. A statistical agency must be able to provide credible information that may be used to evaluate the program and policies of its own department or the government as a whole.
|
From page 36... ...
Census Bureau, and departmental appointment of a career civil servant, as is the case for many statistical agencies, are both methods that can bolster the professional independence of an agency head.9 For a fixed term, it is desirable that it not coincide with the presidential term so that professional considerations, rather than political ones, are more likely to be paramount in the appointment process. Appointment by the President with Senate confirmation for a term that is at the pleasure of the President, as is the case for the head of the Energy 8See the Fundamental Principles of Official Statistics of the U.N.
|
From page 37... ...
112-166) , which took effect on August 10, 2012, kept the commissioner of the National Center for Education Statistics and the director of the Bureau of Justice Statistics as presidential appointees but dropped the requirement for Senate confirmation.10 It also provided that the director of the Census Bureau remain a presidential appointee with enate confirma S tion but have a fixed 5-year term (with one renewal permitted)
|
From page 38... ...
Such advice may be sought in a variety of formal and informal ways, but it should be obtained from both data users and providers and from professional or technical experts in the subject-matter area and in statistical methods and procedures. A strong research program in the agency's subject-matter field can assist in setting priorities and identifying ways to improve an agency's statistical programs (Triplett, 1991)
|
From page 39... ...
11Most statistical agencies have such broad authority, limited by budgetary constraints, departmental requirements, OMB review, and congressional mandates.
|
From page 40... ...
. In addition, the courts sometimes become involved in interpreting laws and regulations that affect statistical agencies, as in a number of issues concerning data confidentiality and Freedom of Information Act requests and in the use of sampling in the population census.
|
From page 41... ...
Not even the appearance of manipulation for political purposes should be allowed. This essential requirement is one reason that statistical agencies are required by Statistical Policy Directive Number 3 12See Observations from the Interagency Technical Working Group on Developing a Supple mental Poverty Measure (March 2010:1,3)
|
From page 42... ...
Practice 3: Continual Development of More Useful Data Federal statistical agencies cannot be static. To provide information of continued relevance for public and policy use, they must continually anticipate data needs for future policy considerations and look for ways to evelop data systems that can serve broad purposes.
|
From page 43... ...
Multiple Data Sources Statistical agencies need to continuously think creatively about using multiple data sources in their statistical programs, including such strategies as the use of small-area estimation with auxiliary data to expand the value of surveys without the need for increased sample sizes,14 or the use of administrative records or possibly Internet sources to supplement, calibrate, and even replace data that would otherwise be collected in a survey.15 Statistical agencies are already using these and other strategies to maximize the relevance and cost-effectiveness of one or more statistical programs. However, there is much more that can be done, including the further development of infrastructure and policies for the federal statistical system as a whole to facilitate cost-effective approaches to the design of statistical programs.
|
From page 44... ...
.17 Survey researchers are actively seeking techniques to maintain and improve both the quality and the cost-effectiveness of surveys (see National Research Council, 2013b) , which are and will remain important components of federal statistical programs.
|
From page 45... ...
It is also possible to develop productive relationships with the custodial agency to make them easier to access and use. That the payoffs can be great is illustrated by many examples of current well-established uses of administrative records for statistical programs.
|
From page 46... ...
This use reduces the non sampling error in the survey estimates for key population groups defined by age, gender, race, and ethnicity.19 There are many ways in which administrative records from federal, state, and even local government agencies could be used more extensively in federal statistical programs. As an example, a study by the Committee on National Statistics (CNSTAT)
|
From page 47... ...
In the longer term, it could be cost-effective to use administrative records to replace selected questions in surveys such as SIPP.22 Another method for using survey and administrative records data to improve data quality and relevance is in models that produce improved estimates for specific quantities of interest, such as small-area estimates of school-age children in poverty or of people with health insurance coverage, which use data from the ACS and administrative sources (see National Research Council, 2000c, 2000d)
|
From page 48... ...
of the sources. Roles for Nontraditional Data Sources Statistical agencies are currently exploring the use of data sources in addition to surveys and administrative records that hold promise to improve the relevance, accuracy, and timeliness of federal statistics (see National esearch Council, 2013a)
|
From page 49... ...
More generally, information that is taken from the Internet cannot usually be described or evaluated according to either a probability survey paradigm or a rules-based administrative records paradigm -- for example, people who post items to sell on an auction website do not comprise any specified population. Another challenge is that statistical agencies lack control over the consistency of nontraditional data over time or among vendors or sites, so that deciding to rely heavily on such data sources carries high risks of compromising key time series if a vendor or site then ceases operation or there are marked changes in data content or population coverage.
|
From page 50... ...
was a major achievement, S authorizing the synchronization of business data among the three principal statistical agencies that produce the nation's key economic statistics -- BEA, BLS, and the Census Bureau. The first formal proposal for data synchronization under CIPSEA involved matching data from BEA's international investment surveys with data from the Survey of Industrial Research and Development, which is now the Business Research, Development, and Inno ation Survey (conducted by the Census Bureau for the NSF)
|
From page 51... ...
Historically, because statistical agencies are oriented toward the mission of their particular department, the longitudinal surveys they developed (and cross-sectional data activities as well) typically focused on subject matter and population groups (or other entities)
|
From page 52... ...
For ex ample, the National Center for Education Statistics is assisting states through the Statewide Longitudinal Data Systems (SLDS) program to develop datas ets from administrative records that follow school children through primary and secondary education and even into higher education and the workforce.32 Operational Methods It is important for statistical agencies to be innovative in the methods used for data collection, processing, estimation, analysis, and dissemination.
|
From page 53... ...
. Statistical agencies have turned to the Internet as a standard vehicle for data dissemination and are increasingly using it as a means of data collection.
|
From page 54... ...
. Some statistical agencies developed detailed quality profiles for some of their major series, such as those produced for the American Housing Survey (Chakrabarty, 1996)
|
From page 55... ...
Departments of Agriculture et al., 2002) , individual statistical agencies have developed quality guidelines for their own data collection programs, which are available on the Internet (see Practice 9 and Appendix A)
|
From page 56... ...
A fourth alternative is to allow researchers to analyze restricted microdata at secure sites maintained by a statistical agency, such as one of the Census Bureau's Research Data Centers located at several universities and research organizations around the country or the National Center for Health Statistics' Research Data 34Such a utility is provided by the Data Enclave of NORC at the University of Chicago, which provides secure access by researchers to selected microdata sets of the USDA Economic Research Service, the National Center for Science and Engineering Statistics, and several other federal agencies and private foundations (see http://www.dataenclave.org/index.php/ home [February 2013]
|
From page 57... ...
. Statistical agencies must be vigilant in their efforts to protect against the increased threats to disclosure from their summary data and microdata products while honoring their obligation to be proactive in seeking ways to provide data to users.
|
From page 58... ...
Practice 6: Cooperation with Data Users Users of federal statistical data span a broad spectrum of interests and needs. They include policy makers, planners, government program administrators, members of Congress and their staffs, and researchers in federal agencies, state and local governments, the business sector, and academia.
|
From page 59... ...
If the data are to be used in court cases, administrative proceedings, or collective bargaining negotiations, an agency should have an explicit and publicly known policy for ensuring that all sides may receive the special tabulations, regardless of which side requested them or paid the cost of the tabulation. Practice 7: Respect for the Privacy and Autonomy of Data Providers Clear policies and effective procedures for respecting the privacy of respondents and, more broadly, protecting the rights and respecting the autonomy of human research participants are critical to maintaining the quality and comprehensiveness of the data that federal statistical agencies provide to policy makers and the public.
|
From page 60... ...
In response, statistical agencies have begun 35For a literature review of public opinion on privacy in the wake of the September 11, 2001, terrorist attacks, see National Research Council (2008a:App.
|
From page 61... ...
to determine that the design meets the ethical requirements for protection.36 Some federal statistical agencies consider certain of their information 36For information about the Common Rule and procedures for the certification of IRBs by the Office for Human Research Protections in the U.S. Department of Health and Human Services, see http://www.hhs.gov/ohrp [February 2013]
|
From page 62... ...
Census Bureau, Title 13 of the U.S. Code, first enacted in 1929, and for the National Agricultural Statistics Service, various provisions in Title 7 of the U.S.
|
From page 63... ...
. To give additional weight and stature to policies that statistical agencies had pursued for decades, OMB issued a Federal Statistical Confidentiality Order on June 27, 1997.
|
From page 64... ...
This dilemma is an important one to federal statistical agencies, and it has stimulated ongoing efforts to develop new statistical and administrative procedures to safeguard confidentiality while permitting more extensive access. An effective federal statistical agency will exercise judgment in determining which of these procedures are best suited to its requirements to serve data users while protecting confidentiality.37 Finally, there is a tension between safeguarding confidentiality and departmental initiatives to consolidate data processing, storage, and maintenance as a way to satisfy requirements of the Federal Information Security Management Act (FISMA)
|
From page 65... ...
. An effective agency is also alert to changes in the economy or in society that may call for changes in the concepts or methods used in particular datasets.40 Yet the need for change often conflicts with the 38The data quality guidelines of statistical agencies in other countries are also helpful; for example, see Statistics Canada (2009)
|
From page 66... ...
Practice 10: An Active Research Program Substantive Research and Analysis A statistical agency should include staff with responsibility for conducting objective substantive analyses of the data that the agency compiles, such as analyses that assess trends over time or compare population groups: • Agency analysts are in a position to understand the need for and purposes of the data from a survey or other data collection program and know how the statistics will be used. Such information has to be available to the agency and understood thoroughly if the survey design is to produce the data required.
|
From page 67... ...
. Research on Methodology and Operations For statistical agencies to be innovative in methods for data collection, analysis, and dissemination, research on methodology and operational procedures must be ongoing.
|
From page 68... ...
Several federal statistical agencies sponsor research using academic principles of cognitive psychology to improve the design of questionnaires (see National Research Council, 1984) , the clarity of data presentation, and the ease of use of electronic data collection and dissemination tools such as the Internet.
|
From page 69... ...
• Position responsibilities are structured to ensure that staff have the opportunity to participate, in ways appropriate to their experience and expertise, in research and development activities to improve data quality and cost-effectiveness of agency operations. • Professional activities, such as publishing in refereed journals and presentations at conferences, are encouraged and recognized, including presentations of technical work in progress with appropriate disclaimers.
|
From page 70... ...
An effective statistical agency maintains and develops a sufficiently large number of in-house staff, including mathematical statisticians, survey researchers, and subject-matter specialists, who are qualified to analyze the agency's data and to plan, design, carry out, and evaluate its core operations so that the agency maintains the integrity of its data and its credibility in planning and fulfilling its mission. Statistical agencies should also maintain and develop staff with the expertise necessary for effective technical and administrative management of contractor resources.
|
From page 71... ...
It endeavors in other ways to ensure that its staff are fully cognizant of the ethics that must guide their actions in order for the agency to maintain its credibility as a source of objective, reliable information for use by all. Practice 12: A Strong Internal and External Evaluation Program Statistical agencies that fully follow such practices as continual development of more useful data, openness about sources and limitations of the data provided, wide dissemination of data, commitment to quality and professional standards of practice, and an active research program will likely be in a good position to make continuous assessments of and improvements in the relevance and quality of their data collection systems.
|
From page 72... ...
As discussed in the next section, including other federal statistical colleagues in this communication, both as users and as collaborators, can also be valuable. Statistical agencies commonly find that it is difficult to discontinue or scale back a particular data series, even when it has largely outlived its usefulness relative to other series, because of objections by users who have
|
From page 73... ...
. 44See, e.g., National Research Council (1985a -- study of the statistical programs of the Immigration and Naturalization Service)
|
From page 74... ...
Initiatives for interrelating and synchronizing data among statistical agencies (including individual data and address lists when permitted by law) can be helpful for such purposes as achieving greater efficiency in drawing samples, evaluating completeness of population coverage, and reducing duplication among statistical programs, as well as reducing respondent burden.
|
From page 75... ...
.46 It encourages CNSTAT at the National Research Council to serve as an independent adviser and reviewer of federal statistical activities. The 1995 reauthorization of the Paper ork Reduction Act created a statutory basis for the existing Inter w agency Council on Statistical Policy, formalizing an arrangement whereby statistical agency heads participate with OMB in activities to coordinate federal statistical programs (see Appendixes A and B)
|
From page 76... ...
Such practices improve statistical estimates, reduce costs, and eliminate duplicate requests for information from the same respondents. In other cases, federal statistical agencies engage in cooperative data collection with state counterparts to let one collection system satisfy the needs of both.
|
From page 77... ...
Toward this goal, in 2008 the FCSM established a Subcommittee on Administrative Records, which is working to develop standards and provide guidance to statistical agencies that will facilitate not only use of administrative records, but also evaluation of their quality and fitness to be part of an agency's data collection, estimation, and evaluation programs. This subcommittee has released two products from its work: one is a compilation of case studies of successful statistical uses of administrative data (Federal Committee on Statistical Methodology, 2009)
|
From page 78... ...
Working together in this way, federal statistical agencies contribute to presenting data in a form that is more relevant to policy concerns and to a stronger statistical system overall. Similar collaborative efforts aimed at integrating not only data dissemination, but also data collection and estimation, using traditional and nontraditional data sources, are critically important to improving the relevance, accuracy, timeliness, and cost-effectiveness of the output from the nation's federal statistical system.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.