Envisioning the 2020 Census (2010) / Chapter Skim
Currently Skimming:
3 Census Bureau Research, Past and Present
3 Census Bureau Research, Past and Present
Pages 59-94
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 59... ...
; Chapter 4 turns to the practical issues of structuring and scheduling research in advance of the decennial census. Section 3–A presents our critique of current and recent trends by the Census Bureau in its research programs; it builds from the context provided by two detailed appendices at the end of this report.
|
From page 60... ...
panel that had coverage measurement research as its explicit charge, which is not the case for our panel. Given our charge, we have focused our own analysis principally on research and development related to census operations; accordingly, it is important to note that our comments on "census research" in what follows should not be interpreted as applying to the Census Bureau's extensive body of coverage measurement research.
|
From page 61... ...
As the quotation from the predecessor panel suggests, our critique applies to Census Bureau research writ larger, including the census tests conducted between 2000 and 2010 and the evaluations and experiments of the 2000 census. 3–A.1 Legacy of Research In Appendix A, we outline the precensus testing activities and formal research, experimentation, and evaluation programs of the 1950–2000 censuses.
|
From page 62... ...
Based on the successful use of the mailout questionnaire in 1960, Congress enacted a brief but powerful amendment to census law in 1964: P 88-530 struck .L. the requirement that decennial census enumerators must personally visit every census household.
|
From page 63... ...
3–A.2 Flaws in Current Census Research and the 2010 CPEX In this section, we briefly describe the principal deficiencies that we observe in the Census Bureau's current approach to research and in the 2010 CPEX in particular. In our assessment, shortcomings in the Census Bureau's research strategy need to be overcome immediately in order to foster an effective research program for the 2020 census.
|
From page 64... ...
Finally, we will mount a post-hoc administrative records census, using administrative records available to the Census Bureau. All of this will better position us for the developmental work we must conduct to improve future decennial census operations.
|
From page 65... ...
• As we noted in our letter report, the Confidentiality/Privacy Notifi cation Experiment is, if anything, contrary to the goals of reducing cost and improving quality. Its paragraph treatment raises the possi bility of mixing census information with data from other government agencies -- i.e., the use of administrative records in census processes -- in an ominous manner.
|
From page 66... ...
The recent cycle of mail-only tests in years ending in 3 or 5, tests involving a field component in years 4 and 6, and a dress rehearsal in year 8 has the advantage of keeping the various parts of census processing in fairly constant operation, so that there is no need to completely rebuild field operations from scratch. But a too-strong focus on these single-shot testing slots has led to poor design choices.
|
From page 67... ...
Try though the tests do to create census-type conditions, localities have little incentive to provide unfettered support other than a sense of civic duty. Finally, the shift in recent years to omnibus tests has created another fundamental flaw in Census Bureau research: almost of necessity, the tests can not build from each other.
|
From page 68... ...
The problem of confounding is well known, yet there are examples of experiments carried out by the Census Bureau, either during the decennial census or during large-scale census tests, in which the experiments have generated very uncertain results due to the simultaneous varying of design factors in addition to those of central interest. For example, the ad hoc Short Form Mail Experiment in 2006 (see Section A–7)
|
From page 69... ...
The choices in the 2010 CPEX seem more suggestive of a "bottom-up" approach -- looking at highly specific parts of the census process and making small adjustments -- than a more visionary "top-down" approach that takes major improvement in the cost-effectiveness of the census (and such wholesale change in operations as is necessary to achieve that improvement) as a guiding principle.3 Such a top-down approach would be predicated on alternative visions for the conduct of a census -- general directions that might be capable of significant effects on census cost or quality.
|
From page 70... ...
goal of the C2SS as an experiment was simply to demonstrate that the Bureau can field the decennial census and a large survey simultaneously. The AREX 2000 experiment was a very useful first step in suggesting the use of administrative records in the census process but, arguably, was still focused too heavily on the potential of administrative records as a replacement for the census (i.e., do counts and distributions match)
|
From page 71... ...
was complemented in the 2000 census by the use of optical character recognition of handwritten responses; indeed, major pieces of postcensus coverage evaluation work made use of the first-time automated capture of handwritten name information. Envisioning the development of handheld computers for use in major census operations, the 2010 census promised to make major advances in the
|
From page 72... ...
. The panel recognized that the 2010 census plan included many major system overhauls -- not only the development of the handheld computers, but also the establishment of a parallel data system with the American Community Survey and retooling of the Census Bureau's geographic resources.
|
From page 73... ...
of housing units. Indeed, a final set of requirements for the devices was only developed between November 2007 and January 2008, and the resulting cost estimate from the contractor as to how expensive it would be to meet those requirements precipitated the "replan." The failures of the handheld development -- coupled with the Census Bureau's decision to forbid Internet response to the 2010 census, despite having offered online response (albeit unadvertised)
|
From page 74... ...
As we have already noted, our review of the testing and experimentation programs of preceding decennial censuses makes it clear that the Census Bureau used to make much greater use of smaller, focused testing activities, and also used to make greater use of other survey vehicles to test changes that might ultimately be adopted for the census. By comparison, the more recent rounds of census research seem to assume that lessons from small experiments, from general survey research, and from foreign censuses and surveys are somehow inapplicable.
|
From page 75... ...
Overcoming this insularity -- and more effectively building from external researchers and international peers -- is a key part of improving Census Bureau research. 3–B KEY STEPS IN RETHINKING THE CENSUS BY RETHINKING RESEARCH Having critiqued the current state of Census Bureau research, we now turn to suggestions for improvement over the coming decade.
|
From page 76... ...
.5 We accept this list for what it is -- a preliminary first cut -- but the first point we make on restructuring census research is related to this listing. The list of objectives contains many good points (although they do sometimes confuse true objectives with the specific tools or procedures intended to achieve those objectives)
|
From page 77... ...
Likewise, the phrase "administrative records census" -- in itself -- is not really a vision; at least an additional level of detail on how (and how well) administrative data might apply to census operations would be necessary to flesh out the idea and make it a tractable model to consider.
|
From page 78... ...
Clearly, the Census Bureau has in place cost models that it uses to develop budget estimates and allocate resources. What is fundamentally unclear is how good those cost models are -- how sensitive they are to varying assumptions, how transparent they are in breaking down costs by component operations, and how flexible they are to estimating the costs of major changes to census operations.
|
From page 79... ...
Like previous National Research Council panels, we think that a master trace sample that saves and links data for a sample of addresses, respondents, and cases through all steps of census processing would be an invaluable tool for providing empirical insight for intercensal testing. The broader notion advanced by the Panel on Research on Future Census Methods of a master trace system -- a technical information infrastructure designed in such a way as to automatically and naturally retain virtually all operational data for later reanalysis -- is a particularly attractive one.
|
From page 80... ...
This group would have the responsibility of retaining and documenting suf ficient data from the 2010 census to be able to comprehensively represent the functioning of all census operations. Such a group would also have the responsibility of assisting Bureau research staff, using current database management tools, to produce re search files to support the assessment of analytic questions con cerning aspects of the 2010 census.
|
From page 81... ...
, the address canvassing operation, and the update-leave operation in 2010, among other sources; • Information from the 2010 Census Coverage Measurement opera tions, including whether addresses were flagged as including omitted or duplicated persons (or whole households) , nonexistent or nonresi dential structures, or erroneously geocoded entries; • Snapshots of Census Bureau–compiled administrative records databases, such as the Statistical Administrative Records System de scribed below; • Derived variables about the nature of the housing unit (or structure)
|
From page 82... ...
One of the CPEX evaluations -- comparing the re sults of detecting whole structures using aerial photography with MAF entries and other sources -- may be a useful part of a broader research program in geographic updating. Generally, the Census Bureau would benefit from a program of field spot-checks, comparison with third party sources (including addresses drawn from administrative records data files)
|
From page 83... ...
, so that the quality and unique contribu tion of local update sources can be assessed and areas with particular need for updating can be identified. • Integration with the American Community Survey field staff: Another original plank of the MTEP was what was known as the Community Address Updating System (CAUS)
|
From page 84... ...
. • Interface with the commercial sector: Just as it is important to com pare the address coverage from compiled administrative records files with the existing MAF, it would also be worthwhile to study the qual ity and coverage of commercially available mailing lists (even if such commercial lists do not become an input source to the MAF)
|
From page 85... ...
As a bold statement of mission, we encourage the Census Bureau to go further than to think of simply getting the development process of handheld computers for NRFU in shape. Rather, we suggest a broader examination of all steps in the census process with the public, stated goal of making the 2020 census as "paperless" as is practicable.
|
From page 86... ...
It also follows that movement toward census processes that are highly automated and as paperless as possible heightens the importance of ensuring that those processes have an audit trail -- that they include outlets for retention, archival, and analysis of operational data such as we recommend for the 2010 census in Section 3–B.3. Having already described many of the points raised in that report, we do not expound on them further.
|
From page 87... ...
Sheer numeric counts aside, the quality and timeliness of administrative records data for even short-form data items, such as race and Hispanic origin and relationship within households, remain open and important questions. Wider use of administrative records in the census also faces formidable legal hurdles, not the least of which are inherent conflicts between the confidentiality and data access provisions in census law (Title 13 of the U.S.
|
From page 88... ...
Particularly as the idea of using administrative records in a va riety of census operations (such as geographic resource updates) is con sidered, the Census Bureau should explore the quality and availabil ity of data files maintained by state and local governments, including local property files, records for "E-911" conversion from rural non city-style addresses to easier-to-locate addresses, and state and county assessors' offices.
|
From page 89... ...
• Examining relative quality of "last resort" enumeration: In those cases in which contact simply cannot be made, the relative quality of dif ferent options for filling the blanks -- for example, proxy information, imputation, and use of administrative records -- should be quantified and evaluated. • Quality of interviews as a function of time from Census Day: It is gen erally well understood that follow-up interviews (as well as indepen dent interviews such as the postenumeration survey that is the heart of coverage measurement operations)
|
From page 90... ...
Panel on Residence Rules in the Decennial Census discussed a wide range of research ideas under the general heading of residence; these generally remain as applicable to the 2010– 2020 planning period as they were to 2000–2010. In terms of questionnaire design, these include further research on replacing the current instructionheavy approach to the basic household count question with a set of smaller, more intuitive questions and more effective presentation of the rationale for the census and specific questions.
|
From page 91... ...
In partic ular, its research program should give some thought to studying the effects on response and cooperation by including questions on citizen ship or immigration status. The arguments that such questions could seriously dampen response and hurt the image of the decennial census as an objective operation are straightforward to make and, we think, are basically compelling, but empirical evidence is important to build ing the case.
|
From page 92... ...
• Change to the unit of enumeration: Since the act authorizing the 1790 census required the count to be counted at their "usual place of abode," the decennial census has used the household as its basic unit of enumeration. In the modern census context, this has involved associating households with specific addresses and, through those ad dresses, with specific geographic locations.
|
From page 93... ...
Going forward, the capacity of the ACS as a unique survey platform in its own right must be explored, including ways for the census and the ACS to support each other: for example, use of parts of the ACS sample as a test bed for experimental census concept and questions.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.