Realizing the Potential of the American Community Survey Challenges, Tradeoffs, and Opportunities (2015) / Chapter Skim
Currently Skimming:
2 Sample Design and Precision of Estimates
2 Sample Design and Precision of Estimates
Pages 23-45
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 23... ...
A detailed discussion of the ACS sample design can be found in the American Community Survey Design and Methodology Report (U.S. Census Bureau, 2014a)
|
From page 24... ...
The comparison captures the contributions of several design features to the dramatic differences in precision between the two data collections. First, the sample size from the ACS 5-year aggregate is only 54 percent of the 2000 cen TABLE 2-1 Design Characteristics of the 2000 Census Long-Form Sample and 2007-2011 ACS Samples 2007-2011 Characteristics 2000 Census American Community Survey Total Sample Size 43,961,414 23,781,823 Number of Tracts 65,746 73,146 Median Tract Sample Size 605 296 Median Tract Design Effect 1.12 1.41 Median Tract Effective Sample Size 533 209 NOTES: There are sample size data for only 72,234 of the 73,146 tracts because a minimum of 50 unweighted sample cases for a subpopulation in a geographic area is required to release a data table.
|
From page 25... ...
Tables 2-2 and 2-3 provide a more detailed overview of the precision and usability of five of the Census Bureau's key variables, using 2008-2012 ACS data. Table 2-2 shows that the median tract-level CV for each of these five key variables meets the Census Bureau's quality threshold (less than or equal to 30 percent)
|
From page 26... ...
FIGURE 2-2 Effective sample sizes in census tracts. NOTE: Only tracts that have a minimum of 50 unweighted sample cases are included.
|
From page 27... ...
× 100%) . Furthermore, data users cannot calculate a meaningful standard error with the standard methods suggested by the Census Bureau if a published estimate is zero or if a derived percentage estimate is 100 percent because in those cases the conventional standard error estimate would be zero, suggesting an implausible degree of certainty 4 Consistent with Census Bureau tradition, ACS estimates are published with 90 percent confidence levels, although ACS data users can convert the published MoEs to the MoEs for a higher confidence level, if needed.
|
From page 28... ...
TABLE 2-2 Usability Analysis: Tract-Level 2008-2012 ACS Data 28 (in percentage) Percentage of Percentage of Percentage of Households Population Percentage of Occupied Housing Receiving SNAP Percentage of Aged 25+ with Persons Living in Units That Are Renter Benefits in Persons Who Are a Graduate or Characteristics Povertya Occupiedb Past 12 Monthsc Foreign Born Professional Degreed National Estimate 14.9 34.5 11.4 12.9 10.6 National Margin of ±0.1 ±0.2 ±0.1 ±0.1 ±0.1 Error All Tractse Tract range 0-100 0-100 0-100 0-100 0-100 Tract mean 16.0 35.2 12.5 12.2 10.3 Tract median 12.6 29.8 9.2 6.9 7.2 Tract median CV 28.3 14.7 29.5 29.7 26.0 Tract median MoE 5.8 6.7 4.6 3.4 3.2 CV ≤ 30% 56 88 51 50 61 CV > 30 and 35 9 29 29 27 ≤ 50% CV > 50% 9 2 20 21 12 Confidence interval 59 80 45 30 17 ≥ 10 percentage points MoE > estimate 4 1 16 15 8 Confidence interval 4 1 16 15 8 lower bound < 0
|
From page 29... ...
tracts) Confidence Interval Width ≥ 10 Percentage Points by CV Category, Excluding Tracts Where CV = 0 CV ≤ 30% 73 84 69 48 24 CV > 30 and ≤ 50% 46 58 33 19 9 CV > 50% 30 32 11 8 6 Median CVs by Tract Total Population Size All Tracts < 1,000 31.3 18.7 27.2 43.6 37.6 1,000-2,999 29.0 16.0 29.6 38.6 31.4 3,000-4,999 28.1 14.3 29.3 29.6 25.9 5,000-6,999 28.2 14.1 29.8 25.2 23.5 7,000+ 27.8 14.4 29.5 20.7 20.8 29 Continued
|
From page 30... ...
TABLE 2-2 Continued 30 Percentage of Percentage of Percentage of Households Population Percentage of Occupied Housing Receiving SNAP Percentage of Aged 25+ with Persons Living in Units That Are Renter Benefits in Persons Who Are a Graduate or Characteristics Povertya Occupiedb Past 12 Monthsc Foreign Born Professional Degreed Median CV by Tract Number of Housing Units All Tracts ≤ 400 32.6 12.0 23.7 29.8 42.9 401-1,000 31.6 18.0 31.8 34.1 35.5 1,001-2,000 28.4 14.9 29.2 30.9 27.4 2,001-4,000 27.4 13.6 29.1 28.1 22.7 4,001-6,000 27.2 13.4 31.8 20.8 17.0 6,001+ 26.4 13.7 40.4 16.0 14.2 aTracts with no persons in the universe for which poverty rates are calculated were excluded from this analysis. bTracts with no households were excluded from this analysis.
|
From page 31... ...
Table 2-2 illustrates that criteria using these two measures may result in opposite conclusions about the precision of ACS estimates. For example, the share of tracts with a CV below or equal to 30 percent, but a confidence interval width that exceeds 10 percentage points, ranges from a low of 24 percent to a high of 84 percent for the five key variables in Table 2-2.
|
From page 32... ...
TABLE 2-3 Distribution of Tracts by Variable Categories and CV Categories: 2008-2012 ACS Data (in percentage) 32 People in Poverty CV ≤ 30% CV > 30% and ≤ 50% CV > 50% Percentage of Total ≤ 3% 1 43 56 8 > 3% and ≤ 10% 25 62 13 32 > 10% and < 20% 66 32 2 31 ≥ 20% and < 30% 89 10 0.6 15 ≥ 30% and < 97% 96 3 0.8 13 ≥ 97% 0 0 0 0 Renter Occupied CV ≤ 30% CV > 30% and ≤ 50% CV > 50% Total ≤ 3% 0.2 17 83 1 > 3% and ≤ 10% 25 61 14 8 > 10% and < 20% 83 16 0.7 21 ≥ 20% and < 30% 98 2 0.2 20 ≥ 30% and < 97% 99 0.3 0.1 49 ≥ 97% 100 0 0 0.5 Households Receiving SNAP Benefits CV ≤ 30% CV > 30% and ≤ 50% CV > 50% Total ≤ 3% 0.4 14 85 16 > 3% and ≤ 10% 21 62 18 35 > 10% and < 20% 77 22 1 28 ≥ 20% and < 30% 97 3 0.2 12 ≥ 30% and < 97% 99 0.7 0.3 9 ≥ 97% 0 0 0 0
|
From page 33... ...
People Who Are Foreign Born CV ≤ 30% CV > 30% and ≤ 50% CV > 50% Total ≤ 3% 3 35 61 28 > 3% and ≤ 10% 37 51 12 32 > 10% and < 20% 81 17 2 19 ≥ 20% and < 30% 96 3 0.4 9 ≥ 30% and < 97% 99 0.6 0.2 12 ≥ 97% 0 0 0 0 Population Aged 25+ with a Graduate or Professional Degree CV ≤ 30% CV > 30% and ≤ 50% CV > 50% Total ≤ 3% 4 38 58 17 > 3% and ≤ 10% 52 43 5 46 > 10% and < 20% 94 6 0.5 23 ≥ 20% and < 30% 99 1 0.4 9 ≥ 30% and < 97% 98 0.7 0.8 5 ≥ 97% 100 0 0 0 NOTES: Table excludes tracts where CV = 0. CV is coefficient of variation.
|
From page 34... ...
EFFORTS TO IMPROVE PRECISION Over the years, data users have been raising concerns about the limited precision of tract-level estimates described above, and the Census Bureau has been exploring ways to address the issue. For example, model-assisted estimation was studied as a mechanism for using administrative records information to improve the precision of ACS estimates at the subcounty level and for tracts in particular (Fay, 2006; Starsinic and Tersine, 2007)
|
From page 35... ...
The old and new sampling plans for census tracts are compared in Table 2-5. The samples reallocated to the smaller sized areas mostly come from large tracts with 4,000 or more housing units, which in general had smaller than average CVs before the reallocation.
|
From page 36... ...
In particular, differential sampling rates at the tract level imply that heterogeneity is introduced when tracts with different sampling rates are combined to create larger geographic areas for analysis. Relative to a sample design with proportional sampling (i.e., uniform sampling rates)
|
From page 37... ...
We chose counties to represent governmental units because they are clearly defined in census data, they are commonly used in federal resource allocation and Census Bureau estimation programs, and counties with small populations are likely to contain small governmental units that also have small populations. Results by quintiles are presented in Table 2-6.
|
From page 38... ...
38 TABLE 2-6 Distribution of Tract Sizes Relative to the Size Distribution of Counties For Each County Quintile, the Percentage of Tracts Belonging to Each Tract Quintile County Quintiles by Population Size 1st 2nd 3rd 4th 5th Tract Quintiles 58- 74,642- 251,644- 627,363- 1,408,481- Total Number of by Population Size 74,641 251,643 627,362 1,408,480 9,787,514 Tracts 1st 1-2,645 23.1% 18.6% 20.9% 19.8% 17.6% 14,639 2nd 2,646-3,567 22.3% 18.9% 19.7% 20.4% 18.7% 14,645 3rd 3,568-4,463 20.9% 19.4% 19.1% 19.9% 20.7% 14,646 4th 4,464-5,673 18.3% 20.4% 19.7% 20.5% 21.1% 14,639 5th 5,674-36,880 15.4% 22.6% 20.7% 19.4% 22.0% 14,646 Total Number of Tracts 14,651 14,660 14,666 14,861 14,377 73,215
|
From page 39... ...
Furthermore, using counties as a proxy for potential benefit of tract equalization to small governmental units, Table 2-6 provides little support for such a position, given that tract sizes appear to be fairly uniformly distributed across various county sizes, including small counties. Beyond the increased sampling rates, the lower CV outcome associated with the smallest governmental units (see Table 2-6)
|
From page 40... ...
That is, tracts still need to be combined in some fashion to produce sufficiently precise estimates because of small sample sizes, but the increases in variances associated with larger geographies from this tract CV smoothing effort is potentially counterproductive and not dealing with the real challenges being faced by ACS data users. The next section presents a case study to illustrate these CV equalization sample design issues and implications.
|
From page 41... ...
The median CV increase for married-couple families was only 0.8 percent, with a maximum of about 2.1 percent. This result is a reflection of the fact that under the old design the sampling rates and resulting final weights of combined tracts were not different enough 5 Neighborhood tabulation areas were created as aggregates of whole census tracts, with a minimum population requirement of 15,000.
|
From page 42... ...
At the 75th and 90th percentiles, the estimated increases to the CVs are 6.5 and 9.1 percent, respectively. Furthermore, there are several neighborhood tabulation areas where the differential sampling rates associated with the new design will result in CVs that are larger by 12 percent or more than they would have been under the old design: see Figure 2-5.
|
From page 43... ...
Emphasis has always been placed on maintaining boundaries of census tracts over time for comparability purposes. The overall picture by neighborhood tabulation areas in New York City shows that in most areas of the city there was a net gain in the precision of estimates due to increases in sample size, but there are a number of notable exceptions that point to the risks inherent in summary statistics.
|
From page 44... ...
However, the level of precision is high enough in comparisons with other neighborhood tabulation areas, and thus suitable for policy development and program planning and implementation. FIGURE 2-6 Effects of differential sampling rates and sample size on CVs for number of married-couple families, New York City neighborhood tabulation areas, ACS 2006-2010.
|
From page 45... ...
RECOMMENDATION 3: Efforts to improve the precision of the American Community Survey estimates for specific small governmental units should be focused on increasing the initial designated sample size while maintaining the optimal nonresponse sampling rate instead of increasing the subsampling rate to 100 percent.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.