Skip to main content

Currently Skimming:

5 Adjusting for Missing Data in Low-Income Surveys
Pages 129-156

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 129...
... The following sections indicate how missing data can lead to biased survey estimates and describe some widely used methods to reduce this effect. Missing data in surveys can be divided usefully into three classes: · Noncoverage.
From page 130...
... Because some state surveys have experienced high nonresponse rates, nonresponse weighting adjustments are likely to be particularly important.The intent of this paper is to describe how they may be applied. All methods for handling missing data aim to reduce their potential biasing effects, but these methods cannot be expected to eliminate the effects of missing data.
From page 131...
... The standard method of attempting to reduce the potentially biasing effect of noncoverage and of unit nonresponse is a "weighting adjustment." Weighting adjustments for these two sources of missing data are described in this paper. Because some state surveys have experienced high nonresponse rates, nonresponse weighting adjustments are likely to be particularly important.]
From page 132...
... Surveys usually collect information from respondents that is not available in administrative data. Both types of studies of low-income populations usually suffer from some form of missing data.
From page 133...
... However, the target population is not completely represented by the sample when either some persons or families are not included in the sampling frame (e.g., the administrative records if used for sampling) or information cannot be obtained for some eligible sampled persons.
From page 134...
... Although the population of interest is all individuals receiving benefits because of substance abuse, assume that the two states (State A and State B) used different frames for sample selection; State A used the RMA client rosters, which covered only 66 percent of the target population, and State B used the Social Security Administration client roster, which was a complete frame.
From page 135...
... The marginal population distributions of demographic variables age, gender, and race usually are available, and these can be used to examine the potential for noncoverage and nonresponse biases, as shown in the following text. Tables 5-1 and 5-2 provide the marginal distributions of the demographic variTABLE 5-1 Hypothetical Population and Sample Distribution for State A SSI Population (%)
From page 136...
... Nonresponse Bias The size of the nonresponse bias depends on the amount of nonresponse and the difference between the respondent and nonrespondent mean values of the study variables. For example, in an equal probability sample (a description of an unequal probability sample is provided in the section on base weights)
From page 137...
... Adjustment factors typically vary among demographic groups, and their purpose is to establish a data set whose sample representation has been adjusted to compensate for the missing nonrespondent data. (We used the term "demographic groups" because race, age, TABLE 5-3 Level of Bias by Nonresponse Rate and Differences in Average Income of Respondents and Nonrespondents Example 1 Sample Size Average Income Example 2 Sample Size Example 3 Average Income Sample Size Average Income Respondents 600 $1,500 410 $1,500 600 $2,000 Nonrespondents 220 $1,100 410 $1,100 220 $1,100 Survey estimate $1,500 $1,500 $2,000 with no nonresponse adjustment Estimated $1,393 $1,300 $1,759 population value Bias $ 107 $ 200 $ 241 NOTE: The data used for the Family Income Survey (FIS)
From page 138...
... The nonresponse adjustment factors are incorporated into the survey weights. The next section reviews the properties of sampling weights in preparation for the discussion of nonresponse adjustment procedures.
From page 139...
... If the sample units are selected with equal probability, the probability of selection is hi = nlN for all sample units, where n is the sample size and N is the number of units in the sampling frame. The base weight, therefore, is wi = N/n for all sampled units.
From page 140...
... The following section provides brief descriptions of various weight adjustment procedures commonly used in large-scale surveys.
From page 141...
... The first step in adjusting for nonresponse is the construction of weighting classes. As discussed in the following text, within each weighting class, the base weights are inflated by the inverse of the response rate so that the sum of the adjusted base weights for respondents is equal to the sum of the base weights for the total eligible sample selected in the weighting class.
From page 142...
... Construction of Nonresponse Adjustment Classes Implementing nonresponse adjustment procedures requires the specification of appropriate weighting classes or cells. Survey responses generally are correlated with certain characteristics of the sample units, and it would be desirable to form classes based on these characteristics.
From page 143...
... If a substantial amount of data are missing for an item on the sampling frame, this variable is probably not appropriate for the purpose of nonresponse adjustments. The variables used to form weighting classes should be effective in distinguishing between subgroups with different response rates.
From page 144...
... We discuss two commonly used procedures (referred to as modeling response propensity) for defining weighting classes using data on auxiliary variables.
From page 145...
... In effect, sample-based nonresponse adjustments distribute the base weights of the nonresponding units to the responding sampled units so that the sum of the adjusted weights over the responding units equals the sum of the base weights for the entire sample. The basic form of the sample-based nonresponse adjustments is a ratio of sums of base weights where the sums extend over specified subsets of the sample defined by response status.
From page 146...
... The procedures for forming appropriate weighting classes for this purpose were discussed earlier. Table 5-5 shows the nonresponse adjustment factors and adjusted weights for the FIS example.
From page 147...
... In this case, the base weights would be ratio adjusted directly to known control totals in a single step. For example, if the classes used for nonresponse adjustment also are used for population-based adjustments, the two-step procedure of first adjusting for nonresponse and then adjusting to known control totals is equivalent to the single population-based adjustment procedure discussed in this section.
From page 148...
... Because the base weights were adjusted to account for the nonresponse (as given in Table 5-5) , these adjusted weights would vary by poststratified adjustment classes.
From page 149...
... ment factors, defined as the ratio of the known family count and the survey estimate. The final survey weights are defined as the product of the base weight and the adjustment factors for nonresponse and posts/ratification.
From page 150...
... Raking Procedure This methodology is referred to as raking ratio estimation because an iterative procedure is used to produce adjustment factors that provide consistency with known marginal population totals. Typically, raking is used in situations where the interior cell counts of a cross-tabulation are unknown or the sample sizes in some cells are too small for efficient estimation (refer to the following section for more information about sufficient cell sample size)
From page 151...
... 151 o u, = ca ~ o o ^ z ~ sit ~ ca ~ *
From page 152...
... Thus, the application of weighting adjustments usually results in lower bias in the associated survey statistics, but at the same time adjustments may result in some increases in variances of the survey estimates. The increases in variance result from the added variability in the sampling weights due to nonresponse and noncoverage adjustments.
From page 153...
... many weighting classes are created with a few respondents in each class, and (2) some weighting classes have very large adjustment factors (possibly due to much higher nonresponse or noncoverage rates in these classes)
From page 154...
... However, it is also very important to pay attention to nonresponse rates for each item in the questionnaire, and data analysts should consider using imputation procedures to compensate for missing items in the state surveys. As discussed earlier, studies of the low-income population usually suffer from missing data.
From page 155...
... The impact of nonresponse bias is usually small in surveys with low nonresponse rates when nonresponse-adjusted weights are used along with the survey data. Although sample weighting cannot take all differences between respondents and nonrespondents into account, the weighting cells that are usually used appear, in general, to reduce the effect of any potential differences between respondents and nonrespondents.
From page 156...
... Alexan dria, VA: American Statistical Association. 1990 A study of procedures to identify and trim extreme sampling weights.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.