Studies of Welfare Populations Data Collection and Research Issues (2002) / Chapter Skim
Currently Skimming:
1 Designing Surveys Acknowledging Nonresponse
1 Designing Surveys Acknowledging Nonresponse
Pages 11-54
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 11... ...
Part I Survey Data
|
From page 12... ...
If one were dealing with two subclasses with equal nonresponse rates that hope is equivalent to a hope that the difference terms are equal to one another. This hope is based on an assumption that nonrespondents will differ from respondents in the same way for both subclasses.
|
From page 13... ...
Couper THE NATURE OF NONRESPONSE ERROR IN SURVEY STATISTICS Sample surveys used to describe low-income populations are effective only when several things go "right." The target population must be defined well, having the geographical and temporal extents that fit the goals of the survey. The sampling frame, the materials used to identify the population, must include the full target population.
|
From page 14... ...
After this brief overview of the basic approaches to reducing the impacts of nonresponse on statistical conclusions from the data concludes, the paper turns to reducing the problem of nonresponse. It reviews current theoretical viewpoints on what causes nonresponse as well as survey design features that have been found to be effective in reducing nonresponse rates.
|
From page 16... ...
These two quantities, the nonresponse rate and the differences between respondents and nonrespondents on the variables of interest, are key issues to surveys of the welfare population. Figures 1-la to 1-ld through show four alternative frequency distributions for respondents and nonrespondents on a hypothetical variable, y, measured on all cases in some target population.
|
From page 17... ...
to to Q en o Q z 7 / Yr Ym as . FIGURE 1-lc Low response rate, nonrespondents similar to respondents.
|
From page 18... ...
However, the size of the bias is small because of the low nonresponse rate. Using the same example as above, with a nonrespondent mean now of $501.00, the bias is .05~$201.00 $501.00)
|
From page 19... ...
19 Cal ·_4 ·_4 o .
|
From page 20... ...
for which target population totals are known. For example, if y were a measure of the length of first employment spell of a welfare leaver, and x were a count of sample welfare leavers, X would be a count of the total number of welfare leavers.
|
From page 21... ...
The reader should note that the nonresponse rates for individual subclasses could be higher or lower than the nonresponse rates for the total sample. For example, it is common that nonresponse rates in large urban areas are higher than nonresponse rates in rural areas.
|
From page 22... ...
That is, if nonrespondents tend to be unemployed versus respondents, on average, this will be true for all subclasses in the sample. If the nonresponse rates were not equal for the two subclasses, then the assumptions of canceling biases is even more complex.
|
From page 23... ...
However, when the response rates of the two subclasses are different, large biases can result. Larger biases in the difference of subclass means arise with larger differences in nonresponse rates in the two subclasses (note the higher absolute value of the bias for any given [ Y-r - Y-m]
|
From page 24... ...
Even in the presence of nonresponse, the nonresponse bias of regression coefficients may be negligible if the model has a specification that reflects all the causes of nonresponse related to the dependent variable. Consider a survey in which respondents differ from nonrespondents in their employment status because there are systematic differences in the representation of different education and race groups among respondents and nonrespondents.
|
From page 25... ...
Postsurvey Compensation for Nonresponse Two principal techniques are used to account for unit nonresponse in the analysis of survey data: weighting and imputation. In computing final statistics, weighting attempts to increase the importance of data from respondents who are in classes with large nonresponse rates and decrease their importance when they
|
From page 26... ...
, Medicaid, and foster care payment receipt, it would be possible to use those variables as adjustment factors. The ideal adjustment factors are those that display variation in response rates and variation on key survey statistics.
|
From page 27... ...
This is not always so. If the adjustment variables were related to response rates but not to the survey variable, then adjustment would do nothing to change the value of the survey statistic.
|
From page 28... ...
The second term reflects the lack of control of the allocation of the sample across the weight classes; this is the term that creates the loss of precision (as well as the fact that the total sample size is reduced from n to Wry, where (~;rh/n) is the response rate.)
|
From page 29... ...
The propensity of locating units is driven by factors related to whether or not the unit moves and the quality of contact information provided at the time of initial data collection. A number of survey design features may affect the likelihood of locating sample units.
|
From page 30... ...
Their lives are more public and accessible through multiple databases such as telephone directories, credit records, voter registration, library registration, membership in churches or religious organizations, or children in schools. Again, we expect that sample units in welfare leaver studies are not particularly rich in these sources of tracking information.
|
From page 31... ...
In telephone surveys, the impediments include "caller ID," "call blocking," or answering machines that filter or restrict direct contact with the household. In most surveys the interviewer has no prior knowledge about the at-home behavior of a given sample household.
|
From page 32... ...
The only information about at-home practices of a sample household is obtained by calling the number. (This imbalance leads to the larger number of calls required to make first contact with a household in telephone surveys; see Groves and Kahn, 1979.)
|
From page 33... ...
Other analysis shows that the exception to this tendency is single-person households with elderly persons, which tend to be home more often than other households. Figure 1-7 shows a similar result for an access impediment in telephone surveys, the answering machine, which now is present in more than 50 percent of homes nationwide (Tucker and O'Neil, 1995~.
|
From page 34... ...
. The Decision to Participate in a Survey Once the interviewer contacts a sample household we believe that the influences on the householder's decision to participate arise from relatively stable features of their environments and backgrounds, fixed features of the survey design, as well as quite transient, unstable features of the interaction between the interviewer and the householder.
|
From page 35... ...
SOURCE: Groves and Couper (1998~. 1-8, which lists influences of the social environment, householder, survey design features, interviewer attributes and behavior, and the contact-level interaction of interviewers and householders.
|
From page 36... ...
This contrast has been commonly observed in part because the urbanicity variable is often available from the sampling frame. The nature of urbanicity effects on response rates has been found to be related to crime rates (House and Wolf, 1978)
|
From page 37... ...
Other factors associated with these also have been studied for their relationship to response rates. These include household structure and characteristics, such as the number and ages of the household members and the quality and upkeep of housing, and the experience of the respondent, such as exposure to situations similar to the interview interaction or a background that provided information or training relevant to the survey topic.
|
From page 38... ...
We have found concepts of reciprocation and authority very important to understanding the behavior of sample persons. In addition, however, four other compliance heuristics described by Cialdini (1984)
|
From page 39... ...
Calls by interviewers to sample households generally are unscheduled events. The amount of discretionary time perceived to be possessed by the householders at the time of contact also will affect their tendency to engage in deliberate, careful consideration of the arguments to participate in the survey.
|
From page 40... ...
This section discusses those features in an indirect manner, by identifying and elaborating the concepts that underlie their effectiveness. Many of the survey design features aimed at gaining cooperation use one or more of the compliance heuristics reviewed earlier.
|
From page 41... ...
The use of race or gender matching by survey organizations may be an attempt to invoke liking through similarity, as well as reducing the potential threat to the householder. Other survey design features do not fit nicely into the compliance heuristics conceptualized by Cialdini.
|
From page 42... ...
Respondent-laterviewer Interaction When interviewers encounter householders, the factors discussed come to bear on the decision to participate. The strategies the interviewer employs to persuade the sample person are determined not only by the interviewer's own ability, expectations, and other variables, but also by features of the survey design and by characteristics of the immediate environment and broader society.
|
From page 43... ...
Experienced interviewers often report that they adapt their approach to the sample unit. Interviewers engage in a continuous search for cues about the attributes of the sample household or the person who answers the door, focusing on those attributes that may be related to one of the basic psychological principles reviewed previously.
|
From page 44... ...
The system is driven by number of interviews per time unit. The second goal, maximizing the probability of obtaining an interview from each sample unit, is the implicit aim of probability sample interviewing.
|
From page 46... ...
We note how similar the goals of a quota sample interviewer are to those of any salesperson, but how different are those of the probability sample interviewer. Given this, it is not surprising that many attempts to use sales techniques in probability sample surveys have not led to large gains in cooperation.
|
From page 48... ...
no single survey design will achieve 100-percent response rates and defenses to nonresponse error should be built into the chosen survey design. The Value of Rich Sampling Frames The list of the target population (or the materials used to construct a list)
|
From page 49... ...
The intent under perfect conditions is that a probability subsample of nonrespondents to the first phase of the survey can yield evidence regarding the likelihood of large nonresponse errors in the first-phase estimates. The "perfect" conditions yield 100 percent response rates on the second-phase cases, thus providing unbiased estimates of the characteristics of the nonrespondent pool.
|
From page 50... ...
Sample surveys suffer from the tyranny of the measurable, with sampling errors dominating design decisions because they can be measured more easily than nonresponse errors. It is tempting to assume the absence of nonresponse error and to maximize sample size to achieve low reported sampling errors.
|
From page 51... ...
Sample persons with prior experiences or relationships with the sponsoring agency for the survey make decisions based partially on how they evaluate those relationships. This may underlie the tendency for those persons dependent on programs to respond at higher levels.
|
From page 52... ...
The higher the nonresponse rate, the higher the risk of nonresponse error, other things being equal. With higher than desired nonresponse rates, the investigators have an obligation to assure themselves that major nonresponse errors are not present, damaging their ability to draw conclusions from the respondentbased statistics.
|
From page 53... ...
Groves 1995 Introductions in Telephone Surveys and Nonresponse. Unpublished paper presented at the Workshop on Interaction in the Standardized Survey Interview, Amsterdam.
|
From page 54... ...
Feinberg 1991 The answering machine poses many questions for telephone survey researchers. Public Opinion Quarterly 55(2)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.