Understanding American Agriculture Challenges for the Agricultural Resource Management Survey (2008) / Chapter Skim
Currently Skimming:
6 Nonresponse, Imputation, and Estimation
6 Nonresponse, Imputation, and Estimation
Pages 107-126
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 107... ...
Nonresponse is readily quantifiable, and, possibly as a result, the National Agricultural Statistics Service (NASS) has concluded that unit nonresponse is the survey's biggest source of nonsampling error.1 In this section we address methods for reporting ARMS nonresponse, consider the nature of both unit and item nonresponse in ARMS, and discuss methods for reducing nonresponse and making nonresponse adjustments in ARMS.
|
From page 108... ...
The noncensus year Phase III response rates are the most troublesome, since they fail to meet the Office of Management and Budget (OMB) threshold of 80 percent, below which the agency must plan for a nonresponse bias analysis (U.S.
|
From page 109... ...
This calculation appropriately reflects the success of the Phase II survey operation in securing responses, but it does not reflect the fact that the sample comes from the Phase I survey, which had its own nonresponse. Thus, the Phase II response rates reported in the table overstate the proportion of the eligible sample that participated in Phase II (and likewise for the Phase III rates)
|
From page 110... ...
Sum the adjusted weighted response rates from each component. ∑RR R3i n3i Cumulative response rate for Phase III = 1i 2i i n3 t Where: Ri is the Phase I response rate for component i Ri is the Phase II response rate for component i Ri is the Phase III response rate for component i ni is the Phase III sample size for component i nt is the total sample size across all components for Phase III
|
From page 111... ...
Response 70.5 51.9 Refusal 23.7 6.2 Inaccessible 5.8 1.3 The proper portrayal of the true extent of nonresponse for understanding the nature of the problem requires that NASS routinely make available not only a cumulative and unadjusted response rate, but also the information needed to independently compute response rates across phases of the survey. This involves showing the disposition of all cases, particularly how the cases in Phase II and Phase III trace back to Phase I
|
From page 112... ...
found little, if any, connection between nonresponse rates and nonresponse bias, and Groves (2006) reported only a small association between nonresponse level and nonresponse bias in a meta-analysis of studies that had validation measures.
|
From page 113... ...
In light of the high nonresponse rates in ARMS, the project should be using a variety of methods to understand the nonrespondent population. Without this kind of information, researchers make survey design changes aimed only at increasing the overall response rate (or stemming the overall decline in response rate)
|
From page 114... ...
. As far as we know, there is no systematic research that establishes how survey timing is related to ARMS data quality.
|
From page 115... ...
Table 6-3 shows the distribution of variables in the 2004 TABLE 6-3 Percentage Distribution of Variables by Missing Data Rate, ARMS Phase III, 2004 Refusal/Unknown Rate 10% or more 4.1 5 to 9.99% 11.5 1 to 4.99% 26.0 0 to 0.99% 58.4 All 100.0
|
From page 116... ...
. However, response rates by variables and characteristics of respondents were not available due to confidentiality concerns.
|
From page 117... ...
About 75 percent of ARMS Phase III survey items fall in the category of items that can be individually refused. Refusals are designated with a minus 1 (–1)
|
From page 118... ...
UnDeRStAnDing AMeRiCAn AgRiCUltURe income) , 15 in Section G (farm assets)
|
From page 119... ...
These imputations are also selected on the basis of the value of the financial and structural variables that are required for the creation of the ERS ARMS Phase III research file. The methodology for imputing these items is discussed below.
|
From page 120... ...
0 UnDeRStAnDing AMeRiCAn AgRiCUltURe other versions of the questionnaire, income in this category is reported for the operator's household and not separately by type of household member. Because of these differences, imputation of the total amount of this income component takes place in two steps.
|
From page 121... ...
In the panel's view, although the current methodology for imputing with partial nonresponse may satisfy the first criterion for means, totals, and perhaps for ratios, it can lead to poor results when the analyst is interested in distributional properties of the population. Another concern was brought to the panel's attention by members of the research community who use the ARMS data.
|
From page 122... ...
Generally speaking, statistical agencies seek to employ estimation methods that result in both the smallest bias and the smallest sampling variance (Statistics Canada, 2003)
|
From page 123... ...
Two summaries are created: a traditional nonresponse-adjusted summary and a calibrated summary. For the former, traditional nonresponse adjustments (reweighting)
|
From page 124... ...
The expenditure estimates are then regenerated with these "calibrated" weights. This calibration process ensures that the ARMS data replicates the official NASS estimates for the 14 selected crop and livestock items set by the board process.
|
From page 125... ...
During this time, NASS and ERS staff prepare for the National Expenditures Board. Preparation of Official estimates.
|
From page 126... ...
Overall, the effects of the various adjustments on statistics estimated using ARMS are not clear. In particular, the interventions in ARMS based on board processes introduce changes that are not replicable in the normal sense expected in scientific research.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.