Understanding American Agriculture Challenges for the Agricultural Resource Management Survey (2008) / Chapter Skim
Currently Skimming:
7 Methods for Analysis of Complex Surveys
7 Methods for Analysis of Complex Surveys
Pages 127-149
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 127... ...
. The economists at ERS have been advised by the mathematical statisticians at the National Agricultural Statistics Service (NASS)
|
From page 128... ...
The survey weights and the replication weights are provided with the ARMS datasets. Recommendation 7.1: NASS should continue to provide survey weights with the ARMS data set, combined with replication weights for variance estimation.
|
From page 129... ...
Recommendation 7.2: NASS and ERS should continue to recommend the design-weighted approach as appropriate for many of the analyses for users of ARMS data and as the best approach for univariate or descriptive statistics. In addition to descriptive inference, ERS staff and researchers in other organizations also use ARMS data in analytical inference, in which econometric models are fitted and inference is made about model parameters.
|
From page 130... ...
0 UnDeRStAnDing AMeRiCAn AgRiCUltURe BOX 7-1 Continued This can be justified as follows. Ignoring the outside term 14/15, inside the square we have the estimate of the total k∑ w k ek minus an estimate of the same total ∈S derived from the S–g set, but multiplied by 15/14, since S–g only represents 14/15 of the sample.
|
From page 131... ...
In the case of ARMS, the variance estimation can be done by repeating the analysis with the replication weights provided by NASS and computing the average sum of squared deviations over the full sample. While this is a theoretically sound approach and has some important practical advantages, its current implementation suffers from a number of specific shortcomings, as outlined below.
|
From page 132... ...
For both of these types of users, the information currently provided as auxiliary data with the ARMS data file is likely to be insufficient, because these users need more specific information on the sampling design and on the nonresponse patterns and adjustments applied to the dataset to include in their procedures or models. All of these drawbacks of the current implementation of variance estimation can be overcome.
|
From page 133... ...
Also, ignoring the sample design information when it is nonignorable could lead to inappropriate estimated variances, confidence intervals, and incorrect p-values in a statistical hypothesis test. For an example of how a nonignorable design can induce bias on the relationship between dependent and independent variables in the linear regression context, see Nathan and Smith (1989)
|
From page 134... ...
UnDeRStAnDing AMeRiCAn AgRiCUltURe assumptions about the nature of the selectivity, as in the classic paper by Heckman (1979)
|
From page 135... ...
MetHODS FOR AnAlySiS OF COMPleX SURveyS lation and the sample design, any finite population quantity of interest that can be expressed as a function of finite population totals, say qn = g(tz1, .
|
From page 136... ...
UnDeRStAnDing AMeRiCAn AgRiCUltURe FIgURE 7-1 Diagram representing classical design-based inference. generate samples, and for each sample an appropriately weighted estima ˆ tor B can be computed that targets Bn.
|
From page 137... ...
MetHODS FOR AnAlySiS OF COMPleX SURveyS FIgURE 7-2 Diagram representing model design-based inference for superpopulation parameters. superpopulation model and a sampling design, both of these components contribute uncertainty.
|
From page 138... ...
UnDeRStAnDing AMeRiCAn AgRiCUltURe mately equal to the sum of its design variance and the model variance of its finite population target Bn. Both quantities can be estimated, the former based on the sampling distribution and the latter based on the superpopulation model.
|
From page 139... ...
For most users, the survey weights and the associated replication variance estimation weights make it possible to perform statistically valid model fitting and inference, without having to gain in-depth knowledge of the ARMS design and weight generation procedures. In this section, we address some additional issues related to implementation of the design-based paradigm related to ARMS.
|
From page 140... ...
0 UnDeRStAnDing AMeRiCAn AgRiCUltURe inference. Sarndal and Swensson (1987)
|
From page 141... ...
The delete-a-group jackknife, the method currently implemented for ARMS, falls in this category and represents a valid design-based variance estimation method. Jackknife variance estimators make it possible to construct confidence intervals for parameters and perform Wald-type hypothesis testing on them.
|
From page 142... ...
As noted earlier, valid model-based analysis of survey data requires a model for which the design and nonresponse are ignorable. In the regression context, this is typically achieved by expanding the model by including so-called design variables as predictors, by themselves and in interaction with other predictors.
|
From page 143... ...
MetHODS FOR AnAlySiS OF COMPleX SURveyS case of multistage or clustered designs, it might also be necessary to specify a variance structure to account for design-induced correlation. Although the specific techniques for specifying the variance structure to account for design-induced correlation are not suggested here, they could include generalized estimating equation (GEE)
|
From page 144... ...
UnDeRStAnDing AMeRiCAn AgRiCUltURe explicit modeling is required. Some relevant work in this area includes Chambers (1986)
|
From page 145... ...
Recommendation 7.4: NASS and ERS should investigate the feasibility of providing sufficient information on the design and nonresponse characteristics of ARMS, in order to perform design-based statistical analysis without using the replicate weights and to allow users to incorporate design and nonresponse information in model-based analyses. ERS researchers in particular might be interested in investigating the appropriateness of these model-based approaches for some types of analyses.
|
From page 146... ...
In view of this need, the ERS staff has taken the initiative to develop materials to assist outside users, albeit on an ad hoc basis. One such publication is Robert Dubman's ERS staff paper, "Variance Estimation With USDA's Farm Costs and Returns Surveys and Agricultural Resource Management Study" (Dubman, 2000)
|
From page 147... ...
The guide could make recommendations to allow users to choose between these different methods.
|
From page 148... ...
for analysis for different types of questions. The best method for analysis depends on the survey design, the sample size, and the type and reason for the analysis.
|
From page 149... ...
Recommendation 7.6: ERS and NASS should collaborate on writing a Guide for Researchers for performing multivariable analyses using data from complex surveys, particularly data from ARMS. In areas in which expertise is not available for writing parts of such a guide, expertise should be sought from the statistics and economics community, especially those with experience in the analysis of survey data from complex survey designs.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.