Understanding American Agriculture Challenges for the Agricultural Resource Management Survey (2008) / Chapter Skim
Currently Skimming:
4 Sample and Questionnaire Design
4 Sample and Questionnaire Design
Pages 62-85
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 62... ...
These methodological issues are addressed in light of survey goals and are evaluated in consideration of the panel's understanding of the current state of the art in survey and questionnaire design. The panel recognizes that the survey has evolved over the years into an "as-is" state that must be taken into account when considering possible improvements.
|
From page 63... ...
• The Phase III survey (cost and returns survey) asks about farm and household economics and farm/farm operator characteristics, includes several questionnaire versions (a general questionnaire, from one to three commodity-specific questionnaires, and a core questionnaire)
|
From page 64... ...
The use of mixed modes for data collection is often an important means of controlling costs. The cooperative agreement arrangement with the National Association of State Departments of Agriculture provides a substantially more economical means of collecting data than would be possible working with any other data collection organization, in the view of NASS.2 These and other measures that are designed in part to achieve cost efficiencies have potential implications for data quality.
|
From page 65... ...
in Phase III is of concern. For example, do the answers to fewer questions obtained by mail in the core questionnaire compare directly with the more detailed answers to the questions obtained by faceto-face interviews in the other versions of the Phase III questionnaire?
|
From page 66... ...
This includes Indian reservations, prison farms, private or university research farms, not-for-profit farms operated by religious organizations, and high school Future Farmers of America farms. These institutional farms do not have the same expenses or income patterns as traditional farms (National Agricultural Statistics Service, 2005, p.
|
From page 67... ...
. The list is constructed and maintained from many different sources, including other NASS surveys and administrative files, as well as thirdparty commercial databases and USDA program files.
|
From page 68... ...
This sample frame construction technique tends to guard against omissions or duplication in the list frame. Indeed, in 2005 the area frame added about 1,600 eligible nonoverlap units to the frame for the Phase III ARMS when combined with the Phase I screened sample.
|
From page 69... ...
SAMPLE DESIGN Three major objectives establish the sample design parameters for the ARMS: adequately representing all size classes, reducing respondent burden, and attaining an expected level of precision. Sample Selection The selection of the target population, described above, is largely done to ensure representation of all size classes of at least $1,000 of agricultural products.
|
From page 70... ...
TABLE 4-2 ARMS Sample by Farm Types and Sizes, 2005 0 $1,000- $100,000- $250,000- $500,000Type of Farm $100,000 $250,000 $500,000 $1,000,000 $1,000,000+ Total Oilseeds, grains, beans 3,830 1,856 1,534 1,371 1,055 9,646 Tobacco 128 50 46 32 19 275 Cotton 65 131 188 269 304 957 Vegetables, melons & potatoes 172 101 131 187 746 1,337 Fruit, tree nuts & berries 795 358 281 291 563 2,288 Greenhouse, nursery 125 470 303 244 810 1,952 Cut Christmas trees 60 21 4 11 23 119 Other crops & hay 1684 241 106 98 131 2,260 Hogs & pigs 87 113 163 256 437 1,056 Milk 99 584 601 420 1,026 2,730 Cattle & calves 2,945 976 518 347 570 5,356 Sheep & goats 139 6 11 7 14 177 Equine 470 4 474 Poultry & eggs 85 347 708 1,203 1,228 3,571 Aquaculture 28 75 17 25 67 212 Other animal 73 19 8 4 8 112 Land/cropland 71 NOL (area) 1,610 Total 10,846 5,360 4,620 4,766 7,001 34,203 SOURCE: Economic Research Service.
|
From page 71... ...
6. The Phase III list sample is then supplemented with farms that were found in the area list operation.
|
From page 72... ...
Following application of the P-B method to selection of the Phase I sample, NASS draws the Phase II and Phase III list samples from "good" reports in Phase I Beginning in 2005, NASS has used sequential interval poisson (SIP)
|
From page 73... ...
The most common way to improve the coefficient of variation involves increasing sample size. The expected level of precision for key ARMS estimates is set forth in NASS Policy and Standards Memoranda (PSM)
|
From page 74... ...
In the past two decades, the science of questionnaire design has been refined to include two groups of specialists who are drawn on in a coordinated effort: content specialists and design specialists. In ARMS, the content specialists are mainly the subject-matter experts in the Economic Research Service (ERS)
|
From page 75... ...
It is often the case that these specializations are based in different agencies, with each agency bringing its strength to the questionnaire design and evaluation process. When properly organized for the task, content and design specialists will perform as an integrated working group, constituting a questionnaire design and evaluation team that reaches out to incorporate the interviewer and the respondent in the process of selecting content and design through field and cognitive testing of each collection mode (Esposito, 2003)
|
From page 76... ...
Potential new or changed questions and combined questionnaires are subjected to at least a rudimentary cognitive review by NASS. The objective of the cognitive pretesting, mainly of paper questionnaires, is to determine if respondents are able to answer the questions, not to measure data quality.
|
From page 77... ...
A recent effort (2003) to develop a short-form, self-administered mail version of ARMS Phase III for the 15-state oversample had some of the character of the methodology research and development-based approach the panel advocates.
|
From page 78... ...
Although there was no experimental control group, NASS was able to conduct two data quality tests comparing the distribution of data for 15 variables and item nonresponse for 9 variables with the concurrently collected enumerator-administered version. However, these limited data quality tests did not permit any conclusion as to which questionnaire was better.
|
From page 79... ...
. One recent formal NASS study of respondent record-keeping practices gave equally discouraging results for a group of 96 farm operations in Missouri and Virginia that had previously reported on the ARMS Phase III survey.
|
From page 80... ...
Key parts of this work would best be conducted in a cognitive or usability laboratory facility. It would be enabled by obtaining a generic clearance from the Office of Management and Budget for testing of all phases of the survey to allow for broader cognitive testing, evaluate the quality of data reported in response to each question, and evaluate the impact of mode of data collection across the three phases.
|
From page 81... ...
In ARMS that could take place when collecting income data and possibly data on other topics. It is hard to separate the sense of privacy created by self-administration in general, whether paper or on computer, from the benefits of computerization in particular, for example, automatically selecting the next question depending on the previous answer(s)
|
From page 82... ...
Consistency of Questions Across Survey versions ARMS Phase III is a multiversion survey. The commodity-specific versions (2-4)
|
From page 83... ...
TABLE 4-4 Variables Included in the Debt Section of the ARMS Phase III Questionnaire, 1996-2005 Number Balance Lender Loan Interest Loan Year Percent for Loan on Other Type of Purpose of of Other Year Type Balance Rate Term Obtained Farm Use Guarantee Loan Loans Loans Loan 3 3 3 3 3 3 3 3 3 3 1996 3 3 3 3 3 3 3 3 3 1997 3 3 3 3 3 3 3 3 3 1998 3 3 3 3 3 3 3 1999 3 3 3 3 3 3 3 3 3 2000 3 3 3 3 3 3 3 3 3 2001 3 3 3 3 3 3 3 3 3 2002 3 3 3 3 3 3 3 3 3 3 2003 3 3 3 3 3 3 3 3 3 3 2004 3 3 3 3 3 3 3 3 3 3 3 2005
|
From page 84... ...
Other variables generated by combining ARMS data with other sources have also been added to the microdata records, such as cost-of-production estimates, farm typology, and commodity cost and return estimates. In some cases there may be both a reduction in respondent burden and an increase in data quality from using administrative data to supplement or replace some survey data.
|
From page 85... ...
Recommendation 4.3: NASS and ERS should explore the collection of auxiliary information on a formal basis, as well as the feasibility of enriching the ARMS data files with information from administrative data sources, geospatial data, and the like.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.