Measuring What We Spend Toward a New Consumer Expenditure Survey (2013) / Chapter Skim
Currently Skimming:
6 Pathway to an Improved Survey
6 Pathway to an Improved Survey
Pages 109-187
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 109... ...
Because the Interview survey had been deemed not to satisfy all user needs, the program also includes the Diary survey. The Diary survey supplies much of the same information but at an even higher level of detail over a shorter period of time by using a different collection mode and a different set of respondents.
|
From page 110... ...
In theory, a number of survey designs can provide the information required by the CPI, collecting a significant level of expenditure data without inflicting the level of burden on households that the current CE does. These designs, including a number of "matrix type" sample designs, involve asking each household only a portion of the total detail required while using weighting and more sophisticated modeling to produce the needed estimates.
|
From page 111... ...
There is an increased emphasis on self-administration of survey components, while creating tools and an infrastructure that will monitor and support the respondent in these endeavors. The field representatives' role will still be important in directly collecting data, but their role will grow to also provide support in additional ways.
|
From page 112... ...
These issues include such things as synchronization of expenditure and nonexpenditure items over similar reference periods, and collecting changes in employment status and other life events. These types of issues are important to the research uses of the CE.
|
From page 113... ...
PANEL'S APPROACH TO DESIGN AND THE COMMONALITIES THAT EMERGED The panel considered many approaches to a redesign of the CE, and sorted through those numerous options by focusing on the following fundamentals: • Improve data quality. • Be mindful that the resources (both out-of-pocket and staff)
|
From page 114... ...
It realizes that several of these options may be at odds with collecting a complete picture of income and expenses from each individual household over longer reporting periods. This is why it is essential for BLS to further clarify its priorities for data uses, recognizing that one survey cannot satisfy all of the possible data users.
|
From page 115... ...
The panel sees a movement toward self-administered data collection with the field representative acting in a support role. However, the prototypes also incorporate interviewing by field representatives.
|
From page 116... ...
This is an essential component of the panel's concept of supported self-administration. The panel discussed many technological alternatives and found one tool that was particularly appealing to the panel across a variety of designs -- the tablet computer.
|
From page 117... ...
Use Administrative Data Appropriately but with Caution The potential use of external records or alternative data sources as a replacement or adjunct to current survey data for the CE is often raised in discussions of a CE redesign. Whether at the aggregate or the micro level,
|
From page 118... ...
. All three prototypes strive for increased use of records, incorporate selfadministration (supported by the field representative, a tablet computer, and a centralized support facility)
|
From page 119... ...
It col lects the detail of expense items as in Design A, while providing a household profile for six months. To do both, it uses a more com plex sample design, collects different information from different samples, and requires more extensive use of modeling to provide expenditure estimates and the household profile.
|
From page 120... ...
Mostly by mail. FIGURE 6-1 Process flow for Design A -- Detailed Expenditures Through Self-Administration.
|
From page 121... ...
In doing so, the panel believes that the survey can collect detailed expenditures accurately with reduced burden on respondents. The goals are to • promote accurate reporting of detailed expenditure data by allow ing sufficient time and space for careful enumeration of expendi tures while using records and receipts; • reduce the effort it takes to report those expenditures by providing support and technology tools; and • reduce respondents' tendencies (often implicitly encouraged in cur rent methods)
|
From page 122... ...
During the initial in-person contact with the household, the field representative would identify the main "household respondent" and assist him or her with completing the Demographics and Life Events Module on the
|
From page 123... ...
In some cases, a field representative visit may be needed to ensure the return of the tablet and capture key missing data. The Demographics and Life Events Module, the Large and Routine Expense Module, and the Income Module would be appropriately modified for a household's second wave of data collection.
|
From page 124... ...
. Role of Field Representative The field representative's role changes radically in Design A, from being the prime interviewer and data recorder to being the facilitator who encourages, trains, and monitors in support of the respondent's thoughtful and accurate data entry.
|
From page 125... ...
While in one sense, the respondent in the proposed design has greater responsibility (and support) than required in current expenditure surveys, in another sense the locus of responsibility is more distributed than before: among the respondent, field representative, and remote monitors (and, in a way, the interface designers and researchers)
|
From page 126... ...
The interface within the tablet converts data entered in the Large and Routine Expense Module and the Income Module to appropriate standardized reporting periods. For paper households, data entry by the field representative or central staff would be required.
|
From page 127... ...
Meeting CE Requirements and Redesign Goals This prototype meets the basic CE requirements laid out in Consumer Expenditure Survey (CE) Data Requirements (Henderson et al., 2011)
|
From page 128... ...
Income estimated Income is reported for a period most convenient for the respondent over the same time to report accurately while being close to the reporting period for period expenditures. Income, like expenditure data, may be modeled for the entire year.
|
From page 129... ...
The field representative would assist the respondent in completing a recall survey of expenditures on a tablet computer. The tablet would be left with the household for use in a one-week supported-journal concurrent collection of expenditures and then mailed back, similar to that described in Design A
|
From page 130... ...
(for 1 week) Real-time upload of data Real-time upload of data Real-time upload of data Household returns tablet or Household returns tablet or Household returns tablet or paper journal, mostly by mail.
|
From page 131... ...
• Household respondents will agree to remain in the panel for 13 months, with three data collection events during that period. • The use of supported self-administration with the tablet, a central support facility, and the field representative allows most respon dents to complete the redesigned recall questionnaire without the need for an in-person interview during waves 2 and 3.
|
From page 132... ...
In this prototype, the tablet is set up with three modules all designed for self-administration: • Demographics and Life Events Module: demographics and other information about the household; • Recall Module: for reporting expenditure, income, and asset data recalled or estimated for the past six months; and • Ongoing Expenditures Module: for recording detailed, ongoing expenditures by household members during the one-week period of supported journal collection.
|
From page 133... ...
The goal is to encourage the use of the tablet as much as possible in subsequent waves while maintaining data quality. The field representative would then assist the respondent in completing the Recall Module reporting expenditures and income for the previous six months.
|
From page 134... ...
Each wave uses a tablet computer with three modules. Wave 1 requires an in-person visit by the field representative with an "assisted" interview using the tablet for the Demographics and Life Events Module and the Recall Module.
|
From page 135... ...
After the initial in-person visit, ideally the field representative never visits the household again. In principle, the household faithfully fills in the Ongoing Expenditures Module (supported journal)
|
From page 136... ...
The role for the central office includes monitoring the cases on a daily basis, phoning the household to intervene if the supported journal is not filled in, and prompting field representatives if their assistance is needed. The central office also fills the role of first-line support for respondent questions, forgotten passwords, and the like.
|
From page 137... ...
Sample size and costs for the subsampled component are provided separately. Assumptions: • Cost per in-person recall module or interview -- $325.
|
From page 138... ...
categories of expense items Income estimated over Questions on income, assets, and "labor force status" are asked on the same time period each wave and for the same reporting period. Complete picture for This prototype focuses on providing an improved picture at the household spending household level over the current CE.
|
From page 139... ...
used in this prototype to construct a full set of microdata for the entire six-month period; • develop appropriate models to "disaggregate" aggregated expenses using data from the one-week supported journal; and • develop successful methodology for a component that will use an intensive interview and process based on prior collation of records and financial software to achieve a budget balance for the year at the household level, as described below. Intensive Subsample in Design B Design Objectives: A relatively small subsample of households who have completed wave 3 of the basic component of Design B would be asked to participate in a more intensive process to provide a full picture of income and expenditures over two consecutive calendar years.
|
From page 140... ...
It may take multiple visits to achieve the required balance. Expenditure and income data reported during waves 2 and 3 of the basic component would be available to the respondent and field representative to work together to bring things in balance.
|
From page 141... ...
Assumptions: • Budget balancing process may take several interviews with the household and may require more experienced field representatives. Expected cost per household -- $800 per wave, with $350 per re fusal contact.
|
From page 142... ...
In doing so, it provides detailed expenditures similar to Design A It provides a complete picture of household expenses and income as in Design B, but for six months (instead of 18)
|
From page 143... ...
• Expenditures recorded for 3 Months Later - Wave 2 ends modified 96-item level aggregation (some smaller categories not In-person visit collected) Fill information gaps Household returns tablet or • More detailed demographics, Tablet retrieved paper journal, mostly by mail income and labor force status 143 FIGURE 6-3 Process flow for Design C -- Dividing Tasks Among Multiple Integrated Samples.
|
From page 144... ...
These data would be used in national and regional estimates, with precision enhanced by modeling back to the base survey and combining with data collected from the household profile component. Household Profile Component: Design C calls for a separate component of the overall sample to focus on providing a complete profile of household expenses and income over two consecutive three-month periods.
|
From page 145... ...
The field representative asks the household to proactively keep receipts and use a supplied tablet computer to record expenditures during the upcoming quarter. Household respondents would enter expenditure amounts in the tablet, indicating the expense category for the item.
|
From page 146... ...
The field representative would leave the tablet with the household for one additional quarter, reemphasizing the need to keep receipts and record expenditures on the tablet. The field representative returns again at the end of the second quarter and conducts an interview using the tablet to fill information gaps.
|
From page 147... ...
Ideally this will be true for both components. On the household profile component, the field representative may have to do more traditional interviewing if the household has not successfully kept up with recording expenditures over
|
From page 148... ...
A complete profile of individual households for microlevel research would be based on data collected for six months on the household profile component, which would be supplemented with estimates of smaller categories of expenses and/or the breakdown of collected aggregates using data from the detailed expenditure component. Infrastructure Infrastructure needs are similar to those in Design A and not repeated here.
|
From page 149... ...
• Incentives -- $150 per detailed expenditure component; $180 per household profile component. • Paper processing for paper households -- $100.
|
From page 150... ...
178. Additional research is needed specifically for this prototype to: • research and develop models for estimation using the base survey and two waves of data collection; and • research and develop models for imputing at the household level "smaller expense items" collected on the detailed expenditure component and not on the household profile component into the household-level dataset to complete the overall household expense profile.
|
From page 151... ...
Larger and aggregated expenses are collected items in the household profile component. Modeling data collected in both modules using correlations with base survey variables is expected to create estimates of detailed expenses with appropriate precision.
|
From page 152... ...
asking re spondents to estimate the "average or typical" amount spent on other types of expense items. This design also incorporates an in tensive subsample in which households will be asked to work with the field representative to balance household income and expenses over two one-year periods.
|
From page 153... ...
It uses a base survey for stratification and sampling, followed by subsamples for a detailed expenditure com ponent and a household profile component. Estimates of detailed expenditures would be made using data from all three components through modeling.
|
From page 154... ...
Among the key questions to be answered with respect to recall and the supported journal in the collection of expenditure data are the following. Recall components: • Can a recall survey be designed that could collect sufficiently ac curate data on spending patterns for individual households at the level of 96 expenditure categories?
|
From page 155... ...
However, it requires modeling daily spending to 96 categories of expenses for six months; there is no experience on which to gauge how well an operational version can be designed and implemented. The panel made an effort to compare data collection costs across prototypes.
|
From page 156... ...
This average cost was divided into the total cost to calculate an annual effective (completed) sample size.
|
From page 157... ...
The prototypes em body the panel's informed opinion about the best avenues to explore, not options from which to choose a survey design. Using the Tablet PC for Self-Administered Data Collection The panel proposes the use of tablet computers with wireless phone cards for supported self-administered data collection in all three proto types as a way to solve some key problems with the CE.
|
From page 158... ...
For some prototype designs, the field representative assists the respondent in self-administration of the initial survey module as a way of both collecting information and training on the tablet. The field representative would leave the tablet computer with the respondent, as well as a package in which to mail back the computer after data collection is completed.
|
From page 159... ...
Ideally, a 24/7 in-house technical support team would be maintained throughout the data collection period to address respondent messages and phone calls very quickly. The field representative remains involved with the household as needed.
|
From page 160... ...
A well-designed interface would make long training sessions unnecessary. Field representatives would be able to assess respondents quickly to determine the most appropriate training or data collection approach.
|
From page 161... ...
Delivery and Mail The first time the tablet enters a household, it is brought by a field representative. The field representative leaves behind the tablet and a mailer package, and explains the incentives.
|
From page 162... ...
Incorporating Cognitive Changes to the Paper-Supported Journal The panel recommends that a paper instrument be available for use by respondents who cannot or will not use a tablet computer to record and submit expenses. The current Diary booklet should not be used and the panel recommends a redesign (Recommendation 6-5)
|
From page 163... ...
Timetable and Priorities It is not the panel's role to impose a specific timetable on BLS for investigation or implementation of the alternative designs discussed here. However, the panel believes that development of a targeted and tightly focused plan is necessary if BLS is to achieve a redesign within the next five years.
|
From page 164... ...
If the tablet-based data collection approach proves feasible, other research and operational questions remain. They include questions such as whether varying the recall length for different expenditures would be effective to reduce burden and improve data quality; whether asking about broader categories of expenditures rather than the detailed items would similarly improve reporting; and how to structure incentives to maximize response.
|
From page 165... ...
Some of the work can be done in parallel with work on the critical path, but it should not detract from the resources focused on addressing the key issues and reaching the key decision points within a reasonable time frame. R ecommendation 6-6: BLS should develop a preliminary roadmap for redesign of the CE within six months.
|
From page 166... ...
decreasing the likelihood of missing data or other factors that affect data quality, or (4) reducing the total costs of fielding a survey (Brehm, 1994; Church, 1993; Dillman, Smyth, and Christian, 2009; Singer et al., 1999)
|
From page 167... ...
Use of an effective incentive design can, therefore, reduce more costly interviewer time and/or achieve a higher rate of participation than when incentives are not used. Guidance for the Consumer Expenditure Program It is critical that some form of incentive structure be put in place in the redesigned CE regardless of what develops as the final data collection design.
|
From page 168... ...
should be seriously considered for any incentive structure to optimize the efficient use of these funds.
|
From page 169... ...
Guidelines for Adopting Newer Technology and Incorporating External Data The panel has proposed three prototype designs for the CE that make use of a tablet computer. Those prototypes do not include recommendations for use of any specific external datasets.
|
From page 170... ...
It is important for the agenda to promote a strategic direction for continuous improvement, creating reductions in: • Data collections and processing costs -- respondents entering sup ported journal data via computer could reduce data collection and processing costs; similarly, the use of administrative data could reduce the amount of information that is needed from the respon dent, further reducing data collection costs; • Measurement error -- reductions can be achieved by tailoring the technology to the user, whether the user is the respondent (com pleting a supported journal) or the interviewer (administering a questionnaire)
|
From page 171... ...
Incorporating External Data The panel developed recommendations regarding the use of extant administrative data for the CE. The potential use of external records or alternative data sources as a replacement or adjunct to current survey data for the CE is often raised in discussions of a CE redesign.
|
From page 172... ...
provide data quality checks for specific retail goods or sets of items (i.e., channels)
|
From page 173... ...
. Utilizing records in this fashion would increase the time burden faced by respondents, but it could have a positive effect on data quality by reducing reporting error.
|
From page 174... ...
The incremental benefits would need to be closely contrasted with the real costs of infrastructure, time, and resources required. If used in a targeted and judicious manner, both aggregate and microdata from such sources could be an effective means to improve overall data quality for the CE.
|
From page 175... ...
Successful survey vendors respond to this environment by building an adaptable staff with complex methodological and statistical skill sets, and by continuously investigating new sample designs, survey methods, and estimation strategies that anticipate future changes. Updating Internal Staff Capabilities In light of this reality, agencies that sponsor and conduct surveys, such as BLS, need to build and maintain flexible and capable organizations and staff.
|
From page 176... ...
Several staff members will need to focus on identifying and acquiring administrative or commercial data to supplement data from the CE surveys. In the process, an evaluation of the quality and accuracy of the administrative records will need to be performed.
|
From page 177... ...
Other agencies have extensive expertise in areas that will be of interest to BLS as it redesigns the CES and other surveys. For example, the Census Bureau, which currently has responsibility for the CE data collection, has expertise in using administrative data to augment survey datasets and is devoting considerable energy to expanding its abilities in this area.
|
From page 178... ...
R ecommendation 6-10: BLS should seek to engage outside experts and organizations with experience in combining the development of tablet computer applications along with appropriate survey methods in de veloping such applications. Targeted Research Needed for Successful Implementation of Design Elements The panel views the CE redesign not as one major effort, but as an ongoing process to continually address changes in the population and in survey methods.
|
From page 179... ...
The panel also recommends the collection of fewer, less detailed, expenditure categories in two of the prototypes, which requires evaluation of how this structure can be used to compute the CPI and how to best collect these data. The panel also recommends research on several other promising areas that may lead to further improvements of the survey to reduce burden and help obtain better quality data, presented in a separate subsection This is by no means a comprehensive list of research areas, but an identification of several areas that need additional research, related to implementation of the proposed designs.
|
From page 180... ...
The choice of a tablet device, the design of the interface, and the addition of any motivation features are all prerequisites for the successful implementation of a tablet device to collect expenditure data. Yet there are fundamental questions about the cost feasibility and ubiquitous use of the tablet technology that need to be addressed.
|
From page 181... ...
Among those who keep electronic records, some may even use specialized software that serves as a single repository of expenditure data, such as tax-related software packages. Within a household, some may rely on a single person to be responsible for all expenditure records, while other households divide this responsibility by the type of expenditure or by the person who made the expenditure.
|
From page 182... ...
evaluate the stability of the estimates. Other important areas for study, especially pertinent to the proposed designs, are the optimum recall period for different types of expenditures and the optimum time between interviews.
|
From page 183... ...
Evaluation of the effectiveness of using more intensive methods. The proposed designs suggest the use of more intensive methods to improve the accuracy of the data collected on all sample members, or the use of more intensive methods on a subsample in statistical adjustments for measurement error. Whether such approaches are warranted and how they are implemented depend on the effectiveness of the more intensive methods to obtain more accurate data.
|
From page 184... ...
Attempts can be made to retrieve expenditure data either from other sources or directly from records that the respondents have retained. These may include credit card/bank account statements, utility statements, pay stubs, and tax records.
|
From page 185... ...
• Research and develop models for imputing at the household level "smaller expense items" collected on the detailed expenditure component and not on the household profile component into the household-level dataset to complete the overall household expense profile. R ecommendation 6-11: BLS should engage in a program of targeted research on the topics listed in this report that will inform the specific redesign of the CE.
|
From page 186... ...
They all incorporate self-administration (supported from the field representative, a tablet computer, and a centralized support facility) as a mode of data collection.
|
From page 187... ...
PATHWAY TO AN IMPROVED SURVEY 187 meet additional challenges in future years. Finally, the panel recommends that BLS reach out to other organizations for assistance in implementing the tablet-based data collection system and the apps that will make it work smoothly.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.