7
Organizational Considerations and Overarching Guidance
This report is intended to provide actionable steps as the Bureau of Labor Statistics (BLS) continues modernization of its Consumer Price Index (CPI) program. While recommendations for advancing the data infrastructure supporting elementary price index estimation, higher level index aggregation, and other aspects of CPI construction were the panel’s primary focus, a few systemic considerations are presented as well in this concluding chapter.
7.1. COORDINATION WITHIN BLS
A shift toward greater use of alternative and nontraditional data sources is a complex task touching on many dimensions of CPI data acquisition and methodology. This feature creates challenges for organizational authority and accountability within BLS. To meet this challenge, BLS should build data modernization into its organizational structure.
Recommendation 7.1: BLS should designate a single, high-level person within the agency, preferably as the deputy commissioner level, whose job is to lead data transformation efforts. Having this responsibility explicitly designated would facilitate a focused, coordinated effort and would ensure accountability. This person also could be the visible point person for coordination with the Bureau of Economic Analysis (BEA), the Census Bureau, U.S. Department of Agriculture (USDA), and other statistical agencies that are likewise in the process of data modernization initiatives. A key objective is to avoid duplicative efforts that likely would arise if data transformation proceeded in a more decentralized (siloed) way within BLS.
The data transformation lead would also be part of the team tasked with developing communication strategies to work with Congress to seek the necessary resources to implement changes and highlight the value of the task to user communities and to the general public.
7.2. INTERAGENCY COLLABORATION
The decentralized design of the U.S. statistical system heightens the need for thoughtful collaboration among statistical agencies as data modernization proceeds. Abraham et al. (2021, p. 16)1 described the situation:
A central set of challenges for realizing the potential of Big Data for economic statistics arises from the way in which the agencies’ collaborations with businesses and with each other are structured. Historically, each of the three main economic statistics agencies—the BLS, Census Bureau, and BEA—has had a well-defined set of largely distinct responsibilities. Although there always has been collaboration among the agencies, each agency collects the information from businesses that it needs for specific statistical series and produces those series independently. In a Big Data world, however, there are compelling reasons for agencies to adopt more integrated data collection and production processes.
Key economic indicators such as national output and income rely on data produced from the multiple statistical agencies identified in the above quote. Although coordination already exists among these agencies, more will be needed for the acquisition and innovative use of alternative data sources in these efforts.
Recommendation 7.2: More extensive collaboration between BLS, the Census Bureau, and BEA—along with other statistical agencies that collect key economic data, such as USDA—is needed to advance the acquisition and use of alternative data sources in the production of economic indicators. More specifically, such coordination will allow the statistical system to negotiate common, unified, comprehensive contracts with companies (once, not multiple times) that collect applicable data.
USDA could be a particularly valuable partner. The department’s Economic Research Service (Food Economics Division), which collects data to inform policies related to federal nutrition assistance programs, has a history of acquiring proprietary scanner and other transactions data for the purpose of estimating food prices, quantities of sales, and acquisition of food for at-home and away-from-home eating (NASEM, 2020). It
___________________
might also be useful to include agencies that perform research and use data extensively, such as the Energy Information Administration, the Office for Financial Research, the Federal Reserve, and the Department of Transportation’s Bureau of Transportation Statistics.
Ideally, collaboration among the statistical agencies would culminate with the creation of a Joint Office to administer collection of electronic data. For example, if the Census Bureau were interested in a large dataset, it would coordinate with its statistical agency partners who would all be able to access the data. Coordinating such acquisition, while adding interagency complexity, has the potential to reduce duplication, save resources, and enhance cross-agency spillovers. Indeed, formalizing institutional coordination of data acquisition would signal commitment by senior leadership of the statistical agencies to the increased use of alternative data. In the case of the CPI, these efforts would help lay the foundation for a world when most transactions leave an electronic record, which, in turn, may ultimately become the principal source of input data for price measurement.
The call for greater interagency collaboration in securing data from outside the statistical system is consistent with similar recommendations put forth in other reports.2 An important task of the joint effort by the statistical agencies would be to create incentives for nongovernment data producers to engage in public-private partnerships. The agencies will need to be creative in establishing mutually beneficial agreements that might, for example, offer value-added products back to data providers in return for data access. For the most part, this work will require breaking new ground although some organizations have made progress along these lines. For example, Adobe Insights (Lasiy, White, and Pandya, 2020) collects merchant data on online transactions to produce timely estimates of spending and quantities of varieties of certain goods. Adobe worked out a deal with retailers to provide benchmark indexes that are useful to clients. This initiative created the opportunity to make comparisons across peer firms, and that inducement was enough to persuade retailers to provide data. Similarly, Statistics Netherlands sent selected indexes back to data companies in return for the source data
___________________
2NASEM (2017, p. 3) recommended that a higher-level cross-agency entity “should assist federal statistical agencies in identifying data sources that can most effectively inform the creation of national statistics, help develop techniques to use data from these sources to compute national statistics while respecting privacy and other protection obligations on the data, and nurture the expertise required to perform these functions” (www.nap.edu/read/24893/chapter/1. The Interagency Council on Statistical Policy (ICSP, www.usa.gov/federal-agencies/federal-interagency-council-on-statistical-policy)—established to “improve communication among the statistical agencies” and which includes membership of all officials at U.S. statistical agencies—may provide a good starting place.
7.3. COLLABORATION AND COMMUNICATION
BLS should cast its collaborative efforts more broadly, beyond the U.S. statistical system, as well. With price measurement in particular, ample opportunities exist to replicate (with needed modifications) innovative approaches to the use of alternative data sources that have been developed by various non-U.S. national statistical offices.
Recommendation 7.3: BLS should enhance its contacts and collaborations with CPI staff in statistical agencies beyond the U.S. system. Other countries have made significant progress on data transformation—specifically in methods blending scanner and web-scraped data with survey sources—and CPI staff would benefit from more fully investigating successes and failures experienced during these efforts.
Some of the most innovative use of alternative datasets has taken place in academic settings, so continued collaboration by BLS with academic and other outside experts is likewise encouraged.
Recommendation 7.4: BLS should enhance its interactions with outside experts (in academia, industry, and elsewhere) through collaborative research to leverage the latest advances in research on price measurement methods.
Existing venues that could be particularly valuable for these efforts include the Federal Economic Statistics Advisory Committee (FESAC) and BLS’s advisory committees, which could play a larger role in guiding data transformation efforts. BLS also could look to models used by statistical agencies in other countries, such as the Statistics Canada advisory board model, which includes real-time consulting on data modernization issues as they arise.3
The type of data modernization for the CPI envisioned in this report will require a technical staff with an expanded set of statistical and computational skills. The panel recognizes that BLS is well aware of these staffing challenges. Accordingly, while BLS should expand collaboration with experts from beyond the agency (and beyond the statistical system), a long-term goal is certainly to expand and redirect its own in-house staff’s skill portfolio.
___________________
3 Information about Statistics Canada advisory groups can be found at www.statcan.gc.ca/en/about/relevant.
Recommendation 7.5: In addition to hiring staff with data science skills, BLS should strive to develop this talent in-house by supporting and rewarding staff who pursue training and educational opportunities to develop the technical expertise that will facilitate data transformation efforts in coming years.
This message that has been articulated frequently in other recent reports. NASEM (2017, p. 149), for example, recommended that “federal statistical agencies should ensure their technical staff receive appropriate training in modern computer science technology including but not limited to database, cryptography, privacy-preserving, and privacy-enhancing technologies.”
Since confidence in and understanding by data users of official statistics is critical, successful modernization of the CPI will require that BLS provide clear and consistent communication about the re-design on an ongoing basis.
Recommendation 7.6: As CPI modernization proceeds, BLS should ensure that key information is readily available to all stakeholders—such as by posting in an easy-to-find location of the website—including advance notice of changes, detail about alternative data sources incorporated, transparency around experimental indexes, and updates on the timeline for the project as it evolves. The agency also should aggressively and frequently communicate with stakeholders in the user and research communities.
Such updates and visibility will be especially important during times of rapid changes, such as the dramatic shift in purchasing patterns during the pandemic and the associated heightened interest in the compiling of CPI relative importance weights during and following the pandemic.
More generally, the key is for BLS to ensure that stakeholders can plan and use data appropriately. A key element of this communication, as alternative data sources are incorporated into the CPI, is to ensure that stakeholders know the specific sources of data and index methodology used for individual components as well as the terms on which BLS obtained the data so that all users are an equal footing in interpreting CPI releases.
One statistical office that undertook aggressive and frequent communication about the use of alternative data sources is the Australian Bureau of Statistics. That agency collaborated with international experts (including statistical offices in other countries) and consulted with key stakeholders (e.g., the Reserve Bank of Australia, the Treasury, and the Department of Social Services) to resolve outstanding methodological issues associated with its data modernization effort, particularly the use of transactions data
to compile the CPI.4 Estimating parallel series based on alternative data for an experimental period, as suggested below, will support data quality and public perceptions of integrity of the data collection changes.
7.4. DATA ACQUISITION AND ACCESS
Beyond content-oriented questions about coverage and representativeness, scope of variables, and transparency regarding methods, the potential of commercial data sources is often limited by legal access hurdles and privacy concerns (NASEM, 2020, p. 124).5 Indeed, sometimes even data sources (e.g., administrative records) from other agencies are inaccessible. BLS documentation indicates that the main obstacles to adopting the kinds of information that could naturally be applied to price measurement, such as web-scraped data, have been administrative and legal (Konny, Williams, and Friedman, 2019, p. 7):
Concerns regarding web scraping have arisen both internally and from respondents…To ensure all alternative data used in research or production is protected under CIPSEA,6 BLS must provide establishments, including those whose data we collect on-line, whether manually or automatically, a pledge of confidentiality promising to use the information for exclusively statistical purposes. In the case of web scraping, BLS cannot proceed without permission of the establishment.
An additional challenge that has been raised is the possibility of a data source that BLS does not directly control disappearing or being significantly altered if the data vendor’s motivations for producing the data change. Further, data purchased from vendors may have been sampled, cleaned, and aggregated in a way that does not best serve the purposes of constructing the CPI. BLS and other statistical agencies must be mindful of the high stakes, in terms of policy and market impacts, associated with methodological changes to the CPI.
___________________
4 For a description of this effort, see www.abs.gov.au/AUSSTATS/abs@.Nsf/39433889d406eeb9ca2570610019e9a5/40fc971083782000ca25768e002c845b!OpenDocument.
5 See Democratizing Our Data: A Manifesto, by Julia Lane, which includes a probe into the bureaucratic and other obstacles that have impeded progress on data modernization in the U.S. statistical system.
6 The Confidential Information Protection and Statistical Efficiency Act (CIPSEA) of 2002 requires that data collected be used strictly for statistical purposes and promises respondents high levels of data protection against disclosure of confidential information. See Implementation Guidance for Title V of the E-Government Act, Confidential Information Protection and Statistical Efficiency Act of 2002 at: https://obamawhitehouse.archives.gov/sites/default/files/omb/assets/omb/inforeg/proposed_cispea_guidance.pdf.
The panel recognizes these challenges. Nonetheless, the case for CPI modernization and the greater use of alternative data is compelling, and the panel believes that the BLS, in coordination with the other statistical agencies, can effectively overcome organizational, administrative, and legal hurdles to move forward while maintaining the high quality the CPI is known for. Moreover, as legal and administrative frameworks become more established and standardized, these efforts should become more routine.
This page intentionally left blank.
