1
Introduction
Education, long an important aspect of social and economic life, has changed substantially in recent decades. The student population has grown in diversity and needs, with more students from economically disadvantaged groups seeking higher education but struggling with the economics. The teaching workforce has changed too—not just demographically, but due to an expected wave of retirements and spot shortages in certain fields. Adults are obtaining education for a broader variety of reasons, with many taking advantage of nondegree industry credentials. Grodsky et al. (2021) offer an analysis of trends and impacts of adult education based on a 2015 follow-on study to the 1982 High School and Beyond Survey. Formal education has shifted dramatically in the direction of online learning, particularly during the COVID-19 pandemic. To provide one example, from April to September 2020, the percentage of households reporting homeschooling of school-age children increased from 5.4 to 11.1 percent (Eggleston and Fields, 2021).
Simultaneously, the digital era has created an explosion of data and statistics that are readily available to the public. Policy makers are increasingly reliant on data, as momentum for evidence-based decision making increases. To support these diversifying and expanding demands, the nation needs an education statistics agency that leads the production and distribution of high-quality data that are useful, timely, and responsive to the needs of many stakeholders. With this report, the panel on A Vision and Roadmap for Education Statistics presents a bold new vision of a federal education statistics agency for the 21st century.
THE NATIONAL CENTER FOR EDUCATION STATISTICS: CONTEXT AND MANDATES, ORGANIZATION, AND PRODUCTS AND SERVICES
“Founded in 1867, NCES is the second oldest and third largest in budget among the Office of Management and Budget’s 13 principal federal statistical agencies.” NCES “provides objective, reliable, and trustworthy statistics about the condition of education through administrative data collections, statistical surveys, longitudinal studies, and assessments” (American Statistical Association et al., 2021, p. 1) (see Box 1-1).
Context and Mandates
Thirteen principal statistical agencies, including NCES, and 96 smaller statistical programs and units comprise the federal statistical system. The chief statistician within the U.S. Office of Management and Budget (OMB) helps to encourage collaboration and cooperation among statistical agencies through the Interagency Council on Statistical Policy (ICSP) (consisting of the heads of the principal statistical agencies and the chief statistical officials from all other cabinet departments), and the Federal Committee on Statistical Methodology (FCSM) (consisting of staff from statistical agencies appointed for their technical expertise). The NCES commissioner represents
the Department of Education (ED) on the ICSP, and selected NCES technical staff serve as appointed members of FCSM. NCES also represents ED on the OMB-chaired Federal Geographic Data Committee.1
In 2020, NCES was the third largest statistical agency in terms of budget, but the ninth largest in terms of staff. “The majority of the 13 Principal Statistical Agencies (PSA) have a line item in the President’s Budget showing the total annual funding request. However, for some PSAs [such as NCES] the funding request is made at the level of their parent organization, who subsequently allocates funds” (U.S. OMB, 2020, p. 8).
The functions most commonly associated with statistical agencies, also referred to as evidence-building activities, are the “collection, compilation, processing, analysis, and dissemination of data and information, to create general purpose, policy- and program-specific (including program evaluation and public health surveillance), or research-oriented statistics and datasets.” (U.S. OMB, 2020, p. 4.) Each statistical agency has a unique set of functions, which are accomplished using multiple approaches.
The official list of typical statistical functions above does not include research. However, research is critical to the PSAs for keeping official functions up to date; for taking advantage of new technologies, new data sources, and new methodologies; and for identifying and solving emerging data gaps (NASEM, 2021b, Practice 5). Typically, the ICSP and FCSM convene working groups consisting of staff members from the PSAs, to address challenging new research issues and to determine promising approaches. NCES staff participate in many of these, including (but not limited to) the Federal Interagency Work Group on Race and Ethnicity Measurement Research, Innovating Data Collection Working Group (formerly called the Adaptive Survey Design Interest Group), the Measuring Sexual Orientation and Gender Identity (SOGI) Research Group, the Federal Partners in Bullying Prevention Working Group, and the COVID-19 Data Strategy and Execution Working Group.
The predecessor of NCES was established in 1867 “for the purpose of collecting such statistics and facts as shall show the condition and progress of education.”2 By the time of the 1994 reauthorization, NCES was a well-established independent statistical agency within the ED’s Office of Educational Research and Improvement (OERI). The Education Sciences Reform Act of 2002 (ESRA)3 replaced OERI with the Institute of Education Sciences (IES). NCES became one of the four centers within IES; the others
___________________
1 For more information, see https://www.fgdc.gov/ [March 2022].
2 20 U.S. Code § 9001(a), Repealed. https://uscode.house.gov/view.xhtml?req=granuleid:USC2000-title20-section9001&num=0&edition=2000 [March 2022].
3https://www.govinfo.gov/content/pkg/BILLS-107hr3801enr/pdf/BILLS-107hr3801enr.pdf [March 2022].
are the National Center for Education Evaluation and Regional Assistance, the National Center for Education Research, and the National Center for Special Education Research.
ESRA nibbled away at NCES’s independence. For example, section 114(f)(5) of ESRA specifies that the director of IES shall “establish necessary procedures for technical and scientific peer review of the activities of the institute, consistent with 116(b)(3),” which states the National Board for Education Sciences “will review and approve procedures for technical and scientific peer review of the activities of the institute”4 (see Chapter 4 for a discussion of the IES and NCES review process). Additionally, some of NCES’s administrative functions, such as budgeting, personnel, and contract control, were centralized within IES (see Chapter 5 for discussion of personnel issues such as hiring and contracting). ESRA also replaced NCES’s Advisory Council on Education Statistics with IES’s National Board for Education Sciences (see Chapter 4 for a discussion of advisory and consulting groups; see Elchert and Pierson (2020) for additional discussion of the decline in NCES authority and autonomy). However, ESRA’s reorganization of research-focused IES could benefit NCES. NCES would be well suited to play an integrative role within IES, in the spirit of a new social science that emphasizes replication, generalization, and data linkage—including linkage between IES experimental data and national survey data, and upgrading the collection of correlational and predictive data. NCES can be envisioned as a research agency in its own right, facilitating collaboration with IES research centers and the academic community, and identifying emerging new data needs while maintaining policy neutrality. These efforts would also enable NCES to be nimbler in the adoption of new approaches and technologies.
Organization
The current organization of NCES is depicted in Figure 5-1. A commissioner heads the Center, and there are three divisions, all of which collect or compile data. Each division represents a different type of data source: Sample Surveys, Administrative Data, and Assessments. The Statistical Standards and Data Confidentiality Staff and the Annual Reports and Information Staff are smaller groups that perform crosscutting functions.
NCES currently employs about 90 full-time equivalent (FTE) employees, with 58 organized in the statistics units and 32 mainly organized
___________________
4https://www.govinfo.gov/content/pkg/BILLS-107hr3801enr/pdf/BILLS-107hr3801enr.pdf [March 2022]. ESRA § 116 (20 U.S. Code § 9516) established the National Board for Education Sciences as a board of directors for IES.
in assessments units (Table D-2, Figure D-1).5 NCES’s current staffing level represents a decline of 23 people (20%) overall and 25 people (30%) for the statistics units from fiscal years (FY) 2003–2021. Compared to the peak in FY 2010, the loss is greater, at 34 FTEs (27%) for NCES as a whole and 28 FTEs (33%) for statistics units. NCES’s annual turnover rate since FY 2018 has ranged from 9 to 11 percent (Table D-3) and is an indicator of the risk for further staff (and knowledge) loss.
Even as NCES’s staffing declined, the scope of the Center’s work increased substantially. For example, the Common Education Data Standards and the Statewide Longitudinal Data Systems (SLDS) Grant Program did not exist in 2003. The National Assessment of Educational Progress (NAEP) and the Integrated Postsecondary Education Data System have undergone substantial expansion since 2003, and EDFacts was a large addition to NCES’s work in 2013. Chapter 5 and Appendix D discuss how NCES utilizes its internal employee resources (funded indirectly from ED’s Salaries and Expenses appropriation) versus its contractors (funded by program appropriations) to manage its scope of work.
Products and Services
“NCES collects, analyzes, and disseminates education statistics at all levels, from preschool through postsecondary and adult education, including statistics on international education” (U.S. OMB, 2020, p. 40) (see Box 1-2 and Table B-1). In addition, NCES supports ED in a variety of ways: strengthening privacy of education data; enhancing the quality and consistency of education data at local, state, and federal levels; and supporting SLDS (see Box 1-3).
CHARGE TO THE PANEL
In 2021, to keep pace with the changes in education and the emergence of new data sources and technology, the director of IES asked the National Academies of Sciences, Engineering, and Medicine to recommend a vision
___________________
5 The statistics count includes the Administrative Data Division, Sample Surveys Division and its predecessors, Statistical Standards and Data Confidentiality Staff, Annual Reports and Information Staff, and the Office of the Commissioner FTEs working on statistics. The assessments count includes the Assessments Division plus one FTE from across multiple employees located in the Office of the Commissioner, who work on assessments for some of their time. The organization of staff into statistics and assessment units does not align with program appropriations (Figure D-2) because staff are paid indirectly through an allocation of the ED’s Salaries and Expenses appropriation. The organization of FTEs does not fully reflect the functional roles of the staff. For example, staff located in a statistics office may also support assessment work.
for NCES to achieve in the next 7 years. The National Academies were asked to consider recent trends and future priorities; to suggest changes to NCES’s portfolio of activities and products, operations, staffing, and use of contractors; and to focus on NCES’s statistical programs and not the assessment programs conducted by the NCES Assessments Division (i.e., the NAEP).6
In response to this request, the Committee on National Statistics appointed an interdisciplinary panel of experts to conduct the study. The panel included experts in education research, policy, and federal, state, and local government programs; as well as experts in statistics, data science, survey methods, data governance and infrastructure, and federal statistical policy. The panel drew heavily on the knowledge and experience of its members to develop a strategic approach that NCES could adopt to remain current and relevant in the face of societal changes.
To increase NCES’s mission impact, the panel was first asked to review trends and developments in the use of survey data, administrative data, and other potential data sources; and to consider NCES’s priorities, operations, staffing, size, and use of contractors. The panel was then to use this information to develop recommendations and key milestones to advance NCES to a future, high-impact state. This report prioritizes areas and activities for NCES to pursue, with recommendations for changes and expansions to NCES’s statistical program areas, data collections, data governance, products, and distribution of resources (i.e., budget implications).
The full Statement of Task is shown in Box 1-4.
INFORMATION GATHERING
To meet the scope of this charge, the panel gathered information broadly, holding eight public sessions to collect information from 23 speakers with stakeholder perspectives, from federal and state government, nonprofit and policy organizations, and academia. During this study, the panel solicited speakers and expert testimony across a wide range of topics. Education topics included higher education, adult literacy, elementary and secondary education, and early childhood education; the curriculum and instruction for mathematics, sciences, and language arts; child development and students with disabilities; career transitions; and student equity and civil rights issues in education. Speakers presented on trends and developments in
___________________
6 IES concurrently commissioned two other studies from the National Academies. One addresses key strategic issues related to the NAEP program, including opportunities to contain costs and increase the use of technology. The second addresses the future of education research at IES, including critical problems for which new research is needed; new methods or approaches for conducting research; and new types of research training investments.
their use of survey data, administrative data, and other data sources. Some speakers informed a potential portfolio of activities by discussing their data infrastructure, governance, collections, and collaborations for data sharing, linkage, and insightful analytics. Others informed a potential portfolio of products, by sharing their high-priority analytic questions in education, data needs, and potential ways for NCES to add value.
NCES staff made multiple presentations to the panel to explain the Center’s mission and organization, operations, staff, use of contractors, and budget, in addition to its survey and administrative data programs, stakeholder engagement and technical assistance, data governance, current programmatic priorities, and recent innovations and initiatives. The panel did not address NCES’s assessment programs, which are discussed in a separate, concurrent National Academies report.
To learn more about NCES’s role in ED’s implementation of the Foundations for Evidence-Based Policymaking Act of 2018 (commonly referred to as the Evidence Act),7 the panel heard from ED’s chief statistical official (i.e., the NCES commissioner) and the chief evaluation officer, and received a statement from the chief data officer.
___________________
7 Pub. L. 115-435. Available: https://www.congress.gov/bill/115th-congress/house-bill/4174.
Given the scope of the charge, the panel also reviewed over 300 documents and dozens of webpages. Documents included information about NCES’s organization, budget, and performance, as well as processes, programs, and stakeholders. In addition, the panel asked NCES a series of questions over a period of several months, which the Center answered in writing.8 The panel also considered advisory reports from the National Institute of Statistical Sciences, a workshop from the National Academy of Education, and editorials from the American Statistical Association. For signals on strategic direction and priorities, the panel considered NCES’s authorizing laws and other mandates, executive orders and presidential memoranda, implementation guidance from OMB, ED’s strategic plans since 2007, ED’s Data Strategy (U.S. ED, 2020), and the Committee on National Statistics’ (CNSTAT’s) Principles and Practices for a Federal Statistical Agency (NASEM, 2021b).
The panel reviewed information on models of data infrastructure, forward-thinking SLDS, new data approaches to studying education, and general data modernization and methodology reports, including from CNSTAT. The panel also reviewed trend data on education topics, such as the demographic composition of students and the teaching workforce, students’ disability status, subjects taught, online learning, and school context. The panel sought information about other federal agencies, particularly the PSAs, to serve as NCES’s peers for comparison, examples, and best practices. Documents included the OMB publications on statistical programs from 2003 to 2020, as well as searches on PSAs’ websites for products, programs, and processes. Finally, the panel deliberated over all evidence during 18 closed meetings.
PANEL’S APPROACH TO THE CHARGE
The panel approached this task by asking what a national statistical agency for education would be and do if it were newly established today, with the same level of resources. This led to a full reimagining of how NCES could meet the growing demands for policy-relevant, education-related statistical analyses and data, focusing on equity and the importance of improving approaches to address the needs of changing demographics. The panel reenvisioned NCES’s stakeholder engagement with key federal, state, and local decision makers and the compelling ideas and priorities that
___________________
8 The questions and answers from NCES and IES are available on request from the project’s Public Access File, along with the other unpublished documents provided by NCES and IES, speaker presentations, and testimony submitted to the panel. Many of the citations in the report are to NCES responses to specific questions from the panel. For access, contact the National Academies of Sciences, Engineering, and Medicine’s Public Access Records Office, https://www8.nationalacademies.org/pa/information.aspx.
engagement could bring to the surface. Engagement could include establishing liaisons in each state to advance data infrastructure and products, encouraging the adoption of Common Education Data Standards, and providing the Center with feedback.
Importantly, NCES can harness opportunities created by rapidly changing technologies and policy developments. New technologies and data sources are readily available and continue to proliferate. Many agencies and organizations have strategically integrated multiple data sources to gain new insights into evolving needs.
The panel found many exciting examples of data-source integration. Federal agencies, including Statistics of Income (SOI), the Economic Research Service, and the National Center for Health Statistics have partnered with other federal agencies, state offices, and organizations to produce new knowledge. The Post-Secondary Employment Outcomes Program, a partnership between the U.S. Census Bureau and higher education systems in 17 states, is one example (U.S. Census Bureau, 2021d). Leading states, such as Massachusetts and Ohio, have partnered state offices across social domains (e.g., education, labor, and health and human services), leveraging their SLDS to understand student outcomes. Other states are partnering with each other for data exchange, such as the Western Interstate Commission for Higher Education and the Midwest Collaborative through the Coleridge Initiative.
Data linkages like these result in cutting-edge insights that integrate research and statistical analyses. A leading example is the work of Raj Chetty and colleagues, studying long-term income and earnings outcomes by blending SOI data with the Department of Housing and Urban Development’s (HUD’s) Moving to Opportunity randomized controlled trial data (Chetty et al., 2016). This study substantially increased the value of both SOI’s and HUD’s data, resulting in new information about HUD’s housing voucher program that is pertinent to decision makers. A second example is the Multi-State Postsecondary Dashboard from the Coleridge Initiative (2022), designed by the states of Kentucky, Ohio, Tennessee, and Indiana, to show earnings and employment regionally, building on both state-specific unemployment insurance and higher education files (Midwest Collaborative, 2020).
This explosion of available data sources coincides with increased demand for actionable data to inform decisions at all levels. The Evidence Act solidified and validated the momentum building towards evidence-driven decision making. The 2021 Presidential Memorandum on Restoring Trust in Government Through Scientific Integrity and Evidence-Based Policymaking (Biden, 2021) has further bolstered the movement, as has the Executive Order on Advancing Racial Equity and Support for Underserved Communities through the Federal Government (Executive Order
13985, 2021), which explicitly emphasizes the need for disaggregated data for studying equity. This movement is not limited to federal agencies. State education agencies, local school districts, private schools, and institutions of higher education are focused on data describing student outcomes, equity, and effective methods, in addition to understanding how their policies affect outcomes. As consumers of education, students and their families are also increasingly seeking data, particularly regarding higher education and adult education. It is with this backdrop of dramatic social changes that the panel reimagines NCES as leading education statistics.
To address these trends in education and statistics, the 5 conclusions and 15 recommendations in this study report provide:
- A vision for what NCES should aspire to be, including roles and responsibilities;
- Methods for NCES to attain that vision, including key milestones;
- A process with goals and specific ideas for creating a future prioritized portfolio of products, including modernization of NCES’s statistical program areas, primary data collections, and data-source acquisitions;
- A future portfolio of activities, including modernization of NCES’s role in data governance and data facilitation;
- A process with goals and suggestions for how NCES can organize and operate, including staffing, size, use of contractors, and implications for its budget; and
- Specific ways to increase the impact of NCES’s educational statistics.
The panel provides process recommendations for some aspects of the charge. While the panel developed and completed a process for evaluating data-content priorities (see Appendix C), the panel has insufficient information to recommend which specific data content should be prioritized or deprioritized. Making such a determination requires a rigorous process with full information on NCES’s resources, the level of effort needed to produce content, the numbers of users and types of uses for each product or collection, and the long-term goals for building new operational infrastructure or leveraging existing data-collection processes. The panel provides a goal for prioritizing data content, a process for evaluating the priority level of various education topics, and specific content ideas, generated from the results of the panel’s prioritization of topics based on limited information (see Chapter 3 and Appendix C).
The panel provides a potential process for modifying NCES’s organization and staffing, with additional specific recommendations on use of contractors. The panel lacks the depth of information to make more specific recommendations on organizational structure, staffing level, and
budget implications. The panel recommends that drivers for organizational and operational changes come from NCES’s strategic goals and objectives. Similarly, the panel provides key milestone dates, but the timing of achieving other recommendations depends on the priorities in NCES’s strategic and implementation plans. While NCES could begin work simultaneously on all but the organizational recommendations (see Chapter 5), the specific milestones and metrics cannot be determined before NCES’s own prioritization of activities and products. Chapter 6 provides significant goalposts and relative milestones for making progress towards the recommendations.
AUDIENCES FOR AND ORGANIZATION OF THE REPORT
This report is of interest to multiple audiences across the nation’s education data stakeholders. First, the panel hopes that NCES leaders and personnel find this report helpful, inspiring, and motivating. Second, the panel wishes that IES and key stakeholders in ED take to heart recommendations for strong partnerships with NCES, to advance education evidence building and implementation of the Evidence Act. Third, this report may be of interest to data-holding agencies and organizations in the public and private sectors that may want to invest in and engage with NCES in data exchange for mutual benefit. Fourth, education researchers are key users of NCES data and partners in advancing the science of education. Fifth, policy makers and practitioners in state and local education agencies may use this report to learn about NCES’s resources and to engage with the Center on potential products and services that would be useful and actionable. Sixth, this report is intended to be accessible to the public—both consumers of education programs and the general taxpaying public. To the extent that NCES is publicly funded, it is obligated to serve the interests of the American people. Finally, this report could serve as a useful template for other federal statistical agencies and units that want to broaden their impact, enhance their engagement, support evidence building, modernize their data sources, or otherwise act strategically. We hope this report serves all these audiences.
Chapter 2 discusses the vision and key recommendations necessary for NCES to meet its mission amid social, technological, and policy shifts and the immense opportunities they present; this chapter addresses the panel’s vision for the future of NCES and reviews trends in the use of various data sources. Chapter 3 discusses data collection and acquisition in terms of topical priorities, such as equity in access and outcomes. It suggests ways for NCES to redirect efforts and resources towards high-value topics, to create a future portfolio of prioritized products and data collections for the Center’s statistical programs. Chapter 4 presents recommendations for engaging with stakeholders and disseminating data and statistical products.
Examples and ideas are discussed, not only for obtaining feedback on the relevance and impact of products, but also for leveraging external resources, such as partnerships and grant-making power, to extend NCES’s ability to serve the education data ecosystem. This chapter also addresses NCES’s priorities for a future portfolio of activities, including data governance and data linkage to increase the value of NCES’s data collections and acquisitions. Chapter 5 presents recommendations for NCES’s operation as an agency, its organization, and how to use its contracting resources; this chapter addresses future operational priorities, including staffing, size, use of contractors, and budget implications. Chapter 6 summarizes the recommendations, all of which address ways to increase the impact of educational statistics produced by NCES. Key goalposts and relative milestones are provided.
Seven appendices contribute detailed information for the interested reader. Appendix A provides a glossary of terms and acronyms used in this report. Appendix B defines and describes a variety of relevant data sources. Appendix C describes the process and criteria for rating the importance or national value of assorted topics in education, as well as NCES’s capacity to collect or acquire data on each topic. Appendix D provides a comparison of the federal statistical agencies on dimensions such as the number of FTEs and contractors, budget, and budget-to-FTE ratio. Appendix E provides information and metrics on IES and NCES product-review processes. Appendix F acknowledges speakers and people who submitted statements or other testimony. Appendix G provides biographical sketches of the panel.
