Summary
In 2021, the director of the Institute of Education Sciences (IES) at the Department of Education (ED) asked the National Academies of Sciences, Engineering, and Medicine to provide a vision for the National Center for Education Statistics (NCES), excluding its Assessments Division, for the next 7 years. The vision would include a modernized portfolio of statistical products and services to increase NCES’s impact. The National Academies were asked to review developments in collecting and using data, consider recent trends and future priorities, and suggest changes to NCES’s programs, operations, staffing, and use of contractors, with a focus on NCES’s statistical programs and not its assessment programs.
In response to this request, the Committee on National Statistics convened an interdisciplinary panel of experts. The panel approached this task by asking what a national statistical agency for education would be and do if it were newly established today, as a means of reimagining how NCES could meet its mission effectively with the same level of resources.
RISE UP TO MEET 21ST-CENTURY EDUCATION DATA ECOSYSTEM NEEDS
Education in the United States is changing rapidly as student bodies become more diverse, more students enroll in postsecondary education, and technology plays a greater role in learning. Education data and policymakers’ appetite for rigorous evidence have advanced rapidly as new data sources come online and the federal government pushes for more and better data, through efforts including the Foundations for Evidence-Based
Policymaking Act of 2018 (commonly called the Evidence Act)1 and the 2021 Presidential Executive Order on Advancing Racial Equity and Support for Underserved Communities through the Federal Government (Executive Order 13985, January 20, 2021).
The panel reimagines NCES as a leader in education statistics, evidence building, and data governance, expanding its role as a data-access facilitator. The panel aspires for NCES to be in full control of how it meets its mission—operating strategically, anticipating environmental changes, and readily adapting to deliver high-value products and services. Finally, the panel envisions NCES as deeply engaged with stakeholders, strengthening data capacity at state and local education agencies, and as a strong partner to ED.
This report cannot take the place of strategic planning, which will require NCES to perform an intensive self-examination and review of the education environment, collaborating with stakeholders and consultants. While understanding that NCES needs to decide for itself how to proceed, the panel presents its thoughts on strategic priorities, data products, services, and operations. This report provides a blueprint of key issues and ways that NCES may seek to resolve them, including operational details and many examples to assist with implementation.
The full report includes 5 conclusions and 15 recommendations. These recommendations require planning, will, and discipline to achieve within NCES’s constraints. The panel suggests that NCES begin by investing in strategic planning, infusing diversity, equity, inclusion, and accessibility (DEIA) considerations throughout its organization and work, exploring alternative datasets, and effectively fulfilling its role as ED’s statistical official. Some recommendations require collaboration with other individuals and agencies. Progress on the fundamental recommendations alone would substantially advance NCES as a leader in the education data ecosystem. NCES is already actively involved in certain recommended actions and can build on them. The Center can also invest some internal resources to activate external resources as a force multiplier, by aligning those external resources to its strategic plan. While there is much work to do, if NCES seriously acts on these recommendations, the Center can take the helm as a meaningful leader in the nation’s education data ecosystem.
Themes and recommendations are summarized in the sections that follow.
___________________
1 Pub. L. 115-435. Available: https://www.congress.gov/bill/115th-congress/house-bill/4174.
FUNDAMENTAL RECOMMENDATIONS CRITICAL FOR ORGANIZATIONAL TRANSFORMATION
The following sections present the fundamental recommendations that are most critical to advancing NCES as a leader in the education data ecosystem. The full report provides details, conclusions, ideas, and suggestions in Chapter 2.
Theme: Develop a Strong Strategic Plan to Make Tough Decisions
RECOMMENDATION 2-1: To direct its future, NCES should develop and implement a bold strategic plan that incentivizes innovation and creative partnerships and that will produce relevant, timely, and reliable statistical products to assist education decision makers at every level of government. NCES should develop and begin implementation of the plan within 1 year of the release of this report.
Strategic planning allows an organization to focus on high-value products and services for maximum effectiveness and mission impact. Not only has the social environment for education statistics changed, NCES’s mission itself has expanded in scope. To meet its mission effectively, any government agency needs to understand its identity and core values, so that it can establish priorities that guide decision making and operations. Strategic planning helps agencies to proactively clarify the tradeoffs and greater goals that must be navigated during decision making.
Such forethought is even more critical for a small agency like NCES, with limited resources, many stakeholders, a proportionally large administrative burden, and unfunded mandates. Strategic planning will allow NCES to determine its identity, values, priorities, and tradeoffs, to understand where it can add the most value, and to determine how it can best move forward in the 21st century. Once NCES institutionalizes its strategic intentions and implements its priorities with discipline, the Center will be proactive and nimble rather than being so “responsive to immediate demands”2 that its long-term progress and overall effectiveness are impaired. Investing time in strategic planning is an essential step towards NCES directing its own future.
NCES’s strategic plan needs to be comprehensive—describing priorities, gaps, and goals, and delineating ways to leverage new tools and technologies to build forward-looking operations and the necessary infrastructure to achieve stated goals. The panel suggests that the plan also indicate the level of effort needed to manifest goals and objectives, possibly in stages as
___________________
2 NCES response to question from the panel, p. 56.
programs and initiatives evolve. Areas to be addressed in the strategic plan are covered in the remainder of this summary.
Theme: Support and Empower NCES to Set Its Own Priorities
RECOMMENDATION 2-2: The secretary of education, director of the Institute of Education Sciences, and NCES commissioner should collaborate to ensure that NCES is independent in developing, producing, and disseminating statistics.
While NCES is responsible for its own strategic planning and priorities, to achieve the Center’s vision, the secretary of education, other offices in ED, and the director of IES need to fully support NCES. It is essential for IES and ED to empower NCES to manifest its vision and strategic priorities. Together, ED, IES, and NCES are advised to revisit and update internal policies and procedures, to ensure that NCES operates under well-established principles and practices for federal statistical agencies (NASEM, 2021b). NCES should have the authority to make decisions on the scope, content, and frequency of its data and statistics; select and promote professional staff based on skills and knowledge; and “be able to meet with members of Congress, congressional staff, and the public to discuss the agency’s statistics, resources, and staffing levels” (NASEM, 2021b, Practice 2, p. 54). NCES should also have highly qualified staff to make decisions on data content based on scientific and professional considerations, and to gather input on data needs from stakeholders and ED officials (NASEM, 2021b, Practices 3, 4, 5, 9). The panel encourages ED, IES, and NCES to collaborate to ensure NCES operates with its full authority as a federal statistical agency and effectively serves its stakeholders in IES and across ED.
Theme: Incentivize Innovation, Experimentation, and Continuous Learning
The panel recommends that NCES’s strategic plan incentivize innovation, experimentation, and continuous learning throughout all facets of the Center. The strategic plan should consider practices that support a culture of innovation (see NASEM, 2021b). For example, not only is it advisable for NCES to hire staff with cutting-edge skills, but the Center should invest in the ongoing development and professional advancement of its existing staff (NASEM, 2021b, Practice 4). Retaining institutional knowledge is critical for the efficient and effective long-term operation of any agency. NCES can also develop an active research program that includes substantive analyses and evaluates new methods, operations, and alternative data sources for fitness for use (NASEM, 2021b, Practice 5). Finally, to accelerate the shift
to a culture of innovation, the panel advises NCES to forge strong partnerships with other components of IES, innovative agencies, the Center’s contractors, and other external experts to identify best practices and pilot new, potentially transformational approaches (NASEM, 2021b, Practice 7).
Theme: Maximize NCES’s Unique Value for Evidence Building
RECOMMENDATION 2-3: The secretary of education, director of the Institute of Education Sciences, and NCES commissioner should immediately take actions to enable the NCES commissioner to most effectively fulfill the responsibilities of the statistical official delineated in the Foundations for Evidence-Based Policymaking Act of 2018 and to support evidence-building needs across the Department of Education.
The Evidence Act effectively expands NCES’s mission by giving statistical agencies new data-acquisition authority, duties to facilitate data access, and new roles and relationships in evidence building. The Evidence Act establishes NCES’s commissioner as the chief statistical official (SO) of ED, to work closely with the chief data officer (CDO), the chief evaluation officer (EO), and others, to advance ED’s development and use of scientifically rigorous evidence. Evidence-based decision making is the purview of the entire ED, and NCES is congressionally mandated to serve a specific role. Implementation of the Evidence Act is relatively nascent3 and this moment presents an opportunity for the secretary of education, the director of IES, and the commissioner of NCES to establish a central role for NCES as a meaningful partner in ED’s evidence-building activities. Together, the three entities are advised to determine how to best maximize the unique value NCES brings as a producer of credible and relevant evidence, a recognized leader in data standards, and a data-access facilitator. The secretary, director, and commissioner should update all related policies, divisions of responsibilities, processes, and practices, to empower the NCES commissioner to effectively perform the SO duties. The SO brings an important connection between the CDO and the EO, by turning data into high-quality information fit to be used to inform policy and decision making.
The IES director, NCES commissioner, and commissioners of other IES centers should collaboratively determine how to leverage NCES’s unique value. The Evidence Act presents evidence goals to be fulfilled and encourages departments to use new techniques, share and analyze data, and form new partnerships. NCES and IES alone can fulfill this need for ED. The panel urges NCES and other centers within IES to come together to build strong
___________________
3 At the time of writing, OMB had not issued guidance for the implementation of Titles II and III or the Evidence Act.
partnerships, grasp these enormous opportunities, and lead ED’s evidence building and research. These ideas are presented in detail in Chapters 2 and 4.
Theme: Adapt to the Changing World of Education by Increasing Diversity and Awareness of Equity Issues
RECOMMENDATION 2-4: NCES should proactively embed diversity, equity, inclusion, and accessibility in all areas of its work and organization, to adapt and serve contemporary communities of the changing world of education.
NCES’s mission includes being relevant and useful to key stakeholders. Thus, the Center’s data and statistics should address contemporary issues, such as changes in student populations and school environments (e.g., online learning). DEIA efforts are critical to ensuring that NCES’s work is relevant and useful to an increasingly diverse set of stakeholders and society. As a significant producer of education statistics, NCES’s data collections, methods, and products should accurately measure contemporary diverse populations and their lived experiences. To maintain relevance, NCES will need to intentionally consider DEIA issues throughout the data life cycle, from data collection through analysis and publication, including by engaging with members of diverse communities and instilling a culture of DEIA throughout the Center’s staff.
Theme: Expand Data-Acquisition Strategies for New Insights
RECOMMENDATION 2-5: To improve its efficiency, timeliness, and relevance, NCES should continually explore alternative data sources for potential use in data and statistical products, conduct studies on the quality of these sources and their fitness for use, and expand responsible access to data from multiple sources and linkage tools. Testing and adoption of new data-science methods for harnessing alternative data should be done in collaboration with other federal statistical agencies, as well as with other components of the Institute of Education Sciences that are actively exploring ways to strengthen the impact of these techniques.
RECOMMENDATION 2-6: For primary collections, NCES should modernize standard language on consent and planned usage, to permit secure secondary uses that enable high-quality follow-up studies, such as through privacy-protected linkages with other data sources.
In recent years, many federal and state agencies have used administrative
data for statistical purposes, including as evidence building for decision making and evaluation of government programs. At the same time, there has been an explosion of alternative data sources, including commercial data; social media and network data; and internet documents, webpages, and videos that can be harvested or “scraped” (see Appendix B). These potential data sources present enormous opportunities for NCES to advance evidence building.
The panel strongly urges data-source exploration and expansion to be part of NCES’s strategic plan. NCES will need to study the quality and fitness for purpose of alternative data sources, as well as the potential benefits and costs of integrating alternative data sources into existing operations and products. The investment is critical to NCES’s culture of evidence-based decision making and the Center’s ability to keep up with stakeholders’ needs for credible, objective, and timely education data and statistics.
In anticipation of data linkage, NCES should reconsider the consent language and planned usage of all primary collections, to support ongoing uses for statistical activities. The language could ask respondents to allow linkages to other data, could ask for recontact for future requests, or could establish other levels of permission, depending on the respondent population. NCES is advised to look to other federal statistical agencies for best practices when drafting statements regarding privacy of survey respondents’ data and its potential uses. NCES is also encouraged to engage with potential respondents when designing appropriate consent language.
ADDITIONAL RECOMMENDATIONS
The following sections elaborate on additional areas to be addressed by NCES’s strategic plan. The full report provides details, conclusions, ideas, and suggestions in Chapters 3, 4, and 5.
Theme: Prioritize Topics, Data Content, and Statistical Information to Increase Relevance
RECOMMENDATION 3-1: NCES should conduct a top-to-bottom review of its data-acquisition activities, to prioritize topics most relevant to understanding contemporary education, and to discontinue activities that are disproportionately costly and burdensome relative to their value.
RECOMMENDATION 3-2: NCES should revisit priorities mandated by Congress and, where appropriate, make recommendations for changes.
Some of NCES’s data collections (such as the Fast Response Survey System and the School Survey on Crime and Safety) are no longer active due to a lack of adequate staffing, and NCES can expect to make additional hard choices due to limited resources. Thus, it is critical that NCES strategically prioritize its data collections and acquisitions based on its strategic plan. The panel illustrates a process that NCES might follow (see Appendix C) and discusses key topics expected to be covered in the Center’s strategic plan.
NCES recognizes that addressing topics of equity and equal access is a top priority. NCES has strong data collections in place for these topics, which collect basic demographic data and data on student outcomes. Data collection is weaker with regard to measuring the educational process. NCES primarily collects data on the traditional education infrastructure and will need to expand measurement of actions outside of that infrastructure, such as early childhood education, adult education, and career and technical training. Greater attention is needed on the access to and use of technology, including, but not limited to, online learning. NCES may be able to expand the extent and usefulness of its data through increasing its linkages with other agencies at all levels. NCES faces multiple federal mandates for data collection and is recommended to work with Congress to determine whether the need for some of these mandates has changed, and to explore whether alternative data-collection approaches might help address its mandates.
Theme: Expand Engagement and Dissemination for Greater Mission Impact
NCES is the nation’s premier statistical agency concerning education. Thus, the Center should be a primary source of data for Congress, the White House, federal agencies, state and local policy makers, students and their families, education practitioners, researchers, the civil rights–monitoring community, the news media, advisory boards, and professional associations. The following suggested engagement and dissemination activities will help NCES to achieve this goal.
Create Engagement Feedback Loops to Ensure Relevance of Products and Services
RECOMMENDATION 4-1: NCES should deepen and broaden its engagement with current and potential data users, to gather continuing feedback about their needs and ways that NCES can meet those needs more effectively. This feedback will help NCES shape its efforts to develop and disseminate standards, provide technical assistance, and strengthen its user community.
RECOMMENDATION 4-2: NCES should actively collaborate with other data-holding federal agencies and organizations to develop useful products and processes, including those that utilize data from alternative sources, to provide timely, policy-relevant insights.
RECOMMENDATION 4-3: NCES should explore and establish creative models for a nimble, ongoing consulting body, supplemented by a pool of ad hoc consultants, to help NCES innovate and be accountable for progress on strategic goals.
NCES has many stakeholders, and it is a challenge to adequately understand all stakeholder needs. Broad engagement is important to identify and monitor the current and emerging issues that drive product and service effectiveness and improvement. Federal and nonfederal organizations that hold and steward data are also critical stakeholders for NCES.
NCES is connected with the federal statistical agencies through the Interagency Council on Statistical Policy, and the Center could leverage those relationships more fully to advance education statistics and insights. Many agencies and organizations are aggressively pursuing partnerships to link data for evidence building. NCES, too, can enhance the value of its data through linkage projects, research, and dissemination. NCES can and should expand its network to support creative evidence building and increase its mission impact.
NCES should engage a nimble and flexible consulting body, not subject to Federal Advisory Committee Act regulations, to provide strategic advice, innovative ideas, and accountability. This body should have full knowledge of NCES’s work and organization and should provide backing when the Center makes difficult decisions. For topics that require particular expertise, NCES should engage a set of ad hoc consultants to complement the ongoing consulting body.
Expand NCES’s Role Enabling Data Access to Serve and Engage Stakeholders
RECOMMENDATION 4-4: NCES should strengthen state capacity to link data across systems, adopt shared data standards, and provide actionable information to state and local education agencies to help improve student learning outcomes. NCES can leverage its Statewide Longitudinal Data Systems Grant Program to achieve this goal.
RECOMMENDATION 4-5: NCES, in collaboration with the Institute of Education Sciences, should establish a joint statistical research program that includes matching internal staff with highly qualified
external researchers, statisticians, and data scientists to develop new data analyses, tools, and publications.
NCES can maximize its mission impact with minimal investment by expanding its role in data governance, by helping to create the processes and metrics that aid the use of information. NCES already performs aspects of data governance and manages restricted data-use licenses. NCES can expand this governance role by assisting states and organizations with data linkage. NCES can streamline data linkage, prepare and curate data, develop templates, and simplify processes. NCES could provide guidance to state and local data providers regarding best practices for establishing and configuring data infrastructure.
The panel finds that NCES is underutilizing its ability to engage states in achieving its goals. NCES can increase its mission impact by aligning the resources of other organizations to the Center’s strategic goals. For example, in implementing its strategic plan, NCES could consider leveraging the goals and outcomes of the Statewide Longitudinal Data Systems Grant Program (NCES, 2021m). The panel recommends that NCES then promote and circulate these intentions to the states and award proposals to advance these goals.
NCES can support its expanded role in data governance and evidence building by leveraging its existing data-licensing program while also directing data analysis. ED will make faster progress in its evidence-building efforts by both broadening the community of researchers and policy makers that can access data for analysis and by signaling important topics, such as those for which NCES especially wants data. For mutual benefit, we recommend NCES establish a joint statistical research program for external researchers and fellows. NCES would also benefit from expanding and modernizing its data-licensing program, to further increase responsible data access for evidence building.
Improve Dissemination, Focusing on Accessibility and Usefulness
RECOMMENDATION 4-6: NCES should release data and data products that are useful, actionable, and timely for local and state education agencies and other stakeholders. To increase timeliness, NCES, in collaboration with the Institute of Education Sciences, should review and revise its internal and external quality assurance processes.
The content of NCES’s data products and tools is important for building a broad user base. Individual stakeholders have distinct needs for content, quality of information, and timeliness. To be more helpful to state and local school districts with few resources for data analysis, NCES should
deliver products that help local education agencies improve their schools and their students’ outcomes. NCES’s Public School District Finance Peer Search (NCES, 2021i) is an approach the panel recommends extending to other topics. NCES could add analytic features, tools, or templates that provide statistical testing for samples.4 For practitioner-oriented products, NCES could partner with the National Center for Education Evaluation and Regional Assistance, to connect users to relevant research in the What Works Clearinghouse (IES, 2021c). Such collaboration could cross-promote products and boost both centers.
Regarding timeliness, NCES is encouraged to engage with stakeholders about their needs and then determine what is feasible, while exploring acceptable quality tradeoffs to decrease lag time. NCES, in collaboration with IES, is also advised to conduct a top-to-bottom review of both NCES and IES Standards Review Office5 quality assurance processes, to revise review criteria for products requiring improved timeliness, with a goal of being transparent about quality tradeoffs.
Theme: Transform Internal Structure and Operations to Align with and Directly Support the Strategic Plan
RECOMMENDATION 5-1: NCES should utilize contractors and creative staffing arrangements to work collaboratively with staff to build internal capacity. To enhance resilience, NCES should also explore greater use of flexible contract types, stronger incentives for contractors to adopt cost-effective innovations, and performance-based requirements.
Within NCES, the statistics units experienced a net loss of 25 full-time equivalent (FTE) employees (30%) between fiscal years (FYs) 2003–2021, though the assessment units had a net gain of 2 FTEs (7%).6 Parts of NCES are understaffed, resulting in the discontinuation of some surveys and the inability to advance new initiatives. NCES relies heavily on contractors to conduct its day-to-day activities. This trend has increased steadily since FY 2003. Today, on average, NCES staff manage 3.7 contracts7 which amounted to $2.8M per year in FY 2020 (U.S. OMB, 2020)—NCES has
___________________
4 See, for example, U.S. Census Bureau’s Statistical Testing Tool. Available: https://www.census.gov/programs-surveys/acs/guidance/statistical-testing-tool.html [March 2022].
5 The Education Sciences Reform Act of 2002 (20 U.S. Code § 9501 et seq.) requires IES to have a peer review process. See: https://ies.ed.gov/director/sro/ and https://ies.ed.gov/director/sro/ppt/Scientific_Peer_Review.pptx [March 2022].
6 IES document provided to the panel, “IES & NCES Historical FTE Data and IES Appropriations Historical”; NCES response to question from the panel, pp. 7–10.
7 NCES response to question from the panel, p. 15.
the highest budget-to-staff ratio among the federal statistical agencies.8 Wide use of contracts and contractors has been effective in extending and supplementing NCES’s staff when staffing allocations from ED were unavailable. However, a contracting approach can be more costly and can, over time, deteriorate NCES’s institutional knowledge. NCES should consider ways to leverage contractors and other external partners to build and maintain institutional knowledge and in-house innovation. NCES should also partner with contractors to create and revise templates for all steps of the data pipeline—from data-use agreements to presentation graphics style—to embed innovations across the Center and its contractors. NCES’s organization by data source encourages stovepiping and can be a barrier to the innovation, blended data, and cross-fertilization that will be central to the Center’s future success. As NCES expands its audience and adapts its value proposition, its organizational structure needs to evolve to fulfill the Center’s new strategic goals.
The panel finds that NCES is in danger of losing institutional knowledge and innovation capabilities because of decreasing staff size, increasing reliance on contractors, and the Center’s current organizational structure. While many organization-related recommendations will be dependent on the specifics of the Center’s strategic plan, some important activities need to take place as part of the strategic-planning process. While writing a strategic plan, NCES is advised to carefully and thoroughly evaluate options for various organizational structures and features, while considering the agency-specific advocacy necessary to make fundamental changes.
___________________
8 This is the average number of dollars calculated as direct funding in FY 2020 divided by the number of FTE permanent staff. It is used to express the average number of dollars managed by each agency staff member. NCES has a budget-to-staff ratio of approximately $2.75M per FTE—more than seven times the median ratio for the 13 principal federal statistical agencies—according to American Statistical Association–compiled data for the 13 federal statistical agencies (Pierson, 2021).
