Summary

In 2002, Congress passed the Education Sciences Reform Act of 2002 (ESRA), authorizing the creation of the Institute of Education Sciences (IES) as the research arm of the Department of Education, and crystallizing the federal government’s commitment to providing “national leadership in expanding fundamental knowledge and understanding of education from early childhood through postsecondary study” (ESRA, 2002). In the 20 years since its founding, IES has had a field-defining impact on education research in the United States.

IES’s activities are accomplished through four centers: the National Center for Education Research (NCER), the National Center for Education Statistics (NCES), the National Center for Education Evaluation and Regional Assistance, and the National Center for Special Education Research (NCSER).¹ This report focuses on NCER and NCSER. The two centers support a wide range of research activities with the broad goal of improving the quality of education in the United States. These research activities span from infancy through adulthood and across multiple education settings, depending on the center. NCER and NCSER also support the development of the next generation of education researchers through various training programs including predoctoral, postdoctoral, early career, and research methods training programs. The centers fund these activities through a

___________________

¹ In December 2004, Congress reauthorized the Individuals with Disabilities Education Act and, in doing so, authorized NCSER as part of IES. NCSER began operation on July 1, 2005.

competitive grant process, and the funding to support research programs is dependent on annual funding appropriated by Congress.

As the 20th anniversary of ESRA approaches, it is time to consider ways that IES can improve its current research activities and plan for future research and training in the education sciences. Such an examination can ensure that IES-funded research moves the field forward on issues that are of critical importance to education and special education policy and practice and that improve learner outcomes.

THE COMMITTEE’S CHARGE AND APPROACH

In response to a request from the Institute of Education Sciences, the National Academies of Sciences, Engineering, and Medicine through its Board on Science Education convened the Committee on the Future of Education Research at the Institute of Education Sciences to provide guidance on the future of education research at the National Center for Education Research and the National Center for Special Education Research. The committee was tasked with providing guidance on critical problems and issues where new research is needed, how to organize the request for applications, new methods and approaches, and new and different kinds of research training investments.

Research and training activities funded by the two centers have transformed education research and generated substantial insights for education policy and practice. However, the landscape of education and education research have changed substantially since the founding of IES due in large part to the portfolio of work funded by NCER and NCSER. These changes have consequences for how NCER and NCSER need to operate in order to effectively maintain their leadership role in education research.

The committee identified five themes that reflect both advances in education research since the founding of IES and major issues that education will face over the next decade: (1) equity in education, (2) technology in education, (3) use and usefulness of education research, (4) heterogeneity in education, and (5) implementation. These themes formed the lens through which the committee approached its task and developed guidance. The committee’s recommendations are intended to help NCER and NCSER continue to produce transformative education research that will allow IES to maintain its status as the premier funder of education research.

KEY FINDINGS AND RECOMMENDATIONS

New Problems and Issues: Project Types and Topics

NCER and NCSER use both project types (Exploration, Development and Innovation, Initial Efficacy and Follow-up, Systematic Replication, and Measurement) and topics to organize grant competitions. Project types are important for NCER and NCSER’s internal processes, as different types of projects result in different request for application (RFA) requirements and different budgets. These project types have also played a normative role in shaping education research, defining a process through which interventions ought to be developed and evaluated—moving from exploration to development to efficacy and finally to replication.

This structure was developed around the fundamental assumption that the challenges facing schools could be addressed by developing and testing interventions that could be packaged and that would increase student achievement across different school and community contexts. Twenty years into this science, however, it is now clear that this model does not account for the complexities of implementation, nor does it reflect what is now known about how evidence influences or drives changes in practice and policy.

Thus, the committee concluded that the current project type structure needs to be revised. The committee proposes an updated framework for the types of research one might undertake that would better reflect (a) the realities of the heterogeneous contexts in which research in education takes place, and (b) the actual ways in which research is used and engaged in education settings.

RECOMMENDATION 4.1:

IES should adopt new categories for types of research that will be more responsive to the needs, structures, resources, and constraints found in educational organizations. The revised types of research should include

• Discovery and Needs Assessment
• Development and Adaptation
• Impact and Heterogeneity
• Knowledge Mobilization
• Measurement

The committee also concluded that while the current set of topics does a good job of representing the field, the way that topics intersect with the present project types poses a challenge. Under the existing project type structure and given IES’s emphasis on designs that allow for causal inferences, topic areas that can be more readily studied with causal designs (i.e.,

large samples, randomized interventions) are viewed as more competitive by reviewers. Further, NCER and NCSER’s focus on student outcomes means that studies that would focus solely on other outcomes in the system are not eligible for funding. And, if investigators focused on outcomes other than those at the level of students are to make their proposal competitive, it means they likely have to change their research questions to focus on students and/or divert project resources to ensure they are meeting IES requirements. As a result, some of the most pressing topics given the current context of education have not received the attention warranted and need focused attention.

RECOMMENDATION 5.1:

Existing constraints or priorities in the RFA structure and review process have narrowed the kinds of studies within topics that are proposed and successfully funded. In order to expand the kinds of studies that are proposed and successfully funded in NCER and NCSER, IES should consider the following:

• Allowing use of outcomes beyond the student level (classroom, school, institution, district) as the primary outcome
• Expanding the choice of research designs for addressing research questions that focus on why, how, and for whom interventions work

In advance of these structural changes, however, the committee recognizes that the current moment of racial reckoning and responding to COVID-19 requires immediate scholarly attention. Given the issues in education that are emerging at breakneck pace and the subsequent demand for assistance from the field, the committee thinks that designating separate competitions for certain topics is warranted in order to signal their importance even though these topics might technically be “fundable” in existing competitions.

RECOMMENDATION 5.2:

Within each of its existing and future topic area competitions, IES should emphasize the need for research focused on equity.

RECOMMENDATION 5.3:

In order to encourage research in areas that are responsive to current needs and are relatively neglected in the current funding portfolio, NCER and NCSER should add the following topics:

• Civil rights policy and practice
• Teacher education and education workforce development
• Education technology and learning analytics

RECOMMENDATION 5.4:

IES should offer new research competitions under NCSER around these topics:

• Teaching practices associated with improved outcomes for students with disabilities
• Classroom and school contexts and structures that support access and inclusion to improved outcomes for students with disabilities
• Issues specific to low-incidence populations

The topics listed above represent priorities identified by the committee based on our understanding of the current state of education research. This list is not intended to be exhaustive or restrictive; rather, these topics are examples of the types of topics that emerge through consistent, focused engagement with the field. Indeed, the committee recognizes that education research is perennially evolving in response to both the production of knowledge as well as the circumstances in the world. For this reason, the committee advises that the list of topics funded by the centers should also evolve in order to remain responsive to the needs of the field. This responsiveness is a necessary component of fulfilling the obligations laid out in ESRA: in order to “sponsor sustained research that will lead to the accumulation of knowledge and understanding of education,” it is important to fully understand not only what knowledge has accumulated, but also where the existing gaps are.

RECOMMENDATION 5.5:

IES should implement a systematic, periodic, and transparent process for analyzing the state of the field and adding or removing topics as appropriate. These procedures should incorporate:

• Mechanisms for engaging with a broad range of stakeholders to identify needs
• Systematic approaches to identifying areas where research is lacking by conducting syntheses of research, creating evidence gap maps, and obtaining input from both practitioners and researchers
• Public-facing and transparent communication about how priority topics are being identified

Methods and Approaches

IES will also need to re-orient its investment in methods and measures. In developing guidance on research to advance new methods and approaches the committee kept in mind that IES’s charge requires that the Institute maintain a focus on “what works.” Since causal questions are inherently comparative, descriptive work is also needed to conceptualize and

describe current practices and the context of schools as a means for full understanding of the comparisons being made. Also, in order to fully understand why and how a particular intervention or program is working, the questions of what works and how it works need to be pursued in concert.

In reviewing the balance of funded work on methods and measures to date, the committee identified key gaps that need to be addressed moving forward. The committee recommends:

RECOMMENDATION 6.1:

IES should develop competitive priorities for research on methods and designs in the following areas:

• Small causal studies
• Understanding implementation and adaptation
• Understanding knowledge mobilization
• Predicting causal effects in local contexts
• Utilizing big data

RECOMMENDATION 6.2:

IES should convene a new competition and review panel for supporting qualitative and mixed-methods approaches to research design and methods.

To respond to the new study types and priority topics and to support the continued growth of methods, new measures and new approaches to measurement will be required. For this reason, we offer a recommendation for IES to consider related to measurement research that will support continued growth in other parts of NCER and NCSER’s portfolio.

RECOMMENDATION 6.3:

IES should develop a competitive priority for the following areas of measurement research:

• Expanding the range of student outcome measures
• Developing and validating measures beyond the student level (e.g., structural and contextual factors that shape student outcomes; teacher outcomes; knowledge mobilization)
• Developing and validating measures related to educational equity
• Using technology to develop new approaches and tools for measurement

Training Programs

The training portfolio offered by NCER and NCSER is an important and vital function that has helped strengthen the education research field, and it is imperative that these programs continue to be offered to education

research scientists. While IES continues these programs, there is also a need for more equitable opportunities and transparency in the offered trainings within both NCER and NCSER. Data that look at who is participating in the training programs are not readily available, and we do not know about the success of training as there are no obvious indicators of success created by IES. There is also a clear opportunity to build on current programs and expand trainings in methods to attend to the high demand among researchers. Finally, IES can implement a variety of strategies that can help broaden participation within its training programs and in turn, continue to strengthen a highly reputable portfolio. The committee recommends:

RECOMMENDATION 7.1:

IES should develop indicators of success for training, collect them from programs, and then make the information publicly available. IES should report the data it already collects on the success of programs and pathways of trainees post-training.

RECOMMENDATION 7.2:

IES should build on its current strengths in methods training and expand in the following areas:

• Methods to address questions of how and why policies and practices work
• Methods that use machine learning, predictive analytics, natural language processing, administrative data, and other like methods

To fully meet the needs of the field as outlined in ESRA, IES has a responsibility to ensure that its training programming is reaching populations of scholars and researchers who need it most. As the committee notes in this report, this is an important issue of equity in the education research community. In addition, there is tangible value in ensuring that the field of education research is diverse insofar as it improves the overall quality of eventual research, increases the likelihood that issues of equity will be taken up in research, and supports the ultimate identity-building of future researchers.

RECOMMENDATION 7.3:

IES should collect and publish information on the racial, ethnic, gender, disability status, disciplinary, and institutional backgrounds (types of institutions including Historically Black Colleges and Universities and Minority-Serving Institutions) of applicants and participants in training at both the individual and institutional levels.

RECOMMENDATION 7.4:

IES should implement a range of strategies to broaden participation in its training programs to achieve greater diversity in the racial, ethnic, and institutional backgrounds of participants. These strategies could include

• Implementing targeted outreach to underrepresented institution types
• Supporting early career mentoring
• Requiring that training program applications clearly articulate a plan for inclusive programming and equitable participation
• Offering supplements to existing research grants to support participation of individuals from underrepresented groups
• Funding short-term research opportunities for undergraduate and graduate students

RFA and Review

The committee concluded that the explicitness of the RFAs used by NCER and NCSER was one of the strengths of the IES grant review system, even as the detailed requirements result in lengthy proposals. Similarly, the committee viewed the review process of IES as a strength. Unlike other agencies (e.g., National Science Foundation), IES program officers have no role in the review process, other than to encourage applicants and provide guidance on the RFAs. Thus, the determination for funding arises only in relation to the final proposal score and the cut-score for that particular year.

Despite these strengths, the committee identified three central challenges that undermine the effectiveness: (1) IES does not publicly share information on its applicants, reviewers, and grantees, making it impossible to track whether the application and review process is resulting an equitable distribution of awards, and if not where in the process disparities are introduced; (2) the current procedures do not provide IES with sufficient information throughout the process to assess the potential impact of projects, including the significance of individual proposals, and the extent to which proposals collectively cohere as a program of research; and (3) the current procedures undermine IES’s ability to be timely and responsive to the needs of the education research community. To address these challenges, the committee recommends:

RECOMMENDATION 8.1:

IES should regularly collect and publish information on the racial, ethnic, gender, disciplinary, and institutional backgrounds of applicants and funded principal investigators (PIs) and co-PIs, composition of review panels, and study samples.

RECOMMENDATION 8.2:

IES should review and fund grants more quickly and re-introduce two application cycles per year.

The committee thinks that attending to the larger structural issues facing NCER and NCSER (see Recommendations 4.1 and 5.1–5.5) will serve to help ensure that funded research is better positioned to be useful for practitioners and policy makers. However, the effects of implementing these recommendations may take several years to emerge, and the committee notes that the field needs useful research as soon as possible. For this reason, we offer two recommendations that may help ameliorate some of the challenges related to usefulness that the committee laid out. First, we suggest that the RFA adjust expectations around collaboration so that stakeholders in communities engaged in funded research are fully included in project planning.

RECOMMENDATION 8.3:

For proposals that include collaborating with LEAs and SEAs, the RFA should require that applicants explain the rationale and preliminary plan for the collaboration in lieu of the current requirement for a letter of support. Upon notification of a successful award, grantees must then provide a comprehensive partnership engagement plan and letter(s) of support in order to receive funding.

The committee also noted the current lack of a consistent plan for engaging practitioner and policy maker perspectives in the application and review process. There are multiple ways that IES might want to leverage these communities, ranging from consistent participation on panels to separate working groups, but the committee notes that practitioner and policy maker communities should be involved in determining the mechanism that works best for IES. The ultimate goal of this work is for IES to define a role for these communities that is both distinct and meaningful, such that these already burdened professionals can maximize their valuable time and effort.

RECOMMENDATION 8.4:

IES should engage a working group representing the practitioner and policy maker communities along with members of the research community to develop realistic mechanisms for incorporating practitioner and policy maker perspectives in the review process systematically across multiple panels.

Enabling Recommendations

Throughout this report, the committee returns to two major issues that constrain IES’s ability to support research that attends to the needs of all students. The first issue is the lack of consistent reporting and analysis related to who applies and is funded in NCER and NCSER competitions, which limits the extent to which IES can ensure that funded research and researchers truly represent the needs of the communities they are intended to serve. Second, IES is afforded a relatively modest budget compared to other federal science agencies. The committee agreed that in order for IES to truly achieve the vision of these recommendations, it is critical to also address both of these issues. As such, the committee recommends:

RECOMMENDATION 9.1:

In addition to implementing the recommendations highlighted above, NCER and NCSER should conduct a comprehensive investigation of the funding processes to identify possible inequities. This analysis should attend to all aspects of the funding process, including application, reviewing, scoring, and monitoring progress. The resulting report should provide insight into barriers to funding across demographic groups and across research types and topics, as well as a plan for ameliorating these inequities.

RECOMMENDATION 9.2:

Congress should re-examine the IES budget, which does not appear to be on par with that of other scientific funding agencies, nor to have the resources to fully implement this suite of recommendations.

REFERENCE