5

Research Topics for NCER and NCSER Grants

The first charge of this committee was to identify critical problems or issues on which new research is needed. We began our response to this charge in Chapter 4 with a discussion of project types (or goals) of studies supported by the Institute of Education Sciences (IES). In this chapter we continue our response to the first charge by considering the other axis of the IES “matrix,” asking what new topics should be addressed by IES-funded research. To inform this question, we heard testimony from a variety of education researchers, practitioners, and other stakeholders across the landscape. We examined data on investments by the National Center for Education Research (NCER) and National Center for Special Education Research (NCSER) in research across topics over time. We also drew on the committee’s diverse and extensive expertise. However, when sitting down to identify new topics for NCER and NCSER to invest in, the committee struggled to identify a clear set of issues that were not already technically “fundable” in IES’s current structure and organization. At almost every suggestion, the committee found a place in the topic structure where a hypothetical study could technically fit. And yet, it is undeniable that IES has accumulated research evidence in some areas far more than in others.

In this chapter, we describe the nature of this challenge. We begin with an overview of how NCER and NCSER use topics to organize their funding opportunities. We then outline barriers within the existing topic structure that prioritize some forms of research at the expense of others. Next, we provide considerations for how NCER and NCSER might develop a mechanism for revisiting these issues in the future to ensure that the development

of research is dynamic, cumulative, and responsive to changing times. We conclude the chapter by identifying a small set of topics that are of critical, immediate importance.

OVERVIEW OF TOPICS

NCER and NCSER use topic areas to communicate research needs and to help manage applications that come in through their grant competitions (see, for example, p. 2 of the FY2021 NCER Education Research Grants request for applications [RFA] and p. 9 of the FY2021 NCSER Special Education Research Grants RFA for a discussion of how they use research topics). Additionally, topic areas allow the research centers to distribute applications across program officers to provide targeted feedback throughout the application process and to efficiently assign applications to peer reviewers with the appropriate expertise.

In FY2021, there were 11 topics supported by NCER and 9 supported by NCSER (see Table 5-1 for the list of topics).

Across all topics in the Education Research and Special Education Research Grants competitions, applicants are invited to submit proposals to any of IES’s five project types: Exploration, Development and Innovation, Initial Efficacy and Follow-up, Replication/Effectiveness,¹ and Measurement. (See Chapter 4 for our proposed revision to these project types.) Jointly, the intersection of types and topics forms a kind of matrix which serves as an organizational framework for the Education Research Grants and the Special Education Research Grants competitions (Schneider, 2021).²

In theory, grouping research into these topics allows NCER and NCSER to be responsive to changes in the field: they can take stock of what has been learned and diagnose where further research is necessary. The committee saw evidence of this in practice. NCER and NCSER routinely add or remove topics based on emerging or changing needs. In FY2021, NCER added a new standing topic focused on Civics Education and Social Studies, which had previously been competed as a special topic in FY2019 and FY2020. NCER and NCSER also removed Education Technology and Technology for Special Education as standalone topics, with the rationale that education technology plays a central role across all topic areas. NCER and NCSER have also at times changed the names of topics to reflect

___________________

¹ For consistency, we include (4) “replication” here, as this is how it has been discussed in Chapter 4. However, more accurately, this project type does not exist in the most recent RFA. Instead, replication studies have a separate RFA altogether.

² Other research grant competitions supported by NCER and NCSER do not rely on this matrix structure.

TABLE 5-1 FY2021 NCER and NCSER Topics

SOURCE: Committee generated; adapted from Request for Applications (IES, 2021).

changes in conventions in the field or to signal to a broader set of scholars that their research is welcome within a given topic area.

NCER and NCSER also use their annual RFAs as a way to provide broad descriptions of its topics and to indicate areas of “Needed Research.” For example, in the FY2021 NCER RFA, the Cognition and Student Learning topic highlighted the need for “Exploratory research to guide the development and testing of education technology products that can personalize instruction.” One tension the research centers face in providing such descriptions is that investigators who do not see their research interests explicitly named in the topic description may choose to modify the goals of their work. Or worse, they may choose to forego applying to IES entirely. As a result, NCSER has in recent years aimed to broaden the kinds of research it supports by removing language that specifies needed research.³

Finally, in addition to their lists of standing topics, NCER and NCSER also include special topics within the Education Research Grants and Special Education Research Grants competitions to respond to pressing issues in the field, or to jumpstart research in areas that have not received

___________________

³ This sentence was modified after release of the report to IES to correct information about the actions that NCSER has taken to broaden the kinds of research it supports.

adequate attention. For example, in FY2019, NCSER opened a special topic focused on Career and Technical Education for Students with Disabilities that continued into FY2020. In FY2020, it included a special topic on English Learners with Disabilities. These special topics, in theory, allow the research centers to adapt to the changing landscape.

THE CHALLENGE OF TOPICS

Overall, the committee agreed that IES’s matrix of possible research areas, organized by topics and project types, corresponded well to the broad network of educational research (again, see our proposal for a revised set of project types in Chapter 4). The challenge for the field is how research has accumulated across this matrix. Some of this is to be expected: Knowledge will naturally accumulate at varying rates across IES’s project types and topics. For a field as diverse as education, it is understandable that researchers would gravitate toward certain programs of research. But in its review of which topics actually receive funding, the committee noted that, in reality, a series of barriers exist both internal and external to IES that hamper the potential for funding for a set of critically important topics.

Our committee’s key takeaway is that the challenge of topics is situated not within the topics themselves. The current set of topics do a good job representing the field. Instead, the committee determined, the challenge lies in how these topics intersect with the present project type structure. Under the existing structure, studies designed toward certain project types lend themselves to demonstrating rigor as described and prioritized by IES (see Chapter 2). In practice, this means that topic areas that can be more readily studied with these methods (i.e., large samples, randomized interventions) are more competitive with reviewers. Further, NCER and NCSER’s focus on student outcomes means that studies that would focus on other outcomes in the system are less likely to receive funding. If investigators focused on outcomes other than those at the level of students are to make their proposals competitive, it means they likely have to change their research questions to focus on students and/or divert project resources to ensure they are meeting IES requirements. The committee’s concern is not that measuring other outcomes is discouraged, but that the requirement to measure students’ academic outcomes takes the focus away from outcomes at other levels, especially for system-level studies.⁴

___________________

⁴ In their analysis of public data on NCER- and NCSER-funded projects, Klager and Tipton (2021) reported that in Development and Innovation, Efficacy, Effectiveness, and Replication studies, 71% of NCER-funded and 72% of NCSER-funded studies focused on student factors as malleable conditions, whereas only 18% of NCER-funded and 20% of NCSER-funded studies focused on teachers and even fewer on classroom, school, or system factors (12% NCER, 8% NCSER).

The Case of Teacher Education

To illustrate this challenge, we use the example of research on teacher education—although we acknowledge that the challenges described can easily be applied to many other areas, a point we return to later in this chapter. There are many reasons IES might want to invest in research on teacher education. There is widespread evidence that teachers play a critical role in improving student outcomes (e.g., Hanushek & Rivkin, 2006), so initial preparation could serve as a key learning opportunity for future educators (e.g., Theobald et al., 2021; Brownell et al., 2019; Ronfeldt et al., 2018). Further, for an organization such as IES, teacher education could play a complementary role to its existing program of research, ensuring, for example, that future educators are equipped with knowledge on effective academic and behavioral interventions for students. Finally, the field of teacher education would benefit a great deal from the investment. Teacher education remains highly localized, with little consistency in how teachers are prepared across programs (CRMSE, 2010; Wilson, Floden, & Ferrini-Mundy, 2001). And causal evidence in teacher education is exceedingly rare (Hill, Mancenido, & Loeb, 2020).

Yet, across 20 years, NCER and NCSER have funded only a handful of studies explicitly focused on teacher education. This is an area of research where there is a clear need, where the challenges have been longstanding, and where the research centers have simply not made much headway. The topic structure does not seem to be the source of the problem. NCER has always maintained a topic focused on Effective Teaching (previously known as Effective Instruction), and the studies on teacher education that have been funded have most commonly fallen under this topic (e.g., Grant #R305M060065 (2006), #R305A180023 (2018)). Grantees have also found a home for teacher education research under Systems, Finance and Policy (#R324A170016), and Researcher-Practitioner Partnerships (#R305H170025), among others. In recent RFAs, NCSER has even expressly called for a concerted focus on teacher education research.

So, while the current topic structure looks as though it could fund this work, in practice, teacher education research has largely gone unfunded. The challenge, it seems, relates to the project type structure employed by NCER and NCSER as well as their study requirements. Notably, we could find no teacher education projects that have been funded under the Initial Efficacy and Follow-Up project type (nor the Measurement project type for that matter). All of this work has been either Exploratory or Development studies. The lack of efficacy trials in teacher education may reflect the challenge in applying research designs developed for other contexts (such as K–12 settings) in preservice teaching programs. Efficacy or Effectiveness studies would require a sufficient number of students within a teacher edu-

cation program as well as a sufficient number of programs. Such cross-site coordination rarely occurs. Nor has teacher education research exhibited substantial progress in methodological development (Hill et al., 2020).

A second challenge is the NCER and NCSER requirement that funded studies focus on and include measures of student outcomes. Researchers who study teacher education face problematic constraints in tailoring their research for IES funding. Eventual student outcomes (once preservice teachers transition into their first teaching jobs) are both distal and often secondary to the target of an intervention. More proximal student outcome options are limited, restricted to the progress made by the students of teacher-candidates during the clinical teaching placement, which is only partially attributable to the candidates themselves. Teacher education is certainly not the only area of research subject to these limitations. Studies that would address subject areas that are not tested for accountability purposes, such as science or social studies, have historically run up against similar challenges.

In sum, we argue that the lack of research on teacher education is not one that could be fixed through the mechanism of topics alone. NCER and NCSER could explicitly call out the need for teacher education research—which may be a good idea in its own right—but without making broader changes to their project type structure and to the outcomes they prioritize, it is unlikely that things would change much beyond the current situation.

Teacher education is just one of the many topics that is likely to face challenges like these. Similar claims could be made about research on changing systems or policy effects, where NCER and NCSER have funded considerably fewer projects than other areas, or on improving teacher or administrator quality through professional development. When problems of research do not naturally lend themselves to randomized controlled trials, or when the direct focus of change is on stakeholders other than students, the pathways to funding at IES’s research centers can be prohibitive.

A SYSTEM FOR UPDATING TOPICS AND RESEARCH PRIORITIES

NCER and NCSER have used a number of mechanisms over the years to modify topics included in their grant competitions, whether by adding, combining, or removing standing topics; changing the names of topics; changing descriptions of topics; or holding occasional special topics or topically focused competitions. While our committee acknowledges that these steps have allowed the research centers to adapt over time, we note that in order to truly respond to the field, IES will need to go a step further. A more systematic, transparent process would strengthen IES’s responsiveness to the needs of the education research community. Such a mechanism

could be used to both (1) assess demand for and awareness of research by key stakeholders, and (2) identify where the supply of research is lacking, inconclusive, or contradictory. Such information can then guide efforts in production, curation, or dissemination of research. Where demand exceeds awareness, IES might then promote greater engagement with existing research products. Where supply exists but not in a usable format to satisfy demand, IES might create more usable practice guides or commission syntheses with plain-language recommendations. Where supply and demand are not aligned, IES can then adapt its research portfolio by adjusting its topic by project type matrix, as well as the questions embedded within those topics and project types.

Although research priority setting is a complex process lacking consensus on best practices, some common themes emerge, such as inclusive stakeholder engagement, relevant criteria and methods for deciding on priorities, and transparency (Viergever et al., 2010; Sibbald, et al., 2009). Numerous methods have been tested in health research, with a summary and critique of their strengths and weaknesses described by the World Health Organization (2020). This publication identifies three categories of criteria: public benefit, scientific feasibility, and cost. It also identifies two types of methods for deciding between priorities: consensus-based approaches and metric-based approaches. Consensus approaches (e.g., Ghaffar et al., 2009) allow for values to drive priorities, but should be balanced by methods that account for diverging perspectives, such as deliberative dialogues (McDonald et al., 2009). Metric-based approaches (e.g., Rudan et al., 2008; Dalkey & Helmer, 1963) aggregate perspectives across a broader audience to generate a list of top priorities, which may then be published or undergo further deliberation (e.g., through the James Lind Alliance).

To expand and strengthen IES’s current processes for identifying new research priorities, we highlight key themes rather than suggest the adoption of a specific method to inform research priorities. In particular, we emphasize the important roles for diverse and equitable stakeholder engagement, evidence synthesis, and greater visibility and transparency.

First, engaging with a broader range of stakeholders (policy makers, practitioners, and other community members, as well as researchers) would build a richer picture of where they perceive needs for better research knowledge. While policy makers, practitioners, and the general public would provide key insights about relevance and benefit, researchers would be better positioned to comment on scientific feasibility, as well as where there are unresolved conflicts or gaps in the research base. Both may offer important perspectives on cost, with the former addressing the cost of implementing potential strategies and the latter addressing the cost of conducting the research. Enlisting existing networks, such as the Regional Comprehensive Centers, Regional Education Laboratories, and professional

associations, can help expand IES’s reach. When analyzing stakeholders’ different roles in the research enterprise (e.g., Brugha & Varvasovszky, 2000; Haddaway et al., 2017), applying an equity lens will be critical for rectifying imbalances in values, opportunities, and impacts (Nasser et al., 2013).

Second, tighter coordination between the priority-setting and evidence synthesis processes would further build understanding of how the evidence base compares to the questions being asked. This could help to identify which topics and questions (1) have existing syntheses which need better dissemination, (2) have existing research which needs to be synthesized, or (3) need more research to be produced. Both NCER and NCSER could work with the National Center for Education Evaluation and Regional Assistance (NCEE) to commission practice guides, syntheses, or evidence gap maps in response to emerging demand. In Chapter 4, we also discussed the import of funding research syntheses within a new Knowledge Mobilization project type. Conducting these syntheses across the different goals could illuminate gaps or surpluses in the progression of research. Specifically, they could reveal where there are needs or potential practices (identified during Discovery and Needs Assessment) that lack adequate intervention development, and where promising interventions (from Development and Adaptation) have not yet been adequately evaluated for effectiveness across the range of populations and contexts needed. Alternately, they could reveal where interventions and programs are proliferating, instead of converging on core components. They could reveal where mismatches between research and practice may motivate further study of knowledge mobilization strategies. They could also reveal where new measures are needed or where common measures are needed.

Third, increasing the visibility and transparency of these processes can engage a wider audience in the research enterprise, helping to build awareness, interest, and trust in both existing and emerging research. With clear routines and timelines for engagement, multiple groups of stakeholders would be able to anticipate when and how to provide input and learn about the perspectives of others in the field.

Some potential instantiations of these themes may be to engage in the following activities at routine intervals, such as every 3 years:

• Form an equity committee that releases data and issues equity reports, documenting areas where research is needed, and reporting who has gotten funded;
• In collaboration with NCEE, provide mechanisms for broad community input through an online suggestion form, surveys, and focus groups embedded within existing networks (e.g., professional associations);

• Hold NCER and NCSER researcher panels and community panels for deeper engagement, chaired by a researcher and an IES program officer, who collaborate to identify issues that both the research and practice community see as important unanswered questions in the field;
• On an ongoing and rotating basis, conduct research syntheses based on existing topics, identifying gaps in the research knowledge. Researchers can apply for the (small) contract to complete the synthesis;
• Delineate and document these processes and outcomes transparently, so that stakeholders understand opportunities for input and how their input is being used.

NEW TOPICS

Implementing the above procedures would provide IES with ongoing information about urgent and emerging needs within the field. But given the current circumstances—including both an unprecedented global pandemic and necessary racial reckoning for continued acts of prejudice and violence against historically marginalized groups in this country—the committee would be remiss if it did not provide specific guidance surrounding topics that likely demand immediate action. The field cannot wait for IES to update its processes for integrating new information from more systematic processes if education is to meet the challenge these historical circumstances demand. Thus, drawing on testimony, commissioned papers, our committee’s collective expertise, and the crosscutting themes identified in this report, we nominate a small number of topics that merit a concerted investment in the coming years. These nominations should not be taken as an exhaustive or restrictive list; rather, they are examples of areas of potential study that emerge when the field is engaged in a process of assessing its needs.

Civil Rights Policy and Practices

Education researchers have produced valuable empirical and conceptual studies on the context of equity in education over the past 20 years. From this literature, it is clear that U.S. schools are more diverse racially and ethnically, but also more segregated and unequal. This paradox is due, in part, to historical legacies of policies related to housing, zoning, and employment, all of which are affected by racial injustice. More recently, in the past two decades, the courts have moved away from desegregation as a remedy for state-sponsored segregation, even as schools continue to be marked by deepening stratification (Gamoran, Collares, & Barfels, 2016).

Economic inequality is also at historic highs (Gamoran, 2015), and the relationship between racial and economic inequality is deeply intertwined and expressed in housing, labor, health, and educational opportunities (Reardon et al., 2021). The COVID-19 pandemic, the opioid crisis, and the struggles to find and maintain reliable housing, food, and health care have deep implications for educational institutions, educational interventions, and the study and measurement of both. For too long, researchers have been trained to elide these contexts in their examinations of educational innovations, and as a result, missed opportunities to build the field’s understanding of the importance of the intimate linkages between the context of schooling and learning and achievement.

IES, through NCER and NCSER, has an opportunity to help build our understanding of how interventions and approaches to teaching, learning, and school processes are informed by these contextual factors for the range of students being educated. In addition, there are important understandings of the within-school practices related to racial and socioeconomic inequality that could be enriched by further robust research (Horsford, 2011). For example, Black, Latinx, and Native American students are far more likely to experience discipline in schools that leads to suspension or expulsion (Losen et al., 2016; Okonofua & Eberhardt, 2015). Also important, students whose identities exist at intersections, such as Black, LGBTQIA, disabled, and/or multilingual children and adolescents, are especially vulnerable to being targeted for harsh discipline that harms their opportunities to learn and predicts greater likelihood for disassociating from school, dropping out, and becoming part of the carceral system as they are referred to police and the courts for behavioral infractions, or simply failing to reach their potential as learners (Scott et al., 2017; Shedd, 2016; Skiba et al., 2011).

Given the challenges within K–12 schooling and for students with disabilities from preschool through age 21, along with the deepening and persistent inequalities that shape school systems, the teaching force, and the learning conditions within and across schools, it is imperative to support and strengthen different epistemological and methodological approaches for investigating issues at the intersection of education and civil rights. As Johnson (2021) argued, IES is not yet equipped with the expertise and systemic data collection and databases to answer questions about racialized mechanisms that shape learning opportunities, experiences with racism and violence in and out of school, and the effects of carceral policies within and out of school. IES could help to support the development of robust metrics to understand race, racialization, and racism in schools and systems; support interventions to remedy inequality; and identify cases that have made progress towards equitable outcomes (Scott et al., 2020).

Consistent with the committee’s focus on equity as a crosscutting theme, and in line with President Biden’s Executive Order on Racial Equity (EO

13985), the committee sees a need for the future of IES-funded research to be purposively oriented toward addressing the needs of underserved communities. To these ends, IES could better support research on equity and civil rights policies by funding research that responds to the education field’s knowledge of how racial injustice in the structures, processes, and practices of schools and systems have an impact on learning and lifetime outcomes by supporting new research on what schools and other education settings can do to mitigate these effects. This might include, for example, research on

• School discipline: Disparities in discipline are well documented, and schools are engaged in a variety of strategies intended to reduce or eliminate these disparities. IES-funded researchers would find willing partners in states and school systems committed to better understanding the conditions that give rise to disparities and the diverse impacts of efforts (such as restorative approaches) to mitigate them.
• Services and supports for students with disabilities: Students with disabilities are likely to have experienced considerable challenges to receiving appropriate supports and services, and considerations for effective mechanisms for engaging students in productive ways educationally are needed.
• COVID-19 and orphans: Over 160,000 children in the United States and 1.5 million worldwide have lost a parent or caregiver. With these numbers likely to grow given unequal access to health care, and with Black, Latinx, and Native American children more likely to have experienced such loss, it is necessary to know how the practice of education can be made responsive to the trauma inflicted by the pandemic on educational opportunities and student well-being, learning, and educational attainment (Cluver, 2021; Imperial College of London, 2021; Treglia et al., 2021).
• Bullying and violence prevention: School violence and bullying pose serious problems, especially for students with intersectional identities based on race, ethnicity, disability, sexual orientation, and gender identity (Esplenage, 2015). Research is needed to identify programs that may work, in specific contexts, to eliminate violence and bullying, with a focus on structural conditions as the source of the problem and the student experience, rather than achievement as the outcome.
• School racial composition: Ongoing research indicates that racial segregation exacerbates inequality because it concentrates Black and Latinx students in high-poverty schools, which tend to be less effective than low-poverty schools (Reardon et al., 2021). Research is needed to examine voluntary and mandatory policies to break

• the link between segregation and concentrated poverty and to ensure high-quality learning opportunities for all children.

Teaching Quality and the Teacher Workforce

Teachers (in both general and special education) serve as the primary interface between students and education in the United States, and yet improving the quality of the teacher workforce represents a notably understudied part of NCER and NCSER’s portfolio. To be clear, many IES-funded studies have relied on teachers, often as the ones who carry out academic or behavioral interventions. Less common are studies that focus specifically on changing the knowledge, skills, practices, and dispositions of the teacher workforce. As described previously in this chapter, many of the reasons for this go beyond the question of topics. With IES’s strong focus on student outcomes, researchers have had fewer avenues for exploring interventions where teacher outcomes are the focus. As we note later in Chapter 6, there has been minimal investment in measurement studies focused on teacher outcomes. The field lacks both IES-funded studies that have focused explicitly on teachers as well as suitable tools for measuring the effects of interventions targeting teachers.⁵

The committee identified research on improving the teaching workforce as a pressing need within both NCER and NCSER. Improving the workforce is a longstanding need but one that has intensified in response to changes in the educational landscape. As articulated in the recent National Academies’ report on the teacher workforce: “Teachers are called on to educate an increasingly diverse student body, to enact culturally responsive pedagogies, and to have a deeper understanding of their students’ socioemotional growth. Integrating these various, layered expectations places substantially new demands on teachers” (NASEM, 2020, p. 187) as educators are tasked with supporting students in the wake of COVID-19.

The committee recognized the need for research addressing teacher education (TEd) and professional development (PD). Although there is substantial empirical evidence about the critical importance of teachers for promoting students’ academic and long-term success (e.g., Chetty, Friedman, & Rockoff, 2014; Aaronson, Barrow, & Sander, 2007; Clotfelter, Ladd, & Vigdor, 2007; Rivkin, Hanushek, & Kain, 2005; Darling-Hammond, 2000), there are sizable knowledge gaps about the initial preparation and

___________________

⁵ The committee wishes to note that school leaders and other professional leaders clearly deserve the same scholarly attention as teachers, and are similarly overlooked in IES’s portfolio for the reasons highlighted in this chapter. Though teachers play a particular role in supporting student achievement because of their direct proximity to students, it is also critically important to understand the impact and potential of other professionals in the school building and throughout the education system.

PD of teachers (Hill, Manciendo, & Loeb, 2021; Phelps & Sykes, 2020; Fryer, 2017; Waitoller & Artiles, 2013; Sindelar, Brownell, & Billingsley, 2010). Research on TEd in this field has been described as “scattered and thin” (Sindelar et al., 2010, p. 13). Reviews of the literature have consistently described the research foundation in these domains as “weak” and identified limitations in the quality and focus of this scholarship (Ronfeldt, 2021; Brownell et al., 2019; National Research Council, 2010; Sindelar et al., 2010; Cochran-Smith & Zeichner, 2009; Wilson et al., 2001). Research on pedagogical practices has been emphasized in the past two decades, but “knowledge accumulation on teacher education … has been uneven and in many areas, sparse” (Brownell et al., 2019, p. 35). Greater support for research on the initial and continuing education of teachers will improve the design and impact of these programs and interventions, improve teacher quality, and ultimately influence the quality of services provided to students. Specific considerations for additional research are as follows:

• Systematically investing in a range of kinds of research studies to bolster knowledge of effective systems of teacher professional learning that better prepares teachers to effectively meet the needs of a range of learners including those with disabilities. For example, in the case of teacher education, this might look like (a) effectiveness studies to establish teacher education practices resulting in improved candidate outcomes, (b) qualitative studies to explore promising practices and underlying mechanisms, and (c) descriptive studies linking program features to long-term candidate and student outcomes. This will contribute to the advancement of a knowledge base that is rich in explanatory and contextualized models.
• Identifying effective approaches for preparing educators for the complexities of the student population, changing professional roles, and the improvement of outcomes for all students.
• Substantiating research programs on teacher learning with a close attention to theory. Scholars have noted the lack of a sustained and coherent approach in the study of TEd and PD (Billingsley & Bettini, 2019; Brownell et al., 2019; Kennedy, 2019).
• Exploring research designs that support causal inferences in the contexts of TEd and PD research.
• Developing measures that are proximal to the goals of teacher education and professional development. As an example, recent advances have been made in measuring teacher content knowledge and establishing parameters for using teacher content knowledge as an outcome measure in cluster randomized trials (e.g., Kelcey et al., 2017; Phelps et al., 2016). Similar lines of research are necessary to

• develop validated, useable measures of teachers’ practice that might complement existing observation tools.
• Studying the broader workforce issues that impact the success of TEd and PD opportunities, including ongoing issues related to teacher supply. Issues of teacher supply are particularly relevant in special education where teacher shortages have existed for decades. In the past 20 years, the landscape of teacher licensure has changed dramatically, with the proliferation of a variety of programs and pathways into the classroom (NASEM, 2020). Researchers have begun to look generally at how licensure pathways shape the teaching workforce (Ronfeldt, 2021), but further work is necessary. In particular, we need further research on how best to support schools in staffing the hardest areas to fill (special education, science, technology, engineering, and math).
• Understanding the intersection between education technology and teacher learning in both TEd and PD. This may include, for example, examining the effectiveness of new online TEd or PD programs. Or, it may involve technology to supplement existing learning opportunities, such as the use of simulations in teacher education or providing automated feedback to educators based on videos of classroom instruction.
• Increasing synergies and complementarities in TEd and PD research in general and special education. Increasing awareness of the complexities of student needs complicates the initial preparation and PD of teachers. Teachers are expected to provide quality instruction and social-emotional learning supports to the range of learners in their classrooms. These expectations include how to differentiate instruction and build trusting relationships to provide genuine support that the range of learners (e.g., gifted, students with disabilities, learners from low-resourced families, culturally and linguistically diverse students) require in today’s schools. These requirements and expectations are inadequately addressed in TEd and PD scholarship. General education teachers get minimal preparation on how to educate students with disabilities. A complicating factor is that TEd and PD in general and special education operate with disparate conceptions of teaching and learning with little cross-pollinations. These systemic barriers disadvantage general and special education teachers while the expectations for coordinated collaborative work continue to increase (e.g., Response to Intervention and Multi-tiered system of supports models). Research in TEd and PD is urgently needed to address these gaps.

Education Technology

Education technology is the use of digital technologies and related software with the goal to enhance learning. A report commissioned by IES, A Compendium of Education Technology Research Funded by NCER and NCSER: 2002–2014 (Yamaguchi & Hall, 2017), defines the uses of technology as support for student learning, support of teachers and instructional practice, and support of research and school improvement. The compendium recognizes that education technology can support the development of metacognitive and social strategies, support learners with special needs, support collaborative learning, extend learning beyond traditional boundaries of the classroom, connect learners who are geographically dispersed, and expand learning beyond formal environments into informal settings such as museums, cultural institutions, and learners’ homes. Similarly, technology has the potential to transform teacher instruction, teacher professional development, and teacher practice. Additionally, schools and their leaders use technology for a range of administrative tasks, to support data-driven decision making, and help devise strategies to increase equity.

IES competed Education Technology as a separate topic from 2008 to 2020 but not in 2021 or 2022. The RFA for the 2022 competition calls for related research in three of the topics: It lists the “development and testing of interventions designed to support all learners in becoming digitally literate citizens in the 21st century, including those which integrate new forms of technology within social studies programs, such as social media, multiuser virtual environments, virtual and augmented reality, and wearables” (p. 13) under the Civics Education and Social Studies topic; “Exploratory research to guide the development and testing of education technology products that can personalize instruction” (p. 14) in the Cognition and Student Learning topic; and calls for researchers to investigate how “technology be leveraged for more effective reading and writing instruction” (p. 19) under the Literacy topic.

The committee expressed concerns about the decision not to separately compete Education Technology at this historic moment in time because the COVID-19 pandemic has shown the critical importance of education technology to support continuity in formal schooling and informal learning (Schwartz et al., 2020). The pandemic has also revealed deep inequities in access to educational technologies across the country. Where education technology was available, the experience of remote learning forced by the pandemic in 2020 and 2021 has shown the deep limitations of current education technology infrastructure, products, practices, and research (Consortium for School Networking, 2021; Sahni et al., 2021; Education Endowment Foundation, 2020; Gallagher & Cottingham, 2020). As a nation, we now also recognize that inequity, lack of diversity, and lack of

inclusion is not only unjust, but also it prevents us from unlocking the full potential of the next generations. Even though the committee recognized that ultimately, education technology needs to serve the specific topics taught in schools, it has become clearer than ever that more research is needed to guide the design of the next generation of education technology tools, and that this research involves many issues that are broader than the specific topics for which IES provides research support. Recent analyses have estimated, for example, that the education technology market will grow by $112.39 billion from 2020 to 2025 (Technavio, 2021). Among the drivers of this growth are artificial intelligence applications, including machine learning, and game-based learning (PRNewswire, 2021). The use of these technologies in the classroom requires a significant, dedicated investment into rigorous research that informs their design to ensure they serve the needs of the learners.

The committee therefore believes that Education Technology proposals should be invited that investigate these broader topics, and that these proposals should be reviewed by a dedicated Education Technology panel. The committee expressed a sense of urgency for this kind of education technology research, as the recognition of the importance of education technology as a result of remote schooling during the pandemic has already begun to result in the development of many new digital tools for learning, support of teachers, and support of schools, which would benefit from this kind of research.

Further, additional research is warranted to more fully explore the relationships between technology and the broader learning environments in which the technology is used. This plays out in two corresponding ways. First, education technology research must examine the ways in which biases become embedded in the design of technology (Scardamalia & Bereiter, 2008). For example, this requires opening up the “black box” of the algorithms for greater transparency in how user profiles and predictive analytics are used to constrain or expand learning opportunities for students based on prior experiences and characteristics (Rospigliosi, 2021). Second, education technology research should examine how technology is embedded within learning environments, or how technology is designed for real-life contexts, social interactions, and cultural influences. This includes how students, teachers, and families use and augment the technology; the role of the “digital divide” in access to resources, including broadband Internet as well as various technological devices; and the moderating influence of peers on students’ use and engagement with technology (van Dijk, 2020; Zheng et al., 2017; Jeong & Hmelo-Silver, 2016).

The above concerns highlight the importance for education technology research to have a strong grounding in the theoretical mechanisms of learning under investigation, to guard against research and technology that

perpetuate bias and inequity. Theoretical transparency will be essential when building and testing new technologies. The kinds of predictive modeling used to personalize instruction for students often depends on a massive amount of student data, demanding close attention to questions about whether the available data are appropriate for the questions being explored, the conditions under which the data were collected, who and what may be missing from the data, how to balance the information gained from the data with the need to protect privacy, and what additional measures may be worth developing (Schwabish & Feng, 2021; Regan & Jesse, 2019). Given inequitable access to education technology, including variation in how schools deploy technology for students across different achievement levels (Lee et al., 2021), ensuring that such research is not extractive and has relevance across a broad range of populations and contexts takes on even greater importance.

Questions that should be addressed in research on Education Technology supported by NCER and NCSER include, but are not limited to

• Development of new pedagogies and theoretical approaches addressing diversity, equity, and inclusion in education technology;
• Ethically aligned design processes for education technology that benefit from knowledge mobilization and focus on diversity, equity, and inclusion;
• Meaningful integration of responsible, accountable, and transparent analytics in learning environments;
• Approaches to personalization, adaptivity, and adaptability that incorporate diversity, equity, and inclusion; focus on transparency; and go beyond learning progressions and adapting for learners’ current level of knowledge;
• Use of artificial intelligence-based approaches for novel education technology solutions, including personalization, adaptivity, and adaptability;
• Measurement approaches for learning outcomes, as well as learner state and learner trait variables, using longitudinal log data from education technology environments;
• Approaches to reliably measuring accountability/attendance versus engagement versus competency in remote learning, and the relative value of each of these outcomes;
• Designing methods of efficacy and effectiveness research harnessing user logs from widely available education technology environments;
• Development of standards for user logging and policies for data collection, storage, and ownership in education technology environments; and

• Effective strategies for the commercialization of successful research prototypes of education technology solutions.

Additional questions that should be addressed in research on Education Technology supported by NCSER could include strategies for use of assistive technology for simulations, games, virtual reality, mixed reality, augmented reality, and similar advanced technologies.

SPECIAL CONSIDERATIONS FOR NCSER

While the current moment motivates the need for further research on specific topics across IES, the committee encourages IES to give specific consideration to pressing challenges facing the field of special education. What makes the re-examination of NCSER’s topics so urgent? Among all groups of students affected by COVID-19, it is becoming increasingly clear that the consequences have been particularly pronounced among students with disabilities. The lack of access to specialized instruction during remote instruction (GAO, 2020), coupled with the fact that students too often lost out on legally mandated services throughout the pandemic (Morris, 2021), presents the very real threat of a further widening of academic and career outcomes between students with and without disabilities. Additionally, as the United States grapples with the consequences of structural racism throughout its institutions, it cannot be overlooked that disability identification is racially stratified, and a better understanding is needed on how special education interventions and other programs function for different subpopulations of students. Finally, key policy shifts in recent years have established an even stronger warrant for the quality of special education practice. The 2017 Supreme Court case Endrew F. v. Douglas County School District established the responsibility that a school district’s special education services produce “appropriate progress” for a given student’s needs (Kauffman et al., 2021; Lemons et al., 2018). In other words, schools are to be held accountable for ensuring that the instruction they provide results in academic and behavioral gains in line with what is established in a child’s Individualized Education Program. This precedent warrants a close investigation into the totality of services students receive. In the next section, we offer several opportunities for enhancing the knowledge base in special education. These are provided as examples only of possible directions.

Expanding a Focus Beyond Identifying Effective Programs

Identifying the programs that are effective for individuals with disabilities and their families has been an important and necessary focus of IES

through NCSER. In addition to program efficacy/effectiveness, it is critical to better understand the teaching practices and instructional contexts in which students with disabilities are provided opportunities for accessing beneficial educational outcomes, both academically and behaviorally. Most of the teaching that teachers do throughout the day is not derived directly from a “program.” They design, implement, and evaluate teaching by taking in resources (curricula, professional development, texts, materials), filtering these resources through their own knowledge and perceived needs of their students while navigating institutional affordances and constraints (e.g., district curricular policies, instructional reform mandates, school assessment initiatives), and then co-constructing teaching-learning processes and outcomes. With this in mind, it is critically important to support programs of research that document the multifaceted processes and contingencies that surround the complex work of teachers.

For example, much has been learned in the past two decades about how people learn (as described in Chapter 2 of this report), although much of that work has been conducted outside of special education contexts. IES, through NCSER, is ideally suited to support work that further extends the learning science work to individuals with disabilities and special education teachers. For example, outside of special education, scholarship in content-area instruction (e.g., mathematics, science, history and civics) has shifted increasingly toward inquiry-oriented approaches to instruction; how do these practices affect the learning outcomes of individuals with disabilities? To what extent are students with disabilities engaged in activities and provided opportunities to access learning with their general education classmates? Pedagogies that vary between general and special education may have real consequences for students with disabilities, because neither field has provided suitable guidance on how to support this population as they navigate activity-based classroom work. How can teachers scaffold these learning activities to ensure that students are developing foundational skills as well as higher-order skills and concepts?

Understanding How School Contexts and Structures Support Inclusion and Access to Improved Outcomes for Students with Disabilities

Perhaps one of the most persistent themes in education for students with disabilities is the provision of access to the general education classroom—for whom, under what conditions, and the instructional arrangements associated with positive outcomes within these arrangements. In light of the standards for special education established through Endrew F., the field must tackle the question of how educators, collectively, can work to ensure that students make appropriate academic progress. Most educators would agree that inclusion in the general education classroom is a goal for students with

disabilities, but researchers have largely ignored the question of whether specific inclusion policies are associated with improved student outcomes. For example, despite the widespread use of co-teaching (where a general educator and special educator provide instruction in the same classroom) as a service delivery model, there is virtually no causal evidence supporting whether the practice actually leads to improved student outcomes (Jones & Winters, 2020; Solis et al., 2012). NCSER is ideally suited to support research that will better inform educators about inclusive practices and models that yield beneficial outcomes for students and their families.

A related area where expanded research is necessary is in better understanding how other contextual factors outside of classroom teachers can positively impact students with disabilities. These factors include professionals (e.g., school psychologists, physical therapists, speech and language therapists); for example, Mulhern (2020) provided causal evidence that school counselors can affect student attainment at levels approaching typical teacher effects. It will also be important to continue expanding research on the role of families in supporting outcomes among students with disabilities. In addition, research on the mediating effects of organizational factors, the layering of multiple (often contradictory) policies, and sociohistorical legacies (e.g., community and school racial segregation) in the implementation and outcomes of inclusive models is urgently needed.

RECOMMENDATIONS

In this chapter, the committee describes its finding that a series of barriers exists both internal and external to IES that hampers the potential for funding for a set of critically important topics. While the current set of topics does a good job representing the field, these constraints limit the extent to which IES can fund research in areas that are pivotal in efforts toward improving student achievement. Ultimately, reimagining the project types alone (as we have recommended in Chapter 4 of this report) will not address the numerous ways that topics, although technically fundable, are unlikely to get funded in the current topic structure. The committee recognizes that without attention to how the Education Sciences Reform Act (ESRA) is enacted in RFA requirements, as well as the review process, it will be difficult to fund research that looks at interventions targeting teachers or systems in particular. Further, as we describe in Chapter 4, fealty to the methodological rigor associated with experimental design has also limited the use of alternative methods for deep understanding of the context in which interventions work (or do not). Finally, the committee recognizes that there are a set of factors (e.g., teacher knowledge and practice, school and district organizational contexts) that matter for supporting student outcomes; it is essential that these factors are attended to in the design and development of studies.

RECOMMENDATION 5.1:

Existing constraints or priorities in the RFA structure and review process have narrowed the kinds of studies within topics that are proposed and successfully funded. In order to expand the kinds of studies that are proposed and successfully funded in NCER and NCSER, IES should consider the following:

• Allowing use of outcomes beyond the student level (classroom, school, institution, district) as the primary outcome
• Expanding the choice of research designs for addressing research questions that focus on why, how, and for whom interventions work

In advance of these structural changes, however, the committee recognizes that the current moment of racial reckoning and responding to COVID-19 require immediate scholarly attention. Given the issues in education that are emerging at breakneck pace and the subsequent demand for assistance from the field, the committee thinks that designating separate competitions for certain topics is warranted in order to signal their importance even though these topics might technically be “fundable” in existing competitions.

RECOMMENDATION 5.2:

Within each of its existing and future topic area competitions, IES should emphasize the need for research focused on equity.

RECOMMENDATION 5.3:

In order to encourage research in areas that are responsive to current needs and are relatively neglected in the current funding portfolio, NCER and NCSER should add the following topics:

• Civil rights policy and practice
• Teacher education and education workforce development
• Education technology and learning analytics

RECOMMENDATION 5.4:

IES should offer new research competitions under NCSER around these topics:

• Teaching practices associated with improved outcomes for students with disabilities
• Classroom and school contexts and structures that support access and inclusion for improved outcomes for students with disabilities
• Issues specific to low-incidence populations

The topics listed above represent priorities identified by the committee based on our understanding of the current state of education research. This list is not intended to be exhaustive or restrictive; rather, these topics are examples of the types of topics that emerge through consistent, focused engagement with the field. Indeed, the committee recognizes that education research is perennially evolving in response to both the production of knowledge as well as the circumstances in the world. For this reason, the committee advises that the list of topics funded by the centers should also evolve in order to remain responsive to the needs of the field. This responsiveness is a necessary component of fulfilling the obligations laid out in ESRA: In order to “sponsor sustained research that will lead to the accumulation of knowledge and understanding of education,” it is important to fully understand not only what knowledge has accumulated, but also where the existing gaps are.

RECOMMENDATION 5.5:

IES should implement a systematic, periodic, and transparent process for analyzing the state of the field and adding or removing topics as appropriate. These procedures should incorporate

• Mechanisms for engaging with a broad range of stakeholders to identify needs
• Systematic approaches to identifying areas where research is lacking by conducting syntheses of research, creating evidence gap maps, and obtaining input from both practitioners and researchers
• Public-facing and transparent communication about how priority topics are being identified

The committee expects that these recommendations, implemented in concert with one another, will allow NCER and NCSER to support research that meets the immediate needs of the field while simultaneously ensuring that it can nimbly adapt to shifting priorities as they inevitably emerge. In the following chapter, we turn to a discussion of how NCER and NCSER might update its work in the area of methods and measures.

REFERENCES

Brugha, R., and Varvasovszky, Z. (2000). Stakeholder analysis: A review. Health Policy and Planning, 15(3), 239–246. https://doi.org/10.1093/heapol/15.3.239.

Center for Research in Mathematics and Science Education (CRMSE). (2010). Breaking the Cycle: An International Comparison of U.S. Mathematics Teacher Preparation. East Lansing: Michigan State University.

Chetty, R., Friedman, J.N., and Rockoff. J.E. (2014). Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood. American Economic Review, 104(9), 2633–2679.

Clotfelter, C.T., Ladd, H.F., and Vigdor, J.L. (2007). Teacher credentials and student achievement: Longitudinal analysis with student fixed effects. Economics of Education Review, Elsevier, 26(6), 673–682.

Cluver, L. (2021, July 20). COVID has created a scale of family loss not seen since AIDS. Guest essay. New York Times. https://www.nytimes.com/2021/07/20/opinion/covid-19-orphans.html.

Cochran-Smith, M., and Zeichner, K.M. (2009). Studying Teacher Education: The Report of the AERA Panel on Research and Teacher Education. Mahwah, NJ: Lawrence Erlbaum Associates.

Consortium for School Networking (2021). EdTechTrends 2021: Members Share their Experience. https://emma-assets.s3.amazonaws.com/paqab/a7a51dcc643bace61c838465c156ffaa/CoSN_EdTech_Trends_Survey_Report_2021.pdf.

Dalkey, N., and Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458–467. http://www.jstor.org/stable/2627117.

Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of state policy evidence. Education Policy Analysis Archives, 8(1). https://doi.org/10.14507/epaa.v8n1.2000.

Education Endowment Foundation (2020). Remote Learning, Rapid Evidence Assessment. London: Author. https://educationendowmentfoundation.org.uk/public/files/Remote_Learning_Rapid_Evidence_Assessment.pdf.

Espelage, D.L. (2015). Emerging issues in school bullying research & prevention science. In E.T. Emmer and E. Sabornie (Eds.), Handbook of Classroom Management: Research, Practice, and Contemporary Issues (pp. 76–93). New York: Taylor & Francis.

Fryer, R.G. (2017). The production of human capital in developed countries: Evidence from 196 randomized field experiments. In A.V. Banerjee and E. Duflo (Eds.), Handbook of Economic Field Experiments, Volume 2 (pp. 95–322). Amsterdam: North-Holland.

Gallagher, H.A., and Cottingham, B. (2020). Improving the Quality of Distance and Blended Learning. Brief No. 8. EdResearch for Recovery Project. https://annenberg.brown.edu/sites/default/files/EdResearch_for_Recovery_Brief_8.pdf.

Gamoran, A. (2015). The Future of Educational Inequality: What Went Wrong and How Can We Fix It? New York: William T. Grant Foundation. http://wtgrantfoundation.org/resource/the-future-of-educational-inequality-what-went-wrong-and-how-can-we-fix-it.

Gamoran, A., Collares, A.C., and Barfels, S. (2016). Does racial isolation in school lead to long-term disadvantages? Labor market consequences of high school racial composition. American Journal of Sociology, 121, 1116–1167.

Ghaffar, A., Collins, T., Matlin, S.A., and Olifson S. (2009). The 3D Combined Approach Matrix: An Improved Tool for Setting Priorities in Research for Health. Global Forum for Health Research. http://www.bvs.hn/Honduras/PIS/MEC3DEnglish.pdf.

Government Accountability Office (GAO). (2020). Distance Learning: Challenges Providing Services to K–12 English Learners and Students with Disabilities during COVID-19. GAO Publication No. 21-43. Washington, DC: Government Printing Office.

Haddaway, N.R., Kohl, C., da Silva, N.R., Schiemann, J., Spök, A., Stewart, R., Sweet, J.B., and Wilhelm, R. (2017). A framework for stakeholder engagement during systematic reviews and maps in environmental management. Environmental Evidence, 6(1), 1–14.

Hanushek, E.A., and Rivkin, S.G. (2006). Teacher quality. In E.A. Hanushek and F. Welch (Eds.), Handbook of the Economics of Education, Volume 2 (pp. 1052–1078). Amsterdam: North Holland.

Hill, H., Mancenido, Z., and Loeb, S. (2021). Effectiveness Research for Teacher Education (EdWorkingPaper: 20-252). Annenberg Institute at Brown University. https://doi.org/10.26300/zhhb-j781.

Hill, H., Mancenido, Z., and Loeb, S. (2020). New Research for Teacher Education (EdWorkingPaper: 20–252). Annenberg Institute at Brown University. https://doi.org/10.26300/zhhb-j781.

Horsford, S.D. (2011). Learning in a Burning House: Educational Inequality, Ideology, and (Dis)Integration. New York: Teachers College Press.

Imperial College of London. (2021). COVID-19 and orphanhood. Orphanhood estimates: United States of America. https://imperialcollegelondon.github.io/orphanhood_calculator/#/country/United%20States%20of%20America.

Institute of Education Sciences (IES) Request for Applications (RFA). (2021). Education Research Grants Program Request for Applications.

Jeong, H., and Hmelo-Silver, C.E. (2016). Seven affordances of computer-supported collaborative learning: How to support collaborative learning? How can technologies help? Educational Psychologist, 51(2), 247–265. https://doi.org/10.1080/00461520.2016.1158654.

Johnson, O. (2021). Future of Methods and Measures in the Field of Education Research. PowerPoint slides. https://www.nationalacademies.org/event/07-07-2021/docs/D63472D5D54F4445CC0761E6392B9E600401A620466B.

Jones, N., and Winters, M. (2021). Are Two Teachers Better Than One? The Effect of Co-Teaching on Students With and Without Disabilities. 2020 APPAM Fall Research Conference.

Kauffman, J.M., Ahrbeck, B., Anastasiou, D., Badar, J., Felder, M., and Hallenbeck, B.A. (2021). Special education policy prospects: Lessons from social policies past. Ex-ceptionality, 29(1), 16–28. https://doi.org/10.1080/09362835.2020.1727326.

Kelcey, B., Spybrook, J., Phelps, G., Jones, N., and Zhang, J. (2017). Designing large-scale multisite and cluster-randomized studies of professional development. The Journal of Experimental Education, 85(3), 389–410.

Kennedy, M.M. (2019). How we learn about teacher learning. Review of Research in Education, 43(1), 138–162. https://doi.org/10.3102/0091732X19838970.

Klager, C.R., and Tipton, E.L. (2021). Commissioned Paper on the Summary of IES Funded Topics. Paper prepared for the National Academies of Sciences, Engineering, and Medicine, Committee on the Future of Education Research at the Institute of Education Sciences in the U.S. Department of Education. https://nap.nationalacademies.org/resource/26428/READY-KlagerTipton_IES_Topic_Analysis_Jan2022v4.pdf.

Lee, D., Huh, Y., Lin, C.Y., Reigeluth, C.M., and Lee, E. (2021). Differences in personalized learning practice and technology use in high- and low-performing learner-centered schools in the United States. Educational Technology Research and Development, 69(2), 1221–1245. https://doi.org/10.1007/s11423-021-09937-y.

Lemons, C.J., Allor, J.H., Al Otaiba, S., and LeJeune, L.M. (2018). 10 research-based tips for enhancing literacy instruction for students with intellectual disability. TEACHING Exceptional Children, 50(4), 220–232. https://doi.org/10.1177/0040059918758162.

Losen, D.J., Keith, M.A., Hobson, C.L., and Martinez, T.E. (2016). Charter Schools, Civil Rights and School Discipline: A Comprehensive Review. Los Angeles: The Civil Rights Project/Proyecto Derechos Civiles.

McDonald, D., Bammer, G., and Deane, P. (2009). Research Integration Using Dialogue Methods. Canberra: ANU E Press. https://press.anu.edu.au/publications/research-integration-using-dialogue-methods.

Morris, A. (2021, September). Parents of students with disabilities try to make up for lost year. The New York Times. https://www.nytimes.com/2021/09/17/nyregion/special-needs-children-coronavirus-pandemic.html.

Mulhern, C. (2020). Beyond Teachers: Estimating Individual Guidance Counselors’ Effects on Educational Attainment. Working paper. http://papers.cmulhern.com/Counselors_Mulhern.pdf.

Nasser, M., Ueffing, E., Welch, V., and Tugwell, P. (2013). An equity lens can ensure an equity-oriented approach to agenda setting and priority setting of Cochrane Reviews. Journal of Clinical Epidemiology, 66(5), 511–521.

National Academies of Sciences, Engineering, and Medicine (NASEM). (2020). Changing Expectations for the K–12 Teacher Workforce: Policies, Preservice Education, Professional Development, and the Workplace. Washington, DC: The National Academies Press. https://doi.org/10.17226/25603.

National Research Council. (2010). Preparing Teachers: Building Evidence for Sound Policy. Washington, DC: The National Academies Press. https://doi.org/10.17226/12882.

Okonofua, J.A., and Eberhardt, J.L. (2015). Two strikes: Race and the disciplining of young students. Psychological Science, 26(5), 617–624. https://doi.org/10.1177/0956797615570365.

Phelps, G., and Sykes, G. (2020). The practice of licensure, the licensure of practice. Phi Delta Kappan, 101(6), 19–23. https://doi.org/10.1177/0031721720909582.

Phelps, G., Kelcey, B., Jones, N., and Liu, S. (2016). Informing estimates of program effects for studies of mathematics professional development using teacher content knowledge outcomes. Evaluation Review, 40(5), 383–409.

PRNewswire. (2021). Edtech Market to grow by USD 112.39 bn from 2020 to 2025. https://www.prnewswire.com/news-releases/edtech-market-to-grow-by-usd-112-39-bn-from-2020-to-2025evolving-opportunities-with-alphabet-inc--blackboard-inc17000-technavio-reports-301412825.html.

Reardon, S.F., Weathers, E.S., Fahle, E.M., Jang, H., and Kalogrides, D. (2021). Is Separate Still Unequal? New Evidence on School Segregation and Racial Academic Achievement Gaps. CEPA Working Paper. Center for Education Policy Analysis, Stanford University.

Regan, P.M., and Jesse, J. (2019). Ethical challenges of edtech, big data and personalized learning: Twenty-first century student sorting and tracking. Ethics and Information Technology, 21(3), 167–179. doi:10.1007/s10676-018-9492-2.

Rivkin, S.G., Hanushek, E.A., and Kain, J.F. (2005). Teachers, schools, and academic achievement. Econometrica, 73, 417–458. https://doi.org/10.1111/j.1468-0262.2005.00584.x.

Ronfeldt, M. (2021). Links among teacher preparation, retention, and teaching effectiveness. National Academy of Education Committee on Evaluating and Improving Teacher Preparation Programs. National Academy of Education.

Ronfeldt, M., Brockman, S.L., and Campbell, S.L. (2018). Does cooperating teachers’ instructional effectiveness improve preservice teachers’ future performance? Educational Researcher, 47(7), 405–418. https://doi.org/10.3102/0013189X18782906.

Rospigliosi, P.A. (2021). The risk of algorithmic injustice for interactive learning environments. Interactive Learning Environments, 29(4), 523–526. https://doi.org/10.1080/1049482.2021.1940485.

Rudan, I., Boschi-Pinto, C., Biloglav, Z., Mulholland, K., and Campbell, H. (2008). Epidemiology and etiology of childhood pneumonia. Bulletin of the World Health Organization, 86(5), 408–416. https://doi.org/10.2471/blt.07.048769.

Sahni, S.D., Polanin, J.R., Zhang, Q., Michaelson, L.E., Caverly, S., Polese, M.L., and Yang, J. (2021). A What Works Clearinghouse Rapid Evidence Review of Distance Learning Programs (WWC 2021-005REV). Washington, DC: U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance, What Works Clearinghouse. https://ies.ed.gov/ncee/wwc/Docs/ReferenceResources/Distance_Learning_RER_508c.pdf.

Scardamalia, M., and Bereiter, C. (2008). Pedagogical biases in educational technologies. Educational Technology, 48(3), 3–11. http://www.jstor.org/stable/44429572.

Schneider, M. (2021). A year for reflection and continued transformation. IES Director’s Blog. https://ies.ed.gov/director/remarks/1-12-2021.asp.

Schwabish, J., and Feng, A. (2021). Do No Harm Guide: Applying Equity Awareness in Data Visualization. Washington DC: Urban Institute. http://hdl.handle.net/20.500.11941/3898.

Schwartz, H.L., Grant, D., Diliberti, M.K., Hunter, G.P., and Setodji, C.M. (2020). Remote Learning Is Here to Stay: Results from the First American School District Panel Survey. Research Report. RR-A956-1. RAND Corporation. https://www.rand.org/pubs/research_reports/RRA956-1.html.

Scott, J., Siegel-Hawley, G., DeBray, E., Frankenberg, E., and McDermott, K. (2020). An Agenda for Restoring Civil Rights in K–12 Federal Education Policy. Boulder, CO: National Education Policy Center. https://nepc.colorado.edu/publication/restoring-civil-rights.

Scott, J., Moses, M.S., Finnigan, K.S., Trujillo, T., and Jackson, D.D. (2017). Law and Order in School and Society: How Discipline and Policing Policies Harm Students of Color, and What We Can Do About It. Boulder, CO: National Education Policy Center. http://nepc.colorado.edu/publication/law-and-order.

Shedd, C. (2016). Unequal City: Race Schools and Perceptions of Injustice. New York: Russell Sage Foundation.

Sibbald, S.L., Singer, P.A., Upshur, R., and Martin, D.K. (2009). Priority setting: What constitutes success? A conceptual framework for successful priority setting. BMC Health Services Research, 9(1), 1–12.

Sindelar, P., Brownell, M.T., and Billingsley, B. (2010). Special education teacher education research: Current status and future directions. Teacher Education and Special Education, 33, 8–24.

Skiba, R.J., Horner, R.H., Chung, C.-G., Rausch, M.K., May, S.L., and Tobin, T. (2011). Race is not neutral: A national investigation of African American and Latino disproportionality in school discipline. School Psychology Review, 40(1), 85–107.

Solis, M., Vaughn, S., Swanson, E., and McCulley, L. (2012). Collaborative models of instruction: The empirical foundations of inclusion and co‐teaching. Psychology in the Schools, 49(5), 498–510.

Technavio. (2021). Edtech Market by Sector and Geography—Forecast and Analysis 2021–2025. Technavio.com.

Theobald, R., Goldhaber, D., Holden, K., and Stein, M. (2021). Special Education Teacher Preparation, Literacy Instructional Alignment, and Reading Achievement for Students with High-Incidence Disabilities. CALDER Working Paper No. 253-0621.

Treglia, D., Cutuli, J.J., Arasteh, K., Bridgeland, J.M., Edson, G., Phillips, S., and Balakrishna, A. (2021). Hidden Pain: Children Who Lost a Parent or Caregiver to COVID-19 and What the Nation Can Do to Help Them. COVID Collaborative.

Van Dijk, J. (2020). The Digital Divide. Cambridge, UK: Polity Press.

Viergever, R.F., Olifson, S., Ghaffar, A., and Terry, R.F. (2010). A checklist for health research priority setting: Nine common themes of good practice. Health Research Policy and Systems, 8(1), 1–9.

Waitoller, F., and Artiles, A.J. (2013). A decade of professional development research for inclusive education: A literature review and notes for a sociocultural research program. Review of Educational Research, 83, 319–356.

Wilson, S.M., Floden, R.E., and Ferrini-Mundy, J. (2001). Teacher Preparation Research: Current Knowledge, Gaps, and Recommendations. Research report prepared for the U.S. Department of Education and the Office for Educational Research and Improvement, No. R-01-3. University of Washington Center for the Study of Teaching and Policy.

World Health Organization. (2020). A Systematic Approach for Undertaking a Research Priority-Setting Exercise: Guidance for WHO Staff. Geneva: Author. https://apps.who.int/iris/handle/10665/334408.

Yamaguchi, R., and Hall, A. (2017). A Compendium of Education Technology Research Funded by NCER and NCSER: 2002–2014. https://ies.ed.gov/ncer/pubs/20170001/.

Zheng, N.-N., Liu, Z.Y., Ren, P.J., Ma, Y.Q., Chen, S.T., Yu, S.Y., Xue, J.R., Chen, B.D., and Wang, F.Y. (2017). Hybrid-augmented intelligence: Collaboration and cognition. Frontiers of Information Technology & Electronic Engineering, 18(2), 153–179. https://doi.org/10.1631/FITEE.1700053.

This page intentionally left blank.