Skip to main content

Currently Skimming:

4 Project Types for NCER/NCSER Grants
Pages 61-100

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 61...
... For 1 Note that in FY2020, 1–3 and 5 have been funded under the Education Research grants competition, whereas 4 is funded under a separate RFA. 2 Whereas grants submitted to the main research funding competitions of NCER and NC SER enter with a specific project type, applications for Research Networks and Research & Development Centers typically encompass multiple project types.
From page 62...
... This is an explanatory, proof-of-concept study, focused on establishing that the intervention can produce an effect under "ideal" conditions (i.e., when 3 While Exploration studies and Development and Innovation studies have remained roughly the same over time, project types (3)
From page 63...
... At the same time, these changes to the fourth project type led to changes to the third. Initially these were referred to as "Efficacy" studies -- which could include replications of previous efficacy studies.
From page 64...
... NOTE: Total grants includes all grant types, including Research Networks and R&D Centers. Project grants include those awarded in specific goals or project types.
From page 65...
... Given the nature of exploratory work, it might be expected that a smaller percentage of these types of studies would progress. In comparison, 30 percent of Development and Innovation grants are later connected to other grants, including 20 percent associated with Efficacy grants and 4 percent with 5 There are no public data available that clearly identify progressions across project types by intervention.
From page 66...
... However, if the originating study resulted in later studies of different project types, it is counted in both; thus the rows do not sum to the "total." The bolded diagonal numbers indicate multiple grants of the same type related to one another. Those above the bolded diagonal numbers indicate goals that progressed forward (e.g., D&I to Efficacy)
From page 67...
... project types. For example, relative to those at research firms, university researchers are far more likely to be involved in Development and Innovation studies than Efficacy studies (Klager & Tipton, 2021)
From page 68...
... The 2022 RFA for Education Research Grants, for example, recommends that researchers proposing Efficacy and Replication studies "describe the setting and implementation conditions," assess "fidelity of implementation and comparison group practice," collect implementation outcomes, and describe a plan for examining fidelity of implementation. Similarly, the RFA requires proposals to describe their sample and recommends that they define a target population.
From page 69...
... " and notes that when these are included, power analyses for related hypothesis tests should be included. Regarding the concerns about connecting points between different project types, however, there has been considerably less work.
From page 70...
... For example, simple online searches of interventions studied via IES grants often result in project or intervention websites, but without any clear indicator of how often the program is adopted. Research on knowledge mobilization suggests that only 17 percent of school and district leaders report accessing research from the WWC "often" (13%)
From page 71...
... Or it may be that even if marketed and available, it is not packaged and supported in the same way that other curricula and programs are. This suggests that there may be a need for approaches that bring together and incentivize partnerships between researchers, communities, education technology companies, publishers, and nonprofits that focus on selling curricula and professional development to schools.
From page 72...
... . While the challenges are clear, much remains to be learned about robust strategies for ensuring that findings from education research reverberate in the decisions of educational leaders and practitioners (Conaway, 2021)
From page 73...
... . Drawing on both the five themes introduced in Chapter 1, and on the previous analysis of how and why interventions may or may not be progressing through the existing project types, the committee identified a set of framing principles for education research that inform the revised project structure.
From page 74...
... Decision makers are inundated with potential interventions and professional development services, in addition to frequently adapting and creating their own, and would benefit from guidance on how to efficiently surface and weigh evidence to compare different options. Altogether, this speaks to a need to better understand knowledge mobilization, including how schools and decision makers identify problems and develop solutions; which interventions, curricula, and programs are currently used in schools; how to get promising evidence into their hands; how educational leaders harness that evidence to guide action; and what conditions support educational leaders to use research more centrally and substantively in their decision making.
From page 75...
... In more practical terms, this means revising the underlying project structure. Like the current structure, we envision four project types, each of which can be crossed with a topic area.
From page 76...
... In examining the abstracts of these reviews, nearly all of them focus on synthesizing the results of randomized trials and high-quality quasi-experiments. Yet in the current structure, while they summarize a broad base of causal research, the research itself is considered "exploratory." New: If, as a field, we are to develop and refine interventions that can successfully improve educational outcomes for students, then it is imperative that these interventions consider the diversity of the educational TABLE 4-4  Current Exploration Study Types SOURCE: Klager & Tipton, 2021 [Commissioned Paper]
From page 77...
... To highlight this anchoring in educational context, we call this new project type Discovery and Needs Assessment. These studies would begin in authentic learning environments, with a focus on observing, measuring, and understanding the varieties of practices and processes on the ground and determining gaps between "what is" and "what could be." By emphasizing the need for situating work in authentic school environments, we are also highlighting the need for a broad range of descriptive work that involves primary data collection.
From page 78...
... This means that the intervention is developed in an optimal condition: that is, one where implementation is highly monitored so that high fidelity is achieved. In the short run, this focus on optimal implementation is ideal, as it allows for 8 The National Network of Education Research–Practice Partnerships now includes 57 members across the United States; see https://nnerpp.rice.edu/.
From page 79...
... Development studies are encouraged to identify a target population that is "narrow," and in practice -- given the smaller resources found in these grants relative to Efficacy studies -- this can mean a population that is local to the researcher and somewhat homogenous. Again, in the short run, this may be optimal, as reduced variation can improve statistical precision.
From page 80...
... Certainly, this increased variation will make it more difficult to estimate a statistically significant average treatment effect in a pilot study. But many scholars in both medicine and education research have already argued that the focus of pilot studies should not be on estimation of the average treatment effect or on testing null hypotheses (e.g., see Westlund & Stuart, 2017)
From page 81...
... , thus not necessarily addressing the heterogeneity found in business-as-usual conditions across education contexts. This prioritization of internal validity can also be seen in the predominance of randomized trials in both efficacy and effectiveness studies.9 The fact that randomization to treatment provides clear identification of a cause-and-effect relationship is not to be disputed.
From page 82...
... Furthermore, a single project type for causal questions, which includes both efficacy and effectiveness studies and addresses heterogeneity as well as the average impact, will elevate matters of external validity to be considered on par with the matters of internal validity. The inclusion of quasi-experiments in this category is of particular importance, as some interventions may simply not be able to be studied using randomized trials.
From page 83...
... By elevating these methodologies, the focus becomes clearly on determining the best evidence for the interventions that schools need, instead of on finding interventions that fit the best methods of evidence. Within randomized trials, combining efficacy and effectiveness studies into a single project type also removes what can be arbitrary distinctions between the two.
From page 84...
... in advance, to develop what might be called a "prospective meta-analysis" and to argue clearly for how these studies in combination will answer the questions posed. Taken together, this combination of experimental and quasi-experimental, efficacy and effectiveness studies into a single project type means that decisions regarding the methods, scale, and purpose of the study would need to be aligned clearly with the intervention proposed, the population in need, and the state of knowledge in the field.
From page 85...
... New: We propose a new project type focused on Knowledge Mobilization. We propose the term "knowledge mobilization" rather than "knowledge utilization" or "research evidence use" because we incorporate into this project type the organization and synthesis of bodies of evidence as well as improvement of the use of research evidence in real-world settings.
From page 86...
... We ultimately decided that by positioning Knowledge Mobilization as a project type, we emphasize that the entire field of education research needs to develop and study the success of these strategies to integrate research with practice. Establishing Knowledge Mobilization as a project type that cuts across multiple topics, rather than as a standalone topic, also recognizes that due to heterogeneity in populations, interventions, implementation, adaptation, and contexts, successful mobilization strategies likely differ by topic.
From page 87...
... Some studies of this type might focus on the broader research enterprise, including whose knowledge is valued, who gets to decide on the implications of knowledge, and who is benefited or harmed by the production or use of that knowledge. Another critical set of questions could explore circumstances leading to inequitable or harmful consequences of knowledge mobilization, particularly research which devalues lived experience or perpetuates deficit narratives (Chicago Beyond, 2019; Doucet, 2019; Kirkland, 2019; Tuck & Yang, 2014)
From page 88...
... Specific knowledge mobilization strategies may differ in the way they affect the production as well as the use of evidence. For example, community-engaged scholarship may strengthen the relevance and rigor of the research produced, through refining the questions and methods to better fit the local context (Balazs & Morello-Frosch, 2013)
From page 89...
... . A project type for Knowledge Mobilization would be newly responsive to these elements of the IES mandate.
From page 90...
... . Each of the other three project types interface both with one another and with this central engine.
From page 91...
... Positioning knowledge mobilization in the center of the engine creates greater opportunities for sharing and applying knowledge across all stages of the research enterprise. Embedding these revised expectations within IES's RFAs would direct the education research community to prioritize these needs in how they conceptualize and conduct research.
From page 92...
... 2. Build on assets within populations and Develop interventions in a wide range Design, develop, and iterate on potential Development communities of interest to develop of purposively selected contexts, so as interventions and implementation strategies and strategies aligned to local context.
From page 93...
... Investigate existing pathways of contribute to sustainment, spread, and scale of knowledge mobilization. successful implementation.
From page 94...
... Methods Develop standards and methods for Develop methods for understanding Explore designs that allow for identifying and QEDs that can be useful when studying treatment effect heterogeneity, testing implementation strategies, including (Stand Alone structural interventions. moderators of effects, and local SMART, single case, factorial, hybrid, steppedPanel)
From page 95...
... Discussion paper prepared for the National Board for Education Sciences. Washington, DC: Institute of Education Sciences.
From page 96...
... . Education Research Grants Program Request for Applications.
From page 97...
... Paper prepared for the National Academies of Sciences, Engineering, and Medicine, Committee on the Future of Education Research at the Institute of Education Sciences in the U.S. Department of Education.
From page 98...
... . Lessons from a school district–university research partnership: The Houston Education Research Consortium.
From page 99...
... . How much do the effects of education and training programs vary across sites? Evidence from past multisite randomized trials.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.