Evaluation Design for Complex Global Initiatives Workshop Summary (2014) / Chapter Skim
Currently Skimming:
9 Evolving Methods in Evaluation Science
9 Evolving Methods in Evaluation Science
Pages 79-86
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 79... ...
• Nonexperimental, observational, and mixed methods can pro vide valuable evaluation information. Many technologies, techniques, and approaches are available in evaluation science.
|
From page 80... ...
As an example, he cited his experience with large smoking cessation programs, which deal with a wide range of smokers who have a variety of motivations to stop smoking, strategies for kicking the habit, and responses both to particular interventions and to when they fail any one attempt to quit smoking. In addition, those delivering the intervention have different personal approaches, further complicating the context of the intervention.
|
From page 81... ...
As a result, Wong said, controlling for context may be neither possible nor desirable because doing so "may in fact be stripping out the thing that is an important trigger to the mechanism." Wong finished his remarks by noting that it helps to have some grounding in the philosophy of science to understand the basis of realist methods and how best to apply them. The realist synthesis has quality reporting standards and training materials available on the web, and there is a discussion group for anyone interested in realist research, he said.1 INNOVATIVE DESIGNS FOR COMPLEX QUESTIONS Emmanuela Gakidou, professor of global health and director of education and training at the Institute for Health Metrics and Evaluation at the University of Washington, discussed three complex interventions that she and her colleagues are evaluating using innovative designs.
|
From page 82... ...
The evaluation is also designed to answer several more specific questions about GAVI's contribution to immunization rates and, ultimately, to reductions in child mortality. The GAVI evaluation uses a mixed methods approach to analyze quantitative and qualitative data from sources, including process evaluation, resource tracking, facility surveys, household surveys, verbal autopsies, FIGURE 9-1 The GAVI Full Country Evaluation sought to understand and quantify the barriers to and drivers of immunization program improvement through routine program monitoring, targeted studies, and full country evaluations, as presented by Gakidou.
|
From page 83... ...
Finally, Gakidou discussed the evaluation of the Malaria Control Policy Assessment, funded by the Bill & Melinda Gates Foundation, which is designed to determine how much of the reduction in child mortality seen in Zambia over the past two decades is a result of scaled-up malaria control interventions. She explained that in addition to malaria control efforts -- distribution of insecticide-treated bed nets and indoor residual spraying -- there has been an expansion of efforts to prevent mother-to-child transmission of HIV through PEPFAR, a transformation and scale-up of immunization programs combined with the development of a new pentavalent vaccine, the introduction of child health weeks, and other health interventions new to Zambia since the mid-1990s.
|
From page 84... ...
As an example, he discussed the Young Lives survey, a longitudinal study of 12,000 children born in two cohorts in Ethiopia, India, Peru, and Vietnam that was designed originally to look at childhood poverty.2 In 2010, when the younger cohort of children reached age 10, Rolleston and his colleagues started including school surveys focused on math and literacy and measuring progress in learning over time. They also looked at school and teacher effectiveness using a longitudinal, within-school design, and included both the indexed children and class peers to get a balanced sample of children at the school and class level.
|
From page 85... ...
While there are inconsistent patterns of explanatory variables, two factors stood out: all teachers in Vietnam received formal teacher training, while 16.5 percent of teachers in India did not, and nearly a third of students in India reported that their teacher was often absent, while teacher absenteeism in Vietnam was exceedingly rare. "But assessing school quality in comparative terms between two systems is quite complex, because what you really need is to be able to measure the value added over time," said Rolleston, "To do that you need to be able to separate the effects of pupils' backgrounds from their prior attainment, which requires a longitudinal design, repeated measures of test scores at the school level, and linked data between teachers, schools, and pupil backgrounds." While difficult to achieve, Rolleston and his team successfully used these sophisticated designs for value-added analysis where they have found big differences in cross-sectional effects compared to longitudinal effects.
|
From page 86... ...
To achieve equity, Vietnam has an emphasis on fundamental or minimum school quality and that, he added, "means that the least advantaged pupils and the most disadvantaged areas do not suffer from as extreme a disadvantage as they do in the other countries in our study." Other important factors for Vietnam's higher levels of student performance are a greater degree of standardization in terms of curricula and textbooks that are more closely matched to pupils' learning levels and abilities, a commitment to mastery by all pupils, and the use of regular assessments. In closing, Rolleston said that, in education, data on learning metrics in developing countries are still inadequate, that there are few rigorous assessments of students' learning performance, and that a robust longitudinal study design is needed to assess school quality.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.