Improving Democracy Assistance Building Knowledge Through Evaluations and Research (2008) / Chapter Skim
Currently Skimming:
Summary
Summary
Pages 1-16
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 1... ...
Despite these substantial expenditures, our understanding of the actual impacts of USAID DG assistance on progress toward democracy remains limited -- and is the subject of much current debate in the policy and scholarly communities. Admittedly, the realities of democracy programming are complicated, given the emphasis on timely responses in politically sensitive environments and flexibility in implementation to account for fluid political circumstances.
|
From page 2... ...
Yet USAID's current evaluation practices do not provide compelling evidence of the impacts of DG programs. While gathering valuable information for project tracking and management, these evaluations usually do not collect data that are critical to making the most accurate and credible determination of project impacts -- such as obtaining baseline measures of targeted outcomes before a project is begun or tracking changes in appropriately selected (or assigned)
|
From page 3... ...
The pilot evaluations should not supplant current evaluations and assessments, but impact evaluations could gradually become a more important part of USAID's portfolio of monitoring and evaluation (M&E) activities as the agency gains experience with such evaluations and determines their value.
|
From page 4... ...
Whether USAID chooses to support such studies or gather them from ongoing academic research, it is important to look at how democracy assistance functions in a range of different initial conditions and trajectories of political change. Such case studies should seek to map out long-term trajectories of political change and to place democracy assistance in the context of national and international factors affecting those trajectories, rather than focus mainly on specific democracy assistance programs.
|
From page 5... ...
What Are Impact Evaluations? Most current evaluations of USAID DG projects, while informative and serving varied purposes for project managers, lack the designs or data needed to provide compelling evidence of whether those projects had their intended effects.
|
From page 6... ...
Current Approaches to Evaluation in USAID The committee's review of current approaches to the evaluation of development assistance in general, and USAID DG programs in particular, found that: • Very few of the evaluations undertaken by international or multilateral development and democracy donors are designed as impact evaluations. There are signs that this is changing as some donors and international agencies are beginning to implement new approaches to evaluation.
|
From page 7... ...
Among both scholars and policy professionals, skeptics worry that the designs for impact evaluations will prove too cumbersome or inflexible to work in fluid and politically sensitive conditions in the field; that such evaluations will be too costly or timeconsuming; or that such studies, in particular randomized designs, are either unethical for or ill suited to the actual projects being carried out in DG programs. Feasibility of Impact Evaluations for DG Projects Recognizing the need to take such concerns seriously, the committee examined a wide range of impact evaluation designs and worked with DG officers at several missions to assess the feasibility of such designs for their current or planned activities.
|
From page 8... ...
Therefore the full committee agreed that the value of such impact evaluations will have to be demonstrated in USAID's own experience. Strategies for Implementation • The committee unanimously recommends that USAID move cautiously but deliberately to implement pilot impact evaluations of several carefully selected projects, including a portion with randomized designs, and expand the use of such impact evaluations as warranted by the results of those pilot evaluations and the needs expressed by USAID mission directors.
|
From page 9... ...
of countries where USAID operates. • While the United States, other donor governments, and international agencies that are making policy in the areas of health or economic assistance are able to draw on databases that are compiled and updated at substantial cost by government or multilateral agencies mandated to collect such data, no comparable source of data on democracy at either the macro or sectoral level currently exists.
|
From page 10... ...
If they wish to have access to data on democracy and democratization comparable to the data relied on by policymakers and foreign assistance agencies in the areas of public health or trade and finance, a substantial government or multilateral effort to improve, develop, and maintain international data on levels and detailed aspects of democracy would be needed. This should not only involve multiple agencies and actors in efforts to initially develop a widely accepted set of sectoral data on democracy and democratic development but should also seek to institutionalize the collection and updating of democracy data for a broad clientele, along the lines of the economic, demographic, and trade data collected by the World Bank, the United Nations, and the International Monetary Fund.
|
From page 11... ...
the cases should include countries with varied outcomes regarding democratic progress or stabilization. • In addition to case studies, a variety of other research methods, both formal and informal (including debriefings of USAID field officers, statistical analyses of international data, and surveys)
|
From page 12... ...
While it will take some time for USAID to learn from undertaking the pilot impact evaluations, it will gain immediately from augmenting its overall learning activities and increasing opportunities for DG staff to actively engage with current research and studies on democratization. Though some committee members believe that the impact evaluations will be more novel and instructive than most current case study and policy reports on democratization, several committee members wish to emphasize the considerable value to policymakers and DG officers of the many books, articles, and reports that have been prepared in recent years by academics, think tanks, and practitioners.
|
From page 13... ...
A DG Evaluation Initiative In support of Recommendations 1 and 4, the committee recommends that USAID develop a five-year DG evaluation initiative, led by a senior USAID official and with special funding, for the following: 1. Undertaking Pilot Impact Evaluations The committee strongly recommends that to accelerate the building of a solid core of knowledge regarding project effectiveness, the DG evaluation initiative should immediately develop and undertake a number of well-designed impact evaluations that test the efficacy of key project models or core development hypotheses that guide USAID DG assistance. A portion of these evaluations should use randomized designs, as these are the most accurate and credible means of ascertaining program impact.
|
From page 14... ...
Pressures to get projects under way, as well as many current contracting practices, thus work against implementing and sustaining impact evaluation designs. One task of the DG evaluation initiative should be to address these issues and explore how to ease the task of undertaking impact evaluations within USAID's contracting and program procedures.
|
From page 15... ...
Videoconferencing and other advanced technologies can be an important supplement, but personal contact and discussion would be extremely important to sharing experiences of success and failure as the evaluation initiative went forward. This includes lessons about both the effectiveness of DG projects and successes and failures in implementing impact evaluations.
|
From page 16... ...
gaining experience with varied impact evaluation designs, including randomized studies, to ascertain how useful they could be for determining the effects of DG projects; (2) focusing on disaggregated, sectoral-level measures to track democratic change; (3)
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.