Skip to main content

Evaluation of PEPFAR (2013) / Chapter Skim
Currently Skimming:

2 Evaluation Scope and Approach
Pages 39-62

The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.


From page 39...
... Leadership Against HIV/AIDS, Tuberculosis, and Malaria Act of 20031 and subsequently reauthorized in the Tom Lantos and Henry J Hyde United States Global Leadership Against HIV/AIDS, Tuberculosis, and Malaria Reauthorization Act of 2008 (hereinafter, the Lantos-Hyde Act of 2008)
From page 40...
... In developing the plan, the planning committee engaged in deliberations through three in-person meetings, two committee teleconferences, and telephonic and electronic communications as needed among working groups composed of subsets of the committee. To inform these deliberations, the planning committee held public sessions to solicit input and gather information from a broad range of stakeholders involved in and affected by PEPFAR.
From page 41...
... Through this information gathering and deliberation, the planning committee developed a conceptual framework for the evaluation that was based on both the committee's expertise and current standards in evaluation methodologies for large-scale programs. The planning phase culminated with the publication of a report describing this conceptual framework and the proposed strategic approach to the evaluation, taking into consideration the requirements for the congressional mandate (IOM and NRC, 2010)
From page 42...
... In the Strategic Approach, the planning committee identified three additional content areas that were not explicitly identified in the legislation but were determined to be critical elements underlying the assessment of the specific content areas requested by Congress. First, the committee deemed it important to review PEPFAR funding in order to determine the level of PEPFAR's investment and to gain insight into how financial support for programs and activities has been determined and distributed over time.
From page 43...
... However, in order to represent the greatest intensity of PEPFAR's investment, the scope of this evaluation was defined to focus on the 31 partner countries submitting an annual Country Operational Plan (COP) at the time of the initiation of the planning phase for this evaluation in 2009.6 In FY 2011, these 31 countries represented 96 percent of PEPFAR's planned funding (OGAC, 2011)
From page 44...
... Data requested from OGAC, implementing agencies, and implementing partners were received as late as October 2012. Recent developments in PEPFAR that were introduced since the main data collection period could not feasibly be assessed by the committee, although key recent developments are noted for context in Chapter 1 and in relevant content areas of subsequent chapters of this report.
From page 45...
... ; mapping of methods and data sources, including key indicators, to the mandated evaluation tasks; developing procedures for data requests; initiating data requests; designing and initiating data quality review methods for data collected directly or received from outside sources; preparing background materials; and continued relationship building with relevant stakeholders such as contacts in PEPFAR countries and at implementing partner organizations. In addition, a major focus of the operational planning phase was to develop and refine processes, frameworks, methods, and instruments for qualitative data collection.
From page 46...
... . Among the inputs to this program impact pathway is the evolving evidence base.
From page 47...
... (distal effects) Strong Health Capacity Strategies Building Systems Individual and Population People Health Impact Trained Service on: Guidance Delivery: Prevention Goods Integration Purchased/ Pyschosocial Coverage Well-Being Planning & Distributed Quality Modeling Cost Efficiency Treatment Services HIV Incidence Access Provided Evolving Evidence Base HIV Beneficiaries Prevalence Served Changes in: Care and Funding & Support Resources Laws and Knowledge Morbidity Policies Attitudes Implemented Behaviors Technical Norms Mortality Addressing Assistance Gender Process & Implementation Results FIGURE 2-1 Program impact pathway for evaluation of PEPFAR's effects on HIV-related health impact for children and adults.
From page 48...
... Although it was a major challenge to directly assess health impact, the evaluation committee was able to use the framework of the program impact pathway to state credible findings about the effects of PEPFAR-supported programs. Assessment of Contribution Although it provides a critical guide for the evaluation, the program impact pathway is of course a simplified view of the implementation of PEPFAR-supported programs.
From page 49...
... EVALUATION SCOPE AND APPROACH 49 Other Donors HARMONIZATION PEPFAR Multilaterals "Three Ones" USG Bilaterals Others Partner Countries • Strategies and policies • Public spending CONTEXTUAL BACKGROUND FACTORS: CULTURE, EPIDEMIOLOGY, POLITICS CONTEXTUAL BACKGROUND FACTORS: CULTURE, EPIDEMIOLOGY, POLITICS • Regulation Total Inputs Private Sector Public Sector Health Other Public Sectors System • NGOs/Nonprofits • Institutional capacity • Child welfare • Firms/For-profits (including M&E) • Primary, Secondary, • Academia • Public goods and Higher Education • Human workforce • Public goods • Financing • Food security quality, cost, access quality, cost, access quality, cost, access AIDS prevention, Other health treatment, and care services Communities Cultural and Social Norms Households and Families Rural and Urban Gender Settings Individuals Income At-Risk Behavior Risk Assets Populations HEALTH IMPACT FIGURE 2-2 Context for PEPFAR contribution in partner countries.
From page 50...
... The evaluation utilized a mix of methods and data sources, including the mapping of investment using financial data, assessing trends over time using program monitoring indicators and clinical data, benchmarking of progress against stated programmatic targets and goals, document reviewing, and analyzing of primary data collected through site visits and semi-structured interviews. Interview Data As the largest component of the data-gathering effort for the evaluation, committee members, IOM staff, and consultants conducted primary data collection through semi-structured interviews.
From page 51...
... The initial selection in advance of each country visit process was based on systematic information gathering from country background research completed by the IOM staff team; input from the PEPFAR mission team and other country stakeholders; and input from committee members. Once in-country, the country visit teams also employed a process of additional snowball sampling by querying scheduled interviewees to identify individuals or organizations who could provide additional information in particular content areas or additional stakeholder perspectives; country visit schedules were structured to allow time for additional interviews to be scheduled to enrich the data collection sample.
From page 52...
... As with the country visit interviews, non-country visit interviewees were selected through purposeful sampling, prioritized based on targeted focus areas within the evaluation and on the process of mapping data sources for evaluation questions. Interviews were conducted using the same methodology as for country visit interviews, utilizing semi-structured interview guides with questions and prompts appropriate to the interviewee(s)
From page 53...
... Financial data were received from OGAC, extracted from publicly available PEPFAR documents, and gathered from other external sources including the Organisation for Economic Co-operation and Development and the Center for Global Development. PEPFAR's program monitoring indicators were received from OGAC, and additional clinical data representing programs implemented by Track 1.0 partners were received from the CDC.
From page 54...
... Interview Data • National prevalence and • Country visit interviews & site visits • USG HQ interviews coverage data • Exit syntheses • Global stakeholders Data Collection Financial Data Financial Data • Planned/approved funding to prime Financial Data • Global donor data partners (n=13) • Obligations and outlays • Planned/approved funding • Planned/approved funding Program Monitoring/Clinical Data Program Monitoring Data • Track 1.0 Partner data (n=13)
From page 55...
... A main reason for the lack of an appropriate comparison is that PEPFAR is widely implemented across many partner countries, which were not selected at random but rather for specific strategic reasons. In addition, an ideal evaluation would use a prospective design, in which data for both intervention and comparison groups would be collected from the beginning of the evaluation.
From page 56...
... Most evaluation questions required the evaluation committee to draw on data that went beyond the indicators that are routinely reported to OGAC. Data from PEPFAR beyond the centrally-reported indicators, such as recommended indicators collected by country programs but not reported to OGAC, data collected independently by the major USG implementing agencies and other implementation partners, financial data by type of partner and expenditures by program activity, results of PEPFAR-supported evaluations, and publications from PEPFAR-supported programs are not managed through processes that allow for ready cataloguing or ready access to what is available.
From page 57...
... with capacity to perform clinical laboratory tests Prevention Number of HIV-positive PEPFAR Output pregnant women who received antiretrovirals to reduce risk of mother-to-child-transmission Percent of HIV-positive National Outcome pregnant women who received antiretrovirals to reduce the risk of mother-to-child-transmission Number of individuals who PEPFAR Output received testing and counseling services for HIV and received their test results Treatment Number of adults and children PEPFAR Output with advanced HIV infection newly enrolled on ART Percent of adults and children with PEPFAR Output advanced HIV infection receiving antiretroviral therapy Percent of adults and children with National Outcome advanced HIV infection receiving antiretroviral therapy Percent of adults and children PEPFAR Outcome known to be alive and on treatment 12 months after initiation of antiretroviral therapy NOTE: Indicator level classified according to 2009 Next Generation Indicator Guidance. ART = antiretroviral therapy; NGIs = Next Generation Indicators; TB = tuberculosis.
From page 58...
... There were, therefore, challenges of interpretation due to heterogeneous data sources with different sampling frames and different data collection systems and criteria, as well as the potential for reporting bias in the responsiveness to data requests from the committee. Chapter 11 presents a discussion of the collection and use of data and information to assess and improve PEPFAR programs and activities, including a forward-looking framework for knowledge management and suggestions for how to develop the means to answer questions posed in the mandate for this evaluation if they are found to be important for future ongoing evaluation.
From page 59...
... . "These changes reflect a progressive transition to a new era of challenges and goals for the program, which include efforts to improve sustainability of the response over time, to enhance coordination with partner governments and other global funding partners, and to support accountable ownership of HIV program delivery by countries themselves.
From page 60...
... 2005b. The President's Emergency Plan for AIDS Relief: Indicators, reporting require ments, and guidelines for focus countries.
From page 61...
... 2009c. The President's Emergency Plan for AIDS Relief: Five-year strategy.


This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More information on Chapter Skim is available.