Evaluation of PEPFAR’s
Contribution (2012–2017)
to
Committee on the Evaluation of Strengthening Human Resources for
Health Capacity in the Republic of Rwanda Under the President’s
Emergency Plan for AIDS Relief (PEPFAR)
Board on Global Health
Health and Medicine Division
A Consensus Study Report of
THE NATIONAL ACADEMIES PRESS
Washington, DC
www.nap.edu
THE NATIONAL ACADEMIES PRESS 500 Fifth Street, NW, Washington, DC 20001
This activity was supported by a contract between the National Academy of Sciences and the U.S. Centers for Disease Control and Prevention. Any opinions, findings, conclusions, or recommendations expressed in this publication do not necessarily reflect the views of any organization or agency that provided support for the project.
International Standard Book Number-13: 978-0-309-67205-4
International Standard Book Number-10: 0-309-67205-8
Digital Object Identifier: https://doi.org/10.17226/25687
Additional copies of this publication are available from the National Academies Press, 500 Fifth Street, NW, Keck 360, Washington, DC 20001; (800) 624-6242 or (202) 334-3313; http://www.nap.edu.
Copyright 2020 by the National Academy of Sciences. All rights reserved.
Printed in the United States of America
Suggested citation: National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR’s contribution (2012–2017) to Rwanda’s Human Resources for Health Program. Washington, DC: The National Academies Press. https://doi.org/10.17226/25687.
The National Academy of Sciences was established in 1863 by an Act of Congress, signed by President Lincoln, as a private, nongovernmental institution to advise the nation on issues related to science and technology. Members are elected by their peers for outstanding contributions to research. Dr. Marcia McNutt is president.
The National Academy of Engineering was established in 1964 under the charter of the National Academy of Sciences to bring the practices of engineering to advising the nation. Members are elected by their peers for extraordinary contributions to engineering. Dr. John L. Anderson is president.
The National Academy of Medicine (formerly the Institute of Medicine) was established in 1970 under the charter of the National Academy of Sciences to advise the nation on medical and health issues. Members are elected by their peers for distinguished contributions to medicine and health. Dr. Victor J. Dzau is president.
The three Academies work together as the National Academies of Sciences, Engineering, and Medicine to provide independent, objective analysis and advice to the nation and conduct other activities to solve complex problems and inform public policy decisions. The National Academies also encourage education and research, recognize outstanding contributions to knowledge, and increase public understanding in matters of science, engineering, and medicine.
Learn more about the National Academies of Sciences, Engineering, and Medicine at www.nationalacademies.org.
Consensus Study Reports published by the National Academies of Sciences, Engineering, and Medicine document the evidence-based consensus on the study’s statement of task by an authoring committee of experts. Reports typically include findings, conclusions, and recommendations based on information gathered by the committee and the committee’s deliberations. Each report has been subjected to a rigorous and independent peer-review process, and it represents the position of the National Academies on the statement of task.
Proceedings published by the National Academies of Sciences, Engineering, and Medicine chronicle the presentations and discussions at a workshop, symposium, or other event convened by the National Academies. The statements and opinions contained in proceedings are those of the participants and are not endorsed by other participants, the planning committee, or the National Academies.
For information about other products and activities of the National Academies, please visit www.nationalacademies.org/about/whatwedo.
COMMITTEE ON THE EVALUATION OF STRENGTHENING HUMAN RESOURCES FOR HEALTH CAPACITY IN THE REPUBLIC OF RWANDA UNDER THE PRESIDENT’S EMERGENCY PLAN FOR AIDS RELIEF (PEPFAR)
ANN E. KURTH (Chair), Linda Koch Lorimer Professor and Dean, Yale School of Nursing
TILL BÄRNIGHAUSEN, Alexander von Humboldt University Professor and Director, Heidelberg Institute of Global Health, University of Heidelberg, Germany; Adjunct Professor of Global Health, Harvard T.H. Chan School of Public Health
ERAN BENDAVID, Associate Professor of Medicine, Stanford University
CARLA CASTILLO-LABORDE, Assistant Professor, Centro de Epidemiología y Políticas de Salud, Facultad de Medicina Clínica Alemana, Universidad del Desarrollo, Santiago, Chile
ELVIN H. GENG, Professor of Medicine, Division of Infectious Diseases, and Director of the Center for Dissemination and Implementation at the Institute for Public Health, Washington University in St. Louis
FASTONE M. GOMA, Dean and Associate Professor, University of Zambia School of Medicine, Lusaka, Zambia
LAURA HOEMEKE, Global Health Policy Consultant; Adjunct Professor, Gillings School of Global Public Health, University of North Carolina at Chapel Hill
ANGELINA KAKOOZA-MWESIGE, Senior Lecturer and Pediatric Neurologist, Department of Pediatrics and Child Health, Makerere University College of Health Sciences, Kampala, Uganda
EMMANUEL B. K. LUYIRIKA, Executive Director, African Palliative Care Association, Kampala, Uganda
MOSA MOSHABELA, Dean and Head, School of Nursing and Public Health, University of KwaZulu-Natal, Durban, South Africa
DENIS NASH, Distinguished Professor and Executive Director, CUNY Institute for Implementation Science in Population Health, City University of New York Graduate School of Public Health and Health Policy
CHARLES OK PANNENBORG, Retired Chief Health Advisor and Chief Health Scientist, The World Bank Group
DEREK J. SLOAN, Senior Clinical Lecturer and Consultant Infectious Diseases Physician, University of St. Andrews, Scotland
SHEILA D. TLOU, Chair, Global HIV Prevention Coalition; Former Director of the Regional Support Team for Eastern and Southern Africa, UNAIDS (until May 2019)
Study Staff
SUSAN F. E. MILNER, Study Director
CECILIA MUNDACA SHAH, Senior Program Officer (until December 2018)
KATYE M. MAGEE, Associate Program Officer (until February 2019)
EMMA FINE, Associate Program Officer (from May 2019)
T. ANH TRAN, Research Associate (until March 2020)
ZARIA FYFFE, Senior Program Assistant (from March 2020)
MICHELLE KVALSUND, National Academy of Medicine Fellow in Osteopathic Medicine
JULIE A. PAVLIN, Senior Director, Board on Global Health
Consultant
BRIDGET B. KELLY, Principal Consultant, Burke Kelly Consulting
EnCompass LLC Consulting Team
SARAH SMITH LUNSFORD, Team Lead and Senior Evaluation Specialist
SIMON HILTEBEITEL, Monitoring and Evaluation Specialist II
KELSEY SIMMONS, Evaluation Specialist II
AMY BHOPAL, Special Projects Associate
REBECCA K. CATHCART, Rwanda-Based Consultant
SYLVESTRE MUSENGIMANA, Rwanda-Based Consultant
Reviewers
This Consensus Study Report was reviewed in draft form by individuals chosen for their diverse perspectives and technical expertise. The purpose of this independent review is to provide candid and critical comments that will assist the National Academies of Sciences, Engineering, and Medicine in making each published report as sound as possible and to ensure that it meets the institutional standards for quality, objectivity, evidence, and responsiveness to the study charge. The review comments and draft manuscript remain confidential to protect the integrity of the deliberative process.
We thank the following individuals for their review of this report:
PAULIN BASINGA, Bill & Melinda Gates Foundation
MOLLY COOKE, University of California, San Francisco
SUNDEEP K. GUPTA, Partners in Hope, Malawi
WILLIAM HOLZEMER, Rutgers University
RICHARD MARLINK, Rutgers Global Health Institute
SANGEETA MOOKHERJI, The George Washington University
KHAMA ROGO, World Bank
KIMBERLY A. SCOTT, Virginia Department of Health
JEAN N. UTUMATWISHIMA, Rwamagana Provincial Hospital
Although the reviewers listed above provided many constructive comments and suggestions, they were not asked to endorse the conclusions or recommendations of this report nor did they see the final draft before its release. The review of this report was overseen by KRISTINE M. GEBBIE, Flinders University School of Nursing and Midwifery, and ANN M. ARVIN, Stanford University School of Medicine. They were responsible
for making certain that an independent examination of this report was carried out in accordance with the standards of the National Academies and that all review comments were carefully considered. Responsibility for the final content rests entirely with the authoring committee and the National Academies.
Preface
The world needs more health workers. The need is acute in low-income country settings with the highest burden of HIV disease, including in sub-Saharan Africa. With 11 percent of the world’s population, 24 percent of the global disease burden, and more than 70 percent of HIV disease,1 yet only 3 percent of the world’s health workforce, this region is ripe for innovation and investment to improve human resources for health (HRH).2 Given this context, the Rwanda HRH Program was an important experiment, one that arose from a unique set of circumstances—a postgenocide need to rebuild, impressive advancement in addressing baseline HIV metrics, and a government dedicated to improving Rwandan health via creation of a stronger and more self-sustaining health system.
The President’s Emergency Plan for AIDS Relief (PEPFAR) and its leadership in the Office of the U.S. Global AIDS Coordinator are mandated to address HIV, and the Government of Rwanda has a commitment to improve the health of all of its people. These goals may seem to be in conflict—a vertical focus on a single disease and a horizontal approach to building a health system able to address all citizens’ basic health needs. The debate over vertical versus horizontal programming has been an ongoing
___________________
1 Kharsany, A. B. M., and Q. A. Karim. 2016. HIV infection and AIDS in sub-Saharan Africa: Current status, challenges and opportunities. Open AIDS 10.
2 IFC (International Finance Corporation). 2019. Sub-Saharan Africa: Health and education. https://www.ifc.org/wps/wcm/connect/REGION__EXT_Content/IFC_External_Corporate_Site/Sub-Saharan+Africa/Priorities/Health+and+Education (accessed November 6, 2019).
one; however, given the growing burden of noncommunicable diseases (accounting for 71 percent of global mortality)3 and the evolution of managing HIV disease as a chronic care model with multiple comorbidities, there may be increasing congruence. In this context, we must admire the creativity of PEPFAR and the Government of Rwanda in partnering to launch this distinctive program.
The Rwanda HRH Program experiment is one from which we believe salient lessons can be drawn for the design and implementation of workforce capacity building that advances both HIV prevention and care and country attainment of universal health access and coverage. Many countries and collaborations can learn from the way the HRH Program was conceived, executed, and evaluated. These lessons are especially timely given the commitment made at the United Nation’s first high-level meeting on universal health coverage (UHC) on September 23, 2019, which recognized the substantial shortfall of workers in low- and middle-income countries and “the need to train, build, and retain a skilled health workforce,” noting in particular “nurses, midwives, and community health workers.”4
Perhaps a primary lesson of the Rwanda HRH paradigm is the limited time line of the Program, foreshortened as it was by a 2-year reduction in PEPFAR funding. Even the original conception of a program designed for less than a decade was arguably too brief to create cohorts of expert clinicians who could be educated, practice and have an impact on care, and educate the next generation of clinicians in turn. Another key lesson is that the degree of structural change needed for sustained gains in institutional capacity for health professional education was underestimated. Likewise, faculty development to take on this work at the University of Rwanda was not as grounded in partnership engagement as it could have been.
Given the vertical mission of PEPFAR, the goal to understand whether the HRH Program funding improved outcomes for people living with HIV (PLHIV) is logical. As a committee, we thought deeply about how one might attempt to answer this as an overarching evaluation question, given the lack of baseline and time-series data, the too brief period of funded work, and the challenges of navigating between a vertical focus on HIV outcomes and a more horizontal nation-building program of health workforce advancement. The committee’s final conclusion was that it would not be possible to determine attribution. But given the continued HIV epidemic in sub-Saharan Africa, as well as the issue of how best to achieve UHC
___________________
3 WHO (World Health Organization). 2018. Noncommunicable diseases. https://www.who.int/news-room/fact-sheets/detail/noncommunicable-diseases (accessed November 6, 2019).
4 United Nations. 2019. Political declaration of the high-level meeting on universal health coverage: “Universal health coverage: Moving together to build a healthier world.” New York: United Nations. https://undocs.org/en/A/RES/74/2 (accessed January 28, 2020).
worldwide, this evaluation was an opportunity to highlight the importance of HRH and how it can affect not only PLHIV, but ultimately everyone’s health.
The study was completed in a relatively accelerated time frame, but with thoughtfulness and methodologic depth. Following from our commitment to learn as much as possible, despite the limitations due to the circumstances of this Program and this evaluation, the committee offers not only findings about this Program, but also suggestions for how future endeavors such as this could be designed to more explicitly enable learning to follow from innovation.
We are deeply grateful for the work of the dedicated staff, consultants, and committee members, and most of all to the Rwandan government representatives and university employees, faculty who participated through U.S. partner institutions, and others throughout the country who participated in this evaluation. We invite the reader to consider ways to apply the lessons to their own work to improve the health of people and populations.
Ann E. Kurth, Chair
Committee on the Evaluation of Strengthening Human Resources for Health Capacity in the Republic of Rwanda Under the President’s Emergency Plan for AIDS Relief (PEPFAR)
This page intentionally left blank.
Acknowledgments
This Consensus Study Report and program evaluation would not have been possible without the invaluable contributions of many individuals. First and foremost, the committee, consultants, and study staff wish to express their sincere appreciation to the 87 anonymous respondents who so generously gave their time to provide their insights during the study’s qualitative interviews.
We also wish to express our sincere thanks to the speakers, listed in Appendix B, and other participants at the committee’s public sessions in Washington, DC, and Kigali, Rwanda, for their time and their willingness to share with us their various perspectives.
In addition, we are grateful to Dr. Khama Rogo and Dr. Daniel Yumbya for providing insight and guidance on the health professional education systems and accreditation processes in East Africa.
We are profoundly grateful to the leadership and staff at the Rwandan Ministry of Health, Rwanda Biomedical Center, University of Rwanda, Rwanda Medical Association, and Rwanda Nurses and Midwives Union for providing the data presented in this study and for their thoughtful input and cooperation. Without the support and commitment, at every juncture, of Dr. Diane Gashumba, Dr. Jean Pierre Nyemazi, Dr. Sabin Nsanzimana, Mr. Joseph Shema, Dr. Parfait Uwaliraye, Dr. Patrck Ndimubanzi, Dr. Jeanine Condo, Dr. Phil Cotton, Dr. David Ntirushwa, and Mr. Andre Gitembagara this report would not have been possible.
We are extremely appreciative of the efforts of Dr. Kristine Gebbie, who twice oversaw reviews of this study, first at the protocol stage and later as a
final manuscript. In addition, we are indebted to the following individuals who gave generously of their time and expertise to review the study’s protocol in December 2018: Dr. Sangeeta Mookherji, The George Washington University; Dr. Sam Phiri, Lighthouse Trust, Malawi; Dr. Shoshanna Sofaer, City University of New York; Dr. Allison P. Squires, New York University; and Dr. Kathryn Whetten, Duke University.
We also wish to thank staff with the President’s Emergency Plan for AIDS Relief (PEPFAR) and the U.S. Centers for Disease Control and Prevention (CDC) Rwanda and with CDC Atlanta, most especially Ms. Emma Mtiro, for their support and guidance over the course of the study.
Finally, many within the National Academies of Sciences, Engineering, and Medicine also provided assistance with the study. We would like to thank Rebecca Morgan with the National Academies’ Research Center staff for her assistance with research. We also wish to thank Ron Brown, Daniel Cesnalis, Anna Isabel Camilo Javier, Julie Wiltshire, and Skip Day (Office of the Chief Financial Officer); Lauren Shern, Taryn Young, and Bettina Seliber (HMD Executive Office); Maryjo Oster and Dorothy Zolandz (Report Review Committee); Marc Gold (Office of the General Counsel); Andrew Grafton, Greta Gorman, Nicole Joy, Jeanay Butler, and Sadaf Faraz (Office of Communications); and Stephanie Miceli (Office of News and Public Information).
Contents
1 INTRODUCTION: EVALUATION SCOPE AND APPROACH
Rwanda Human Resources for Health Program
Committee’s Approach to the Charge
2 EVALUATION DESIGN, METHODS, AND LIMITATIONS
3 HUMAN RESOURCES FOR HEALTH PROGRAM CONTEXT, VISION, AND DESIGN
Context in Rwanda Leading Up to the Program
Programmatic Management Approaches and Challenges
5 INSTITUTIONAL CAPACITY FOR HEALTH PROFESSIONAL EDUCATION
Recruitment and Retention of Health Professional Educators
Curricula and Programs for Health Professional Education
Accreditation and Specialty Programs
Research and Professional Development
Health in the Rwandan Labor Market
Recruitment and Retention of Health Workers
Upgrading and Procuring Equipment
7 EFFECTS ON HUMAN RESOURCES FOR HEALTH AND QUALITY OF CARE
Overall Effect of the Human Resources for Health Program
Sustainability and Institutionalization
Overarching Evaluation Conclusions
Implications for HIV and Human Resources for Health Programming
This page intentionally left blank.
Box, Figures, and Tables
BOX
FIGURES
S-1 Key findings: successes and challenges
1-2 Theoretical causal pathway for the evaluation
3-1 The size of the Rwandan economy from 2000–2018 in current U.S. dollars
3-2 Health expenditures in Rwanda from 2000–2016 (in purchasing power parity per capita)
3-3 Proportion of health expenditure in Rwanda by revenue source
3-4 PEPFAR Rwanda planned funding by program area and percent allocation for HSS
3-5 MOH expenditures FY 2010/2011–FY 2014/2015 (U.S. dollars)
6-1 Doctors, nurses, and midwives per 10,000 population
6-2 Total physician specialists graduated under the HRH Program by year
6-3 Total nursing specialists graduated under the HRH Program by specialty
6-4 Career trajectory of interviewed HRH Program graduates following graduation
6-5 Career trajectory of interviewed HRH Program trained nursing respondents
7-1 HIV prevalence among Rwandan adults aged 15–49
7-2 Antiretroviral therapy coverage, 2010–2017
7-3 New HIV infections, 1990–2018
7-4 PMTCT need and number of pregnant women on ARVs, 2011–2016
7-5 Data-driven causal pathway
TABLES
2-1 HIV Prevalence and ART Coverage in Facility Microsystem Districts
2-2 Interview Respondent Sample
2-3 Interview Topics by Respondent Type
3-1 PEPFAR Funding Sources That Contributed to the HRH Program
3-2 HRH Program Budget by Year (U.S. Dollars)
3-3 CDC Disbursements for the HRH Program by Year (U.S. Dollars)
3-5 MOH Health Workforce Expenditures with Category Breakdowns by Year (U.S. Dollars)
3-6 Evolution of the HRH Program’s Goals and Approaches
3-7 HRH Program Expenditures of PEPFAR Investments (U.S. Dollars)
5-1 Comparative Analysis of MHA and MSN Programs
5-2 Required CPD Credits by Profession
6-1 Number of Health Practitioners in Rwanda by Level
6-2 University of Rwanda Medical Student Graduation Numbers by Program
6-3 Number of Practitioners Receiving Their License
6-4 Distribution of All Physician Specialists and General Practitioners by Health Facility Level
6-5 Equipment by District Procured with PEPFAR Support Under the HRH Program, 2013–2017
6-6 Equipment by Site Procured with PEPFAR Support Under the HRH Program, 2013–2017
6-7 Types of Equipment Procured with PEPFAR Support Under the HRH Program, 2013–2017
7-1 Progress Toward 90-90-90 Treatment Cascade Targets in Rwanda
7-2 UNAIDS and PEPFAR Estimates of PMTCT Coverage in Rwanda, 2011–2016
This page intentionally left blank.
Acronyms and Abbreviations
APEFE | Association pour la Promotion de l’Éducation et de la Formation à l’Etranger |
ART | antiretroviral therapy |
ARV | antiretroviral (drug) |
BSN | Bachelor of Science in Nursing |
CD4 | cluster of differentiation 4 |
CDC | U.S. Centers for Disease Control and Prevention |
CHAI | Clinton Health Access Initiative |
CHUB | Centre Hospitalier Universitaire de Butare/University Teaching Hospital, Butare |
CHUK | Centre Hospitalier Universitaire de Kigali/University Teaching Hospital, Kigali |
CHW | community health worker |
CMHS | College of Medicine and Health Sciences |
COP | country operational plan |
COSECSA | College of Surgeons of East, Central, and Southern Africa |
CPD | continuing professional development |
ESTHER | Ensemble pour une solidarité thérapeutique hospitalière en réseau/Network for Therapeutic Solidarity in Hospitals |
FCS(ECSA) | Fellowship of the College of Surgeons of East, Central, and Southern Africa |
FY | fiscal year |
GDP | gross domestic product |
GHWA | Global Health Workforce Alliance |
GP | general practitioner |
HRH | human resources for health |
HSS | health systems strengthening |
HSSP IV | Health Sector Strategic Plan 4 |
LMIC | low- and middle-income country |
M&E | monitoring and evaluation |
MDG | Millennium Development Goal |
MHA | Master of Hospital and Healthcare Administration |
MIFOTRA | Ministry of Public Service and Labour |
MINECOFIN | Ministry of Finance and Economic Planning |
M.Med. | Master of Medicine |
MOE | Ministry of Education |
MOH | Ministry of Health |
MOU | memorandum of understanding |
MSN | Master of Science in Nursing |
NGO | nongovernmental organization |
NISR | National Institute of Statistics of Rwanda |
PEPFAR | President’s Emergency Plan for AIDS Relief |
PLHIV | people living with HIV |
PMTCT | prevention of mother-to-child transmission |
PrEP | preexposure prophylaxis |
RBC | Rwanda Biomedical Center |
RPHIA | Rwanda Population-Based HIV Impact Assessment |
RWF | Rwandan Francs |
Sida | Swedish International Development Cooperation Agency |
SMART | specific, measurable, achievable, relevant, and time-based |
SPH | School of Public Health |
SPIU | Single Project Implementation Unit |
TAGGS | Tracking Accountability in Government Grants |
TDF | tenofovir disoproxil fumarate |
THET | Tropical Health Education Trust |
UGHE | University of Global Health Equity |
UHC | universal health coverage |
UNAIDS | Joint United Nations Programme on HIV/AIDS |
USAID | United States Agency for International Development |
USD | U.S. dollar |
USI | U.S. institution |
WHO | World Health Organization |
This page intentionally left blank.