Lessons Learned From Implementing Prospective, Multicountry Mixed-Methods Evaluations for Gavi and the Global Fund.

INTRODUCTION: As global health programs have become increasingly complex, corresponding evaluations must be designed to assess the full complexity of these programs. Gavi and the Global Fund have commissioned 2 such evaluations to assess the full spectrum of their investments using a prospective mixed-methods approach. We aim to describe lessons learned from implementing these evaluations. METHODS: This article presents a synthesis of lessons learned based on the Gavi and Global Fund prospective mixed-methods evaluations, with each evaluation considered a case study. The lessons are based on the evaluation team's experience from over 7 years (2013-2020) implementing these evaluations. The Centers for Disease Control and Prevention Framework for Evaluation in Public Health was used to ground the identification of lessons learned. RESULTS: We identified 5 lessons learned that build on existing evaluation best practices and include a mix of practical and conceptual considerations. The lessons cover the importance of (1) including an inception phase to engage stakeholders and inform a relevant, useful evaluation design; (2) aligning on the degree to which the evaluation is embedded in the program implementation; (3) monitoring programmatic, organizational, or contextual changes and adapting the evaluation accordingly; (4) hiring evaluators with mixed-methods expertise and using tools and approaches that facilitate mixing methods; and (5) contextualizing recommendations and clearly communicating their underlying strength of evidence. CONCLUSION: Global health initiatives, particularly those leveraging complex interventions, should consider embedding evaluations to understand how and why the programs are working. These initiatives can learn from the lessons presented here to inform the design and implementation of such evaluations.

Partnership and Participation-A Social Network Analysis of the 2017 Global Fund Application Process in the Democratic Republic of the Congo and Uganda.

Background: The Global Fund to Fight AIDS, Tuberculosis and Malaria was founded in 2002 as a public-private partnership between governments, the private sector, civil society, and populations affected by the three diseases. A key principle of the Global Fund is country ownership in accessing funding through "engagement of in-country stakeholders, including key and vulnerable populations, communities, and civil society." Research documenting whether diverse stakeholders are actually engaged and on how stakeholder engagement affects processes and outcomes of grant applications is limited. Objective: To examine representation during the 2017 Global Fund application process in the Democratic Republic of the Congo (DRC) and Uganda and the benefits and drawbacks of partnership to the process. Methods: We developed a mixed-methods social network survey to measure network structure and assess perceptions of how working together in partnership with other individuals/organizations affected perceived effectiveness, efficiency, and country ownership of the application process. Surveys were administered from December 2017-May 2018, initially to a set of central actors, followed by any individuals named during the surveys (up to 10) as collaborators. Network analyses were conducted using R. Findings: Collaborators spanning many organizations and expertise areas contributed to the 2017 applications (DRC: 152 nodes, 237 ties; Uganda: 118 nodes, 241 ties). Participation from NGOs and civil society representatives was relatively strong, with most of their ties being to different organization types, Uganda (63%), and DRC (67%), highlighting their collaborative efforts across the network. Overall, the perceived benefits of partnership were high, including very strong ratings for effectiveness in both countries. Perceived drawbacks of partnership were minimal; however, less than half of respondents thought partnership helped reduce transaction costs or financial costs, suggesting an inclusive and participatory process may come with short-term efficiency tradeoffs. Conclusions: Social network analysis can be useful for identifying who is included and excluded from the process, which can support efforts to ensure stronger, more meaningful engagement in future Global Fund application processes.