Measuring for Success: Evaluating Leadership Training Programs for Sustainable Impact.

ABSTRACT

Background: In an era of global health security challenges such as the COVID-19 pandemic, there is greater need for strong leadership. Over the past decades, significant investments have been made in global health leadership development programs by governments and philanthropic organizations to address this need. Evaluating the societal impact of these programs remains challenging, despite consensus on the importance of public health leadership. Objective: This article identifies the gaps and highlights the critical role of monitoring and evaluation approaches in assessing the impact of global health leadership programs. Importantly, we also propose the theory of change (TOC) as a common framework and identify a set of tools and indicators that leadership programs can adapt and use. Methods: We carried out an informal review of major global health leadership programs, including a literature review on leadership program evaluation approaches. Current practices in assessing the short- to long-term outcomes of leadership training programs were explored and synthesized. We also examined use of program theory frameworks, such as theory of change to guide the evaluation strategy. We find the TOC approach can be enhanced by integrating evaluation-specific frameworks and establishing broad stakeholder buy-in. We highlight measurement challenges, proposed outcome indicators and evaluation methodologies, and outline the future direction for such efforts. Findings: Most evaluation of current leadership programs is focused on short-term individual-level outcomes, while reports on long-term societal impact were limited. Reciprocal impacts on and benefits for the "host" organizations were not included in evaluation metrics. Most programs had program logic or result chains, but with no well-articulated program theories. Conclusion: Key stakeholders involved in leadership training programs benefit from the evidence of rigorous program evaluations to inform decisions that address barriers in fostering global health leadership and improving population health outcomes. Insight into reciprocal change in host organizations is important. Evaluation of global health leadership training must go beyond the individual trainee and encompass organizational and community-level impacts. Documentation of long-lasting organizational and societal impacts is essential for donors to appreciate the return on their investment. Key Takeaways Evaluation plays an important role in understanding how leadership development takes place and how it contributes to improving public health outcomes.Making the case for investments in leadership development programs requires robust evidence from monitoring and evaluation strategies that link investments beyond the individual-level to longer-term societal impacts.The first critical step towards a strategy for success is for leadership programs to clearly build, articulate, share, and use their program theories or theories of change.Theories of change help identify the pathways (and potential tensions) through which leadership development programs effect change at the individual, organizational and community levels.Evaluation methods that examine outcomes of leadership programs should be multi-method, multi-level, and where possible include counterfactual outcomes.Allocation of funds to evaluate on-going and long-lasting societal impact of leadership programs should be a routine practice.