How to evaluate the implementation of complex health programmes in low-income settings: the approach of the Gavi Full Country Evaluations.

Vaccination, like most other public health services, relies on a complex package of intervention components, functioning systems and committed actors to achieve universal coverage. Despite significant investment in immunization programmes, national coverage trends have slowed and equity gaps have grown. This paper describes the design and implementation of the Gavi Full Country Evaluations, a multi-country, prospective, mixed-methods approach whose goal was to monitor and evaluate processes, inputs, outputs and outcomes of immunization programmes in Bangladesh, Mozambique, Uganda and Zambia. We implemented the Full Country Evaluations from 2013 to 2018 with the goal of identifying the drivers of immunization programme improvement to support programme implementation and increase equitable immunization coverage. The framework supported methodological and paradigmatic flexibility to respond to a broad range of evaluation and implementation research questions at global, national and cross-country levels, but was primarily underpinned by a focus on evaluating processes and identifying the root causes of implementation breakdowns. Process evaluation was driven by theories of change for each Gavi funding stream (e.g. Health Systems Strengthening) or activity, ranging from global policy development to district-level programme implementation. Mixing of methods increased in relevance and rigour over time as we learned to build multiple methods into increasingly tailored evaluation questions. Evaluation teams in country-based research institutes increasingly strengthened their level of embeddedness with immunization programmes as the emphasis shifted over time to focus more heavily on the use of findings for programme learning and adaptation. Based on our experiences implementing this approach, we recommend it for the evaluation of other complex interventions, health programmes or development assistance.

Implementation of the World's largest measles-rubella mass vaccination campaign in Bangladesh: a process evaluation.

BACKGROUND: Gavi, the Vaccine Alliance, supported a mass vaccination Measles-Rubella Campaign (MRC) in Bangladesh during January-February 2014. METHODS: We conducted a mixed-method process evaluation to understand the successes and challenges in implementation of the MRC. We reviewed documents for the MRC and the immunization programme in Bangladesh; observed meetings, vaccination sessions, and health facilities; and conducted 58 key informant interviews, 574 exit interviews with caregivers and 156 brief surveys with stakeholders involved in immunization. Our theory of Change for vaccination delivery guided our assessment of ideal implementation milestones and indicators to compare with the actual implementation processes. RESULTS: We identified challenges relating to country-wide political unrest, administrative and budgetary delays, shortage of transportation, problems in registration of target populations, and fears about safety of the vaccine. Despite these issues, a number of elements contributed to the successful launch of the MRC. These included: the comprehensive design of the campaign; strong partnerships between immunization authorities in the government system, Alliance partners, and civil society actors; and motivated and skilled health workers at different levels of the health system. CONCLUSIONS: The successful implementation of the MRC in spite of numerous contextual and operational challenges demonstrated the adaptive capacity of the national immunization programme and its partners that has positive implications for future introductions of Gavi-supported vaccines.

Maternal and child health and neighborhood context: the selection and construction of area-level variables.

BACKGROUND: Increasingly, maternal and child health researchers are employing the statistical approach of multilevel modeling analysis to simultaneously examine the relationship between contextual and individual determinants and maternal and child health outcomes. This review addresses the following questions: (1) What categories of neighborhood characteristics have been addressed? (2) How were those neighborhood characteristics operationalized? METHODS: A literature review identified 31 relevant articles published between January 1999 and March 2004. The articles were read with special attention toward the measurement of neighborhood characteristics. RESULTS: Twelve categories of neighborhood characteristics represented in the articles include income/wealth, employment, family structure, population composition, housing, mobility, education, occupation, social resources, violence and crime, deviant behavior and physical conditions. A wide diversity of approaches was used to measure these characteristics. The most widely utilized source of data was that of administrative records from the census or local government authorities. Although most authors provided theoretical explanations of their choice to examine broad neighborhood constructs, few were explicit about why certain indicators were selected to measure these constructs. CONCLUSIONS: There are theoretical, methodological and practical barriers in the measurement of the neighborhood context which must be addressed for the field to move forward. These barriers are discussed and recommendations are made for addressing them in future research.