A realist systematic review of evidence from low- and middle-income countries of interventions to improve immunization data use.

BACKGROUND: The use of routine immunization data by health care professionals in low- and middle-income countries remains an underutilized resource in decision-making. Despite the significant resources invested in developing national health information systems, systematic reviews of the effectiveness of data use interventions are lacking. Applying a realist review methodology, this study synthesized evidence of effective interventions for improving data use in decision-making. METHODS: We searched PubMed, POPLINE, Centre for Agriculture and Biosciences International Global Health, and African Journals Online for published literature. Grey literature was obtained from conference, implementer, and technical agency websites and requested from implementing organizations. Articles were included if they reported on an intervention designed to improve routine data use or reported outcomes related to data use, and targeted health care professionals as the principal data users. We developed a theory of change a priori for how we expect data use interventions to influence data use. Evidence was then synthesized according to data use intervention type and level of the health system targeted by the intervention. RESULTS: The searches yielded 549 articles, of which 102 met our inclusion criteria, including 49 from peer-reviewed journals and 53 from grey literature. A total of 66 articles reported on immunization data use interventions and 36 articles reported on data use interventions for other health sectors. We categorized 68 articles as research evidence and 34 articles as promising strategies. We identified ten primary intervention categories, including electronic immunization registries, which were the most reported intervention type (n = 14). Among the research evidence from the immunization sector, 32 articles reported intermediate outcomes related to data quality and availability, data analysis, synthesis, interpretation, and review. Seventeen articles reported data-informed decision-making as an intervention outcome, which could be explained by the lack of consensus around how to define and measure data use. CONCLUSIONS: Few immunization data use interventions have been rigorously studied or evaluated. The review highlights gaps in the evidence base, which future research and better measures for assessing data use should attempt to address.

Current Challenges and Future Possibilities for Immunization Information Systems.

Immunizations have proven to be an important tool for public health and for reducing the impact of vaccine preventable diseases. To realize the maximum benefits of immunizations, a coordinated effort between public policy, health care providers and health systems is required to increase vaccination coverage and to ensure high-quality data are available to inform clinical and public health interventions. Immunization information systems (IIS) are confidential, population-based, computerized databases that record all immunization doses administered by participating providers to persons residing within a given geopolitical area. The key output of an IIS is high-quality data for use in targeting and monitoring immunization program activities and providing clinical decision support at the point of care. To be truly effective, IISs need to form a nationwide network and repository of immunization data. Since the early 2000s Centers for Disease Control and Prevention has made strides to help IIS move toward a nationwide network through efforts focused on improving infrastructure and functionality, such as the IIS Minimum Functional Standards, and the IIS Annual Report, a self-reported data collection of IIS progress toward achieving the functional standards. While these efforts have helped immunization programs achieve more functional standards, there is a need to shift focus from infrastructure and functionality improvements to high data quality through objective measurement of IIS performance and evaluating critical outcomes. Additionally, realizing the vision of a nationwide repository of high-quality immunization data requires tackling the many challenges that impact data quality and availability including those related to policy, data sharing, data use, aging IIS technology, sustainability, and participation in the IIS. This paper describes the current state of IIS in the United States, critical challenges impacting the quality of data in IIS, and potential components of a future IIS model to address these challenges.

Data management needs assessment for the scale-up of district health information system and introduction of routine (essential) immunization module in Bauchi State, Nigeria, 2015.

Introduction: the National Primary Health Care Development Agency, African Field Epidemiology Network, United States Centers for Disease Control and Prevention and the Bill and Melinda Gates Foundation are implementing a Routine Immunization (RI) Module as part of their Routine Health Data Management System based on the 2013 - 2015 Accountability Framework for RI in Nigeria. To inform planning and evidence-based decision making, a data management needs assessment was conducted in Bauchi state which was one of the states selected for the deployment of the DHIS2 RI module. Methods: desk reviews were conducted, and a semi-structured questionnaire was administered in four Local Government Areas (LGAs) in Bauchi state that were selected based on the initial evaluation of the performance of all 20 Bauchi LGAs. Ganjuwa and Shira were selected as high-performing LGAs and Alkaleri and Bogoro as low-performing LGAs. Four Health Facilities (HF) were selected in each LGA based on rural or urban classification, type of HFs (private or public), security and accessibility. Results: local Immunization Officers (LIOs) prepare monthly reports in high-performing LGAs, and Community Health Care workers are mostly (69%) responsible for report compilation at the HFs. Shira and Alkaleri met 77% and 44% of training indicator targets, respectively, in the previous 12 months. Data recording and reporting was the type of training received the most by health facility personnel. Functioning refrigerators were in all visited LGAs, working thermometer and updated temperature monitoring charts were available in all the cold chain stores. However, no health facility reported having available computers for data-related activities. Conclusion: this assessment provided an improved understanding of the Bauchi state Routine Health Data Management System and informed the content of the state-wide scale-up.

Assessment of immunization data quality of routine reports in Ho municipality of Volta region, Ghana.

BACKGROUND: Immunization has been an important public health intervention for preventing and reducing child morbidity and mortality over the years and coverage has increased in the past decades. However, the validity of the data from immunization coverages is usually disputed. Immunization data from health facilities show poor concordance between tallied registers and monthly reports as they are reported to higher levels of the health system. The study assessed the quality of data from routine immunization of some health facilities in the Ho central municipality in the Volta region of Ghana. METHODS: A descriptive cross-sectional study was used to review routine immunization data in tallied registers and reports submitted to the Municipal Health Directorate (MHD) from January to December, 2015. Simple random sampling was used to select three health facilities in Ho central municipality. The World Health Organization (WHO) Data Quality Self-assessment (DQS) tool was the main instrument used to present and analyze data for accuracy and discrepancy level between the tallied registers and reports. A template was created in Microsoft excel which automatically presented accuracy and discrepancy levels when data was entered. Ethical approval for the study was obtained from Ghana Health Service Ethics Review Committee. RESULTS: The result showed discrepancies between recounted tallies at the facilities and reports submitted to the MHD. Accuracy ratios of 102, 64 and 94% for Bacillus Calmette Guerin (BCG), Pentavalent (Penta) vaccine dose 3 and Measles 2 respectively indicating underreporting for BCG and over reporting for the rest were obtained. There was 460 over reported data to the municipal level representing accuracy ratio of 80% and discrepancy level of 20%. CONCLUSIONS: Immunization data was characterized by underreporting and overreporting, hence not accurate and lacked quality. Immunization data quality should be a priority among health staff at health facilities.

Assessment of a novel scanner-supported system for processing of child health and immunization data in Uganda.

BACKGROUND: Electronic data capturing has the potential to improve data quality and user-friendliness compared to manually processed, paper-based documentation systems. The MyChild system uses an innovative approach to process immunization data by employing detachable vouchers integrated into a vaccination booklet which are then scanned and converted into individual-level health data. The aim was to evaluate the MyChild data capturing system by assessing the proportion of correctly processed vouchers and to compare the user-friendliness in term of time spent on documentation and health worker experiences with the standard health information system at health facilities in Uganda. METHODS: We used a mixed method approach. Documented data were manually copied and compared to processed health records to calculate the proportion of correctly registered vouchers. To compare time spend on documentation we did a continuous observational time-motion study and analyzed data using a Mann-Whitney U test. Semi-structured interviews were conducted to assess health workers' experiences and analyzed using conventional content analysis. Data was collected in 14 health facilities in two districts in Uganda using different systems. RESULTS: The MyChild system processed 97% (224 of 231) of the vouchers correctly. Recording using the MyChild system increased time spend on documentation of vaccination follow-up visits by 24 s compared to the standard system (02:25 vs. 02:01 min/child, Mann-Whitney U = 6293, n1 = 115, n2 = 151, p < 0.001 two-tailed, Z = - 3.861, r = 0.186). However, high variance between health centers using the same health information system suggests that documentation time differences can be attributed to other factors than the way information was processed. Health workers perceived both health management information systems as predominantly functional and easy to use, while the MyChild system achieved a higher level of satisfaction. CONCLUSIONS: The MyChild system electronically processes individual-level immunization data correctly without increasing significantly time spent on recording and is appreciated by health providers making it a potential solution to overcome shortcomings of present paper-based health information systems in health centers.

Improving immunization data quality in Peru and Mexico: Two case studies highlighting challenges and lessons learned.

INTRODUCTION: The Global Vaccine Action Plan and the Regional Immunization Action Plan of the Americas call for countries to improve immunization data quality. Immunization information systems, particularly electronic immunization registries (EIRs), can help to facilitate program management and increase coverage. However, little is known about efforts to develop and implement such systems in low- and middle-income countries. We present the experiences of Mexico and Peru in implementing EIRs. METHODS: We conducted case studies of an EIR in Mexico and of a population registry in Peru. Information was gathered from technical documents, stakeholder focus groups, site visits, and semi-structured interviews of national stakeholders. We organized findings into narratives that emphasized challenges and lessons learned. RESULTS: Mexico built one of the world's first EIRs, incorporating novel features such as local-level tracking of patients; however, insufficient resources and poor data registration practices led to the system's discontinuation. Peru created an information system to improve affiliation to social programs, including the immunization program and quality of demographic data. Mexico's experience highlights lessons in failed sustainability of an EIR and a laudable effort to reform a country's information system. Peru's demonstrates that attempts to improve health and other data may strengthen health systems, including immunization data. Major challenges in information system implementation and sustainability in Peru and Mexico related to funding, clear governance structures, and resistance among health workers. DISCUSSION: These case studies reinforce the need for countries to ensure adequate funding, plans for sustainability, and health worker capacity-building activities before implementing EIRs. They also suggest new approaches to implementation, including economic incentives for sub-national administrative levels and opportunities to link efforts to improve immunization data with other health and political priorities. More information on best practices is needed to ensure the successful adoption and sustainability of immunization registries in low- and middle-income countries.