Evaluating the "Zindagi Mehfooz" Electronic Immunization Registry and Suite of Digital Health Interventions to Improve the Coverage and Timeliness of Immunization Services in Sindh, Pakistan: Mixed Methods Study.

BACKGROUND: The Zindagi Mehfooz (safe life; ZM) electronic immunization registry (EIR) is a comprehensive suite of digital health interventions that aims to improve equitable access, timeliness, and coverage of child immunizations through a smartphone-based app for vaccinators, web-based dashboards for supervisors and managers, text message alerts and reminders for caregivers, and a call center. It has been implemented at scale in Sindh Province, Pakistan. OBJECTIVE: This study aimed to present findings from an evaluation of the ZM-EIR suite of digital health interventions in order to improve data availability and use as a contribution, among other immunization program interventions, to enhanced immunization outcomes for children aged 12-23 months in Sindh Province. METHODS: The mixed methods study included (1) analysis of ZM-EIR system data to identify high-, moderate-, and low-adoption and compliance sites; (2) in-depth interviews with caregivers, vaccinators, supervisors, and managers in the Expanded Program for Immunization (EPI); and (3) pre-post outcome evaluation using vaccine coverage from the Multiple Indicator Cluster Surveys (MICS) 2014 and 2018-2019. Key outcomes of interest were improved data availability, use and contribution to immunization outcomes, including receipt of individual antigens (Bacillus Calmette-Guérin [BCG], pentavalent [Penta] 1-3, measles), full immunization (all antigens), and zero-dose children defined as children aged 6-23 months who have not received the first dosage of the diphtheria-pertussis-tetanus 1/Penta vaccine. RESULTS: By registering newborns, providing alerts and reminders, and tracking their immunization completion, the ZM-EIR improved data availability and use in the EPI. The ZM-EIR was well received by EPI administrators, supervisors, vaccinators, and caregivers. The key benefit highlighted by ZM-EIR users was a list of children who missed scheduled vaccines (defaulters). Through greater availability and use of data, the ZM-EIR implementation, as part of a broader package of immunization program-strengthening activities in Sindh Province, may have contributed to an increase in immunization coverage and timeliness for BCG vaccinations and a decrease in zero-dose children in 2018-2019 from 2014. Additional findings from the study included the dual burden of reporting on paper and gender-related considerations of female caregivers not wanting to provide their phone numbers to male vaccinators, creating barriers to greater uptake of the ZM-EIR. CONCLUSIONS: The ZM-EIR is a promising technology platform that has increased the availability and use of immunization data, which may have contributed, along with other intensive immunization program interventions, to improvements in immunization outcomes through systematic registration of children, alerts and reminders, and increased use of data for planning and monitoring by the EPI. TRIAL REGISTRATION: ISRCTN Registry ISRCTN23078223; https://doi.org/10.1186/ISRCTN23078223.

Electronic Immunization Registry in Rwanda: Qualitative Study of Health Worker Experiences.

BACKGROUND: Monitoring childhood immunization programs is essential for health systems. Despite the introduction of an electronic immunization registry called e-Tracker in Rwanda, challenges such as lacking population denominators persist, leading to implausible reports of coverage rates of more than 100%. OBJECTIVE: This study aimed to assess the extent to which the immunization e-Tracker responds to stakeholders' needs and identify key areas for improvement. METHODS: In-depth interviews were conducted with all levels of e-Tracker users including immunization nurses, data managers, and supervisors from health facilities in 5 districts of Rwanda. We used an interview guide based on the constructs of the Human, Organization, and Technology-Fit (HOT-Fit) framework, and we analyzed and summarized our findings using the framework. RESULTS: Immunization nurses reported using the e-Tracker as a secondary data entry tool in addition to paper-based forms, which resulted in considerable dissatisfaction among nurses. While users acknowledged the potential of a digital tool compared to paper-based systems, they also reported the need for improvement of functionalities to support their work, such as digital client appointment lists, lists of defaulters, search and register functions, automated monthly reports, and linkages to birth notifications and the national identity system. CONCLUSIONS: Reducing dual documentation for users can improve e-Tracker use and user satisfaction. Our findings can help identify additional digital health interventions to support and strengthen the health information system for the immunization program.

Twenty Years of Progress and the Way Forward: Immunization Registry Information System in Korea.

The National Immunization Program in The Republic of Korea offers mandatory and free vaccinations to children under 12, regulated by the Infectious Disease Prevention and Control Act. Tracking vaccination coverage is crucial for population protection and public health strategies. Since 2002, the Immunization Registry Information System (IRIS) has been used nationwide to capture vaccination data. This study reviewed documents related to IRIS's establishment and development. The Republic of Korea legally supports IRIS's construction and data collection, integrating vaccination data with the Ministry of the Interior and Safety's resident registration to minimize errors. This collaboration also facilitates cost reimbursement and digital registration, promoting wider vaccination coverage. IRIS manages expense claims once vaccination details are logged, and authorized medical institutions can access these records in real-time. Since 2015, the Korea Disease Control and Prevention Agency has been compiling annual data on national vaccination coverage. IRIS also sends automated reminders in 12 languages, reports adverse effects, and issues vaccination certificates. However, IRIS lacks integration between vaccine and disease registries, unlike countries such as England, Denmark, and the Netherlands. Improving integration capabilities could enhance IRIS's support for public health through an integrated information system.

Outcomes and Costs of the Transition From a Paper-Based Immunization System to a Digital Immunization System in Vietnam: Mixed Methods Study.

BACKGROUND: The electronic National Immunization Information System (NIIS) was introduced nationwide in Vietnam in 2017. Health workers were expected to use the NIIS alongside the legacy paper-based system. Starting in 2018, Hanoi and Son La provinces transitioned to paperless reporting. Interventions to support this transition included data guidelines and training, internet-based data review meetings, and additional supportive supervision visits. OBJECTIVE: This study aims to assess (1) changes in NIIS data quality and use, (2) changes in immunization program outcomes, and (3) the economic costs of using the NIIS versus the traditional paper system. METHODS: This mixed methods study took place in Hanoi and Son La provinces. It aimed to analyses pre- and postintervention data from various sources including the NIIS; household and health facility surveys; and interviews to measure NIIS data quality, data use, and immunization program outcomes. Financial data were collected at the national, provincial, district, and health facility levels through record review and interviews. An activity-based costing approach was conducted from a health system perspective. RESULTS: NIIS data timeliness significantly improved from pre- to postintervention in both provinces. For example, the mean number of days from birth date to NIIS registration before and after intervention dropped from 18.6 (SD 65.5) to 5.7 (SD 31.4) days in Hanoi (P<.001) and from 36.1 (SD 94.2) to 11.7 (40.1) days in Son La (P<.001). Data from Son La showed that the completeness and accuracy improved, while Hanoi exhibited mixed results, possibly influenced by the COVID-19 pandemic. Data use improved; at postintervention, 100% (667/667) of facilities in both provinces used NIIS data for activities beyond monthly reporting compared with 34.8% (202/580) in Hanoi and 29.4% (55/187) in Son La at preintervention. Across nearly all antigens, the percentage of children who received the vaccine on time was higher in the postintervention cohort compared with the preintervention cohort. Up-front costs associated with developing and deploying the NIIS were estimated at US $0.48 per child in the study provinces. The commune health center level showed cost savings from changing from the paper system to the NIIS, mainly driven by human resource time savings. At the administrative level, incremental costs resulted from changing from the paper system to the NIIS, as some costs increased, such as labor costs for supportive supervision and additional capital costs for equipment associated with the NIIS. CONCLUSIONS: The Hanoi and Son La provinces successfully transitioned to paperless reporting while maintaining or improving NIIS data quality and data use. However, improvements in data quality were not associated with improvements in the immunization program outcomes in both provinces. The COVID-19 pandemic likely had a negative influence on immunization program outcomes, particularly in Hanoi. These improvements entail up-front financial costs.

Routine Vaccination Coverage in an Adolescent Transgender Population in a Large Tertiary Care Center in the United States.

INTRODUCTION: To address healthcare disparities among transgender and gender diverse (TGD) adolescents, we examined vaccination coverage in those receiving gender-affirming care. METHOD: Our study analyzed de-identified data (2013-2022) from a tertiary care clinic. Comparing vaccination rates of 203 TGD adolescent patients to age-matched peers in New York State using CDC National Immunization Survey-Teen data. RESULTS: We found TGD patients had similar vaccination coverage to the general adolescent population. Notably, TGD patients had significantly higher up-to-date human papillomavirus vaccination coverage (76.4%, CI=69.9, 82.0) than the NYS adolescent population (60.5%, CI=53.0, 67.5). DISCUSSION: This suggests TGD adolescents at gender-affirming clinics maintain or exceed vaccination rates of their cisgender counterparts. Future research should explore vaccination rates for TGD individuals without access to gender-affirming care.

Locations of COVID-19 vaccination provision: Urban-rural differences.

PURPOSE: Our goal was to compare locations of COVID-19 vaccine provision in urban and rural communities over the course of the pandemic. METHODS: We used the Iowa Immunization Registry Information System (IRIS) to identify the organizations providing COVID-19 vaccines (eg, pharmacies, public health departments, and medical providers). Proportions of first-dose vaccines by organization type and patient census-based statistical area were generated. We calculated Chi-square tests to assess differences among metropolitan, micropolitan, and noncore communities. FINDINGS: IRIS data revealed that 64% (n = 2,043,251) of Iowans received their first COVID-19 vaccine between December 14, 2020, and December 31, 2022. For metropolitan-dwelling individuals, most first doses were administered at pharmacies (53%), with similar trends observed for micropolitan (49%) and noncore (42%) individuals. The second most common location for metropolitan individuals was medical practices (17%); public health clinics and departments were the second most common provider for micropolitan (26%) and noncore (33%) individuals. These trends shifted over time. In December 2020, hospitals were the most common vaccine provider for everyone, but by December 2022, medical providers were the most common source for metropolitan individuals, and pharmacies were most common for micropolitan and noncore individuals. CONCLUSIONS: Trends in the type of vaccine provider differentiated metropolitan residents from micropolitan and noncore residents. For the latter groups, local public health departments played a more significant role. Across all groups, pharmacists emerged as a critical vaccine provider. Our findings can be used to plan for seasonal vaccine campaigns as well as potential future mass vaccination campaigns.

CoWIN: The Future of Universal Immunization Program in India.

CoWIN (Covid Vaccine Intelligence Network) is a web-based platform owned and operated by the Government of India for facilitating COVID-19 vaccination of the citizens. This project is considered a tremendous success, apparent from the sheer number of hassle-free and successful COVID-19 vaccinations. We believe that the CoWIN system is an ideal platform for capturing all the vaccine-related data in the Universal Immunization program (UIP) as well as providing an all-inclusive vaccine certificate in digital format. Advantages and challenges of upscaling the CoWIN system to include UIP and a vision statement of how this can be achieved are discussed in this article. We also enlist the state-level solutions currently in use for digital immunization records, which can serve as case studies for upscaling the CoWIN system.

Methodological changes implemented over time to support accurate and timely COVID-19 vaccine coverage estimates: Ontario, Canada.

The COVID-19 vaccination program implementation in Ontario, Canada has spanned multiple years and is ongoing. To meet the challenges of the program, Ontario developed and implemented a new electronic COVID-19 immunization registry, COVaxON, which captures individual-level data on all doses administered in the province enabling comprehensive coverage assessment. However, the need for ongoing COVID-19 vaccine coverage assessments over a multi-year vaccination program posed challenges necessitating methodological changes. This paper describes Ontario's COVID-19 immunization registry, the methods implemented over time to allow for the ongoing assessment of vaccine coverage by age, and the impact of those methodological changes. Throughout the course of the vaccination program, four different methodological approaches were used to calculate age-specific coverage estimates using vaccination data (numerator) obtained from COVaxON. Age-specific numerators were initially calculated using age at time of first dose (method A), but were updated to the age at coverage assessment (method B). Database enhancements allowed for the exclusion of deceased individuals from the numerator (method C). Population data (denominator) was updated to 2022 projections from the 2021 national census following their availability (method D). The impact was most evident in older age groups where vaccine uptake was high. For example, coverage estimates for individuals aged 70-79 years of age for at least one dose decreased from 104.9 % (method B) to 95.0 % (method D). Thus, methodological changes improved estimates such that none exceeded 100 %. Ontario's COVID-19 immunization registry has been transformational for vaccine program surveillance. The implementation of a single registry for COVID-19 vaccines was essential for comprehensive near real-time coverage assessment, and enabled new uses of the data to support additional components of vaccine program surveillance. The province is well positioned to build on what has been achieved as a result of the COVID-19 pandemic and expand the registry to other routine vaccination programs.

Usability and Acceptability of Electronic Immunization Registry Data Entry Workflows From the Health Care Worker Perspective in Siaya, Kenya (Part 3): Pre-Post Study.

BACKGROUND: Digital health tools such as electronic immunization registries (EIRs) have the potential to improve patient care and alleviate the challenges that arise from the use of paper-based clinic records for reporting. To address some of these challenges, the Kenya Ministry of Health and the International Training and Education Center for Health Kenya implemented an EIR system in 161 immunizing clinics in Siaya County between 2018 and 2019. The successful implementation of digital health tools depends on many factors, one of which is alignment between the technology and the context in which it is used. One important aspect of that implementation context is the perceptions of the health care workers (HCWs) using the EIR. OBJECTIVE: This study aimed to evaluate HCWs' perceptions of the usability and acceptability of multiple clinic workflows using the new EIR. METHODS: We performed a mixed methods pre-post study using semistructured interviews with HCWs at 6 facilities in Siaya County, Kenya. We interviewed HCWs at each facility 4 times: at baseline and once after the implementation of 3 different workflow modifications (n=24 interviews). The baseline state was dual data entry with paper records and the EIR. We then implemented 3 workflow modifications for 1 full day each: fully paperless data entry, preparation of an appointment diary before patient visits for the day, and a combination of the 2 workflows. We compared ratings and themes across interviews after each of the 4 workflows to understand the changes in the usability and acceptability of the EIR. RESULTS: HCWs considered the EIR clinic workflows to be usable and acceptable. Of the modified workflows, HCWs perceived the fully paperless workflow most favorably. In all workflows, HCWs' perceived benefits included ease of clinical decision-making using the EIR, reduced mental burden of data entry when using the EIR, and ease of identification of errors. Perceived barriers to the workflow included contextual challenges such as staffing shortages and lack of network connectivity, EIR platform challenges such as errors in saving records and missing fields, and workflow challenges such as the dual data entry burden of using paper and digital tools simultaneously. CONCLUSIONS: Fully paperless EIR implementation shows great promise from a workflow acceptability standpoint, contingent upon the presence of supporting contextual clinic factors and the resolution of system performance and design challenges. Rather than trying to identify a singular best workflow, future efforts should provide adequate flexibility for HCWs to implement the new system in their unique clinic context. Future EIR implementation stands to benefit from continued monitoring of EIR adoption acceptability during implementation, both for Siaya's program and for other efforts around the globe, as digital health interventions become more widely used.

Time Utilization Among Immunization Clinics Using an Electronic Immunization Registry (Part 2): Time and Motion Study of Modified User Workflows.

BACKGROUND: Digital health interventions have the potential to improve the provision of health care services through digitized data collection and management. Low- and middle-income countries are beginning to introduce electronic immunization registries (EIRs) into their routine immunization services to better capture and store childhood vaccination information. Especially in Africa, where 25% of children remain unimmunized or underimmunized, technologies that can help identify children due for a vaccination are particularly important for improving vaccination coverage. However, an improved understanding of the effectiveness of these systems is needed to develop and deploy sustainable EIRs in low- and middle-income countries. OBJECTIVE: We conducted an interventional pretest-posttest design study that sought to improve time efficiency through workflow modifications in Kenyan immunization clinics. Our aim was to describe how activity times differed after introducing workflow modifications that could potentially reduce the time needed to perform routine data entry activities. Our intent was to demonstrate changes in efficiency when moving from the existing dual-data entry workflow to a future paperless workflow by health facility size and experience length of health care workers (HCWs). METHODS: We tested how 3 workflow modifications would affect time utilization among HCWs using the EIR at the point of care compared with baseline immunization clinic workflows. Our outcome of interest was the time taken to complete individual activities and a patient's total time in the clinic where we compared the time spent during the baseline workflow with that during the modified workflow. We used a standardized tool to observe and document the immunization clinic workflow. To estimate differences in time utilization, we used bivariate analyses and fit multivariate linear mixed-effects models. RESULTS: Our study found that for HCWs using an EIR, the introduction of modified workflows decreased the amount of time needed to provide services to children seen in the immunization clinic. With a baseline mean time of 10 minutes spent per child, this decreased by about 3 minutes when the preparation modification was introduced and almost 5 minutes for the paperless and combined modifications. Results pertaining to the EIR's performance and ability to connect to the internet were particularly insightful about potential causes of delays. CONCLUSIONS: We were able to conduct a concise clinical simulation exercise by introducing modified workflows and estimating their impact on time utilization in immunization clinics using an EIR. We found that the paperless workflow provided the largest time savings when delivering services, although this was threatened by poor EIR performance and internet connectivity. This study demonstrated that not only should digital health interventions be built and adapted for particular use cases but existing user workflows also need to adapt to new technology.

Integration of a Digital Health Intervention Into Immunization Clinic Workflows in Kenya: Qualitative, Realist Evaluation of Technology Usability.

BACKGROUND: In an effort to increase vaccination coverage in low-resource settings, digital tools have been introduced to better track immunization records, improve data management practices, and provide improved access to vaccination coverage data for decision-making. Despite the potential of these electronic systems to improve the provision of health services, few digital health interventions have been institutionalized at scale in low- and middle-income countries. OBJECTIVE: In this paper, we aimed to describe how health care workers in Kenya had integrated an electronic immunization registry into their immunization clinic workflows and to use these findings to inform the development of a refined program theory on the registry's usability. METHODS: Informed by realist methodology, we developed a program theory to explain usability of the electronic immunization registry. We designed a qualitative study based on our theory to describe the barriers and facilitators influencing data entry and use. Qualitative data were collected through semistructured interviews with users and workflow observations of immunization clinic sessions. Our findings were summarized by context-mechanism-outcome relationships formed after analyzing our key themes across interviews and workflow observations. Using these relationships, we were able to identify common rules for future implementers. RESULTS: Across the 12 facilities included in our study, 19 health care workers were interviewed, and 58 workflow sessions were observed. The common rules developed from our qualitative findings are as follows: rule 1-ensure that the users complete training to build familiarity with the system, understand the value of the system and data, and know where to find support; rule 2-confirm that the system captures all data needed for users to provide routine health care services and is easy to navigate; rule 3-identify work-arounds for poor network, system performance, and too few staff or resources; and rule 4-make users aware of expected changes to their workflow, and how these changes might differ over time and by facility size or number of patients. Upon study completion, we revised the program theory to reflect the importance of the goals and workflows of electronic immunization registries aligning with reality. CONCLUSIONS: We created a deeper understanding of the underlying mechanisms for usability of the registry. We found that the electronic immunization registry had high acceptability among users; however, there were numerous barriers to using the system, even under ideal conditions, causing a misalignment between the system and the reality of the users' workflows and their environment. Human-centered design and human-factors methods can assist during pilot stages to better align systems with users' needs and again after scale-up to ensure that interventions are suitable for all user settings.

Introduction and implementation of an immunization information system in the Indonesian province of Daerah Istimewa Yogyakarta: lessons for scaling-up.

BACKGROUND: Immunization is critical to saving children from infections. To increase vaccination coverage, valid and real-time data are needed. Accordingly, it is essential to have a good report system that serves as defaulter tracking to prevent children's immunization failure. The Daerah Istimewa Yogyakarta (DIY) Health Office introduced an electronic immunization registry and successfully implemented it for more than five years. It is the only individual-based record system in Indonesia that has been sustainably operated for a long time. Yet, no systematic assessment of this system has been conducted to date. This study examines the Sistem Informasi Imunisasi Terpadu (SIMUNDU) introduction and implementation process with a view to extracting lessons that could inform scalability and sustainability across the country. METHODS: This study used an explanatory sequential mixed-method design, which collected quantitative data from 142 participants and qualitative data from nine participants. The data entry clerk at a health facility was systematically selected to participate in the survey, while in the key informant interview, the informant was selected based on the survey result. A descriptive and thematic approach was adopted to analyze the quantitative and qualitative data. Results from across the two approaches were integrated for comparison and contrast. RESULTS: Findings are presented according to three core themes that emerged from the data: system strengths, potential threats, weakness and opportunities for scaling-up. Strengths, i.e., factors contributing to the success of SIMUNDU, include management, system performance, people's behavior, and resources. Potential threats to sustaining the system include individual capacity, technical or system issues, and high workload. Opportunities - i.e., a promising factor that influences the SIMUNDU ability to operate sustainably - such as continuity, expectation, and the possibility of scaling up. CONCLUSIONS: SIMUNDU is a promising innovation for Indonesia, beyond DIY. There is agreement about the potential for scaling up this IIS to other provinces. The experience of implementing this system in DIY over the past five years has shown that the benefits outweigh the challenges, and SIMUNDU has grown into a robust yet user-friendly system.

Determinants to immunization information system implementation in independent community pharmacies.

BACKGROUND: Immunization information systems (IISs) facilitate consolidated vaccination data within each state. Many have limited pharmacy participation, especially Alabama. As pharmacists increasingly engage in delivery of vaccines recommended across the life span, it is critical to understand the barriers to IIS implementation, particularly in rural community pharmacies where access to primary care may be limited. OBJECTIVE: The purpose of this study was to identify barriers and facilitators to IIS implementation in rural, independent community pharmacies in Alabama. METHODS: Qualitative interviews with rural, independent community pharmacy personnel and IIS experts in both states with high IIS participation and Alabama (state with low IIS participation) were conducted. States with high participation were identified as those with ≥75% of adults recorded in their respective state IIS. Less than 25% of Alabama adults were recorded in the state IIS at the time of this study. Deductive coding using CFIR constructs was conducted with a second coder to ensure inter-rater reliability. CFIR Rating Rules were applied to the coded data to allow for identification of constructs that have the greatest impact on implementation. RESULTS: A total of twenty-five participants (16 pharmacy personnel; 9 IIS experts) were interviewed. During interviews, 32 of 39 CFIR constructs were mentioned and 11 constructs with a strong influence (+2 or -2) were identified. These included, "adaptability", "complexity", "compatibility", "available resources", "access to knowledge and information", "needs and resources of those served by the organization", "peer pressure", "external policy and initiatives", "knowledge and beliefs about the innovation", "engaging key stakeholders", and "engaging innovation participants". CONCLUSIONS: This qualitative study explored perceived barriers and facilitators to IIS implementation in the rural, independent community pharmacy setting from the perspectives of pharmacy personnel and IIS experts. Factors identified can be used to inform the development of resources and implementation strategies to improve IIS uptake and participation.

Factors associated with human papillomavirus and meningococcal vaccination among adolescents living in rural and urban areas.

Background: Studies have shown that adolescent vaccination rates with human papillomavirus (HPV) and quadrivalent meningococcal conjugate (MenACWY) vaccines are lower in rural areas of the U.S. than in urban areas. We sought to determine factors associated with vaccine acceptance in these two settings. Methods: We conducted a cross-sectional survey of 536 parents or guardians of teens age 13 through 15 years in select rural and urban counties of Minnesota and Wisconsin. We collected information on demographic variables, receipt of adolescent vaccines, and attitudes toward HPV vaccine in particular. Multivariable logistic regression models were used to assess associations between covariates and outcomes of interest (HPV vaccine receipt and MenACWY receipt). Results: Of the 536 respondents, 267 (50%) resided in a rural county. Most respondents were female (78%) and non-Hispanic White (88%). About half (52%) of teens of the surveyed parents received the three vaccines recommended specifically for adolescents: 90% received tetanus-diphtheria-acellular pertussis (Tdap), 84% received MenACWY, and 60% received one or more doses of HPV vaccine. Rural and urban parents surveyed differed on several covariates relating to teen's health services, parent's demographics, and household characteristics. Parent's perception of the importance that their healthcare providers placed on vaccination with HPV and MenACWY were independently associated with receipt of each of those vaccines (odds ratio [OR] 6.37, 95% confidence interval [CI] 2.90-13.96 and OR 2.15, 95% CI 1.07-4.31, respectively). Parents of vaccinated teens were less likely to report concerns about potential harm from the HPV vaccine or having heard stories about health problems caused by the HPV vaccine. Conclusion: Teen receipt of HPV vaccine and MenACWY appears to be influenced by parents' perception of vaccine importance, provider recommendations, and concerns regarding potential harm from the HPV vaccine. Continued education of providers and parents of the importance of adolescent vaccinations is warranted.

Added Value of Electronic Immunization Registries in Low- and Middle-Income Countries: Observational Case Study in Tanzania.

BACKGROUND: There is growing interest and investment in electronic immunization registries (EIRs) in low- and middle-income countries. EIRs provide ready access to patient- and aggregate-level service delivery data that can be used to improve patient care, identify spatiotemporal trends in vaccination coverage and dropout, inform resource allocation and program operations, and target quality improvement measures. The Government of Tanzania introduced the Tanzania Immunization Registry (TImR) in 2017, and the system has since been rolled out in 3736 facilities in 15 regions. OBJECTIVE: The aims of this study are to conceptualize the additional ways in which EIRs can add value to immunization programs (beyond measuring vaccine coverage) and assess the potential value-add using EIR data from Tanzania as a case study. METHODS: This study comprised 2 sequential phases. First, a comprehensive list of ways EIRs can potentially add value to immunization programs was developed through stakeholder interviews. Second, the added value was evaluated using descriptive and regression analyses of TImR data for a prioritized subset of program needs. RESULTS: The analysis areas prioritized through stakeholder interviews were population movement, missed opportunities for vaccination (MOVs), continuum of care, and continuous quality improvement. The included TImR data comprised 958,870 visits for 559,542 patients from 2359 health facilities. Our analyses revealed that few patients sought care outside their assigned facility (44,733/810,568, 5.52% of applicable visits); however, this varied by region; facility urbanicity, type, ownership, patient volume, and duration of TImR system use; density of facilities in the immediate area; and patient age. Analyses further showed that MOVs were highest among children aged <12 months (215,576/831,018, 25.94% of visits included an MOV and were applicable visits); however, there were few significant differences based on other individual or facility characteristics. Nearly half (133,337/294,464, 45.28%) of the children aged 12 to 35 months were fully vaccinated or had received all doses except measles-containing vaccine-1 of the 14-dose under-12-month schedule (ie, through measles-containing vaccine-1), and facility and patient characteristics associated with dropout varied by vaccine. The continuous quality improvement analysis showed that most quality issues (eg, MOVs) were concentrated in <10% of facilities, indicating the potential for EIRs to target quality improvement efforts. CONCLUSIONS: EIRs have the potential to add value to immunization stakeholders at all levels of the health system. Individual-level electronic data can enable new analyses to understand service delivery or care-seeking patterns, potential risk factors for underimmunization, and where challenges occur. However, to achieve this potential, country programs need to leverage and strengthen the capacity to collect, analyze, interpret, and act on the data. As EIRs are introduced and scaled in low- and middle-income countries, implementers and researchers should continue to share real-world examples and build an evidence base for how EIRs can add value to immunization programs, particularly for innovative uses.

The Transition to an Entirely Digital Immunization Registry in Ha Noi Province and Son La Province, Vietnam: Readiness Assessment Study.

BACKGROUND: Vietnam is one of the first low- to middle-income countries to develop and implement a national-scale electronic immunization registry. This system was finalized into the National Immunization Information System (NIIS) and scaled up to a national-level system in 2017. As a result, immunization coverage and the timeliness of vaccinations have drastically improved. The time spent on planning and reporting vaccinations has drastically reduced; as a result, vaccination planning and reporting has become more accurate and effective. However, to date, end users have been tasked with managing both the NIIS and paper-based systems in parallel until a formal assessment of the readiness to fully transition to the NIIS is conducted. OBJECTIVE: This study aims to evaluate the readiness to move to an entirely digital NIIS in 2 provinces of Vietnam-Ha Noi and Son La. METHODS: All health facilities were surveyed to assess their infrastructure, capacity, and need for human resources. NIIS end users were observed and interviewed to evaluate their NIIS knowledge and skill sets. Data from immunization cards and facility paper-based logbooks were compared with data from the NIIS, and vaccine stocks at selected facilities were tallied and compared with data from the NIIS. RESULTS: Of the 990 health facilities evaluated, most used the NIIS to enter and track immunizations (987/990, 99.7%) and vaccine stocks (889/990, 90.8%). Most had stable electricity (971/990, 98.1%), at least 1 computer (986/990, 99.6%), and ≥2 trained NIIS end users (825/990, 83.3%). End users reported that the NIIS supported them in managing and reporting immunization data and saving them time (725/767, 94.5%). Although many end users were able to perform basic skills, almost half struggled with performing more complex tasks. Immunization data were compiled from the NIIS and immunization cards (338/378, 89.4%) and paper-based logbooks (254/269, 94.4%). However, only 54.5% (206/378) of immunization IDs matched, 57% (13/23) of Bacillus Calmette-Guérin vaccination records were accurate, and 70% (21/30) of the facilities had consistent physical vaccine stock balances. The feedback received from NIIS end users suggests that more supportive supervision, frequent refresher training for strengthening their skill sets, and detailed standardized guides for improving data quality are needed. CONCLUSIONS: The readiness to transition to a digital system is promising; however, additional resources are required to address the timeliness, completeness, and accuracy of the data.

Influenza immunization coverage of children with sickle cell disease.

OBJECTIVES: To assess receipt of annual flu immunization among children living with sickle cell disease (SCD). METHODS: Receipt of flu immunization (2014-2019) by SCD status was assessed among all Michigan children <18 years of age using the statewide immunization registry. Logistic regression was used to estimate the odds of annual flu immunization by SCD status and age. RESULTS: Annual flu immunization coverage was higher among children with SCD (46.9%; n = 751) than without (23.2%; n = 2,012,846). The annual adjusted odds of flu immunization for those with SCD were 2.8 (95% CI: 2.5-3.1) times higher than for those without SCD; there were no significant differences by age among children with SCD. Among those without SCD, adolescents aged 13-17 were 2.2 (95% CI: 2.2-2.2) times less likely to receive annual flu immunization than children 6-35 months. CONCLUSIONS: Children with SCD had higher annual flu immunization rates than those without SCD, but >50% remain unimmunized.

Current Challenges and Future Possibilities for Immunization Information Systems.

Immunizations have proven to be an important tool for public health and for reducing the impact of vaccine preventable diseases. To realize the maximum benefits of immunizations, a coordinated effort between public policy, health care providers and health systems is required to increase vaccination coverage and to ensure high-quality data are available to inform clinical and public health interventions. Immunization information systems (IIS) are confidential, population-based, computerized databases that record all immunization doses administered by participating providers to persons residing within a given geopolitical area. The key output of an IIS is high-quality data for use in targeting and monitoring immunization program activities and providing clinical decision support at the point of care. To be truly effective, IISs need to form a nationwide network and repository of immunization data. Since the early 2000s Centers for Disease Control and Prevention has made strides to help IIS move toward a nationwide network through efforts focused on improving infrastructure and functionality, such as the IIS Minimum Functional Standards, and the IIS Annual Report, a self-reported data collection of IIS progress toward achieving the functional standards. While these efforts have helped immunization programs achieve more functional standards, there is a need to shift focus from infrastructure and functionality improvements to high data quality through objective measurement of IIS performance and evaluating critical outcomes. Additionally, realizing the vision of a nationwide repository of high-quality immunization data requires tackling the many challenges that impact data quality and availability including those related to policy, data sharing, data use, aging IIS technology, sustainability, and participation in the IIS. This paper describes the current state of IIS in the United States, critical challenges impacting the quality of data in IIS, and potential components of a future IIS model to address these challenges.

Estimating Population-Level Effects of the Acellular Pertussis Vaccine Using Routinely Collected Immunization Data.

BACKGROUND: Measuring and reporting the different population-level effects of the acellular pertussis vaccine on pertussis disease in addition to direct effects can increase the cost-effectiveness of a vaccine. METHODS: We conducted a retrospective cohort study of children born between 1 January 2008 and 31 December 2017, in King County, Washington, who were enrolled in the Washington State Immunization Information System. Diphtheria, tetanus toxoids, and acellular pertussis (DTaP) vaccination data from in the Washington State Immunization Information System was linked with pertussis case data from Public Health Seattle and King County. Census-level vaccination coverage was estimated as proportion of age-appropriately vaccinated children residing in it. Direct vaccine effectiveness was estimated by comparing pertussis risk in fully vaccinated and undervaccinated children. Population-level vaccine effectiveness (VE) measures were estimated by comparing pertussis risk in census tracts in the highest quartile for vaccination coverage with that in the lowest quartile. RESULTS: For direct protection, estimated VE was 76% (95% confidence interval, 63%-84%) in low-vaccination-coverage clusters, and it decreased to 47% (13%-68%) in high-coverage clusters, after adjustment for potential confounders. The estimated indirect VE was 45.0% (95% confidence interval, 1%-70%), the total VE 93.9% (91%-96%), and the overall VE 42.2% (19%-60%). CONCLUSION: Our findings suggest that DTaP vaccination provided direct as well as indirect protection in the highly immunized King County, Washington. Routine DTaP vaccination programs may have the potential to provide not only protection for vaccinated individuals but also for the undervaccinated individuals living in the same area.

Identifying Opportunities to Strengthen the Public Health Informatics Infrastructure: Exploring Hospitals' Challenges with Data Exchange.

Policy Points Even though most hospitals have the technological ability to exchange data with public health agencies, the majority continue to experience challenges. Most challenges are attributable to the general resources of public health agencies, although workforce limitations, technology issues such as a lack of data standards, and policy uncertainty around reporting requirements also remain prominent issues. Ongoing funding to support the adoption of technology and strengthen the development of the health informatics workforce, combined with revising the promotion of the interoperability scoring approach, will likely help improve the exchange of electronic data between hospitals and public health agencies. CONTEXT: The novel coronavirus 2019 (COVID-19) pandemic has highlighted significant barriers in the exchange of essential information between hospitals and local public health agencies. Thus it remains important to clarify the specific issues that hospitals may face in reporting to public health agencies to inform focused approaches to improve the information exchange for the current pandemic as well as ongoing public health activities and population health management. METHODS: This study uses cross-sectional data of acute-care, nonfederal hospitals from the 2017 American Hospital Association Annual Survey and Information Technology supplement. Guided by the technology-organization-environment framework, we coded the responses to a question regarding the challenges that hospitals face in submitting data to public health agencies by using content analysis according to the type of challenge (i.e., technology, organization, or environment), responsible entity (i.e., hospital, public health agency, vendor, multiple), and the specific issue described. We used multivariable logistic and multinomial regression to identify characteristics of hospitals associated with experiencing the types of challenges. FINDINGS: Our findings show that of the 2,794 hospitals in our analysis, 1,696 (61%) reported experiencing at least one challenge in reporting health data to a public health agency. Organizational issues were the most frequently reported type of challenge, noted by 1,455 hospitals. The most common specific issue, reported by 1,117 hospitals, was the general resources of public health agencies. An advanced EHR system and participation in a health information exchange both decreased the likelihood of not reporting experiencing a challenge and increased the likelihood of reporting an organizational challenge. CONCLUSIONS: Our findings inform policy recommendations such as improving data standards, increasing funding for public health agencies to improve their technological capabilities, offering workforce training programs, and increasing clarity of policy specifications and reporting. These approaches can improve the exchange of information between hospitals and public health agencies.