The impact of computer-generated messages on childhood immunization coverage.

INTRODUCTION: Recent evaluations of computer-generated reminder/recall messages have suggested that they are an inexpensive, labor-saving method of improving office visitation rates of childhood immunization providers. This study assesses the sustained impact of computer-generated messages on immunization coverage during the first two years of life. DESIGN: Randomized, controlled trial. SETTING: County health department in the Denver metropolitan area. STUDY PARTICIPANTS: Children (n = 1227) 60 to 90 days of age who had received the first dose of diphtheria-tetanus-pertussis (DTP) and/or poliovirus vaccines. INTERVENTION: Households of children were randomized into four groups to receive: telephone messages followed by letters (Group A); telephone messages alone (Group B); letters only (Group C); or no notification (Group D). Households in the intervention groups (A, B, and C) received up to five computer-generated telephone messages and/or up to four letters each time their children became due for immunization(s). MAIN OUTCOME MEASURE: Immunization series completion at 24 months of age. RESULTS: Children whose families were randomized to receive any of the interventions were 21% more likely to have completed the immunization series by 24 months of age than were children randomized into the control group (49.2% vs 40.9%; RR [rate ratio] = .21; CI [confidence interval] = 1.01, 1.44). While not statistically significant, children in Group A were 23% more likely to complete their immunization series by 24 months of age than those in the control group (50.2% vs 40.9%; RR = 1.23; CI = 1.00, 1.52). No differences were detected among the intervention groups. The costs per additional child completing the series by 24 months of age in Group A was $226 ($79 after start-up costs were discounted). CONCLUSION: Computer-generated contacts, either by phone or by mail (or both combined), used each time vaccines become due, are efficacious in increasing immunization coverage of children under 2 years of age.

Vaccination practices, policies, and management factors associated with high vaccination coverage levels in Georgia public clinics. Georgia Immunization Program Evaluation Team.

BACKGROUND: Controlling vaccine-preventable diseases by achieving high childhood vaccination coverage levels is a national priority. However, there are few, if any, comprehensive evaluations of state immunization programs in the United States, and little attention has been given to the importance of vaccination clinic management style and staff motivation. OBJECTIVE: To evaluate the factors associated with the increase in childhood vaccination coverage levels from 53% in 1988 to 89% in 1994 in Georgia's public health clinics. DESIGN: A 1994 mail survey obtaining information on clinic vaccination policies and practices and management practices. SETTING: All 227 public health clinics in Georgia. PARTICIPANTS: Clinic nurses responsible for vaccination services. OUTCOME MEASURE: The 1994 clinic-specific coverage level for 21- to 23-month-old children for 4 doses of diphtheria and tetanus toxoids and pertussis vaccine, 3 doses of polio vaccine, and 1 dose of a measles-containing vaccine as determined by an independent state assessment of clinic coverage levels. RESULTS: Univariate analysis showed that higher coverage levels were significantly (P<.05) associated with smaller clinic size, higher proportions of clientele enrolled in the Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), being a nonurban clinic, and numerous vaccination practices and policies. Multivariable analysis showed that only 8 of greater than 150 factors remained associated with higher coverage levels, including having no waiting time to be seen, having telephone reminder systems, conducting home visits for defaulters, and restricting WIC vouchers when a child was undervaccinated. Motivational factors related to higher coverage included clinic lead nurses receiving an incentive to raise coverage and lead nurses participating in assessments of clinic coverage levels by state immunization staff. CONCLUSIONS: No single factor is responsible for raising vaccination coverage levels. Efforts to improve coverage should include local assessment to provide feedback on performance and identify appropriate local solutions. Coordinating with WIC, conducting recall and reminder activities, motivating clinic staff, and having staff participate in decisions are important in raising vaccination levels.

The impact of interventions by a community-based organization on inner-city vaccination coverage: Fulton County, Georgia, 1992-1993.

OBJECTIVE: To evaluate the impact of interventions by a community-based organization on immunization rates. DESIGN: Controlled community intervention trial. SETTING AND PARTICIPANTS: Children aged 3 to 59 months in Fulton County, Georgia, who were patients of 1 of 4 public clinics (clinic based), or residents of 1 of 9 inner-city communities (residence based). INTERVENTIONS: (1) Clinic-based intervention included monthly review of clinic vaccination records to identify undervaccinated children followed by contact with family (reminder-recall strategy); (2) residence-based intervention included door-to-door assessment and education campaigns followed by mobile van vaccinations, temporary on-site vaccination stations, free child care and transportation to providers, incentives of food and baby products, focus groups, and coalitions with local organizations (community saturation with vaccination messages and opportunities). OUTCOME MEASURES: Change in vaccination rates after 1 year based on clinic record reviews and population surveys. RESULTS: For clinic-based intervention, series completion rates improved from 43% (87/204) to 58% (99/170) in intervention clinics (P=.003), while rates in control clinics did not change from the baseline of 52% (81/157 to 78/150), for a net difference between intervention and control arms of +15 percentage points (P=.046). For residence-based intervention, age-appropriate vaccination rates improved from 44% (154/347) to 61% (260/429) in intervention communities (+17 percentage points; P<.001) compared with improvement of 44% (78/178) to 58% (129/221) for control communities (+14 percentage points; P=.004), but the difference between arms was not significant (+3 percentage points, P=.78). CONCLUSIONS: Reminder-recall activities by the community-based organization improved vaccination rates in intervention clinics compared with control clinics. A statistically significant impact on vaccination rates could not be detected for residence-based interventions by the community-based organization.

Impact of measurement and feedback on vaccination coverage in public clinics, 1988-1994.

OBJECTIVE: To investigate whether a reported rise in vaccination coverage in Georgia public clinics during the period 1988 through 1994 was artifactual or real and, if real, to determine the extent to which the rise could be associated with a program of measurement and feedback. DESIGN: Examination of data from Georgia public clinics, doses-administered records, and National Health Interview Surveys. SETTING/PARTICIPANTS: Children attending Georgia public clinics. INTERVENTION: Measurement of vaccination coverage and feedback to providers. MAIN OUTCOME MEASURE: Vaccination coverage rates. RESULTS: For the period 1988 through 1994, 136 004 Georgia public clinic vaccination records for children 21 to 23 months of age were reviewed. Median series-completion rates at public clinics rose from 53% to 89%, while indexes of under-vaccination fell: missed opportunities for simultaneous vaccination (6% to 0%), lost contact for more than 12 months (14% to 1%), and first vaccination more than 1 month late (19% to 8%). According to the independent doses-administered database, the proportion of children starting the primary series very late (> or =12 months old) fell from 14% to 6%, and the series-completion index rose from 64% to 83%, suggesting that improvements could not be wholly ascribed to better clinic record keeping. In 1988, vaccination coverage of children 24 months of age in the National Health Interview Survey (NHIS) was 53%, identical to median public clinic coverage in Georgia; in 1993, NHIS coverage was 60%, while median public clinic coverage in Georgia was 90%, suggesting that the rise in coverage in Georgia public clinics exceeded national trends. Patterns within the coverage changes suggest an association with the process of measurement and feedback. CONCLUSIONS: A marked increase in vaccination coverage occurred in Georgia public clinics associated with a program of annual measurement and feedback.

Information as intervention: how Georgia used vaccination coverage data to double public sector vaccination coverage in seven years.

All states are now required by federal law to measure immunization coverage in each public clinic in their jurisdiction once a year. This law is based on data suggesting a twofold increase of immunization coverage in public clinics in Georgia during a seven-year period when the state developed a system for measuring clinic coverage and using these data to stimulate immunization performance. Review of the history of the development of the Georgia system suggests that measurement alone is not sufficient to raise coverage, however. In Georgia, measurement was coupled with a vigorous program of feedback of coverage data, provision of incentives for good performance, and exchange of information among clinics. The Centers for Disease Control and Prevention (CDC) has summarized the Georgia system with the acronym AFIX--Assessment, Feedback, Incentives, eXchange of information--and recommends that all state immunization program managers test and adapt this methodology. The article comments on the development of the Georgia system and describes why CDC believes other states should adopt it.

Effectiveness of computer-generated telephone messages in increasing clinic visits.

OBJECTIVE: To evaluate the effectiveness of computer-generated telephone reminder calls in increasing kept appointment rates in a public health setting. DESIGN: Randomized controlled trial. SETTING: Public health clinic, Georgia. PATIENTS: Five hundred seventeen clients with scheduled appointments during a 4-week period at immunization, women, infant, and children; well-child; or family-planning programs. INTERVENTION: A single computer-generated telephone reminder 1 day before each client's scheduled appointment. MAIN OUTCOME MEASURE: Rates of kept appointments. RESULTS: Of the 277 clients assigned to receive the intervention, 144 (52%) kept their appointments, compared with only 78 (32.5%) of 240 who were not assigned to receive a message (P < .05). Improvement in kept appointment rates associated with receiving the message was highest for the immunization-program (183% increase, P < .05), with increases of 64%, 53%, and 44% for the well-child; women, infant, and children; and family-planning programs, respectively. CONCLUSIONS: These results suggest a simple and effective method to increase kept appointment rates in a variety of public health programs.

A randomized trial of the effectiveness of computer-generated telephone messages in increasing immunization visits among preschool children.

OBJECTIVE: To assess the effectiveness of computer-generated telephone reminder and recall messages in increasing preschool immunization visits. DESIGN: Randomized, controlled trial. SETTING: Fourteen counties in urban and rural Georgia. STUDY PARTICIPANTS: Children (N = 8002) who were younger than 2 years; had telephone numbers listed in preexisting computerized immunization databases; and were due or late for immunization(s) during the 4-month enrollment period. INTERVENTION: Households of children were randomized to receive or not receive a general or vaccine-specific computer-generated telephone reminder or recall message the day before the child was due, or immediately after randomization if the child was late. MAIN OUTCOME MEASURE: The rates of immunization visits during the 30-day follow-up period. RESULTS: Of the 4636 children whose households were randomized to receive a message, 1684 (36.3%) visited the health department within 30 days compared with 955 (28.4%) of the 3366 children whose households were not contacted (risk ratio [RR] = 1.28; 95% confidence interval [CI] = 1.20 to 1.37; P < .01). Immunization visits were more frequent (41.1%) among the 3257 children whose households actually received the message (RR = 1.45; 95% CI = 1.36 to 1.56; P < .01). Improvement in immunization visits was similar for general and specific messages, greater for recall than reminder messages, and greatest for children who were late for the third dose of the diphtheria-tetanus-pertussis vaccine and the measles-mumps-rubella vaccine. CONCLUSION: These data suggest a simple and effective way to increase preschool immunization visits, particularly for vaccines associated with the lowest immunization rates.

Evaluation of telephoned computer-generated reminders to improve immunization coverage at inner-city clinics.

The authors evaluated the effectiveness of computer-generated telephoned reminders used to raise the rates of on-time immunization among preschool-age children in two public clinics in Atlanta, GA. The overall effect of the intervention on immunization levels appeared to be minimal (crude relative risk = 1.07, 95 percent confidence interval = 0.78, 1.46), in part because only about 80 percent of children in both the randomly selected intervention group and in the control group were members of a household with a telephone number listed in clinic records. However, logistic regression analysis indicated that 36 of 68 children (52.9 percent) in the intervention group whose households were reached were vaccinated within 30 days of their due dates, compared to 31 of 75 children (41.3 percent) in the control group whose household telephone numbers were recorded but not called (adjusted odds ratio = 2.12, 95 percent confidence interval = 1.01, 4.46). This analysis indicates that telephoned reminders demonstrated a level of effectiveness in improving immunization levels at inner-city clinics that recommends further trial and study.

Monitoring progress toward US preschool immunization goals.

The United States has achieved over 97% immunization of children by school age and has reduced the incidence of vaccine-preventable diseases by more than 90% since the prevaccination era. However, children often do not receive immunizations at the recommended age, and in densely populated urban areas this delay in immunization has led to epidemics of measles. Correctable deficiencies of the immunization delivery system have been identified in these areas. To respond to needs, the public health infrastructure must be strengthened, and active participation from the private sector must be obtained, both in delivery of immunizations and in assessment of performance. Appropriate action must be stimulated by the provision of timely information on immunization coverage and on indicators of program performance at the local level.

A million dollar measles outbreak: epidemiology, risk factors, and a selective revaccination strategy.

Between February 8 and April 4, 1986, an outbreak of measles occurred in the State of Arkansas. A total of 489 suspected measles cases were reported from 53 counties; 86 schools statewide reported suspected measles cases. There were 284 cases confirmed in 18 counties; 23.6 percent among students in one university and 41.2 percent among students in kindergarten through 12th grade in 32 schools. An epidemiologic investigation was carried out to evaluate risk factors for vaccine failure and to assess the effectiveness of a selective revaccination strategy in the outbreak setting. A cohort study conducted at a junior high school showed that, compared with students vaccinated against measles at ages 15 months or older, those vaccinated at ages 12-14 months had a three-fold increased risk of measles (relative risk 3.2, 95 percent confidence interval 1.5, 6.9). For schools reporting measles, the Arkansas Department of Health and the Department of Education jointly required reimmunization of students vaccinated at ages younger than 15 months and the exclusion of students not vaccinated at ages 15 months or older until they were vaccinated or until 2 weeks after the last rash onset. To implement these recommendations, more than 100,000 doses of combined measles-mumps-rubella vaccine were distributed at a cost greater than $1 million.