Using cost-effectiveness analysis to evaluate targeting strategies: the case of vitamin A supplementation.

Given the demonstrated efficacy of vitamin A supplements in reducing childhood mortality, health officials now have to decide whether it would be efficient to target the supplements to high risk children. Decisions about targeting are complex because they depend on a number of factors; the degree of clustering of preventable deaths, the cost of the intervention, the side-effects of the intervention, the cost of identifying the high risk group, and the accuracy of the 'diagnosis' of risk. A cost-effectiveness analysis was used in the Philippines to examine whether vitamin A supplements should be given universally to all children 6-59 months, targeted broadly to children suffering from mild, moderate, or severe malnutrition, or targeted narrowly to pre-schoolers with moderate and severe malnutrition. The first year average cost of the universal approach was US$67.21 per death averted compared to $144.12 and $257.20 for the broad and narrow targeting approaches respectively. When subjected to sensitivity analysis the conclusion about the most cost-effective strategy was robust to changes in underlying assumptions such as the efficacy of supplements, clustering of deaths, and toxicity. Targeting vitamin A supplements to high risk children is not an efficient use of resources. Based on the results of this cost-effectiveness analysis and a consideration of alternate strategies, it is apparent that vitamin A, like immunization, should be provided to all pre-schoolers in the developing world. Issues about targeting public health interventions can usefully be addressed by cost-effectiveness analysis.

PIP: It has been established that vitamin A supplementation can help reduce levels of child mortality. Findings are reported from a cost-effectiveness study in the Philippines undertaken to determine whether vitamin A supplements should be given universally to all children age 6-59 months; targeted broadly to children with mild, moderate, or severe malnutrition; or targeted narrowly to preschoolers with moderate and severe malnutrition. Whether to target supplementation depends upon the degree of clustering of preventable deaths, the cost of the intervention, the side effects of the intervention, the cost of identifying the high risk group, and the accuracy of the diagnosis of risk. The first year average cost of the universal approach would be US$67.21 per death averted, $144.12 for the broad targeting approach, and $257.20 for the narrow approach. Targeting vitamin A supplements to high-risk children is therefore not an efficient use of resources. Vitamin A, like immunization, should be provided to all preschoolers in the developing world.

Improving primary health care through systematic supervision: a controlled field trial.

Most primary health care services in developing countries are delivered by staff working in peripheral facilities where supervision is problematic. This study examined whether systematic supervision using an objective set of indicators could improve health worker performance. A checklist was developed by the Philippine Department of Health which assigned a score from 0 to 3 on each of 20 indicators which were clearly defined. The checklist was implemented in 4 remote provinces with 6 provinces from the same regions serving as a control area. In all 10 provinces, health facilities were randomly selected and surveyed before implementation of the checklist and again 6 months later. Performance, as measured by the combined scores on the 20 indicators, improved 42% (95% Cl = 29% to 55%) in the experimental group compared to 18% (95% Cl = 9% to 27%) in the control group. In the experimental, but not in the control facilities, there was a correlation between frequency of supervision and improvements in scores. The initial cost of implementing the checklist was US $ 19.92 per health facility and the annual recurrent costs were estimated at $ 1.85. Systematic supervision using clearly defined and quantifiable indicators can improve service delivery considerably, at modest cost.

Data utilization and analytical skills among mid-level health programme managers in a developing country.

Mid-level health managers in a developing country were studied to examine the extent to which they are able to use and analyse data they receive from a new health information system. Among 168 managers filling out a self-administered questionnaire, 52% could compute a simple cost-effectiveness ratio and 27% were able to calculate proportions. Only 43% of the managers were able to construct a cumulative graph similar to the one recommended by WHO. Facility with these tasks was not related to levels of training or age. Twelve out of 25 programme managers interviewed in depth did not know how well their particular programme had performed in the previous year. Similarly, only six of these managers knew their best and poorest performing districts. The results of this study suggest; 1) Training managers in data analysis and use is critical if health information systems are to actually improve health care delivery. 2) Data intended for the use of programme managers need to be presented in simple ways. 3) Further social research is required to understand how managers perceive and use data. 4) Efforts to ensure the use of data should not be seen as something to be 'added on' after information systems are in place.

PIP: Mid-level health managers in a developing country were studies to assess to what extent they were able to analyze data received from a new health information system. Among 168 managers filling out a self-administered questionnaire, 52% could compute a simple cost effectiveness ratio, and 27% were able to calculate proportions. Only 11% could compute a weighted average. Among the 10 EPI managers who filled out questionnaires, only 7 constructed a cumulative graph correctly and 5 interpreted it right. Only 43% of the managers were able to construct a cumulative graph similar to the one recommended by the World Health Organization. Constructing and interpreting bar charts was also not done very well; however, a simple pie chart was correctly interpreted by 76% of the respondents. The average score for all managers surveyed was 8.1 correct answers out of a possible 20. Facility with these tasks was not related to levels of training or age. Twelve out of 25 program managers interviewed in-depth did not know how well their particular program had performed in the previous year. Thirteen of the 25 (52%) knew the answer to within +or-5% for the specified indicator, 36% of the managers knew their poorest performing district (or province), and only 24% knew both their best and worst performing districts, 57% of all the managers, including 84% of those who participated in the structured interviews, responded to a closed question that the data they received from the health information system was acceptably accurate. The results suggest that: 1) Training managers in data analysis and use is crucial to actually improving health care delivery. 2) Data intended for program managers needs to be presented in simple ways. 3) Further social research is required to measure how managers scrutinize and use data. 4) Efforts should ensure the use of data when information systems are installed.

Missed opportunities for immunization during visits for curative care: a randomized cross-over trial in Sudan.

Infants who come to health facilities for curative care in developing countries are usually not vaccinated at the same time. To assess what could be done a randomized cross-over study was carried out in twelve urban health centres in Sudan where two approaches were investigated: (1) the place for vaccination was moved very close to the consulting room, and (2) the doctor seeing the infant wrote a prescription recommending vaccination for the child. On average, 55% of the infants needing immunization were vaccinated when either of these approaches was followed. No difference was found between the two interventions in terms of the proportion of eligible children who were immunized (mean difference, 2%; 95% Cl, -4% to +7%). The more sick an infant appeared to be to the mother, the more likely she was to refuse vaccination. Older infants and infants not previously vaccinated were also less likely to be immunized. The number of missed opportunities can thus be reduced using these simple approaches. However, to immunize infants who are sick, unvaccinated, or have limited access to health facilities will require more social mobilization, health education, and outreach activities.

PIP: Infants who come to health facilities for curative care in developing countries are usually not vaccinated at the same time. To assess what could be done, a randomized cross-over study was carried out in 12 urban health centers in Sudan where 2 approaches were investigated: where the site for vaccination was moved very close to the consulting room and when the doctor seeing the infant wrote a prescription recommending vaccination for the child. On the average, 55% of the infants who required immunization were vaccinated when either of these approaches was followed. No difference was found between the 2 intervention approaches in terms of the proportion of eligible children who were immunized (mean difference, 2%; 95% confidence interval, -4%-+7%). The sicker the infant appeared to the mother, the more likely she was to refuse vaccination. Older infants and those not previously vaccinated were also less likely to be immunized. The number of missed opportunities can thus be reduced using these simple approaches. However, to immunize infants who are sick, unvaccinated, or have limited access to health facilities will require more social mobilization, health education, and outreach activities.

Health education interventions in developing countries: a methodological review of published articles.

Some 67 journal articles that described and evaluated health education programmes in developing countries were read by two independent reviewers who examined the methodology used in the studies. Of the articles 47% provided a sufficiently detailed description of the educational intervention to allow replication and 40% described the educational level of the intended audience. Only 21% were controlled studies employing sample sizes greater than 60 individuals or two clusters, although six studies used randomized or quasi-randomized designs. Of the studies 33% looked at changes in health status while another 33% used observable changes in health behaviour as an endpoint. There was good agreement between the reviewers on whether these characteristics were present. Only three of the articles contained all four methodological attributes described above. The results of these articles suggests that successful health education depends on using a few messages, of proven benefit, repeatedly, and in many forums. It is important to improve the methodological quality of health education research. This can be done by using controlled, preferably randomized, designs, ensuring adequate sample sizes, examining only objective changes in behaviour or, better yet, changes in morbidity or mortality. Research reports should describe in detail the educational intervention employed and the target audience.

The changing age structure of diphtheria patients: evidence for the effectiveness of EPI in the Sudan.

During an outbreak of diphtheria in Khartoum, Sudan, in 1988, only 19.1% of patients admitted to hospital were under 5 years of age. This is considerably less than the proportion of such patients seen during a similar outbreak in Khartoum in 1978 (49.5%) and also less than the proportion (55.2%) of under-5-year-olds reported for all inpatients with diphtheria in the Sudan during 1979-86. Cluster surveys carried out between 1981 and 1989 demonstrate that vaccination coverage was much higher for under-5-year-olds (about 65% for the third dose of diphtheria-pertussis-tetanus vaccine (DPT3] than for children of school age (less than 20% for DPT3) at the time of the 1988 outbreak. These results indicate that improved vaccination coverage led to the shift in the age distribution of diphtheria patients seen during the 1988 outbreak. It is unlikely that these data are affected by the type of biases that usually plague disease surveillance systems and can therefore be used as a simple way of assessing the effectiveness of the Expanded Programme on Immunization (EPI).

Missed opportunities for immunization during visits for curative care: practical reasons for their occurrence.

A study of missed opportunities for immunization was conducted in 11 health facilities in Khartoum. Mothers (236) who had brought their infant children to a facility for a reason other than immunizations were interviewed. Of these infants, 58% were missing at least 1 antigen and 29% had never been immunized. The obstacles to being vaccinated included the mothers not having brought their immunization cards, the mothers' fear of having their children immunized when ill, health care workers' ignorance of the recommended ages and dose intervals for vaccines, and the vaccination areas being too far away from and out of sight of patient waiting areas.

The accuracy of mother's reports about their children's vaccination status.

Estimates of measles vaccination coverage in the Sudan vary on average by 23 percentage points, depending on whether or not information supplied by mothers who have lost their children's vaccination cards is included. To determine the accuracy of mother's reports, we collected data during four large coverage surveys in which illiterate mothers with vaccination cards were asked about their children's vaccination status and their answers were compared with the information given on the cards. Mothers' replies were very accurate. For example, for measles vaccination, the data supplied were both sensitive (87%) and specific (79%) compared with those on the vaccination cards. For both DPT and measles vaccination, accurate estimates of the true coverage rates could therefore be obtained by relying solely on mothers' reports. Within +/- 1 month, 78% of the women knew the age at which their children had received their first dose of poliovaccine. Ignoring mothers' reports of their children's vaccination status could therefore result in serious underestimates of the true vaccination coverage. A simple method of dealing with the problem posed by lost vaccination cards during coverage surveys is also suggested.

The accuracy of mothers' reports about their children's vaccination status

Estimates of measles vaccination coverage in the Sudan vary on average by 23 percentage points, depending on whether or not information supplied by mothers who have lost their children's vaccination cards is included. To determine the accuracy of mothers' reports, we collected data during four large coverage surveys in which illiterate mothers with vaccination cards were asked about their children's vaccination status and their answers were compared with the information given on the cards

Mothers' replies were very accurate. For example, for measles vaccination, the data supplied were both sensitive (87 percent) and specific (79 percent) compared with those on the vaccination cards. For both DPT and measles vaccination, accurate estimates of the true coverage rates could therefore be obtained by relying solely on mothers' reports. Within more or less one month, 78 percent of the women knew the age at which their children had received their first dose of poliovaccine

Ignoring mothers' reports of their children's vaccination status could therefore result in serious underestimates of the true vaccination coverage. A simple method of dealing with the problem posed by lost vaccination cards during coverage surveys is also suggested(AU)

Well child clinics and mass vaccination campaigns: an evaluation of strategies for improving the coverage of primary health care in a developing country.

Millions of children in developing countries are dying from diseases that could be prevented or treated by simple interventions. To examine ways to improve the delivery of these basic services, we evaluated well child clinics and mass vaccination campaigns under operational conditions in a rural area of Nicaragua. We found that mass vaccination campaigns using volunteers reached 77.1 per cent of the population under age six. At stationary well child clinics in which villages were invited to a health center and a small food ration was used as an incentive, attendance improved to 94.1 per cent. Similar attendance levels (99.2 per cent) were attained by mobile well child clinics also using a food incentive. Attendance at stationary clinics decreased with the distance of the village from the health center. However, stationary clinics took up only half as much health workers' time as mobile clinics. Our results suggest that stationary clinics employing food as an incentive could be used for villages or neighborhoods close to a health center while mobile clinics offering food should be reserved for more isolated villages.

Improvement in coverage of primary health care in a developing country through use of food incentives.

To improve attendance at mobile clinics for children food incentives were offered to attenders in a rural municipality in northern Nicaragua. Clinic attendance in villages where food incentives were offered was higher than that in control villages (96.5% vs 63.3% of child population, p less than 0.005). When food was later offered in control villages, attendance rose by 60.2% to full attendance (p less than 0.001). Some of the large amounts of non-emergency food aid available could be offered as incentives to increase the use of basic health services in developing countries.