Evolution of the monitoring and evaluation strategies to support the World Health Organization's Global Programme to Eliminate Lymphatic Filariasis.

The Global Programme to Eliminate Lymphatic Filariasis (GPELF) was established with the ambitious goal of eliminating LF as a public health problem. The remarkable success of the GPELF over the past 2 decades in carrying out its principal strategy of scaling up and scaling down mass drug administration has relied first on the development of a rigorous monitoring and evaluation (M&E) framework and then the willingness of the World Health Organization and its community of partners to modify this framework in response to the practical experiences of national programmes. This flexibility was facilitated by the strong partnership that developed among researchers, LF programme managers and donors willing to support the necessary research agenda. This brief review summarizes the historical evolution of the GPELF M&E strategies and highlights current research needed to achieve the elimination goal.

A Multicountry Comparison of Three Coverage Evaluation Survey Sampling Methodologies for Neglected Tropical Diseases.

Coverage evaluation surveys (CESs) are an important complement to routinely reported drug coverage estimates following mass drug administration for neglected tropical diseases (NTDs). Although the WHO recommends the routine use of CESs, they are rarely implemented. Reasons for this low uptake are multifaceted; one is uncertainty on the best sampling method. We conducted a multicountry study to compare the statistical characteristics, cost, time, and complexity of three commonly used CES sampling methods: the Expanded Program on Immunization's (EPI's) 30 × 7 cluster survey, a stratified design with systematic sampling within strata to enable lot quality assurance sampling (S-LQAS) decision rules, and probability sampling with segmentation (PSS). The three CES methods were used in Burkina Faso, Honduras, Malawi, and Uganda, and results were compared across the country sites. All three CES methods were found to be feasible. The S-LQAS approach took the least amount of time to complete and, consequently, was the least expensive; however, all three methods cost less than $5,000 per district. The PSS design resulted in an unbiased, equal-probability sample of the target populations. By contrast, the EPI approach had inherent bias related to the selection of households. Because of modifications needed to maintain feasibility, the S-LQAS method also resulted in a non-probability sample with less precision than the other two methods. Given the comparable cost and time of the three sampling methods and the statistical advantages of the PSS method, the PSS method was deemed to be the best for CESs in NTD programs.

Soil-transmitted helminthiasis in four districts in Bangladesh: household cluster surveys of prevalence and intervention status.

BACKGROUND: In 2016, after 8 years of twice-annual nationwide preventive chemotherapy (PC) administration to school-age children (SAC), the Bangladesh Ministry of Health & Family Welfare (MOHFW) sought improved impact and intervention monitoring data to assess progress toward the newly adopted goal of eliminating soil-transmitted helminthiasis (STH) as a public health problem. METHODS: We surveyed four Bangladeshi districts between August and October 2017. We conducted a multi-stage, cluster-sample, household survey which produced equal-probability samples for preschool-age children (PSAC; 1-4 years), SAC (5-14 years), and adults (≥ 15 years). Standardized questionnaires were administered, using Android-based smart phones running an Open Data Kit application. Stool samples were collected and testing for STH prevalence and infection intensity used the Kato-Katz technique. RESULTS: In all, 4318 stool samples were collected from 7164 participants. Estimates of STH prevalence by risk group in three of the four surveyed districts ranged from 3.4 to 5.0%, all with upper, 1-sided 95% confidence limits < 10%. However, STH prevalence estimates in Sirajganj District ranged from 23.4 to 29.1%. Infections in that district were spatially focal; four of the 30 survey clusters had > 50% prevalence in at least one risk group. Among all tested specimens, Ascaris lumbricoides was the most common STH parasite [8.2% (n = 352)], followed by Trichuris trichiura [0.9% (n = 37)], and hookworm [0.6% (n = 27)]. In each district, PC coverage among SAC was above the 75% program target but did not exceed 45% among PSAC in any district. Improved sanitation at home, school, or work was over 90% in all districts. CONCLUSIONS: In the three low-prevalence districts, the MOHFW is considering decreasing the frequency of mass drug administration, per World Health Organization (WHO) guidelines. Also, the MOHFW will focus programmatic resources and supervisory efforts on Sirajganj District. Despite considering WHO guidance, the MOHFW will not expand PC administration to women of reproductive age partly due to the low prevalence of hookworm and T. trichiura, the STH parasites that contribute most to morbidity in that risk group. Data collected from surveys such as ours would help effectively guide future STH control efforts in Bangladesh and elsewhere.

The rationale and cost-effectiveness of a confirmatory mapping tool for lymphatic filariasis: Examples from Ethiopia and Tanzania.

Endemicity mapping is required to determining whether a district requires mass drug administration (MDA). Current guidelines for mapping LF require that two sites be selected per district and within each site a convenience sample of 100 adults be tested for antigenemia or microfilaremia. One or more confirmed positive tests in either site is interpreted as an indicator of potential transmission, prompting MDA at the district-level. While this mapping strategy has worked well in high-prevalence settings, imperfect diagnostics and the transmission potential of a single positive adult have raised concerns about the strategy's use in low-prevalence settings. In response to these limitations, a statistically rigorous confirmatory mapping strategy was designed as a complement to the current strategy when LF endemicity is uncertain. Under the new strategy, schools are selected by either systematic or cluster sampling, depending on population size, and within each selected school, children 9-14 years are sampled systematically. All selected children are tested and the number of positive results is compared against a critical value to determine, with known probabilities of error, whether the average prevalence of LF infection is likely below a threshold of 2%. This confirmatory mapping strategy was applied to 45 districts in Ethiopia and 10 in Tanzania, where initial mapping results were considered uncertain. In 42 Ethiopian districts, and all 10 of the Tanzanian districts, the number of antigenemic children was below the critical cutoff, suggesting that these districts do not require MDA. Only three Ethiopian districts exceeded the critical cutoff of positive results. Whereas the current World Health Organization guidelines would have recommended MDA in all 55 districts, the present results suggest that only three of these districts requires MDA. By avoiding unnecessary MDA in 52 districts, the confirmatory mapping strategy is estimated to have saved a total of $9,293,219.

Tetanus Immunity among Women Aged 15 to 39 Years in Cambodia: a National Population-Based Serosurvey, 2012.

To monitor progress toward maternal and neonatal tetanus elimination (MNTE) in Cambodia, we conducted a nationwide serosurvey of tetanus immunity in 2012. Multistage cluster sampling was used to select 2,154 women aged 15 to 39 years. Tetanus toxoid antibodies in serum samples were measured by gold-standard double-antigen enzyme-linked immunosorbent assay (DAE) and a novel multiplex bead assay (MBA). Antibody concentrations of ≥0.01 IU/ml by DAE or the equivalent for MBA were considered seroprotective. Estimated tetanus seroprotection was 88% (95% confidence interval [CI], 86 to 89%); 64% (95% CI, 61 to 67%) of women had antibody levels of ≥1.0 IU/ml. Seroprotection was significantly lower (P < 0.001) among women aged 15 to 19 years (63%) and 20 to 24 years (87%) than among those aged ≥25 years (96%), among nulliparous women than among parous women (71 versus 97%), and among those living in the western region than among those living in other regions (82 versus 89%). The MBA showed high sensitivity (99% [95% CI, 98 to 99%]) and specificity (92% [95% CI, 88 to 95%]) compared with DAE. Findings were compatible with MNTE in Cambodia (≥80% protection). Tetanus immunity gaps should be addressed through strengthened routine immunization and targeted vaccination campaigns. Incorporating tetanus testing in national serosurveys using MBAs, which can measure immunity to multiple pathogens simultaneously, may be beneficial for monitoring MNTE.

Accuracy of Coverage Survey Recall following an Integrated Mass Drug Administration for Lymphatic Filariasis, Schistosomiasis, and Soil-Transmitted Helminthiasis.

BACKGROUND: Achieving target coverage levels for mass drug administration (MDA) is essential to elimination and control efforts for several neglected tropical diseases (NTD). To ensure program goals are met, coverage reported by drug distributors may be validated through household coverage surveys that rely on respondent recall. This is the first study to assess accuracy in such surveys. METHODOLOGY/PRINCIPAL FINDINGS: Recall accuracy was tested in a series of coverage surveys conducted at 1, 6, and 12 months after an integrated MDA in Togo during which three drugs (albendazole, ivermectin, and praziquantel) were distributed. Drug distribution was observed during the MDA to ensure accurate recording of persons treated during the MDA. Information was obtained for 506, 1131, and 947 persons surveyed at 1, 6, and 12 months, respectively. Coverage (defined as the percentage of persons taking at least one of the MDA medications) within these groups was respectively 88.3%, 87.4%, and 80.0%, according to the treatment registers; it was 87.9%, 91.4% and 89.4%, according to survey responses. Concordance between respondents and registers on swallowing at least one pill was >95% at 1 month and >86% at 12 months; the lower concordance at 12 months was more likely due to difficulty matching survey respondents with the year-old treatment register rather than inaccurate responses. Respondents generally distinguished between pills similar in appearance; concordance for recall of which pills were taken was over 80% in each survey. SIGNIFICANCE: In this population, coverage surveys provided remarkably consistent coverage estimates for up to one year following an integrated MDA. It is not clear if similar consistency will be seen in other settings, however, these data suggest that in some settings coverage surveys might be conducted as much as one year following an MDA without compromising results. This might enable integration of post-MDA coverage measurement into large, multipurpose, periodic surveys, thereby conserving resources.

Predictors of health worker performance after Integrated Management of Childhood Illness training in Benin: a cohort study.

BACKGROUND: Correct treatment of potentially life-threatening illnesses (PLTIs) in children under 5 years, such as malaria, pneumonia, and diarrhea, can substantially reduce mortality. The Integrated Management of Childhood Illness (IMCI) strategy has been shown to improve treatment of child illnesses, but multiple studies have shown that gaps in health worker performance remain after training. To better understand factors related to health worker performance, we analyzed 9,330 patient consultations in Benin from 2001-2002, after training one of the first cohorts of 32 health workers in IMCI. METHODS: With data abstracted from patient registers specially designed for IMCI-trained health workers, we examined associations between health facility-, health worker-, and patient-level factors and 10 case-management outcomes for PLTIs. RESULTS: Altogether, 63.6% of children received treatment for all their PLTIs in accordance with IMCI guidelines, and 77.8% received life-saving treatment (i.e., clinically effective treatment, even if not exactly in accordance with IMCI guidelines). Performance of individual health workers varied greatly, from 15-88% of patients treated correctly, on average. Multivariate regression analyses identified several factors that might have influenced case-management quality, many outside a manager's direct control. Younger health workers significantly outperformed older ones, and infants received better care than older children. Children with danger signs, those with more complex illnesses, and those with anemia received worse care. Health worker supervision was associated with improved performance for some outcomes. CONCLUSIONS: A variety of factors, some outside the direct control of program managers, can influence health worker practices. An understanding of these influences can help inform the development of strategies to improve performance.

Unprogrammed deworming in the Kibera slum, Nairobi: implications for control of soil-transmitted helminthiases.

BACKGROUND: Programs for control of soil-transmitted helminth (STH) infections are increasingly evaluating national mass drug administration (MDA) interventions. However, "unprogrammed deworming" (receipt of deworming drugs outside of nationally-run STH control programs) occurs frequently. Failure to account for these activities may compromise evaluations of MDA effectiveness. METHODS: We used a cross-sectional study design to evaluate STH infection and unprogrammed deworming among infants (aged 6-11 months), preschool-aged children (PSAC, aged 1-4 years), and school-aged children (SAC, aged 5-14 years) in Kibera, Kenya, an informal settlement not currently receiving nationally-run MDA for STH. STH infection was assessed by triplicate Kato-Katz. We asked heads of households with randomly-selected children about past-year receipt and source(s) of deworming drugs. Local non-governmental organizations (NGOs) and school staff participating in school-based deworming were interviewed to collect information on drug coverage. RESULTS: Of 679 children (18 infants, 184 PSAC, and 477 SAC) evaluated, 377 (55%) reported receiving at least one unprogrammed deworming treatment during the past year. PSAC primarily received treatments from chemists (48.3%) or healthcare centers (37.7%); SAC most commonly received treatments at school (55.0%). Four NGOs reported past-year deworming activities at 47 of >150 schools attended by children in our study area. Past-year deworming was negatively associated with any-STH infection (34.8% vs 45.4%, p = 0.005). SAC whose most recent deworming medication was sourced from a chemist were more often infected with Trichuris (38.0%) than those who received their most recent treatment from a health center (17.3%) or school (23.1%) (p = 0.05). CONCLUSION: Unprogrammed deworming was received by more than half of children in our study area, from multiple sources. Both individual-level treatment and unprogrammed preventive chemotherapy may serve an important public health function, particularly in the absence of programmed deworming; however, they may also lead to an overestimation of programmed MDA effectiveness. A standardized, validated tool is needed to assess unprogrammed deworming.

Impact of a community-based lymphedema management program on episodes of Adenolymphangitis (ADLA) and lymphedema progression--Odisha State, India.

BACKGROUND: Lymphedema management programs have been shown to decrease episodes of adenolymphangitis (ADLA), but the impact on lymphedema progression and of program compliance have not been thoroughly explored. Our objectives were to determine the rate of ADLA episodes and lymphedema progression over time for patients enrolled in a community-based lymphedema management program. We explored the association between program compliance and ADLA episodes as well as lymphedema progression. METHODOLOGY/PRINCIPAL FINDINGS: A lymphedema management program was implemented in Odisha State, India from 2007-2010 by the non-governmental organization, Church's Auxiliary for Social Action, in consultation with the Centers for Disease Control and Prevention. A cohort of patients was followed over 24 months. The crude 30-day rate of ADLA episodes decreased from 0.35 episodes per person-month at baseline to 0.23 at 24 months. Over the study period, the percentage of patients who progressed to more severe lymphedema decreased (P-value  = 0.0004), while those whose lymphedema regressed increased over time (P-value<0.0001). Overall compliance to lymphedema management, lagged one time point, appeared to have little to no association with the frequency of ADLA episodes among those without entry lesions (RR = 0.87 (0.69, 1.10)) and was associated with an increased rate (RR = 1.44 (1.11, 1.86)) among those with entry lesions. Lagging compliance two time points, it was associated with a decrease in the rate of ADLA episodes among those with entry lesions (RR = 0.77 (95% CI: 0.59, 0.99)) and was somewhat associated among those without entry lesions (RR = 0.83 (95% CI: 0.64, 1.06)). Compliance to soap was associated with a decreased rate of ADLA episodes among those without inter-digital entry lesions. CONCLUSIONS/SIGNIFICANCE: These results indicate that a community-based lymphedema management program is beneficial for lymphedema patients for both ADLA episodes and lymphedema. It is one of the first studies to demonstrate an association between program compliance and rate of ADLA episodes.

Potential barriers to healthcare in Malawi for under-five children with cough and fever: a national household survey.

Failure to access healthcare is an important contributor to child mortality in many developing countries. In a national household survey in Malawi, we explored demographic and socioeconomic barriers to healthcare for childhood illnesses and assessed the direct and indirect costs of seeking care. Using a cluster-sample design, we selected 2,697 households and interviewed 1,669 caretakers. The main reason for households not being surveyed was the absence of a primary caretaker in the household. Among 2,077 children aged less than five years, 504 episodes of cough and fever during the previous two weeks were reported. A trained healthcare provider was visited for 48.0% of illness episodes. A multivariate regression model showed that children from the poorest households (p = 0.02) and children aged > 12 months (p = 0.02) were less likely to seek care when ill compared to those living in wealthier households and children of higher age-group respectively. Families from rural households spent more time travelling compared to urban households (68.9 vs 14.1 minutes; p < 0.001). In addition, visiting a trained healthcare provider was associated with longer travel time (p < 0.001) and higher direct costs (p < 0.001) compared to visiting an untrained provider. Thus, several barriers to accessing healthcare in Malawi for childhood illnesses exist. Continued efforts to reduce these barriers are needed to narrow the gap in the health and healthcare equity in Malawi.

Transmission assessment surveys (TAS) to define endpoints for lymphatic filariasis mass drug administration: a multicenter evaluation.

BACKGROUND: Lymphatic filariasis (LF) is targeted for global elimination through treatment of entire at-risk populations with repeated annual mass drug administration (MDA). Essential for program success is defining and confirming the appropriate endpoint for MDA when transmission is presumed to have reached a level low enough that it cannot be sustained even in the absence of drug intervention. Guidelines advanced by WHO call for a transmission assessment survey (TAS) to determine if MDA can be stopped within an LF evaluation unit (EU) after at least five effective rounds of annual treatment. To test the value and practicality of these guidelines, a multicenter operational research trial was undertaken in 11 countries covering various geographic and epidemiological settings. METHODOLOGY: The TAS was conducted twice in each EU with TAS-1 and TAS-2 approximately 24 months apart. Lot quality assurance sampling (LQAS) formed the basis of the TAS survey design but specific EU characteristics defined the survey site (school or community), eligible population (6-7 year olds or 1(st)-2(nd) graders), survey type (systematic or cluster-sampling), target sample size, and critical cutoff (a statistically powered threshold below which transmission is expected to be no longer sustainable). The primary diagnostic tools were the immunochromatographic (ICT) test for W. bancrofti EUs and the BmR1 test (Brugia Rapid or PanLF) for Brugia spp. EUs. PRINCIPAL FINDINGS/CONCLUSIONS: In 10 of 11 EUs, the number of TAS-1 positive cases was below the critical cutoff, indicating that MDA could be stopped. The same results were found in the follow-up TAS-2, therefore, confirming the previous decision outcome. Sample sizes were highly sex and age-representative and closely matched the target value after factoring in estimates of non-participation. The TAS was determined to be a practical and effective evaluation tool for stopping MDA although its validity for longer-term post-MDA surveillance requires further investigation.

Evaluating health worker performance in Benin using the simulated client method with real children.

BACKGROUND: The simulated client (SC) method for evaluating health worker performance utilizes surveyors who pose as patients to make surreptitious observations during consultations. Compared to conspicuous observation (CO) by surveyors, which is commonly done in developing countries, SC data better reflect usual health worker practices. This information is important because CO can cause performance to be better than usual. Despite this advantage of SCs, the method's full potential has not been realized for evaluating performance for pediatric illnesses because real children have not been utilized as SCs. Previous SC studies used scenarios of ill children that were not actually brought to health workers. During a trial that evaluated a quality improvement intervention in Benin (the Integrated Management of Childhood Illness [IMCI] strategy), we conducted an SC survey with adult caretakers as surveyors and real children to evaluate the feasibility of this approach and used the results to assess the validity of CO. METHODS: We conducted an SC survey and a CO survey (one right after the other) of health workers in the same 55 health facilities. A detailed description of the SC survey process was produced. Results of the two surveys were compared for 27 performance indicators using logistic regression modeling. RESULTS: SC and CO surveyors observed 54 and 185 consultations, respectively. No serious problems occurred during the SC survey. Performance levels measured by CO were moderately higher than those measured by SCs (median CO - SC difference = 16.4 percentage-points). Survey differences were sometimes much greater for IMCI-trained health workers (median difference = 29.7 percentage-points) than for workers without IMCI training (median difference = 3.1 percentage-points). CONCLUSION: SC surveys can be done safely with real children if appropriate precautions are taken. CO can introduce moderately large positive biases, and these biases might be greater for health workers exposed to quality improvement interventions. TRIAL NUMBER: http://clinicaltrials.gov Identifier NCT00510679.

Trends in health worker performance after implementing the Integrated Management of Childhood Illness strategy in Benin.

OBJECTIVE: Training health workers to use Integrated Management of Childhood Illness (IMCI) guidelines can improve care for ill children in outpatient settings in developing countries. However, even after IMCI training, important performance gaps exist. One potential reason is that the effect of training can rapidly wane. Our aim was to determine if the performance of IMCI-trained health workers deteriorated over 3 years. METHODS: We studied two departments in Benin. First, we performed a record review of 32 IMCI-trained health workers during the first year of IMCI implementation (2001-2002). Second, we analysed data from cross-sectional health facility surveys from 2001 to 2004 that represented the entire study area. Primary outcomes were the proportion of children under 5 years old with potentially life-threatening illnesses who received either recommended or adequate treatment, and among all children, an index of overall guideline adherence. Secondary outcomes reflected the treatment of individual diseases. Outcomes were calculated monthly, and time trends were evaluated with regression modelling. RESULTS: The record review included 9393 consultations, and the surveys included 411 consultations performed by 105 health workers. For both data sources, performance trends were essentially flat for nearly all outcomes. Absolute levels of performance revealed substantial performance gaps. CONCLUSIONS: We found no evidence that performance declined over 3 years after IMCI training. However, important performance gaps found immediately after IMCI training persisted and should be addressed.

Pandemic (H1N1) 2009 and Hajj Pilgrims who received Predeparture Vaccination, Egypt.

In Egypt, vaccination against pandemic (H1N1) 2009 virus was required of pilgrims departing for the 2009 Hajj. A survey of 551 pilgrims as they returned to Egypt found 542 (98.1% [weighted]) reported receiving the vaccine; 6 (1.0% [weighted]) were infected with influenza virus A (H3N2) but none with pandemic (H1N1) 2009 virus.

Impact of a malaria-control project in Benin that included the integrated management of childhood illness strategy.

OBJECTIVES: To estimate the impact of the Integrated Management of Childhood Illness (IMCI) strategy on early-childhood mortality, we evaluated a malaria-control project in Benin that implemented IMCI and promoted insecticide-treated nets (ITNs). METHODS: We conducted a before-and-after intervention study that included a nonrandomized comparison group. We used the preceding birth technique to measure early-childhood mortality (risk of dying before age 30 months), and we used health facility surveys and household surveys to measure process indicators. RESULTS: Most process indicators improved in the area covered by the intervention. Notably, because ITNs were also promoted in the comparison area children's ITN use increased by about 20 percentage points in both areas. Regarding early-childhood mortality, the trend from baseline (1999-2001) to follow-up (2002-2004) for the intervention area (13.0% decrease; P < .001) was 14.1% (P < .001) lower than was the trend for the comparison area (1.3% increase; P = .46). CONCLUSIONS: Mortality decreased in the intervention area after IMCI and ITN promotion. ITN use increased similarly in both study areas, so the mortality impact of ITNs in the 2 areas might have canceled each other out. Thus, the mortality reduction could have been primarily attributable to IMCI's effect on health care quality and care-seeking.

The rise and fall of supervision in a project designed to strengthen supervision of Integrated Management of Childhood Illness in Benin.

OBJECTIVE: In developing countries, supervision is a widely recognized strategy for improving health worker performance; and anecdotally, maintaining regular, high-quality supervision is difficult. However, remarkably little research has explored in depth why supervision is so challenging. METHODS: In the context of a trial to improve health worker adherence to Integrated Management of Childhood Illness (IMCI) guidelines and strengthen supervision in southeastern Benin, we used record reviews, focus group discussions, key informant interviews, and cross-sectional surveys to examine the supervision process. FINDINGS: Initially, little IMCI supervision occurred. The frequency increased substantially after implementing a series of workshops, but then deteriorated. Quantitative and qualitative data revealed obstacles to supervision at multiple levels of the health system. Based on supervisors' opinions, the main problems were: poor coordination; inadequate management skills and ineffective management teams; a lack of motivation; problems related to decentralization; health workers sometimes resisting IMCI implementation; and less priority given to IMCI supervision because of incentives for non-supervision activities, a lack of leadership, and an expectation of integrated supervision. To this list, based on our observations, we add: the increasing supervision workload, time required for non-supervision activities, project interventions not always being implemented as planned, and the loss of particularly effective supervisors. In terms of correctly completing steps of the supervision process, the quality of supervision was generally good. CONCLUSIONS: Managers should monitor supervision, understand the evolving influences on supervision, and use their resources and authority to both promote supervision and remove impediments to supervision. Support from leaders can be crucial, thus donors and politicians should help make supervision a true priority. As with front-line clinicians, supervisors are health workers who need support. We emphasize the importance of research to identify effective and affordable strategies for improving supervision frequency and quality. (ClinicalTrials.gov number NCT00510679).

Integrating NTD mapping protocols: Can surveys for trachoma and urinary schistosomiasis be done simultaneously?

We determined whether the school-based "disease mapping" methodology used to assess urinary schistosomiasis (SCH) is useful for determining trachoma interventions and whether the district-based approach recommended for trachoma is useful for SCH control programs. We conducted two separate integrated surveys in eight districts of central Nigeria: school based and district based. A total of 17,189 children were examined for trachoma and 16,238 children were examined for hematuria from 363 schools and 2,149 households. School surveys identified 67 communities warranting praziquantel drug treatment of SCH and 142 trachoma-endemic communities warranting trachoma control activities. In district-level estimates, we identified 24 communities for praziquantel treatment and 0 for trachoma intervention. Integrating trachoma into SCH school-based surveys, and SCH into trachoma surveys, was quick and easy, but in this setting, school-based surveys were more useful for identifying communities where intervention is warranted.

Improving pneumonia case-management in Benin: a randomized trial of a multi-faceted intervention to support health worker adherence to Integrated Management of Childhood Illness guidelines.

BACKGROUND: Pneumonia is a leading cause of death among children under five years of age. The Integrated Management of Childhood Illness strategy can improve the quality of care for pneumonia and other common illnesses in developing countries, but adherence to these guidelines could be improved. We evaluated an intervention in Benin to support health worker adherence to the guidelines after training, focusing on pneumonia case management. METHODS: We conducted a randomized trial. After a health facility survey in 1999 to assess health care quality before Integrated Management of Childhood Illness training, health workers received training plus either study supports (job aids, non-financial incentives and supervision of workers and supervisors) or "usual" supports. Follow-up surveys were conducted in 2001, 2002 and 2004. Outcomes were indicators of health care quality for Integrated Management-defined pneumonia. Further analyses included a graphical pathway analysis and multivariable logistic regression modelling to identify factors influencing case-management quality. RESULTS: We observed 301 consultations of children with non-severe pneumonia that were performed by 128 health workers in 88 public and private health facilities. Although outcomes improved in both intervention and control groups, we found no statistically significant difference between groups. However, training proceeded slowly, and low-quality care from untrained health workers diluted intervention effects. Per-protocol analyses suggested that health workers with training plus study supports performed better than those with training plus usual supports (20.4 and 19.2 percentage-point improvements for recommended treatment [p=0.08] and "recommended or adequate" treatment [p=0.01], respectively). Both groups tended to perform better than untrained health workers. Analyses of treatment errors revealed that incomplete assessment and difficulties processing clinical findings led to missed pneumonia diagnoses, and missed diagnoses led to inadequate treatment. Increased supervision frequency was associated with better care (odds ratio for recommended treatment=2.1 [95% confidence interval: 1.13.9] per additional supervisory visit). CONCLUSION: Integrated Management of Childhood Illness training was useful, but insufficient, to achieve high-quality pneumonia case management. Our study supports led to additional improvements, although large gaps in performance still remained. A simple graphical pathway analysis can identify specific, common errors that health workers make in the case-management process; this information could be used to target quality improvement activities, such as supervision (ClinicalTrials.gov number NCT00510679).

A multifaceted intervention to improve health worker adherence to integrated management of childhood illness guidelines in Benin.

OBJECTIVES: We evaluated an intervention to support health workers after training in Integrated Management of Childhood Illness (IMCI), a strategy that can improve outcomes for children in developing countries by encouraging workers' use of evidence-based guidelines for managing the leading causes of child mortality. METHODS: We conducted a randomized trial in Benin. We administered a survey in 1999 to assess health care quality before IMCI training. Health workers then received training plus either study supports (job aids, nonfinancial incentives, and supervision of workers and supervisors) or usual supports. Follow-up surveys conducted in 2001 to 2004 assessed recommended treatment, recommended or adequate treatment, and an index of overall guideline adherence. RESULTS: We analyzed 1244 consultations. Performance improved in both intervention and control groups, with no significant differences between groups. However, training proceeded slowly, and low-quality care from health workers without IMCI training diluted intervention effects. Per-protocol analyses revealed that workers with IMCI training plus study supports provided better care than did those with training plus usual supports (27.3 percentage-point difference for recommended treatment; P < .05), and both groups outperformed untrained workers. CONCLUSIONS: IMCI training was useful but insufficient. Relatively inexpensive supports can lead to additional improvements.