Conducting a Large Public Health Data Collection Project in Uganda: Methods, Tools, and Lessons Learned.

We report on the implementation experience of carrying out data collection and other activities for a public health evaluation study on whether U.S. President's Emergency Plan for AIDS Relief (PEPFAR) investment improved utilization of health services and health system strengthening in Uganda. The retrospective study period focused on the PEPFAR scale-up, from mid-2005 through mid-2011, a period of expansion of PEPFAR programing and health services. We visited 315 health care facilities in Uganda in 2011 and 2012 to collect routine health management information system data forms, as well as to conduct interviews with health system leaders. An earlier phase of this research project collected data from all 112 health district headquarters, reported elsewhere. This article describes the lessons learned from collecting data from health care facilities, project management, useful technologies, and mistakes. We used several new technologies to facilitate data collection, including portable document scanners, smartphones, and web-based data collection, along with older but reliable technologies such as car batteries for power, folding tables to create space, and letters of introduction from appropriate authorities to create entrée. Research in limited-resource settings requires an approach that values the skills and talents of local people, institutions and government agencies, and a tolerance for the unexpected. The development of personal relationships was key to the success of the project. We observed that capacity building activities were repaid many fold, especially in data management and technology.

District Health Officer Perceptions of PEPFAR's Influence on the Health System in Uganda, 2005-2011.

BACKGROUND: Vertically oriented global health initiatives (GHIs) addressing the HIV/AIDS epidemic, including the President's Emergency Plan for AIDS Relief (PEPFAR), have successfully contributed to reducing HIV/AIDS related morbidity and mortality. However, there is still debate about whether these disease-specific programs have improved or harmed health systems overall, especially with respect to non-HIV health needs. METHODS: As part of a larger evaluation of PEPFAR's effects on the health system between 2005-2011, we collected qualitative and quantitative data through semi-structured interviews with District Health Officers (DHOs) from all 112 districts in Uganda. We asked DHOs to share their perceptions about the ways in which HIV programs (largely PEPFAR in the Ugandan context) had helped and harmed the health system. We then identified key themes among their responses using qualitative content analysis. RESULTS: Ugandan DHOs said PEPFAR had generally helped the health system by improving training, integrating HIV and non-HIV care, and directly providing resources. To a lesser extent, DHOs said PEPFAR caused the health system to focus too narrowly on HIV/AIDS, increased workload for already overburdened staff, and encouraged doctors to leave public sector jobs for higher-paid positions with HIV/AIDS programs. CONCLUSION: Health system leaders in Uganda at the district level were appreciative of resources aimed at HIV they could often apply for broader purposes. As HIV infection becomes a chronic disease requiring strong health systems to manage sustained patient care over time, Uganda's weak health systems will require broad infrastructure improvements inconsistent with narrow vertical health programming.

Did PEPFAR investments result in health system strengthening? A retrospective longitudinal study measuring non-HIV health service utilization at the district level.

OBJECTIVES : PEPFAR's initial rapid scale-up approach was largely a vertical effort focused fairly exclusively on AIDS. The purpose of our research was to identify spill-over health system effects, if any, of investments intended to stem the HIV epidemic over a 6-year period with evidence from Uganda. The test of whether there were health system expansions (aside from direct HIV programming) was evidence of increases in utilization of non-HIV services-such as outpatient visits, in-facility births or immunizations-that could be associated with varying levels of PEPFAR investments at the district level. METHODS : Uganda's Health Management Information System article-based records were available from mid-2005 onwards. We visited all 112 District Health offices to collect routine monthly reports (which contain data aggregated from monthly facility reports) and annual reports (which contain data aggregated from annual facility reports). Counts of individuals on anti-retroviral therapy (ART) at year-end served as our primary predictor variable. We grouped district-months into tertiles of high, medium or low PEPFAR investment based on their total reported number of patients on ART at the end of the year. We generated incidence-rate ratios, interpreted as the relative rate of the outcome measure in relation to the lowest investment PEPFAR tertile, holding constant control variables in the model. RESULTS : We found PEPFAR investment overall was associated with small declines in service volumes in several key areas of non-HIV care (outpatient care for young children, TB tests and in-facility deliveries), after adjusting for sanitation, elementary education and HIV prevalence. For example, districts with medium and high ART investment had 11% fewer outpatient visits for children aged 4 and younger compared with low investment districts, incidence rate ratio (IRR) of 0.89 for high investment compared with low (95% CI, 0.85-0.94) and IRR of 0.93 for medium compared with low (0.90-0.96). Similarly, 22% fewer TB sputum tests were performed in high investment districts compared with low investment, [IRR 0.78 (0.72-0.85)] and 13% fewer in medium compared with low, [IRR 0.88 (0.83-0.94)]. Districts with medium and high ART investment had 5% fewer in-facility deliveries compared with low investment districts [IRR 0.95 for high compared with low, (91-1.00) and 0.96 for medium compared with low (0.93-0.99)]. Although not statistically significant, the rate of maternal deaths in high investment district-months was 13% lower than observed in low investment districts. CONCLUSIONS : This study sought to understand whether PEPFAR, as a vertical programme, may have had a spill-over effect on the health system generally, as measured by utilization. Our conclusion is that it did not, at least not in Uganda.