Documenting HIV research-utilization activities, outputs and outcomes: examples and lessons learned from Project SOAR.

The importance of using research findings to inform policy and program decisions is well recognized, but the literature on measuring research utilization activities is scarce. As funding to support some areas of research wanes or remains stagnant, the need to document the value of investing in research by its' effect on improved programs and policies becomes increasingly necessary. We present the experience of Project SOAR, a six-year USAID-funded project focusing on HIV/AIDS-related implementation research, to demonstrate measurement of research utilization. We follow the project's research-utilization logic model, including inputs, activities, outputs, and outcomes. We present tools the project developed and examples from project studies and discuss what works, remaining challenges and how to overcome them, and lessons learned. We then make recommendations for incorporating research-utilization activities and measurement in implementation-research studies.

Engaging Stakeholders, from Inception and Throughout the Study, is Good Research Practice to Promote use of Findings.

The need for research-informed programming and policy making is well established. However, there is limited evidence that, when researchers actively promote utilization of research findings, stakeholders use such findings for decision making in low- and middle-income countries (LMIC). A common barrier for research uptake in LMIC is that researchers focus on passive dissemination of final findings as the primary vehicle to affect research uptake. A more active approach to facilitating research utilization (RU) is necessary. Project SOAR, a six-year USAID-funded operations research project, recognized this gap and developed an approach to include the end data users in the research process from inception to final results dissemination. In this commentary, we make recommendations for active facilitation of research uptake using emerging lessons from SOAR's RU process that focuses on ongoing engagement of stakeholders throughout the life of the study.

Research gaps in routine health information system design barriers to data quality and use in low- and middle-income countries: A literature review.

Despite the potential impact of health information system (HIS) design barriers on health data quality and use and, ultimately, health outcomes in low- and middle-income countries (LMICs), no comprehensive literature review has been conducted to study them in this context. We therefore conducted a formal literature review to understand system design barriers to data quality and use in LMICs and to identify any major research gaps related understanding how system design affects data use. We conducted an electronic search across 4 scientific databases-PubMed, Web of Science, Embase, and Global Health-and consulted a data use expert. Following a systematic inclusion and exclusion process, 316 publications (316 abstracts and 18 full papers) were included in the review. We found a paucity of scientific publications that explicitly describe system design factors that hamper data quality or data use for decision making. Although user involvement, work flow, human-computer interactions, and user experience are critical aspects of system design, our findings suggest that these issues are not discussed or conceptualized in the literature. Findings also showed that individual training efforts focus primarily on imparting data analysis skills. The adverse impact of HIS design barriers on data integrity and health system performance may be even bigger in LMICs than elsewhere, leading to errors in population health management and clinical care. We argue for integrating systems thinking into HIS strengthening efforts to reduce the HIS design-user reality gap.

Moving data off the shelf and into action: an intervention to improve data-informed decision making in Côte d'Ivoire.

BACKGROUND: Improving a health system requires data, but too often they are unused or under-used by decision makers. Without interventions to improve the use of data in decision making, health systems cannot meet the needs of the populations they serve. In 2008, in Côte d'Ivoire, data were largely unused in health decision-making processes. OBJECTIVE: To implement and evaluate an intervention to improve the use of data in decision making in Cote d'Ivoire. DESIGN: From 2008 to 2012, Cote d'Ivoire sought to improve the use of national health data through an intervention that broadens participation in and builds links between data collection and decision-making processes; identifies information needs; improves data quality; builds capacity to analyze, synthesize, and interpret data; and develops policies to support data use. To assess the results, a Performance of Routine Information System Management Assessment was conducted before and after the intervention using a combination of purposeful and random sampling. In 2008, the sample consisted of the central level, 12 districts, and 119 facilities, and in 2012, the sample consisted of the central level, 20 districts, and 190 health facilities. To assess data use, we developed dichotomous indicators: discussions of analysis findings, decisions taken based on the analysis, and decisions referred to upper management for action. We aggregated the indicators to generate a composite, continuous index of data use. RESULTS: From 2008 to 2012, the district data-use score increased from 40 to 70%; the facility score remained the same - 38%. The central score is not reported, because of a methodological difference in the two assessments. CONCLUSIONS: The intervention improved the use of data in decision making at the district level in Côte d'Ivoire. This study provides an example of, and guidance for, implementing a large-scale intervention to improve data-informed decision making.

Impact of a decision-support tool on decision making at the district level in Kenya.

BACKGROUND: In many countries, the responsibility for planning and delivery of health services is devolved to the subnational level. Health programs, however, often fall short of efficient use of data to inform decisions. As a result, programs are not as effective as they can be at meeting the health needs of the populations they serve. In Kenya, a decision-support tool, the District Health Profile (DHP) tool was developed to integrate data from health programs, primarily HIV, at the district level and to enable district health management teams to review and monitor program progress for specific health issues to make informed service delivery decisions. METHODS: Thirteen in-depth interviews were conducted with ten tool users and three non-users in six districts to qualitatively assess the process of implementing the tool and its effect on data-informed decision making at the district level. The factors that affected use or non-use of the tool were also investigated. Respondents were selected via convenience sample from among those that had been trained to use the DHP tool except for one user who was self-taught to use the tool. Selection criteria also included respondents from urban districts with significant resources as well as respondents from more remote, under-resourced districts. RESULTS: Findings from the in-depth interviews suggest that among those who used it, the DHP tool had a positive effect on data analysis, review, interpretation, and sharing at the district level. The automated function of the tool allowed for faster data sharing and immediate observation of trends that facilitated data-informed decision making. All respondents stated that the DHP tool assisted them to better target existing services in need of improvement and to plan future services, thus positively influencing program improvement. CONCLUSIONS: This paper stresses the central role that a targeted decision-support tool can play in making data aggregation, analysis, and presentation easier and faster. The visual synthesis of data facilitates the use of information in health decision making at the district level of a health system and promotes program improvement. The experience in Kenya can be applied to other countries that face challenges making district-level, data-informed decisions with data from fragmented information systems.

Improving the use of health data for health system strengthening.

BACKGROUND: Good quality and timely data from health information systems are the foundation of all health systems. However, too often data sit in reports, on shelves or in databases and are not sufficiently utilised in policy and program development, improvement, strategic planning and advocacy. Without specific interventions aimed at improving the use of data produced by information systems, health systems will never fully be able to meet the needs of the populations they serve. OBJECTIVE: To employ a logic model to describe a pathway of how specific activities and interventions can strengthen the use of health data in decision making to ultimately strengthen the health system. DESIGN: A logic model was developed to provide a practical strategy for developing, monitoring and evaluating interventions to strengthen the use of data in decision making. The model draws on the collective strengths and similarities of previous work and adds to those previous works by making specific recommendations about interventions and activities that are most proximate to affect the use of data in decision making. The model provides an organizing framework for how interventions and activities work to strengthen the systematic demand, synthesis, review, and use of data. RESULTS: The logic model and guidance are presented to facilitate its widespread use and to enable improved data-informed decision making in program review and planning, advocacy, policy development. Real world examples from the literature support the feasible application of the activities outlined in the model. CONCLUSIONS: The logic model provides specific and comprehensive guidance to improve data demand and use. It can be used to design, monitor and evaluate interventions, and to improve demand for, and use of, data in decision making. As more interventions are implemented to improve use of health data, those efforts need to be evaluated.

Comparative acceptability of combined and progestin-only injectable contraceptives in Kenya.

OBJECTIVE: We compared 12-month continuation rates, menstrual bleeding patterns and other aspects of acceptability between users of Cyclofem and users of Depo-Provera. METHODS: The life-table method was used to calculate quarterly continuation rates. In all, 360 Kenyan women were randomly assigned to one of the two contraceptives. User-satisfaction questionnaires were administered at 6 and 12 months or at discontinuation, whichever occurred first. RESULTS: The 1-year continuation rate was 75.4% for Depo-Provera users versus 56.5% for Cyclofem users (p<.001). Main reasons for discontinuation included difficulty making clinic visits (45.1% for Cyclofem vs. 40% for Depo-Provera), menstrual changes (14.1% vs. 12.5%) and nonmenstrual problems (15.5% vs. 12.5%). None of the Depo-Provera users and 8.5% of the Cyclofem users claimed frequency of visits as the main reason for discontinuation. In all, 70.6% of the Depo-Provera users were amenorrheic after 12 months, as were 20.8% of the Cyclofem users. CONCLUSIONS: The 1-year continuation rate was higher for Depo-Provera than for Cyclofem. There was no important difference in discontinuation rates because of menstrual problems; the difference mainly reflected the frequency of visits required.