Bayesian network models with decision tree analysis for management of childhood malaria in Malawi.

BACKGROUND: Malaria is a major cause of death in children under five years old in low- and middle-income countries such as Malawi. Accurate diagnosis and management of malaria can help reduce the global burden of childhood morbidity and mortality. Trained healthcare workers in rural health centers manage malaria with limited supplies of malarial diagnostic tests and drugs for treatment. A clinical decision support system that integrates predictive models to provide an accurate prediction of malaria based on clinical features could aid healthcare workers in the judicious use of testing and treatment. We developed Bayesian network (BN) models to predict the probability of malaria from clinical features and an illustrative decision tree to model the decision to use or not use a malaria rapid diagnostic test (mRDT). METHODS: We developed two BN models to predict malaria from a dataset of outpatient encounters of children in Malawi. The first BN model was created manually with expert knowledge, and the second model was derived using an automated method. The performance of the BN models was compared to other statistical models on a range of performance metrics at multiple thresholds. We developed a decision tree that integrates predictions with the costs of mRDT and a course of recommended treatment. RESULTS: The manually created BN model achieved an area under the ROC curve (AUC) equal to 0.60 which was statistically significantly higher than the other models. At the optimal threshold for classification, the manual BN model had sensitivity and specificity of 0.74 and 0.42 respectively, and the automated BN model had sensitivity and specificity of 0.45 and 0.68 respectively. The balanced accuracy values were similar across all the models. Sensitivity analysis of the decision tree showed that for values of probability of malaria below 0.04 and above 0.40, the preferred decision that minimizes expected costs is not to perform mRDT. CONCLUSION: In resource-constrained settings, judicious use of mRDT is important. Predictive models in combination with decision analysis can provide personalized guidance on when to use mRDT in the management of childhood malaria. BN models can be efficiently derived from data to support clinical decision making.

Design and implementation of a clinical laboratory information system in a low-resource setting.

BACKGROUND: Reducing laboratory errors presents a significant opportunity for both cost reduction and healthcare quality improvement. This is particularly true in low-resource settings where laboratory errors are further exacerbated by poor infrastructure and shortages in a trained workforce. Informatics interventions can be used to address some of the sources of laboratory errors. OBJECTIVES: This article describes the development process for a clinical laboratory information system (LIS) that leverages informatics interventions to address problems in the laboratory testing process at a hospital in a low-resource setting. METHODS: We designed interventions using informatics methods for previously identified problems in the laboratory testing process at a clinical laboratory in a low-resource setting. First, we reviewed a pre-existing LIS functionality assessment toolkit and consulted with laboratory personnel. This provided requirements that were developed into a LIS with interventions designed to address the problems that had been identified. We piloted the LIS at the Kamuzu Central Hospital in Lilongwe, Malawi. RESULTS: We implemented a series of informatics interventions in the form of a LIS to address sources of laboratory errors and support the entire laboratory testing process. Custom hardware was built to support the ordering of laboratory tests and review of laboratory test results. CONCLUSION: Our experience highlights the potential of using informatics interventions to address systemic problems in the laboratory testing process in low-resource settings. Implementing these interventions may require innovation of new hardware to address various contextual issues. We strongly encourage thorough testing of such innovations to reduce the risk of failure when implemented.

Design and implementation of a clinical laboratory information system in a low-resource setting

Background: Reducing laboratory errors presents a significant opportunity for both cost reduction and healthcare quality improvement. This is particularly true in low-resource settings where laboratory errors are further exacerbated by poor infrastructure and shortages in a trained workforce. Informatics interventions can be used to address some of the sources of laboratory errors.Objectives: This article describes the development process for a clinical laboratory information system (LIS) that leverages informatics interventions to address problems in the laboratory testing process at a hospital in a low-resource setting.Methods: We designed interventions using informatics methods for previously identified problems in the laboratory testing process at a clinical laboratory in a low-resource setting. First, we reviewed a pre-existing LIS functionality assessment toolkit and consulted with laboratory personnel. This provided requirements that were developed into a LIS with interventions designed to address the problems that had been identified. We piloted the LIS at the Kamuzu Central Hospital in Lilongwe, Malawi.Results: We implemented a series of informatics interventions in the form of a LIS to address sources of laboratory errors and support the entire laboratory testing process. Custom hardware was built to support the ordering of laboratory tests and review of laboratory test results.Conclusion: Our experience highlights the potential of using informatics interventions to address systemic problems in the laboratory testing process in low-resource settings. Implementing these interventions may require innovation of new hardware to address various contextual issues. We strongly encourage thorough testing of such innovations to reduce the risk of failure when implemented

Assessing Ghana's eHealth workforce: implications for planning and training.

BACKGROUND: eHealth-the proficient application of information and communication technology to support healthcare delivery-has been touted as one of the best solutions to address quality and accessibility challenges in healthcare. Although eHealth could be of more value to health systems in low- and middle-income countries (LMICs) where resources are limited, identification of a competent workforce which can develop and maintain eHealth systems is a key barrier to adoption. Very little is known about the actual or optimal states of the eHealth workforce needs of LMICs. The objective of this study was to develop a framework to characterize and assess the eHealth workforce of hospitals in LMICs. METHODS: To characterize and assess the sufficiency of the workforce, we designed this study in twofold. First, we developed a general framework to categorize the eHealth workforce at any LMIC setting. Second, we combined qualitative data, using semi-structured interviews and the Workload Indicator of Staffing Needs (WISN) to assess the sufficiency of the eHealth workforce in selected hospitals in a LMIC setting like Ghana. RESULTS: We surveyed 76 (60%) of the eHealth staff from three hospitals in Ghana-La General Hospital, University of Ghana Hospital, and Greater Accra Regional Hospital. We identified two main eHealth cadres, technical support/information technology (IT) and health information management (HIM). While the HIM cadre presented diversity in expertise, the IT group was dominated by training in Science (42%) and Engineering (55%), and the majority (87%) had at least a bachelor's degree. Health information clerk (32%), health information officer (25%), help desk specialist (20%), and network administrator (11%) were the most dominant roles. Based on the WISN assessment, the eHealth workforce at all the surveyed sites was insufficient. La General and University of Ghana were operating at 10% of required IT staff capacity, while Ridge was short by 42%. CONCLUSIONS: We have developed a framework to characterize and assess the eHealth workforce in LMICs. Applying it to a case study in Ghana has given us a better understanding of potential eHealth staffing needs in LMICs, while providing the quantitative basis for building the requisite human capital to drive eHealth initiatives. Educators can also use our results to explore competency gaps and refine curricula for burgeoning training programs. The findings of this study can serve as a springboard for other LMICs to assess the effects of a well-trained eHealth workforce on the return on eHealth investments.

User-centered design and usability testing of RxMAGIC: a prescription management and general inventory control system for free clinic dispensaries.

BACKGROUND: To address challenges related to medication management in underserved settings, we developed a system for Prescription Management And General Inventory Control, or RxMAGIC, in collaboration with the Birmingham Free Clinic in Pittsburgh, Pennsylvania. RxMAGIC is an interoperable, web-based medication management system designed to standardize and streamline the dispensing practice and improve inventory control in a free clinic setting. This manuscript describes the processes used to design, develop, and deploy RxMAGIC. METHODS: We transformed data from previously performed mixed-methods needs assessment studies into functional user requirements using agile development methods. Requirements took the form of user stories that were prioritized to drive implementation of RxMAGIC as a web-application. A functional prototype was developed and tested to understand its perceived usefulness before developing a production system. Prior to deployment, we evaluated the usability of RxMAGIC with six users to diagnose potential interaction challenges that may be avoided through redesign. The results from this study were similarly prioritized and informed the final features of the production system. RESULTS: We developed 45 user stories that acted as functional requirements to incrementally build RxMAGIC. Integrating with the electronic health record at the clinic was a requirement for deployment. We utilized health data standards to communicate with the existing order entry system; an outgoing electronic prescribing framework was leveraged to send prescription data to RxMAGIC. The results of the usability study were positive, with all tested features receiving a mean score of four or five (i.e. somewhat easy or easy, respectively) on a five-point Likert scale assessing ease of completion, thus demonstrating the system's simplicity and high learnability. RxMAGIC was deployed at the clinic in October 2016 over a two-week period. CONCLUSIONS: We built RxMAGIC, an open-source, pharmacist-facing dispensary management information system that augments the pharmacist's ability to efficiently deliver medication services in a free clinic setting. RxMAGIC provides electronic dispensing and automated inventory management and alerting capabilities. We deployed RxMAGIC at the Birmingham Free Clinic and measured its usability with potential users. In future work, we plan to continue to measure the impact of RxMAGIC on pharmacist efficiency and satisfaction.

Measuring time utilization of pharmacists in the Birmingham Free Clinic dispensary.

BACKGROUND: Free and charitable clinics are a critical part of America's healthcare safety net. Although informatics tools have the potential to mitigate many of the organizational and service-related challenges facing these clinics, little research attention has been paid to the workflows and potential impact of electronic systems in these settings. In previous work, we performed a qualitative investigation at a free clinic dispensary to identify workflow challenges that may be alleviated through introduction of informatics interventions. However, this earlier study did not quantify the magnitude of these challenges. Time-motion studies offer a precise standard in quantifying healthcare workers' time expenditures on clinical activities, and can provide valuable insight into system specifications. These data, informed by a lean healthcare perspective, provide a quality improvement framework intended to maximize value and eliminate waste in inefficient workflow processes. METHODS: We performed a continuous observation time-motion study in the Birmingham Free Clinic dispensary. Two researchers followed pharmacists over the course of three general clinic sessions and recorded the duration of specific tasks. Pharmacists were then asked to identify tasks as value-added or non-value-added to facilitate calculation of the value quotient, a metric used to determine a workflow's level of efficiency. RESULTS: Four high-level workflow categories occupied almost 95 % of pharmacist time: prescription (Rx) preparation (39.8 %), clinician interaction (21.5 %), EMR operations (14.8 %), and patient interaction (18.7 %). Pharmacists invested the largest portion of time in prescription preparation, with 21.8 % of pharmacist time spent handwriting medication labels. Based on value categorizations made by the pharmacists, the average value quotient was found to be 40.3 %, indicating that pharmacists spend more than half of their time completing tasks they consider to be non-value-added. CONCLUSIONS: Our results show that pharmacists spend a large portion of their time preparing prescriptions, primarily the handwritten labeling of medication bottles and documentation tasks, which is not an optimal utilization of pharmacist expertise. The value quotient further supports that there are many wasteful tasks that may benefit from workflow redesign and health information technology, which could result in efficiency improvements for pharmacists.

Assessing Perceived Challenges to Laboratory Testing at a Malawian Referral Hospital.

Adequate laboratory infrastructure in sub-Saharan Africa is vital for tackling the burden of infectious diseases such as human immunodeficiency virus and acquired immune deficiency syndrome, malaria, and tuberculosis, yet laboratories are ill-integrated into the diagnostic and care delivery process in low-resource settings. Although much of the literature focuses on disease-specific challenges around laboratory testing, we sought to identify horizontal challenges to the laboratory testing process through interviews with clinicians involved in the diagnostic process. Based on 22 interviews with physicians, nurses, clinical officers, medical students, and laboratory technicians, technologists and supervisors, we identified 12 distinct challenges in the areas of staff, materials, workflow, and the blood bank. These challenges underscore the informational challenges that compound more visible resource shortages in the laboratory testing process, which lend themselves to horizontal strengthening efforts around the diagnostic process.

Understanding the dispensary workflow at the Birmingham Free Clinic: a proposed framework for an informatics intervention.

BACKGROUND: The Birmingham Free Clinic (BFC) in Pittsburgh, Pennsylvania, USA is a free, walk-in clinic that serves medically uninsured populations through the use of volunteer health care providers and an on-site medication dispensary. The introduction of an electronic medical record (EMR) has improved several aspects of clinic workflow. However, pharmacists' tasks involving medication management and dispensing have become more challenging since EMR implementation due to its inability to support workflows between the medical and pharmaceutical services. To inform the design of a systematic intervention, we conducted a needs assessment study to identify workflow challenges and process inefficiencies in the dispensary. METHODS: We used contextual inquiry to document the dispensary workflow and facilitate identification of critical aspects of intervention design specific to the user. Pharmacists were observed according to contextual inquiry guidelines. Graphical models were produced to aid data and process visualization. We created a list of themes describing workflow challenges and asked the pharmacists to rank them in order of significance to narrow the scope of intervention design. RESULTS: Three pharmacists were observed at the BFC. Observer notes were documented and analyzed to produce 13 themes outlining the primary challenges pharmacists encounter during dispensation at the BFC. The dispensary workflow is labor intensive, redundant, and inefficient when integrated with the clinical service. Observations identified inefficiencies that may benefit from the introduction of informatics interventions including: medication labeling, insufficient process notification, triple documentation, and inventory control. CONCLUSIONS: We propose a system for Prescription Management and General Inventory Control (RxMAGIC). RxMAGIC is a framework designed to mitigate workflow challenges and improve the processes of medication management and inventory control. While RxMAGIC is described in the context of the BFC dispensary, we believe it will be generalizable to pharmacies in other low-resource settings, both domestically and internationally.

Barriers to using eHealth data for clinical performance feedback in Malawi: A case study.

INTRODUCTION: Sub-optimal performance of healthcare providers in low-income countries is a critical and persistent global problem. The use of electronic health information technology (eHealth) in these settings is creating large-scale opportunities to automate performance measurement and provision of feedback to individual healthcare providers, to support clinical learning and behavior change. An electronic medical record system (EMR) deployed in 66 antiretroviral therapy clinics in Malawi collects data that supervisors use to provide quarterly, clinic-level performance feedback. Understanding barriers to provision of eHealth-based performance feedback for individual healthcare providers in this setting could present a relatively low-cost opportunity to significantly improve the quality of care. OBJECTIVE: The aims of this study were to identify and describe barriers to using EMR data for individualized audit and feedback for healthcare providers in Malawi and to consider how to design technology to overcome these barriers. METHODS: We conducted a qualitative study using interviews, observations, and informant feedback in eight public hospitals in Malawi where an EMR system is used. We interviewed 32 healthcare providers and conducted seven hours of observation of system use. RESULTS: We identified four key barriers to the use of EMR data for clinical performance feedback: provider rotations, disruptions to care processes, user acceptance of eHealth, and performance indicator lifespan. Each of these factors varied across sites and affected the quality of EMR data that could be used for the purpose of generating performance feedback for individual healthcare providers. CONCLUSION: Using routinely collected eHealth data to generate individualized performance feedback shows potential at large-scale for improving clinical performance in low-resource settings. However, technology used for this purpose must accommodate ongoing changes in barriers to eHealth data use. Understanding the clinical setting as a complex adaptive system (CAS) may enable designers of technology to effectively model change processes to mitigate these barriers.

Computer-supported feedback message tailoring: theory-informed adaptation of clinical audit and feedback for learning and behavior change.

BACKGROUND: Evidence shows that clinical audit and feedback can significantly improve compliance with desired practice, but it is unclear when and how it is effective. Audit and feedback is likely to be more effective when feedback messages can influence barriers to behavior change, but barriers to change differ across individual health-care providers, stemming from differences in providers' individual characteristics. DISCUSSION: The purpose of this article is to invite debate and direct research attention towards a novel audit and feedback component that could enable interventions to adapt to barriers to behavior change for individual health-care providers: computer-supported tailoring of feedback messages. We argue that, by leveraging available clinical data, theory-informed knowledge about behavior change, and the knowledge of clinical supervisors or peers who deliver feedback messages, a software application that supports feedback message tailoring could improve feedback message relevance for barriers to behavior change, thereby increasing the effectiveness of audit and feedback interventions. We describe a prototype system that supports the provision of tailored feedback messages by generating a menu of graphical and textual messages with associated descriptions of targeted barriers to behavior change. Supervisors could use the menu to select messages based on their awareness of each feedback recipient's specific barriers to behavior change. We anticipate that such a system, if designed appropriately, could guide supervisors towards giving more effective feedback for health-care providers. A foundation of evidence and knowledge in related health research domains supports the development of feedback message tailoring systems for clinical audit and feedback. Creating and evaluating computer-supported feedback tailoring tools is a promising approach to improving the effectiveness of clinical audit and feedback.

Computer-Supported Feedback Message Tailoring for Healthcare Providers in Malawi: Proof-of-Concept.

Although performance feedback has the potential to help clinicians improve the quality and safety of care, healthcare organizations generally lack knowledge about how this guidance is best provided. In low-resource settings, tools for theory-informed feedback tailoring may enhance limited clinical supervision resources. Our objectives were to establish proof-of-concept for computer-supported feedback message tailoring in Malawi, Africa. We conducted this research in five stages: clinical performance measurement, modeling the influence of feedback on antiretroviral therapy (ART) performance, creating a rule-based message tailoring process, generating tailored messages for recipients, and finally analysis of performance and message tailoring data. We retrospectively generated tailored messages for 7,448 monthly performance reports from 11 ART clinics. We found that tailored feedback could be routinely generated for four guideline-based performance indicators, with 35% of reports having messages prioritized to optimize the effect of feedback. This research establishes proof-of-concept for a novel approach to improving the use of clinical performance feedback in low-resource settings and suggests possible directions for prospective evaluations comparing alternative designs of feedback messages.

Informatics solutions for bridging the gap between clinical and laboratory services in a low-resource setting.

Background: There has been little formal analysis of laboratory systems in resource-limited settings, despite widespread consensus around the importance of a strong laboratory infrastructure. Objectives: This study details the informational challenges faced by the laboratory at Kamuzu Central Hospital, a tertiary health facility in Malawi; and proposes ways in which informatics can bolster the efficiency and role of low-resource laboratory systems. Methods: We evaluated previously-collected data on three different aspects of laboratory use. A four-week quality audit of laboratory test orders quantified challenges associated with collecting viable specimens for testing. Data on tests run by the laboratory over a one-year period described the magnitude of the demand for laboratory services. Descriptive information about the laboratory workflow identified informational process breakdowns in the pre-analytical and post-analytical phases and was paired with a 24-hour sample of laboratory data on results reporting. Results: The laboratory conducted 242 242 tests over a 12-month period. The four-week quality audit identified 54% of samples as untestable. Prohibitive paperwork errors were identified in 16% of samples. Laboratory service workflows indicated a potential process breakdown in sample transport and results reporting resulting from the lack of assignment of these tasks to any specific employee cadre. The study of result reporting time showed a mean of almost six hours, with significant variation. Conclusions: This analysis identified challenges in each phase of laboratory testing. Informatics could improve the management of this information by streamlining test ordering and the communication of test orders to the laboratory and results back to the ordering physician.

Applying lessons learnt from the 'DOTS' Tuberculosis Model to monitoring and evaluating persons with diabetes mellitus in Blantyre, Malawi.

The global burden of diabetes mellitus (DM) is immense and predicted to reach 438 million by 2030, with 80% of the cases being in the developing world. The management of chronic non-communicable diseases like DM is poor in most resource-limited settings, and the 'directly observed therapy, short course' (DOTS) framework for tuberculosis control has been proposed as a feasible way to improve this situation. In late 2009, aspects of the DOTS model were applied to the management of persons with DM in the diabetes clinic in Queen Elizabeth Central Hospital, Blantyre, Malawi, and a point-of-care electronic medical record system was set up to support and monitor patients in care. This is the first quarterly and cumulative report of persons with DM registered for care stratified by treatment outcomes, complications and medication history up to 31 December 2010. There were 170 new patients registered between October and December 2010, with 1864 ever registered by 31 December 2010. Most patients were alive and in care; 3 died, 53 defaulted and 3 transferred out. Of those on oral hypoglycaemic agents, metformin was most commonly used. Complications were common. The monitoring and evaluation will be further refined, and at the same time, the systems developed in Blantyre will be expanded to other parts of the country.

The feasibility of automating audit and feedback for ART guideline adherence in Malawi.

OBJECTIVE: To determine the feasibility of using electronic medical record (EMR) data to provide audit and feedback of antiretroviral therapy (ART) clinical guideline adherence to healthcare workers (HCWs) in Malawi. MATERIALS AND METHODS: We evaluated recommendations from Malawi's ART guidelines using GuideLine Implementability Appraisal criteria. Recommendations that passed selected criteria were converted into ratio-based performance measures. We queried representative EMR data to determine the feasibility of generating feedback for each performance measure, summed clinical encounters representing each performance measure's denominator, and then measured the distribution of encounter frequency for individual HCWs across nurse and clinical officer groups. RESULTS: We analyzed 423,831 encounters in the EMR data and generated automated feedback for 21 recommendations (12%) from Malawi's ART guidelines. We identified 11 nurse recommendations and eight clinical officer recommendations. Individual nurses and clinical officers had an average of 45 and 59 encounters per month, per recommendation, respectively. Another 37 recommendations (21%) would support audit and feedback if additional routine EMR data are captured and temporal constraints are modeled. DISCUSSION: It appears feasible to implement automated guideline adherence feedback that could potentially improve HCW performance and supervision. Feedback reports may support workplace learning by increasing HCWs' opportunities to reflect on their performance. CONCLUSION: A moderate number of recommendations from Malawi's ART guidelines can be used to generate automated guideline adherence feedback using existing EMR data. Further study is needed to determine the receptivity of HCWs to peer comparison feedback and barriers to implementation of automated audit and feedback in low-resource settings.

Experience implementing a point-of-care electronic medical record system for primary care in Malawi.

Due to the fact that health care professionals in Malawi are often overstretched, the use and quality of health data can be compromised. The Malawi Health Management Information System (HMIS) has streamlined data collection and reporting and increased the use of data to improve care. Obstacles remain, including incomplete reporting and low staff morale. With the Baobab Health Trust and the Malawi Ministry of Health, Partners In Health piloted an innovative point-of-care data system for primary care that functions alongside OpenMRS, an open source medical record platform. The system has given access to a patient-level primary care dataset in real time. Initial results highlight some of the benefits of a point-of-care system such as improved data quality, emphasize the importance of sharing data with clinical practitioners, and shed light on how this approach could strengthen HMIS.

Touchscreen task efficiency and learnability in an electronic medical record at the point-of-care.

The objective of this study was to determine the relative efficiency of novices compared to a prediction of skilled use when performing tasks using the touchscreen interface of an EMR developed in Malawi. We observed novice users performing touchscreen tasks and recorded timestamp data from their performances. Using a predictive human performance modeling tool, the authors predicted the skilled task performance time for each task. Efficiency and rates of error were evaluated with respect to user interface design. Nineteen participants performed 31 EMR tasks seven times for a total of 4,123 observed performances. We analyzed twelve representative tasks leaving 1,596 performances featuring six user interface designs. Mean novice performance time was significantly slower than mean predicted skilled performance time (p<0.001). However, novices performed faster than the predicted skilled level in 208 (13%) of successful task performances. These findings suggest the user interface design supports a primary design goal of the EMR--to allow novice users to perform tasks efficiently and effectively.