Extending an open-source tool to measure data quality: case report on Observational Health Data Science and Informatics (OHDSI).

INTRODUCTION: As the health system seeks to leverage large-scale data to inform population outcomes, the informatics community is developing tools for analysing these data. To support data quality assessment within such a tool, we extended the open-source software Observational Health Data Sciences and Informatics (OHDSI) to incorporate new functions useful for population health. METHODS: We developed and tested methods to measure the completeness, timeliness and entropy of information. The new data quality methods were applied to over 100 million clinical messages received from emergency department information systems for use in public health syndromic surveillance systems. DISCUSSION: While completeness and entropy methods were implemented by the OHDSI community, timeliness was not adopted as its context did not fit with the existing OHDSI domains. The case report examines the process and reasons for acceptance and rejection of ideas proposed to an open-source community like OHDSI.

Risk of angioedema associated with levetiracetam compared with phenytoin: Findings of the observational health data sciences and informatics research network.

Recent adverse event reports have raised the question of increased angioedema risk associated with exposure to levetiracetam. To help address this question, the Observational Health Data Sciences and Informatics research network conducted a retrospective observational new-user cohort study of seizure patients exposed to levetiracetam (n = 276,665) across 10 databases. With phenytoin users (n = 74,682) as a comparator group, propensity score-matching was conducted and hazard ratios computed for angioedema events by per-protocol and intent-to-treat analyses. Angioedema events were rare in both the levetiracetam and phenytoin groups (54 vs. 71 in per-protocol and 248 vs. 435 in intent-to-treat). No significant increase in angioedema risk with levetiracetam was seen in any individual database (hazard ratios ranging from 0.43 to 1.31). Meta-analysis showed a summary hazard ratio of 0.72 (95% confidence interval [CI] 0.39-1.31) and 0.64 (95% CI 0.52-0.79) for the per-protocol and intent-to-treat analyses, respectively. The results suggest that levetiracetam has the same or lower risk for angioedema than phenytoin, which does not currently carry a labeled warning for angioedema. Further studies are warranted to evaluate angioedema risk across all antiepileptic drugs.

Characterizing treatment pathways at scale using the OHDSI network.

Observational research promises to complement experimental research by providing large, diverse populations that would be infeasible for an experiment. Observational research can test its own clinical hypotheses, and observational studies also can contribute to the design of experiments and inform the generalizability of experimental research. Understanding the diversity of populations and the variance in care is one component. In this study, the Observational Health Data Sciences and Informatics (OHDSI) collaboration created an international data network with 11 data sources from four countries, including electronic health records and administrative claims data on 250 million patients. All data were mapped to common data standards, patient privacy was maintained by using a distributed model, and results were aggregated centrally. Treatment pathways were elucidated for type 2 diabetes mellitus, hypertension, and depression. The pathways revealed that the world is moving toward more consistent therapy over time across diseases and across locations, but significant heterogeneity remains among sources, pointing to challenges in generalizing clinical trial results. Diabetes favored a single first-line medication, metformin, to a much greater extent than hypertension or depression. About 10% of diabetes and depression patients and almost 25% of hypertension patients followed a treatment pathway that was unique within the cohort. Aside from factors such as sample size and underlying population (academic medical center versus general population), electronic health records data and administrative claims data revealed similar results. Large-scale international observational research is feasible.

Observational Health Data Sciences and Informatics (OHDSI): Opportunities for Observational Researchers.

The vision of creating accessible, reliable clinical evidence by accessing the clincial experience of hundreds of millions of patients across the globe is a reality. Observational Health Data Sciences and Informatics (OHDSI) has built on learnings from the Observational Medical Outcomes Partnership to turn methods research and insights into a suite of applications and exploration tools that move the field closer to the ultimate goal of generating evidence about all aspects of healthcare to serve the needs of patients, clinicians and all other decision-makers around the world.

An Interactive User Interface for Drug Labeling to Improve Readability and Decision-Making.

FDA-approved prescribing information (also known as product labeling or labels) contain critical safety information for health care professionals. Drug labels have often been criticized, however, for being overly complex, difficult to read, and rife with overwarning, leading to high cognitive load. In this project, we aimed to improve the usability of drug labels by increasing the 'signal-to-noise ratio' and providing meaningful information to care providers based on patient-specific comorbidities and concomitant medications. In the current paper, we describe the design process and resulting web application, known as myDrugLabel. Using the Structured Product Label documents as a base, we describe the process of label personalization, readability improvements, and integration of diverse evidence sources, including the medical literature from PubMed, pharmacovigilance reports from FDA adverse event reporting system (FAERS), and social media signals directly into the label.

Regenstrief Institute's Medical Gopher: a next-generation homegrown electronic medical record system.

OBJECTIVE: Regenstrief Institute developed one of the seminal computerized order entry systems, the Medical Gopher, for implementation at Wishard Hospital nearly three decades ago. Wishard Hospital and Regenstrief remain committed to homegrown software development, and over the past 4 years we have fully rebuilt Gopher with an emphasis on usability, safety, leveraging open source technologies, and the advancement of biomedical informatics research. Our objective in this paper is to summarize the functionality of this new system and highlight its novel features. MATERIALS AND METHODS: Applying a user-centered design process, the new Gopher was built upon a rich-internet application framework using an agile development process. The system incorporates order entry, clinical documentation, result viewing, decision support, and clinical workflow. We have customized its use for the outpatient, inpatient, and emergency department settings. RESULTS: The new Gopher is now in use by over 1100 users a day, including an average of 433 physicians caring for over 3600 patients daily. The system includes a wizard-like clinical workflow, dynamic multimedia alerts, and a familiar 'e-commerce'-based interface for order entry. Clinical documentation is enhanced by real-time natural language processing and data review is supported by a rapid chart search feature. DISCUSSION: As one of the few remaining academically developed order entry systems, the Gopher has been designed both to improve patient care and to support next-generation informatics research. It has achieved rapid adoption within our health system and suggests continued viability for homegrown systems in settings of close collaboration between developers and providers.

Speaking the same language? International variations in the safety information accompanying top-selling prescription drugs.

BACKGROUND: The official prescribing information document distributed with a prescription drug is a key source of safety information, but it may include excessive or insufficient details. OBJECTIVES: To compare prescribing information approved by the US Food and Drug Administration with the UK, Canada and Australia to identify content differences in safety warnings. METHODS: For 20 top-selling prescription drugs, we used an automated natural language processing tool to calculate the number and severity of reported adverse drug reactions (ADRs). We fit hierarchical Poisson models and included fixed effects for other prescribing information characteristics. Separately, we analysed the appearance and content of 'black box' warnings. RESULTS: There was substantial variation in safety content of approved prescribing information. Canada had the highest median ADRs per drug (138 (IQR 86-234)) and the UK had the lowest (84 (IQR 51-111)). The number of ADRs reported was on average 50% higher in Canada compared with the USA (ratio of ADRs/document: 1.5, 95% CI 1.4 to 1.6, p<0.001). By contrast, there were on average 15% fewer ADRs listed in the UK compared with the USA (ratio of ADRs/document 0.85 (95% CI 0.78 to 0.93, p<0.001), and 21% fewer ADRs listed in Australia compared with the USS (ratio of ADRs/document 0.79, 95% CI 0.74 to 0.85, p<0.001). There were no variations in ADR severity. The presence and qualitative content of boxed warnings also showed substantial diversity. CONCLUSIONS: International variations exist in the presentation of safety data in drug prescribing information, which may have important implications for patient safety. Better international coordination is necessary to enhance use of this information for patient decision-making.

Adherence to drug-drug interaction alerts in high-risk patients: a trial of context-enhanced alerting.

OBJECTIVE: Drug-drug interaction (DDI) alerting is an important form of clinical decision support, yet physicians often fail to attend to critical DDI warnings due to alert fatigue. We previously described a model for highlighting patients at high risk of a DDI by enhancing alerts with relevant laboratory data. We sought to evaluate the effect of this model on alert adherence in high-risk patients. METHODS: A 6-month randomized controlled trial involving 1029 outpatient physicians was performed. The target interactions were all DDIs known to cause hyperkalemia. Alerts in the intervention group were enhanced with the patient's most recent potassium and creatinine levels. The control group received unmodified alerts. High -risk patients were those with baseline potassium >5.0 mEq/l and/or creatinine ≥1.5 mg/dl (132 µmol/l). RESULTS: We found no significant difference in alert adherence in high-risk patients between the intervention group (15.3%) and the control group (16.8%) (p=0.71). Adherence in normal risk patients was significantly lower in the intervention group (14.6%) than in the control group (18.6%) (p<0.01). In neither group did physicians increase adherence in patients at high risk. CONCLUSIONS: Physicians adhere poorly to hyperkalemia-associated DDI alerts even in patients with risk factors for a clinically significant interaction, and the display of relevant laboratory data in these alerts did not improve adherence levels in the outpatient setting. Further research is necessary to determine optimal strategies for conveying patient-specific DDI risk.

Literature based drug interaction prediction with clinical assessment using electronic medical records: novel myopathy associated drug interactions.

Drug-drug interactions (DDIs) are a common cause of adverse drug events. In this paper, we combined a literature discovery approach with analysis of a large electronic medical record database method to predict and evaluate novel DDIs. We predicted an initial set of 13197 potential DDIs based on substrates and inhibitors of cytochrome P450 (CYP) metabolism enzymes identified from published in vitro pharmacology experiments. Using a clinical repository of over 800,000 patients, we narrowed this theoretical set of DDIs to 3670 drug pairs actually taken by patients. Finally, we sought to identify novel combinations that synergistically increased the risk of myopathy. Five pairs were identified with their p-values less than 1E-06: loratadine and simvastatin (relative risk or RR = 1.69); loratadine and alprazolam (RR = 1.86); loratadine and duloxetine (RR = 1.94); loratadine and ropinirole (RR = 3.21); and promethazine and tegaserod (RR = 3.00). When taken together, each drug pair showed a significantly increased risk of myopathy when compared to the expected additive myopathy risk from taking either of the drugs alone. Based on additional literature data on in vitro drug metabolism and inhibition potency, loratadine and simvastatin and tegaserod and promethazine were predicted to have a strong DDI through the CYP3A4 and CYP2D6 enzymes, respectively. This new translational biomedical informatics approach supports not only detection of new clinically significant DDI signals, but also evaluation of their potential molecular mechanisms.

A successful model and visual design for creating context-aware drug-drug interaction alerts.

Evaluating the potential harm of a drug-drug interaction (DDI) requires knowledge of a patient's relevant co-morbidities and risk factors. Current DDI alerts lack such patient-specific contextual data. In this paper, we present an efficient model for integrating pertinent patient data into DDI alerts. This framework is designed to be interoperable across multiple drug knowledge bases and clinical information systems. To evaluate the model, we generated a set of contextual DDI data using our local drug knowledge base then conducted an evaluation study of a prototype contextual alert design. The alert received favorable ratings from study subjects, who agreed it was an improvement over traditional alerts and was likely to support clinical management and save physician time. This framework may ultimately help reduce alert fatigue through the dynamic display of DDI alerts based on patient risk.

ADESSA: A Real-Time Decision Support Service for Delivery of Semantically Coded Adverse Drug Event Data.

Evaluating medications for potential adverse events is a time-consuming process, typically involving manual lookup of information by physicians. This process can be expedited by CDS systems that support dynamic retrieval and filtering of adverse drug events (ADE's), but such systems require a source of semantically-coded ADE data. We created a two-component system that addresses this need. First we created a natural language processing application which extracts adverse events from Structured Product Labels and generates a standardized ADE knowledge base. We then built a decision support service that consumes a Continuity of Care Document and returns a list of patient-specific ADE's. Our database currently contains 534,125 ADE's from 5602 product labels. An NLP evaluation of 9529 ADE's showed recall of 93% and precision of 95%. On a trial set of 30 CCD's, the system provided adverse event data for 88% of drugs and returned these results in an average of 620ms.

Data visualization speeds review of potential adverse drug events in patients on multiple medications.

Patients on multiple medications are at increased risk for adverse drug events. While physicians can reduce this risk by regularly reviewing the side-effect profiles of their patients' medications, this process can be time-consuming. We created a decision support system designed to expedite reviewing potential adverse reactions through information visualization. The system includes a database containing 16,340 unique drug and side-effect pairs, representing 250 common medications. A numeric score is assigned to each pair reflecting the strength of association between drug and effect. Based on these scores, the system generates graphical adverse reaction maps for any user-selected combination of drugs. A study comparing speed and accuracy of retrieving side-effect data using this tool versus UpToDate demonstrated a 60% reduction in time to complete a query (61 s vs. 155 s, p < 0.0001) with no decrease in accuracy. These findings suggest that information visualization can significantly expedite review of potential adverse drug events.

Electronic medical billing records and public health surveillance: comparison of two systems used during the 1996 Atlanta Olympic Games.

For selected diagnoses of public health interest during the 1996 Olympic Games, the authors compared data concurrently obtained on the same patient population by two separate surveillance systems: (1) an existing hospital electronic medical billing records system and (2) a system based on manual record abstraction. Counts of total patient visits closely agreed, though the two systems differed considerably in some diagnostic categories, especially injuries. The authors concluded that while causation, risk factors, and illness severity are not reflected directly in standard International Classification of Diseases (ICD) codes, and "E" codes to indicate causation may not be used, special-purpose surveillance systems based on existing computerized medical records may be as effective as manual data abstracting.