Increasing trust in real-world evidence through evaluation of observational data quality.

OBJECTIVE: Advances in standardization of observational healthcare data have enabled methodological breakthroughs, rapid global collaboration, and generation of real-world evidence to improve patient outcomes. Standardizations in data structure, such as use of common data models, need to be coupled with standardized approaches for data quality assessment. To ensure confidence in real-world evidence generated from the analysis of real-world data, one must first have confidence in the data itself. MATERIALS AND METHODS: We describe the implementation of check types across a data quality framework of conformance, completeness, plausibility, with both verification and validation. We illustrate how data quality checks, paired with decision thresholds, can be configured to customize data quality reporting across a range of observational health data sources. We discuss how data quality reporting can become part of the overall real-world evidence generation and dissemination process to promote transparency and build confidence in the resulting output. RESULTS: The Data Quality Dashboard is an open-source R package that reports potential quality issues in an OMOP CDM instance through the systematic execution and summarization of over 3300 configurable data quality checks. DISCUSSION: Transparently communicating how well common data model-standardized databases adhere to a set of quality measures adds a crucial piece that is currently missing from observational research. CONCLUSION: Assessing and improving the quality of our data will inherently improve the quality of the evidence we generate.

Reopening businesses and risk of COVID-19 transmission.

The true risk of a COVID-19 resurgence as states reopen businesses is unknown. In this paper, we used anonymized cell-phone data to quantify the potential risk of COVID-19 transmission in business establishments by building a Business Risk Index that measures transmission risk over time. The index was built using two metrics, visits per square foot and the average duration of visits, to account for both density of visits and length of time visitors linger in the business. We analyzed trends in traffic patterns to 1,272,260 businesses across eight states from January 2020 to June 2020. We found that potentially risky traffic behaviors at businesses decreased by 30% by April. Since the end of April, the risk index has been increasing as states reopen. There are some notable differences in trends across states and industries. Finally, we showed that the time series of the average Business Risk Index is useful for forecasting future COVID-19 cases at the county-level (P < 0.001). We found that an increase in a county's average Business Risk Index is associated with an increase in positive COVID-19 cases in 1 week (IRR: 1.16, 95% CI: (1.1-1.26)). Our risk index provides a way for policymakers and hospital decision-makers to monitor the potential risk of COVID-19 transmission from businesses based on the frequency and density of visits to businesses. This can serve as an important metric as states monitor and evaluate their reopening strategies.

Deep phenotyping of 34,128 adult patients hospitalised with COVID-19 in an international network study.

Comorbid conditions appear to be common among individuals hospitalised with coronavirus disease 2019 (COVID-19) but estimates of prevalence vary and little is known about the prior medication use of patients. Here, we describe the characteristics of adults hospitalised with COVID-19 and compare them with influenza patients. We include 34,128 (US: 8362, South Korea: 7341, Spain: 18,425) COVID-19 patients, summarising between 4811 and 11,643 unique aggregate characteristics. COVID-19 patients have been majority male in the US and Spain, but predominantly female in South Korea. Age profiles vary across data sources. Compared to 84,585 individuals hospitalised with influenza in 2014-19, COVID-19 patients have more typically been male, younger, and with fewer comorbidities and lower medication use. While protecting groups vulnerable to influenza is likely a useful starting point in the response to COVID-19, strategies will likely need to be broadened to reflect the particular characteristics of individuals being hospitalised with COVID-19.

Deep phenotyping of 34,128 patients hospitalised with COVID-19 and a comparison with 81,596 influenza patients in America, Europe and Asia: an international network study.

Background In this study we phenotyped individuals hospitalised with coronavirus disease 2019 (COVID-19) in depth, summarising entire medical histories, including medications, as captured in routinely collected data drawn from databases across three continents. We then compared individuals hospitalised with COVID-19 to those previously hospitalised with influenza. Methods We report demographics, previously recorded conditions and medication use of patients hospitalised with COVID-19 in the US (Columbia University Irving Medical Center [CUIMC], Premier Healthcare Database [PHD], UCHealth System Health Data Compass Database [UC HDC], and the Department of Veterans Affairs [VA OMOP]), in South Korea (Health Insurance Review & Assessment [HIRA]), and Spain (The Information System for Research in Primary Care [SIDIAP] and HM Hospitales [HM]). These patients were then compared with patients hospitalised with influenza in 2014-19. Results 34,128 (US: 8,362, South Korea: 7,341, Spain: 18,425) individuals hospitalised with COVID-19 were included. Between 4,811 (HM) and 11,643 (CUIMC) unique aggregate characteristics were extracted per patient, with all summarised in an accompanying interactive website (http://evidence.ohdsi.org/Covid19CharacterizationHospitalization/). Patients were majority male in the US (CUIMC: 52%, PHD: 52%, UC HDC: 54%, VA OMOP: 94%,) and Spain (SIDIAP: 54%, HM: 60%), but were predominantly female in South Korea (HIRA: 60%). Age profiles varied across data sources. Prevalence of asthma ranged from 4% to 15%, diabetes from 13% to 43%, and hypertensive disorder from 24% to 70% across data sources. Between 14% and 33% were taking drugs acting on the renin-angiotensin system in the 30 days prior to hospitalisation. Compared to 81,596 individuals hospitalised with influenza in 2014-19, patients admitted with COVID-19 were more typically male, younger, and healthier, with fewer comorbidities and lower medication use. Conclusions We provide a detailed characterisation of patients hospitalised with COVID-19. Protecting groups known to be vulnerable to influenza is a useful starting point to minimize the number of hospital admissions needed for COVID-19. However, such strategies will also likely need to be broadened so as to reflect the particular characteristics of individuals hospitalised with COVID-19.

Risk of lower extremity amputations in people with type 2 diabetes mellitus treated with sodium-glucose co-transporter-2 inhibitors in the USA: A retrospective cohort study.

AIMS: To examine the incidence of amputation in patients with type 2 diabetes mellitus (T2DM) treated with sodium glucose co-transporter 2 (SGLT2) inhibitors overall, and canagliflozin specifically, compared with non-SGLT2 inhibitor antihyperglycaemic agents (AHAs). MATERIALS AND METHODS: Patients with T2DM newly exposed to SGLT2 inhibitors or non-SGLT2 inhibitor AHAs were identified using the Truven MarketScan database. The incidence of below-knee lower extremity (BKLE) amputation was calculated for patients treated with SGLT2 inhibitors, canagliflozin, or non-SGLT2 inhibitor AHAs. Patients newly exposed to canagliflozin and non-SGLT2 inhibitor AHAs were matched 1:1 on propensity scores, and a Cox proportional hazards model was used for comparative analysis. Negative controls (outcomes not believed to be associated with any AHA) were used to calibrate P values. RESULTS: Between April 1, 2013 and October 31, 2016, 118 018 new users of SGLT2 inhibitors, including 73 024 of canagliflozin, and 226 623 new users of non-SGLT2 inhibitor AHAs were identified. The crude incidence rates of BKLE amputation were 1.22, 1.26 and 1.87 events per 1000 person-years with SGLT2 inhibitors, canagliflozin and non-SGLT2 inhibitor AHAs, respectively. For the comparative analysis, 63 845 new users of canagliflozin were matched with 63 845 new users of non-SGLT2 inhibitor AHAs, resulting in well-balanced baseline covariates. The incidence rates of BKLE amputation were 1.18 and 1.12 events per 1000 person-years with canagliflozin and non-SGLT2 inhibitor AHAs, respectively; the hazard ratio was 0.98 (95% confidence interval 0.68-1.41; P = .92, calibrated P = .95). CONCLUSIONS: This real-world study observed no evidence of increased risk of BKLE amputation for new users of canagliflozin compared with non-SGLT2 inhibitor AHAs in a broad population of patients with T2DM.

Incidence of diabetic ketoacidosis among patients with type 2 diabetes mellitus treated with SGLT2 inhibitors and other antihyperglycemic agents.

AIMS: To estimate and compare incidence of diabetes ketoacidosis (DKA) among patients with type 2 diabetes who are newly treated with SGLT2 inhibitors (SGLT2i) versus non-SGLT2i antihyperglycemic agents (AHAs) in actual clinical practice. METHODS: A new-user cohort study design using a large insurance claims database in the US. DKA incidence was compared between new users of SGLT2i and new users of non-SGLT2i AHAs pair-matched on exposure propensity scores (EPS) using Cox regression models. RESULTS: Overall, crude incidence rates (95% CI) per 1000 patient-years for DKA were 1.69 (1.22-2.30) and 1.83 (1.58-2.10) among new users of SGLT2i (n=34,442) and non-SGLT2i AHAs (n=126,703). These rates more than doubled among patients with prior insulin prescriptions but decreased by more than half in analyses that excluded potential autoimmune diabetes (PAD). The hazard ratio (95% CI) for DKA comparing new users of SGLT2i to new users of non-SGLT2i AHAs was 1.91 (0.94-4.11) (p=0.09) among the 30,196 EPS-matched pairs overall, and 1.13 (0.43-3.00) (p=0.81) among the 27,515 EPS-matched pairs that excluded PAD. CONCLUSIONS: This was the first observational study that compared DKA risk between new users of SGLT2i and non-SGLT2i AHAs among patients with type 2 diabetes, and overall no statistically significant difference was detected.

Risk Prediction for Ischemic Stroke and Transient Ischemic Attack in Patients Without Atrial Fibrillation: A Retrospective Cohort Study.

BACKGROUND: Stroke mainly occurs in patients without atrial fibrillation (AF). This study explored risk prediction models for ischemic stroke and transient ischemic attack (TIA) in patients without AF. METHODS: Three US-based healthcare databases (Truven MarketScan Commercial Claims and Encounters [CCAE], Medicare Supplemental [MDCR], and Optum Clinformatics [Optum]) were used to establish patient cohorts without AF during the index period of 2008-2012. The performance of 2 existing models (CHADS2 and CHA2DS2-VASc) for predicting stroke and TIA was examined by fitting a logistic regression to a training dataset and evaluating predictive accuracy in a validation dataset (area under the curve, AUC) using patients with complete follow-up of 1 or 3 years, separately. RESULTS: The commercial populations were younger and had fewer comorbidities than Medicare-eligible population. The incidence proportions of ischemic stroke and TIA during 1 and 3 years of follow-up were .5% and 1.9% (CCAE), .6% and 2.2% (Optum), and 4.6% and 13.1% (MDCR), respectively. The models performed consistently across all 3 databases, with the AUC ranging from .69 to .77 and from .68 to .73 for 1- and 3-year prediction, respectively. Predictive accuracy was lower than the initial work of CHADS2 evaluation in patients with AF (AUC: .82), but consistent with a subsequent meta-analysis of CHADS2 (.60-.80) and CHA2DS2-VASc performance (.64-.79). CONCLUSION: Although the existing schemes for predicting ischemic stroke and TIA in patients with AF can be applied to patients without AF with comparable predictive accuracy, the evidence suggests that there is room for improvement in these models' performance.

Characterizing treatment pathways at scale using the OHDSI network.

Observational research promises to complement experimental research by providing large, diverse populations that would be infeasible for an experiment. Observational research can test its own clinical hypotheses, and observational studies also can contribute to the design of experiments and inform the generalizability of experimental research. Understanding the diversity of populations and the variance in care is one component. In this study, the Observational Health Data Sciences and Informatics (OHDSI) collaboration created an international data network with 11 data sources from four countries, including electronic health records and administrative claims data on 250 million patients. All data were mapped to common data standards, patient privacy was maintained by using a distributed model, and results were aggregated centrally. Treatment pathways were elucidated for type 2 diabetes mellitus, hypertension, and depression. The pathways revealed that the world is moving toward more consistent therapy over time across diseases and across locations, but significant heterogeneity remains among sources, pointing to challenges in generalizing clinical trial results. Diabetes favored a single first-line medication, metformin, to a much greater extent than hypertension or depression. About 10% of diabetes and depression patients and almost 25% of hypertension patients followed a treatment pathway that was unique within the cohort. Aside from factors such as sample size and underlying population (academic medical center versus general population), electronic health records data and administrative claims data revealed similar results. Large-scale international observational research is feasible.

Multisite Evaluation of a Data Quality Tool for Patient-Level Clinical Data Sets.

INTRODUCTION: Data quality and fitness for analysis are crucial if outputs of analyses of electronic health record data or administrative claims data should be trusted by the public and the research community. METHODS: We describe a data quality analysis tool (called Achilles Heel) developed by the Observational Health Data Sciences and Informatics Collaborative (OHDSI) and compare outputs from this tool as it was applied to 24 large healthcare datasets across seven different organizations. RESULTS: We highlight 12 data quality rules that identified issues in at least 10 of the 24 datasets and provide a full set of 71 rules identified in at least one dataset. Achilles Heel is a freely available software that provides a useful starter set of data quality rules with the ability to add additional rules. We also present results of a structured email-based interview of all participating sites that collected qualitative comments about the value of Achilles Heel for data quality evaluation. DISCUSSION: Our analysis represents the first comparison of outputs from a data quality tool that implements a fixed (but extensible) set of data quality rules. Thanks to a common data model, we were able to compare quickly multiple datasets originating from several countries in America, Europe and Asia.

Feasibility and utility of applications of the common data model to multiple, disparate observational health databases.

OBJECTIVES: To evaluate the utility of applying the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) across multiple observational databases within an organization and to apply standardized analytics tools for conducting observational research. MATERIALS AND METHODS: Six deidentified patient-level datasets were transformed to the OMOP CDM. We evaluated the extent of information loss that occurred through the standardization process. We developed a standardized analytic tool to replicate the cohort construction process from a published epidemiology protocol and applied the analysis to all 6 databases to assess time-to-execution and comparability of results. RESULTS: Transformation to the CDM resulted in minimal information loss across all 6 databases. Patients and observations excluded were due to identified data quality issues in the source system, 96% to 99% of condition records and 90% to 99% of drug records were successfully mapped into the CDM using the standard vocabulary. The full cohort replication and descriptive baseline summary was executed for 2 cohorts in 6 databases in less than 1 hour. DISCUSSION: The standardization process improved data quality, increased efficiency, and facilitated cross-database comparisons to support a more systematic approach to observational research. Comparisons across data sources showed consistency in the impact of inclusion criteria, using the protocol and identified differences in patient characteristics and coding practices across databases. CONCLUSION: Standardizing data structure (through a CDM), content (through a standard vocabulary with source code mappings), and analytics can enable an institution to apply a network-based approach to observational research across multiple, disparate observational health databases.

Applying standardized drug terminologies to observational healthcare databases: a case study on opioid exposure.

Observational healthcare databases represent a valuable resource for health economics, outcomes research, quality of care, drug safety, epidemiology and comparative effectiveness research. The methods used to identify a population for study in an observational healthcare database with the desired drug exposures of interest are complex and not consistent nor apparent in the published literature. Our research evaluates three drug classification systems and their impact on prevalence in the analysis of observational healthcare databases using opioids as a case in point. The standard terminologies compiled in the Observational Medical Outcomes Partnership's Common Data Model vocabulary were used to facilitate the identification of populations with opioid exposures. This study analyzed three distinct observational healthcare databases and identified patients with at least one exposure to an opioid as defined by drug codes derived through the application of three classification systems. Opioid code sets were created for each of the three classification systems and the number of identified codes was summarized. We estimated the prevalence of opioid exposure in three observational healthcare databases using the three defined code sets. In addition we compared the number of drug codes and distinct ingredients that were identified using these classification systems. We found substantial variation in the prevalence of opioid exposure identified using an individual classification system versus a composite method using multiple classification systems. To ensure transparent and reproducible research publications should include a description of the process used to develop code sets and the complete code set used in studies.