Potential impact of blood cholesterol guidelines on statin treatment in the U.S. population using interrupted time series analysis.

BACKGROUND: The 2013 ACC/AHA Guideline was a paradigm shift in lipid management and identified the four statin-benefit groups. Many have studied the guideline's potential impact, but few have investigated its potential long-term impact on MACE. Furthermore, most studies also ignored the confounding effect from the earlier release of generic atorvastatin in Dec 2011. METHODS: To evaluate the potential (long-term) impact of the 2013 ACC/AHA Guideline release in Nov 2013 in the U.S., we investigated the association of the 2013 ACC/AHA Guideline with the trend changes in 5-Year MACE survival and three other statin-related outcomes (statin use, optimal statin use, and statin adherence) while controlling for generic atorvastatin availability using interrupted time series analysis, called the Chow's test. Specifically, we conducted a retrospective study using U.S. nationwide de-identified claims and electronic health records from Optum Labs Database Warehouse (OLDW) to follow the trends of 5-Year MACE survival and statin-related outcomes among four statin-benefit groups that were identified in the 2013 ACC/AHA Guideline. Then, Chow's test was used to discern trend changes between generic atorvastatin availability and guideline potential impact. RESULTS: 197,021 patients were included (ASCVD: 19,060; High-LDL: 33,907; Diabetes: 138,159; High-ASCVD-Risk: 5,895). After the guideline release, the long-term trend (slope) of 5-Year MACE Survival for the Diabetes group improved significantly (P = 0.002). Optimal statin use for the ASCVD group also showed immediate improvement (intercept) and long-term positive changes (slope) after the release (P < 0.001). Statin uses did not have significant trend changes and statin adherence remained unchanged in all statin-benefit groups. Although no other statistically significant trend changes were found, overall positive trend change or no changes were observed after the 2013 ACC/AHA Guideline release. CONCLUSIONS: The 2013 ACA/AHA Guideline release is associated with trend improvements in the long-term MACE Survival for Diabetes group and optimal statin use for ASCVD group. These significant associations might indicate a potential positive long-term impact of the 2013 ACA/AHA Guideline on better health outcomes for primary prevention groups and an immediate potential impact on statin prescribing behaviors in higher-at-risk groups. However, further investigation is required to confirm the causal effect of the 2013 ACA/AHA Guideline.

Machine Learning for Risk Prediction of Recurrent AKI in Adult Patients After Hospital Discharge.

Recurrent AKI has been found common among hospitalized patients after discharge, and early prediction may allow timely intervention and optimized post-discharge treatment [1]. There are significant gaps in the literature regarding the risk prediction on the post-AKI population, and most current works only included a limited number of pre-selected variables [2]. In this study, we built and compared machine learning models using both knowledge-based and data-driven features in predicting the risk of recurrent AKI within 1-year of discharge. Our results showed that the additional use of data-driven features statistically improved the model performances, with best AUC=0.766 by using logistic regression.

Post-Implementation Outcomes of a Remote Patient Monitoring Program After Emergency Department Discharge.

To better communicate and improve post-visit outcomes, a remote patient monitoring (RPM) program was implemented for patients discharged from emergency departments (ED) across 10 hospitals. The solution was offered to patients at the time of ED discharge and staffed by a group of care coordinators to respond to questions/urgent needs. Of 107,477 consecutive patients offered RPM, 28,425 patients (26.4%) engaged with the program. Activated patients with RPM were less likely to return to the ED within 90 days of their index visit [19.8% compared to 23.6%, p<.001]. While activation rates were modest, we observed fewer return visits to the ED in patients using RPM, with a 16.2% lower hazard of returning in the next year. Future research is needed to understand methods to improve RPM activation, any causal effects of RPM activation on return ED visits, and external validation of these findings.

Associations Between Psychosocial Needs, Carbohydrate-Counting Behavior, and App Satisfaction: A Randomized Crossover App Trial on 92 Adults With Diabetes.

To examine whether psychosocial needs in diabetes care are associated with carbohydrate counting and if carbohydrate counting is associated with satisfaction with diabetes applications' usability, a randomized crossover trial of 92 adults with type 1 or 2 diabetes requiring insulin therapy tested two top-rated diabetes applications, mySugr and OnTrack Diabetes. Survey responses on demographics, psychosocial needs (perceived competence, autonomy, and connectivity), carbohydrate-counting frequency, and application satisfaction were modeled using mixed-effect linear regressions to test associations. Participants ranged between 19 and 74 years old (mean, 54 years) and predominantly had type 2 diabetes (70%). Among the three tested domains of psychosocial needs, only competence-not autonomy or connectivity-was found to be associated with carbohydrate-counting frequency. No association between carbohydrate-counting behavior and application satisfaction was found. In conclusion, perceived competence in diabetes care is an important factor in carbohydrate counting; clinicians may improve adherence to carbohydrate counting with strategies designed to improve perceived competence. Carbohydrate-counting behavior is complex; its impact on patient satisfaction of diabetes application usability is multifactorial and warrants consideration of patient demographics such as sex as well as application features for automated carbohydrate counting.

Evaluation of four machine learning models for signal detection.

Background: Logistic regression-based signal detection algorithms have benefits over disproportionality analysis due to their ability to handle potential confounders and masking factors. Feature exploration and developing alternative machine learning algorithms can further strengthen signal detection. Objectives: Our objective was to compare the signal detection performance of logistic regression, gradient-boosted trees, random forest and support vector machine models utilizing Food and Drug Administration adverse event reporting system data. Design: Cross-sectional study. Methods: The quarterly data extract files from 1 October 2017 through 31 December 2020 were downloaded. Due to an imbalanced outcome, two training sets were used: one stratified on the outcome variable and another using Synthetic Minority Oversampling Technique (SMOTE). A crude model and a model with tuned hyperparameters were developed for each algorithm. Model performance was compared against a reference set using accuracy, precision, F1 score, recall, the receiver operating characteristic area under the curve (ROCAUC), and the precision-recall curve area under the curve (PRCAUC). Results: Models trained on the balanced training set had higher accuracy, F1 score and recall compared to models trained on the SMOTE training set. When using the balanced training set, logistic regression, gradient-boosted trees, random forest and support vector machine models obtained similar performance evaluation metrics. The gradient-boosted trees hyperparameter tuned model had the highest ROCAUC (0.646) and the random forest crude model had the highest PRCAUC (0.839) when using the balanced training set. Conclusion: All models trained on the balanced training set performed similarly. Logistic regression models had higher accuracy, precision and recall. Logistic regression, random forest and gradient-boosted trees hyperparameter tuned models had a PRCAUC ⩾ 0.8. All models had an ROCAUC ⩾ 0.5. Including both disproportionality analysis results and additional case report information in models resulted in higher performance evaluation metrics than disproportionality analysis alone.

Personalized statin treatment plan using counterfactual approach with multi-objective optimization over benefits and risks.

Background: Statins are a class of drugs that lower cholesterol levels in the blood by inhibiting an enzyme called 3-hydroxy-3-methyl-glutaryl-coenzyme A (HMG-CoA) reductase. High cholesterol levels can lead to plaque buildup in the arteries, which can cause Atherosclerotic Cardiovascular Disease(ASCVD). Statins can reduce the risk of ASCVD events by about 25-35% but they might be associated with symptoms such as muscle pain, liver damage, or diabetes. As a result, this leads to a strong reason to discontinue statin therapy, which increases the risk of cardiovascular events and mortality and becomes a public-health problem.To solve this problem, in the previous work, we proposed a framework to produce a proactive strategy, called a personalized statin treatment plan (PSTP) to minimize the risks of statin-associated symptoms and therapy discontinuation when prescribing statin. In our previous PSTP framework, three limitations remain, and they can influence PSTP usability: (1) Not taking the counterfactual predictions and confounding bias into account. (2) The balance between multiple drug-prescribing objectives (especially trade-off objectives), such as tradeoff between benefits and risks. (3) Evaluating PSTP in retrospective data. Objectives: This manuscript aimed to provide solutions for the three abovementioned problems to improve PSTP robustness to produce a proactive strategy for statin prescription that can maximize the benefits (low-density lipoprotein cholesterol (LDL-C) reduction) and minimize risks (statin-associated symptoms and therapy discontinuation) at the same time. Methods: We applied overlapping weighting counterfactual survival risk prediction (CP), multiple objective optimization (MOO), and clinical trial simulation (CTS) which consists of Random Arms, Clinical Guideline arms, PSTP Arms, and Practical Arms to improve the PSTP framework and usability. Results: In addition to highly balanced covariates, in the CTS, the revised PSTP showed improvements in lowering the SAS risks overall compared to other arms across all time points by at most 7.5% to at least 1.0% (Fig. 8(a)). It also has the better flexibility of identifying the optimal Statin across all time points within one year. Conclusion: We demonstrated feasibility of robust and trustworthy counterfactual survival risk prediction model. In CTS, we also demonstrated the PSTP with Pareto optimization can personalize optimal balance between Statin benefits and risks.

Advantages and disadvantages of using theory-based versus data-driven models with social and behavioral determinants of health data.

OBJECTIVE: Theory-based research of social and behavioral determinants of health (SBDH) found SBDH-related patterns in interventions and outcomes for pregnant/birthing people. The objectives of this study were to replicate the theory-based SBDH study with a new sample, and to compare these findings to a data-driven SBDH study. MATERIALS AND METHODS: Using deidentified public health nurse-generated Omaha System data, 2 SBDH indices were computed separately to create groups based on SBDH (0-5+ signs/symptoms). The data-driven SBDH index used multiple linear regression with backward elimination to identify SBDH factors. Changes in Knowledge, Behavior, and Status (KBS) outcomes, numbers of interventions, and adjusted R-squared statistics were computed for both models. RESULTS: There were 4109 clients ages 13-40 years. Outcome patterns aligned with the original research: KBS increased from admission to discharge with Knowledge improving the most; discharge KBS decreased as SBDH increased; and interventions increased as SBDH increased. Slopes of the data-driven model were steeper, showing clearer KBS trends for data-driven SBDH groups. The theory-based model adjusted R-squared was 0.54 (SE = 0.38) versus 0.61 (SE = 0.35) for the data-driven model with an entirely different set of SBDH factors. CONCLUSIONS: The theory-based approach provided a framework to identity patterns and relationships and may be applied consistently across studies and populations. In contrast, the data-driven approach can provide insights based on novel patterns for a given dataset and reveal insights and relationships not predicted by existing theories. Data-driven methods may be an advantage if there is sufficiently comprehensive SBDH data upon which to create the data-driven models.

Performance of subgrouped proportional reporting ratios in the US Food and Drug Administration (FDA) adverse event reporting system.

BACKGROUND: Many signal detection algorithms give the same weight to information from all products and patients, which may result in signals being masked or false positives being flagged as potential signals. Subgrouped analysis can be used to help correct for this. RESEARCH DESIGN AND METHODS: The publicly available US Food and Drug Administration Adverse Event Reporting System quarterly data extract files from 1 January 2015 through 30 September 2017 were utilized. A proportional reporting ratio (PRR) analysis subgrouped by either age, sex, ADE report type, seriousness of ADE, or reporter was compared to the crude PRR analysis using sensitivity, specificity, precision, and c-statistic. RESULTS: Subgrouping by age (n = 78, 34.5% increase), sex (n = 67, 15.5% increase), and reporter (n = 64, 10.3% increase) identified more signals than the crude analysis. Subgrouping by either age or sex increased both the sensitivity and precision. Subgrouping by report type or seriousness resulted in fewer signals (n = 50, -13.8% for both). Subgrouped analyses had higher c-statistic values, with age having the highest (0.468). CONCLUSIONS: Subgrouping by either age or sex produced more signals with higher sensitivity and precision than the crude PRR analysis. Subgrouping by these variables can unmask potentially important associations.

App Design Features Important for Diabetes Self-management as Determined by the Self-Determination Theory on Motivation: Content Analysis of Survey Responses From Adults Requiring Insulin Therapy.

BACKGROUND: Using a diabetes app can improve glycemic control; however, the use of diabetes apps is low, possibly due to design issues that affect patient motivation. OBJECTIVE: This study aimed to describes how adults with diabetes requiring insulin perceive diabetes apps based on 3 key psychological needs (competence, autonomy, and connectivity) described by the Self-Determination Theory (SDT) on motivation. METHODS: This was a qualitative analysis of data collected during a crossover randomized laboratory trial (N=92) testing 2 diabetes apps. Data sources included (1) observations during app testing and (2) survey responses on desired app features. Guided by the SDT, coding categories included app functions that could address psychological needs for motivation in self-management: competence, autonomy, and connectivity. RESULTS: Patients described design features that addressed needs for competence, autonomy, and connectivity. To promote competence, electronic data recording and analysis should help patients track and understand blood glucose (BG) results necessary for planning behavior changes. To promote autonomy, BG trend analysis should empower patients to set safe and practical personalized behavioral goals based on time and the day of the week. To promote connectivity, app email or messaging function could share data reports and communicate with others on self-management advice. Additional themes that emerged are the top general app designs to promote positive user experience: patient-friendly; automatic features of data upload; voice recognition to eliminate typing data; alert or reminder on self-management activities; and app interactivity of a sound, message, or emoji change in response to keeping or not keeping BG in the target range. CONCLUSIONS: The application of the SDT was useful in identifying motivational app designs that address the psychological needs of competence, autonomy, and connectivity. User-centered design concepts, such as being patient-friendly, differ from the SDT because patients need a positive user experience (ie, a technology need). Patients want engaging diabetes apps that go beyond data input and output. Apps should be easy to use, provide personalized analysis reports, be interactive to affirm positive behaviors, facilitate data sharing, and support patient-clinician communication.

Decision rules for personalized statin treatment prescriptions over multi-objectives.

In our previous study, we demonstrated the feasibility of producing a proactive statin prescription strategy - a personalized statin treatment plan (PSTP) - using neural networks with big data. However, its non-transparency limited result interpretations and clinical usability. To improve the transparency of our previous approach with minimal compromise to the maximal statin treatment benefit-to-risk ratio, this study proposed a five-step pipeline approach called the decision rules for statin treatment (DRST). Steps 1-3 of our proposed pipeline improved our previous PSTP model in optimizing individual benefit-to-risk ratio; Step 4 used a decision tree model (DRST) to provide straightforward rules in the initial statin treatment plan; Step 5 aimed to evaluate the efficacy of these decision rules by conducting a clinical trial simulation. We included 107,739 de-identified patient data from Optum Labs Database Warehouse in this study. The final decision rules were compact and efficient, resulting from a decision tree with only a maximum depth of 3 and 11 nodes. The DRST identified three factors that are easily obtainable at the point of care: age, low-density lipoprotein cholesterol (LDL-C) level, and age-adjusted Charlson score. Moreover, it also identified six subpopulations that can benefit most from these decision rules. In our clinical trial simulations, DRST was found to improve statin benefit in LDL-C reduction by 4.15 percentage points (pp) and reduce risks of statin-associated symptoms (SAS) and statin discontinuation by 11.71 and 3.96 pp, respectively, when compared to the standard of care. Moreover, these DRST results were only less than 0.6 pp suboptimal to PSTP, demonstrating that building DRST that provide transparency with minimal compromise to the maximal benefit-to-risk ratio of statin treatments is feasible.

Clinical Data Cohort Quality Improvement: The Case of the Medication Data in The University of Minnesota's Clinical Data Repository.

Clinical and translational research centers (CTRCs) have emerged as key centers for electronic medical record related research through integrated data repositories (IDRs) and the 'secondary use' of clinical data. Researchers accessing and pre-processing ever increasing amounts of electronic medical records for data mining tasks have a growing need for best practice approaches for clinical data quality assessment and improvement. This project focused on a large data extract for 7 statin medication prescriptions for patients with cardiovascular disease. After the initial data extraction, we proceeded to analyze the data for completeness, correctness, currency, and percentage populated using established data quality frameworks. Assessment of the said data was performed through medication possession ratios, medication discontinuation reasons, and drug dosages. When we compared distributions of data elements such as drug dosage before and after changes were introduced by our pre-processing protocols, only a minimal noticeable difference was found as the clinical data cohort quality assessment and pre-processing were completed without substantially altering the original data structure. Our study demonstrated practical steps for clinical data cohort quality improvement using medication data and illustrates a best practice approach in clinical data cohort quality improvement for any data mining tasks.

Assessing the use of prescription drugs and dietary supplements in obese respondents in the National Health and Nutrition Examination Survey.

INTRODUCTION: Obesity is a common disease and a known risk factor for many other conditions such as hypertension, type 2 diabetes, and cancer. Treatment options for obesity include lifestyle changes, pharmacotherapy, and surgical interventions such as bariatric surgery. In this study, we examine the use of prescription drugs and dietary supplements by the individuals with obesity. METHODS: We conducted a cross-sectional analysis of the National Health and Nutrition Examination Survey (NHANES) data 2003-2018. We used multivariate logistic regression to analyze the correlations of demographics and obesity status with the use of prescription drugs and dietary supplement use. We also built machine learning models to classify prescription drug and dietary supplement use using demographic data and obesity status. RESULTS: Individuals with obesity are more likely to take cardiovascular agents (OR = 2.095, 95% CI 1.989-2.207) and metabolic agents (OR = 1.658, 95% CI 1.573-1.748) than individuals without obesity. Gender, age, race, poverty income ratio, and insurance status are significantly correlated with dietary supplement use. The best performing model for classifying prescription drug use had the accuracy of 74.3% and the AUROC of 0.82. The best performing model for classifying dietary supplement use had the accuracy of 65.3% and the AUROC of 0.71. CONCLUSIONS: This study can inform clinical practice and patient education of the use of prescription drugs and dietary supplements and their correlation with obesity.

Discovering novel drug-supplement interactions using SuppKG generated from the biomedical literature.

OBJECTIVE: Develop a novel methodology to create a comprehensive knowledge graph (SuppKG) to represent a domain with limited coverage in the Unified Medical Language System (UMLS), specifically dietary supplement (DS) information for discovering drug-supplement interactions (DSI), by leveraging biomedical natural language processing (NLP) technologies and a DS domain terminology. MATERIALS AND METHODS: We created SemRepDS (an extension of an NLP tool, SemRep), capable of extracting semantic relations from abstracts by leveraging a DS-specific terminology (iDISK) containing 28,884 DS terms not found in the UMLS. PubMed abstracts were processed using SemRepDS to generate semantic relations, which were then filtered using a PubMedBERT model to remove incorrect relations before generating SuppKG. Two discovery pathways were applied to SuppKG to identify potential DSIs, which are then compared with an existing DSI database and also evaluated by medical professionals for mechanistic plausibility. RESULTS: SemRepDS returned 158.5% more DS entities and 206.9% more DS relations than SemRep. The fine-tuned PubMedBERT model (significantly outperformed other machine learning and BERT models) obtained an F1 score of 0.8605 and removed 43.86% of semantic relations, improving the precision of the relations by 26.4% over pre-filtering. SuppKG consists of 56,635 nodes and 595,222 directed edges with 2,928 DS-specific nodes and 164,738 edges. Manual review of findings identified 182 of 250 (72.8%) proposed DS-Gene-Drug and 77 of 100 (77%) proposed DS-Gene1-Function-Gene2-Drug pathways to be mechanistically plausible. DISCUSSION: With added DS terminology to the UMLS, SemRepDS has the capability to find more DS-specific semantic relationships from PubMed than SemRep. The utility of the resulting SuppKG was demonstrated using discovery patterns to find novel DSIs. CONCLUSION: For the domain with limited coverage in the traditional terminology (e.g., UMLS), we demonstrated an approach to leverage domain terminology and improve existing NLP tools to generate a more comprehensive knowledge graph for the downstream task. Even this study focuses on DSI, the method may be adapted to other domains.

Producing personalized statin treatment plans to optimize clinical outcomes using big data and machine learning.

Almost half of Americans 65 years of age and older take statins, which are highly effective in lowering low-density lipoprotein cholesterol, preventing atherosclerotic cardiovascular disease (ASCVD), and reducing all-cause mortality. Unfortunately, ∼50% of patients prescribed statins do not obtain these critical benefits because they discontinue use within one year of treatment initiation. Therefore, statin discontinuation has been identified as a major public health concern due to the increased morbidity, mortality, and healthcare costs associated with ASCVD. In clinical practice, statin-associated symptoms (SAS) often result in dose reduction or discontinuation of these life-saving medications. Currently, physician decision-making in statin prescribing typically relies on only a few patient data elements. Physicians then employ reactive strategies to manage SAS concerns after they manifest (e.g., offering an alternative statin treatment plan or a statin holiday). A preferred approach would be a proactive strategy to identify the optimal treatment plan (statin agent + dosage) to prevent/minimize SAS and statin discontinuation risks for a particular individual prior to initiating treatment. Given that using a single patient's data to identify the optimal statin regimen is inadequate to ensure that the harms of statin use are minimized, alternative tactics must be used to address this problem. In this proof-of-concept study, we explore the use of a machine-learning personalized statin treatment plan (PSTP) platform to assess the numerous statin treatment plans available and identify the optimal treatment plan to prevent/minimize harms (SAS and statin discontinuation) for an individual. Our study leveraged de-identified administrative insurance claims data from the OptumLabs® Data Warehouse, which includes medical and pharmacy claims, laboratory results, and enrollment records for more than 130 million commercial and Medicare Advantage (MA) enrollees, to successfully develop the PSTP platform. In this study, we found three results: (1) the PSTP platform recommends statin prescription with significantly lower risks of SAS and discontinuation compared with standard-practice, (2) because machine learning can consider many more dimensions of data, the performance of the proactive prescription strategy with machine-learning support is better, especially the artificial neural network approach, and (3) we demonstrate a method of incorporating optimization constraints for individualized patient-centered medicine and shared decision making. However, more research into its clinical use is needed. These promising results show the feasibility of using machine learning and big data approaches to produce personalized healthcare treatment plans and support the precision-health agenda.

Clinical Data Repositories: Primary Sources of Drug-to-Drug Interaction Detection and Risk Assessment.

Background: Polypharmacy can be a source of adverse drug events including those caused by drug to drug interaction (DDI) exposures. Web-based DDI databases are available to researchers for the identification of potential DDI exposures. Rather than relying on potentially incomplete DDI databases, large clinical data repositories (CDR) which are integrated data sources fed with millions of heterogeneous electronic health records (EHRs) containing real-world data should be leveraged for data driven DDI identification. Objective: To explore and validate the viability of clinical data repositories as data driven resources for clinically important adverse drug events detection and surveillance. Methods: This work leverages a minimum clinical data set from the University of Minnesota's CDR to identify drugs that have statin to drug interaction (SDI) potential and compares the findings with results of web based DDI databases. Using an SDI identification matrix, we identified several potential novel SDI drugs that were not mentioned in the web-based sources but explored through our study as drugs with SDI potential. Results: Drugs flagged by our SDI identification matrix but not mentioned in the web-based sources include Lysine, Ketotifen, Latanoprost, Methylcellulose, Oxazepam, Linseed Oil, and others. Conclusion: Our findings identified potential gaps regarding the completeness, currency, and overall reliability of open source and commercial DDI databases. CDRs can be a primary source for identifying drug to drug interactions. Keywords: clinical data repository, drug to drug interaction databases, drug to drug interaction, statin to drug interaction, polypharmacy, statin to drug interaction identification matrix, adverse drug event, statin.

Application of Causal Discovery Algorithms in Studying the Nephrotoxicity of Remdesivir Using Longitudinal Data from the EHR.

Remdesivir has been widely used for the treatment of Coronavirus (COVID) in hospitalized patients, but its nephrotoxicity is still under investigation1. Given the paucity of knowledge regarding the mechanism and optimal treatment of the development of acute kidney injury (AKI) in the setting of COVID, we analyzed the role of remdesivir and built multifactorial causal models of COVID-AKI by applying causal discovery machine learning techniques. Risk factors of COVID-AKI and renal function measures were represented in a temporal sequence using longitudinal data from EHR. Our models successfully recreated known causal pathways to changes in renal function and interactions with each other and examined the consistency of high-level causal relationships over a 4-day course of remdesivir. Results indicated a need for assessment of renal function on day 2 and 3 use of remdesivir, while uncovering that remdesivir may pose less risk to AKI than existing conditions of chronic kidney disease.

Claims-Based Score for the Prediction of Bleeding in a Contemporary Cohort of Patients Receiving Oral Anticoagulation for Venous Thromboembolism.

Background Current scores for bleeding risk assessment in patients with venous thromboembolism (VTE) undergoing oral anticoagulation have limited predictive capacity. We developed and internally validated a bleeding prediction model using healthcare claims data. Methods and Results We selected patients with incident VTE initiating oral anticoagulation in the 2011 to 2017 MarketScan databases. Hospitalized bleeding events were identified using validated algorithms in the 180 days after VTE diagnosis. We evaluated demographic factors, comorbidities, and medication use before oral anticoagulation initiation as potential predictors of bleeding using stepwise selection of variables in Cox models run on 1000 bootstrap samples of the patient population. Variables included in >60% of all models were selected for the final analysis. We internally validated the model using bootstrapping and correcting for optimism. We included 165 434 patients with VTE and initiating oral anticoagulation, of whom 2294 had a bleeding event. After undergoing the variable selection process, the final model included 20 terms (15 main effects and 5 interactions). The c-statistic for the final model was 0.68 (95% CI, 0.67-0.69). The internally validated c-statistic corrected for optimism was 0.68 (95% CI, 0.67-0.69). For comparison, the c-statistic of the Hypertension, Abnormal Renal/Liver Function, Stroke, Bleeding History or Predisposition, Labile International Normalized Ratio, Elderly (>65 Years), Drugs/Alcohol Concomitantly (HAS-BLED) score in this population was 0.62 (95% CI, 0.61-0.63). Conclusions We have developed a novel model for bleeding prediction in VTE using large healthcare claims databases. Performance of the model was moderately good, highlighting the urgent need to identify better predictors of bleeding to inform treatment decisions.

When text simplification is not enough: could a graph-based visualization facilitate consumers' comprehension of dietary supplement information?

OBJECTIVE: Dietary supplements are widely used. However, dietary supplements are not always safe. For example, an estimated 23 000 emergency room visits every year in the United States were attributed to adverse events related to dietary supplement use. With the rapid development of the Internet, consumers usually seek health information including dietary supplement information online. To help consumers access quality online dietary supplement information, we have identified trustworthy dietary supplement information sources and built an evidence-based knowledge base of dietary supplement information-the integrated DIetary Supplement Knowledge base (iDISK) that integrates and standardizes dietary supplement related information across these different sources. However, as information in iDISK was collected from scientific sources, the complex medical jargon is a barrier for consumers' comprehension. The objective of this study is to assess how different approaches to simplify and represent dietary supplement information from iDISK will affect lay consumers' comprehension. MATERIALS AND METHODS: Using a crowdsourcing platform, we recruited participants to read dietary supplement information in 4 different representations from iDISK: (1) original text, (2) syntactic and lexical text simplification (TS), (3) manual TS, and (4) a graph-based visualization. We then assessed how the different simplification and representation strategies affected consumers' comprehension of dietary supplement information in terms of accuracy and response time to a set of comprehension questions. RESULTS: With responses from 690 qualified participants, our experiments confirmed that the manual approach, as expected, had the best performance for both accuracy and response time to the comprehension questions, while the graph-based approach ranked the second outperforming other representations. In some cases, the graph-based representation outperformed the manual approach in terms of response time. CONCLUSIONS: A hybrid approach that combines text and graph-based representations might be needed to accommodate consumers' different information needs and information seeking behavior.

Content Analysis: First-Time Patient User Challenges with Top-Rated Commercial Diabetes Apps.

Background/Introduction: Using a mobile application (app) may improve diabetes self-management. However, the use of diabetes apps is low, possibly due to design and usability issues. The purpose of this study was to identify barriers to app use among adult patients with diabetes who were testing diabetes apps for the first time. Materials and Methods: We conducted a content analysis of observation notes and patient comments collected during the testing of two top commercially available diabetes apps as part of a crossover randomized trial. Participants were adult patients with type 1 or type 2 diabetes on insulin therapy. We analyzed field notes and transcriptions of audio recordings. Open coding derived categories of usability issues, which then were grouped into themes and subthemes on usability problem types. Results: A total of 92 adult Android smartphone users were recruited online (e.g., Facebook) and in-person postings. Three major themes described problems with data input, app report display and presentation, and self-learning options. Data entry modes were problematic because of overcrowded app screens, complicated "save data" steps, and a lack of data entry confirmation. The app icons, wording, entry headings, and analysis reports were not intuitive to understand. Participants wanted self-learning options (e.g., pop-up messages) during app use. Conclusions: Patient testing of top commercially available diabetes apps revealed key usability design issues in data entry, app report, and self-help learning options. Good app training for patients is necessary for both initial use and long-term use of diabetes apps to support self-management.

Assessing the enrichment of dietary supplement coverage in the Unified Medical Language System.

OBJECTIVE: We sought to assess the need for additional coverage of dietary supplements (DS) in the Unified Medical Language System (UMLS) by investigating (1) the overlap between the integrated DIetary Supplements Knowledge base (iDISK) DS ingredient terminology and the UMLS and (2) the coverage of iDISK and the UMLS over DS mentions in the biomedical literature. MATERIALS AND METHODS: We estimated the overlap between iDISK and the UMLS by mapping iDISK to the UMLS using exact and normalized strings. The coverage of iDISK and the UMLS over DS mentions in the biomedical literature was evaluated via a DS named-entity recognition (NER) task within PubMed abstracts. RESULTS: The coverage analysis revealed that only 30% of iDISK terms can be matched to the UMLS, although these cover over 99% of iDISK concepts. A manual review revealed that a majority of the unmatched terms represented new synonyms, rather than lexical variants. For NER, iDISK nearly doubles the precision and achieves a higher F1 score than the UMLS, while maintaining a competitive recall. DISCUSSION: While iDISK has significant concept overlap with the UMLS, it contains many novel synonyms. Furthermore, almost 3000 of these overlapping UMLS concepts are missing a DS designation, which could be provided by iDISK. The NER experiments show that the specialization of iDISK is useful for identifying DS mentions. CONCLUSIONS: Our results show that the DS representation in the UMLS could be enriched by adding DS designations to many concepts and by adding new synonyms.