A flexible symbolic regression method for constructing interpretable clinical prediction models.

Machine learning (ML) models trained for triggering clinical decision support (CDS) are typically either accurate or interpretable but not both. Scaling CDS to the panoply of clinical use cases while mitigating risks to patients will require many ML models be intuitively interpretable for clinicians. To this end, we adapted a symbolic regression method, coined the feature engineering automation tool (FEAT), to train concise and accurate models from high-dimensional electronic health record (EHR) data. We first present an in-depth application of FEAT to classify hypertension, hypertension with unexplained hypokalemia, and apparent treatment-resistant hypertension (aTRH) using EHR data for 1200 subjects receiving longitudinal care in a large healthcare system. FEAT models trained to predict phenotypes adjudicated by chart review had equivalent or higher discriminative performance (p < 0.001) and were at least three times smaller (p < 1 × 10-6) than other potentially interpretable models. For aTRH, FEAT generated a six-feature, highly discriminative (positive predictive value = 0.70, sensitivity = 0.62), and clinically intuitive model. To assess the generalizability of the approach, we tested FEAT on 25 benchmark clinical phenotyping tasks using the MIMIC-III critical care database. Under comparable dimensionality constraints, FEAT's models exhibited higher area under the receiver-operating curve scores than penalized linear models across tasks (p < 6 × 10-6). In summary, FEAT can train EHR prediction models that are both intuitively interpretable and accurate, which should facilitate safe and effective scaling of ML-triggered CDS to the panoply of potential clinical use cases and healthcare practices.

From benchmark to bedside: transfer learning from social media to patient-provider text messages for suicide risk prediction.

OBJECTIVE: Compared to natural language processing research investigating suicide risk prediction with social media (SM) data, research utilizing data from clinical settings are scarce. However, the utility of models trained on SM data in text from clinical settings remains unclear. In addition, commonly used performance metrics do not directly translate to operational value in a real-world deployment. The objectives of this study were to evaluate the utility of SM-derived training data for suicide risk prediction in a clinical setting and to develop a metric of the clinical utility of automated triage of patient messages for suicide risk. MATERIALS AND METHODS: Using clinical data, we developed a Bidirectional Encoder Representations from Transformers-based suicide risk detection model to identify messages indicating potential suicide risk. We used both annotated and unlabeled suicide-related SM posts for multi-stage transfer learning, leveraging customized contemporary learning rate schedules. We also developed a novel metric estimating predictive models' potential to reduce follow-up delays with patients in distress and used it to assess model utility. RESULTS: Multi-stage transfer learning from SM data outperformed baseline approaches by traditional classification performance metrics, improving performance from 0.734 to a best F1 score of 0.797. Using this approach for automated triage could reduce response times by 15 minutes per urgent message. DISCUSSION: Despite differences in data characteristics and distribution, publicly available SM data benefit clinical suicide risk prediction when used in conjunction with contemporary transfer learning techniques. Estimates of time saved due to automated triage indicate the potential for the practical impact of such models when deployed as part of established suicide prevention interventions. CONCLUSIONS: This work demonstrates a pathway for leveraging publicly available SM data toward improving risk assessment, paving the way for better clinical care and improved clinical outcomes.

Automated Detection of Cognitive Distortions in Text Exchanges Between Clinicians and People With Serious Mental Illness.

OBJECTIVE: The authors tested whether natural language processing (NLP) methods can detect and classify cognitive distortions in text messages between clinicians and people with serious mental illness as effectively as clinically trained human raters. METHODS: Text messages (N=7,354) were collected from 39 clients in a randomized controlled trial of a 12-week texting intervention. Clinical annotators labeled messages for common cognitive distortions: mental filtering, jumping to conclusions, catastrophizing, "should" statements, and overgeneralizing. Multiple NLP classification methods were applied to the same messages, and performance was compared. RESULTS: A tuned model that used bidirectional encoder representations from transformers (F1=0.62) achieved performance comparable to that of clinical raters in classifying texts with any distortion (F1=0.63) and superior to that of other models. CONCLUSIONS: NLP methods can be used to effectively detect and classify cognitive distortions in text exchanges, and they have the potential to inform scalable automated tools for clinical support during message-based care for people with serious mental illness.

Backdoor Adjustment of Confounding by Provenance for Robust Text Classification of Multi-institutional Clinical Notes.

Natural Language Processing (NLP) methods have been broadly applied to clinical tasks. Machine learning and deep learning approaches have been used to improve the performance of clinical NLP. However, these approaches require sufficiently large datasets for training, and trained models have been shown to transfer poorly across sites. These issues have led to the promotion of data collection and integration across different institutions for accurate and portable models. However, this can introduce a form of bias called confounding by provenance. When source-specific data distributions differ at deployment, this may harm model performance. To address this issue, we evaluate the utility of backdoor adjustment for text classification in a multi-site dataset of clinical notes annotated for mentions of substance abuse. Using an evaluation framework devised to measure robustness to distributional shifts, we assess the utility of backdoor adjustment. Our results indicate that backdoor adjustment can effectively mitigate for confounding shift.

Distinct Roles of Glutamine Metabolism in Benign and Malignant Cartilage Tumors With IDH Mutations.

Enchondromas and chondrosarcomas are common cartilage neoplasms that are either benign or malignant, respectively. The majority of these tumors harbor mutations in either IDH1 or IDH2. Glutamine metabolism has been implicated as a critical regulator of tumors with IDH mutations. Using genetic and pharmacological approaches, we demonstrated that glutaminase-mediated glutamine metabolism played distinct roles in enchondromas and chondrosarcomas with IDH1 or IDH2 mutations. Glutamine affected cell differentiation and viability in these tumors differently through different downstream metabolites. During murine enchondroma-like lesion development, glutamine-derived α-ketoglutarate promoted hypertrophic chondrocyte differentiation and regulated chondrocyte proliferation. Deletion of glutaminase in chondrocytes with Idh1 mutation increased the number and size of enchondroma-like lesions. In contrast, pharmacological inhibition of glutaminase in chondrosarcoma xenografts reduced overall tumor burden partially because glutamine-derived non-essential amino acids played an important role in preventing cell apoptosis. This study demonstrates that glutamine metabolism plays different roles in tumor initiation and cancer maintenance. Supplementation of α-ketoglutarate and inhibiting GLS may provide a therapeutic approach to suppress enchondroma and chondrosarcoma tumor growth, respectively. © 2022 The Authors. Journal of Bone and Mineral Research published by Wiley Periodicals LLC on behalf of American Society for Bone and Mineral Research (ASBMR).

Predicting Drug Blood-Brain Barrier Penetration with Adverse Event Report Embeddings.

Adverse event reports (AER) are widely used for post-market drug safety surveillance and drug repurposing, with the assumption that drugs with similar side-effects may have similar therapeutic effects also. In this study, we used distributed representations of drugs derived from the Food and Drug Administration (FDA) AER system using aer2vec, a method of representing AER, with drug embeddings emerging from a neural network trained to predict the probability of adverse drug effects given observed drugs. We combined these representations with molecular features to predict permeability of the blood-brain barrier to drugs, a prerequisite to their application to treat conditions of the central nervous system. Across multiple machine learning classifiers, the addition of distributed representations improved performance over prior methods using drug-drug similarity estimates derived from discrete representations of AER system data. Embedding-based approaches outperformed those using discrete statistics, with improvements in absolute AUC of 5% and 9%, corresponding to improvements of 9% and 13% over performance with molecular features only. Performance was retained when reducing embedding dimensions from 500 to 6, indicating that they are neither attributable to overfitting, nor to a difference in the number of trainable parameters. These results indicate that aer2vec distributed representations carry information that is valuable for drug repurposing.

Augmenting aer2vec: Enriching distributed representations of adverse event report data with orthographic and lexical information.

Adverse Drug Events (ADEs) are prevalent, costly, and sometimes preventable. Post-marketing drug surveillance aims to monitor ADEs that occur after a drug is released to market. Reports of such ADEs are aggregated by reporting systems, such as the Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS). In this paper, we consider the topic of how best to represent data derived from reports in FAERS for the purpose of detecting post-marketing surveillance signals, in order to inform regulatory decision making. In our previous work, we developed aer2vec, a method for deriving distributed representations (concept embeddings) of drugs and side effects from ADE reports, establishing the utility of distributional information for pharmacovigilance signal detection. In this paper, we advance this line of research further by evaluating the utility of encoding orthographic and lexical information. We do so by adapting two Natural Language Processing methods, subword embedding and vector retrofitting, which were developed to encode such information into word embeddings. Models were compared for their ability to distinguish between positive and negative examples in a set of manually curated drug/ADE relationships, with both aer2vec enhancements offering advantages in performances over baseline models, and best performance obtained when retrofitting and subword embeddings were applied in concert. In addition, this work demonstrates that models leveraging distributed representations do not require extensive manual preprocessing to perform well on this pharmacovigilance signal detection task, and may even benefit from information that would otherwise be lost during the normalization and standardization process.

Retrofitting Vector Representations of Adverse Event Reporting Data to Structured Knowledge to Improve Pharmacovigilance Signal Detection.

Adverse drug events (ADE) are prevalent and costly. Clinical trials are constrained in their ability to identify potential ADEs, motivating the development of spontaneous reporting systems for post-market surveillance. Statistical methods provide a convenient way to detect signals from these reports but have limitations in leveraging relationships between drugs and ADEs given their discrete count-based nature. A previously proposed method, aer2vec, generates distributed vector representations of ADE report entities that capture patterns of similarity but cannot utilize lexical knowledge. We address this limitation by retrofitting aer2vec drug embeddings to knowledge from RxNorm and developing a novel retrofitting variant using vector rescaling to preserve magnitude. When evaluated in the context of a pharmacovigilance signal detection task, aer2vec with retrofitting consistently outperforms disproportionality metrics when trained on minimally preprocessed data. Retrofitting with rescaling results in further improvements in the larger and more challenging of two pharmacovigilance reference sets used for evaluation.

A maximum likelihood approach to electronic health record phenotyping using positive and unlabeled patients.

OBJECTIVE: Phenotyping patients using electronic health record (EHR) data conventionally requires labeled cases and controls. Assigning labels requires manual medical chart review and therefore is labor intensive. For some phenotypes, identifying gold-standard controls is prohibitive. We developed an accurate EHR phenotyping approach that does not require labeled controls. MATERIALS AND METHODS: Our framework relies on a random subset of cases, which can be specified using an anchor variable that has excellent positive predictive value and sensitivity independent of predictors. We proposed a maximum likelihood approach that efficiently leverages data from the specified cases and unlabeled patients to develop logistic regression phenotyping models, and compare model performance with existing algorithms. RESULTS: Our method outperformed the existing algorithms on predictive accuracy in Monte Carlo simulation studies, application to identify hypertension patients with hypokalemia requiring oral supplementation using a simulated anchor, and application to identify primary aldosteronism patients using real-world cases and anchor variables. Our method additionally generated consistent estimates of 2 important parameters, phenotype prevalence and the proportion of true cases that are labeled. DISCUSSION: Upon identification of an anchor variable that is scalable and transferable to different practices, our approach should facilitate development of scalable, transferable, and practice-specific phenotyping models. CONCLUSIONS: Our proposed approach enables accurate semiautomated EHR phenotyping with minimal manual labeling and therefore should greatly facilitate EHR clinical decision support and research.

Tracing Tumor Evolution in Sarcoma Reveals Clonal Origin of Advanced Metastasis.

Cellular heterogeneity is frequently observed in cancer, but the biological significance of heterogeneous tumor clones is not well defined. Using multicolor reporters and CRISPR-Cas9 barcoding, we trace clonal dynamics in a mouse model of sarcoma. We show that primary tumor growth is associated with a reduction in clonal heterogeneity. Local recurrence of tumors following surgery or radiation therapy is driven by multiple clones. In contrast, advanced metastasis to the lungs is driven by clonal selection of a single metastatic clone (MC). Using RNA sequencing (RNA-seq) and in vivo assays, we identify candidate suppressors of metastasis, namely, Rasd1, Reck, and Aldh1a2. These genes are downregulated in MCs of the primary tumors prior to the formation of metastases. Overexpression of these suppressors of metastasis impair the ability of sarcoma cells to colonize the lungs. Overall, this study reveals clonal dynamics during each step of tumor progression, from initiation to growth, recurrence, and distant metastasis.

Designing risk prediction models for ambulatory no-shows across different specialties and clinics.

Objective: As available data increases, so does the opportunity to develop risk scores on more refined patient populations. In this paper we assessed the ability to derive a risk score for a patient no-showing to a clinic visit. Methods: Using data from 2 264 235 outpatient appointments we assessed the performance of models built across 14 different specialties and 55 clinics. We used regularized logistic regression models to fit and assess models built on the health system, specialty, and clinic levels. We evaluated fits based on their discrimination and calibration. Results: Overall, the results suggest that a relatively robust risk score for patient no-shows could be derived with an average C-statistic of 0.83 across clinic level models and strong calibration. Moreover, the clinic specific models, even with lower training set sizes, often performed better than the more general models. Examination of the individual models showed that risk factors had different degrees of predictability across the different specialties. Implementation of optimal modeling strategies would lead to capturing an additional 4819 no-shows per-year. Conclusion: Overall, this work highlights both the opportunity for and the importance of leveraging the available electronic health record data to develop more refined risk models.

An analogous wood barrel theory to explain the occurrence of hormesis: A case study of sulfonamides and erythromycin on Escherichia coli growth.

Hormesis has aroused much attention during the past two decades and may have great implications on many fields, including toxicology and risk assessment. However, the observation of hormesis remains challenged under laboratory conditions. To determine favorable conditions under which to observe hormesis, we investigated the hormetic responses of Escherichia coli (E. coli) upon exposure of different concentrations of sulfonamides and erythromycin at different time points and in different culture media: Luria-Bertani (LB) broth and Mueller Hinton (MH) broth. Our results reveal that the antibiotics, both individually and combined, produce hormetic effects on E. coli growth in MH broth at the stationary phase, with the maximum stimulatory response increasing with time. However, in LB broth, the hormetic response was not observed, which can be explained by an analogous "wood barrel theory". Our study suggests that the culture medium and time should be taken into consideration in hormetic studies, and compound mixtures should also receive more attention for their potential to induce hormesis.

The Comparison of the Combined Toxicity between Gram-negative and Gram-positive Bacteria: a Case Study of Antibiotics and Quorum-sensing Inhibitors.

Quorum-sensing inhibitors (QSIs) are being used increasingly in diverse fields, and are likely to end up in the environment, where they may encounter the antibiotics and consequently cause joint effects on biological systems. However, the potential joint effects of QSIs and antibiotics have received little attention. In this study, the joint effects of antibiotics, represented by sulfonamides (SAs) and penicillin, as well as three potential QSIs, were investigated using both Gram-negative (Escherichia coli, E. coli) and Gram-positive bacteria (Bacillus subtilis, B. subtilis). It was found that E. coli tend to be more sensitive to the individual drugs than B. subtilis, whereas the joint effects on the two bacteria showed no difference regarding the same combination of antibiotics and QSIs. In general, SAs presented additive effects with γ-Valerolactone and 2-Pyrrolidinone, but antagonistic effects with L-(+)-Prolinol; penicillin exhibited antagonistic effects with all three QSIs. Moreover, it was found that the rate of resistance in E. coli against the individual antibiotics was reduced through the addition of the QSIs, which suggests a promising use of the QSIs in the bacterial infection treatment. This study also offers a valuable reference for the risk assessment of the antibiotics and QSIs in the real environment.