Ethical and regulatory challenges of large language models in medicine.

With the rapid growth of interest in and use of large language models (LLMs) across various industries, we are facing some crucial and profound ethical concerns, especially in the medical field. The unique technical architecture and purported emergent abilities of LLMs differentiate them substantially from other artificial intelligence (AI) models and natural language processing techniques used, necessitating a nuanced understanding of LLM ethics. In this Viewpoint, we highlight ethical concerns stemming from the perspectives of users, developers, and regulators, notably focusing on data privacy and rights of use, data provenance, intellectual property contamination, and broad applications and plasticity of LLMs. A comprehensive framework and mitigating strategies will be imperative for the responsible integration of LLMs into medical practice, ensuring alignment with ethical principles and safeguarding against potential societal risks.

Identifying data-driven subtypes of major depressive disorder with electronic health records.

BACKGROUND: Efforts to reduce the heterogeneity of major depressive disorder (MDD) by identifying subtypes have not yet facilitated treatment personalization or investigation of biology, so novel approaches merit consideration. METHODS: We utilized electronic health records drawn from 2 academic medical centers and affiliated health systems in Massachusetts to identify data-driven subtypes of MDD, characterizing sociodemographic features, comorbid diagnoses, and treatment patterns. We applied Latent Dirichlet Allocation (LDA) to summarize diagnostic codes followed by agglomerative clustering to define patient subgroups. RESULTS: Among 136,371 patients (95,034 women [70 %]; 41,337 men [30 %]; mean [SD] age, 47.0 [14.0] years), the 15 putative MDD subtypes were characterized by comorbidities and distinct patterns in medication use. There was substantial variation in rates of selective serotonin reuptake inhibitor (SSRI) use (from a low of 62 % to a high of 78 %) and selective norepinephrine reuptake inhibitor (SNRI) use (from 4 % to 21 %). LIMITATIONS: Electronic health records lack reliable symptom-level data, so we cannot examine the extent to which subtypes might differ in clinical presentation or symptom dimensions. CONCLUSION: These data-driven subtypes, drawing on representative clinical cohorts, merit further investigation for their utility in identifying more homogeneous patient populations for basic as well as clinical investigation.

Artificial intelligence education: An evidence-based medicine approach for consumers, translators, and developers.

Current and future healthcare professionals are generally not trained to cope with the proliferation of artificial intelligence (AI) technology in healthcare. To design a curriculum that caters to variable baseline knowledge and skills, clinicians may be conceptualized as "consumers", "translators", or "developers". The changes required of medical education because of AI innovation are linked to those brought about by evidence-based medicine (EBM). We outline a core curriculum for AI education of future consumers, translators, and developers, emphasizing the links between AI and EBM, with suggestions for how teaching may be integrated into existing curricula. We consider the key barriers to implementation of AI in the medical curriculum: time, resources, variable interest, and knowledge retention. By improving AI literacy rates and fostering a translator- and developer-enriched workforce, innovation may be accelerated for the benefit of patients and practitioners.

The Unintended Consequences of Discount Regularization: Improving Regularization in Certainty Equivalence Reinforcement Learning.

Discount regularization, using a shorter planning horizon when calculating the optimal policy, is a popular choice to restrict planning to a less complex set of policies when estimating an MDP from sparse or noisy data (Jiang et al., 2015). It is commonly understood that discount regularization functions by de-emphasizing or ignoring delayed effects. In this paper, we reveal an alternate view of discount regularization that exposes unintended consequences. We demonstrate that planning under a lower discount factor produces an identical optimal policy to planning using any prior on the transition matrix that has the same distribution for all states and actions. In fact, it functions like a prior with stronger regularization on state-action pairs with more transition data. This leads to poor performance when the transition matrix is estimated from data sets with uneven amounts of data across state-action pairs. Our equivalence theorem leads to an explicit formula to set regularization parameters locally for individual state-action pairs rather than globally. We demonstrate the failures of discount regularization and how we remedy them using our state-action-specific method across simple empirical examples as well as a medical cancer simulator.

Reward Design For An Online Reinforcement Learning Algorithm Supporting Oral Self-Care.

While dental disease is largely preventable, professional advice on optimal oral hygiene practices is often forgotten or abandoned by patients. Therefore patients may benefit from timely and personalized encouragement to engage in oral self-care behaviors. In this paper, we develop an online reinforcement learning (RL) algorithm for use in optimizing the delivery of mobile-based prompts to encourage oral hygiene behaviors. One of the main challenges in developing such an algorithm is ensuring that the algorithm considers the impact of current actions on the effectiveness of future actions (i.e., delayed effects), especially when the algorithm has been designed to run stably and autonomously in a constrained, real-world setting characterized by highly noisy, sparse data. We address this challenge by designing a quality reward that maximizes the desired health outcome (i.e., high-quality brushing) while minimizing user burden. We also highlight a procedure for optimizing the hyperparameters of the reward by building a simulation environment test bed and evaluating candidates using the test bed. The RL algorithm discussed in this paper will be deployed in Oralytics. To the best of our knowledge, Oralytics is the first mobile health study utilizing an RL algorithm designed to prevent dental disease by optimizing the delivery of motivational messages supporting oral self-care behaviors.

Online model selection by learning how compositional kernels evolve.

Motivated by the need for efficient, personalized learning in mobile health, we investigate the problem of online compositional kernel selection for multi-task Gaussian Process regression. Existing composition selection methods do not satisfy our strict criteria in health; selection must occur quickly, and the selected kernels must maintain the appropriate level of complexity, sparsity, and stability as data arrives online. We introduce the Kernel Evolution Model (KEM), a generative process on how to evolve kernel compositions in a way that manages the bias-variance trade-off as we observe more data about a user. Using pilot data, we learn a set of kernel evolutions that can be used to quickly select kernels for new test users. KEM reliably selects high-performing kernels for a range of synthetic and real data sets, including two health data sets.

An interpretable RL framework for pre-deployment modeling in ICU hypotension management.

Computational methods from reinforcement learning have shown promise in inferring treatment strategies for hypotension management and other clinical decision-making challenges. Unfortunately, the resulting models are often difficult for clinicians to interpret, making clinical inspection and validation of these computationally derived strategies challenging in advance of deployment. In this work, we develop a general framework for identifying succinct sets of clinical contexts in which clinicians make very different treatment choices, tracing the effects of those choices, and inferring a set of recommendations for those specific contexts. By focusing on these few key decision points, our framework produces succinct, interpretable treatment strategies that can each be easily visualized and verified by clinical experts. This interrogation process allows clinicians to leverage the model's use of historical data in tandem with their own expertise to determine which recommendations are worth investigating further e.g. at the bedside. We demonstrate the value of this approach via application to hypotension management in the ICU, an area with critical implications for patient outcomes that lacks data-driven individualized treatment strategies; that said, our framework has broad implications on how to use computational methods to assist with decision-making challenges on a wide range of clinical domains.

Do clinicians follow heuristics in prescribing antidepressants?

BACKGROUND: While clinicians commonly learn heuristics to guide antidepressant treatment selection, surveys suggest real-world prescribing practices vary widely. We aimed to determine the extent to which antidepressant prescriptions were consistent with commonly-advocated heuristics for treatment selection. METHODS: This retrospective longitudinal cohort study examined electronic health records from psychiatry and non-psychiatry practice networks affiliated with two large academic medical centers between March 2008 and December 2017. Patients included 45,955 individuals with a major depressive disorder or depressive disorder not otherwise specified diagnosis who were prescribed at least one of 11 common antidepressant medications. Specific clinical features that may impact prescribing choices were extracted from coded data, and analyzed for association with index prescription in logistic regression models adjusted for sociodemographic variables and provider type. RESULTS: Multiple clinical features yielded 10% or greater change in odds of prescribing, including overweight and underweight status and sexual dysfunction. These heuristics were generally applied similarly across hospital systems and psychiatrist and non-psychiatrist providers. LIMITATIONS: These analyses rely on coded clinical data, which is likely to substantially underestimate prevalence of particular clinical features. Additionally, numerous other features that may impact prescribing choices are not able to be modeled. CONCLUSION: Our results confirm the hypothesis that clinicians apply heuristics on the basis of clinical features to guide antidepressant prescribing, although the magnitude of these effects is modest, suggesting other patient- or clinician-level factors have larger effects. FUNDING: This work was funded by NSF GRFP (grant no. DGE1745303), Harvard SEAS, the Center for Research on Computation and Society at Harvard, the Harvard Data Science Initiative, and a grant from the National Institute of Mental Health (grant no. 1R01MH106577).

Efficiently identifying individuals at high risk for treatment resistance in major depressive disorder using electronic health records.

BACKGROUND: With the emergence of evidence-based treatments for treatment-resistant depression, strategies to identify individuals at greater risk for treatment resistance early in the course of illness could have clinical utility. We sought to develop and validate a model to predict treatment resistance in major depressive disorder using coded clinical data from the electronic health record. METHODS: We identified individuals from a large health system with a diagnosis of major depressive disorder receiving an index antidepressant prescription, and used a tree-based machine learning classifier to build a risk stratification model to identify those likely to experience treatment resistance. The resulting model was validated in a second health system. RESULTS: In the second health system, the extra trees model yielded an AUC of 0.652 (95% CI: 0.623-0.682); with sensitivity constrained at 0.80, specificity was 0.358 (95% CI: 0.300-0.413). Lift in the top quintile was 1.99 (95% CI: 1.76-2.22). Including additional data for the 4 weeks following treatment initiation did not meaningfully improve model performance. LIMITATIONS: The extent to which these models generalize across additional health systems will require further investigation. CONCLUSION: Electronic health records facilitated stratification of risk for treatment-resistant depression and demonstrated generalizability to a second health system. Efforts to improve upon such models using additional measures, and to understand their performance in real-world clinical settings, are warranted.

Designing Reinforcement Learning Algorithms for Digital Interventions: Pre-Implementation Guidelines.

Online reinforcement learning (RL) algorithms are increasingly used to personalize digital interventions in the fields of mobile health and online education. Common challenges in designing and testing an RL algorithm in these settings include ensuring the RL algorithm can learn and run stably under real-time constraints, and accounting for the complexity of the environment, e.g., a lack of accurate mechanistic models for the user dynamics. To guide how one can tackle these challenges, we extend the PCS (predictability, computability, stability) framework, a data science framework that incorporates best practices from machine learning and statistics in supervised learning to the design of RL algorithms for the digital interventions setting. Furthermore, we provide guidelines on how to design simulation environments, a crucial tool for evaluating RL candidate algorithms using the PCS framework. We show how we used the PCS framework to design an RL algorithm for Oralytics, a mobile health study aiming to improve users' tooth-brushing behaviors through the personalized delivery of intervention messages. Oralytics will go into the field in late 2022.

Power Constrained Bandits.

Contextual bandits often provide simple and effective personalization in decision making problems, making them popular tools to deliver personalized interventions in mobile health as well as other health applications. However, when bandits are deployed in the context of a scientific study-e.g. a clinical trial to test if a mobile health intervention is effective-the aim is not only to personalize for an individual, but also to determine, with sufficient statistical power, whether or not the system's intervention is effective. It is essential to assess the effectiveness of the intervention before broader deployment for better resource allocation. The two objectives are often deployed under different model assumptions, making it hard to determine how achieving the personalization and statistical power affect each other. In this work, we develop general meta-algorithms to modify existing algorithms such that sufficient power is guaranteed while still improving each user's well-being. We also demonstrate that our meta-algorithms are robust to various model mis-specifications possibly appearing in statistical studies, thus providing a valuable tool to study designers.

Preferential Mixture-of-Experts: Interpretable Models that Rely on Human Expertise As Much As Possible.

We propose Preferential MoE, a novel human-ML mixture-of-experts model that augments human expertise in decision making with a data-based classifier only when necessary for predictive performance. Our model exhibits an interpretable gating function that provides information on when human rules should be followed or avoided. The gating function is maximized for using human-based rules, and classification errors are minimized. We propose solving a coupled multi-objective problem with convex subproblems. We develop approximate algorithms and study their performance and convergence. Finally, we demonstrate the utility of Preferential MoE on two clinical applications for the treatment of Human Immunodeficiency Virus (HIV) and management of Major Depressive Disorder (MDD).

Generalization in Clinical Prediction Models: The Blessing and Curse of Measurement Indicator Variables.

OBJECTIVE: Specific factors affecting generalizability of clinical prediction models are poorly understood. Our main objective was to investigate how measurement indicator variables affect external validity in clinical prediction models for predicting onset of vasopressor therapy. DESIGN: We fit logistic regressions on retrospective cohorts to predict vasopressor onset using two classes of variables: seemingly objective clinical variables (vital signs and laboratory measurements) and more subjective variables denoting recency of measurements. SETTING: Three cohorts from two tertiary-care academic hospitals in geographically distinct regions, spanning general inpatient and critical care settings. PATIENTS: Each cohort consisted of adult patients (age greater than or equal to 18 yr at time of hospitalization), with lengths of stay between 6 and 600 hours, and who did not receive vasopressors in the first 6 hours of hospitalization or ICU admission. Models were developed on each of the three derivation cohorts and validated internally on the derivation cohort and externally on the other two cohorts. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: The prevalence of vasopressors was 0.9% in the general inpatient cohort and 12.4% and 11.5% in the two critical care cohorts. Models utilizing both classes of variables performed the best in-sample, with C-statistics for predicting vasopressor onset in 4 hours of 0.862 (95% CI, 0.844-0.879), 0.822 (95% CI, 0.793-0.852), and 0.889 (95% CI, 0.880-0.898). Models solely using the subjective variables denoting measurement recency had poor external validity. However, these practice-driven variables helped adjust for differences between the two hospitals and led to more generalizable models using clinical variables. CONCLUSIONS: We developed and externally validated models for predicting the onset of vasopressors. We found that practice-specific features denoting measurement recency improved local performance and also led to more generalizable models if they are adjusted for during model development but discarded at validation. The role of practice-specific features such as measurement indicators in clinical prediction modeling should be carefully considered if the goal is to develop generalizable models.

How machine-learning recommendations influence clinician treatment selections: the example of the antidepressant selection.

Decision support systems embodying machine learning models offer the promise of an improved standard of care for major depressive disorder, but little is known about how clinicians' treatment decisions will be influenced by machine learning recommendations and explanations. We used a within-subject factorial experiment to present 220 clinicians with patient vignettes, each with or without a machine-learning (ML) recommendation and one of the multiple forms of explanation. We found that interacting with ML recommendations did not significantly improve clinicians' treatment selection accuracy, assessed as concordance with expert psychopharmacologist consensus, compared to baseline scenarios in which clinicians made treatment decisions independently. Interacting with incorrect recommendations paired with explanations that included limited but easily interpretable information did lead to a significant reduction in treatment selection accuracy compared to baseline questions. These results suggest that incorrect ML recommendations may adversely impact clinician treatment selections and that explanations are insufficient for addressing overreliance on imperfect ML algorithms. More generally, our findings challenge the common assumption that clinicians interacting with ML tools will perform better than either clinicians or ML algorithms individually.

Predicting change in diagnosis from major depression to bipolar disorder after antidepressant initiation.

We aimed to develop and validate classification models able to identify individuals at high risk for transition from a diagnosis of depressive disorder to one of bipolar disorder. This retrospective health records cohort study applied outpatient clinical data from psychiatry and nonpsychiatry practice networks affiliated with two large academic medical centers between March 2008 and December 2017. Participants included 67,807 individuals with a diagnosis of major depressive disorder or depressive disorder not otherwise specified and no prior diagnosis of bipolar disorder, who received at least one of the nine antidepressant medications. The main outcome was at least one diagnostic code reflective of a bipolar disorder diagnosis within 3 months of index antidepressant prescription. Logistic regression and random forests using diagnostic and procedure codes as well as sociodemographic features were used to predict this outcome, with discrimination and calibration assessed in a held-out test set and then a second academic medical center. Among 67,807 individuals who received at least one antidepressant medication, 925 (1.36%) subsequently received a diagnosis of bipolar disorder within 3 months. Models incorporating coded diagnoses and procedures yielded a mean area under the receiver operating characteristic curve of 0.76 (ranging from 0.73 to 0.80). Standard supervised machine learning methods enabled development of discriminative and transferable models to predict transition to bipolar disorder. With further validation, these scores may enable physicians to more precisely calibrate follow-up intensity for high-risk patients after antidepressant initiation.

Learning Predictive and Interpretable Timeseries Summaries from ICU Data.

Machine learning models that utilize patient data across time (rather than just the most recent measurements) have increased performance for many risk stratification tasks in the intensive care unit. However, many of these models and their learned representations are complex and therefore difficult for clinicians to interpret, creating challenges for validation. Our work proposes a new procedure to learn summaries of clinical timeseries that are both predictive and easily understood by humans. Specifically, our summaries consist of simple and intuitive functions of clinical data (e.g. "falling mean arterial pressure"). Our learned summaries outperform traditional interpretable model classes and achieve performance comparable to state-of-the-art deep learning models on an in-hospital mortality classification task.

The myth of generalisability in clinical research and machine learning in health care.

An emphasis on overly broad notions of generalisability as it pertains to applications of machine learning in health care can overlook situations in which machine learning might provide clinical utility. We believe that this narrow focus on generalisability should be replaced with wider considerations for the ultimate goal of building machine learning systems that are useful at the bedside.

Identifying Distinct, Effective Treatments for Acute Hypotension with SODA-RL: Safely Optimized Diverse Accurate Reinforcement Learning.

Hypotension in critical care settings is a life-threatening emergency that must be recognized and treated early. While fluid bolus therapy and vasopressors are common treatments, it is often unclear which interventions to give, in what amounts, and for how long. Observational data in the form of electronic health records can provide a source for helping inform these choices from past events, but often it is not possible to identify a single best strategy from observational data alone. In such situations, we argue it is important to expose the collection of plausible options to a provider. To this end, we develop SODA-RL: Safely Optimized, Diverse, and Accurate Reinforcement Learning, to identify distinct treatment options that are supported in the data. We demonstrate SODA-RL on a cohort of 10,142 ICU stays where hypotension presented. Our learned policies perform comparably to the observed physician behaviors, while providing different, plausible alternatives for treatment decisions.

Interpretable Batch IRL to Extract Clinician Goals in ICU Hypotension Management.

Exposing and understanding the motivations of clinicians is an important step for building robust assistive agents as well as improving care. In this work, we focus on understanding the motivations for clinicians managing hypotension in the ICU. We model the ICU interventions as a batch, sequential decision making problem and develop a novel interpretable batch variant of Adversarial Inverse Reinforcement Learning algorithm that not only learns rewards which induce treatment policies similar to clinical treatments, but also ensure that the learned functional form of rewards is consistent with the decision mechanisms of clinicians in the ICU. We apply our approach to understanding vasopressor and IVfluid administration in the ICU and posit that this interpretability enables inspection and validation of the rewards robustly.

Assessment of a Prediction Model for Antidepressant Treatment Stability Using Supervised Topic Models.

Importance: In the absence of readily assessed and clinically validated predictors of treatment response, pharmacologic management of major depressive disorder often relies on trial and error. Objective: To assess a model using electronic health records to identify predictors of treatment response in patients with major depressive disorder. Design, Setting, and Participants: This retrospective cohort study included data from 81 630 adults with a coded diagnosis of major depressive disorder from 2 academic medical centers in Boston, Massachusetts, including outpatient primary and specialty care clinics from December 1, 1997, to December 31, 2017. Data were analyzed from January 1, 2018, to March 15, 2020. Exposures: Treatment with at least 1 of 11 standard antidepressants. Main Outcomes and Measures: Stable treatment response, intended as a proxy for treatment effectiveness, defined as continued prescription of an antidepressant for 90 days. Supervised topic models were used to extract 10 interpretable covariates from coded clinical data for stability prediction. With use of data from 1 hospital system (site A), generalized linear models and ensembles of decision trees were trained to predict stability outcomes from topic features that summarize patient history. Held-out patients from site A and individuals from a second hospital system (site B) were evaluated. Results: Among the 81 630 adults (56 340 women [69%]; mean [SD] age, 48.46 [14.75] years; range, 18.0-80.0 years), 55 303 reached a stable response to their treatment regimen during follow-up. For held-out patients from site A, the mean area under the receiver operating characteristic curve (AUC) for discrimination of the general stability outcome was 0.627 (95% CI, 0.615-0.639) for the supervised topic model with 10 covariates. In evaluation of site B, the AUC was 0.619 (95% CI, 0.610-0.627). Building models to predict stability specific to a particular drug did not improve prediction of general stability even when using a harder-to-interpret ensemble classifier and 9256 coded covariates (specific AUC, 0.647; 95% CI, 0.635-0.658; general AUC, 0.661; 95% CI, 0.648-0.672). Topics coherently captured clinical concepts associated with treatment response. Conclusions and Relevance: The findings suggest that coded clinical data available in electronic health records may facilitate prediction of general treatment response but not response to specific medications. Although greater discrimination is likely required for clinical application, the results provide a transparent baseline for such studies.