Design of a multicenter randomized controlled trial of a post-discharge suicide prevention intervention for high-risk psychiatric inpatients: The Veterans Coordinated Community Care Study.

BACKGROUND: The period after psychiatric hospital discharge is one of elevated risk for suicide-related behaviors (SRBs). Post-discharge clinical outreach, although potentially effective in preventing SRBs, would be more cost-effective if targeted at high-risk patients. To this end, a machine learning model was developed to predict post-discharge suicides among Veterans Health Administration (VHA) psychiatric inpatients and target a high-risk preventive intervention. METHODS: The Veterans Coordinated Community Care (3C) Study is a multicenter randomized controlled trial using this model to identify high-risk VHA psychiatric inpatients (n = 850) randomized with equal allocation to either the Coping Long Term with Active Suicide Program (CLASP) post-discharge clinical outreach intervention or treatment-as-usual (TAU). The primary outcome is SRBs over a 6-month follow-up. We will estimate average treatment effects adjusted for loss to follow-up and investigate the possibility of heterogeneity of treatment effects. RESULTS: Recruitment is underway and will end September 2024. Six-month follow-up will end and analysis will begin in Summer 2025. CONCLUSION: Results will provide information about the effectiveness of CLASP versus TAU in reducing post-discharge SRBs and provide guidance to VHA clinicians and policymakers about the implications of targeted use of CLASP among high-risk psychiatric inpatients in the months after hospital discharge. CLINICAL TRIALS REGISTRATION: ClinicalTrials.Gov identifier: NCT05272176 (https://www. CLINICALTRIALS: gov/ct2/show/NCT05272176).

Predicting Suicides Among US Army Soldiers After Leaving Active Service.

Importance: The suicide rate of military servicemembers increases sharply after returning to civilian life. Identifying high-risk servicemembers before they leave service could help target preventive interventions. Objective: To develop a model based on administrative data for regular US Army soldiers that can predict suicides 1 to 120 months after leaving active service. Design, Setting, and Participants: In this prognostic study, a consolidated administrative database was created for all regular US Army soldiers who left service from 2010 through 2019. Machine learning models were trained to predict suicides over the next 1 to 120 months in a random 70% training sample. Validation was implemented in the remaining 30%. Data were analyzed from March 2023 through March 2024. Main outcome and measures: The outcome was suicide in the National Death Index. Predictors came from administrative records available before leaving service on sociodemographics, Army career characteristics, psychopathologic risk factors, indicators of physical health, social networks and supports, and stressors. Results: Of the 800 579 soldiers in the cohort (84.9% male; median [IQR] age at discharge, 26 [23-33] years), 2084 suicides had occurred as of December 31, 2019 (51.6 per 100 000 person-years). A lasso model assuming consistent slopes over time discriminated as well over all but the shortest risk horizons as more complex stacked generalization ensemble machine learning models. Test sample area under the receiver operating characteristic curve ranged from 0.87 (SE = 0.06) for suicides in the first month after leaving service to 0.72 (SE = 0.003) for suicides over 120 months. The 10% of soldiers with highest predicted risk accounted for between 30.7% (SE = 1.8) and 46.6% (SE = 6.6) of all suicides across horizons. Calibration was for the most part better for the lasso model than the super learner model (both estimated over 120-month horizons.) Net benefit of a model-informed prevention strategy was positive compared with intervene-with-all or intervene-with-none strategies over a range of plausible intervention thresholds. Sociodemographics, Army career characteristics, and psychopathologic risk factors were the most important classes of predictors. Conclusions and relevance: These results demonstrated that a model based on administrative variables available at the time of leaving active Army service can predict suicides with meaningful accuracy over the subsequent decade. However, final determination of cost-effectiveness would require information beyond the scope of this report about intervention content, costs, and effects over relevant horizons in relation to the monetary value placed on preventing suicides.

Patterns and correlates of mental healthcare utilization during the COVID-19 pandemic among individuals with pre-existing mental disorder.

How did mental healthcare utilization change during the COVID-19 pandemic period among individuals with pre-existing mental disorder? Understanding utilization patterns of these at-risk individuals and identifying those most likely to exhibit increased utilization could improve patient stratification and efficient delivery of mental health services. This study leveraged large-scale electronic health record (EHR) data to describe mental healthcare utilization patterns among individuals with pre-existing mental disorder before and during the COVID-19 pandemic and identify correlates of high mental healthcare utilization. Using EHR data from a large healthcare system in Massachusetts, we identified three "pre-existing mental disorder" groups (PMD) based on having a documented mental disorder diagnosis within the 6 months prior to the March 2020 lockdown, related to: (1) stress-related disorders (e.g., depression, anxiety) (N = 115,849), (2) serious mental illness (e.g., schizophrenia, bipolar disorders) (N = 11,530), or (3) compulsive behavior disorders (e.g., eating disorder, OCD) (N = 5,893). We also identified a "historical comparison" group (HC) for each PMD (N = 113,604, 11,758, and 5,387, respectively) from the previous year (2019). We assessed the monthly number of mental healthcare visits from March 13 to December 31 for PMDs in 2020 and HCs in 2019. Phenome-wide association analyses (PheWAS) were used to identify clinical correlates of high mental healthcare utilization. We found the overall number of mental healthcare visits per patient during the pandemic period in 2020 was 10-12% higher than in 2019. The majority of increased visits was driven by a subset of high mental healthcare utilizers (top decile). PheWAS results indicated that correlates of high utilization (prior mental disorders, chronic pain, insomnia, viral hepatitis C, etc.) were largely similar before and during the pandemic, though several conditions (e.g., back pain) were associated with high utilization only during the pandemic. Limitations included that we were not able to examine other risk factors previously shown to influence mental health during the pandemic (e.g., social support, discrimination) due to lack of social determinants of health information in EHR data. Mental healthcare utilization among patients with pre-existing mental disorder increased overall during the pandemic, likely due to expanded access to telemedicine. Given that clinical correlates of high mental healthcare utilization in a major hospital system were largely similar before and during the COVID-19 pandemic, resource stratification based on known risk factor profiles may aid hospitals in responding to heightened mental healthcare needs during a pandemic.

Comparative effectiveness of remote digital gamified and group CBT skills training interventions for anxiety and depression among college students: Results of a three-arm randomised controlled trial.

Digital interventions can enhance access to healthcare in under-resourced settings. However, guided digital interventions may be costly for low- and middle-income countries, despite their effectiveness. In this randomised control trial, we evaluated the effectiveness of two digital interventions designed to address this issue: (1) a Cognitive Behavioral Therapy Skills Training (CST) intervention that increased scalability by using remote online group administration; and (2) the SuperBetter gamified self-guided CBT skills training app, which uses other participants rather than paid staff as guides. The study was implemented among anxious and/or depressed South African undergraduates (n = 371) randomised with equal allocation to Remote Group CST, SuperBetter, or a MoodFlow mood monitoring control. Symptoms were assessed with the Generalized Anxiety Disorder-7 (GAD-7) and the Patient Health Questionnaire-9 (PHQ-9). Intention-to-treat analysis found effect sizes at the high end of prior digital intervention trials, including significantly higher adjusted risk differences (ARD; primary outcome) in joint anxiety/depression remission at 3-months and 6-months for Remote Group CST (ARD = 23.3-18.9%, p = 0.001-0.035) and SuperBetter (ARD = 12.7-22.2%, p = 0.047-0.006) than MoodFlow and mean combined PHQ-9/GAD-7 scores (secondary outcome) significantly lower for Remote Group CST and SuperBetter than MoodFlow. These results illustrate how innovative delivery methods can increase the scalability of standard one-on-one guided digital interventions. PREREGISTRATION INTERNATIONAL STANDARD RANDOMISED CONTROLLED TRIAL NUMBER (ISRTCN) SUBMISSION #: 47,089,643.

Validation of an ICD-Code-Based Case Definition for Psychotic Illness Across Three Health Systems.

BACKGROUND AND HYPOTHESIS: Psychosis-associated diagnostic codes are increasingly being utilized as case definitions for electronic health record (EHR)-based algorithms to predict and detect psychosis. However, data on the validity of psychosis-related diagnostic codes is limited. We evaluated the positive predictive value (PPV) of International Classification of Diseases (ICD) codes for psychosis. STUDY DESIGN: Using EHRs at 3 health systems, ICD codes comprising primary psychotic disorders and mood disorders with psychosis were grouped into 5 higher-order groups. 1133 records were sampled for chart review using the full EHR. PPVs (the probability of chart-confirmed psychosis given ICD psychosis codes) were calculated across multiple treatment settings. STUDY RESULTS: PPVs across all diagnostic groups and hospital systems exceeded 70%: Mass General Brigham 0.72 [95% CI 0.68-0.77], Boston Children's Hospital 0.80 [0.75-0.84], and Boston Medical Center 0.83 [0.79-0.86]. Schizoaffective disorder PPVs were consistently the highest across sites (0.80-0.92) and major depressive disorder with psychosis were the most variable (0.57-0.79). To determine if the first documented code captured first-episode psychosis (FEP), we excluded cases with prior chart evidence of a diagnosis of or treatment for a psychotic illness, yielding substantially lower PPVs (0.08-0.62). CONCLUSIONS: We found that the first documented psychosis diagnostic code accurately captured true episodes of psychosis but was a poor index of FEP. These data have important implications for the case definitions used in the development of risk prediction models designed to predict or detect undiagnosed psychosis.

Validation of an ICD-code-based case definition for psychotic illness across three health systems.

Background and Hypothesis: Early detection of psychosis is critical for improving outcomes. Algorithms to predict or detect psychosis using electronic health record (EHR) data depend on the validity of the case definitions used, typically based on diagnostic codes. Data on the validity of psychosis-related diagnostic codes is limited. We evaluated the positive predictive value (PPV) of International Classification of Diseases (ICD) codes for psychosis. Study Design: Using EHRs at three health systems, ICD codes comprising primary psychotic disorders and mood disorders with psychosis were grouped into five higher-order groups. 1,133 records were sampled for chart review using the full EHR. PPVs (the probability of chart-confirmed psychosis given ICD psychosis codes) were calculated across multiple treatment settings. Study Results: PPVs across all diagnostic groups and hospital systems exceeded 70%: Massachusetts General Brigham 0.72 [95% CI 0.68-0.77], Boston Children's Hospital 0.80 [0.75-0.84], and Boston Medical Center 0.83 [0.79-0.86]. Schizoaffective disorder PPVs were consistently the highest across sites (0.80-0.92) and major depressive disorder with psychosis were the most variable (0.57-0.79). To determine if the first documented code captured first-episode psychosis (FEP), we excluded cases with prior chart evidence of a diagnosis of or treatment for a psychotic illness, yielding substantially lower PPVs (0.08-0.62). Conclusions: We found that the first documented psychosis diagnostic code accurately captured true episodes of psychosis but was a poor index of FEP. These data have important implications for the development of risk prediction models designed to predict or detect undiagnosed psychosis.

Developing an individualized treatment rule for Veterans with major depressive disorder using electronic health records.

Efforts to develop an individualized treatment rule (ITR) to optimize major depressive disorder (MDD) treatment with antidepressant medication (ADM), psychotherapy, or combined ADM-psychotherapy have been hampered by small samples, small predictor sets, and suboptimal analysis methods. Analyses of large administrative databases designed to approximate experiments followed iteratively by pragmatic trials hold promise for resolving these problems. The current report presents a proof-of-concept study using electronic health records (EHR) of n = 43,470 outpatients beginning MDD treatment in Veterans Health Administration Primary Care Mental Health Integration (PC-MHI) clinics, which offer access not only to ADMs but also psychotherapy and combined ADM-psychotherapy. EHR and geospatial databases were used to generate an extensive baseline predictor set (5,865 variables). The outcome was a composite measure of at least one serious negative event (suicide attempt, psychiatric emergency department visit, psychiatric hospitalization, suicide death) over the next 12 months. Best-practices methods were used to adjust for nonrandom treatment assignment and to estimate a preliminary ITR in a 70% training sample and to evaluate the ITR in the 30% test sample. Statistically significant aggregate variation was found in overall probability of the outcome related to baseline predictors (AU-ROC = 0.68, S.E. = 0.01), with test sample outcome prevalence of 32.6% among the 5% of patients having highest predicted risk compared to 7.1% in the remainder of the test sample. The ITR found that psychotherapy-only was the optimal treatment for 56.0% of patients (roughly 20% lower risk of the outcome than if receiving one of the other treatments) and that treatment type was unrelated to outcome risk among other patients. Change in aggregate treatment costs of implementing this ITR would be negligible, as 16.1% fewer patients would be prescribed ADMs and 2.9% more would receive psychotherapy. A pragmatic trial would be needed to confirm the accuracy of the ITR.

Toward personalized care for insomnia in the US Army: a machine learning model to predict response to cognitive behavioral therapy for insomnia.

STUDY OBJECTIVES: The standard of care for military personnel with insomnia is cognitive behavioral therapy for insomnia (CBT-I). However, only a minority seeking insomnia treatment receive CBT-I, and little reliable guidance exists to identify those most likely to respond. As a step toward personalized care, we present results of a machine learning (ML) model to predict CBT-I response. METHODS: Administrative data were examined for n = 1,449 nondeployed US Army soldiers treated for insomnia with CBT-I who had moderate-severe baseline Insomnia Severity Index (ISI) scores and completed 1 or more follow-up ISIs 6-12 weeks after baseline. An ensemble ML model was developed in a 70% training sample to predict clinically significant ISI improvement (reduction of at least 2 standard deviations on the baseline ISI distribution). Predictors included a wide range of military administrative and baseline clinical variables. Model accuracy was evaluated in the remaining 30% test sample. RESULTS: 19.8% of patients had clinically significant ISI improvement. Model area under the receiver operating characteristic curve (standard error) was 0.60 (0.03). The 20% of test-sample patients with the highest probabilities of improvement were twice as likely to have clinically significant improvement compared with the remaining 80% (36.5% vs 15.7%; χ21 = 9.2, P = .002). Nearly 85% of prediction accuracy was due to 10 variables, the most important of which were baseline insomnia severity and baseline suicidal ideation. CONCLUSIONS: Pending replication, the model could be used as part of a patient-centered decision-making process for insomnia treatment. Parallel models will be needed for alternative treatments before such a system is of optimal value. CITATION: Gabbay FH, Wynn GH, Georg MW, et al. Toward personalized care for insomnia in the US Army: a machine learning model to predict response to cognitive behavioral therapy for insomnia. J Clin Sleep Med. 2024;20(6):921-931.

Predicting Homelessness Among Transitioning U.S. Army Soldiers.

INTRODUCTION: This study develops a practical method to triage Army transitioning service members (TSMs) at highest risk of homelessness to target a preventive intervention. METHODS: The sample included 4,790 soldiers from the Study to Assess Risk and Resilience in Servicemembers-Longitudinal Study (STARRS-LS) who participated in 1 of 3 Army STARRS 2011-2014 baseline surveys followed by the third wave of the STARRS-LS online panel surveys (2020-2022). Two machine learning models were trained: a Stage-1 model that used administrative predictors and geospatial data available for all TSMs at discharge to identify high-risk TSMs for initial outreach; and a Stage-2 model estimated in the high-risk subsample that used self-reported survey data to help determine highest risk based on additional information collected from high-risk TSMs once they are contacted. The outcome in both models was homelessness within 12 months after leaving active service. RESULTS: Twelve-month prevalence of post-transition homelessness was 5.0% (SE=0.5). The Stage-1 model identified 30% of high-risk TSMs who accounted for 52% of homelessness. The Stage-2 model identified 10% of all TSMs (i.e., 33% of high-risk TSMs) who accounted for 35% of all homelessness (i.e., 63% of the homeless among high-risk TSMs). CONCLUSIONS: Machine learning can help target outreach and assessment of TSMs for homeless prevention interventions.