Illusory generalizability of clinical prediction models.

It is widely hoped that statistical models can improve decision-making related to medical treatments. Because of the cost and scarcity of medical outcomes data, this hope is typically based on investigators observing a model's success in one or two datasets or clinical contexts. We scrutinized this optimism by examining how well a machine learning model performed across several independent clinical trials of antipsychotic medication for schizophrenia. Models predicted patient outcomes with high accuracy within the trial in which the model was developed but performed no better than chance when applied out-of-sample. Pooling data across trials to predict outcomes in the trial left out did not improve predictions. These results suggest that models predicting treatment outcomes in schizophrenia are highly context-dependent and may have limited generalizability.

Clinical and Financial Outcomes Associated With a Workplace Mental Health Program Before and During the COVID-19 Pandemic.

Importance: Investment in workplace wellness programs is increasing despite concerns about lack of clinical benefit and return on investment (ROI). In contrast, outcomes from workplace mental health programs, which treat mental health difficulties more directly, remain mostly unknown. Objective: To determine whether participation in an employer-sponsored mental health benefit was associated with improvements in depression and anxiety, workplace productivity, and ROI as well as to examine factors associated with clinical improvement. Design, Setting, and Participants: This cohort study included participants in a US workplace mental health program implemented by 66 employers across 40 states from January 1, 2018, to January 1, 2021. Participants were employees who enrolled in the mental health benefit program and had at least moderate anxiety or depression, at least 1 appointment, and at least 2 outcome assessments. Intervention: A digital platform that screened individuals for common mental health conditions and provided access to self-guided digital content, care navigation, and video and in-person psychotherapy and/or medication management. Main Outcomes and Measures: Primary outcomes were the Patient Health Questionnaire-9 for depression (range, 0-27) score and the Generalized Anxiety Disorder 7-item scale (range, 0-21) score. The ROI was calculated by comparing the cost of treatment to salary costs for time out of the workplace due to mental health symptoms, measured with the Sheehan Disability Scale. Data were collected through 6 months of follow-up and analyzed using mixed-effects regression. Results: A total of 1132 participants (520 of 724 who reported gender [71.8%] were female; mean [SD] age, 32.9 [8.8] years) were included. Participants reported improvements from pretreatment to posttreatment in depression (b = -6.34; 95% CI, -6.76 to -5.91; Cohen d = -1.11; 95% CI, -1.18 to -1.03) and anxiety (b = -6.28; 95% CI, -6.77 to -5.91; Cohen d = -1.21; 95% CI, -1.30 to -1.13). Symptom change per log-day of treatment was similar post-COVID-19 vs pre-COVID-19 for depression (b = 0.14; 95% CI, -0.10 to 0.38) and anxiety (b = 0.08; 95% CI, -0.22 to 0.38). Workplace salary savings at 6 months at the federal median wage was US $3440 (95% CI, $2730-$4151) with positive ROI across all wage groups. Conclusions and Relevance: Results of this cohort study suggest that an employer-sponsored workplace mental health program was associated with large clinical effect sizes for employees and positive financial ROI for employers.

Trends in inpatient care for psychiatric disorders in NHS hospitals across England, 1998/99-2019/20: an observational time series analysis.

PURPOSE: It is unclear how hospitals are responding to the mental health needs of the population in England, against a backdrop of diminishing resources. We aimed to document patterns in hospital activity by psychiatric disorder and how these have changed over the last 22 years. METHODS: In this observational time series analysis, we used routinely collected data on all NHS hospitals in England from 1998/99 to 2019/20. Trends in hospital admissions and bed days for psychiatric disorders were smoothed using negative binomial regression models with year as the exposure and rates (per 1000 person-years) as the outcome. When linear trends were not appropriate, we fitted segmented negative binomial regression models with one change-point. We stratified by gender and age group [children (0-14 years); adults (15 years +)]. RESULTS: Hospital admission rates and bed days for all psychiatric disorders decreased by 28.4 and 38.3%, respectively. Trends were not uniform across psychiatric disorders or age groups. Admission rates mainly decreased over time, except for anxiety and eating disorders which doubled over the 22-year period, significantly increasing by 2.9% (AAPC = 2.88; 95% CI: 2.61-3.16; p < 0.001) and 3.4% (AAPC = 3.44; 95% CI: 3.04-3.85; p < 0.001) each year. Inpatient hospital activity among children showed more increasing and pronounced trends than adults, including an increase of 212.9% for depression, despite a 63.8% reduction for adults with depression during the same period. CONCLUSION: In the last 22 years, there have been overall reductions in hospital activity for psychiatric disorders. However, some disorders showed pronounced increases, pointing to areas of growing need for inpatient psychiatric care, especially among children.

The promise of machine learning in predicting treatment outcomes in psychiatry.

For many years, psychiatrists have tried to understand factors involved in response to medications or psychotherapies, in order to personalize their treatment choices. There is now a broad and growing interest in the idea that we can develop models to personalize treatment decisions using new statistical approaches from the field of machine learning and applying them to larger volumes of data. In this pursuit, there has been a paradigm shift away from experimental studies to confirm or refute specific hypotheses towards a focus on the overall explanatory power of a predictive model when tested on new, unseen datasets. In this paper, we review key studies using machine learning to predict treatment outcomes in psychiatry, ranging from medications and psychotherapies to digital interventions and neurobiological treatments. Next, we focus on some new sources of data that are being used for the development of predictive models based on machine learning, such as electronic health records, smartphone and social media data, and on the potential utility of data from genetics, electrophysiology, neuroimaging and cognitive testing. Finally, we discuss how far the field has come towards implementing prediction tools in real-world clinical practice. Relatively few retrospective studies to-date include appropriate external validation procedures, and there are even fewer prospective studies testing the clinical feasibility and effectiveness of predictive models. Applications of machine learning in psychiatry face some of the same ethical challenges posed by these techniques in other areas of medicine or computer science, which we discuss here. In short, machine learning is a nascent but important approach to improve the effectiveness of mental health care, and several prospective clinical studies suggest that it may be working already.

Convergent molecular, cellular, and cortical neuroimaging signatures of major depressive disorder.

Major depressive disorder emerges from the complex interactions of biological systems that span genes and molecules through cells, networks, and behavior. Establishing how neurobiological processes coalesce to contribute to depression requires a multiscale approach, encompassing measures of brain structure and function as well as genetic and cell-specific transcriptional data. Here, we examine anatomical (cortical thickness) and functional (functional variability, global brain connectivity) correlates of depression and negative affect across three population-imaging datasets: UK Biobank, Brain Genomics Superstruct Project, and Enhancing NeuroImaging through Meta Analysis (ENIGMA; combined n ≥ 23,723). Integrative analyses incorporate measures of cortical gene expression, postmortem patient transcriptional data, depression genome-wide association study (GWAS), and single-cell gene transcription. Neuroimaging correlates of depression and negative affect were consistent across three independent datasets. Linking ex vivo gene down-regulation with in vivo neuroimaging, we find that transcriptional correlates of depression imaging phenotypes track gene down-regulation in postmortem cortical samples of patients with depression. Integrated analysis of single-cell and Allen Human Brain Atlas expression data reveal somatostatin interneurons and astrocytes to be consistent cell associates of depression, through both in vivo imaging and ex vivo cortical gene dysregulation. Providing converging evidence for these observations, GWAS-derived polygenic risk for depression was enriched for genes expressed in interneurons, but not glia. Underscoring the translational potential of multiscale approaches, the transcriptional correlates of depression-linked brain function and structure were enriched for disorder-relevant molecular pathways. These findings bridge levels to connect specific genes, cell classes, and biological pathways to in vivo imaging correlates of depression.

Symptom clusters in adolescent depression and differential response to treatment: a secondary analysis of the Treatment for Adolescents with Depression Study randomised trial.

BACKGROUND: Better understanding of the heterogeneity of treatment responses could help to improve care for adolescents with depression. We analysed data from a clinical trial to assess whether specific symptom clusters responded differently to various treatments. METHODS: For this secondary analysis, we used data from the Treatment for Adolescents with Depression Study (TADS), in which 439 US adolescents aged 12-17 with a DSM-IV diagnosis of major depressive disorder and a minimum score of 45 on the Children's Depression Rating Scale-Revised (CDRS-R) were randomly assigned (1:1:1:1) to treatment with fluoxetine, cognitive behavioural therapy (CBT), fluoxetine plus CBT, or pill placebo. Our analysis focuses on the acute phase of the trial (ie, the first 12 weeks). Groups of co-occurring symptoms were established by clustering scores for each CDRS-R item at baseline with Ward's method, with Euclidean distances for hierarchical agglomerative clustering. We then used a linear mixed-effects model to investigate the relationship between symptom clusters and treatment efficacy, with the sum of symptom scores within each cluster as the dependent measure. As fixed effects, we entered cluster, time, and treatment assignment, with all two-way and three-way interactions, into the model. The random effect providing better fit was established to be a by-subject random slope for cluster based on improvement in the Schwarz-Bayesian information criterion. OUTCOMES: We identified two symptom clusters: cluster 1 comprised depressed mood, difficulty having fun, irritability, social withdrawal, sleep disturbance, impaired schoolwork, excessive fatigue, and low self-esteem, and cluster 2 comprised increased appetite, physical complaints, excessive weeping, decreased appetite, excessive guilt, morbid ideation, and suicidal ideation. For cluster 1 symptoms, CDRS-R scores were reduced by 5·8 points (95% CI 2·8-8·9) in adolescents treated with fluoxetine plus CBT, and by 4·1 points (1·1-7·1) in those treated with fluoxetine, compared with those given placebo. For cluster 2 symptoms, no significant differences in improvements in CDRS-R scores were detected between the active treatment and placebo groups. INTERPRETATION: Response to fluoxetine and CBT among adolescents with depression is heterogeneous. Clinicians should consider clinical profile when selecting therapeutic modality. The contrast in response patterns between symptom clusters could provide opportunities to improve treatment efficacy by gearing the development of new therapies towards the resolution of specific symptoms. FUNDING: Conselho Nacional de Desenvolvimento Científico e Tecnológico.

Machine learning and big data in psychiatry: toward clinical applications.

Psychiatry is a medical field concerned with the treatment of mental illness. Psychiatric disorders broadly relate to higher functions of the brain, and as such are richly intertwined with social, cultural, and experiential factors. This makes them exquisitely complex phenomena that depend on and interact with a large number of variables. Computational psychiatry provides two ways of approaching this complexity. Theory-driven computational approaches employ mechanistic models to make explicit hypotheses at multiple levels of analysis. Data-driven machine-learning approaches can make predictions from high-dimensional data and are generally agnostic as to the underlying mechanisms. Here, we review recent advances in the use of big data and machine-learning approaches toward the aim of alleviating the suffering that arises from psychiatric disorders.

Association between physical exercise and mental health in 1·2 million individuals in the USA between 2011 and 2015: a cross-sectional study.

BACKGROUND: Exercise is known to be associated with reduced risk of all-cause mortality, cardiovascular disease, stroke, and diabetes, but its association with mental health remains unclear. We aimed to examine the association between exercise and mental health burden in a large sample, and to better understand the influence of exercise type, frequency, duration, and intensity. METHODS: In this cross-sectional study, we analysed data from 1 237 194 people aged 18 years or older in the USA from the 2011, 2013, and 2015 Centers for Disease Control and Prevention Behavioral Risk Factors Surveillance System survey. We compared the number of days of bad self-reported mental health between individuals who exercised and those who did not, using an exact non-parametric matching procedure to balance the two groups in terms of age, race, gender, marital status, income, education level, body-mass index category, self-reported physical health, and previous diagnosis of depression. We examined the effects of exercise type, duration, frequency, and intensity using regression methods adjusted for potential confounders, and did multiple sensitivity analyses. FINDINGS: Individuals who exercised had 1·49 (43·2%) fewer days of poor mental health in the past month than individuals who did not exercise but were otherwise matched for several physical and sociodemographic characteristics (W=7·42 × 1010, p<2·2 × 10-16). All exercise types were associated with a lower mental health burden (minimum reduction of 11·8% and maximum reduction of 22·3%) than not exercising (p<2·2 × 10-16 for all exercise types). The largest associations were seen for popular team sports (22·3% lower), cycling (21·6% lower), and aerobic and gym activities (20·1% lower), as well as durations of 45 min and frequencies of three to five times per week. INTERPRETATION: In a large US sample, physical exercise was significantly and meaningfully associated with self-reported mental health burden in the past month. More exercise was not always better. Differences as a function of exercise were large relative to other demographic variables such as education and income. Specific types, durations, and frequencies of exercise might be more effective clinical targets than others for reducing mental health burden, and merit interventional study. FUNDING: Cloud computing resources were provided by Microsoft.

Predicting Barriers to Treatment for Depression in a U.S. National Sample: A Cross-Sectional, Proof-of-Concept Study.

OBJECTIVE: Even though safe and effective treatments for depression are available, many individuals with a diagnosis of depression do not obtain treatment. This study aimed to develop a tool to identify persons who might not initiate treatment among those who acknowledge a need. METHODS: Data were aggregated from the 2008-2014 U.S. National Survey on Drug Use and Health (N=391,753), including 20,785 adults given a diagnosis of depression by a health care provider in the 12 months before the survey. Machine learning was applied to self-report survey items to develop strategies for identifying individuals who might not get needed treatment. RESULTS: A derivation cohort aggregated between 2008 and 2013 was used to develop a model that identified the 30.6% of individuals with depression who reported needing but not getting treatment. When applied to independent responses from the 2014 cohort, the model identified 72% of those who did not initiate treatment (p<.01), with a balanced accuracy that was also significantly above chance (71%, p<.01). For individuals who did not get treatment, the model predicted 10 (out of 15) reasons that they endorsed as barriers to treatment, with balanced accuracies between 53% and 65% (p<.05 for all). CONCLUSIONS: Considerable work is needed to improve follow-up and retention rates after the critical initial meeting in which a patient is given a diagnosis of depression. Routinely collected information about patients with depression could identify those at risk of not obtaining needed treatment, which may inform the development and implementation of interventions to reduce the prevalence of untreated depression.

Multivariate Pattern Analysis of Genotype-Phenotype Relationships in Schizophrenia.

Genetic risk variants for schizophrenia have been linked to many related clinical and biological phenotypes with the hopes of delineating how individual variation across thousands of variants corresponds to the clinical and etiologic heterogeneity within schizophrenia. This has primarily been done using risk score profiling, which aggregates effects across all variants into a single predictor. While effective, this method lacks flexibility in certain domains: risk scores cannot capture nonlinear effects and do not employ any variable selection. We used random forest, an algorithm with this flexibility designed to maximize predictive power, to predict 6 cognitive endophenotypes in a combined sample of psychiatric patients and controls (N = 739) using 77 genetic variants strongly associated with schizophrenia. Tenfold cross-validation was applied to the discovery sample and models were externally validated in an independent sample of similar ancestry (N = 336). Linear approaches, including linear regression and task-specific polygenic risk scores, were employed for comparison. Random forest models for processing speed (P = .019) and visual memory (P = .036) and risk scores developed for verbal (P = .042) and working memory (P = .037) successfully generalized to an independent sample with similar predictive strength and error. As such, we suggest that both methods may be useful for mapping a limited set of predetermined, disease-associated SNPs to related phenotypes. Incorporating random forest and other more flexible algorithms into genotype-phenotype mapping inquiries could contribute to parsing heterogeneity within schizophrenia; such algorithms can perform as well as standard methods and can capture a more comprehensive set of potential relationships.

PTSD and the War of Words.

Trauma-related symptoms among veterans of military engagement have been documented at least since the time of the ancient Greeks.1 Since the third edition of the Diagnostic and Statistical Manual in 1980, this condition has been known as posttraumatic stress disorder, but the name has changed repeatedly over the past century, including shell shock, war neurosis, and soldier's heart. Using over 14 million articles in the digital archives of the New York Times, Associated Press, and Reuters, we quantify historical changes in trauma-related terminology over the past century. These data suggest that posttraumatic stress disorder has historically peaked in public awareness after the end of US military engagements, but denoted by a different name each time-a phenomenon that could impede clinical and scientific progress.