Optimized Risk Score to Predict Mortality in Patients With Cardiogenic Shock in the Cardiac Intensive Care Unit.

Background Mortality prediction in critically ill patients with cardiogenic shock can guide triage and selection of potentially high-risk treatment options. Methods and Results We developed and externally validated a checklist risk score to predict in-hospital mortality among adults admitted to the cardiac intensive care unit with Society for Cardiovascular Angiography & Interventions Shock Stage C or greater cardiogenic shock using 2 real-world data sets and Risk-Calibrated Super-sparse Linear Integer Modeling (RiskSLIM). We compared this model to those developed using conventional penalized logistic regression and published cardiogenic shock and intensive care unit mortality prediction models. There were 8815 patients in our training cohort (in-hospital mortality 13.4%) and 2237 patients in our validation cohort (in-hospital mortality 22.8%), and there were 39 candidate predictor variables. The final risk score (termed BOS,MA2) included maximum blood urea nitrogen ≥25 mg/dL, minimum oxygen saturation <88%, minimum systolic blood pressure <80 mm Hg, use of mechanical ventilation, age ≥60 years, and maximum anion gap ≥14 mmol/L, based on values recorded during the first 24 hours of intensive care unit stay. Predicted in-hospital mortality ranged from 0.5% for a score of 0 to 70.2% for a score of 6. The area under the receiver operating curve was 0.83 (0.82-0.84) in training and 0.76 (0.73-0.78) in validation, and the expected calibration error was 0.9% in training and 2.6% in validation. Conclusions Developed using a novel machine learning method and the largest cardiogenic shock cohorts among published models, BOS,MA2 is a simple, clinically interpretable risk score that has improved performance compared with existing cardiogenic-shock risk scores and better calibration than general intensive care unit risk scores.

Optimized short-forms of the Cognitive Distortions Questionnaire.

INTRODUCTION: The Cognitive Distortions Questionnaire (CD-Quest) is a self-report questionnaire that assesses common cognitive distortions. Although the CD-Quest has excellent psychometric properties, its length may limit its use. METHODS: We attempted to develop short-forms of the CD-Quest using RiskSLIM - a machine learning method to build short-form scales that can be scored by hand. Each short-form was fit to maximize concordance with the total CD-Quest score for a specified number of items based on an objective function, in this case R2, by selecting an optimal subset of items and an optimal set of small integer weights. The models were trained in a sample of US undergraduate students (N = 906). We then validated each short-form on five independent samples: two samples of undergraduate students in Brazil (Ns = 182, 183); patients with depression in Brazil (N = 62); patients with social anxiety disorder in the US (N = 198); and psychiatric outpatients in Turkey (N = 269). RESULTS: A 9-item short-form with integer scoring was created that reproduced the total 15-item CD-Quest score in all validation samples with excellent accuracy (R2 = 90.4-93.6%). A 5-item ultra-short-form had good accuracy (R2 = 78.2-85.5%). DISCUSSION: A 9-item short-form and a 5-item ultra-short-form of the CD-Quest both reproduced full CD-Quest scores with excellent to good accuracy. These shorter versions of the full CD-Quest could facilitate measurement of cognitive distortions for users with limited time and resources.

Development and Validation of a Model to Predict Posttraumatic Stress Disorder and Major Depression After a Motor Vehicle Collision.

Importance: A substantial proportion of the 40 million people in the US who present to emergency departments (EDs) each year after traumatic events develop posttraumatic stress disorder (PTSD) or major depressive episode (MDE). Accurately identifying patients at high risk in the ED would facilitate the targeting of preventive interventions. Objectives: To develop and validate a prediction tool based on ED reports after a motor vehicle collision to predict PTSD or MDE 3 months later. Design, Setting, and Participants: The Advancing Understanding of Recovery After Trauma (AURORA) study is a longitudinal study that examined adverse posttraumatic neuropsychiatric sequalae among patients who presented to 28 US urban EDs in the immediate aftermath of a traumatic experience. Enrollment began on September 25, 2017. The 1003 patients considered in this diagnostic/prognostic report completed 3-month assessments by January 31, 2020. Each patient received a baseline ED assessment along with follow-up self-report surveys 2 weeks, 8 weeks, and 3 months later. An ensemble machine learning method was used to predict 3-month PTSD or MDE from baseline information. Data analysis was performed from November 1, 2020, to May 31, 2021. Main Outcomes and Measures: The PTSD Checklist for DSM-5 was used to assess PTSD and the Patient Reported Outcomes Measurement Information System Depression Short-Form 8b to assess MDE. Results: A total of 1003 patients (median [interquartile range] age, 34.5 [24-43] years; 715 [weighted 67.9%] female; 100 [weighted 10.7%] Hispanic, 537 [weighted 52.7%] non-Hispanic Black, 324 [weighted 32.2%] non-Hispanic White, and 42 [weighted 4.4%] of non-Hispanic other race or ethnicity were included in this study. A total of 274 patients (weighted 26.6%) met criteria for 3-month PTSD or MDE. An ensemble machine learning model restricted to 30 predictors estimated in a training sample (patients from the Northeast or Midwest) had good prediction accuracy (mean [SE] area under the curve [AUC], 0.815 [0.031]) and calibration (mean [SE] integrated calibration index, 0.040 [0.002]; mean [SE] expected calibration error, 0.039 [0.002]) in an independent test sample (patients from the South). Patients in the top 30% of predicted risk accounted for 65% of all 3-month PTSD or MDE, with a mean (SE) positive predictive value of 58.2% (6.4%) among these patients at high risk. The model had good consistency across regions of the country in terms of both AUC (mean [SE], 0.789 [0.025] using the Northeast as the test sample and 0.809 [0.023] using the Midwest as the test sample) and calibration (mean [SE] integrated calibration index, 0.048 [0.003] using the Northeast as the test sample and 0.024 [0.001] using the Midwest as the test sample; mean [SE] expected calibration error, 0.034 [0.003] using the Northeast as the test sample and 0.025 [0.001] using the Midwest as the test sample). The most important predictors in terms of Shapley Additive Explanations values were symptoms of anxiety sensitivity and depressive disposition, psychological distress in the 30 days before motor vehicle collision, and peritraumatic psychosomatic symptoms. Conclusions and Relevance: The results of this study suggest that a short set of questions feasible to administer in an ED can predict 3-month PTSD or MDE with good AUC, calibration, and geographic consistency. Patients at high risk can be identified in the ED for targeting if cost-effective preventive interventions are developed.

Developing an optimal short-form of the PTSD Checklist for DSM-5 (PCL-5).

BACKGROUND: Although several short-forms of the posttraumatic stress disorder (PTSD) Checklist (PCL) exist, all were developed using heuristic methods. This report presents the results of analyses designed to create an optimal short-form PCL for DSM-5 (PCL-5) using both machine learning and conventional scale development methods. METHODS: The short-form scales were developed using independent datasets collected by the Army Study to Assess Risk and Resilience among Service members. We began by using a training dataset (n = 8,917) to fit short-form scales with between 1 and 8 items using different statistical methods (exploratory factor analysis, stepwise logistic regression, and a new machine learning method to find an optimal integer-scored short-form scale) to predict dichotomous PTSD diagnoses determined using the full PCL-5. A smaller subset of best short-form scales was then evaluated in an independent validation sample (n = 11,728) to select one optimal short-form scale based on multiple operating characteristics (area under curve [AUC], calibration, sensitivity, specificity, net benefit). RESULTS: Inspection of AUCs in the training sample and replication in the validation sample led to a focus on 4-item integer-scored short-form scales selected with stepwise regression. Brier scores in the validation sample showed that a number of these scales had comparable calibration (0.015-0.032) and AUC (0.984-0.994), but that one had consistently highest net benefit across a plausible range of decision thresholds. CONCLUSIONS: The recommended 4-item integer-scored short-form PCL-5 generates diagnoses that closely parallel those of the full PCL-5, making it well-suited for screening.

Association of an Electroencephalography-Based Risk Score With Seizure Probability in Hospitalized Patients.

Importance: Continuous electroencephalography (EEG) use in critically ill patients is expanding. There is no validated method to combine risk factors and guide clinicians in assessing seizure risk. Objective: To use seizure risk factors from EEG and clinical history to create a simple scoring system associated with the probability of seizures in patients with acute illness. Design, Setting, and Participants: We used a prospective multicenter (Emory University Hospital, Brigham and Women's Hospital, and Yale University Hospital) database containing clinical and electrographic variables on 5427 continuous EEG sessions from eligible patients if they had continuous EEG for clinical indications, excluding epilepsy monitoring unit admissions. We created a scoring system model to estimate seizure risk in acutely ill patients undergoing continuous EEG. The model was built using a new machine learning method (RiskSLIM) that is designed to produce accurate, risk-calibrated scoring systems with a limited number of variables and small integer weights. We validated the accuracy and risk calibration of our model using cross-validation and compared its performance with models built with state-of-the-art logistic regression methods. The database was developed by the Critical Care EEG Research Consortium and used data collected over 3 years. The EEG variables were interpreted using standardized terminology by certified reviewers. Exposures: All patients had more than 6 hours of uninterrupted EEG recordings. Main Outcomes and Measures: The main outcome was the average risk calibration error. Results: There were 5427 continuous EEGs performed on 4772 participants (2868 men, 49.9%; median age, 61 years) performed at 3 institutions, without further demographic stratification. Our final model, 2HELPS2B, had an area under the curve of 0.819 and average calibration error of 2.7% (95% CI, 2.0%-3.6%). It included 6 variables with the following point assignments: (1) brief (ictal) rhythmic discharges (B[I]RDs) (2 points); (2) presence of lateralized periodic discharges, lateralized rhythmic delta activity, or bilateral independent periodic discharges (1 point); (3) prior seizure (1 point); (4) sporadic epileptiform discharges (1 point); (5) frequency greater than 2.0 Hz for any periodic or rhythmic pattern (1 point); and (6) presence of "plus" features (superimposed, rhythmic, sharp, or fast activity) (1 point). The probable seizure risk of each score was 5% for a score of 0, 12% for a score of 1, 27% for a score of 2, 50% for a score of 3, 73% for a score of 4, 88% for a score of 5, and greater than 95% for a score of 6 or 7. Conclusions and Relevance: The 2HELPS2B model is a quick accurate tool to aid clinical judgment of the risk of seizures in critically ill patients.

The World Health Organization Adult Attention-Deficit/Hyperactivity Disorder Self-Report Screening Scale for DSM-5.

Importance: Recognition that adult attention-deficit/hyperactivity disorder (ADHD) is common, seriously impairing, and usually undiagnosed has led to the development of adult ADHD screening scales for use in community, workplace, and primary care settings. However, these scales are all calibrated to DSM-IV criteria, which are narrower than the recently developed DSM-5 criteria. Objectives: To update for DSM-5 criteria and improve the operating characteristics of the widely used World Health Organization Adult ADHD Self-Report Scale (ASRS) for screening. Design, Setting, and Participants: Probability subsamples of participants in 2 general population surveys (2001-2003 household survey [n = 119] and 2004-2005 managed care subscriber survey [n = 218]) who completed the full 29-question self-report ASRS, with both subsamples over-sampling ASRS-screened positives, were blindly administered a semistructured research diagnostic interview for DSM-5 adult ADHD. In 2016, the Risk-Calibrated Supersparse Linear Integer Model, a novel machine-learning algorithm designed to create screening scales with optimal integer weights and limited numbers of screening questions, was applied to the pooled data to create a DSM-5 version of the ASRS screening scale. The accuracy of the new scale was then confirmed in an independent 2011-2012 clinical sample of patients seeking evaluation at the New York University Langone Medical Center Adult ADHD Program (NYU Langone) and 2015-2016 primary care controls (n = 300). Data analysis was conducted from April 4, 2016, to September 22, 2016. Main Outcomes and Measures: The sensitivity, specificity, area under the curve (AUC), and positive predictive value (PPV) of the revised ASRS. Results: Of the total 637 participants, 44 (37.0%) household survey respondents, 51 (23.4%) managed care respondents, and 173 (57.7%) NYU Langone respondents met DSM-5 criteria for adult ADHD in the semistructured diagnostic interview. Of the respondents who met DSM-5 criteria for adult ADHD, 123 were male (45.9%); mean (SD) age was 33.1 (11.4) years. A 6-question screening scale was found to be optimal in distinguishing cases from noncases in the first 2 samples. Operating characteristics were excellent at the diagnostic threshold in the weighted (to the 8.2% DSM-5/Adult ADHD Clinical Diagnostic Scale population prevalence) data (sensitivity, 91.4%; specificity, 96.0%; AUC, 0.94; PPV, 67.3%). Operating characteristics were similar despite a much higher prevalence (57.7%) when the scale was applied to the NYU Langone clinical sample (sensitivity, 91.9%; specificity, 74.0%; AUC, 0.83; PPV, 82.8%). Conclusions and Relevance: The new ADHD screening scale is short, easily scored, detects the vast majority of general population cases at a threshold that also has high specificity and PPV, and could be used as a screening tool in specialty treatment settings.

Clinical Prediction Models for Sleep Apnea: The Importance of Medical History over Symptoms.

STUDY OBJECTIVE: Obstructive sleep apnea (OSA) is a treatable contributor to morbidity and mortality. However, most patients with OSA remain undiagnosed. We used a new machine learning method known as SLIM (Supersparse Linear Integer Models) to test the hypothesis that a diagnostic screening tool based on routinely available medical information would be superior to one based solely on patient-reported sleep-related symptoms. METHODS: We analyzed polysomnography (PSG) and self-reported clinical information from 1,922 patients tested in our clinical sleep laboratory. We used SLIM and 7 state-of-the-art classification methods to produce predictive models for OSA screening using features from: (i) self-reported symptoms; (ii) self-reported medical information that could, in principle, be extracted from electronic health records (demographics, comorbidities), or (iii) both. RESULTS: For diagnosing OSA, we found that model performance using only medical history features was superior to model performance using symptoms alone, and similar to model performance using all features. Performance was similar to that reported for other widely used tools: sensitivity 64.2% and specificity 77%. SLIM accuracy was similar to state-of-the-art classification models applied to this dataset, but with the benefit of full transparency, allowing for hands-on prediction using yes/no answers to a small number of clinical queries. CONCLUSION: To predict OSA, variables such as age, sex, BMI, and medical history are superior to the symptom variables we examined for predicting OSA. SLIM produces an actionable clinical tool that can be applied to data that is routinely available in modern electronic health records, which may facilitate automated, rather than manual, OSA screening. COMMENTARY: A commentary on this article appears in this issue on page 159.