External Validation of Stockholm3 in a Retrospective German Clinical Cohort.

BACKGROUND AND OBJECTIVE: Stockholm3 is a comprehensive blood test amalgamating protein biomarkers, genetic indicators, and clinical data to predict clinically significant prostate cancer risk (International Society of Urological Pathology grade ≥2 upon biopsy). Our study aims to externally validate Stockholm3 and compare its performance with the use of prostate-specific antigen (PSA) and the Rotterdam Prostate Cancer Risk Calculator (RPCRC) for clinically significant prostate cancer detection. METHODS: We gathered data from men subjected to prostate biopsies at the Martini-Klinik, Germany, between 2014 and 2017. Participants were selected based on elevated PSA levels or suspicious digital rectal examinations, all undergoing a 10-12-core systematic biopsy without a magnetic resonance imaging-targeted biopsy. We assessed Stockholm3 and RPCRC performance for clinically significant prostate cancer detection. Furthermore, we compared the proportion of men recommended for biopsy and biopsy outcomes with Stockholm3 and RPCRC against PSA ≥3 ng/ml. KEY FINDINGS AND LIMITATIONS: Our study encompassed 405 biopsied men, with a median age of 66 yr (interquartile range [IQR]: 60-72), PSA levels at 7 ng/ml (IQR: 5.2-10.8), and Stockholm3 scores at 18 (IQR: 10-34). Among them, 128 men (31%) received clinically significant prostate cancer diagnoses. Employing the recommended Stockholm3 threshold (≥15) could have reduced unnecessary biopsies by 52%, while detecting 92% of clinically significant cases compared with using PSA ≥3 ng/ml as a biopsy criterion. Both Stockholm3 and RPCRC exhibited strong discrimination, with area under the curve values of 0.80 (95% confidence interval [CI]: 0.76-0.85) and 0.75 (95% CI: 0.70-0.80), respectively. Stockholm3 demonstrated good calibration, while RPCRC underestimated the risk compared with observed outcomes. Moreover, Stockholm3 yielded positive clinical net benefits, whereas RPCRC yielded negative net benefits for clinically relevant thresholds. CONCLUSIONS AND CLINICAL IMPLICATIONS: Stockholm3 utilization could detect 92% of clinically significant prostate cancer cases while simultaneously reducing unnecessary biopsies by 52%, compared with the PSA ≥3 ng/ml criterion, based on our analysis within a cohort of men who underwent systematic biopsies. PATIENT SUMMARY: In a German clinical cohort of 405 men, Stockholm3, a blood test for early prostate cancer detection, exhibited favorable clinical benefits. It identified a substantial number of clinically significant cases while reducing unnecessary biopsies by over half in men without the disease and those with clinically nonsignificant prostate cancer.

A machine learning-based predictive model for the in-hospital mortality of critically ill patients with atrial fibrillation.

BACKGROUND: Atrial fibrillation (AF) is common among intensive care unit (ICU) patients and significantly raises the in-hospital mortality rate. Existing scoring systems or models have limited predictive capabilities for AF patients in ICU. Our study developed and validated machine learning models to predict the risk of in-hospital mortality in ICU patients with AF. METHODS AND RESULTS: Medical Information Mart for Intensive Care (MIMIC)-IV dataset and eICU Collaborative Research Database (eICU-CRD) were analyzed. Among ten classifiers compared, adaptive boosting (AdaBoost) showed better performance in predicting all-cause mortality in AF patients. A compact model with 15 features was developed and validated. Both the all variable and compact models exhibited excellent performance with area under the receiver operating characteristic curves (AUCs) of 1(95%confidence interval [CI]: 1.0-1.0) in the training set. In the MIMIC-IV testing set, the AUCs of the all variable and compact models were 0.978 (95% CI: 0.973-0.982) and 0.977 (95% CI: 0.972-0.982), respectively. In the external validation set, the AUCs of all variable and compact models were 0.825 (95% CI: 0.815-0.834) and 0.807 (95% CI: 0.796-0.817), respectively. CONCLUSION: An AdaBoost-based predictive model was subjected to internal and external validation, highlighting its strong predictive capacity for assessing the risk of in-hospital mortality in ICU patients with AF.

Development and validation of a new scoring system to discriminate between uncomplicated and complicated appendicitis.

A scoring system to discriminate between uncomplicated and complicated appendicitis is beneficial to determine the optimal treatment for acute appendicitis. We developed a scoring system to discriminate between uncomplicated and complicated appendicitis and assessed the clinical usefulness of the scoring system using external validation. A total of 299 patients with acute appendicitis were retrospectively reviewed. One hundred and ninety-nine patients were assigned to the model development group, while the other 100 patients were assigned to an external validation group. A scoring system for complicated appendicitis was created using a final multivariate logistic regression model with six independent predictors. The area under the receiver operating characteristic curve of the scoring system was 0.882 (95% confidence interval: 0.835-0.929). The cutoff point of the scoring system was 12, and the sensitivity and specificity were 82.9% and 86.2%, respectively. In the external validation group, the area under the receiver operating characteristic curve of the scoring system was 0.868 (95% confidence interval 0.794-0.942), and there was no significant difference between the groups in the area under the receiver operating characteristic curve (P = 0.750). Our newly developed scoring system may contribute to prompt determination of the optimal treatment for acute appendicitis.

External validation and comparison of four prediction scores for severe maternal morbidity.

BACKGROUND: Severe maternal morbidity is increasing in the United States. Several tools and scores exist to stratify an individual's risk of severe maternal morbidity. OBJECTIVE: We sought to examine and compare the validity of four scoring systems for predicting severe maternal morbidity. STUDY DESIGN: This was a retrospective cohort study of all individuals in the Consortium on Safe Labor dataset, which was conducted from 2002 to 2008. Individuals were excluded if they had missing information on risk factors. Severe maternal morbidity was defined based on the Centers for Disease Control and Prevention excluding blood transfusion. Blood transfusion was excluded due to concerns regarding the specificity of ICD codes for this indicator and its variable clinical significance. Risk scores were calculated for each participant using the Assessment of Perinatal Excellence, California Maternal Quality Care Collaborative, Obstetric Comorbidity Index, and Modified Obstetric Comorbidity Index. We calculated the probability of severe maternal morbidity according to the risk scores. The discriminative performance of the prediction score was examined by the areas under receiver operating characteristic curves and their 95% confidence intervals. The area under the curve for each score was compared using the bootstrap resampling. Calibration plots were developed for each score to examine the goodness-of-fit. The concordance probability method was used to define an optimal cutoff point for the best-performing score. RESULTS: Of 153, 463 individuals, 1,115 (0.7%) had severe maternal morbidity. The California Maternal Quality Care Collaborative scoring system had a significantly higher area under the curve [95% confidence interval] (0.78 [0.77-0.80]) compared to the Assessment of Perinatal Excellence scoring system, Obstetric Comorbidity Index and Modified Obstetric Comorbidity Index scoring systems 0.75 [0.73-0.76],. 0.67 [0.65-0.68], 0.66 [0.70-0.73]; P < 0.001). Calibration plots showed excellent concordance between the predicted and actual severe maternal morbidity for the Assessment of Perinatal Excellence scoring system and Obstetric Comorbidity Index (both Hosmer-Lemeshow test P-values = 1.00, suggesting goodness-of-fit). CONCLUSION: This study validated four risk-scoring systems to predict severe maternal morbidity. Both California Maternal Quality Care Collaborative and Assessment of Perinatal Excellence scoring systems had good discrimination to predict severe maternal morbidity. The Assessment of Perinatal Excellence score and the Obstetric Comorbidity Index had goodness-of-fit. At ideal calculated cut-off points, the Assessment of Perinatal Excellence score had the highest sensitivity of the four scores at 71%, indicating that better scoring systems are still needed for predicting severe maternal morbidity.

Derivation and external validation of predictive models for invasive mechanical ventilation in intensive care unit patients with COVID-19.

BACKGROUND: This study aimed to develop prognostic models for predicting the need for invasive mechanical ventilation (IMV) in intensive care unit (ICU) patients with COVID-19 and compare their performance with the Respiratory rate-OXygenation (ROX) index. METHODS: A retrospective cohort study was conducted using data collected between March 2020 and August 2021 at three hospitals in Rio de Janeiro, Brazil. ICU patients aged 18 years and older with a diagnosis of COVID-19 were screened. The exclusion criteria were patients who received IMV within the first 24 h of ICU admission, pregnancy, clinical decision for minimal end-of-life care and missing primary outcome data. Clinical and laboratory variables were collected. Multiple logistic regression analysis was performed to select predictor variables. Models were based on the lowest Akaike Information Criteria (AIC) and lowest AIC with significant p values. Assessment of predictive performance was done for discrimination and calibration. Areas under the curves (AUC)s were compared using DeLong's algorithm. Models were validated externally using an international database. RESULTS: Of 656 patients screened, 346 patients were included; 155 required IMV (44.8%), 191 did not (55.2%), and 207 patients were male (59.8%). According to the lowest AIC, arterial hypertension, diabetes mellitus, obesity, Sequential Organ Failure Assessment (SOFA) score, heart rate, respiratory rate, peripheral oxygen saturation (SpO2), temperature, respiratory effort signals, and leukocytes were identified as predictors of IMV at hospital admission. According to AIC with significant p values, SOFA score, SpO2, and respiratory effort signals were the best predictors of IMV; odds ratios (95% confidence interval): 1.46 (1.07-2.05), 0.81 (0.72-0.90), 9.13 (3.29-28.67), respectively. The ROX index at admission was lower in the IMV group than in the non-IMV group (7.3 [5.2-9.8] versus 9.6 [6.8-12.9], p < 0.001, respectively). In the external validation population, the area under the curve (AUC) of the ROX index was 0.683 (accuracy 63%), the AIC model showed an AUC of 0.703 (accuracy 69%), and the lowest AIC model with significant p values had an AUC of 0.725 (accuracy 79%). CONCLUSIONS: In the development population of ICU patients with COVID-19, SOFA score, SpO2, and respiratory effort signals predicted the need for IMV better than the ROX index. In the external validation population, although the AUCs did not differ significantly, the accuracy was higher when using SOFA score, SpO2, and respiratory effort signals compared to the ROX index. This suggests that these variables may be more useful in predicting the need for IMV in ICU patients with COVID-19. GOV IDENTIFIER: NCT05663528.

Prospective external validation of radiomics-based predictive model of distant metastasis after dynamic tumor tracking stereotactic body radiation therapy in patients with non-small-cell lung cancer: A multi-institutional analysis.

BACKGROUND AND PURPOSE: This study aims to externally validate a predictive model for distant metastasis (DM) with computed tomography (CT)-based radiomics features in prospectively enrolled non-small-cell lung cancer patients undergoing dynamic tumor-tracking stereotactic body radiation therapy (DTT-SBRT). MATERIALS AND METHODS: The study collected retrospective data from 567 patients across 11 institutions as the training dataset and prospectively enrolled 42 patients from four institutions as the external test dataset. Four clinical features were collected, and 944 CT-based radiomic features were extracted from gross tumor volumes. After standardization and feature selection, DM predictive models were developed using fine and gray regression (FG) and random survival forest (RSF), incorporating clinical and radiomic features, and their combinations within the training dataset. Then, the model was applied to the test dataset, dividing patients into high- and low-risk groups based on medians of risk scores. Model performance was assessed using the concordance index (C-index), and the statistical significance between groups was evaluated using Gray's test. RESULTS: In the training dataset, 122 of 567 patients (21.5%) developed DM, compared to 9 of 42 patients (21.4%) in the test dataset. In the test dataset, the C-indices of the clinical, radiomics, and hybrid models with FG were 0.559, 0.544, and 0.560, respectively, whereas those with RSF were 0.576, 0.604, and 0.627, respectively. The hybrid model with RSF, which exhibited the best predictive performance of all models, identified 7 of 23 patients (30.4%) as high risk and 2 of 19 patients (10.5%) as low risk for DM incidence in the test dataset (p = 0.116). CONCLUSION: Although predictive models for DM lack significance when applied to prospectively enrolled cases undergoing DTT-lung SBRT, the model with RSF exhibits a consistent capacity to effectively classify patients at a high risk of developing DM.

External validation of a novel nomogram for diagnosis of Protein Energy Wasting in adult hemodialysis patients.

Background: Protein Energy Wasting (PEW) has high incidence in adult hemodialysis patients and refers to a state of decreased protein and energy substance. It has been demonstrated that PEW highly affects the quality of survival and increases the risk of death. Nevertheless, its diagnostic criteria are complex in clinic. To simplify the diagnosis method of PEW in adult hemodialysis patients, we previously established a novel clinical prediction model that was well-validated internally using bootstrapping. In this multicenter cross-sectional study, we aimed to externally validate this nomogram in a new cohort of adult hemodialysis patients. Methods: The novel prediction model was built by combining four independent variables with part of the International Society of Renal Nutrition and Metabolism (ISRNM) diagnostic criteria including albumin, total cholesterol, and body mass index (BMI). We evaluated the performance of the new model using discrimination (Concordance Index), calibration plots, and Clinical Impact Curve to assess its predictive utility. Results: From September 1st, 2022 to August 31st, 2023, 1,158 patients were screened in five medical centers in Shanghai. 622 (53.7%) hemodialysis patients were included for analysis. The PEW predictive model was acceptable discrimination with the area under the curve of 0.777 (95% CI 0.741-0.814). Additionally, the model revealed well-fitted calibration curves. The McNemar test showed the novel model had similar diagnostic efficacy with the gold standard diagnostic method (p > 0.05). Conclusion: Our results from this cross-sectional external validation study further demonstrate that the novel model is a valid tool to identify PEW in adult hemodialysis patients effectively.

Validation of a model for outcome prediction after endovascular treatment for ischemic stroke.

INTRODUCTION: The recently developed MR-PREDICTS@24 h model showed excellent performance in the MR-CLEAN Registry cohort in patients presenting within 12 h from onset. However, its applicability to an U.S. population and to patients presenting beyond 12 h from last known normal are still undetermined. We aim to externally validate the MR-PREDICTS@24 h model in a new geographic setting and in the late window. METHODS: In this retrospective analysis of a prospectively collected database from a comprehensive stroke center in the United States, we included patients with intracranial carotid artery or middle cerebral artery M1 or M2 segment occlusions who underwent endovascular therapy and applied the MR-PREDICTS@24 h formula to estimate the probabilities of functional outcome at day 90. The primary endpoint was the modified Rankin Scale (mRS) at 90 days. RESULTS: We included 1246 patients, 879 in the early (<12 h) and 367 in the late (≥12 h) cohort. For both cohorts, calibration and discrimination of the model were accurate throughout mRS levels, with absolute differences between estimated and predicted proportions ranging from 1% to 5%. Calibration metrics and curve inspections showed good performance for estimating the probabilities of mRS ≤ 1 to mRS ≤ 5 for the early cohort. For the late cohort, predictions were reliable for the probabilities of mRS ≤ 1 to mRS ≤ 4. CONCLUSION: The MR-PREDICTS@24 h was transferrable to a real-world U.S.-based cohort in the early window and showed consistently accurate predictions for patients presenting in the late window without need for updating.

NEWS2, S/F-ratio and ROX-index at emergency department for the prediction of adverse outcomes in COVID-19 patients: An external validation study.

BACKGROUND: In the context of the COVID-19 pandemic, the early and accurate identification of patients at risk of deterioration was crucial in overcrowded and resource-limited emergency departments. This study conducts an external validation for the evaluation of the performance of the National Early Warning Score 2 (NEWS2), the S/F ratio, and the ROX index at ED admission in a large cohort of COVID-19 patients from Colombia, South America, assessing the net clinical benefit with decision curve analysis. METHODS: A prospective cohort study was conducted on 6907 adult patients with confirmed COVID-19 admitted to a tertiary care ED in Colombia. The study evaluated the diagnostic performance of NEWS2, S/F ratio, and ROX index scores at ED admission using the area under the receiver operating characteristic curve (AUROC) for discrimination, calibration, and decision curve analysis for the prediction of intensive care unit admission, invasive mechanical ventilation, and in-hospital mortality. RESULTS: We included 6907 patients who presented to the ED with confirmed SARS-CoV-2 infection from March 2020 to November 2021. Mean age was 51 (35-65) years and 50.4% of patients were males. The rate of intensive care unit admission was 28%, and in-hospital death was 9.8%. All three scores have good discriminatory performance for the three outcomes based on the AUROC. S/F ratio showed miscalibration at low predicted probabilities and decision curve analysis indicated that the NEWS2 score provided a greater net benefit compared to other scores across at a 10% threshold to decide ED admission at a high-level of care facility. CONCLUSIONS: The NEWS2, S/F ratio, and ROX index at ED admission have good discriminatory performances in COVID-19 patients for the prediction of adverse outcomes, but the NEWS2 score has a higher net benefit underscoring its clinical utility in optimizing patient management and resource allocation in emergency settings.

Predicting negative attitudes towards suicide in social media texts: prediction model development and validation study.

Background: Implementing machine learning prediction of negative attitudes towards suicide may improve health outcomes. However, in previous studies, varied forms of negative attitudes were not adequately considered, and developed models lacked rigorous external validation. By analyzing a large-scale social media dataset (Sina Weibo), this paper aims to fully cover varied forms of negative attitudes and develop a classification model for predicting negative attitudes as a whole, and then to externally validate its performance on population and individual levels. Methods: 938,866 Weibo posts with relevant keywords were downloaded, including 737,849 posts updated between 2009 and 2014 (2009-2014 dataset), and 201,017 posts updated between 2015 and 2020 (2015-2020 dataset). (1) For model development, based on 10,000 randomly selected posts from 2009 to 2014 dataset, a human-based content analysis was performed to manually determine labels of each post (non-negative or negative attitudes). Then, a computer-based content analysis was conducted to automatically extract psycholinguistic features from each of the same 10,000 posts. Finally, a classification model for predicting negative attitudes was developed on selected features. (2) For model validation, on the population level, the developed model was implemented on remaining 727,849 posts from 2009 to 2014 dataset, and was externally validated by comparing proportions of negative attitudes between predicted and human-coded results. Besides, on the individual level, similar analyses were performed on 300 randomly selected posts from 2015 to 2020 dataset, and the developed model was externally validated by comparing labels of each post between predicted and actual results. Results: For model development, the F1 and area under ROC curve (AUC) values reached 0.93 and 0.97. For model validation, on the population level, significant differences but very small effect sizes were observed for the whole sample (χ 2 1 = 32.35, p < 0.001; Cramer's V = 0.007, p < 0.001), men (χ 2 1 = 9.48, p = 0.002; Cramer's V = 0.005, p = 0.002), and women (χ 2 1 = 25.34, p < 0.001; Cramer's V = 0.009, p < 0.001). Besides, on the individual level, the F1 and AUC values reached 0.76 and 0.74. Conclusion: This study demonstrates the efficiency and necessity of machine learning prediction of negative attitudes as a whole, and confirms that external validation is essential before implementing prediction models into practice.

Assessment of Early Vascular Aging Ambulatory Score (EVAAs): A Large Population-based External Validation Study.

BACKGROUND: Pulse Wave Velocity (PWV) remains the gold-standard method to assess Early Vascular Aging (EVA) defined by arterial stiffness. However, its high cost, time-consuming process, and need for qualified medical staff shows the importance of identifying alternative methods for the EVA evaluation. OBJECTIVE: In order to simplify the process of assessing patients' EVA, we recently developed the Early Vascular Aging Ambulatory score (EVAAs), a simple tool to predict the risk of EVA. The aim of the present study was the external validation of EVAAs in an independent population. METHODS: Eight hundred seventy-nine (46.3% men) patients who were referred to our Hypertension ESH Excellence Center were included in this study. The mean age was 46.43 ± 22.87 years. EVA was evaluated in two different ways. The first assessment included c-f PWV values, whereas the second one included EVAAs without the direct measurement of carotid-femoral PWV. RESULTS: The null hypothesis was that the prediction of EVA based on EVAAs does not present any statistically significant difference compared to the prediction based on the calculation from c-f PWV. Mean squared error (MSE) was used for the assessment of the null hypothesis, which was found to be 0.40. The results revealed that the EVAAs show the probability of EVA with 0.98 sensitivity and 0.75 specificity. The EVAAs present 95% positive predictive value and 92% negative predictive value. CONCLUSION: Our study revealed that EVAAs could be as reliable as the carotid-femoral PWV to identify patients with EVA. Hence, we hope that EVAAs will be a useful tool in clinical practice.

External Validation of the Machine Learning-Based Thermographic Indices for Rheumatoid Arthritis: A Prospective Longitudinal Study.

External validation is crucial in developing reliable machine learning models. This study aimed to validate three novel indices-Thermographic Joint Inflammation Score (ThermoJIS), Thermographic Disease Activity Index (ThermoDAI), and Thermographic Disease Activity Index-C-reactive protein (ThermoDAI-CRP)-based on hand thermography and machine learning to assess joint inflammation and disease activity in rheumatoid arthritis (RA) patients. A 12-week prospective observational study was conducted with 77 RA patients recruited from rheumatology departments of three hospitals. During routine care visits, indices were obtained at baseline and week 12 visits using a pre-trained machine learning model. The performance of these indices was assessed cross-sectionally and longitudinally using correlation coefficients, the area under the receiver operating curve (AUROC), sensitivity, specificity, and positive and negative predictive values. ThermoDAI and ThermoDAI-CRP correlated with CDAI, SDAI, and DAS28-CRP cross-sectionally (ρ = 0.81; ρ = 0.83; ρ = 0.78) and longitudinally (ρ = 0.55; ρ = 0.61; ρ = 0.60), all p < 0.001. ThermoDAI and ThermoDAI-CRP also outperformed Patient Global Assessment (PGA) and PGA + C-reactive protein (CRP) in detecting changes in 28-swollen joint counts (SJC28). ThermoJIS had an AUROC of 0.67 (95% CI, 0.58 to 0.76) for detecting patients with swollen joints and effectively identified patients transitioning from SJC28 > 1 at baseline visit to SJC28 ≤ 1 at week 12 visit. These results support the effectiveness of ThermoJIS in assessing joint inflammation, as well as ThermoDAI and ThermoDAI-CRP in evaluating disease activity in RA patients.

Limited Generalizability of Retrospective Single-Center Cohort Study in Comparison to Multicenter Cohort Study on Prognosis of Hepatocellular Carcinoma.

Introduction: We aimed to evaluate the generalizability of retrospective single-center cohort studies on prognosis of hepatocellular carcinoma (HCC) by comparing overall survival (OS) after various treatments between a nationwide multicenter cohort and a single-center cohort of HCC patients. Methods: Patients newly diagnosed with HCC between January 2008 and December 2018 were analyzed using data from the Korean Primary Liver Cancer Registry (multicenter cohort, n=16,443), and the Asan Medical Center HCC registry (single-center cohort, n=15,655). The primary outcome, OS after initial treatment, was compared between the two cohorts for both the entire population and for subcohorts with Child-Pugh A liver function (n=2797 and n=5151, respectively) treated according to the Barcelona-Clinic-Liver-Cancer (BCLC) strategy, using Log rank test and Cox proportional hazard models. Results: Patients of BCLC stages 0 and A (59.3% vs 35.2%) and patients who received curative treatment (42.1% vs 32.1%) were more frequently observed in the single-center cohort (Ps<0.001). Multivariable analysis revealed significant differences between the two cohorts in OS according to type of treatment: the multicenter cohort was associated with higher risk of mortality among patients who received curative (adjusted hazard ratio [95% confidence interval], 1.48 [1.39-1.59]) and non-curative (1.22 [1.17-1.27]) treatments, whereas the risk was lower in patients treated with systemic therapy (0.83 [0.74-0.92]) and best supportive care (0.85 [0.79-0.91]). Subcohort analysis also demonstrated significantly different OS between the two cohorts, with a higher risk of mortality in multicenter cohort patients who received chemoembolization (1.72 [1.48-2.00]) and ablation (1.44 [1.08-1.92]). Conclusion: Comparisons of single-center and multicenter cohorts of HCC patients revealed significant differences in OS according to treatment modality after adjustment for prognostic variables. Therefore, the results of retrospective single-center cohort studies of HCC treatments may not be generalizable to real-world practice.

Multicenter external validation of prediction models for clinical outcomes after spinal fusion for lumbar degenerative disease.

BACKGROUND: Clinical prediction models (CPM), such as the SCOAP-CERTAIN tool, can be utilized to enhance decision-making for lumbar spinal fusion surgery by providing quantitative estimates of outcomes, aiding surgeons in assessing potential benefits and risks for each individual patient. External validation is crucial in CPM to assess generalizability beyond the initial dataset. This ensures performance in diverse populations, reliability and real-world applicability of the results. Therefore, we externally validated the tool for predictability of improvement in oswestry disability index (ODI), back and leg pain (BP, LP). METHODS: Prospective and retrospective data from multicenter registry was obtained. As outcome measure minimum clinically important change was chosen for ODI with ≥ 15-point and ≥ 2-point reduction for numeric rating scales (NRS) for BP and LP 12 months after lumbar fusion for degenerative disease. We externally validate this tool by calculating discrimination and calibration metrics such as intercept, slope, Brier Score, expected/observed ratio, Hosmer-Lemeshow (HL), AUC, sensitivity and specificity. RESULTS: We included 1115 patients, average age 60.8 ± 12.5 years. For 12-month ODI, area-under-the-curve (AUC) was 0.70, the calibration intercept and slope were 1.01 and 0.84, respectively. For NRS BP, AUC was 0.72, with calibration intercept of 0.97 and slope of 0.87. For NRS LP, AUC was 0.70, with calibration intercept of 0.04 and slope of 0.72. Sensitivity ranged from 0.63 to 0.96, while specificity ranged from 0.15 to 0.68. Lack of fit was found for all three models based on HL testing. CONCLUSIONS: Utilizing data from a multinational registry, we externally validate the SCOAP-CERTAIN prediction tool. The model demonstrated fair discrimination and calibration of predicted probabilities, necessitating caution in applying it in clinical practice. We suggest that future CPMs focus on predicting longer-term prognosis for this patient population, emphasizing the significance of robust calibration and thorough reporting.

Comparative Analysis of Machine Learning Models for Prediction of Acute Liver Injury in Sepsis Patients.

Introduction: Acute liver injury (ALI) is a common complication of sepsis and is associated with adverse clinical outcomes. We aimed to develop a model to predict the risk of ALI in patients with sepsis after hospitalization. Methods: Medical records of 3196 septic patients treated at the Lishui Central Hospital in Zhejiang Province from January 2015 to May 2023 were selected. Cohort 1 was divided into ALI and non-ALI groups for model training and internal validation. The initial laboratory test results of the study subjects were used as features for machine learning (ML), and models built using nine different ML algorithms were compared to select the best algorithm and model. The predictive performance of model stacking methods was then explored. The best model was externally validated in Cohort 2. Results: In Cohort 1, LightGBM demonstrated good stability and predictive performance with an area under the curve (AUC) of 0.841. The top five most important variables in the model were diabetes, congestive heart failure, prothrombin time, heart rate, and platelet count. The LightGBM model showed stable and good ALI risk prediction ability in the external validation of Cohort 2 with an AUC of 0.815. Furthermore, an online prediction website was developed to assist healthcare professionals in applying this model more effectively. Conclusions: The Light GBM model can predict the risk of ALI in patients with sepsis after hospitalization.

External validation of risk prediction model for gestational diabetes: Individual participant data meta-analysis of randomized trials.

BACKGROUND: An original validated risk prediction model with good discriminatory prognostic performance for predicting gestational diabetes (GDM) diagnosis, has been updated for recent international association of diabetes in pregnancy study group (IADPSG) diagnostic criteria. However, the updated model is yet to be externally validated on an international dataset. AIMS: To perform an external validation of the updated risk prediction model to evaluate model indices such as discrimination and calibration based on data from the International Weight Management in Pregnancy (i-WIP) Collaborative Group. MATERIALS AND METHODS: The i -WIP dataset was used to validate the GDM prediction tool across discrimination and model calibration. RESULTS: Overall 7689 individual patient data were included, with 17.4 % with GDM, however only 113 cases were available using IADPSG (International Association of Diabetes and Pregnancy Groups) criteria for 75 g OGTT glucose load and ACOG (American College of Obstetricians and Gynecologists) for 100 g glucose load and having the routine clinical risk factor data. The GDM model was moderately discriminatory (Area Under the Curve (AUC) of 0.67; 95 % CI 0.59 to 0.75), Sensitivity 81.0 % (95 % CI 66.7 % to 90.9 %), specificity 53 % (40.3 % to 65.4 %). The GDM score showed reasonable calibration for predicting GDM (slope = 0.84, CITL = 0.77). Imputation for missing data increased the sample to n = 253, and vastly improved the discrimination and calibration of the model to AUC = 78 (95 % CI 72 to 85), sensitivity (81 %, 95 % CI 66.7 % to 90.9 %) and specificity (75 %, 95 % CI 68.8 % to 81 %). CONCLUSION: The updated GDM model showed promising discrimination in predicting GDM in an international population sourced from RCT individual patient data. External validations are essential in order for the risk prediction area to advance, and we demonstrate the utility of using existing RCT data from different global settings. Despite limitations associated with harmonising the data to the variable types in the model, the validation model indices were reasonable, supporting generalizability across continents and populations.

A latent class analysis of cognitive-affective heterogeneity in current intermittent explosive disorder.

Intermittent Explosive Disorder (IED) is a common, chronic, and impairing psychological condition characterized by recurrent, affective aggressive behavior. IED is associated with a host of cognitive and affective symptoms not included in the diagnostic criteria which may be a valuable indicator of heterogeneity in IED-such information can be useful to enhance understanding and treatment of this disorder in mental health settings. A preliminary investigation conducted on cognitive-affective symptom heterogeneity in individuals with a history of IED demonstrated that level of emotional dysregulation primarily differentiated IED subgroups, however the sample size was limited, and almost half of the individuals did not have current IED (only lifetime IED). The present study addressed these limitations by conducting a latent class analysis of cognitive-affective symptoms among a large (n = 504) sample of individuals diagnosed with current IED. The latent IED classes were then externally validated on several adverse outcomes, historical precursors, and demographic variables. Statistical and clinical indicators supported a four-class model, with classes primarily distinguished by patients' severity of emotion dysregulation. The two moderate emotion-dysregulated classes both endorsed callous-unemotional traits and low empathy relative to other classes, a finding which differs from the initial investigation. An external validation of the four classes revealed that they significantly differed on severity outcomes (e.g., aggression, suicide attempts, antisocial behavior, global functioning, comorbidities) and historical precursors (e.g., aversive parental care, childhood maltreatment). These findings provide further insight into the heterogeneity within IED and the associations of such variability with important precursors and clinical outcomes.

Validating CircaCP: a generic sleep-wake cycle detection algorithm for unlabelled actigraphy data.

Sleep-wake (SW) cycle detection is a key step for extracting temporal sleep metrics from actigraphy. Various supervised learning algorithms have been developed, yet their generalizability from sensor to sensor or study to study is questionable. In this paper, we detail and validate an unsupervised algorithm-CircaCP-for detecting SW cycles from actigraphy. It first uses a robust cosinor model to estimate circadian rhythm, then searches for a single change point (CP) within each circadian cycle. Using CircaCP, we estimated sleep/wake onset times (S/WOTs) from 2125 individuals' data in the MESA sleep study and compared the estimated S/WOTs against self-reported S/WOT event markers, using Bland-Altman analysis as well as variance component analysis. On average, SOTs estimated by CircaCP were 3.6 min behind those reported by event markers, and WOTs by CircaCP were less than 1 min behind those reported by markers. These differences accounted for less than 0.2% variability in S/WOTs, considering other sources of between-subject variations. Rooted in first principles of human circadian rhythms, our algorithm transferred seamlessly from children's hip-worn ActiGraph data to ageing adults' wrist-worn Actiwatch data. The generalizability of our algorithm suggests that it can be widely applied to actigraphy collected by other sensors and studies.

Survival after resection of malignant peripheral nerve sheath tumors: Introducing and validating a novel type-specific prognostic model.

Background: This study aimed to assess the performance of currently available risk calculators in a cohort of patients with malignant peripheral nerve sheath tumors (MPNST) and to create an MPNST-specific prognostic model including type-specific predictors for overall survival (OS). Methods: This is a retrospective multicenter cohort study of patients with MPNST from 11 secondary or tertiary centers in The Netherlands, Italy and the United States of America. All patients diagnosed with primary MPNST who underwent macroscopically complete surgical resection from 2000 to 2019 were included in this study. A multivariable Cox proportional hazard model for OS was estimated with prespecified predictors (age, grade, size, NF-1 status, triton status, depth, tumor location, and surgical margin). Model performance was assessed for the Sarculator and PERSARC calculators by examining discrimination (C-index) and calibration (calibration plots and observed-expected statistic; O/E-statistic). Internal-external cross-validation by different regions was performed to evaluate the generalizability of the model. Results: A total of 507 patients with primary MPNSTs were included from 11 centers in 7 regions. During follow-up (median 8.7 years), 211 patients died. The C-index was 0.60 (95% CI 0.53-0.67) for both Sarculator and PERSARC. The MPNST-specific model had a pooled C-index of 0.69 (95%CI 0.65-0.73) at validation, with adequate discrimination and calibration across regions. Conclusions: The MPNST-specific MONACO model can be used to predict 3-, 5-, and 10-year OS in patients with primary MPNST who underwent macroscopically complete surgical resection. Further validation may refine the model to inform patients and physicians on prognosis and support them in shared decision-making.