Multicenter external validation of prediction models for clinical outcomes after spinal fusion for lumbar degenerative disease.

BACKGROUND: Clinical prediction models (CPM), such as the SCOAP-CERTAIN tool, can be utilized to enhance decision-making for lumbar spinal fusion surgery by providing quantitative estimates of outcomes, aiding surgeons in assessing potential benefits and risks for each individual patient. External validation is crucial in CPM to assess generalizability beyond the initial dataset. This ensures performance in diverse populations, reliability and real-world applicability of the results. Therefore, we externally validated the tool for predictability of improvement in oswestry disability index (ODI), back and leg pain (BP, LP). METHODS: Prospective and retrospective data from multicenter registry was obtained. As outcome measure minimum clinically important change was chosen for ODI with ≥ 15-point and ≥ 2-point reduction for numeric rating scales (NRS) for BP and LP 12 months after lumbar fusion for degenerative disease. We externally validate this tool by calculating discrimination and calibration metrics such as intercept, slope, Brier Score, expected/observed ratio, Hosmer-Lemeshow (HL), AUC, sensitivity and specificity. RESULTS: We included 1115 patients, average age 60.8 ± 12.5 years. For 12-month ODI, area-under-the-curve (AUC) was 0.70, the calibration intercept and slope were 1.01 and 0.84, respectively. For NRS BP, AUC was 0.72, with calibration intercept of 0.97 and slope of 0.87. For NRS LP, AUC was 0.70, with calibration intercept of 0.04 and slope of 0.72. Sensitivity ranged from 0.63 to 0.96, while specificity ranged from 0.15 to 0.68. Lack of fit was found for all three models based on HL testing. CONCLUSIONS: Utilizing data from a multinational registry, we externally validate the SCOAP-CERTAIN prediction tool. The model demonstrated fair discrimination and calibration of predicted probabilities, necessitating caution in applying it in clinical practice. We suggest that future CPMs focus on predicting longer-term prognosis for this patient population, emphasizing the significance of robust calibration and thorough reporting.

Survival after resection of malignant peripheral nerve sheath tumors: Introducing and validating a novel type-specific prognostic model.

Background: This study aimed to assess the performance of currently available risk calculators in a cohort of patients with malignant peripheral nerve sheath tumors (MPNST) and to create an MPNST-specific prognostic model including type-specific predictors for overall survival (OS). Methods: This is a retrospective multicenter cohort study of patients with MPNST from 11 secondary or tertiary centers in The Netherlands, Italy and the United States of America. All patients diagnosed with primary MPNST who underwent macroscopically complete surgical resection from 2000 to 2019 were included in this study. A multivariable Cox proportional hazard model for OS was estimated with prespecified predictors (age, grade, size, NF-1 status, triton status, depth, tumor location, and surgical margin). Model performance was assessed for the Sarculator and PERSARC calculators by examining discrimination (C-index) and calibration (calibration plots and observed-expected statistic; O/E-statistic). Internal-external cross-validation by different regions was performed to evaluate the generalizability of the model. Results: A total of 507 patients with primary MPNSTs were included from 11 centers in 7 regions. During follow-up (median 8.7 years), 211 patients died. The C-index was 0.60 (95% CI 0.53-0.67) for both Sarculator and PERSARC. The MPNST-specific model had a pooled C-index of 0.69 (95%CI 0.65-0.73) at validation, with adequate discrimination and calibration across regions. Conclusions: The MPNST-specific MONACO model can be used to predict 3-, 5-, and 10-year OS in patients with primary MPNST who underwent macroscopically complete surgical resection. Further validation may refine the model to inform patients and physicians on prognosis and support them in shared decision-making.

Limited Generalizability of Retrospective Single-Center Cohort Study in Comparison to Multicenter Cohort Study on Prognosis of Hepatocellular Carcinoma.

Introduction: We aimed to evaluate the generalizability of retrospective single-center cohort studies on prognosis of hepatocellular carcinoma (HCC) by comparing overall survival (OS) after various treatments between a nationwide multicenter cohort and a single-center cohort of HCC patients. Methods: Patients newly diagnosed with HCC between January 2008 and December 2018 were analyzed using data from the Korean Primary Liver Cancer Registry (multicenter cohort, n=16,443), and the Asan Medical Center HCC registry (single-center cohort, n=15,655). The primary outcome, OS after initial treatment, was compared between the two cohorts for both the entire population and for subcohorts with Child-Pugh A liver function (n=2797 and n=5151, respectively) treated according to the Barcelona-Clinic-Liver-Cancer (BCLC) strategy, using Log rank test and Cox proportional hazard models. Results: Patients of BCLC stages 0 and A (59.3% vs 35.2%) and patients who received curative treatment (42.1% vs 32.1%) were more frequently observed in the single-center cohort (Ps<0.001). Multivariable analysis revealed significant differences between the two cohorts in OS according to type of treatment: the multicenter cohort was associated with higher risk of mortality among patients who received curative (adjusted hazard ratio [95% confidence interval], 1.48 [1.39-1.59]) and non-curative (1.22 [1.17-1.27]) treatments, whereas the risk was lower in patients treated with systemic therapy (0.83 [0.74-0.92]) and best supportive care (0.85 [0.79-0.91]). Subcohort analysis also demonstrated significantly different OS between the two cohorts, with a higher risk of mortality in multicenter cohort patients who received chemoembolization (1.72 [1.48-2.00]) and ablation (1.44 [1.08-1.92]). Conclusion: Comparisons of single-center and multicenter cohorts of HCC patients revealed significant differences in OS according to treatment modality after adjustment for prognostic variables. Therefore, the results of retrospective single-center cohort studies of HCC treatments may not be generalizable to real-world practice.

Validation of the Health Assessment Tool (HAT) based on four aging cohorts from the Swedish National study on Aging and Care.

BACKGROUND: As global aging accelerates, routinely assessing the functional status and morbidity burden of older patients becomes paramount. The aim of this study is to assess the validity of the comprehensive clinical and functional Health Assessment Tool (HAT) based on four cohorts of older adults (60 + years) from the Swedish National study on Aging and Care (SNAC) spanning urban, suburban, and rural areas. METHODS: The HAT integrates five health indicators (gait speed, global cognition, number of chronic diseases, and basic and instrumental activities of daily living), providing an individual-level score between 0 and 10. The tool was constructed using nominal response models, first separately for each cohort and then in a harmonized dataset. Outcomes included all-cause mortality over a maximum follow-up of 16 years and unplanned hospital admissions over a maximum of 3 years of follow-up. The predictive capacity was assessed through the area under the curve (AUC) using logistic regressions. For time to death, Cox regressions were performed, and Harrell's C-indices were reported. Results from the four cohorts were pooled using individual participant data meta-analysis and compared with those from the harmonized dataset. RESULTS: The HAT demonstrated high predictive capacity across all cohorts as well as in the harmonized dataset. In the harmonized dataset, the AUC was 0.84 (95% CI 0.81-0.87) for 1-year mortality, 0.81 (95% CI 0.80-0.83) for 3-year mortality, 0.80 (95% CI 0.79-0.82) for 5-year mortality, 0.69 (95% CI 0.67-0.70) for 1-year unplanned admissions, and 0.69 (95% CI 0.68-0.70) for 3-year unplanned admissions. The Harrell's C for time-to-death throughout 16 years of follow-up was 0.75 (95% CI 0.74-0.75). CONCLUSIONS: The HAT is a highly predictive, clinically intuitive, and externally valid instrument with potential for better addressing older adults' health needs and optimizing risk stratification at the population level.

External validation and comparison of magnetic resonance imaging-based risk prediction models for prostate biopsy stratification.

PURPOSE: Magnetic resonance imaging (MRI) is a promising tool for risk assessment, potentially reducing the burden of unnecessary prostate biopsies. Risk prediction models that incorporate MRI data have gained attention, but their external validation and comparison are essential for guiding clinical practice. The aim is to externally validate and compare risk prediction models for the diagnosis of clinically significant prostate cancer (csPCa). METHODS: A cohort of 4606 patients across fifteen European tertiary referral centers were identified from a prospective maintained database between January 2016 and April 2023. Transrectal or transperineal image-fusion MRI-targeted and systematic biopsies for PI-RADS score of ≥ 3 or ≥ 2 depending on patient characteristics and physician preferences. Probabilities for csPCa, defined as International Society of Urological Pathology (ISUP) grade ≥ 2, were calculated for each patients using eight models. Performance was characterized by area under the receiver operating characteristic curve (AUC), calibration, and net benefit. Subgroup analyses were performed across various clinically relevant subgroups. RESULTS: Overall, csPCa was detected in 2154 (47%) patients. The models exhibited satisfactory performance, demonstrating good discrimination (AUC ranging from 0.75 to 0.78, p < 0.001), adequate calibration, and high net benefit. The model described by Alberts showed the highest clinical utility for threshold probabilities between 10 and 20%. Subgroup analyses highlighted variations in models' performance, particularly when stratified according to PSA level, biopsy technique and PI-RADS version. CONCLUSIONS: We report a comprehensive external validation of risk prediction models for csPCa diagnosis in patients who underwent MRI-targeted and systematic biopsies. The model by Alberts demonstrated superior clinical utility and should be favored when determining the need for a prostate biopsy.

Identifying radiogenomic associations of breast cancer based on DCE-MRI by using Siamese Neural Network with manufacturer bias normalization.

BACKGROUND AND PURPOSE: The immunohistochemical test (IHC) for Human Epidermal Growth Factor Receptor 2 (HER2) and hormone receptors (HR) provides prognostic information and guides treatment for patients with invasive breast cancer. The objective of this paper is to establish a non-invasive system for identifying HER2 and HR in breast cancer using dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI). METHODS: In light of the absence of high-performance algorithms and external validation in previously published methods, this study utilizes 3D deep features and radiomics features to represent the information of the Region of Interest (ROI). A Siamese Neural Network was employed as the classifier, with 3D deep features and radiomics features serving as the network input. To neutralize manufacturer bias, a batch effect normalization method, ComBat, was introduced. To enhance the reliability of the study, two datasets, Predict Your Therapeutic Response with Imaging and moLecular Analysis (I-SPY 1) and I-SPY 2, were incorporated. I-SPY 2 was utilized for model training and validation, while I-SPY 1 was exclusively employed for external validation. Additionally, a breast tumor segmentation network was trained to improve radiomic feature extraction. RESULTS: The results indicate that our approach achieved an average Area Under the Curve (AUC) of 0.632, with a Standard Error of the Mean (SEM) of 0.042 for HER2 prediction in the I-SPY 2 dataset. For HR prediction, our method attained an AUC of 0.635 (SEM 0.041), surpassing other published methods in the AUC metric. Moreover, the proposed method yielded competitive results in other metrics. In external validation using the I-SPY 1 dataset, our approach achieved an AUC of 0.567 (SEM 0.032) for HR prediction and 0.563 (SEM 0.033) for HER2 prediction. CONCLUSION: This study proposes a non-invasive system for identifying HER2 and HR in breast cancer. Although the results do not conclusively demonstrate superiority in both tasks, they indicate that the proposed method achieved good performance and is a competitive classifier compared to other reference methods. Ablation studies demonstrate that both radiomics features and deep features for the Siamese Neural Network are beneficial for the model. The introduced manufacturer bias normalization method has been shown to enhance the method's performance. Furthermore, the external validation of the method enhances the reliability of this research. Source code, pre-trained segmentation network, Radiomics and deep features, data for statistical analysis, and Supporting Information of this article are online at: https://github.com/FORRESTHUACHEN/Siamese_Neural_Network_based_Brest_cancer_Radiogenomic.

Internal and External Validation of Machine Learning Models for Predicting Acute Kidney Injury Following Non-Cardiac Surgery Using Open Datasets.

This study developed and validated a machine learning model to accurately predict acute kidney injury (AKI) after non-cardiac surgery, aiming to improve patient outcomes by assessing its clinical feasibility and generalizability. We conducted a retrospective cohort study using data from 76,032 adults who underwent non-cardiac surgery at a single tertiary medical center between March 2019 and February 2021, and used data from 5512 patients from the VitalDB open dataset for external model validation. The predictive variables for model training consisted of demographic, preoperative laboratory, and intraoperative data, including calculated statistical values such as the minimum, maximum, and mean intraoperative blood pressure. When predicting postoperative AKI, our gradient boosting machine model incorporating all the variables achieved the best results, with AUROC values of 0.868 and 0.757 for the internal and external validations using the VitalDB dataset, respectively. The model using intraoperative data performed best in internal validation, while the model with preoperative data excelled in external validation. In this study, we developed a predictive model for postoperative AKI in adult patients undergoing non-cardiac surgery using preoperative and intraoperative data, and external validation demonstrated the efficacy of open datasets for generalization in medical artificial modeling research.

Large-scale external validation and meta-analysis of gene methylation biomarkers in tumor tissue for colorectal cancer prognosis.

BACKGROUND: DNA methylation biomarkers in colorectal cancer (CRC) tissue hold potential as prognostic indicators. However, individual studies have yielded heterogeneous results, and external validation is largely absent. We conducted a comprehensive external validation and meta-analysis of previously suggested gene methylation biomarkers for CRC prognosis. METHODS: We performed a systematic search to identify relevant studies investigating gene methylation biomarkers for CRC prognosis until March 2024. Our external validation cohort with long-term follow-up included 2303 patients with CRC from 22 hospitals in southwest Germany. We used Cox regression analyses to assess associations between previously suggested gene methylation biomarkers and prognosis, adjusting for clinical variables. We calculated pooled hazard ratios (HRs) and their 95% confidence intervals (CIs) using random-effects models. FINDINGS: Of 151 single gene and 29 multiple gene methylation biomarkers identified from 121 studies, 37 single gene and seven multiple gene biomarkers were significantly associated with CRC prognosis after adjustment for clinical variables. Moreover, the directions of these associations with prognosis remained consistent between the original studies and our validation analyses. Seven single biomarkers and two multi-biomarker signatures were significantly associated with CRC prognosis in the meta-analysis, with a relatively strong level of evidence for CDKN2A, WNT5A, MLH1, and EVL. INTERPRETATION: In a comprehensive evaluation of the so far identified gene methylation biomarkers for CRC prognosis, we identified candidates with potential clinical relevance for further investigation. FUNDING: The German Research Council, the Interdisciplinary Research Program of the National Center for Tumor Diseases (NCT), Germany, the German Federal Ministry of Education and Research.

External validation of the 2-year mortality prediction tool in hemodialysis patients developed using a Bayesian network.

Background: In recent years, a number of predictive models have appeared to predict the risk of medium-term mortality in hemodialysis patients, but only one, limited to patients aged over 70 years, has undergone sufficiently powerful external validation. Recently, using a national learning database and an innovative approach based on Bayesian networks and 14 carefully selected predictors, we have developed a clinical prediction tool to predict all-cause mortality at 2 years in all incident hemodialysis patients. In order to generalize the results of this tool and propose its use in routine clinical practice, we carried out an external validation using an independent external validation database. Methods: A regional, multicenter, observational, retrospective cohort study was conducted to externally validate the tool for predicting 2-year all-cause mortality in incident and prevalent hemodialysis patients. This study recruited a total of 142 incident and 697 prevalent adult hemodialysis patients followed up in one of the eight Association pour l'Utilisation du Rein Artificiel dans la région Lyonnaise (AURAL) Alsace dialysis centers. Results: In incident patients, the 2-year all-cause mortality prediction tool had an area under the receiver curve (AUC-ROC) of 0.73, an accuracy of 65%, a sensitivity of 71% and a specificity of 63%. In prevalent patients, the performance for the external validation were similar in terms of AUC-ROC, accuracy and specificity, but was lower in term of sensitivity. Conclusion: The tool for predicting all-cause mortality at 2 years, developed using a Bayesian network and 14 routinely available explanatory variables, obtained satisfactory external validation in incident patients, but sensitivity was insufficient in prevalent patients.

Don't be misled: 3 misconceptions about external validation of clinical prediction models.

Clinical prediction models provide risks of health outcomes that can inform patients and support medical decisions. However, most models never make it to actual implementation in practice. A commonly heard reason for this lack of implementation is that prediction models are often not externally validated. While we generally encourage external validation, we argue that an external validation is often neither sufficient nor required as an essential step before implementation. As such, any available external validation should not be perceived as a license for model implementation. We clarify this argument by discussing 3 common misconceptions about external validation. We argue that there is not one type of recommended validation design, not always a necessity for external validation, and sometimes a need for multiple external validations. The insights from this paper can help readers to consider, design, interpret, and appreciate external validation studies.

External validation of the EFFECT mortality prediction scale in patients admitted for acute heart failure in Araba, Spain.

AIM: To validate the EFFECT (Enhanced Feedback for Effective Cardiac Treatment) scales, which predict mortality at 1 month and 1 year after admission, in a defined cohort of patients admitted to the Araba University Hospital (HUA) with a diagnosis of acutely decompensated heart failure. METHOD: External validation study of a predictive model, in a retrospective cohort of patients admitted between October 1, 2020 and September 30, 2021. RESULTS: A total of 550 patients were included. The two scales demonstrated good overall discriminatory ability in our series, with an area under ROC (0.755 y 0.756) and values in Brier score (0.094 y 0.194) similar to the original series. Calibration was assessed using the Hosmer-Lemeshow test and calibration plots and was also adequate. All this despite the fact that significant differences were observed in many clinical characteristics between our series and the original one. CONCLUSIONS: The EFFECT scales showed good predictive ability and transportability. The one-month prediction scale was also useful for predicting mortality at one year. For both time periods, mortality was similar in the groups established in the original as low and very low risk.

External validation of the computer aided risk scoring system in predicting in-hospital mortality following emergency medical admissions.

BACKGROUND: Clinical prediction models have the potential to improve the quality of care and enhance patient safety outcomes. A Computer-aided Risk Scoring system (CARSS) was previously developed to predict in-hospital mortality following emergency admissions based on routinely collected blood tests and vitals. We aimed to externally validate the CARSS model. METHODS: In this retrospective external validation study, we considered all adult (≥18 years) emergency medical admissions discharged between 11/11/2020 and 11/11/2022 from The Rotherham Foundation Trust (TRFT), UK. We assessed the predictive performance of the CARSS model based on its discriminative (c-statistic) and calibration characteristics (calibration slope and calibration plots). RESULTS: Out of 32,774 admissions, 20,422 (62.3 %) admissions were included. The TRFT sample had similar demographic characteristics to the development sample but had higher mortality (6.1 % versus 5.7 %). The CARSS model demonstrated good discrimination (c-statistic 0.87 [95 % CI 0.86-0.88]) and good calibration to the TRFT dataset (slope = 1.03 [95 % CI 0.98-1.08] intercept = 0 [95 % CI -0.06-0.07]) after re-calibrating for differences in baseline mortality (intercept = 0.96 [95 % CI 0.90-1.03] before re-calibration). CONCLUSION: In summary, the CARSS model is externally validated after correcting the baseline risk of death between development and validation datasets. External validation of the CARSS model showed that it under-predicted in-hospital mortality. Re-calibration of this model showed adequate performance in the TRFT dataset.

External validation of the ARCH score in patients undergoing aortic arch reconstruction under circulatory arrest.

BACKGROUND: Aortic arch surgery with hypothermic circulatory arrest (HCA) carries a higher risk of morbidity and mortality compared to routine cardiac surgical procedures. The newly developed ARCH (arch reconstruction under circulatory arrest with hypothermia) score has not been externally validated. We sought to externally validate this score in our local population. METHODS: All consecutive open aortic arch surgeries with HCA performed between 2014 and 2023 were included. Univariable and multivariable analyses were performed. Model discrimination was assessed by the C-statistic with 95% confidence intervals as part of the receiver operating characteristic (ROC) curve analysis. Model performance was visualized by a calibration plot and quantified by the Brier score. RESULTS: A total of 760 patients (38.3% females) were included. The mean age was 61 (±13.6) years, with 56.4% of patients' age >60 years. The procedures were carried out mostly emergently or urgently (59.6%). Total arch replacement was performed in 32.5% of the patients, and aortic root procedures were carried out in 74.6%. In-hospital death occurred in 64 patients (8.4%), and stroke occurred in 5.4%. The C-statistic revealed a low discriminatory ability for predicting in-hospital mortality (area under the ROC curve, 0.62; 95% confidence interval, 0.54-0.69; P = .002); however, model calibration was found to be excellent (Brier score of 0.07). CONCLUSIONS: The ARCH score for in-hospital mortality showed low discriminatory ability in our local population, although with excellent ability for prediction of mortality.

Establishing a Validation Infrastructure for Imaging-Based Artificial Intelligence Algorithms Before Clinical Implementation.

With promising artificial intelligence (AI) algorithms receiving FDA clearance, the potential impact of these models on clinical outcomes must be evaluated locally before their integration into routine workflows. Robust validation infrastructures are pivotal to inspecting the accuracy and generalizability of these deep learning algorithms to ensure both patient safety and health equity. Protected health information concerns, intellectual property rights, and diverse requirements of models impede the development of rigorous external validation infrastructures. The authors propose various suggestions for addressing the challenges associated with the development of efficient, customizable, and cost-effective infrastructures for the external validation of AI models at large medical centers and institutions. The authors present comprehensive steps to establish an AI inferencing infrastructure outside clinical systems to examine the local performance of AI algorithms before health practice or systemwide implementation and promote an evidence-based approach for adopting AI models that can enhance radiology workflows and improve patient outcomes.

External validation of population pharmacokinetic models of tacrolimus in Thai adult liver transplant recipients.

OBJECTIVE: Several population pharmacokinetic models of tacrolimus in liver transplant patients were built, and their predictability was evaluated in their settings. However, the extrapolation in the prediction was unclear. This study aimed to evaluate the predictive performance of published tacrolimus models in adult liver transplant recipients using data from the Thai population as an external dataset. METHODS: The selected published models were systematically searched and evaluated for their quality. The external dataset of patients who underwent the first liver transplant and received immediate-release tacrolimus was used to assess the predictive performance of each selected model. Trough concentrations between 3 and 6 months were retrospectively collected to evaluate the predictability of each model using prediction-based diagnostics, simulation-based diagnostics, and Bayesian forecasting. RESULTS: Sixty-seven patients with 360 trough concentrations and eight selected published models were included in this study. None of the models met the predictive precision criteria in prediction-based diagnostics. Meanwhile, four published population pharmacokinetic models showed a normal distribution in NPDE testing. Regarding Bayesian forecasting, all models improved their forecasts with at least one prior information data point. CONCLUSION: Bayesian forecasting is more accurate and precise than other testing methods for predicting drug concentrations. However, none of the evaluated models provides satisfactory predictive performance for generalization to Thai liver transplant patients. This underscores the need for future research to develop population PK models tailored to the Thai population. Such efforts should consider the inclusion of nonlinear pharmacokinetics and region-specific factors, including genetic variability, to improve model accuracy and applicability.

External Validation of the Kidney Failure Risk Equation Among Urban Community-Based Chinese Patients With CKD.

Rationale & Objective: The Kidney Failure Risk Equations have been proven to perform well in multinational databases, whereas validation in Asian populations is lacking. This study sought to externally validate the equations in a community-based chronic kidney disease cohort in China. Study Design: A retrospective cohort study. Setting & Participants: Patients with and estimated glomerular filtration rate (eGFR) < 60 mL/min/1.73 m2 dwelling in an industrialized coastal city of China. Exposure: Age, sex, eGFR, and albuminuria were included in the 4-variable model, whereas serum calcium, phosphate, bicarbonate, and albumin levels were added to the previously noted variables in the 8-variable model. Outcome: Initiation of long-term dialysis treatment. Analytical Approach: Model discrimination, calibration, and clinical utility were evaluated by Harrell's C statistic, calibration plots, and decision curve analysis, respectively. Results: A total of 4,587 participants were enrolled for validation of the 4-variable model, whereas 1,414 were enrolled for the 8-variable model. The median times of follow-up were 4.0 (interquartile range: 2.6-6.3) years for the 4-variable model and 3.4 (2.2-5.6) years for the 8-variable model. For the 4-variable model, the C statistics were 0.750 (95% CI: 0.615-0.885) for the 2-year model and 0.766 (0.625-0.907) for the 5-year model, whereas the values were 0.756 (0.629-0.883) and 0.774 (0.641-0.907), respectively, for the 8-variable model. Calibration was acceptable for both the 4-variable and 8-variable models. Decision curve analysis for the models at the 5-year scale performed better throughout different net benefit thresholds than the eGFR-based (<30 mL/min/1.73 m2) strategy. Limitations: A large proportion of patients lack albuminuria measurements, and only a subset of population could provide complete data for the 8-variable equation. Conclusions: The kidney failure risk equations showed acceptable discrimination and calibration and better clinical utility than the eGFR-based strategy for incidence of kidney failure among community-based urban Chinese patients with chronic kidney disease.

Accurate and reliable risk evaluation of chronic kidney disease (CKD) prognosis can be helpful for physicians to make decisions concerning treatment opportunity and therapeutic strategy. The kidney failure risk equation is an outstanding model for predicting risk of kidney failure among patients with CKD. However, the equation is lacking validation among Chinese populations. In the current study, we demonstrated that the equation had good discrimination among an urban community-based cohort of patients with CKD in China. The calibration was also acceptable. Decision curve analysis also showed that the equation performed better than a traditional kidney function-based strategy. The results provide the basis for using predictions derived from the kidney failure risk equation to improve the management of patients with CKD in community settings in China.

Nomogram model for predicting cancer-specific mortality in patients with early-onset colorectal cancer: a competing risk analysis insight from the SEER database and an external validation cohort.

Background: Early-onset colorectal cancer (EOCRC) is increasing in incidence and poses a growing threat. Urgent research is needed, especially in survival analysis, to enhance comprehension and treatment strategies. This study aimed to explore the risk factors associated with cancer-specific mortality (CSM) and other-cause mortality (OCM) in patients with EOCRC. Additionally, the study aimed to develop a nomogram predicting CSM using a competitive risk model and validate its accuracy through the use of training, using internal and external cohorts. Methods: Data from EOCRC patients were collected from the Surveillance, Epidemiology, and End Results (SEER) database (2008-2017). EOCRC patients who were treated at a tertiary hospital in northeast China between 2014 and 2020 were also included in the study. The SEER data were divided into the training and validation sets at a 7:3 ratio. A univariate Cox regression model was employed to identify prognostic factors. Subsequently, multivariate Cox regression models were applied to ascertain the presence of independent risk factors. A nomogram was generated to visualize the results, which were evaluated using the concordance index (C-index), area under the curve (AUC), and calibration curves. The clinical utility was assessed via decision curve analysis (DCA). Results: Multivariable Cox regression analysis demonstrated that factors such as race, tumor differentiation, levels of carcinoembryonic antigen (CEA), marital status, histological type, American Joint Committee on Cancer (AJCC) stage, and surgical status were independent risk factors for CSM in EOCRC patients. In addition, age, gender, chemotherapy details, CEA levels, marital status, and AJCC stage were established as independent risk factors for OCM in individuals diagnosed with EOCRC. A nomogram was developed using the identified independent risk factors, demonstrating excellent performance with a C-index of 0.806, 0.801, and 0.810 for the training, internal validation, and external validation cohorts, respectively. The calibration curves and AUC further confirmed the accuracy and discriminative ability of the nomogram. Furthermore, the DCA results indicated that the model had good clinical value. Conclusions: In this study, a competing risk model for CSM was developed in EOCRC patients. The model demonstrates a high level of predictive accuracy, providing valuable insights into the treatment decision-making process.

External Validation of a Machine Learning Model for Schizophrenia Classification.

Background and Objective: Excellent generalizability is the precondition for the widespread practical implementation of machine learning models. In our previous study, we developed the schizophrenia classification model (SZ classifier) to identify potential schizophrenia patients in the Japanese population. The SZ classifier has exhibited impressive performance during internal validation. However, ensuring the robustness and generalizability of the SZ classifier requires external validation across independent sample sets. In this study, we aimed to present an external validation of the SZ classifier using outpatient data. Methods: The SZ classifier was trained by using online survey data, which incorporate demographic, health-related, and social comorbidity features. External validation was conducted using an outpatient sample set which is independent from the sample set during the model development phase. The model performance was assessed based on the sensitivity and misclassification rates for schizophrenia, bipolar disorder, and major depression patients. Results: The SZ classifier demonstrated a sensitivity of 0.75 when applied to schizophrenia patients. The misclassification rates were 59% and 55% for bipolar disorder and major depression patients, respectively. Conclusions: The SZ classifier currently encounters challenges in accurately determining the presence or absence of schizophrenia at the individual level. Prior to widespread practical implementation, enhancements are necessary to bolster the accuracy and diminish the misclassification rates. Despite the current limitations of the model, such as poor specificity for certain psychiatric disorders, there is potential for improvement if including multiple types of psychiatric disorders during model development.

External validation and updating of the Rossi nomogram for predicting cesarean delivery following induction: is the Bishop score valuable?

OBJECTIVE: This study sought to validate the Rossi nomogram in a Chinese population and then to include the Bishop score to see if it has an effect on the accuracy of the nomogram. MATERIALS AND METHODS: The Rossi predictive model was applied and externally validated in a retrospective cohort from August 2017 and July 2023 in a Chinese tertiary-level medical center. For the revision and updating of the models, the regression coefficients of all the predictors (except race) were re-estimated and then the cervical Bishop score at the time of induction was added. Each model's performance was measured using the receiver-operating characteristic and calibration plots. Decision curve analysis determined the range of the probability threshold for each prediction model that would be of clinical value. RESULTS: A total of 721 women met the inclusion criteria, of whom 183 (25.4%) underwent a cesarean delivery. The calibration demonstrated the underestimation of the original model, with an area under the curve (AUC) of 0.789 (95% confidence interval [CI] 0.753-0.825, p < 0.001). After recalibrating the original model, the discriminative performance was improved from 0.789 to 0.803. Moreover, the discriminatory power of the updated model was further improved when the Bishop score at the time of induction was added to the recalibrated multivariable model. Indeed, the updated model demonstrated good calibration and discriminatory power, with an AUC of 0.811. The decision curve analysis indicated that all the models (original, recalibrated, and updated) provided higher net benefits of between 0 and 60% of the probability threshold, which indicates the benefits of using the models to make decisions concerning patients who fall within the identified range of the probability threshold. The net benefits of the updated model were higher than those of the original model and the recalibrated model. CONCLUSION: The nomogram used to predict cesarean delivery following induction developed by Rossi et al. has been validated in a Chinese population in this study. More specifically, adaptation to a Chinese population by excluding ethnicity and including the Bishop score prior to induction gave rise to better performance. The three models (original, recalibrated, and updated) offer higher net benefits when the probability threshold is between 0 and 60%.

External validation of predictive models for new vertebral fractures following percutaneous vertebroplasty.

OBJECTIVE: To investigate the external validation and scalability of four predictive models regarding new vertebral fractures following percutaneous vertebroplasty. METHODS: Utilizing retrospective data acquired from two centers, compute the area under the curve (AUC), calibration curve, and Kaplan-Meier plot to assess the model's discrimination and calibration. RESULTS: In the external validation of Zhong et al.'s 2015 predictive model for the probability of new fractures post-vertebroplasty, the AUC for re-fracture at 1, 2, and 3 years postoperatively was 0.570, 0.617, and 0.664, respectively. The AUC for Zhong et al.'s 2016 predictive model for the probability of new fractures in neighboring vertebrae was 0.738. Kaplan-Meier plot results for both models indicated a significantly lower incidence of re-fracture in low-risk patients compared to high-risk patients. Li et al.'s 2021 model had an AUC of 0.518, and its calibration curve suggested an overestimation of the probability of new fractures. Li et al.'s 2022 model had an AUC of 0.556, and its calibration curve suggested an underestimation of the probability of new fractures. CONCLUSION: The external validation of four models demonstrated that the predictive model proposed by Zhong et al. in 2016 exhibited superior external generalization capabilities.