Development and validation of a prediction model for heart failure in patients with heart valvular regurgitation.

AIMS: Patients with heart valvular regurgitation is increasing; early screening of potential patients developing heart failure (HF) is crucial. METHODS: From 1 November 2019 to 31 October 2023, a total of 509 patients with heart valvular regurgitation hospitalized in the Department of Cardiovascular Disease of the First Affiliated Hospital of Guangzhou University of Traditional Medicine were enrolled. Three hundred fifty-six cases were selected as the training set for modelling, and 153 cases were selected as the validation set for the internal validation of the model. RESULTS: A predictive model of heart failure with the following nine risk factors was developed: atrial fibrillation (AF), pulmonary infection (PI), coronary artery disease (CAD), creatinine (CREA), low-density lipoprotein cholesterol (LDL-C), d-dimer (DDi), left ventricular end-diastolic diameter (LVEDd), mitral regurgitation (MR) and aortic regurgitation (AR). The model was evaluated by the C-index [the training set: area under curve (AUC) 0.937, 95% confidence interval (CI) 0.911-0.963; the validation set: AUC 0.928, 95% CI 0.890-0.967]. Hosmer-Lemeshow test (the training set: χ2 10.908, P = 0.207; the validation set: χ2 4.896, P = 0.769) revealed that both the training and validation sets performed well in terms of model differentiation and calibration. Decision curve analysis showed that both the training and validation sets have higher net benefits, indicating that the model has good utility. Ten-fold cross-validation showed that the training set has high similarities with the validation set, which means that the model has good stability. CONCLUSIONS: The occurrence of heart failure in patients with valvular regurgitation has a significant correlation with AF, PI, CAD, CREA, LDL-C, DDi, LVEDd, MR and AR. Based on these risk factors, a prediction model for heart failure was developed and validated, which showed good differentiation and utility, high accuracy and stability, providing a method for predicting heart failure.

Don't be misled: 3 misconceptions about external validation of clinical prediction models.

Clinical prediction models provide risks of health outcomes that can inform patients and support medical decisions. However, most models never make it to actual implementation in practice. A commonly heard reason for this lack of implementation is that prediction models are often not externally validated. While we generally encourage external validation, we argue that an external validation is often neither sufficient nor required as an essential step before implementation. As such, any available external validation should not be perceived as a license for model implementation. We clarify this argument by discussing 3 common misconceptions about external validation. We argue that there is not one type of recommended validation design, not always a necessity for external validation, and sometimes a need for multiple external validations. The insights from this paper can help readers to consider, design, interpret, and appreciate external validation studies.

COVID-19: The Development and Validation of a New Mortality Risk Score.

Background: The coronavirus disease 2019 (COVID-19) pandemic has found the whole world unprepared for its correct management. Italy was the first European country to experience the spread of the SARS-CoV-2 virus at the end of February 2020. As a result of hospital overcrowding, the quality of care delivered was not always optimal. A substantial number of patients admitted to non-ICU units could have been treated at home. It would have been extremely useful to have a score that, based on personal and clinical characteristics and simple blood tests, could have predicted with sufficient reliability the probability that a patient had or did not have a disease that could have led to their death. This study aims to develop a scoring system to identify which patients with COVID-19 are at high mortality risk upon hospital admission, to expedite and enhance clinical decision making. Methods: A retrospective analysis was performed to develop a multivariable prognostic prediction model. Results: Derivation and external validation cohorts were obtained from two Italian University Hospital databases, including 388 (10.31% deceased) and 1357 (7.68% deceased) patients with confirmed COVID-19, respectively. A multivariable logistic model was used to select seven variables associated with in-hospital death (age, baseline oxygen saturation, hemoglobin value, white blood cell count, percentage of neutrophils, platelet count, and creatinine value). Calibration and discrimination were satisfactory with a cumulative AUC for prediction mortality of 0.924 (95% CI: 0.893-0.944) in derivation cohorts and 0.808 (95% CI: 0.886-0.828) in external validation cohorts. The risk score obtained was compared with the ISARIC 4C Mortality Score, and with all the other most important scores considered so far, to evaluate the risk of death of patients with COVID-19. It performed better than all the above scores to evaluate the predictability of dying. Its sensitivity, specificity, and AUC were higher than the other COVID-19 scoring systems when the latter were calculated for the 388 patients in our derivation cohort. Conclusions: In conclusion, the CZ-COVID-19 Score may help all physicians by identifying those COVID-19 patients who require more attention to provide better therapeutic regimens or, on the contrary, by identifying those patients for whom hospitalization is not necessary and who could therefore be sent home without overcrowding healthcare facilities. We developed and validated a new risk score based on seven variables for upon-hospital admission of COVID-19 patients. It is very simple to calculate and performs better than all the other similar scores to evaluate the predictability of dying.

The development and internal validation of a multivariable model predicting 6-month mortality for people with opioid use disorder presenting to community drug services in England: a protocol.

BACKGROUND: People with opioid use disorder have substantially higher standardised mortality rates compared to the general population; however, lack of clear individual prognostic information presents challenges to prioritise or target interventions within drug treatment services. Previous prognostic models have been developed to estimate the risk of developing opioid use disorder and opioid-related overdose in people routinely prescribed opioids but, to our knowledge, none have been developed to estimate mortality risk in people accessing drug services with opioid use disorder. Initial presentation to drug services is a pragmatic time to evaluate mortality risk given the contemporaneous routine collection of prognostic indicators and as a decision point for appropriate service prioritisation and targeted intervention delivery. This study aims to develop and internally validate a model to estimate 6-month mortality risk for people with opioid use disorder from prognostic indicators recorded at initial assessment in drug services in England. METHODS: An English national dataset containing records from individuals presenting to drug services between 1 April 2013 and 1 April 2023 (n > 800,000) (the National Drug Treatment Monitoring System (NDTMS)) linked to their lifetime hospitalisation and death records (Hospital Episode Statistics-Office of National Statistics (HES-ONS)). Twelve candidate prognostic indicator variables were identified based on literature review of demographic and clinical features associated with increased mortality for people in treatment for opioid use disorder. Variables will be extracted at initial presentation to drug services with mortality measured at 6 months. Two multivariable Cox regression models will be developed one for 6-month all-cause mortality and one for 6-month drug-related mortality using backward elimination with a fractional polynomial approach for continuous variables. Internal validation will be undertaken using bootstrapping methods. Discrimination of both models will be reported using Harrel's c and d-statistics. Calibration curves and slopes will be presented comparing expected and observed event rates. DISCUSSION: The models developed and internally validated in this study aim to improve clinical assessment of mortality risk for people with opioid use disorder presenting to drug services in England. External validation in different populations will be required to develop the model into a tool to assist future clinical decision-making.

Surveillance of catheter-associated bloodstream infections: development and validation of a fully automated algorithm.

BACKGROUND: Most surveillance systems for catheter-related bloodstream infections (CRBSI) and central line-associated bloodstream infections (CLABSI) are based on manual chart review. Our objective was to validate a fully automated algorithm for CRBSI and CLABSI surveillance in intensive care units (ICU). METHODS: We developed a fully automated algorithm to detect CRBSI, CLABSI and ICU-onset bloodstream infections (ICU-BSI) in patients admitted to the ICU of a tertiary care hospital in Switzerland. The parameters included in the algorithm were based on a recently performed systematic review. Structured data on demographics, administrative data, central vascular catheter and microbiological results (blood cultures and other clinical cultures) obtained from the hospital's data warehouse were processed by the algorithm. Validation for CRBSI was performed by comparing results with prospective manual BSI surveillance data over a 6-year period. CLABSI were retrospectively assessed over a 2-year period. RESULTS: From January 2016 to December 2021, 854 positive blood cultures were identified in 346 ICU patients. The median age was 61.7 years [IQR 50-70]; 205 (24%) positive samples were collected from female patients. The algorithm detected 5 CRBSI, 109 CLABSI and 280 ICU-BSI. The overall CRBSI and CLABSI incidence rates determined by automated surveillance for the period 2016 to 2021 were 0.18/1000 catheter-days (95% CI 0.06-0.41) and 3.86/1000 catheter days (95% CI: 3.17-4.65). The sensitivity, specificity, positive predictive and negative predictive values of the algorithm for CRBSI, were 83% (95% CI 43.7-96.9), 100% (95% CI 99.5-100), 100% (95% CI 56.5-100), and 99.9% (95% CI 99.2-100), respectively. One CRBSI was misclassified as an ICU-BSI by the algorithm because the same bacterium was identified in the blood culture and in a lower respiratory tract specimen. Manual review of CLABSI from January 2020 to December 2021 (n = 51) did not identify any errors in the algorithm. CONCLUSIONS: A fully automated algorithm for CRBSI and CLABSI detection in critically-ill patients using only structured data provided valid results. The next step will be to assess the feasibility and external validity of implementing it in several hospitals with different electronic health record systems.

Development of a rapid LC-MS/MS method for simultaneous quantification of ten commonly used antibiotic drugs in human serum.

The use of TDM in clinical practice to monitor the plasma levels of antibiotics administered to critically ill patients is a well-established approach that allows for optimization of the patient's response to drug therapy, considering the characteristics of the drug, the clinical and physiological status of the patient and any peculiar of the pathogen that caused the clinical picture. In our laboratory, we have developed a single LC-MS/MS analysis for dosing the serum concentration of an antibacterial panel composed of eight antibacterial and two selective inhibitors. The method presented used a certified material furnished by a commercial company and was internally validated using the EMA guidelines. The results have shown high sensitivity, precision, and accuracy, a lower matrix effect combined with simple sample preparation and a time-saving procedure. We have evaluated the recovery rate and matrix effect by testing serum samples without pathological index and serum pools obtained from haemolysed, icteric, or lipemic samples. The assay has shown a recovery range between 94% and 101%.

A Prediction Model of Preeclampsia in Hyperglycemia Pregnancy.

Purpose: To investigate the risk factors associated with preeclampsia in hyperglycemic pregnancies and develop a predictive model based on routine pregnancy care. Patients and Methods: The retrospective collection of clinical data was performed on 951 pregnant women with hyperglycemia, including those diagnosed with diabetes in pregnancy (DIP) and gestational diabetes mellitus (GDM), who delivered after 34 weeks of gestation at the Maternal and Child Health Hospital Affiliated to Anhui Medical University between January 2017 and December 2019. Observation indicators included liver and kidney function factors testing at 24-29+6 weeks gestation, maternal age, and basal blood pressure. The indicators were screened univariately, and the "rms" package in R language was applied to explore the factors associated with PE in HIP pregnancy by stepwise regression. Multivariable logistic regression analysis was used to develop the prediction model. Based on the above results, a nomogram was constructed to predict the risk of PE occurrence in pregnant women with HIP. Then, the model was evaluated from three aspects: discrimination, calibration, and clinical utility. The internal validation was performed using the bootstrap procedure. Results: Multivariate logistic regression analysis showed that cystatin C, uric acid, glutamyl aminotransferase, blood urea nitrogen, and basal systolic blood pressure as predictors of PE in pregnancy with HIP. The predictive model yielded an area under curve (AUC) value of 0.8031 (95% CI: 0.7383-0.8679), with an optimal threshold of 0.0805, at which point the sensitivity was 0.8307 and specificity of 0.6604. Hosmer-Lemeshow test values were P = 0.3736, Brier score value was 0.0461. After 1000 Bootstrap re-samplings for internal validation, the AUC was 0.7886, the Brier score was 0.0478 and the predicted probability of the calibration curve was similar to the actual probability. A nomogram was constructed based on the above to visualize the model. Conclusion: This study developed a model for predicting PE in pregnant women with HIP, achieving high predictive performance of PE risk through the information of routine pregnancy care.

Performance of the streamlined quality outcomes database web-based calculator: internal and external validation.

BACKGROUND CONTEXT: With an increasing number of web-based calculators designed to provide the probabilities of an individual achieving improvement after lumbar spine surgery, there is a need to determine the accuracy of these models. PURPOSE: To perform an internal and external validation study of the reduced Quality Outcomes Database web-based Calculator (QOD-Calc). STUDY DESIGN: Observational longitudinal cohort. PATIENT SAMPLE: Patients enrolled study-wide in Quality Outcomes Database (QOD) and patients enrolled in DaneSpine at a single institution who had elective lumbar spine surgery with baseline data to complete QOD-Calc and 12-month postoperative data. OUTCOME MEASURES: Oswestry Disability Index (ODI), Numeric Rating Scales (NRS) for back and leg pain, EuroQOL-5D (EQ-5D). METHODS: Baseline data elements were entered into QOD-Calc to determine the probability for each patient having Any Improvement and 30% Improvement in NRS leg pain, back pain, EQ-5D and ODI. These probabilities were compared with the actual 12-month postop data for each of the QOD and DaneSpine cases. Receiver-operating characteristics analyses were performed and calibration plots created to assess model performance. RESULTS: 24,755 QOD cases and 8,105 DaneSpine lumbar cases were included in the analysis. QOD-Calc had acceptable to outstanding ability (AUC: 0.694-0.874) to predict Any Improvement in the QOD cohort and moderate to acceptable ability (AUC: 0.658-0.747) to predict 30% Improvement. QOD-Calc had acceptable to exceptional ability (AUC: 0.669-0.734) to predict Any improvement and moderate to exceptional ability (AUC: 0.619-0.862) to predict 30% Improvement in the DaneSpine cohort. AUCs for the DaneSpine cohort was consistently lower that the AUCs for the QOD validation cohort. CONCLUSION: QOD-Calc performs well in predicting outcomes in a patient population that is similar to the patients that was used to develop it. Although still acceptable, model performance was slightly worse in a distinct population, despite the fact that the sample was more homogenous. Model performance may also be attributed to the low discrimination threshold, with close to 90% of cases reporting Any Improvement in outcome. Prediction models may need to be developed that are highly specific to the characteristics of the population.

Internal validation of the GA118-24B Genetic Analyzer, a stable capillary electrophoresis system for forensic DNA identification.

The GA118-24B Genetic Analyzer (hereafter, "GA118-24B") is an independently developed capillary electrophoresis instrument. In the present research, we designed a series of validation experiments to test its performance at detecting DNA fragments compared to the Applied Biosystems 3500 Genetic Analyzer (hereafter, "3500"). Three commercially available autosomal short tandem repeat multiplex kits were used in this validation. The results showed that GA118-24B had acceptable spectral calibration for three kits. The results of accuracy and concordance studies were also satisfactory. GA118-24B showed excellent precision, with a standard deviation of less than 0.1 bp. Sensitivity and mixture studies indicated that GA118-24B could detect low-template DNA and complex mixtures as well as the results generated by 3500 in parallel experiments. Based on the experimental results, we set specific analytical and stochastic thresholds. Besides, GA118-24B showed superiority than 3500 within certain size ranges in the resolution study. Instead of conventional commercial multiplex kits, GA118-24B performed stably on a self-developed eight-dye multiplex system, which were not performed on 3500 Genetic Analyzer. We compared our validation results with those of previous research and found our results to be convincing. Overall, we conclude that GA118-24B is a stable and reliable genetic analyzer for forensic DNA identification.

Internal validation of reduced PCR reaction volume of the Qiagen Investigator® Argus X-12 QS Kit from blood samples on FTA cards.

The onus of proof in criminal cases is beyond any reasonable doubt, and the issue on the lack of complete internal validation data can be manipulated when it comes to justifying the validity and reliability of the X-chromosomal short tandem repeats analysis for court representation. Therefore, this research evaluated the efficiency of the optimized 60% reduced volumes for polymerase chain reaction (PCR) amplification using the Qiagen Investigator® Argus X-12 QS Kit, as well as the capillary electrophoresis (CE) sample preparation for blood samples on Flinder's Technology Associates (FTA) cards. Good-quality DNA profile (3000-12,000 RFU) from the purified blood sample on FTA card (1.2 mm) were obtained using the optimized PCR (10.0 µL of PCR reaction volume and 21 cycles) and CE (9.0 µL Hi-Di™ Formamide and 0.3 µL DNA Size Standard 550 [BTO] and 27 s injection time) conditions. The analytical and stochastic thresholds were 100 and 200 RFU, respectively. Hence, the internal validation data supported the use of the optimized 60% reduced PCR amplification reaction volume of the Qiagen Investigator® Argus X-12 QS Kit as well as the CE sample preparation for producing reliable DNA profiles that comply with the quality assurance standards for forensic DNA testing laboratories, while optimizing the analytical cost.

A Validated Composite Score Demonstrates Potential Superiority to MELD-Based Systems in Predicting Short-Term Survival in Patients with Liver Cirrhosis and Spontaneous Bacterial Peritonitis-A Preliminary Study.

BACKGROUND: Spontaneous bacterial peritonitis (SBP) is a severe complication in cirrhosis patients with ascites, leading to high mortality rates if not promptly treated. However, specific prediction models for SBP are lacking. AIMS: This study aimed to compare commonly used cirrhotic prediction models (CTP score, MELD, MELD-Na, iMELD, and MELD 3.0) for short-term mortality prediction and develop a novel model to improve mortality prediction. METHODS: Patients with the first episode of SBP were included. Prognostic values for mortality were assessed using AUROC analysis. A novel prediction model was developed and validated. RESULTS: In total, 327 SBP patients were analyzed, with HBV infection as the main etiologies. MELD 3.0 demonstrated the highest AUROC among the traditional models. The novel model, incorporating HRS, exhibited superior predictive accuracy for in-hospital in all patients and 3-month mortality in HBV-cirrhosis, with AUROC values of 0.827 and 0.813 respectively, surpassing 0.8. CONCLUSIONS: MELD 3.0 score outperformed the CTP score and showed a non-significant improvement compared to other MELD-based scores, while the novel SBP model demonstrated impressive accuracy. Internal validation and an HBV-related cirrhosis subgroup sensitivity analysis supported these findings, highlighting the need for a specific prognostic model for SBP and the importance of preventing HRS development to improve SBP prognosis.

Quality indices for topic model selection and evaluation: a literature review and case study.

BACKGROUND: Topic models are a class of unsupervised machine learning models, which facilitate summarization, browsing and retrieval from large unstructured document collections. This study reviews several methods for assessing the quality of unsupervised topic models estimated using non-negative matrix factorization. Techniques for topic model validation have been developed across disparate fields. We synthesize this literature, discuss the advantages and disadvantages of different techniques for topic model validation, and illustrate their usefulness for guiding model selection on a large clinical text corpus. DESIGN, SETTING AND DATA: Using a retrospective cohort design, we curated a text corpus containing 382,666 clinical notes collected between 01/01/2017 through 12/31/2020 from primary care electronic medical records in Toronto Canada. METHODS: Several topic model quality metrics have been proposed to assess different aspects of model fit. We explored the following metrics: reconstruction error, topic coherence, rank biased overlap, Kendall's weighted tau, partition coefficient, partition entropy and the Xie-Beni statistic. Depending on context, cross-validation and/or bootstrap stability analysis were used to estimate these metrics on our corpus. RESULTS: Cross-validated reconstruction error favored large topic models (K ≥ 100 topics) on our corpus. Stability analysis using topic coherence and the Xie-Beni statistic also favored large models (K = 100 topics). Rank biased overlap and Kendall's weighted tau favored small models (K = 5 topics). Few model evaluation metrics suggested mid-sized topic models (25 ≤ K ≤ 75) as being optimal. However, human judgement suggested that mid-sized topic models produced expressive low-dimensional summarizations of the corpus. CONCLUSIONS: Topic model quality indices are transparent quantitative tools for guiding model selection and evaluation. Our empirical illustration demonstrated that different topic model quality indices favor models of different complexity; and may not select models aligning with human judgment. This suggests that different metrics capture different aspects of model goodness of fit. A combination of topic model quality indices, coupled with human validation, may be useful in appraising unsupervised topic models.

Development and internal validation of a non-invasive clinical tool to predict sufficient omega-3 levels in early pregnancy.

BACKGROUND: Complications from preterm birth (PTB) are the leading cause of death and disability in those under five years. Whilst the role of omega-3 (n-3) supplementation in reducing PTB is well-established, growing evidence suggests supplementation use in those replete may increase the risk of early PTB. AIM: To develop a non-invasive tool to identify individuals with total n-3 serum levels above 4.3% of total fatty acids in early pregnancy. METHODS: We conducted a prospective observational study recruiting 331 participants from three clinical sites in Newcastle, Australia. Eligible participants (n = 307) had a singleton pregnancy between 8 and 20 weeks' gestation at recruitment. Data on factors associated with n-3 serum levels were collected using an electronic questionnaire; these included estimated intake of n-3 (including food type, portion size, frequency of consumption), n-3 supplementation, and sociodemographic factors. The optimal cut-point of estimated n-3 intake that predicted mothers with total serum n-3 levels likely above 4.3% was developed using multivariate logistic regression, adjusting for maternal age, body mass index, socioeconomic status, and n-3 supplementation use. Total serum n-3 levels above 4.3% was selected as previous research has demonstrated that mothers with these levels are at increased risk of early PTB if they take additional n-3 supplementation during pregnancy. Models were evaluated using various performance metrics including sensitivity, specificity, area under receiver operator characteristic (AUROC) curve, true positive rate (TPR) at 10% false positive rate (FPR), Youden Index, Closest to (0,1) Criteria, Concordance Probability, and Index of Union. Internal validation was performed using 1000-bootstraps to generate 95% confidence intervals for performance metrics generated. RESULTS: Of 307 eligible participants included for analysis, 58.6% had total n-3 serum levels above 4.3%. The optimal model had a moderate discriminative ability (AUROC 0.744, 95% CI 0.742-0.746) with 84.7% sensitivity, 54.7% specificity and 37.6% TPR at 10% FPR. CONCLUSIONS: Our non-invasive tool was a moderate predictor of pregnant women with total serum n-3 levels above 4.3%; however, its performance is not yet adequate for clinical use. TRIAL REGISTRATION: This trial was approved by the Hunter New England Human Research Ethics Committee of the Hunter New England Local Health District (Reference 2020/ETH00498 on 07/05/2020 and 2020/ETH02881 on 08/12/2020).

Development and internal validation of a model to predict long-term survival of ANCA associated vasculitis.

Objectives: Risk stratification and prognosis prediction are critical for appropriate management of anti-neutrophil cytoplasmic antibody (ANCA) associated vasculitis (AAV). Herein, we aim to develop and internally validate a prediction model specifically for long-term survival of patients with AAV. Methods: We thoroughly reviewed the medical charts of patients with AAV admitted to Peking Union Medical College Hospital from January 1999 to July 2019. The Least Absolute Shrinkage and Selection Operator method and the COX proportional hazard regression was used to develop the prediction model. The Harrell's concordance index (C-index), calibration curves and Brier scores were calculated to evaluate the model performance. The model was internally validated by bootstrap resampling methods. Results: A total of 653 patients were included in the study, including 303 patients with microscopic polyangiitis, 245 patients with granulomatosis with polyangiitis and 105 patients with eosinophilic granulomatosis with polyangiitis, respectively. During a median follow-up of 33 months (interquartile range 15-60 months), 120 deaths occurred. Age at admission, chest and cardiovascular involvement, serum creatinine grade, hemoglobin levels at baseline and AAV sub-types were selected as predictive parameters in the final model. The optimism-corrected C-index and integrated Brier score of our prediction model were 0.728 and 0.109. The calibration plots showed fine agreement between observed and predicted probability of all-cause death. The decision curve analysis (DCA) showed that in a wide range of threshold probabilities, our prediction model had higher net benefits compared with the revised five factor score (rFFSand) and the birmingham vasculitis activity score (BVAS) system. Conclusion: Our model performs well in predicting outcomes of AAV patients. Patients with moderate-to-high probability of death should be followed closely and personalized monitoring plan should be scheduled.

Application of artificial intelligence for overall survival risk stratification in oropharyngeal carcinoma: A validation of ProgTOOL.

BACKGROUND: In recent years, there has been a surge in machine learning-based models for diagnosis and prognostication of outcomes in oncology. However, there are concerns relating to the model's reproducibility and generalizability to a separate patient cohort (i.e., external validation). OBJECTIVES: This study primarily provides a validation study for a recently introduced and publicly available machine learning (ML) web-based prognostic tool (ProgTOOL) for overall survival risk stratification of oropharyngeal squamous cell carcinoma (OPSCC). Additionally, we reviewed the published studies that have utilized ML for outcome prognostication in OPSCC to examine how many of these models were externally validated, type of external validation, characteristics of the external dataset, and diagnostic performance characteristics on the internal validation (IV) and external validation (EV) datasets were extracted and compared. METHODS: We used a total of 163 OPSCC patients obtained from the Helsinki University Hospital to externally validate the ProgTOOL for generalizability. In addition, PubMed, OvidMedline, Scopus, and Web of Science databases were systematically searched according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. RESULTS: The ProgTOOL produced a predictive performance of 86.5% balanced accuracy, Mathew's correlation coefficient of 0.78, Net Benefit (0.7) and Brier score (0.06) for overall survival stratification of OPSCC patients as either low-chance or high-chance. In addition, out of a total of 31 studies found to have used ML for the prognostication of outcomes in OPSCC, only seven (22.6%) reported a form of EV. Three studies (42.9%) each used either temporal EV or geographical EV while only one study (14.2%) used expert as a form of EV. Most of the studies reported a reduction in performance when externally validated. CONCLUSION: The performance of the model in this validation study indicates that it may be generalized, therefore, bringing recommendations of the model for clinical evaluation closer to reality. However, the number of externally validated ML-based models for OPSCC is still relatively small. This significantly limits the transfer of these models for clinical evaluation and subsequently reduces the likelihood of the use of these models in daily clinical practice. As a gold standard, we recommend the use of geographical EV and validation studies to reveal biases and overfitting of these models. These recommendations are poised to facilitate the implementation of these models in clinical practice.

Predictive models to assess risk of extended length of stay in adults with spinal deformity and lumbar degenerative pathology: development and internal validation.

BACKGROUND CONTEXT: Postoperative recovery after adult spinal deformity (ASD) operations is arduous, fraught with complications, and often requires extended hospital stays. A need exists for a method to rapidly predict patients at risk for extended length of stay (eLOS) in the preoperative setting. PURPOSE: To develop a machine learning model to preoperatively estimate the likelihood of eLOS following elective multi-level lumbar/thoracolumbar spinal instrumented fusions (≥3 segments) for ASD. STUDY DESIGN/SETTING: Retrospectively from a state-level inpatient database hosted by the Health care cost and Utilization Project. PATIENT SAMPLE: Of 8,866 patients of age ≥50 with ASD undergoing elective lumbar or thoracolumbar multilevel instrumented fusions. OUTCOME MEASURES: The primary outcome was eLOS (>7 days). METHODS: Predictive variables consisted of demographics, comorbidities, and operative information. Significant variables from univariate and multivariate analyses were used to develop a logistic regression-based predictive model that use six predictors. Model accuracy was assessed through area under the curve (AUC), sensitivity, and specificity. RESULTS: Of 8,866 patients met inclusion criteria. A saturated logistic model with all significant variables from multivariate analysis was developed (AUC=0.77), followed by generation of a simplified logistic model through stepwise logistic regression (AUC=0.76). Peak AUC was reached with inclusion of six selected predictors (combined anterior and posterior approach, surgery to both lumbar and thoracic regions, ≥8 level fusion, malnutrition, congestive heart failure, and academic institution). A cutoff of 0.18 for eLOS yielded a sensitivity of 77% and specificity of 68%. CONCLUSIONS: This predictive model can facilitate identification of adults at risk for eLOS following elective multilevel lumbar/thoracolumbar spinal instrumented fusions for ASD. With a fair diagnostic accuracy, the predictive calculator will ideally enable clinicians to improve preoperative planning, guide patient expectations, enable optimization of modifiable risk factors, facilitate appropriate discharge planning, stratify financial risk, and accurately identify patients who may represent high-cost outliers. Future prospective studies that validate this risk assessment tool on external datasets would be valuable.

There is no such thing as a validated prediction model.

BACKGROUND: Clinical prediction models should be validated before implementation in clinical practice. But is favorable performance at internal validation or one external validation sufficient to claim that a prediction model works well in the intended clinical context? MAIN BODY: We argue to the contrary because (1) patient populations vary, (2) measurement procedures vary, and (3) populations and measurements change over time. Hence, we have to expect heterogeneity in model performance between locations and settings, and across time. It follows that prediction models are never truly validated. This does not imply that validation is not important. Rather, the current focus on developing new models should shift to a focus on more extensive, well-conducted, and well-reported validation studies of promising models. CONCLUSION: Principled validation strategies are needed to understand and quantify heterogeneity, monitor performance over time, and update prediction models when appropriate. Such strategies will help to ensure that prediction models stay up-to-date and safe to support clinical decision-making.

Development and internal validation of a prediction model for acute kidney injury following cardiac valve replacement surgery.

BACKGROUND: Acute kidney injury (AKI) is a common complication after cardiac surgery. This study aims to develop and validate a risk model for predicting AKI after cardiac valve replacement surgery. METHODS: Data from patients undergoing surgical valve replacement between January 2015 and December 2018 in our hospital were retrospectively analyzed. The subjects were randomly divided into a derivation cohort and a validation cohort at a ratio of 7:3. The primary outcome was defined as AKI within 7 days after surgery. Logistic regression analysis was conducted to select risk predictors for developing the prediction model. Receiver operator characteristic curve (ROC), calibration plot and clinical decision curve analysis (DCA) will be used to evaluate the discrimination, precision and clinical benefit of the prediction model. RESULTS: A total of 1159 patients were involved in this study. The prevalence of AKI following surgery was 37.0% (429/1159). Logistic regression analysis showed that age, hemoglobin, fibrinogen, serum uric acid, cystatin C, bicarbonate, and cardiopulmonary bypass time were independent risk factors associated with AKI after surgical valve replacement (all P < 0.05). The areas under the ROC curves (AUCs) in the derivation cohort and the validation cohort were 0.777 (95% CI 0.744-0.810) and 0.760 (95% CI 0.706-0.813), respectively. The calibration plots indicated excellent consistency between the prediction probability and actual probability. DCA demonstrated great clinical benefit of the prediction model. CONCLUSIONS: We developed a prediction model for predicting AKI after cardiac valve replacement surgery that was internally validated to have good discrimination, calibration, and clinical practicability.

Establishment of prediction model for severe acute pancreatitis complicated with abdominal hypertension / 中华胰腺病杂志

Objective:To construct the prediction model of SAP complicated with intra-abdominal hypertension (IAH), and evaluate the prediction efficiency of the model.Methods:The clinical data of 322 SAP patients admitted to the emergency department of Cangzhou Hospital of Integrated Chinese and Western Medicine in Hebei Province from January 2017 to December 2021 were retrospectively analyzed. They were divided into IAH group ( n=153) and control group ( n=169) according to whether they had IAH complications or not. The clinical characteristics and laboratory test results of the two groups were compared. Multifactor logistic step-up regression was used to analyze the risk factors of SAP patients complicated with IAH. A nomogram model for predicting SAP complicated with IAH was established by using R software. The receiver operating characteristic curve (ROC) of the model was plotted, and the area under the curve (AUC) was calculated to evaluate its prediction efficiency. Calibration chart, Hosmer-Lemesshow test and decision curve analysis were used to evaluate the prediction accuracy and clinical application value of the model. The Bootstrap method was applied to verify the model internally. Results:In IAH group, cases with body mass index, CRP, procalcitonin (PCT), WBC, acute physiological and chronic health assessmentⅡ (APACHEⅡ) score, modified CT Severity Index score (MCTSI), incidence of complications (abdominal effusion, abdominal infection, gastrointestinal dysfunction, shock, multiple organ dysfunction syndrome), mechanical ventilation, the number of high-volume fluid reactivation (24 h≥4 L) were more than those in control group; serum albumin and serum calcium in IAH group were lower than those in control group, and the differences were statistically significant (all P value <0.05). Multivariate logistic regression analysis showed that serum albumin ( OR=0.815, 95% CI 0.710-0.937), CRP ( OR=1.005, 95% CI 1.002-1.008), MCTSI ( OR=2.043, 95% CI 1.695-2.463), complication of gastrointestinal dysfunction ( OR=4.179, 95% CI 2.170-8.049), and high-volume fluid resuscitation ( OR=4.265, 95% CI 2.269-8.015) were independent risk factors for IAH in SAP.The Nomogram prediction model was established using the five factors above as parameters, and the AUC value for predicting IAH complication was 0.886. The Hosmer-Lemesshow test showed a high consistency between the prediction results and the actual clinical observation results ( P=0.189). The results of decision curve analysis showed that the prediction probability of the model was between 10% and 85%, which could bring more benefits to patients. Conclusions:The early prediction model of SAP with concurrent IAH is successfully established, which can better predict the risk of SAP with concurrent IAH.

Development and validation of a prediction model for early mortality after transcatheter aortic valve implantation (TAVI) based on the Netherlands Heart Registration (NHR): The TAVI-NHR risk model.

BACKGROUND: The currently available mortality prediction models (MPM) have suboptimal performance when predicting early mortality (30-days) following transcatheter aortic valve implantation (TAVI) on various external populations. We developed and validated a new TAVI-MPM based on a large number of predictors with recent data from a national heart registry. METHODS: We included all TAVI-patients treated in the Netherlands between 2013 and 2018, from the Netherlands Heart Registration. We used logistic-regression analysis based on the Akaike Information Criterion for variable selection. We multiply imputed missing values, but excluded variables with >30% missing values. For internal validation, we used ten-fold cross-validation. For temporal (prospective) validation, we used the 2018-data set for testing. We assessed discrimination by the c-statistic, predicted probability accuracy by the Brier score, and calibration by calibration graphs, and calibration-intercept and calibration slope. We compared our new model to the updated ACC-TAVI and IRRMA MPMs on our population. RESULTS: We included 9144 TAVI-patients. The observed early mortality was 4.0%. The final MPM had 10 variables, including: critical-preoperative state, procedure-acuteness, body surface area, serum creatinine, and diabetes-mellitus status. The median c-statistic was 0.69 (interquartile range [IQR] 0.646-0.75). The median Brier score was 0.038 (IQR 0.038-0.040). No signs of miscalibration were observed. The c-statistic's temporal-validation was 0.71 (95% confidence intervals 0.64-0.78). Our model outperformed the updated currently available MPMs ACC-TAVI and IRRMA (p value < 0.05). CONCLUSION: The new TAVI-model used additional variables and showed fair discrimination and good calibration. It outperformed the updated currently available TAVI-models on our population. The model's good calibration benefits preprocedural risk-assessment and patient counseling.