Efficacy comparison of different machine learning models to predict adverse inhospital outcome in patients with severe trauma / 中华创伤杂志

ABSTRACT

Objective:To compare the predictive performance of different machine learning models using pre-hospital data to predict adverse inhospital outcome in patients with severe trauma.Methods:A retrospective cohort study was conducted to analyze the clinical data of 100 135 patients with severe trauma from the National Trauma Data Bank (NTDB) from January 2017 to December 2018. There were 69 644 males and 30 480 females apart from 11 patients with missing gender information, with the range age of 16-89 years [(50.1±21.1)years]. Clinical characteristics included demographic information (sex and age), trauma type (blunt or penetrating trauma), pre-hospital time [emergency medical services (EMS) response time, EMS scene time, and EMS transport time], pre-hospital vital signs (systolic blood pressure, pulse rate, respiratory rate, and oxygen saturation), trauma score [Glasgow coma score (GCS) and injury severity score (ISS)]. The original data were divided into the training set (in the year 2017) and the testing set (in the year 2018) according to the year of admission, including 50 429 patients in the training set and 49 706 patients in the testing set. The patients were classified into non-adverse outcome group ( n=94 526) and adverse outcome group ( n=5 609), according to whether they had an adverse outcome or not. There were 2 808 patients with adverse outcome in the training set and 2 801 patients with adverse outcome in the testing set. All models were built based on the training set. Eight machine learning algorithms consisting of neural network (NNET), naive Bayes (NB), gradient boosting machine (GBM), adaptive boosting (Ada), random forest (RF), bagging tree (BT), categorical boosting (CatBoost) and extreme gradient boosting (XGB) were used to construct prediction models for clinical outcomes among patients with severe trauma based on their clinical features. Models were evaluated according to the sensitivity, specificity, area under the receiver operating characteristic (ROC) curve (AUC) and Hosmer-Lemeshow goodness-of-fit test. Results:Of the NNET, NB, GBM, Ada, RF, BT, CatBoost and XGB models in the testing set, the sensitivity was 0.84, 0.83, 0.27, 0.79, 0.83, 0.81, 0.62 and 0.78, respectively; the specificity was 0.79, 0.76, 0.81, 0.79, 0.79, 0.74, 0.83 and 0.79, respectively; the AUC was 0.89 (95% CI 0.88, 0.90), 0.86 (95% CI 0.85, 0.87), 0.54 (95% CI 0.53, 0.55), 0.86 (95% CI 0.85, 0.87), 0.88 (95% CI 0.88, 0.90), 0.83 (95% CI 0.82, 0.85), 0.77 (95% CI 0.76, 0.79) and 0.86 (95% CI 0.85, 0.87), respectively. The NNET model had the best differentiation. In terms of calibration degree, both NNET and NB showed good performance ( P>0.05 for Hosmer-Lemeshow goodness-of-fit test). Conclusion:The NNET model has a favorable predictive performance for adverse inhospital outcome in patients with severe trauma, which may provide a reference for the rapid prediction of prognosis in patients with severe trauma.