Reduce the Cold Start of COVID-19 In-hospital Mortality Prediction Models via Transfer Learning

ABSTRACT

At the beginning of the breakout of a new disease, the healthcare community almost always has little experience in treating patients of this kind. Similarly, due to insufficient patient records at the early stage of a pandemic, it is difficult to train an in-hospital mortality prediction model specific to the new disease. We call this the 'cold start' problem of mortality prediction models. In this paper, we aim to study the cold start problem of 3-days ahead COVID-19 mortality prediction models by the following two steps: (i) Train XGBoost [1] and logistic regression 3-days ahead mortality prediction models on MIMIC3, a publicly available ICU patient dataset [2];(ii) Apply those MIMIC3 models to COVID-19 patients and then use the prediction scores as a new feature to train COVID-19 3-days ahead mortality prediction models. Retrospective experiments are conducted on a real-world COVID-19 patient dataset(n = 1,287) collected in US from June 2020 to February 2021 with a mixed cohort of both ICU and Non-ICU patients. Since the dataset is imbalanced(death rate = 7.8%), we primarily focus on the relative improvement of AUPR. We trained models with and without MIMIC3 scores on the first 200, 400,..., 1000 patients respectively and then tested on the next 200 incoming patients. The results show a diminishing positive transfer effect of AUPR from 5.36% for the first 200 patients(death rate = 5.5%) to 3.58% for all 1,287 patients. Meanwhile the AUROC scores largely remain unchanged, regardless of the number of patients in the training set. What's more, the p-value of t-test suggests that the cold start problem disappears for a dataset larger than 600 COVID-19 patients. To conclude, we demonstrate the possibility of mitigating the cold start problem via the proposed method. © 2022 IEEE.