Domain Shifts in Machine Learning Based Covid-19 Diagnosis From Blood Tests.
J Med Syst
; 46(5): 23, 2022 Mar 29.
Article
in English
| MEDLINE | ID: covidwho-1763426
ABSTRACT
Many previous studies claim to have developed machine learning models that diagnose COVID-19 from blood tests. However, we hypothesize that changes in the underlying distribution of the data, so called domain shifts, affect the predictive performance and reliability and are a reason for the failure of such machine learning models in clinical application. Domain shifts can be caused, e.g., by changes in the disease prevalence (spreading or tested population), by refined RT-PCR testing procedures (way of taking samples, laboratory procedures), or by virus mutations. Therefore, machine learning models for diagnosing COVID-19 or other diseases may not be reliable and degrade in performance over time. We investigate whether domain shifts are present in COVID-19 datasets and how they affect machine learning methods. We further set out to estimate the mortality risk based on routinely acquired blood tests in a hospital setting throughout pandemics and under domain shifts. We reveal domain shifts by evaluating the models on a large-scale dataset with different assessment strategies, such as temporal validation. We present the novel finding that domain shifts strongly affect machine learning models for COVID-19 diagnosis and deteriorate their predictive performance and credibility. Therefore, frequent re-training and re-assessment are indispensable for robust models enabling clinical utility.
Keywords
Full text:
Available
Collection:
International databases
Database:
MEDLINE
Main subject:
COVID-19
Type of study:
Diagnostic study
/
Experimental Studies
/
Observational study
/
Prognostic study
Limits:
Humans
Language:
English
Journal:
J Med Syst
Year:
2022
Document Type:
Article
Affiliation country:
S10916-022-01807-1
Similar
MEDLINE
...
LILACS
LIS