Your browser doesn't support javascript.
The Pitfalls of Using Open Data to Develop Deep Learning Solutions for COVID-19 Detection in Chest X-Rays.
Harkness, Rachael; Hall, Geoff; Frangi, Alejandro F; Ravikumar, Nishant; Zucker, Kieran.
  • Harkness R; CISTIB Centre for Computational Imaging and Simulation Technologies in Biomedicine, School of Computing.
  • Hall G; University of Leeds, Leeds, LS2 9JT, United Kingdom.
  • Frangi AF; University of Leeds, Leeds, LS2 9JT, United Kingdom.
  • Ravikumar N; Leeds Institute of Medical Research at St James's, United Kingdom.
  • Zucker K; Leeds Teaching Hospitals NHS Trust, Leeds, LS9 7TF, United Kingdom.
Stud Health Technol Inform ; 290: 679-683, 2022 Jun 06.
Article in English | MEDLINE | ID: covidwho-1933570
ABSTRACT
Since the emergence of COVID-19, deep learning models have been developed to identify COVID-19 from chest X-rays. With little to no direct access to hospital data, the AI community relies heavily on public data comprising numerous data sources. Model performance results have been exceptional when training and testing on open-source data, surpassing the reported capabilities of AI in pneumonia-detection prior to the COVID-19 outbreak. In this study impactful models are trained on a widely used open-source data and tested on an external test set and a hospital dataset, for the task of classifying chest X-rays into one of three classes COVID-19, non-COVID pneumonia and no-pneumonia. Classification performance of the models investigated is evaluated through ROC curves, confusion matrices and standard classification metrics. Explainability modules are implemented to explore the image features most important to classification. Data analysis and model evalutions show that the popular open-source dataset COVIDx is not representative of the real clinical problem and that results from testing on this are inflated. Dependence on open-source data can leave models vulnerable to bias and confounding variables, requiring careful analysis to develop clinically useful/viable AI tools for COVID-19 detection in chest X-rays.
Subject(s)
Keywords

Full text: Available Collection: International databases Database: MEDLINE Main subject: Deep Learning / COVID-19 Type of study: Diagnostic study / Experimental Studies / Prognostic study Limits: Humans Language: English Journal: Stud Health Technol Inform Journal subject: Medical Informatics / Health Services Research Year: 2022 Document Type: Article

Similar

MEDLINE

...
LILACS

LIS


Full text: Available Collection: International databases Database: MEDLINE Main subject: Deep Learning / COVID-19 Type of study: Diagnostic study / Experimental Studies / Prognostic study Limits: Humans Language: English Journal: Stud Health Technol Inform Journal subject: Medical Informatics / Health Services Research Year: 2022 Document Type: Article