Improvement and Multi-Population Generalizability of a Deep Learning-Based Chest Radiograph Severity Score for COVID-19

PurposeTo improve and test the generalizability of a deep learning-based model for assessment of COVID-19 lung disease severity on chest radiographs (CXRs) from different patient populations. Materials and MethodsA published convolutional Siamese neural network-based model previously trained on hospitalized patients with COVID-19 was tuned using 250 outpatient CXRs. This model produces a quantitative measure of COVID-19 lung disease severity (pulmonary x-ray severity (PXS) score). The model was evaluated on CXRs from four test sets, including 3 from the United States (patients hospitalized at an academic medical center (N=154), patients hospitalized at a community hospital (N=113), and outpatients (N=108)) and 1 from Brazil (patients at an academic medical center emergency department (N=303)). Radiologists from both countries independently assigned reference standard CXR severity scores, which were correlated with the PXS scores as a measure of model performance (Pearson r). The Uniform Manifold Approximation and Projection (UMAP) technique was used to visualize the neural network results. ResultsTuning the deep learning model with outpatient data improved model performance in two United States hospitalized patient datasets (r=0.88 and r=0.90, compared to baseline r=0.86). Model performance was similar, though slightly lower, when tested on the United States outpatient and Brazil emergency department datasets (r=0.86 and r=0.85, respectively). UMAP showed that the model learned disease severity information that generalized across test sets. ConclusionsPerformance of a deep learning-based model that extracts a COVID-19 severity score on CXRs improved using training data from a different patient cohort (outpatient versus hospitalized) and generalized across multiple populations.

COVID-19 Outpatient Screening: a Prediction Score for Adverse Events

BackgroundWe sought to develop an automatable score to predict hospitalization, critical illness, or death in patients at risk for COVID-19 presenting for urgent care during the Massachusetts outbreak. MethodsSingle-center study of adult outpatients seen in respiratory illness clinics (RICs) or the emergency department (ED), including development (n = 9381, March 7-May 2) and prospective (n = 2205, May 3-14) cohorts. Data was queried from Partners Enterprise Data Warehouse. Outcomes were hospitalization, critical illness or death within 7 days. We developed the COVID-19 Acuity Score (CoVA) using automatically extracted data from the electronic medical record and learning-to-rank ordinal logistic regression modeling. Calibration was assessed using predicted-to-observed ratio (E/O). Discrimination was assessed by C-statistics (AUC). ResultsIn the development cohort, 27.3%, 7.2%, and 1.1% of patients experienced hospitalization, critical illness, or death, respectively; and in the prospective cohort, 26.1%, 6.3%, and 0.5%. CoVA showed excellent performance in the development cohort (concurrent validation) for hospitalization (E/O: 1.00, AUC: 0.80); for critical illness (E/O: 1.00, AUC: 0.82); and for death (E/O: 1.00, AUC: 0.87). Performance in the prospective cohort (prospective validation) was similar for hospitalization (E/O: 1.01, AUC: 0.76); for critical illness (E/O 1.03, AUC: 0.79); and for death (E/O: 1.63, AUC=0.93). Among 30 predictors, the top five were age, diastolic blood pressure, blood oxygen saturation, COVID-19 testing status, and respiratory rate. ConclusionsCoVA is a prospectively validated automatable score to assessing risk for adverse outcomes related to COVID-19 infection in the outpatient setting.

Automated assessment of COVID-19 pulmonary disease severity on chest radiographs using convolutional Siamese neural networks

PurposeTo develop an automated measure of COVID-19 pulmonary disease severity on chest radiographs (CXRs), for longitudinal disease evaluation and clinical risk stratification. Materials and MethodsA convolutional Siamese neural network-based algorithm was trained to output a measure of pulmonary disease severity on anterior-posterior CXRs (pulmonary x-ray severity (PXS) score), using weakly-supervised pretraining on ~160,000 images from CheXpert and transfer learning on 314 CXRs from patients with COVID-19. The algorithm was evaluated on internal and external test sets from different hospitals, containing 154 and 113 CXRs respectively. The PXS score was correlated with a radiographic severity score independently assigned by two thoracic radiologists and one in-training radiologist. For 92 internal test set patients with follow-up CXRs, the change in PXS score was compared to radiologist assessments of change. The association between PXS score and subsequent intubation or death was assessed. ResultsThe PXS score correlated with the radiographic pulmonary disease severity score assigned to CXRs in the COVID-19 internal and external test sets ({rho}=0.84 and {rho}=0.78 respectively). The direction of change in PXS score in follow-up CXRs agreed with radiologist assessment ({rho}=0.74). In patients not intubated on the admission CXR, the PXS score predicted subsequent intubation or death within three days of hospital admission (area under the receiver operator characteristic curve=0.80 (95%CI 0.75-0.85)). ConclusionA Siamese neural network-based severity score automatically measures COVID-19 pulmonary disease severity in chest radiographs, which can be scaled and rapidly deployed for clinical triage and workflow optimization. SUMMARYA convolutional Siamese neural network-based algorithm can calculate a continuous radiographic pulmonary disease severity score in COVID-19 patients, which can be used for longitudinal disease evaluation and clinical risk stratification. KEY RESULTSO_LIA Siamese neural network-based severity score correlates with radiologist-annotated pulmonary disease severity on chest radiographs from patients with COVID-19 ({rho}=0.84 and {rho}=0.78 in internal and external test sets respectively). C_LIO_LIThe direction of change in the severity score in follow-up radiographs is concordant with radiologist assessment ({rho}=0.74). C_LIO_LIThe admission chest radiograph severity score can help predict subsequent intubation or death within three days of admission (receiver operator characteristic area under the curve=0.80). C_LI