Deep Learning for Lung Cancer Detection on Screening CT Scans: Results of a Large-Scale Public Competition and an Observer Study with 11 Radiologists.

PURPOSE: To determine whether deep learning algorithms developed in a public competition could identify lung cancer on low-dose CT scans with a performance similar to that of radiologists. MATERIALS AND METHODS: In this retrospective study, a dataset consisting of 300 patient scans was used for model assessment; 150 patient scans were from the competition set and 150 were from an independent dataset. Both test datasets contained 50 cancer-positive scans and 100 cancer-negative scans. The reference standard was set by histopathologic examination for cancer-positive scans and imaging follow-up for at least 2 years for cancer-negative scans. The test datasets were applied to the three top-performing algorithms from the Kaggle Data Science Bowl 2017 public competition: grt123, Julian de Wit and Daniel Hammack (JWDH), and Aidence. Model outputs were compared with an observer study of 11 radiologists that assessed the same test datasets. Each scan was scored on a continuous scale by both the deep learning algorithms and the radiologists. Performance was measured using multireader, multicase receiver operating characteristic analysis. RESULTS: The area under the receiver operating characteristic curve (AUC) was 0.877 (95% CI: 0.842, 0.910) for grt123, 0.902 (95% CI: 0.871, 0.932) for JWDH, and 0.900 (95% CI: 0.870, 0.928) for Aidence. The average AUC of the radiologists was 0.917 (95% CI: 0.889, 0.945), which was significantly higher than grt123 (P = .02); however, no significant difference was found between the radiologists and JWDH (P = .29) or Aidence (P = .26). CONCLUSION: Deep learning algorithms developed in a public competition for lung cancer detection in low-dose CT scans reached performance close to that of radiologists.Keywords: Lung, CT, Thorax, Screening, Oncology Supplemental material is available for this article. © RSNA, 2021.

Automated Assessment of COVID-19 Reporting and Data System and Chest CT Severity Scores in Patients Suspected of Having COVID-19 Using Artificial Intelligence.

Background The coronavirus disease 2019 (COVID-19) pandemic has spread across the globe with alarming speed, morbidity, and mortality. Immediate triage of patients with chest infections suspected to be caused by COVID-19 using chest CT may be of assistance when results from definitive viral testing are delayed. Purpose To develop and validate an artificial intelligence (AI) system to score the likelihood and extent of pulmonary COVID-19 on chest CT scans using the COVID-19 Reporting and Data System (CO-RADS) and CT severity scoring systems. Materials and Methods The CO-RADS AI system consists of three deep-learning algorithms that automatically segment the five pulmonary lobes, assign a CO-RADS score for the suspicion of COVID-19, and assign a CT severity score for the degree of parenchymal involvement per lobe. This study retrospectively included patients who underwent a nonenhanced chest CT examination because of clinical suspicion of COVID-19 at two medical centers. The system was trained, validated, and tested with data from one of the centers. Data from the second center served as an external test set. Diagnostic performance and agreement with scores assigned by eight independent observers were measured using receiver operating characteristic analysis, linearly weighted κ values, and classification accuracy. Results A total of 105 patients (mean age, 62 years ± 16 [standard deviation]; 61 men) and 262 patients (mean age, 64 years ± 16; 154 men) were evaluated in the internal and external test sets, respectively. The system discriminated between patients with COVID-19 and those without COVID-19, with areas under the receiver operating characteristic curve of 0.95 (95% CI: 0.91, 0.98) and 0.88 (95% CI: 0.84, 0.93), for the internal and external test sets, respectively. Agreement with the eight human observers was moderate to substantial, with mean linearly weighted κ values of 0.60 ± 0.01 for CO-RADS scores and 0.54 ± 0.01 for CT severity scores. Conclusion With high diagnostic performance, the CO-RADS AI system correctly identified patients with COVID-19 using chest CT scans and assigned standardized CO-RADS and CT severity scores that demonstrated good agreement with findings from eight independent observers and generalized well to external data. © RSNA, 2020 Supplemental material is available for this article.

Brock malignancy risk calculator for pulmonary nodules: validation outside a lung cancer screening population.

OBJECTIVE: To assess the performance of the Brock malignancy risk model for pulmonary nodules detected in routine clinical setting. METHODS: In two academic centres in the Netherlands, we established a list of patients aged ≥40 years who received a chest CT scan between 2004 and 2012, resulting in 16 850 and 23 454 eligible subjects. Subsequent diagnosis of lung cancer until the end of 2014 was established through linking with the National Cancer Registry. A nested case-control study was performed (ratio 1:3). Two observers used semiautomated software to annotate the nodules. The Brock model was separately validated on each data set using ROC analysis and compared with a solely size-based model. RESULTS: After the annotation process the final analysis included 177 malignant and 695 benign nodules for centre A, and 264 malignant and 710 benign nodules for centre B. The full Brock model resulted in areas under the curve (AUCs) of 0.90 and 0.91, while the size-only model yielded significantly lower AUCs of 0.88 and 0.87, respectively (p<0.001). At 10% malignancy risk, the threshold suggested by the British Thoracic Society, sensitivity of the full model was 75% and 81%, specificity was 85% and 84%, positive predictive values were 14% and 10% at negative predictive value (NPV) of 99%. The optimal threshold was 6% for centre A and 8% for centre B, with NPVs >99%. DISCUSSION: The Brock model shows high predictive discrimination of potentially malignant and benign nodules when validated in an unselected, heterogeneous clinical population. The high NPV may be used to decrease the number of nodule follow-up examinations.

Towards automatic pulmonary nodule management in lung cancer screening with deep learning.

The introduction of lung cancer screening programs will produce an unprecedented amount of chest CT scans in the near future, which radiologists will have to read in order to decide on a patient follow-up strategy. According to the current guidelines, the workup of screen-detected nodules strongly relies on nodule size and nodule type. In this paper, we present a deep learning system based on multi-stream multi-scale convolutional networks, which automatically classifies all nodule types relevant for nodule workup. The system processes raw CT data containing a nodule without the need for any additional information such as nodule segmentation or nodule size and learns a representation of 3D data by analyzing an arbitrary number of 2D views of a given nodule. The deep learning system was trained with data from the Italian MILD screening trial and validated on an independent set of data from the Danish DLCST screening trial. We analyze the advantage of processing nodules at multiple scales with a multi-stream convolutional network architecture, and we show that the proposed deep learning system achieves performance at classifying nodule type that surpasses the one of classical machine learning approaches and is within the inter-observer variability among four experienced human observers.

Lung-RADS Category 4X: Does It Improve Prediction of Malignancy in Subsolid Nodules?

Purpose To evaluate the added value of Lung CT Screening Reporting and Data System (Lung-RADS) assessment category 4X over categories 3, 4A, and 4B for differentiating between benign and malignant subsolid nodules (SSNs). Materials and Methods SSNs on all baseline computed tomographic (CT) scans from the National Lung Cancer Trial that would have been classified as Lung-RADS category 3 or higher were identified, resulting in 374 SSNs for analysis. An experienced screening radiologist volumetrically segmented all solid cores and located all malignant SSNs visible on baseline scans. Six experienced chest radiologists independently determined which nodules to upgrade to category 4X, a recently introduced category for lesions that demonstrate additional features or imaging findings that increase the suspicion of malignancy. Malignancy rates of purely size-based categories and category 4X were compared. Furthermore, the false-positive rates of category 4X lesions were calculated and observer variability was assessed by using Fleiss κ statistics. Results The observers upgraded 15%-24% of the SSNs to category 4X. The malignancy rate for 4X nodules varied from 46% to 57% per observer and was substantially higher than the malignancy rates of categories 3, 4A, and 4B SSNs without observer intervention (9%, 19%, and 23%, respectively). On average, the false-positive rate for category 4X nodules was 7% for category 3 SSNs, 7% for category 4A SSNs, and 19% for category 4B SSNs. Of the falsely upgraded benign lesions, on average 27% were transient. The agreement among the observers was moderate, with an average κ value of 0.535 (95% confidence interval: 0.509, 0.561). Conclusion The inclusion of a 4X assessment category for lesions suspicious for malignancy in a nodule management tool is of added value and results in high malignancy rates in the hands of experienced radiologists. Proof of the transient character of category 4X lesions at short-term follow-up could avoid unnecessary invasive management. © RSNA, 2017.

Pulmonary Nodule Detection in CT Images: False Positive Reduction Using Multi-View Convolutional Networks.

We propose a novel Computer-Aided Detection (CAD) system for pulmonary nodules using multi-view convolutional networks (ConvNets), for which discriminative features are automatically learnt from the training data. The network is fed with nodule candidates obtained by combining three candidate detectors specifically designed for solid, subsolid, and large nodules. For each candidate, a set of 2-D patches from differently oriented planes is extracted. The proposed architecture comprises multiple streams of 2-D ConvNets, for which the outputs are combined using a dedicated fusion method to get the final classification. Data augmentation and dropout are applied to avoid overfitting. On 888 scans of the publicly available LIDC-IDRI dataset, our method reaches high detection sensitivities of 85.4% and 90.1% at 1 and 4 false positives per scan, respectively. An additional evaluation on independent datasets from the ANODE09 challenge and DLCST is performed. We showed that the proposed multi-view ConvNets is highly suited to be used for false positive reduction of a CAD system.