Deep Learning for Lung Cancer Detection on Screening CT Scans: Results of a Large-Scale Public Competition and an Observer Study with 11 Radiologists.

PURPOSE: To determine whether deep learning algorithms developed in a public competition could identify lung cancer on low-dose CT scans with a performance similar to that of radiologists. MATERIALS AND METHODS: In this retrospective study, a dataset consisting of 300 patient scans was used for model assessment; 150 patient scans were from the competition set and 150 were from an independent dataset. Both test datasets contained 50 cancer-positive scans and 100 cancer-negative scans. The reference standard was set by histopathologic examination for cancer-positive scans and imaging follow-up for at least 2 years for cancer-negative scans. The test datasets were applied to the three top-performing algorithms from the Kaggle Data Science Bowl 2017 public competition: grt123, Julian de Wit and Daniel Hammack (JWDH), and Aidence. Model outputs were compared with an observer study of 11 radiologists that assessed the same test datasets. Each scan was scored on a continuous scale by both the deep learning algorithms and the radiologists. Performance was measured using multireader, multicase receiver operating characteristic analysis. RESULTS: The area under the receiver operating characteristic curve (AUC) was 0.877 (95% CI: 0.842, 0.910) for grt123, 0.902 (95% CI: 0.871, 0.932) for JWDH, and 0.900 (95% CI: 0.870, 0.928) for Aidence. The average AUC of the radiologists was 0.917 (95% CI: 0.889, 0.945), which was significantly higher than grt123 (P = .02); however, no significant difference was found between the radiologists and JWDH (P = .29) or Aidence (P = .26). CONCLUSION: Deep learning algorithms developed in a public competition for lung cancer detection in low-dose CT scans reached performance close to that of radiologists.Keywords: Lung, CT, Thorax, Screening, Oncology Supplemental material is available for this article. © RSNA, 2021.

CT-Detected Subsolid Nodules: A Predictor of Lung Cancer Development at Another Location?

The purpose of this case-cohort study was to investigate whether the frequency and computed tomography (CT) features of pulmonary nodules posed a risk for the future development of lung cancer (LC) at a different location. Patients scanned between 2004 and 2012 at two Dutch academic hospitals were cross-linked with the Dutch Cancer Registry. All patients who were diagnosed with LC by 2014 and a random selection of LC-free patients were considered. LC patients who were determined to be LC-free at the time of the scan and all LC-free patients with an adequate scan were included. The nodule count and types (solid, part-solid, ground-glass, and perifissural) were recorded per scan. Age, sex, and other CT measures were included to control for confounding factors. The cohort included 163 LC patients and 1178 LC-free patients. Cox regression revealed that the number of ground-glass nodules and part-solid nodules present were positively correlated to future LC risk. The area under the receiver operating curve of parsimonious models with and without nodule type information were 0.827 and 0.802, respectively. The presence of subsolid nodules in a clinical setting may be a risk factor for future LC development in another pulmonary location in a dose-dependent manner. Replication of the results in screening cohorts is required for maximum utility of these findings.

Deep Learning for Malignancy Risk Estimation of Pulmonary Nodules Detected at Low-Dose Screening CT.

Background Accurate estimation of the malignancy risk of pulmonary nodules at chest CT is crucial for optimizing management in lung cancer screening. Purpose To develop and validate a deep learning (DL) algorithm for malignancy risk estimation of pulmonary nodules detected at screening CT. Materials and Methods In this retrospective study, the DL algorithm was developed with 16 077 nodules (1249 malignant) collected -between 2002 and 2004 from the National Lung Screening Trial. External validation was performed in the following three -cohorts -collected between 2004 and 2010 from the Danish Lung Cancer Screening Trial: a full cohort containing all 883 nodules (65 -malignant) and two cancer-enriched cohorts with size matching (175 nodules, 59 malignant) and without size matching (177 -nodules, 59 malignant) of benign nodules selected at random. Algorithm performance was measured by using the area under the receiver operating characteristic curve (AUC) and compared with that of the Pan-Canadian Early Detection of Lung Cancer (PanCan) model in the full cohort and a group of 11 clinicians composed of four thoracic radiologists, five radiology residents, and two pulmonologists in the cancer-enriched cohorts. Results The DL algorithm significantly outperformed the PanCan model in the full cohort (AUC, 0.93 [95% CI: 0.89, 0.96] vs 0.90 [95% CI: 0.86, 0.93]; P = .046). The algorithm performed comparably to thoracic radiologists in cancer-enriched cohorts with both random benign nodules (AUC, 0.96 [95% CI: 0.93, 0.99] vs 0.90 [95% CI: 0.81, 0.98]; P = .11) and size-matched benign nodules (AUC, 0.86 [95% CI: 0.80, 0.91] vs 0.82 [95% CI: 0.74, 0.89]; P = .26). Conclusion The deep learning algorithm showed excellent performance, comparable to thoracic radiologists, for malignancy risk estimation of pulmonary nodules detected at screening CT. This algorithm has the potential to provide reliable and reproducible malignancy risk scores for clinicians, which may help optimize management in lung cancer screening. © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Tammemägi in this issue.

Brock malignancy risk calculator for pulmonary nodules: validation outside a lung cancer screening population.

OBJECTIVE: To assess the performance of the Brock malignancy risk model for pulmonary nodules detected in routine clinical setting. METHODS: In two academic centres in the Netherlands, we established a list of patients aged ≥40 years who received a chest CT scan between 2004 and 2012, resulting in 16 850 and 23 454 eligible subjects. Subsequent diagnosis of lung cancer until the end of 2014 was established through linking with the National Cancer Registry. A nested case-control study was performed (ratio 1:3). Two observers used semiautomated software to annotate the nodules. The Brock model was separately validated on each data set using ROC analysis and compared with a solely size-based model. RESULTS: After the annotation process the final analysis included 177 malignant and 695 benign nodules for centre A, and 264 malignant and 710 benign nodules for centre B. The full Brock model resulted in areas under the curve (AUCs) of 0.90 and 0.91, while the size-only model yielded significantly lower AUCs of 0.88 and 0.87, respectively (p<0.001). At 10% malignancy risk, the threshold suggested by the British Thoracic Society, sensitivity of the full model was 75% and 81%, specificity was 85% and 84%, positive predictive values were 14% and 10% at negative predictive value (NPV) of 99%. The optimal threshold was 6% for centre A and 8% for centre B, with NPVs >99%. DISCUSSION: The Brock model shows high predictive discrimination of potentially malignant and benign nodules when validated in an unselected, heterogeneous clinical population. The high NPV may be used to decrease the number of nodule follow-up examinations.

In vivo growth of 60 non-screening detected lung cancers: a computed tomography study.

Current pulmonary nodule management guidelines are based on nodule volume doubling time, which assumes exponential growth behaviour. However, this is a theory that has never been validated in vivo in the routine-care target population. This study evaluates growth patterns of untreated solid and subsolid lung cancers of various histologies in a non-screening setting.Growth behaviour of pathology-proven lung cancers from two academic centres that were imaged at least three times before diagnosis (n=60) was analysed using dedicated software. Random-intercept random-slope mixed-models analysis was applied to test which growth pattern most accurately described lung cancer growth. Individual growth curves were plotted per pathology subgroup and nodule type.We confirmed that growth in both subsolid and solid lung cancers is best explained by an exponential model. However, subsolid lesions generally progress slower than solid ones. Baseline lesion volume was not related to growth, indicating that smaller lesions do not grow slower compared to larger ones.By showing that lung cancer conforms to exponential growth we provide the first experimental basis in the routine-care setting for the assumption made in volume doubling time analysis.

Visual discrimination of screen-detected persistent from transient subsolid nodules: An observer study.

PURPOSE: To evaluate whether, and to which extent, experienced radiologists are able to visually correctly differentiate transient from persistent subsolid nodules from a single CT examination alone and to determine CT morphological features to make this differentiation. MATERIALS AND METHODS: We selected 86 transient and 135 persistent subsolid nodules from the National Lung Screening Trial (NLST) database. Four experienced radiologists visually assessed a predefined list of morphological features and gave a final judgment on a continuous scale (0-100). To assess observer performance, area under the receiver operating characteristic (ROC) curve was calculated. Statistical differences of morphological features between transient and persistent lesions were calculated using Chi-square. Inter-observer agreement of morphological features was evaluated by percentage agreement. RESULTS: Forty-nine lesions were excluded by at least 2 observers, leaving 172 lesions for analysis. On average observers were able to differentiate transient from persistent subsolid nodules ≥ 10 mm with an area under the curve of 0.75 (95% CI 0.67-0.82). Nodule type, lesion margin, presence of a well-defined border, and pleural retraction showed significant differences between transient and persistent lesions in two observers. Average pair-wise percentage agreement for these features was 81%, 64%, 47% and 89% respectively. Agreement for other morphological features varied from 53% to 95%. CONCLUSION: The visual capacity of experienced radiologists to differentiate persistent and transient subsolid nodules is moderate in subsolid nodules larger than 10 mm. Performance of the visual assessment of CT morphology alone is not sufficient to generally abandon a short-term follow-up for subsolid nodules.

Automatic segmentation of the solid core and enclosed vessels in subsolid pulmonary nodules.

Subsolid pulmonary nodules are commonly encountered in lung cancer screening and clinical routine. Compared to other nodule types, subsolid nodules are associated with a higher malignancy probability for which the size and mass of the nodule and solid core are important indicators. However, reliably measuring these characteristics on computed tomography (CT) can be hampered by the presence of vessels encompassed by the nodule, since vessels have similar CT attenuation as solid cores. This can affect treatment decisions and patient management. We present a method based on voxel classification to automatically identify vessels and solid cores in given subsolid nodules on CT. Three experts validated our method on 170 screen-detected subsolid nodules from the Multicentric Italian Lung Disease trial. The agreement between the proposed method and the observers was substantial for vessel detection and moderate for solid core detection, which was similar to the inter-observer agreement. We found a relatively high variability in the inter-observer agreement and low method-observer agreements for delineating the borders of vessels and solid cores, illustrating the difficulty of this task. However, 92.4% of the proposed vessel and 80.6% of the proposed solid core segmentations were labeled as usable in clinical practice by the majority of experts.

Incidental perifissural nodules on routine chest computed tomography: lung cancer or not?

OBJECTIVES: Perifissural nodules (PFNs) are a common finding on chest CT, and are thought to represent non-malignant lesions. However, data outside a lung cancer-screening setting are currently lacking. METHODS: In a nested case-control design, out of a total cohort of 16,850 patients ≥ 40 years of age who underwent routine chest CT (2004-2012), 186 eligible subjects with incident lung cancer and 511 controls without were investigated. All non-calcified nodules ≥ 4 mm were semi-automatically annotated. Lung cancer location and subject characteristics were recorded. RESULTS: Cases (56 % male) had a median age of 64 years (IQR 59-70). Controls (60 % male) were slightly younger (p<0.01), median age of 61 years (IQR 51-70). A total of 262/1,278 (21 %) unique non-calcified nodules represented a PFN. None of these were traced to a lung malignancy over a median follow-up of around 4.5 years. PFNs were most often located in the lower lung zones (72 %, p<0.001). Median diameter was 4.6 mm (range: 4.0-8.1), volume 51 mm3 (range: 32-278). Some showed growth rates < 400 days. CONCLUSIONS: Our data show that incidental PFNs do not represent lung cancer in a routine care, heterogeneous population. This confirms prior screening-based results. KEY POINTS: • One-fifth of non-calcified nodules represented a perifissural nodule in our non-screening population. • PFNs fairly often show larger size, and can show interval growth. • When morphologically resembling a PFN, nodules are nearly certainly not a malignancy. • The assumed benign aetiology of PFNs seems valid outside the screening setting.

Malignancy estimation of Lung-RADS criteria for subsolid nodules on CT: accuracy of low and high risk spectrum when using NLST nodules.

PURPOSE: Lung-RADS proposes malignancy probabilities for categories 2 (<1%) and 4B (>15%). The purpose of this study was to quantify and compare malignancy rates for Lung-RADS 2 and 4B subsolid nodules (SSNs) on a nodule base. METHODS: We identified all baseline SSNs eligible for Lung-RADS 2 and 4B in the National Lung Screening Trial (NLST) database. Solid cores and nodule locations were annotated using in-house software. Malignant SSNs were identified by an experienced radiologist using NLST information. Malignancy rates and percentages of persistence were calculated. RESULTS: Of the Lung-RADS 2SSNs, 94.3% (1790/1897) could be located on chest CTs. Likewise, 95.1% (331/348) of part-solid nodules ≥6 mm in diameter could be located. Of these, 120 had a solid core ≥8 mm, corresponding to category 4B. Category 2 SSNs showed a malignancy rate of 2.5%, exceeding slightly the proposed rate of <1%. Category 4B SSNs showed a malignancy rate of 23.9%. In both categories one third of benign lesions were transient. CONCLUSION: Malignancy probabilities for Lung-RADS 2 and 4B generally match malignancy rates in SSNs. An option to include also category 2 SSNs for upgrade to 4X designed for suspicious nodules might be useful in the future. Integration of short-term follow-up to confirm persistence would prevent unnecessary invasive work-up in 4B SSNs. KEY POINTS: • Malignancy probabilities for Lung-RADS 2/4B generally match malignancy risks in SSNs. • Transient rate between low-risk Lung-RADS 2 and high-risk 4B lesions were similar. • Upgrade of highly suspicious Lung-RADS 2 SSNs to Lung-RADS 4X might be useful. • Up to one third of the benign high-risk Lung-RADS 4B lesions were transient. • Short-term follow-up confirming persistence would avoid unnecessary invasive work-up of 4B lesions.

Towards automatic pulmonary nodule management in lung cancer screening with deep learning.

The introduction of lung cancer screening programs will produce an unprecedented amount of chest CT scans in the near future, which radiologists will have to read in order to decide on a patient follow-up strategy. According to the current guidelines, the workup of screen-detected nodules strongly relies on nodule size and nodule type. In this paper, we present a deep learning system based on multi-stream multi-scale convolutional networks, which automatically classifies all nodule types relevant for nodule workup. The system processes raw CT data containing a nodule without the need for any additional information such as nodule segmentation or nodule size and learns a representation of 3D data by analyzing an arbitrary number of 2D views of a given nodule. The deep learning system was trained with data from the Italian MILD screening trial and validated on an independent set of data from the Danish DLCST screening trial. We analyze the advantage of processing nodules at multiple scales with a multi-stream convolutional network architecture, and we show that the proposed deep learning system achieves performance at classifying nodule type that surpasses the one of classical machine learning approaches and is within the inter-observer variability among four experienced human observers.

Lung-RADS Category 4X: Does It Improve Prediction of Malignancy in Subsolid Nodules?

Purpose To evaluate the added value of Lung CT Screening Reporting and Data System (Lung-RADS) assessment category 4X over categories 3, 4A, and 4B for differentiating between benign and malignant subsolid nodules (SSNs). Materials and Methods SSNs on all baseline computed tomographic (CT) scans from the National Lung Cancer Trial that would have been classified as Lung-RADS category 3 or higher were identified, resulting in 374 SSNs for analysis. An experienced screening radiologist volumetrically segmented all solid cores and located all malignant SSNs visible on baseline scans. Six experienced chest radiologists independently determined which nodules to upgrade to category 4X, a recently introduced category for lesions that demonstrate additional features or imaging findings that increase the suspicion of malignancy. Malignancy rates of purely size-based categories and category 4X were compared. Furthermore, the false-positive rates of category 4X lesions were calculated and observer variability was assessed by using Fleiss κ statistics. Results The observers upgraded 15%-24% of the SSNs to category 4X. The malignancy rate for 4X nodules varied from 46% to 57% per observer and was substantially higher than the malignancy rates of categories 3, 4A, and 4B SSNs without observer intervention (9%, 19%, and 23%, respectively). On average, the false-positive rate for category 4X nodules was 7% for category 3 SSNs, 7% for category 4A SSNs, and 19% for category 4B SSNs. Of the falsely upgraded benign lesions, on average 27% were transient. The agreement among the observers was moderate, with an average κ value of 0.535 (95% confidence interval: 0.509, 0.561). Conclusion The inclusion of a 4X assessment category for lesions suspicious for malignancy in a nodule management tool is of added value and results in high malignancy rates in the hands of experienced radiologists. Proof of the transient character of category 4X lesions at short-term follow-up could avoid unnecessary invasive management. © RSNA, 2017.

Subsolid pulmonary nodule morphology and associated patient characteristics in a routine clinical population.

OBJECTIVES: To determine the presence and morphology of subsolid pulmonary nodules (SSNs) in a non-screening setting and relate them to clinical and patient characteristics. METHODS: A total of 16,890 reports of clinically obtained chest CT (06/2011 to 11/2014, single-centre) were searched describing an SSN. Subjects with a visually confirmed SSN and at least two thin-slice CTs were included. Nodule volumes were measured. Progression was defined as volume increase exceeding the software interscan variation. Nodule morphology, location, and patient characteristics were evaluated. RESULTS: Fifteen transient and 74 persistent SSNs were included (median follow-up 19.6 [8.3-36.8] months). Subjects with an SSN were slightly older than those without (62 vs. 58 years; p = 0.01), but no gender predilection was found. SSNs were mostly located in the upper lobes. Women showed significantly more often persistent lesions than men (94 % vs. 69 %; p = 0.002). Part-solid lesions were larger (1638 vs. 383 mm3; p < 0.001) and more often progressive (68 % vs. 38 %; p = 0.02), compared to pure ground-glass nodules. Progressive SSNs were rare under the age of 50 years. Logistic regression analysis did not identify additional nodule parameters of future progression, apart from part-solid nature. CONCLUSIONS: This study confirms previously reported characteristics of SSNs and associated factors in a European, routine clinical population. KEY POINTS: • SSNs in women are significantly more often persistent compared to men. • SSN persistence is not associated with age or prior malignancy. • The majority of (persistent) SSNs are located in the upper lung lobes. • A part-solid nature is associated with future nodule growth. • Progressive solitary SSNs are rare under the age of 50 years.

Fleischner recommendations for the management of subsolid pulmonary nodules: high awareness but limited conformance - a survey study.

OBJECTIVES: The aim of this study was to assess awareness and conformance to the Fleischner society recommendations for the management of subsolid pulmonary nodules (SSN) in clinical practice. METHODS: An online questionnaire with four imaging cases was sent to 1579 associates from the European Respiratory Society and 757 from the European Society of Thoracic Imaging. Each respondent was asked to choose from several options which one they thought was the indicated management for the nodule presented. Awareness and conformance to the Fleischner recommendations (FR) were assessed and correlated to respondents characteristics. RESULTS: In total, 119 radiologists (response rate 16.0 %) and 243 pulmonologists (response rate 16.5 %) were included. Awareness of the FR was higher in radiologists than in pulmonologists (93 % vs. 70 %, p < 0.001), as was implementation in daily practice (66 % vs. 47 %, p < 0.001). Radiologists conformed to FR in rates of 31, 69, 68, and 82 %, and pulmonologists in 12, 43, 70, and 75 % for cases 1 to 4, respectively. Overmanagement was common. Conformance in SSN management was associated with awareness, working in an academic practice, larger practice size, teaching residents, and higher SSN exposure. CONCLUSIONS: Although awareness of the Fleischner recommendations for SSN management is widespread, management choices in clinical practice show large heterogeneity. KEY POINTS: • Guideline awareness among clinicians is widespread, but conformance shows large heterogeneity. • Awareness and conformance is significantly higher among radiologists than pulmonologists. • Overmanagement is common, which may lead to avoidable financial and physical burden.

Automatic classification of pulmonary peri-fissural nodules in computed tomography using an ensemble of 2D views and a convolutional neural network out-of-the-box.

In this paper, we tackle the problem of automatic classification of pulmonary peri-fissural nodules (PFNs). The classification problem is formulated as a machine learning approach, where detected nodule candidates are classified as PFNs or non-PFNs. Supervised learning is used, where a classifier is trained to label the detected nodule. The classification of the nodule in 3D is formulated as an ensemble of classifiers trained to recognize PFNs based on 2D views of the nodule. In order to describe nodule morphology in 2D views, we use the output of a pre-trained convolutional neural network known as OverFeat. We compare our approach with a recently presented descriptor of pulmonary nodule morphology, namely Bag of Frequencies, and illustrate the advantages offered by the two strategies, achieving performance of AUC = 0.868, which is close to the one of human experts.

Impact on Patient Safety and Satisfaction of Implementation of an Outpatient Clinic in Interventional Radiology (IPSIPOLI-Study): A Quasi-Experimental Prospective Study.

PURPOSE: Interventional radiology (IR) procedures are associated with high rates of preparation and planning errors. In many centers, pre-procedural consultation and screening of patients is performed by referring physicians. Interventional radiologists have better knowledge about procedure details and risks, but often only get acquainted with the patient in the procedure room. We hypothesized that patient safety (PS) and patient satisfaction (PSAT) in elective IR procedures would improve by implementation of a pre-procedural visit to an outpatient IR clinic. MATERIAL AND METHODS: IRB approval was obtained and informed consent was waived. PS and PSAT were measured in patients undergoing elective IR procedures before (control group; n = 110) and after (experimental group; n = 110) implementation of an outpatient IR clinic. PS was measured as the number of process deviations. PSAT was assessed using a questionnaire measuring Likert scores of three dimensions: interpersonal care aspects, information/communication, and patient participation. Differences in PS and PSAT between the two groups were compared using an independent t test. RESULTS: The average number of process deviations per patient was 0.39 in the control group compared to 0.06 in the experimental group (p < 0.001). In 9.1 % patients in the control group, no legal informed consent was obtained compared to 0 % in the experimental group. The mean overall Likert score was significantly higher in the experimental group compared to the control group: 2.68 (SD 0.314) versus 2.48 (SD 0.381) (p < 0.001). CONCLUSION: PS and PSAT improve significantly if patients receive consultation and screening in an IR outpatient clinic prior to elective IR procedures.

Association of varicosities and concomitant deep venous thrombosis in patients with superficial venous thrombosis, a systematic review.

BACKGROUND: In patients with superficial venous thrombosis (SVT) co-existence of deep venous thrombosis (DVT) can be present. Varicosities are considered as a risk factor for both SVT and DVT separately. However, current evidence is contradictory whether varicosities are associated with an increased or reduced prevalence of concomitant DVT in patients with SVT. OBJECTIVES: To determine the diagnostic value of both presence and absence of varicosities in the detection of concomitant DVT in non-hospitalized patients with SVT. METHODS: In MEDLINE and EMBASE, a systematic search was performed to collect all published studies on this topic. The selected papers were critically appraised. By diagnostic 2 × 2 tables prior probabilities and predictive values were computed. RESULTS: Six relevant articles were identified. The prior probability of concomitant DVT in patients referred from primary care to the outpatient clinic varied between 13 and 34%. In five studies, absence of varicosities was related to a higher probability of concomitant DVT (33-44%) compared to a presence of varicosities (3-23%). The sixth study showed an inversed, non-significant association: DVT was present in 21% of patients with SVT on non-varicose veins versus in 35% of patients with SVT on varicose veins. CONCLUSION: In five out of six studies on patients with SVT in outpatient settings, absence of varicosities was related to a higher probability of concomitant DVT. Further research is needed to determine whether an assessment of varicosities in general practice could result in an improved selection of patients who require additional imaging to detect or exclude DVT.