Explainable deep learning-based survival prediction for non-small cell lung cancer patients undergoing radical radiotherapy.

BACKGROUND AND PURPOSE: Survival is frequently assessed using Cox proportional hazards (CPH) regression; however, CPH may be too simplistic as it assumes a linear relationship between covariables and the outcome. Alternative, non-linear machine learning (ML)-based approaches, such as random survival forests (RSFs) and, more recently, deep learning (DL) have been proposed; however, these techniques are largely black-box in nature, limiting explainability. We compared CPH, RSF and DL to predict overall survival (OS) of non-small cell lung cancer (NSCLC) patients receiving radiotherapy using pre-treatment covariables. We employed explainable techniques to provide insights into the contribution of each covariable on OS prediction. MATERIALS AND METHODS: The dataset contained 471 stage I-IV NSCLC patients treated with radiotherapy. We built CPH, RSF and DL OS prediction models using several baseline covariable combinations. 10-fold Monte-Carlo cross-validation was employed with a split of 70%:10%:20% for training, validation and testing, respectively. We primarily evaluated performance using the concordance index (C-index) and integrated Brier score (IBS). Local interpretable model-agnostic explanation (LIME) values, adapted for use in survival analysis, were computed for each model. RESULTS: The DL method exhibited a significantly improved C-index of 0.670 compared to the CPH and a significantly improved IBS of 0.121 compared to the CPH and RSF approaches. LIME values suggested that, for the DL method, the three most important covariables in OS prediction were stage, administration of chemotherapy and oesophageal mean radiation dose. CONCLUSION: We show that, using pre-treatment covariables, a DL approach demonstrates superior performance over CPH and RSF for OS prediction and use explainable techniques to provide transparency and interpretability.

PhysVENeT: a physiologically-informed deep learning-based framework for the synthesis of 3D hyperpolarized gas MRI ventilation.

Functional lung imaging modalities such as hyperpolarized gas MRI ventilation enable visualization and quantification of regional lung ventilation; however, these techniques require specialized equipment and exogenous contrast, limiting clinical adoption. Physiologically-informed techniques to map proton (1H)-MRI ventilation have been proposed. These approaches have demonstrated moderate correlation with hyperpolarized gas MRI. Recently, deep learning (DL) has been used for image synthesis applications, including functional lung image synthesis. Here, we propose a 3D multi-channel convolutional neural network that employs physiologically-informed ventilation mapping and multi-inflation structural 1H-MRI to synthesize 3D ventilation surrogates (PhysVENeT). The dataset comprised paired inspiratory and expiratory 1H-MRI scans and corresponding hyperpolarized gas MRI scans from 170 participants with various pulmonary pathologies. We performed fivefold cross-validation on 150 of these participants and used 20 participants with a previously unseen pathology (post COVID-19) for external validation. Synthetic ventilation surrogates were evaluated using voxel-wise correlation and structural similarity metrics; the proposed PhysVENeT framework significantly outperformed conventional 1H-MRI ventilation mapping and other DL approaches which did not utilize structural imaging and ventilation mapping. PhysVENeT can accurately reflect ventilation defects and exhibits minimal overfitting on external validation data compared to DL approaches that do not integrate physiologically-informed mapping.

A hybrid model- and deep learning-based framework for functional lung image synthesis from multi-inflation CT and hyperpolarized gas MRI.

BACKGROUND: Hyperpolarized gas MRI is a functional lung imaging modality capable of visualizing regional lung ventilation with exceptional detail within a single breath. However, this modality requires specialized equipment and exogenous contrast, which limits widespread clinical adoption. CT ventilation imaging employs various metrics to model regional ventilation from non-contrast CT scans acquired at multiple inflation levels and has demonstrated moderate spatial correlation with hyperpolarized gas MRI. Recently, deep learning (DL)-based methods, utilizing convolutional neural networks (CNNs), have been leveraged for image synthesis applications. Hybrid approaches integrating computational modeling and data-driven methods have been utilized in cases where datasets are limited with the added benefit of maintaining physiological plausibility. PURPOSE: To develop and evaluate a multi-channel DL-based method that combines modeling and data-driven approaches to synthesize hyperpolarized gas MRI lung ventilation scans from multi-inflation, non-contrast CT and quantitatively compare these synthetic ventilation scans to conventional CT ventilation modeling. METHODS: In this study, we propose a hybrid DL configuration that integrates model- and data-driven methods to synthesize hyperpolarized gas MRI lung ventilation scans from a combination of non-contrast, multi-inflation CT and CT ventilation modeling. We used a diverse dataset comprising paired inspiratory and expiratory CT and helium-3 hyperpolarized gas MRI for 47 participants with a range of pulmonary pathologies. We performed six-fold cross-validation on the dataset and evaluated the spatial correlation between the synthetic ventilation and real hyperpolarized gas MRI scans; the proposed hybrid framework was compared to conventional CT ventilation modeling and other non-hybrid DL configurations. Synthetic ventilation scans were evaluated using voxel-wise evaluation metrics such as Spearman's correlation and mean square error (MSE), in addition to clinical biomarkers of lung function such as the ventilated lung percentage (VLP). Furthermore, regional localization of ventilated and defect lung regions was assessed via the Dice similarity coefficient (DSC). RESULTS: We showed that the proposed hybrid framework is capable of accurately replicating ventilation defects seen in the real hyperpolarized gas MRI scans, achieving a voxel-wise Spearman's correlation of 0.57 ± 0.17 and an MSE of 0.017 ± 0.01. The hybrid framework significantly outperformed CT ventilation modeling alone and all other DL configurations using Spearman's correlation. The proposed framework was capable of generating clinically relevant metrics such as the VLP without manual intervention, resulting in a Bland-Altman bias of 3.04%, significantly outperforming CT ventilation modeling. Relative to CT ventilation modeling, the hybrid framework yielded significantly more accurate delineations of ventilated and defect lung regions, achieving a DSC of 0.95 and 0.48 for ventilated and defect regions, respectively. CONCLUSION: The ability to generate realistic synthetic ventilation scans from CT has implications for several clinical applications, including functional lung avoidance radiotherapy and treatment response mapping. CT is an integral part of almost every clinical lung imaging workflow and hence is readily available for most patients; therefore, synthetic ventilation from non-contrast CT can provide patients with wider access to ventilation imaging worldwide.

Implementable Deep Learning for Multi-sequence Proton MRI Lung Segmentation: A Multi-center, Multi-vendor, and Multi-disease Study.

BACKGROUND: Recently, deep learning via convolutional neural networks (CNNs) has largely superseded conventional methods for proton (1 H)-MRI lung segmentation. However, previous deep learning studies have utilized single-center data and limited acquisition parameters. PURPOSE: Develop a generalizable CNN for lung segmentation in 1 H-MRI, robust to pathology, acquisition protocol, vendor, and center. STUDY TYPE: Retrospective. POPULATION: A total of 809 1 H-MRI scans from 258 participants with various pulmonary pathologies (median age (range): 57 (6-85); 42% females) and 31 healthy participants (median age (range): 34 (23-76); 34% females) that were split into training (593 scans (74%); 157 participants (55%)), testing (50 scans (6%); 50 participants (17%)) and external validation (164 scans (20%); 82 participants (28%)) sets. FIELD STRENGTH/SEQUENCE: 1.5-T and 3-T/3D spoiled-gradient recalled and ultrashort echo-time 1 H-MRI. ASSESSMENT: 2D and 3D CNNs, trained on single-center, multi-sequence data, and the conventional spatial fuzzy c-means (SFCM) method were compared to manually delineated expert segmentations. Each method was validated on external data originating from several centers. Dice similarity coefficient (DSC), average boundary Hausdorff distance (Average HD), and relative error (XOR) metrics to assess segmentation performance. STATISTICAL TESTS: Kruskal-Wallis tests assessed significances of differences between acquisitions in the testing set. Friedman tests with post hoc multiple comparisons assessed differences between the 2D CNN, 3D CNN, and SFCM. Bland-Altman analyses assessed agreement with manually derived lung volumes. A P value of <0.05 was considered statistically significant. RESULTS: The 3D CNN significantly outperformed its 2D analog and SFCM, yielding a median (range) DSC of 0.961 (0.880-0.987), Average HD of 1.63 mm (0.65-5.45) and XOR of 0.079 (0.025-0.240) on the testing set and a DSC of 0.973 (0.866-0.987), Average HD of 1.11 mm (0.47-8.13) and XOR of 0.054 (0.026-0.255) on external validation data. DATA CONCLUSION: The 3D CNN generated accurate 1 H-MRI lung segmentations on a heterogenous dataset, demonstrating robustness to disease pathology, sequence, vendor, and center. EVIDENCE LEVEL: 4. TECHNICAL EFFICACY: Stage 1.

A Dual-Channel Deep Learning Approach for Lung Cavity Estimation From Hyperpolarized Gas and Proton MRI.

BACKGROUND: Hyperpolarized gas MRI can quantify regional lung ventilation via biomarkers, including the ventilation defect percentage (VDP). VDP is computed from segmentations derived from spatially co-registered functional hyperpolarized gas and structural proton (1 H)-MRI. Although acquired at similar lung inflation levels, they are frequently misaligned, requiring a lung cavity estimation (LCE). Recently, single-channel, mono-modal deep learning (DL)-based methods have shown promise for pulmonary image segmentation problems. Multichannel, multimodal approaches may outperform single-channel alternatives. PURPOSE: We hypothesized that a DL-based dual-channel approach, leveraging both 1 H-MRI and Xenon-129-MRI (129 Xe-MRI), can generate LCEs more accurately than single-channel alternatives. STUDY TYPE: Retrospective. POPULATION: A total of 480 corresponding 1 H-MRI and 129 Xe-MRI scans from 26 healthy participants (median age [range]: 11 [8-71]; 50% females) and 289 patients with pulmonary pathologies (median age [range]: 47 [6-83]; 51% females) were split into training (422 scans [88%]; 257 participants [82%]) and testing (58 scans [12%]; 58 participants [18%]) sets. FIELD STRENGTH/SEQUENCE: 1.5-T, three-dimensional (3D) spoiled gradient-recalled 1 H-MRI and 3D steady-state free-precession 129 Xe-MRI. ASSESSMENT: We developed a multimodal DL approach, integrating 129 Xe-MRI and 1 H-MRI, in a dual-channel convolutional neural network. We compared this approach to single-channel alternatives using manually edited LCEs as a benchmark. We further assessed a fully automatic DL-based framework to calculate VDPs and compared it to manually generated VDPs. STATISTICAL TESTS: Friedman tests with post hoc Bonferroni correction for multiple comparisons compared single-channel and dual-channel DL approaches using Dice similarity coefficient (DSC), average boundary Hausdorff distance (average HD), and relative error (XOR) metrics. Bland-Altman analysis and paired t-tests compared manual and DL-generated VDPs. A P value < 0.05 was considered statistically significant. RESULTS: The dual-channel approach significantly outperformed single-channel approaches, achieving a median (range) DSC, average HD, and XOR of 0.967 (0.867-0.978), 1.68 mm (37.0-0.778), and 0.066 (0.246-0.045), respectively. DL-generated VDPs were statistically indistinguishable from manually generated VDPs (P = 0.710). DATA CONCLUSION: Our dual-channel approach generated LCEs, which could be integrated with ventilated lung segmentations to produce biomarkers such as the VDP without manual intervention. EVIDENCE LEVEL: 4. TECHNICAL EFFICACY: Stage 1.

Large-scale investigation of deep learning approaches for ventilated lung segmentation using multi-nuclear hyperpolarized gas MRI.

Respiratory diseases are leading causes of mortality and morbidity worldwide. Pulmonary imaging is an essential component of the diagnosis, treatment planning, monitoring, and treatment assessment of respiratory diseases. Insights into numerous pulmonary pathologies can be gleaned from functional lung MRI techniques. These include hyperpolarized gas ventilation MRI, which enables visualization and quantification of regional lung ventilation with high spatial resolution. Segmentation of the ventilated lung is required to calculate clinically relevant biomarkers. Recent research in deep learning (DL) has shown promising results for numerous segmentation problems. Here, we evaluate several 3D convolutional neural networks to segment ventilated lung regions on hyperpolarized gas MRI scans. The dataset consists of 759 helium-3 (3He) or xenon-129 (129Xe) volumetric scans and corresponding expert segmentations from 341 healthy subjects and patients with a wide range of pathologies. We evaluated segmentation performance for several DL experimental methods via overlap, distance and error metrics and compared them to conventional segmentation methods, namely, spatial fuzzy c-means (SFCM) and K-means clustering. We observed that training on combined 3He and 129Xe MRI scans using a 3D nn-UNet outperformed other DL methods, achieving a mean ± SD Dice coefficient of 0.963 ± 0.018, average boundary Hausdorff distance of 1.505 ± 0.969 mm, Hausdorff 95th percentile of 5.754 ± 6.621 mm and relative error of 0.075 ± 0.039. Moreover, limited differences in performance were observed between 129Xe and 3He scans in the testing set. Combined training on 129Xe and 3He yielded statistically significant improvements over the conventional methods (p < 0.0001). In addition, we observed very strong correlation and agreement between DL and expert segmentations, with Pearson correlation of 0.99 (p < 0.0001) and Bland-Altman bias of - 0.8%. The DL approach evaluated provides accurate, robust and rapid segmentations of ventilated lung regions and successfully excludes non-lung regions such as the airways and artefacts. This approach is expected to eliminate the need for, or significantly reduce, subsequent time-consuming manual editing.

Deep learning in structural and functional lung image analysis.

The recent resurgence of deep learning (DL) has dramatically influenced the medical imaging field. Medical image analysis applications have been at the forefront of DL research efforts applied to multiple diseases and organs, including those of the lungs. The aims of this review are twofold: (i) to briefly overview DL theory as it relates to lung image analysis; (ii) to systematically review the DL research literature relating to the lung image analysis applications of segmentation, reconstruction, registration and synthesis. The review was conducted following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. 479 studies were initially identified from the literature search with 82 studies meeting the eligibility criteria. Segmentation was the most common lung image analysis DL application (65.9% of papers reviewed). DL has shown impressive results when applied to segmentation of the whole lung and other pulmonary structures. DL has also shown great potential for applications in image registration, reconstruction and synthesis. However, the majority of published studies have been limited to structural lung imaging with only 12.9% of reviewed studies employing functional lung imaging modalities, thus highlighting significant opportunities for further research in this field. Although the field of DL in lung image analysis is rapidly expanding, concerns over inconsistent validation and evaluation strategies, intersite generalisability, transparency of methodological detail and interpretability need to be addressed before widespread adoption in clinical lung imaging workflow.