Improving Vessel Segmentation with Multi-Task Learning and Auxiliary Data Available Only During Model Training.

Liver vessel segmentation in magnetic resonance imaging data is important for the computational analysis of vascular remodeling, associated with a wide spectrum of diffuse liver diseases. Existing approaches rely on contrast enhanced imaging data, but the necessary dedicated imaging sequences are not uniformly acquired. Images without contrast enhancement are acquired more frequently, but vessel segmentation is challenging, and requires large-scale annotated data. We propose a multi-task learning framework to segment vessels in liver MRI without contrast. It exploits auxiliary contrast enhanced MRI data available only during training to reduce the need for annotated training examples. Our approach draws on paired native and contrast enhanced data with and without vessel annotations for model training. Results show that auxiliary data improves the accuracy of vessel segmentation, even if they are not available during inference. The advantage is most pronounced if only few annotations are available for training, since the feature representation benefits from the shared task structure. A validation of this approach to augment a model for brain tumor segmentation confirms its benefits across different domains. An auxiliary informative imaging modality can augment expert annotations even if it is only available during training.

Correlation of histologic, imaging, and artificial intelligence features in NAFLD patients, derived from Gd-EOB-DTPA-enhanced MRI: a proof-of-concept study.

OBJECTIVE: To compare unsupervised deep clustering (UDC) to fat fraction (FF) and relative liver enhancement (RLE) on Gd-EOB-DTPA-enhanced MRI to distinguish simple steatosis from non-alcoholic steatohepatitis (NASH), using histology as the gold standard. MATERIALS AND METHODS: A derivation group of 46 non-alcoholic fatty liver disease (NAFLD) patients underwent 3-T MRI. Histology assessed steatosis, inflammation, ballooning, and fibrosis. UDC was trained to group different texture patterns from MR data into 10 distinct clusters per sequence on unenhanced T1- and Gd-EOB-DTPA-enhanced T1-weighted hepatobiliary phase (T1-Gd-EOB-DTPA-HBP), then on T1 in- and opposed-phase images. RLE and FF were quantified on identical sequences. Differences of these parameters between NASH and simple steatosis were evaluated with χ2- and t-tests, respectively. Linear regression and Random Forest classifier were performed to identify associations between histological NAFLD features, RLE, FF, and UDC patterns, and then determine predictors able to distinguish simple steatosis from NASH. ROC curves assessed diagnostic performance of UDC, RLE, and FF. Finally, we tested these parameters on 30 validation cohorts. RESULTS: For the derivation group, UDC-derived features from unenhanced and T1-Gd-EOB-DTPA-HBP, plus from T1 in- and opposed-phase, distinguished NASH from simple steatosis (p ≤ 0.001 and p = 0.02, respectively) with 85% and 80% accuracy, respectively, while RLE and FF distinguished NASH from simple steatosis (p ≤ 0.001 and p = 0.004, respectively), with 83% and 78% accuracy, respectively. On multivariate regression analysis, RLE and FF correlated only with fibrosis (p = 0.040) and steatosis (p ≤ 0.001), respectively. Conversely, UDC features, using Random Forest classifier predictors, correlated with all histologic NAFLD components. The validation group confirmed these results for both approaches. CONCLUSION: UDC, RLE, and FF could independently separate NASH from simple steatosis. UDC may predict all histologic NAFLD components. CLINICAL RELEVANCE STATEMENT: Using gadoxetic acid-enhanced MR, fat fraction (FF > 5%) can diagnose NAFLD, and relative liver enhancement can distinguish NASH from simple steatosis. Adding AI may let us non-invasively estimate the histologic components, i.e., fat, ballooning, inflammation, and fibrosis, the latter the main prognosticator. KEY POINTS: • Unsupervised deep clustering (UDC) and MR-based parameters (FF and RLE) could independently distinguish simple steatosis from NASH in the derivation group. • On multivariate analysis, RLE could predict only fibrosis, and FF could predict only steatosis; however, UDC could predict all histologic NAFLD components in the derivation group. • The validation cohort confirmed the findings for the derivation group.

Dynamic memory to alleviate catastrophic forgetting in continual learning with medical imaging.

Medical imaging is a central part of clinical diagnosis and treatment guidance. Machine learning has increasingly gained relevance because it captures features of disease and treatment response that are relevant for therapeutic decision-making. In clinical practice, the continuous progress of image acquisition technology or diagnostic procedures, the diversity of scanners, and evolving imaging protocols hamper the utility of machine learning, as prediction accuracy on new data deteriorates, or models become outdated due to these domain shifts. We propose a continual learning approach to deal with such domain shifts occurring at unknown time points. We adapt models to emerging variations in a continuous data stream while counteracting catastrophic forgetting. A dynamic memory enables rehearsal on a subset of diverse training data to mitigate forgetting while enabling models to expand to new domains. The technique balances memory by detecting pseudo-domains, representing different style clusters within the data stream. Evaluation of two different tasks, cardiac segmentation in magnetic resonance imaging and lung nodule detection in computed tomography, demonstrate a consistent advantage of the method.

CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation.

Segmentation of abdominal organs has been a comprehensive, yet unresolved, research field for many years. In the last decade, intensive developments in deep learning (DL) introduced new state-of-the-art segmentation systems. Despite outperforming the overall accuracy of existing systems, the effects of DL model properties and parameters on the performance are hard to interpret. This makes comparative analysis a necessary tool towards interpretable studies and systems. Moreover, the performance of DL for emerging learning approaches such as cross-modality and multi-modal semantic segmentation tasks has been rarely discussed. In order to expand the knowledge on these topics, the CHAOS - Combined (CT-MR) Healthy Abdominal Organ Segmentation challenge was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI), 2019, in Venice, Italy. Abdominal organ segmentation from routine acquisitions plays an important role in several clinical applications, such as pre-surgical planning or morphological and volumetric follow-ups for various diseases. These applications require a certain level of performance on a diverse set of metrics such as maximum symmetric surface distance (MSSD) to determine surgical error-margin or overlap errors for tracking size and shape differences. Previous abdomen related challenges are mainly focused on tumor/lesion detection and/or classification with a single modality. Conversely, CHAOS provides both abdominal CT and MR data from healthy subjects for single and multiple abdominal organ segmentation. Five different but complementary tasks were designed to analyze the capabilities of participating approaches from multiple perspectives. The results were investigated thoroughly, compared with manual annotations and interactive methods. The analysis shows that the performance of DL models for single modality (CT / MR) can show reliable volumetric analysis performance (DICE: 0.98 ± 0.00 / 0.95 ± 0.01), but the best MSSD performance remains limited (21.89 ± 13.94 / 20.85 ± 10.63 mm). The performances of participating models decrease dramatically for cross-modality tasks both for the liver (DICE: 0.88 ± 0.15 MSSD: 36.33 ± 21.97 mm). Despite contrary examples on different applications, multi-tasking DL models designed to segment all organs are observed to perform worse compared to organ-specific ones (performance drop around 5%). Nevertheless, some of the successful models show better performance with their multi-organ versions. We conclude that the exploration of those pros and cons in both single vs multi-organ and cross-modality segmentations is poised to have an impact on further research for developing effective algorithms that would support real-world clinical applications. Finally, having more than 1500 participants and receiving more than 550 submissions, another important contribution of this study is the analysis on shortcomings of challenge organizations such as the effects of multiple submissions and peeking phenomenon.

[Machine learning in radiology : Terminology from individual timepoint to trajectory]. / Maschinelles Lernen in der Radiologie : Begriffsbestimmung vom Einzelzeitpunkt bis zur Trajektorie.

METHODICAL ISSUE: Machine learning (ML) algorithms have an increasingly relevant role in radiology tackling tasks such as the automatic detection and segmentation of diagnosis-relevant markers, the quantification of progression and response, and their prediction in individual patients. STANDARD RADIOLOGICAL METHODS: ML algorithms are relevant for all image acquisition techniques from computed tomography (CT) and magnetic resonance imaging (MRI) to ultrasound. However, different modalities result in different challenges with respect to standardization and variability. METHODICAL INNOVATIONS: ML algorithms are increasingly able to analyze longitudinal data for the training of prediction models. This is relevant since it enables the use of comprehensive information for predicting individual progression and response, and the associated support of treatment decisions by ML models. PERFORMANCE: The quality of detection and segmentation algorithms of lesions has reached an acceptable level in several areas. The accuracy of prediction models is still increasing, but is dependent on the availability of representative training data. ACHIEVEMENTS: The development of ML algorithms in radiology is progressing although many solutions are still at a validation stage. It is accompanied by a parallel and increasingly interlinked development of basic methods and techniques which will gradually be put into practice in radiology. PRACTICAL CONSIDERATIONS: Two factors will impact the relevance of ML in radiological practice: the thorough validation of algorithms and solutions, and the creation of representative diverse data for the training and validation in a realistic context.