On the Importance of Domain Awareness in Classifier Interpretations in Medical Imaging.

Translating the success of deep learning-based computer-assisted classification into clinical adaptation hinges on the ability to explain a prediction's causality. Post-hoc interpretability approaches, especially counterfactual techniques, have shown both technical and psychological potential. Nevertheless, currently dominant approaches utilize heuristic, unvalidated methodology. Thereby, they potentially operate the underlying networks outside their validated domain, adding doubt in the predictor's abilities instead of generating knowledge and trust. In this work, we investigate this out-of-distribution problem for medical image pathology classifiers and propose marginalization techniques and evaluation procedures to overcome it. Furthermore, we propose a complete domain-aware pipeline for radiology environments. Its validity is demonstrated on a synthetic and two publicly available image datasets. Specifically, we evaluate using the CBIS-DDSM/DDSM mammography collection and the Chest X-ray14 radiographs. Our solution shows, both quantitatively and qualitatively, a significant reduction of localization ambiguity and clearer conveying results.

Employing similarity to highlight differences: On the impact of anatomical assumptions in chest X-ray registration methods.

To facilitate both the detection and the interpretation of findings in chest X-rays, comparison with a previous image of the same patient is very valuable to radiologists. Today, the most common approach for deep learning methods to automatically inspect chest X-rays disregards the patient history and classifies only single images as normal or abnormal. Nevertheless, several methods for assisting in the task of comparison through image registration have been proposed in the past. However, as we illustrate, they tend to miss specific types of pathological changes like cardiomegaly and effusion. Due to assumptions on fixed anatomical structures or their measurements of registration quality, they produce unnaturally deformed warp fields impacting visualization of differences between moving and fixed images. We aim to overcome these limitations, through a new paradigm based on individual rib pair segmentation for anatomy penalized registration. Our method proves to be a natural way to limit the folding percentage of the warp field to 1/6 of the state of the art while increasing the overlap of ribs by more than 25%, implying difference images showing pathological changes overlooked by other methods. We develop an anatomically penalized convolutional multi-stage solution on the National Institutes of Health (NIH) data set, starting from less than 25 fully and 50 partly labeled training images, employing sequential instance memory segmentation with hole dropout, weak labeling, coarse-to-fine refinement and Gaussian mixture model histogram matching. We statistically evaluate the benefits of our method and highlight the limits of currently used metrics for registration of chest X-rays.

Multi-Task Fusion for Improving Mammography Screening Data Classification.

Machine learning and deep learning methods have become essential for computer-assisted prediction in medicine, with a growing number of applications also in the field of mammography. Typically these algorithms are trained for a specific task, e.g., the classification of lesions or the prediction of a mammogram's pathology status. To obtain a comprehensive view of a patient, models which were all trained for the same task(s) are subsequently ensembled or combined. In this work, we propose a pipeline approach, where we first train a set of individual, task-specific models and subsequently investigate the fusion thereof, which is in contrast to the standard model ensembling strategy. We fuse model predictions and high-level features from deep learning models with hybrid patient models to build stronger predictors on patient level. To this end, we propose a multi-branch deep learning model which efficiently fuses features across different tasks and mammograms to obtain a comprehensive patient-level prediction. We train and evaluate our full pipeline on public mammography data, i.e., DDSM and its curated version CBIS-DDSM, and report an AUC score of 0.962 for predicting the presence of any lesion and 0.791 for predicting the presence of malignant lesions on patient level. Overall, our fusion approaches improve AUC scores significantly by up to 0.04 compared to standard model ensembling. Moreover, by providing not only global patient-level predictions but also task-specific model results that are related to radiological features, our pipeline aims to closely support the reading workflow of radiologists.