A Taxonomy of Uncertainty Events in Visual Analytics.

Visual analytics (VA) has become a standard tool to process and analyze data visually to generate novel insights. Unfortunately, each component can introduce uncertainty in the visual analytics process. These uncertainty events can originate from many effects and need to be differentiated. In this work, we propose a taxonomy of potential uncertainty events in the visual analytics cycle. Here, we structure the taxonomy along the components included in the visual analytics cycle. Based on this taxonomy, we provide a list of dependencies between these events. At last, we show how to use our taxonomy by providing a real-world example.

Stroke-GFCN: ischemic stroke lesion prediction with a fully convolutional graph network.

Purpose: The interpretation of image data plays a critical role during acute brain stroke diagnosis, and promptly defining the requirement of a surgical intervention will drastically impact the patient's outcome. However, determining stroke lesions purely from images can be a daunting task. Many studies proposed automatic segmentation methods for brain stroke lesions from medical images in different modalities, though heretofore results do not satisfy the requirements to be clinically reliable. We investigate the segmentation of brain stroke lesions using a geometric deep learning model that takes advantage of the intrinsic interconnected diffusion features in a set of multi-modal inputs consisting of computer tomography (CT) perfusion parameters. Approach: We propose a geometric deep learning model for the segmentation of ischemic stroke brain lesions that employs spline convolutions and unpooling/pooling operators on graphs to excerpt graph-structured features in a fully convolutional network architecture. In addition, we seek to understand the underlying principles governing the different components of our model. Accordingly, we structure the experiments in two parts: an evaluation of different architecture hyperparameters and a comparison with state-of-the-art methods. Results: The ablation study shows that deeper layers obtain a higher Dice coefficient score (DCS) of up to 0.3654. Comparing different pooling and unpooling methods shows that the best performing unpooling method is the proportional approach, yet it often smooths the segmentation border. Unpooling achieves segmentation results more adapted to the lesion boundary corroborated with systematic lower values of Hausdorff distance. The model performs at the level of state-of-the-art models without optimized training methods, such as augmentation or patches, with a DCS of 0.4553±0.0031. Conclusions: We proposed and evaluated an end-to-end trainable fully convolutional graph network architecture using spline convolutional layers for the ischemic stroke lesion prediction. We propose a model that employs graph-based operations to predict acute stroke brain lesions from CT perfusion parameters. Our results prove the feasibility of using geometric deep learning to solve segmentation problems, and our model shows a better performance than other models evaluated. The proposed model achieves improved metric values for the DCS metric, ranging from 8.61% to 69.05%, compared with other models trained under the same conditions. Next, we compare different pooling and unpooling operations in relation to their segmentation results, and we show that the model can produce segmentation outputs that adapt to irregular segmentation boundaries when using simple heuristic unpooling operations.

Tissue Outcome Prediction in Patients with Proximal Vessel Occlusion and Mechanical Thrombectomy Using Logistic Models.

Perfusion CT is established to aid selection of patients with proximal intracranial vessel occlusion for thrombectomy in the extended time window. Selection is mostly based on simple thresholding of perfusion parameter maps, which, however, does not exploit the full information hidden in the high-dimensional perfusion data. We implemented a multiparametric mass-univariate logistic model to predict tissue outcome based on data from 405 stroke patients with acute proximal vessel occlusion in the anterior circulation who underwent mechanical thrombectomy. Input parameters were acute multimodal CT imaging (perfusion, angiography, and non-contrast) as well as basic demographic and clinical parameters. The model was trained with the knowledge of recanalization status and final infarct localization. We found that perfusion parameter maps (CBF, CBV, and Tmax) were sufficient for tissue outcome prediction. Compared with single-parameter thresholding-based models, our logistic model had comparable volumetric accuracy, but was superior with respect to topographical accuracy (AUC of receiver operating characteristic). We also found higher spatial accuracy (Dice index) in an independent internal but not external cross-validation. Our results highlight the value of perfusion data compared with non-contrast CT, CT angiography and clinical information for tissue outcome-prediction. Multiparametric logistic prediction has high potential to outperform the single-parameter thresholding-based approach. In the future, the combination of tissue and functional outcome prediction might provide an individual biomarker for the benefit from mechanical thrombectomy in acute stroke care.

Ten Open Challenges in Medical Visualization.

The medical domain has been an inspiring application area in visualization research for many years already, but many open challenges remain. The driving forces of medical visualization research have been strengthened by novel developments, for example, in deep learning, the advent of affordable VR technology, and the need to provide medical visualizations for broader audiences. At IEEE VIS 2020, we hosted an Application Spotlight session to highlight recent medical visualization research topics. With this article, we provide the visualization community with ten such open challenges, primarily focused on challenges related to the visualization of medical imaging data. We first describe the unique nature of medical data in terms of data preparation, access, and standardization. Subsequently, we cover open visualization research challenges related to uncertainty, multimodal and multiscale approaches, and evaluation. Finally, we emphasize challenges related to users focusing on explainable AI, immersive visualization, P4 medicine, and narrative visualization.

Visualizing Multimodal Deep Learning for Lesion Prediction.

A U-Net is a type of convolutional neural network that has been shown to output impressive results in medical imaging segmentation tasks. Still, neural networks in general form a black box that is hard to interpret, especially by noncomputer scientists. This work provides a visual system that allows users to examine U-Nets that were trained to predict brain lesions caused by stroke using multimodal imaging. We provide several visualization views that allow users to load trained U-Nets, run them on different patient data, and examine the results while visually following the computation of the U-Net. With these visualizations, we can provide useful information for our medical collaborators showing how the training database can be improved and which features are best learned by the neural network.

Hierarchical Image Semantics Using Probabilistic Path Propagations for Biomedical Research.

Image segmentation is an important subtask in biomedical research applications, such as estimating the position and shape of a tumor. Unfortunately, advanced image segmentation methods are not widely applied in research applications as they often miss features, such as uncertainty communication, and may lack an intuitive approach for the use of the underlying algorithm. To solve this problem, this paper fuses a fuzzy and a hierarchical segmentation approach together, thus providing a flexible multiclass segmentation method based on probabilistic path propagations. By utilizing this method, analysts and physicians can map their mental model of image components and their composition to higher level objects. The probabilistic segmentation of higher order components is propagated along the user-defined hierarchy to highlight the potential of improvement resulting in each level of hierarchy by providing an intuitive representation. The effectiveness of this approach is demonstrated by evaluating our segmentations of biomedical datasets, comparing it to the state-of-the-art segmentation approaches, and an extensive user study.