HUMAN-MACHINE COLLABORATION FOR MEDICAL IMAGE SEGMENTATION.

Image segmentation is a ubiquitous step in almost any medical image study. Deep learning-based approaches achieve state-of-the-art in the majority of image segmentation benchmarks. However, end-to-end training of such models requires sufficient annotation. In this paper, we propose a method based on conditional Generative Adversarial Network (cGAN) to address segmentation in semi-supervised setup and in a human-in-the-loop fashion. More specifically, we use the generator in the GAN to synthesize segmentations on unlabeled data and use the discriminator to identify unreliable slices for which expert annotation is required. The quantitative results on a conventional standard benchmark show that our method is comparable with the state-of-the-art fully supervised methods in slice-level evaluation, despite of requiring far less annotated data.

Analyzing the co-localization of substantia nigra hyper-echogenicities and iron accumulation in Parkinson's disease: A multi-modal atlas study with transcranial ultrasound and MRI.

BACKGROUND: Transcranial B-mode sonography (TCS) can detect hyperechogenic speckles in the area of the substantia nigra (SN) in Parkinson's disease (PD). These speckles correlate with iron accumulation in the SN tissue, but an exact volumetric localization in and around the SN is still unknown. Areas of increased iron content in brain tissue can be detected in vivo with magnetic resonance imaging, using quantitative susceptibility mapping (QSM). METHODS: In this work, we i) acquire, co-register and transform TCS and QSM imaging from a cohort of 23 PD patients and 27 healthy control subjects into a normalized atlas template space and ii) analyze and compare the 3D spatial distributions of iron accumulation in the midbrain, as detected by a signal increase (TCS+ and QSM+) in both modalities. RESULTS: We achieved sufficiently accurate intra-modal target registration errors (TRE<1 mm) for all MRI volumes and multi-modal TCS-MRI co-localization (TRE<4 mm) for 66.7% of TCS scans. In the caudal part of the midbrain, enlarged TCS+ and QSM+ areas were located within the SN pars compacta in PD patients in comparison to healthy controls. More cranially, overlapping TCS+ and QSM+ areas in PD subjects were found in the area of the ventral tegmental area (VTA). CONCLUSION: Our findings are concordant with several QSM-based studies on iron-related alterations in the area SN pars compacta. They substantiate that TCS+ is an indicator of iron accumulation in Parkinson's disease within and in the vicinity of the SN. Furthermore, they are in favor of an involvement of the VTA and thereby the mesolimbic system in Parkinson's disease.

Gaussian process uncertainty in age estimation as a measure of brain abnormality.

Multivariate regression models for age estimation are a powerful tool for assessing abnormal brain morphology associated to neuropathology. Age prediction models are built on cohorts of healthy subjects and are built to reflect normal aging patterns. The application of these multivariate models to diseased subjects usually results in high prediction errors, under the hypothesis that neuropathology presents a similar degenerative pattern as that of accelerated aging. In this work, we propose an alternative to the idea that pathology follows a similar trajectory than normal aging. Instead, we propose the use of metrics which measure deviations from the mean aging trajectory. We propose to measure these deviations using two different metrics: uncertainty in a Gaussian process regression model and a newly proposed age weighted uncertainty measure. Consequently, our approach assumes that pathologic brain patterns are different to those of normal aging. We present results for subjects with autism, mild cognitive impairment and Alzheimer's disease to highlight the versatility of the approach to different diseases and age ranges. We evaluate volume, thickness, and VBM features for quantifying brain morphology. Our evaluations are performed on a large number of images obtained from a variety of publicly available neuroimaging databases. Across all features, our uncertainty based measurements yield a better separation between diseased subjects and healthy individuals than the prediction error. Finally, we illustrate differences in the disease pattern to normal aging, supporting the application of uncertainty as a measure of neuropathology.

DeepNAT: Deep convolutional neural network for segmenting neuroanatomy.

We introduce DeepNAT, a 3D Deep convolutional neural network for the automatic segmentation of NeuroAnaTomy in T1-weighted magnetic resonance images. DeepNAT is an end-to-end learning-based approach to brain segmentation that jointly learns an abstract feature representation and a multi-class classification. We propose a 3D patch-based approach, where we do not only predict the center voxel of the patch but also neighbors, which is formulated as multi-task learning. To address a class imbalance problem, we arrange two networks hierarchically, where the first one separates foreground from background, and the second one identifies 25 brain structures on the foreground. Since patches lack spatial context, we augment them with coordinates. To this end, we introduce a novel intrinsic parameterization of the brain volume, formed by eigenfunctions of the Laplace-Beltrami operator. As network architecture, we use three convolutional layers with pooling, batch normalization, and non-linearities, followed by fully connected layers with dropout. The final segmentation is inferred from the probabilistic output of the network with a 3D fully connected conditional random field, which ensures label agreement between close voxels. The roughly 2.7million parameters in the network are learned with stochastic gradient descent. Our results show that DeepNAT compares favorably to state-of-the-art methods. Finally, the purely learning-based method may have a high potential for the adaptation to young, old, or diseased brains by fine-tuning the pre-trained network with a small training sample on the target application, where the availability of larger datasets with manual annotations may boost the overall segmentation accuracy in the future.

Three-dimensional sonographic examination of the midbrain for computer-aided diagnosis of movement disorders.

We present a novel approach to transcranial B-mode sonography for Parkinson's disease (PD) diagnosis by using 3-D ultrasound (3-DUS). We reconstructed bilateral 3-DUS volumes of the midbrain and substantia nigra echogenicities (SNE) and report results of a more objective abnormality detection in (PD). For classification, we analyzed volumetric measurements of midbrain and SNE in subjects with PD and healthy controls (HC). After blinded segmentation of these structures in 22/23 subjects (11 PD, 11 HC) and by two observers with varying prior experience in this technique, the classification algorithm yielded up to 91% sensitivity and 64% specificity using the larger volume of both SNE as a single-dimensional features and up to 90.9% sensitivity and 72.7% specificity using a multidimensional feature set with midbrain and both SNE volumes. This pilot study indicates that our TC-3-D-US approach is technically feasible and less dependent on the investigator's experience and good bone windows. Our pilot study yielded a fairly high sensitivity and specificity in differentiating between subjects with PD and HC.

Ultrasound confidence maps using random walks.

Advances in ultrasound system development have led to a substantial improvement of image quality and to an increased use of ultrasound in clinical practice. Nevertheless, ultrasound attenuation and shadowing artifacts cannot be entirely avoided and continue to challenge medical image computing algorithms. We introduce a method for estimating a per-pixel confidence in the information depicted by ultrasound images, referred to as an ultrasound confidence map, which emphasizes uncertainty in attenuated and/or shadow regions. Our main novelty is the modeling of the confidence estimation problem within a random walks framework by taking into account ultrasound specific constraints. The solution to the random walks equilibrium problem is global and takes the entire image content into account. As a result, our method is applicable to a variety of ultrasound image acquisition setups. We demonstrate the applicability of our confidence maps for ultrasound shadow detection, 3D freehand ultrasound reconstruction, and multi-modal image registration.

The 2D analytic signal for envelope detection and feature extraction on ultrasound images.

The fundamental property of the analytic signal is the split of identity, meaning the separation of qualitative and quantitative information in form of the local phase and the local amplitude, respectively. Especially the structural representation, independent of brightness and contrast, of the local phase is interesting for numerous image processing tasks. Recently, the extension of the analytic signal from 1D to 2D, covering also intrinsic 2D structures, was proposed. We show the advantages of this improved concept on ultrasound RF and B-mode images. Precisely, we use the 2D analytic signal for the envelope detection of RF data. This leads to advantages for the extraction of the information-bearing signal from the modulated carrier wave. We illustrate this, first, by visual assessment of the images, and second, by performing goodness-of-fit tests to a Nakagami distribution, indicating a clear improvement of statistical properties. The evaluation is performed for multiple window sizes and parameter estimation techniques. Finally, we show that the 2D analytic signal allows for an improved estimation of local features on B-mode images.

Locally adaptive Nakagami-based ultrasound similarity measures.

The derivation of statistically optimal similarity measures for intensity-based registration is possible by modeling the underlying image noise distribution. The parameters of these distributions are, however, commonly set heuristically across all images. In this article, we show that the estimation of the parameters on the present images largely improves the registration, which is a consequence of the more accurate characterization of the image noise. More precisely, instead of having constant parameters over the entire image domain, we estimate them on patches, leading to a local adaptation of the similarity measure. While this basic idea of creating locally adaptive metrics is interesting for various fields of application, we present the derivation for ultrasound imaging. The domain of ultrasound is particularly appealing for this approach, due to the inherent contamination with speckle noise. Furthermore, there exist detailed analyses of suitable noise distributions in the literature. We present experiments for applying a bivariate Nakagami distribution that facilitates modeling of several scattering scenarios prominent in medical ultrasound. Depending on the number of scatterers per resolution cell and the presence of coherent structures, different Nakagami parameters are required to obtain a valid approximation of the intensity statistics and to account for distributional locality. Our registration results on radio-frequency ultrasound data confirm the theoretical necessity for a spatial adaptation of similarity metrics.

Midbrain segmentation in transcranial 3D ultrasound for Parkinson diagnosis.

Ultrasound examination of the human brain through the temporal bone window, also called transcranial ultrasound (TC-US), is a completely non-invasive and cost-efficient technique, which has established itself for differential diagnosis of Parkinson's Disease (PD) in the past decade. The method requires spatial analysis of ultrasound hyperechogenicities produced by pathological changes within the Substantia Nigra (SN), which belongs to the basal ganglia within the midbrain. Related work on computer aided PD diagnosis shows the urgent need for an accurate and robust segmentation of the midbrain from 3D TC-US, which is an extremely difficult task due to poor image quality of TC-US. In contrast to 2D segmentations within earlier approaches, we develop the first method for semi-automatic midbrain segmentation from 3D TC-US and demonstrate its potential benefit on a database of 11 diagnosed Parkinson patients and 11 healthy controls.

The 2D analytic signal on RF and B-mode ultrasound images.

The fundamental property of the analytic signal is the split of identity, meaning the separation of quantitative and qualitative information in form of the local phase and the local amplitude, respectively. Especially the structural representation, independent of brightness and contrast, of the local phase is interesting for numerous image processing tasks. Recently, the extension of the analytic signal from 1D to 2D, covering also intrinsic 2D structures, was proposed. We show the advantages of this improved concept on ultrasound RF and B-mode images. Precisely, we use the 2D analytic signal for the envelope detection of RF data. This leads to advantages for the extraction of the information-bearing signal from the modulated carrier wave. We illustrate this, first, by visual assessment of the images, and second, by performing goodness-of-fit tests to a Nakagami distribution, indicating a clear improvement of statistical properties. Finally, we show that the 2D analytic signal allows for an improved estimation of local features on B-mode images.

Fiducial-free registration procedure for navigated bronchoscopy.

Navigated bronchoscopy has been developed by various groups within the last decades. Systems based on CT data and electromagnetic tracking enable the visualization of the position and orientation of the bronchoscope, forceps, and biopsy tools within CT data. Therefore registration between the tracking space and the CT volume is required. Standard procedures are based on point-based registration methods that require selecting corresponding natural landmarks in both coordinate systems by the examiner. We developed a novel algorithm for a fully automatic registration procedure in navigated bronchoscopy based on the trajectory recorded during routine examination of the airways at the beginning of an intervention. The proposed system provides advantages in terms of an unchanged medical workflow and high accuracy. We compared the novel method with point-based and ICP-based registration. Experiments demonstrate that the novel method transforms up to 97% of tracking points inside the segmented airways, which was the best performance compared to the other methods.