Machine learning-based augmented reality for improved surgical scene understanding.

In orthopedic and trauma surgery, AR technology can support surgeons in the challenging task of understanding the spatial relationships between the anatomy, the implants and their tools. In this context, we propose a novel augmented visualization of the surgical scene that mixes intelligently the different sources of information provided by a mobile C-arm combined with a Kinect RGB-Depth sensor. Therefore, we introduce a learning-based paradigm that aims at (1) identifying the relevant objects or anatomy in both Kinect and X-ray data, and (2) creating an object-specific pixel-wise alpha map that permits relevance-based fusion of the video and the X-ray images within one single view. In 12 simulated surgeries, we show very promising results aiming at providing for surgeons a better surgical scene understanding as well as an improved depth perception.

Learning from multiple experts with random forests: application to the segmentation of the midbrain in 3D ultrasound.

In the field of computer aided medical image analysis, it is often difficult to obtain reliable ground truth for evaluating algorithms or supervising statistical learning procedures. In this paper we present a new method for training a classification forest from images labelled by variably performing experts, while simultaneously evaluating the performance of each expert. Our approach builds upon state-of-the-art randomized classification forest techniques for medical image segmentation and recent methods for the fusion of multiple expert decisions. By incorporating the performance evaluation within the training phase, we obtain a novel forest framework for learning from conflicting expert decisions, accounting for both inter- and intra-expert variability. We demonstrate on a synthetic example that our method allows to retrieve the correct segmentation among other incorrectly labelled images, and we present an application to the automatic segmentation of the midbrain in 3D transcranial ultrasound images.

Three-dimensional sonographic examination of the midbrain for computer-aided diagnosis of movement disorders.

We present a novel approach to transcranial B-mode sonography for Parkinson's disease (PD) diagnosis by using 3-D ultrasound (3-DUS). We reconstructed bilateral 3-DUS volumes of the midbrain and substantia nigra echogenicities (SNE) and report results of a more objective abnormality detection in (PD). For classification, we analyzed volumetric measurements of midbrain and SNE in subjects with PD and healthy controls (HC). After blinded segmentation of these structures in 22/23 subjects (11 PD, 11 HC) and by two observers with varying prior experience in this technique, the classification algorithm yielded up to 91% sensitivity and 64% specificity using the larger volume of both SNE as a single-dimensional features and up to 90.9% sensitivity and 72.7% specificity using a multidimensional feature set with midbrain and both SNE volumes. This pilot study indicates that our TC-3-D-US approach is technically feasible and less dependent on the investigator's experience and good bone windows. Our pilot study yielded a fairly high sensitivity and specificity in differentiating between subjects with PD and HC.

Detection of substantia nigra echogenicities in 3D transcranial ultrasound for early diagnosis of Parkinson disease.

Parkinson's disease (PD) is a neurodegenerative movement disorder caused by decay of dopaminergic cells in the substantia nigra (SN), which are basal ganglia residing within the midbrain area. In the past two decades, transcranial B-mode sonography (TCUS) has emerged as a viable tool in differential diagnosis of PD and recently has been shown to have promising potential as a screening technique for early detection of PD, even before onset of motor symptoms. In TCUS imaging, the degeneration of SN cells becomes visible as bright and hyper-echogenic speckle patches (SNE) in the midbrain. Recent research proposes the usage of 3D ultrasound imaging in order to make the application of the TCUS technique easier and more objective. In this work, for the first time, we propose an automatic 3D SNE detection approach based on random forests, with a novel formulation of SNE probability that relies on visual context and anatomical priors. On a 3D-TCUS dataset of 11 PD patients and 11 healthy controls, we demonstrate that our SNE detection approach yields promising results with a sensitivity and specificity of around 83%.

Fast multiple organ detection and localization in whole-body MR dixon sequences.

Automatic localization of multiple anatomical structures in medical images provides important semantic information with potential benefits to diverse clinical applications. Aiming at organ-specific attenuation correction in PET/MR imaging, we propose an efficient approach for estimating location and size of multiple anatomical structures in MR scans. Our contribution is three-fold: (1) we apply supervised regression techniques to the problem of anatomy detection and localization in whole-body MR, (2) we adapt random ferns to produce multidimensional regression output and compare them with random regression forests, and (3) introduce the use of 3D LBP descriptors in multi-channel MR Dixon sequences. The localization accuracy achieved with both fern- and forest-based approaches is evaluated by direct comparison with state of the art atlas-based registration, on ground-truth data from 33 patients. Our results demonstrate improved anatomy localization accuracy with higher efficiency and robustness.

A machine learning approach for deformable guide-wire tracking in fluoroscopic sequences.

Deformable guide-wire tracking in fluoroscopic sequences is a challenging task due to the low signal to noise ratio of the images and the apparent complex motion of the object of interest. Common tracking methods are based on data terms that do not differentiate well between medical tools and anatomic background such as ribs and vertebrae. A data term learned directly from fluoroscopic sequences would be more adapted to the image characteristics and could help to improve tracking. In this work, our contribution is to learn the relationship between features extracted from the original image and the tracking error. By randomly deforming a guide-wire model around its ground truth position in one single reference frame, we explore the space spanned by these features. Therefore, a guide-wire motion distribution model is learned to reduce the intrisic dimensionality of this feature space. Random deformations and the corresponding features can be then automatically generated. In a regression approach, the function mapping this space to the tracking error is learned. The resulting data term is integrated into a tracking framework based on a second-order MAP-MRF formulation which is optimized by QPBO moves yielding high-quality tracking results. Experiments conducted on two fluoroscopic sequences show that our approach is a promising alternative for deformable tracking of guide-wires.