Grouping Boundary Proposals for Fast Interactive Image Segmentation.

Geodesic models are known as an efficient tool for solving various image segmentation problems. Most of existing approaches only exploit local pointwise image features to track geodesic paths for delineating the objective boundaries. However, such a segmentation strategy cannot take into account the connectivity of the image edge features, increasing the risk of shortcut problem, especially in the case of complicated scenario. In this work, we introduce a new image segmentation model based on the minimal geodesic framework in conjunction with an adaptive cut-based circular optimal path computation scheme and a graph-based boundary proposals grouping scheme. Specifically, the adaptive cut can disconnect the image domain such that the target contours are imposed to pass through this cut only once. The boundary proposals are comprised of precomputed image edge segments, providing the connectivity information for our segmentation model. These boundary proposals are then incorporated into the proposed image segmentation model, such that the target segmentation contours are made up of a set of selected boundary proposals and the corresponding geodesic paths linking them. Experimental results show that the proposed model indeed outperforms state-of-the-art minimal paths-based image segmentation approaches.

Computing geodesic paths encoding a curvature prior for curvilinear structure tracking.

In this paper, we introduce an efficient method for computing curves minimizing a variant of the Euler-Mumford elastica energy, with fixed endpoints and tangents at these endpoints, where the bending energy is enhanced with a user-defined and data-driven scalar-valued term referred to as the curvature prior. In order to guarantee that the globally optimal curve is extracted, the proposed method involves the numerical computation of the viscosity solution to a specific static Hamilton-Jacobi-Bellman (HJB) partial differential equation (PDE). For that purpose, we derive the explicit Hamiltonian associated with this variant model equipped with a curvature prior, discretize the resulting HJB PDE using an adaptive finite difference scheme, and solve it in a single pass using a generalized fast-marching method. In addition, we also present a practical method for estimating the curvature prior values from image data, designed for the task of accurately tracking curvilinear structure centerlines. Numerical experiments on synthetic and real-image data illustrate the advantages of the considered variant of the elastica model with a prior curvature enhancement in complex scenarios where challenging geometric structures appear.

Geodesic Models With Convexity Shape Prior.

The minimal geodesic models established upon the eikonal equation framework are capable of finding suitable solutions in various image segmentation scenarios. Existing geodesic-based segmentation approaches usually exploit image features in conjunction with geometric regularization terms, such as euclidean curve length or curvature-penalized length, for computing geodesic curves. In this paper, we take into account a more complicated problem: finding curvature-penalized geodesic paths with a convexity shape prior. We establish new geodesic models relying on the strategy of orientation-lifting, by which a planar curve can be mapped to an high-dimensional orientation-dependent space. The convexity shape prior serves as a constraint for the construction of local geodesic metrics encoding a particular curvature constraint. Then the geodesic distances and the corresponding closed geodesic paths in the orientation-lifted space can be efficiently computed through state-of-the-art Hamiltonian fast marching method. In addition, we apply the proposed geodesic models to the active contours, leading to efficient interactive image segmentation algorithms that preserve the advantages of convexity shape prior and curvature penalization.

Trajectory Grouping With Curvature Regularization for Tubular Structure Tracking.

Tubular structure tracking is a crucial task in the fields of computer vision and medical image analysis. The minimal paths-based approaches have exhibited their strong ability in tracing tubular structures, by which a tubular structure can be naturally modeled as a minimal geodesic path computed with a suitable geodesic metric. However, existing minimal paths-based tracing approaches still suffer from difficulties such as the shortcuts and short branches combination problems, especially when dealing with the images involving complicated tubular tree structures or background. In this paper, we introduce a new minimal paths-based model for minimally interactive tubular structure centerline extraction in conjunction with a perceptual grouping scheme. Basically, we take into account the prescribed tubular trajectories and curvature-penalized geodesic paths to seek suitable shortest paths. The proposed approach can benefit from the local smoothness prior on tubular structures and the global optimality of the used graph-based path searching scheme. Experimental results on both synthetic and real images prove that the proposed model indeed obtains outperformance comparing with the state-of-the-art minimal paths-based tubular structure tracing algorithms.

Geodesic Paths for Image Segmentation With Implicit Region-Based Homogeneity Enhancement.

Minimal paths are regarded as a powerful and efficient tool for boundary detection and image segmentation due to its global optimality and the well-established numerical solutions such as fast marching method. In this paper, we introduce a flexible interactive image segmentation model based on the Eikonal partial differential equation (PDE) framework in conjunction with region-based homogeneity enhancement. A key ingredient in the introduced model is the construction of local geodesic metrics, which are capable of integrating anisotropic and asymmetric edge features, implicit region-based homogeneity features and/or curvature regularization. The incorporation of the region-based homogeneity features into the metrics considered relies on an implicit representation of these features, which is one of the contributions of this work. Moreover, we also introduce a way to build simple closed contours as the concatenation of two disjoint open curves. Experimental results prove that the proposed model indeed outperforms state-of-the-art minimal paths-based image segmentation approaches.

A Generalized Asymmetric Dual-Front Model for Active Contours and Image Segmentation.

The Voronoi diagram-based dual-front scheme is known as a powerful and efficient technique for addressing the image segmentation and domain partitioning problems. In the basic formulation of existing dual-front approaches, the evolving contour can be considered as the interfaces of adjacent Voronoi regions. Among these dual-front models, a crucial ingredient is regarded as the geodesic metrics by which the geodesic distances and the corresponding Voronoi diagram can be estimated. In this paper, we introduce a new dual-front model based on asymmetric quadratic metrics. These metrics considered are built by the integration of the image features and a vector field derived from the evolving contour. The use of the asymmetry enhancement can reduce the risk for the segmentation contours being stuck at false positions, especially when the initial curves are far away from the target boundaries or the images have complicated intensity distributions. Moreover, the proposed dual-front model can be applied for image segmentation in conjunction with various region-based homogeneity terms. The numerical experiments on both synthetic and real images show that the proposed dual-front model indeed achieves encouraging results.

Minimal Paths for Tubular Structure Segmentation With Coherence Penalty and Adaptive Anisotropy.

The minimal path method has proven to be particularly useful and efficient in tubular structure segmentation applications. In this paper, we propose a new minimal path model associated with a dynamic Riemannian metric embedded with an appearance feature coherence penalty and an adaptive anisotropy enhancement term. The features that characterize the appearance and anisotropy properties of a tubular structure are extracted through the associated orientation score. The proposed the dynamic Riemannian metric is updated in the course of the geodesic distance computation carried out by the efficient single-pass fast marching method. Compared to the state-of-the-art minimal path models, the proposed minimal path model is able to extract the desired tubular structures from a complicated vessel tree structure. In addition, we propose an efficient prior path-based method to search for vessel radius value at each centerline position of the target. Finally, we perform the numerical experiments on both synthetic and real images. The quantitive validation is carried out on retinal vessel images. The results indicate that the proposed model indeed achieves a promising performance.

Interactive retinal vessel centreline extraction and boundary delineation using anisotropic fast marching and intensities consistency.

In this paper, we propose a new interactive retinal vessels extraction method with anisotropic fast marching (AFM) based on the observation that one vessel may have the property of local intensities consistency. Our goal is to extract both the centrelines and boundaries between two given points. The proposed method consists of two stages: the first stage aims to finding the vessel centrelines using AFM and local intensities consistency roughly, while the second stage is to refine the centrelines from the previous stage using constrained Riemannian metric based AFM, and get the boundaries of the vessels simultaneously. Experiments show that results of our method outperform the classical minimal path method [1].

Tagged template deformation.

Model-based approaches are very popular for medical image segmentation as they carry useful prior information on the target structure. Among them, the implicit template deformation framework recently bridged the gap between the efficiency and flexibility of level-set region competition and the robustness of atlas deformation approaches. This paper generalizes this method by introducing the notion of tagged templates. A tagged template is an implicit model in which different subregions are defined. In each of these subregions, specific image features can be used with various confidence levels. The tags can be either set manually or automatically learnt via a process also hereby described. This generalization therefore greatly widens the scope of potential clinical application of implicit template deformation while maintaining its appealing algorithmic efficiency. We show the great potential of our approach in myocardium segmentation of ultrasound images.

Longitudinal deformation models, spatial regularizations and learning strategies to quantify Alzheimer's disease progression.

In the context of Alzheimer's disease, two challenging issues are (1) the characterization of local hippocampal shape changes specific to disease progression and (2) the identification of mild-cognitive impairment patients likely to convert. In the literature, (1) is usually solved first to detect areas potentially related to the disease. These areas are then considered as an input to solve (2). As an alternative to this sequential strategy, we investigate the use of a classification model using logistic regression to address both issues (1) and (2) simultaneously. The classification of the patients therefore does not require any a priori definition of the most representative hippocampal areas potentially related to the disease, as they are automatically detected. We first quantify deformations of patients' hippocampi between two time points using the large deformations by diffeomorphisms framework and transport these deformations to a common template. Since the deformations are expected to be spatially structured, we perform classification combining logistic loss and spatial regularization techniques, which have not been explored so far in this context, as far as we know. The main contribution of this paper is the comparison of regularization techniques enforcing the coefficient maps to be spatially smooth (Sobolev), piecewise constant (total variation) or sparse (fused LASSO) with standard regularization techniques which do not take into account the spatial structure (LASSO, ridge and ElasticNet). On a dataset of 103 patients out of ADNI, the techniques using spatial regularizations lead to the best classification rates. They also find coherent areas related to the disease progression.

Efficient brain lesion segmentation using multi-modality tissue-based feature selection and support vector machines.

Support vector machines (SVM) are machine learning techniques that have been used for segmentation and classification of medical images, including segmentation of white matter hyper-intensities (WMH). Current approaches using SVM for WMH segmentation extract features from the brain and classify these followed by complex post-processing steps to remove false positives. The method presented in this paper combines advanced pre-processing, tissue-based feature selection and SVM classification to obtain efficient and accurate WMH segmentation. Features from 125 patients, generated from up to four MR modalities [T1-w, T2-w, proton-density and fluid attenuated inversion recovery(FLAIR)], differing neighbourhood sizes and the use of multi-scale features were compared. We found that although using all four modalities gave the best overall classification (average Dice scores of 0.54 ± 0.12, 0.72 ± 0.06 and 0.82 ± 0.06 respectively for small, moderate and severe lesion loads); this was not significantly different (p = 0.50) from using just T1-w and FLAIR sequences (Dice scores of 0.52 ± 0.13, 0.71 ± 0.08 and 0.81 ± 0.07). Furthermore, there was a negligible difference between using 5 × 5 × 5 and 3 × 3 × 3 features (p = 0.93). Finally, we show that careful consideration of features and pre-processing techniques not only saves storage space and computation time but also leads to more efficient classification, which outperforms the one based on all features with post-processing.

Registration of free-breathing 3D+t abdominal perfusion CT images via co-segmentation.

Dynamic contrast-enhanced computed tomography (DCE-CT) is a valuable imaging modality to assess tissues properties, particularly in tumours, by estimating pharmacokinetic parameters from the evolution of pixels intensities in 3D+t acquisitions. However, this requires a registration of the whole sequence of volumes, which is challenging especially when the patient breathes freely. In this paper, we propose a generic, fast and automatic method to address this problem. As standard iconic registration methods are not robust to contrast intake, we rather rely on the segmentation of the organ of interest. This segmentation is performed jointly with the registration of the sequence within a novel co-segmentation framework. Our approach is based on implicit template deformation, that we extend to a co-segmentation algorithm which provides as outputs both a segmentation of the organ of interest in every image and stabilising transformations for the whole sequence. The proposed method is validated on 15 datasets acquired from patients with renal lesions and shows improvement in terms of registration and estimation of pharmacokinetic parameters over the state-of-the-art method.

Incorporating shape variability in image segmentation via implicit template deformation.

Implicit template deformation is a model-based segmentation framework that was successfully applied in several medical applications. In this paper, we propose a method to learn and use prior knowledge on shape variability in such framework. This shape prior is learnt via an original and dedicated process in which both an optimal template and principal modes of variations are estimated from a collection of shapes. This learning strategy requires neither a pre-alignment of the training shapes nor one-to-one correspondences between shape sample points. We then generalize the implicit template deformation formulation to automatically select the most plausible deformation as a shape prior. This novel framework maintains the two main properties of implicit template deformation: topology preservation and computational efficiency. Our approach can be applied to any organ with a possibly complex shape but fixed topology. We validate our method on myocardium segmentation from cardiac magnetic resonance short-axis images and demonstrate segmentation improvement over standard template deformation.

Joint co-segmentation and registration of 3D ultrasound images.

Contrast-enhanced ultrasound (CEUS) allows a visualization of the vascularization and complements the anatomical information provided by conventional ultrasound (US). However, these images are inherently subject to noise and shadows, which hinders standard segmentation algorithms. In this paper, we propose to use simultaneously the different information coming from 3D US and CEUS images to address the problem of kidney segmentation. To that end, we introduce a generic framework for joint co-segmentation and registration that seeks objects having the same shape in several images. From this framework, we derive both an ellipsoid co-detection and a model-based co-segmentation algorithm. These methods rely on voxel-classification maps that we estimate using random forests in a structured way. This yields a fast and fully automated pipeline, in which an ellipsoid is first estimated to locate the kidney in both US and CEUS volumes and then deformed to segment it accurately. The proposed method outperforms state-of-the-art results (by dividing the kidney volume error by two) on a clinically representative database of 64 images.

Automatic detection and segmentation of kidneys in 3D CT images using random forests.

Kidney segmentation in 3D CT images allows extracting useful information for nephrologists. For practical use in clinical routine, such an algorithm should be fast, automatic and robust to contrast-agent enhancement and fields of view. By combining and refining state-of-the-art techniques (random forests and template deformation), we demonstrate the possibility of building an algorithm that meets these requirements. Kidneys are localized with random forests following a coarse-to-fine strategy. Their initial positions detected with global contextual information are refined with a cascade of local regression forests. A classification forest is then used to obtain a probabilistic segmentation of both kidneys. The final segmentation is performed with an implicit template deformation algorithm driven by these kidney probability maps. Our method has been validated on a highly heterogeneous database of 233 CT scans from 89 patients. 80% of the kidneys were accurately detected and segmented (Dice coefficient > 0.90) in a few seconds per volume.

Contour detection and completion for inpainting and segmentation based on topological gradient and fast marching algorithms.

We combine in this paper the topological gradient, which is a powerful method for edge detection in image processing, and a variant of the minimal path method in order to find connected contours. The topological gradient provides a more global analysis of the image than the standard gradient and identifies the main edges of an image. Several image processing problems (e.g., inpainting and segmentation) require continuous contours. For this purpose, we consider the fast marching algorithm in order to find minimal paths in the topological gradient image. This coupled algorithm quickly provides accurate and connected contours. We present then two numerical applications, to image inpainting and segmentation, of this hybrid algorithm.

Segmentation of 3D tubular objects with adaptive front propagation and minimal tree extraction for 3D medical imaging.

We present a new fast approach for segmentation of thin branching structures, like vascular trees, based on Fast-Marching (FM) and Level Set (LS) methods. FM allows segmentation of tubular structures by inflating a "long balloon" from a user given single point. However, when the tubular shape is rather long, the front propagation may blow up through the boundary of the desired shape close to the starting point. Our contribution is focused on a method to propagate only the useful part of the front while freezing the rest of it. We demonstrate its ability to segment quickly and accurately tubular and tree-like structures. We also develop a useful stopping criterion for the causal front propagation. We finally derive an efficient algorithm for extracting an underlying 1D skeleton of the branching objects, with minimal path techniques. Each branch being represented by its centerline, we automatically detect the bifurcations, leading to the "Minimal Tree" representation. This so-called "Minimal Tree" is very useful for visualization and quantification of the pathologies in our anatomical data sets. We illustrate our algorithms by applying it to several arteries datasets.

3D Brain Segmentation Using Dual-Front Active Contours with Optional User Interaction.

Important attributes of 3D brain cortex segmentation algorithms include robustness, accuracy, computational efficiency, and facilitation of user interaction, yet few algorithms incorporate all of these traits. Manual segmentation is highly accurate but tedious and laborious. Most automatic techniques, while less demanding on the user, are much less accurate. It would be useful to employ a fast automatic segmentation procedure to do most of the work but still allow an expert user to interactively guide the segmentation to ensure an accurate final result. We propose a novel 3D brain cortex segmentation procedure utilizing dual-front active contours which minimize image-based energies in a manner that yields flexibly global minimizers based on active regions. Region-based information and boundary-based information may be combined flexibly in the evolution potentials for accurate segmentation results. The resulting scheme is not only more robust but much faster and allows the user to guide the final segmentation through simple mouse clicks which add extra seed points. Due to the flexibly global nature of the dual-front evolution model, single mouse clicks yield corrections to the segmentation that extend far beyond their initial locations, thus minimizing the user effort. Results on 15 simulated and 20 real 3D brain images demonstrate the robustness, accuracy, and speed of our scheme compared with other methods.

Single quantum dot tracking based on perceptual grouping using minimal paths in a spatiotemporal volume.

Semiconductor quantum dots (QDs) are new fluorescent probes with great promise for ultrasensitive biological imaging. When detected at the single-molecule level, QD-tagged molecules can be observed and tracked in the membrane of live cells over unprecedented durations. The motion of these individual molecules, recorded in sequences of fluorescence images, can reveal aspects of the dynamics of cellular processes that remain hidden in conventional ensemble imaging. Due to QD complex optical properties, such as fluorescence intermittency, the quantitative analysis of these sequences is, however, challenging and requires advanced algorithms. We present here a novel approach, which, instead of a frame by frame analysis, is based on perceptual grouping in a spatiotemporal volume. By applying a detection process based on an image fluorescence model, we first obtain an unstructured set of points. Individual molecular trajectories are then considered as minimal paths in a Riemannian metric derived from the fluorescence image stack. These paths are computed with a variant of the fast marching method and few parameters are required. We demonstrate the ability of our algorithm to track intermittent objects both in sequences of synthetic data and in experimental measurements obtained with individual QD-tagged receptors in the membrane of live neurons. While developed for tracking QDs, this method can, however, be used with any fluorescent probes.