Unsupervised registration of 3D knee implant components to biplanar X-ray images.

BACKGROUND: Registration of three-dimensional (3D) knee implant components to radiographic images provides the 3D position of the implants which aids to analyze the component alignment after total knee arthroplasty. METHODS: We present an automatic 3D to two-dimensional (2D) registration using biplanar radiographic images based on a hybrid similarity measure integrating region and edge-based information. More precisely, this measure is herein defined as a weighted combination of an edge potential field-based similarity, which represents the relation between the external contours of the component projections and an edge potential field estimated on the two radiographic images, and an object specificity property, which is based on the distinction of the region-label inside and outside of the object. RESULTS: The accuracy of our 3D/2D registration algorithm was assessed on a sample of 64 components (32 femoral components and 32 tibial components). In our tests, we obtained an average of the root mean square error (RMSE) of 0.18 mm, which is significantly lower than that of both single similarity methods, supporting our hypothesis of better stability and accuracy with the proposed approach. CONCLUSION: Our method, which provides six accurate registration parameters (three rotations and three translations) without requiring any fiducial markers, makes it possible to perform the important analyses on the rotational alignment of the femoral and tibial components on a large number of cases. In addition, this method can be extended to register other implants or bones.

CoSOV1Net: A Cone- and Spatial-Opponent Primary Visual Cortex-Inspired Neural Network for Lightweight Salient Object Detection.

Salient object-detection models attempt to mimic the human visual system's ability to select relevant objects in images. To this end, the development of deep neural networks on high-end computers has recently achieved high performance. However, developing deep neural network models with the same performance for resource-limited vision sensors or mobile devices remains a challenge. In this work, we propose CoSOV1net, a novel lightweight salient object-detection neural network model, inspired by the cone- and spatial-opponent processes of the primary visual cortex (V1), which inextricably link color and shape in human color perception. Our proposed model is trained from scratch, without using backbones from image classification or other tasks. Experiments on the most widely used and challenging datasets for salient object detection show that CoSOV1Net achieves competitive performance (i.e., Fß=0.931 on the ECSSD dataset) with state-of-the-art salient object-detection models while having a low number of parameters (1.14 M), low FLOPS (1.4 G) and high FPS (211.2) on GPU (Nvidia GeForce RTX 3090 Ti) compared to the state of the art in lightweight or nonlightweight salient object-detection tasks. Thus, CoSOV1net has turned out to be a lightweight salient object-detection model that can be adapted to mobile environments and resource-constrained devices.

3D biplanar reconstruction of lower limbs using nonlinear statistical models.

Three-dimensional (3D) reconstruction of lower limbs is of great interest in surgical planning, computer assisted surgery, and for biomechanical applications. The use of 3D imaging modalities such as computed tomography (CT) scan and magnetic resonance imaging (MRI) has limitations such as high radiation and expense. Therefore, three-dimensional reconstruction methods from biplanar X-ray images represent an attractive alternative. In this paper, we present a new unsupervised 3D reconstruction method for the patella, talus, and pelvis using calibrated biplanar (45- and 135-degree oblique) radiographic images and a prior information on the geometric/anatomical structure of these complex bones. A multidimensional scaling (MDS)-based nonlinear dimensionality reduction algorithm is applied to exploit this prior geometric/anatomical information. It represents relevant deformations existing in the training set. Our method is based on a hybrid-likelihood using regions and contours. The edge-based notion represents the relation between the external contours of the bone projections and an edge potential field estimated on the radiographic images. Region-based notion is the non-overlapping ratio between segmented and projected bone regions of interest (RoIs). Our automatic 3D reconstruction model entails stochastically minimizing an energy function allowing an estimation of deformation parameters of the bone shape. This 3D reconstruction method has been successfully tested on 13 biplanar radiographic image pairs, yielding very promising results.

Salient Object Detection by LTP Texture Characterization on Opposing Color Pairs under SLICO Superpixel Constraint.

The effortless detection of salient objects by humans has been the subject of research in several fields, including computer vision, as it has many applications. However, salient object detection remains a challenge for many computer models dealing with color and textured images. Most of them process color and texture separately and therefore implicitly consider them as independent features which is not the case in reality. Herein, we propose a novel and efficient strategy, through a simple model, almost without internal parameters, which generates a robust saliency map for a natural image. This strategy consists of integrating color information into local textural patterns to characterize a color micro-texture. It is the simple, yet powerful LTP (Local Ternary Patterns) texture descriptor applied to opposing color pairs of a color space that allows us to achieve this end. Each color micro-texture is represented by a vector whose components are from a superpixel obtained by the SLICO (Simple Linear Iterative Clustering with zero parameter) algorithm, which is simple, fast and exhibits state-of-the-art boundary adherence. The degree of dissimilarity between each pair of color micro-textures is computed by the FastMap method, a fast version of MDS (Multi-dimensional Scaling) that considers the color micro-textures' non-linearity while preserving their distances. These degrees of dissimilarity give us an intermediate saliency map for each RGB (Red-Green-Blue), HSL (Hue-Saturation-Luminance), LUV (L for luminance, U and V represent chromaticity values) and CMY (Cyan-Magenta-Yellow) color space. The final saliency map is their combination to take advantage of the strength of each of them. The MAE (Mean Absolute Error), MSE (Mean Squared Error) and Fß measures of our saliency maps, on the five most used datasets show that our model outperformed several state-of-the-art models. Being simple and efficient, our model could be combined with classic models using color contrast for a better performance.

Unraveling reproducible dynamic states of individual brain functional parcellation.

Data-driven parcellations are widely used for exploring the functional organization of the brain, and also for reducing the high dimensionality of fMRI data. Despite the flurry of methods proposed in the literature, functional brain parcellations are not highly reproducible at the level of individual subjects, even with very long acquisitions. Some brain areas are also more difficult to parcellate than others, with association heteromodal cortices being the most challenging. An important limitation of classical parcellations is that they are static, that is, they neglect dynamic reconfigurations of brain networks. In this paper, we proposed a new method to identify dynamic states of parcellations, which we hypothesized would improve reproducibility over static parcellation approaches. For a series of seed voxels in the brain, we applied a cluster analysis to regroup short (3 min) time windows into "states" with highly similar seed parcels. We split individual time series of the Midnight scan club sample into two independent sets of 2.5 hr (test and retest). We found that average within-state parcellations, called stability maps, were highly reproducible (over 0.9 test-retest spatial correlation in many instances) and subject specific (fingerprinting accuracy over 70% on average) between test and retest. Consistent with our hypothesis, seeds in heteromodal cortices (posterior and anterior cingulate) showed a richer repertoire of states than unimodal (visual) cortex. Taken together, our results indicate that static functional parcellations are incorrectly averaging well-defined and distinct dynamic states of brain parcellations. This work calls to revisit previous methods based on static parcellations, which includes the majority of published network analyses of fMRI data. Our method may, thus, impact how researchers model the rich interactions between brain networks in health and disease.

Multimodal Change Detection in Remote Sensing Images Using an Unsupervised Pixel Pairwise Based Markov Random Field Model.

This work presents a Bayesian statistical approach to the multimodal change detection (CD) problem in remote sensing imagery. More precisely, we formulate the multimodal CD problem in the unsupervised Markovian framework. The main novelty of the proposed Markovian model lies in the use of an observation field built up from a pixel pairwise modeling and on the bitemporal heterogeneous satellite image pair. Such modeling allows us to rely instead on a robust visual cue, with the appealing property of being quasi-invariant to the imaging (multi-) modality. To use this observation cue as part of a stochastic likelihood model, we first rely on a preliminary iterative estimation technique that takes into account the variety of the laws in the distribution mixture and estimates the parameters of the Markovian mixture model. Once this estimation step is completed, the Maximum a posteriori (MAP) solution of the change detection map, based on the previously estimated parameters, is then computed with a stochastic optimization process. Experimental results and comparisons involving a mixture of different types of imaging modalities confirm the robustness of the proposed approach.

A Multi-Objective Decision Making Approach for Solving the Image Segmentation Fusion Problem.

Image segmentation fusion is defined as the set of methods which aim at merging several image segmentations, in a manner that takes full advantage of the complementarity of each one. Previous relevant researches in this field have been impeded by the difficulty in identifying an appropriate single segmentation fusion criterion, providing the best possible, i.e., the more informative, result of fusion. In this paper, we propose a new model of image segmentation fusion based on multi-objective optimization which can mitigate this problem, to obtain a final improved result of segmentation. Our fusion framework incorporates the dominance concept in order to efficiently combine and optimize two complementary segmentation criteria, namely, the global consistency error and the F-measure (precision-recall) criterion. To this end, we present a hierarchical and efficient way to optimize the multi-objective consensus energy function related to this fusion model, which exploits a simple and deterministic iterative relaxation strategy combining the different image segments. This step is followed by a decision making task based on the so-called "technique for order performance by similarity to ideal solution". Results obtained on two publicly available databases with manual ground truth segmentations clearly show that our multi-objective energy-based model gives better results than the classical mono-objective one.

A perceptual map for gait symmetry quantification and pathology detection.

BACKGROUND: The gait movement is an essential process of the human activity and the result of collaborative interactions between the neurological, articular and musculoskeletal systems, working efficiently together. This explains why gait analysis is important and increasingly used nowadays for the diagnosis of many different types (neurological, muscular, orthopedic, etc.) of diseases. This paper introduces a novel method to quickly visualize the different parts of the body related to an asymmetric movement in the human gait of a patient for daily clinical usage. The proposed gait analysis algorithm relies on the fact that the healthy walk has (temporally shift-invariant) symmetry properties in the coronal plane. The goal is to provide an inexpensive and easy-to-use method, exploiting an affordable consumer depth sensor, the Kinect, to measure the gait asymmetry and display results in a perceptual way. METHOD: We propose a multi-dimensional scaling mapping using a temporally shift invariant distance, allowing us to efficiently visualize (in terms of perceptual color difference) the asymmetric body parts of the gait cycle of a subject. We also propose an index computed from this map and which quantifies locally and globally the degree of asymmetry. RESULTS: The proposed index is proved to be statistically significant and this new, inexpensive, marker-less, non-invasive, easy to set up, gait analysis system offers a readable and flexible tool for clinicians to analyze gait characteristics and to provide a fast diagnostic. CONCLUSION: This system, which estimates a perceptual color map providing a quick overview of asymmetry existing in the gait cycle of a subject, can be easily exploited for disease progression, recovery cues from post-operative surgery (e.g., to check the healing process or the effect of a treatment or a prosthesis) or might be used for other pathologies where gait asymmetry might be a symptom.

Local symmetry detection in natural images using a particle filtering approach.

In this paper, we propose an algorithm to detect smooth local symmetries and contours of ribbon-like objects in natural images. The detection is formulated as a spatial tracking task using a particle filtering approach, extracting one part of a structure at a time. Using an adaptive local geometric model, the method can detect straight reflection symmetries in perfectly symmetrical objects as well as smooth local symmetries in curved elongated objects. In addition, the proposed approach jointly estimates spine and contours, making it possible to generate back ribbon objects. Experiments for local symmetry detection have been conducted on a recent extension of the Berkeley segmentation data sets. We also show that it is possible to retrieve specific geometrical objects using intuitive prior structural information.

A MultiScale Particle Filter Framework for Contour Detection.

We investigate the contour detection task in complex natural images. We propose a novel contour detection algorithm which jointly tracks at two scales small pieces of edges called edgelets. This multiscale edgelet structure naturally embeds semi-local information and is the basic element of the proposed recursive Bayesian modeling. Prior and transition distributions are learned offline using a shape database. Likelihood functions are learned online, thus are adaptive to an image, and integrate color and gradient information via local, textural, oriented, and profile gradient-based features. The underlying model is estimated using a sequential Monte Carlo approach, and the final soft contour detection map is retrieved from the approximated trajectory distribution. We also propose to extend the model to the interactive cut-out task. Experiments conducted on the Berkeley Segmentation data sets show that the proposed MultiScale Particle Filter Contour Detector method performs well compared to competing state-of-the-art methods.

Evaluating perceptual maps of asymmetries for gait symmetry quantification and pathology detection.

The gait movement is a complex and essential process of the human activity. Yet, many types of diseases (neurological, muscular, orthopedic, etc.) can be diagnosed from the gait analysis. This paper introduces a novel method to quickly visualize the different body parts related to an (temporally shift-invariant) asymmetric movement in the human gait of a patient for daily clinical usage. The goal is to provide a cheap and easy-to-use method that measures the gait asymmetry and display results in a perceptual and intuitive way. This method relies on an affordable consumer depth sensor, the Kinect, which is very suitable for small room and fast diagnostic, since it is easy to setup and marker-less.

An energy-based model for the image edge-histogram specification problem.

In this correspondence, we present an original energy-based model that achieves the edge-histogram specification of a real input image and thus extends the exact specification method of the image luminance (or gray level) distribution recently proposed by Coltuc et al. Our edge-histogram specification approach is stated as an optimization problem in which each edge of a real input image will tend iteratively toward some specified gradient magnitude values given by a target edge distribution (or a normalized edge histogram possibly estimated from a target image). To this end, a hybrid optimization scheme combining a global and deterministic conjugate-gradient-based procedure and a local stochastic search using the Metropolis criterion is proposed herein to find a reliable solution to our energy-based model. Experimental results are presented, and several applications follow from this procedure.

Analyzing gait pathologies using a depth camera.

In this paper, we present a new approach to construct a 3D human skeleton model, which is then used to quantify gait pathologies, using a depth camera. First, thanks to the depth map, we obtain a human depth silhouette in 3D, from which our method is based to estimate each body part position. Second, the angle between the upper and lower legs of the 3D skeleton model is calculated. Finally, we show that using only this angle information is enough to quantify motion asymmetry. This result has been verified through an experimental study with 3 different subjects. Due to its advantages (simple, markerless and low-cost), this method is a promising solution for gait clinics in the future.

MDS-based multiresolution nonlinear dimensionality reduction model for color image segmentation.

In this paper, we present an efficient coarse-to-fine multiresolution framework for multidimensional scaling and demonstrate its performance on a large-scale nonlinear dimensionality reduction and embedding problem in a texture feature extraction step for the unsupervised image segmentation problem. We demonstrate both the efficiency of our multiresolution algorithm and its real interest to learn a nonlinear low-dimensional representation of the texture feature set of an image which can then subsequently be exploited in a simple clustering-based segmentation algorithm. The resulting segmentation procedure has been successfully applied on the Berkeley image database, demonstrating its efficiency compared to the best existing state-of-the-art segmentation methods recently proposed in the literature.

Depth Energy Image for gait symmetry quantification.

This paper introduces a new quantification method for gait symmetry based on depth information acquired from a structured light system. First, the new concept of Depth Energy Image is introduced to better visualize gait asymmetry problems. Then a simple index is computed from this map to quantify motion symmetry. Results are presented for six subjects with and without gait problems. Since the method is markerless and cheap, it could be a very promising solution in the future for gait clinics.

A label field fusion bayesian model and its penalized maximum rand estimator for image segmentation.

This paper presents a novel segmentation approach based on a Markov random field (MRF) fusion model which aims at combining several segmentation results associated with simpler clustering models in order to achieve a more reliable and accurate segmentation result. The proposed fusion model is derived from the recently introduced probabilistic Rand measure for comparing one segmentation result to one or more manual segmentations of the same image. This non-parametric measure allows us to easily derive an appealing fusion model of label fields, easily expressed as a Gibbs distribution, or as a nonstationary MRF model defined on a complete graph. Concretely, this Gibbs energy model encodes the set of binary constraints, in terms of pairs of pixel labels, provided by each segmentation results to be fused. Combined with a prior distribution, this energy-based Gibbs model also allows for definition of an interesting penalized maximum probabilistic rand estimator with which the fusion of simple, quickly estimated, segmentation results appears as an interesting alternative to complex segmentation models existing in the literature. This fusion framework has been successfully applied on the Berkeley image database. The experiments reported in this paper demonstrate that the proposed method is efficient in terms of visual evaluation and quantitative performance measures and performs well compared to the best existing state-of-the-art segmentation methods recently proposed in the literature.

Segmentation by fusion of histogram-based k-means clusters in different color spaces.

This paper presents a new, simple, and efficient segmentation approach, based on a fusion procedure which aims at combining several segmentation maps associated to simpler partition models in order to finally get a more reliable and accurate segmentation result. The different label fields to be fused in our application are given by the same and simple (K-means based) clustering technique on an input image expressed in different color spaces. Our fusion strategy aims at combining these segmentation maps with a final clustering procedure using as input features, the local histogram of the class labels, previously estimated and associated to each site and for all these initial partitions. This fusion framework remains simple to implement, fast, general enough to be applied to various computer vision applications (e.g., motion detection and segmentation), and has been successfully applied on the Berkeley image database. The experiments herein reported in this paper illustrate the potential of this approach compared to the state-of-the-art segmentation methods recently proposed in the literature.

DCT-based complexity regularization for EM tomographic reconstruction.

This paper introduces a simple algorithm for tomographic reconstruction based on the use of a complexity regularization term. The regularization is formulated in the discrete cosine transform (DCT) domain by promoting a low-noise reconstruction having a high sparsity in the frequency domain. The resulting algorithm simply alternates between a maximum-likelihood (ML) expectation-maximization (EM) update and a decreasing sparsity constraint in the DCT domain. Applications to SPECT reconstruction and comparisons with a classical estimator using the best available regularization terms are given in order to illustrate the potential of our reconstruction technique.

Segmentation framework based on label field fusion.

In this paper, we put forward a novel fusion framework that mixes together label fields instead of observation data as is usually the case. Our framework takes as input two label fields: a quickly estimated and to-be-refined segmentation map and a spatial region map that exhibits the shape of the main objects of the scene. These two label fields are fused together with a global energy function that is minimized with a deterministic iterative conditional mode algorithm. As explained in the paper, the energy function may implement a pure fusion strategy or a fusion-reaction function. In the latter case, a data-related term is used to make the optimization problem well posed. We believe that the conceptual simplicity, the small number of parameters, the use of a simple and fast deterministic optimizer that admits a natural implementation on a parallel architecture are among the main advantages of our approach. Our fusion framework is adapted to various computer vision applications among which are motion segmentation, motion estimation and occlusion detection.

Localization of shapes using statistical models and stochastic optimization.

In this paper, we present a new model for deformations of shapes. A pseudo-likelihood is based on the statistical distribution of the gradient vector field of the gray level. The prior distribution is based on the Probabilistic Principal Component Analysis (PPCA). We also propose a new model based on mixtures of PPCA that is useful in the case of greater variability in the shape. A criterion of global or local object specificity based on a preliminary color segmentation of the image, is included into the model. The localization of a shape in an image is then viewed as minimizing the corresponding Gibbs field. We use the Exploration/Selection (E/S) stochastic algorithm in order to find the optimal deformation. This yields a new unsupervised statistical method for localization of shapes. In order to estimate the statistical parameters for the gradient vector field of the gray level, we use an Iterative Conditional Estimation (ICE) procedure. The color segmentation of the image can be computed with an Exploration/Selection/Estimation (ESE) procedure.