Vesalius: high-resolution in silico anatomization of spatial transcriptomic data using image analysis.

Characterization of tissue architecture promises to deliver insights into development, cell communication, and disease. In silico spatial domain retrieval methods have been developed for spatial transcriptomics (ST) data assuming transcriptional similarity of neighboring barcodes. However, domain retrieval approaches with this assumption cannot work in complex tissues composed of multiple cell types. This task becomes especially challenging in cellular resolution ST methods. We developed Vesalius to decipher tissue anatomy from ST data by applying image processing technology. Vesalius uniquely detected territories composed of multiple cell types and successfully recovered tissue structures in high-resolution ST data including in mouse brain, embryo, liver, and colon. Utilizing this tissue architecture, Vesalius identified tissue morphology-specific gene expression and regional specific gene expression changes for astrocytes, interneuron, oligodendrocytes, and entorhinal cells in the mouse brain.

Block-cyclic stochastic coordinate descent for deep neural networks.

We present a stochastic first-order optimization algorithm, named block-cyclic stochastic coordinate descent (BCSC), that adds a cyclic constraint to stochastic block-coordinate descent in the selection of both data and parameters. It uses different subsets of the data to update different subsets of the parameters, thus limiting the detrimental effect of outliers in the training set. Empirical tests in image classification benchmark datasets show that BCSC outperforms state-of-the-art optimization methods in generalization leading to higher accuracy within the same number of update iterations. The improvements are consistent across different architectures and datasets, and can be combined with other training techniques and regularizations.

Adaptive Regularization of Some Inverse Problems in Image Analysis.

We present an adaptive regularization scheme for optimizing composite energy functionals arising in image analysis problems. The scheme automatically trades off data fidelity and regularization depending on the current data fit during the iterative optimization, so that regularization is strongest initially, and wanes as data fidelity improves, with the weight of the regularizer being minimized at convergence. We also introduce a Huber loss function in both data fidelity and regularization terms, and present an efficient convex optimization algorithm based on the alternating direction method of multipliers (ADMM) using the equivalent relation between the Huber function and the proximal operator of the one-norm. We illustrate and validate our adaptive Huber-Huber model on synthetic and real images in segmentation, motion estimation, and denoising problems.

Shape Matching Using Multiscale Integral Invariants.

We present a shape descriptor based on integral kernels. Shape is represented in an implicit form and it is characterized by a series of isotropic kernels that provide desirable invariance properties. The shape features are characterized at multiple scales which form a signature that is a compact description of shape over a range of scales. The shape signature is designed to be invariant with respect to group transformations which include translation, rotation, scaling, and reflection. In addition, the integral kernels that characterize local shape geometry enable the shape signature to be robust with respect to undesirable perturbations while retaining discriminative power. Use of our shape signature is demonstrated for shape matching based on a number of synthetic and real examples.

Semiautomated thyroid volumetry using 3D CT: prospective comparison with measurements obtained using 2D ultrasound, 2D CT, and water displacement method of specimen.

OBJECTIVE: The objective of our study was to measure thyroid volumes using semiautomated 3D CT and to compare the 3D CT volumes with volumes measured using 2D ultrasound, 2D CT, and the water displacement method. SUBJECTS AND METHODS: In 47 patients, 2D ultrasound volumes and 2D CT volumes of the thyroid gland were estimated using the ellipsoid volume formula, and 3D CT volumes were calculated using semiautomated reconstructive techniques. All volume data were compared with thyroid specimen volumes obtained using the water displacement method and were statistically analyzed using the one-way ANOVA, the Pearson correlation coefficient (R), linear regression, and the concordance correlation coefficient (CCC). The processing time of semiautomated 3D CT thyroid volumetry was measured. RESULTS: The paired mean differences ± SD between the three imaging-determined volumes and the specimen volumes were 0.8 ± 3.1 mL for 2D ultrasound, 4.0 ± 4.7 mL for 2D CT, and 0.2 ± 2.5 mL for 3D CT. A significant difference in the mean thyroid volume was found between 2D CT and specimen volumes (p = 0.016) compared with the other pairs (p = 0.937 for 2D ultrasound mean volume vs specimen mean volume, and p = 0.999 for 3D CT mean volume vs specimen mean volume). Between specimen volume and 2D ultrasound volume, specimen volume and 2D CT volume, and specimen volume and 3D CT volume, R values were 0.885, 0.724, and 0.929, respectively, and CCC values were 0.876, 0.598, and 0.925, respectively. The mean processing time of semiautomated 3D CT thyroid volumetry was 7.0 minutes. CONCLUSION: Thyroid volumes measured using 2D ultrasound or semiautomated 3D CT are substantially close to thyroid specimen volumes measured using the water displacement method. Semiautomated 3D CT thyroid volumetry can provide a more reliable measure of thyroid volume than 2D ultrasound.

A population-based tissue probability map-driven level set method for fully automated mammographic density estimations.

PURPOSE: A major challenge when distinguishing glandular tissues on mammograms, especially for area-based estimations, lies in determining a boundary on a hazy transition zone from adipose to glandular tissues. This stems from the nature of mammography, which is a projection of superimposed tissues consisting of different structures. In this paper, the authors present a novel segmentation scheme which incorporates the learned prior knowledge of experts into a level set framework for fully automated mammographic density estimations. METHODS: The authors modeled the learned knowledge as a population-based tissue probability map (PTPM) that was designed to capture the classification of experts' visual systems. The PTPM was constructed using an image database of a selected population consisting of 297 cases. Three mammogram experts extracted regions for dense and fatty tissues on digital mammograms, which was an independent subset used to create a tissue probability map for each ROI based on its local statistics. This tissue class probability was taken as a prior in the Bayesian formulation and was incorporated into a level set framework as an additional term to control the evolution and followed the energy surface designed to reflect experts' knowledge as well as the regional statistics inside and outside of the evolving contour. RESULTS: A subset of 100 digital mammograms, which was not used in constructing the PTPM, was used to validate the performance. The energy was minimized when the initial contour reached the boundary of the dense and fatty tissues, as defined by experts. The correlation coefficient between mammographic density measurements made by experts and measurements by the proposed method was 0.93, while that with the conventional level set was 0.47. CONCLUSIONS: The proposed method showed a marked improvement over the conventional level set method in terms of accuracy and reliability. This result suggests that the proposed method successfully incorporated the learned knowledge of the experts' visual systems and has potential to be used as an automated and quantitative tool for estimations of mammographic breast density levels.

Tracking using motion estimation with physically motivated inter-region constraints.

We propose a method for tracking structures (e.g., ventricles and myocardium) in cardiac images (e.g., magnetic resonance) by propagating forward in time a previous estimate of the structures using a new physically motivated motion estimation scheme. Our method estimates motion by regularizing only within structures so that differing motions among different structures are not mixed. It simultaneously satisfies the physical constraints at the interface between a fluid and a medium that the normal component of the fluid's motion must match the normal component of the medium's motion and the No-Slip condition, which states that the tangential velocity approaches zero near the interface. We show that these conditions lead to partial differential equations with Robin boundary conditions at the interface, which couple the motion between structures. We show that propagating a segmentation across frames using our motion estimation scheme leads to more accurate segmentation than traditional motion estimation that does not use physical constraints. Our method is suited to interactive segmentation, prominently used in commercial applications for cardiac analysis, where segmentation propagation is used to predict a segmentation in the next frame. We show that our method leads to more accurate predictions than a popular and recent interactive method used in cardiac segmentation.

Joint estimation of shape and deformation for the detection of lesions in dynamic contrast-enhanced breast MRI.

We propose a mathematical framework for simultaneously delineating the boundary of object and estimating its temporal motion in the application of lesion detection in a dynamic contrast-enhanced (DCE) breast MRI sequence where both the appearance and the shape of region of interest is assumed to change in time. A unified energy functional for a joint segmentation and registration is proposed based on the assumption that the statistical properties of dynamic intensity curves within a region of interest are homogeneous. Our algorithm is designed to provide the morphological properties of the enhanced region and its dynamic intensity profiles, called kinetic signatures, in the analysis of DCE imagery since these features are considered as significant cues in understanding images. The proposed energy comprises a combination of a segmentation energy and a registration energy. The segmentation energy is developed based on a convex formulation being insensitive to the initialization. The registration energy is designed to compensate motion artifacts that are usually involved in the temporal imaging procedure. The major objective of this work is to provide a mathematical framework for a joint segmentation and registration on a dynamic sequence of images, and we demonstrate the mutual benefit of the estimation of temporal deformations for the registration step and the localization of regions of interest for the segmentation step. The effectiveness of the developed algorithm has been demonstrated on a number of clinical DCE breast MRI data in the application of breast lesion detection and the results show its potential to improve the accuracy and the efficiency in the diagnosis of breast cancer.

Multipass active contours for an adaptive contour map.

Isocontour mapping is efficient for extracting meaningful information from a biomedical image in a topographic analysis. Isocontour extraction from real world medical images is difficult due to noise and other factors. As such, adaptive selection of contour generation parameters is needed. This paper proposes an algorithm for generating an adaptive contour map that is spatially adjusted. It is based on the modified active contour model, which imposes successive spatial constraints on the image domain. The adaptability of the proposed algorithm is governed by the energy term of the model. This work focuses on mammograms and the analysis of their intensity. Our algorithm employs the Mumford-Shah energy functional, which considers an image's intensity distribution. In mammograms, the brighter regions generally contain significant information. Our approach exploits this characteristic to address the initialization and local optimum problems of the active contour model. Our algorithm starts from the darkest region; therefore, local optima encountered during the evolution of contours are populated in less important regions, and the important brighter regions are reserved for later stages. For an unrestricted initial contour, our algorithm adopts an existing technique without re-initialization. To assess its effectiveness and robustness, the proposed algorithm was tested on a set of mammograms.

Nonlinear registration of serial coronary CT angiography (CCTA) for assessment of changes in atherosclerotic plaque.

PURPOSE: Coronary CT angiography (CCTA) is a high-resolution three-dimensional imaging technique for the evaluation of coronary arteries in suspected or confirmed coronary artery disease (CAD). Coregistration of serial CCTA scans would allow precise superimposition of images obtained at two different points in time, which could aid in recognition of subtle changes and precise monitoring of coronary plaque progression or regression. To this end, the authors aimed at developing a fully automatic nonlinear volume coregistration for longitudinal CCTA scan pairs. METHODS: The algorithm combines global displacement and local deformation using nonlinear volume coregistration with a volume-preserving constraint. Histogram matching of intensities between two serial scans is performed prior to nonlinear coregistration with dense nonparametric local deformation in which sum of squared differences is used as a similarity measure. The approximate segmentation of coronary arteries obtained from commercially available software provides initial anatomical landmarks for the coregistration algorithm that help localize and emphasize the structure of interest. To avoid possible bias caused by incorrect segmentation, the authors convolve the Gaussian kernel with the segmented binary coronary tree mask and define an extended weighted region of interest. A multiresolution approach is employed to represent coarse-to-fine details of both volumes and the energy function is optimized using a gradient descent method. The authors applied the algorithm in ten paired CCTA datasets (20 scans in total) obtained within 10.7 +/- 5.7 months from each other on a dual source CT scanner to monitor progression of CAD. RESULTS: Serial CCTA coregistration was successful in 9/10 cases as visually confirmed. The global displacement and local deformation of target registration error obtained from four anatomical landmarks were 2.22 +/- 1.15 and 1.56 +/- 0.74 mm, respectively, and the inverse consistency error of local deformation was 0.14 +/- 0.06 mm. The observer variability between two expert observers was 1.31 +/- 0.91 mm. CONCLUSIONS: The proposed coregistration algorithm demonstrates potential to accurately register serial CCTA scans, which may allow direct comparison of calcified and noncalcified atherosclerotic plaque changes between the two scans.

Segmentation of regions of interest in mammograms in a topographic approach.

This paper presents a novel method for the segmentation of regions of interest in mammograms. The algorithm concurrently delineates the boundaries of the breast boundary, the pectoral muscle, as well as dense regions that include candidate masses. The resulting representation constitutes an analysis of the global structure of the object in the mammogram. We propose a topographic representation called the isocontour map, in which a salient region forms a dense quasi-concentric pattern of contours. The topological and geometrical structure of the image is analyzed using an inclusion tree that is a hierarchical representation of the enclosure relationships between contours. The "saliency" of a region is measured topologically as the minimum nesting depth. Features at various scales are analyzed in multiscale isocontour maps, and we demonstrate that the multiscale scheme provides an efficient way of achieving better delineations. Experimental results demonstrate that the proposed method has potential as the basis for a prompting system in mammogram mass detection.

Geometric feature-based multimodal image registration of contrast-enhanced cardiac CT with gated myocardial perfusion SPECT.

PURPOSE: Cardiac computed tomography (CT) and single photon emission computed tomography (SPECT) provide clinically complementary information in the diagnosis of coronary artery disease (CAD). Fused anatomical and physiological data acquired sequentially on separate scanners can be coregistered to accurately diagnose CAD in specific coronary vessels. METHODS: A fully automated registration method is presented utilizing geometric features from a reliable segmentation of gated myocardial perfusion SPECT (MPS) volumes, where regions of myocardium and blood pools are extracted and used as an anatomical mask to de-emphasize the inhomogeneities of intensity distribution caused by perfusion defects and physiological variations. A multiresolution approach is employed to represent coarse-to-fine details of both volumes. The extracted voxels from each level are aligned using a similarity measure with a piecewise constant image model and minimized using a gradient descent method. The authors then perform limited nonlinear registration of gated MPS to adjust for phase differences by automatic cardiac phase matching between CT and MPS. For phase matching, they incorporate nonlinear registration using thin-plate-spline-based warping. Rigid registration has been compared with manual alignment (n=45) on 20 stress/rest MPS and coronary CTA data sets acquired from two different sites and five stress CT perfusion data sets. Phase matching was also compared to expert visual assessment. RESULTS: As compared with manual alignment obtained from two expert observers, the mean and standard deviation of absolute registration errors of the proposed method for MPS were 4.3 +/- 3.5, 3.6 +/- 2.6, and 3.6 +/- 2.1 mm for translation and 2.1 +/- 3.2 degrees, 0.3 +/- 0.8 degree, and 0.7 +/- 1.2 degrees for rotation at site A and 3.8 +/- 2.7, 4.0 +/- 2.9, and 2.2 +/- 1.8mm for translation and 1.1 +/- 2.0 degrees, 1.6 +/- 3.1 degrees, and 1.9 +/- 3.8 degrees for rotation at site B. The results for CT perfusion were 3.0 +/- 2.9, 3.5 +/- 2.4, and 2.8 +/- 1.0 mm for translation and 3.0 +/- 2.4 degrees, 0.6 +/- 0.9 degree, and 1.2 +/- 1.3 degrees for rotation. The registration error shows that the proposed method achieves registration accuracy of less than 1 voxel (6.4 x 6.4 x 6.4 mm) misalignment. The proposed method was robust for different initializations in the range from -80 to 70, -80 to 70, and -50 to 50 mm in the x-, y-, and z-axes, respectively. Validation results of finding best matching phase showed that best matching phases were not different by more than two phases, as visually determined. CONCLUSIONS: The authors have developed a fast and fully automated method for registration of contrast cardiac CT with gated MPS which includes nonlinear cardiac phase matching and is capable of registering these modalities with accuracy <10 mm in 87% of the cases.

AUTOMATED MULTI-MODALITY REGISTRATION OF 64-SLICE CORONARY CT ANGIOGRAPHY WITH MYOCARDIAL PERFUSION SPECT.

A multi-modality image registration algorithm for the alignment of myocardial perfusion SPECT (MPS) and coronary computed tomography angiography (CTA) scans is presented in this work. Coronary CTA and MPS provides clinically complementary information in the diagnosis of coronary artery disease. An automated registration algorithm is proposed utilizing segmentation results of MPS volumes, where regions of myocardium and blood pools are extracted and used as an anatomical mask. Using a variational framework, we adopt an energy functional with a piecewise constant image model and optimize it numerically with a gradient descent algorithm. The computational efficiency and robustness of the proposed automatic registration of CTA with MPS have been demonstrated by the experiments that yielded an average error smaller than an MPS voxel size.

Multimodal data integration for computer-aided ablation of atrial fibrillation.

Image-guided percutaneous interventions have successfully replaced invasive surgical methods in some cardiologic practice, where the use of 3D-reconstructed cardiac images, generated by magnetic resonance imaging (MRI) and computed tomography (CT), plays an important role. To conduct computer-aided catheter ablation of atrial fibrillation accurately, multimodal information integration with electroanatomic mapping (EAM) data and MRI/CT images is considered in this work. Specifically, we propose a variational formulation for surface reconstruction and incorporate the prior shape knowledge, which results in a level set method. The proposed method enables simultaneous reconstruction and registration under nonrigid deformation. Promising experimental results show the potential of the proposed approach.

Locally rotation, contrast, and scale invariant descriptors for texture analysis.

Textures within real images vary in brightness, contrast, scale and skew as imaging conditions change. To enable recognition of textures in real images, it is necessary to employ a similarity measure which is invariant to these properties. Furthermore, since textures often appear on undulating surfaces, such invariances must necessarily be local rather than global. Despite these requirements, it is only relatively recently that texture recognition algorithms with local scale and affine invariance properties have begun to be reported. Typically, they comprise detecting feature points followed by geometric normalization prior to description. We describe a method based on invariant combinations of linear filters. Unlike previous methods, we introduce a novel family of filters, which provide scale invariance, resulting in a texture description invariant to local changes in orientation, contrast and scale and robust to local skew. Significantly, the family of filters enable local scale invariants to be defined without using a scale selection principle or a large number of filters. A texture discrimination method based on the A2 similarity measure applied to histograms derived from our filter responses outperforms existing methods for retrieval and classification results for both the Brodatz textures and the UIUC database, which has been designed to require local invariance.

Integral invariants for shape matching.

For shapes represented as closed planar contours, we introduce a class of functionals which are invariant with respect to the Euclidean group and which are obtained by performing integral operations. While such integral invariants enjoy some of the desirable properties of their differential counterparts, such as locality of computation (which allows matching under occlusions) and uniqueness of representation (asymptotically), they do not exhibit the noise sensitivity associated with differential quantities and, therefore, do not require presmoothing of the input shape. Our formulation allows the analysis of shapes at multiple scales. Based on integral invariants, we define a notion of distance between shapes. The proposed distance measure can be computed efficiently and allows warping the shape boundaries onto each other; its computation results in optimal point correspondence as an intermediate step. Numerical results on shape matching demonstrate that this framework can match shapes despite the deformation of subparts, missing parts and noise. As a quantitative analysis, we report matching scores for shape retrieval from a database.