Tracking visible boundary of objects using occlusion adaptive motion snake.

We propose a novel technique for tracking the visible boundary of a video object in the presence of occlusion. Starting with an initial contour that is interactively specified by the user and may be automatically refined by using intra-energy terms, the proposed technique employs piecewise contour prediction using local motion and color information on both sides of the contour segment, and contour snapping using scale-invariant intra-frame and inter-frame energy terms. The piecewise (segmented) nature of the contour prediction scheme and modeling of the motion on both sides of each contour segment enable accurate determination of whether and where the tracked boundary is occluded by another object. The proposed snake energy terms are associated with contour segments (as opposed to node points) and they are scale/resolution independent to allow multi-resolution contour tracking without the need to retune the weights of the energy terms at each resolution level. This facilitates contour prediction at coarse resolution and snapping at fine resolution with high accuracy. Experimental results are provided to illustrate the performance of the proposed occlusion detection algorithm and the novel snake energy terms that enable visible boundary tracking in the presence of occlusion.

End-to-end color printer calibration by total least squares regression.

Neugebauer modeling plays an important role in obtaining end-to-end device characterization profiles for halftone color printer calibration. This paper proposes total least square (TLS) regression methods to estimate the parameters of various Neugebauer models. Compared to the traditional least squares (LS) based methods, the TLS approach is physically more appropriate for the printer modeling problem because it accounts for errors in the measured reflectance of both the primaries and the modeled samples. A TLS method based on print measurements from single-colorant step-wedges is first developed. The method is then extended to incorporate multicolorant print measurements using an iterative algorithm. The LS and TLS techniques are compared through tests performed on two color printers, one employing conventional rotated halftone screens and the other using a dot-on-dot halftone screen configuration. Our experiments indicate that the TLS methods yield a consistent and significant improvement over the LS-based techniques for model parameter estimation. The gains from the TLS method are particularly significant when the number of patches for which measured data is available is limited.

A new motion-compensated reduced-order model Kalman filter for space-varying restoration of progressive and interlaced video.

We propose a new approach for motion-compensated, reduced order model Kalman filtering for restoration of progressive and interlaced video. In the case of interlaced inputs, the proposed filter also performs deinterlacing. In contrast to the literature, both motion-compensation and reduced-order state modeling are achieved by augmenting the observation equation, as opposed to modifying the state-transition equation. The proposed modeling, which includes the two-dimensional (2-D) reduced order model Kalman filtering (ROMKF) of Angwin and Kaufman as a special case, results in significant performance improvement in fixed-lag Kalman filtering of space-varying blurred images. This is demonstrated by experimental results.

Robust, object-based high-resolution image reconstruction from low-resolution video.

We propose a robust, object-based approach to high-resolution image reconstruction from video using the projections onto convex sets (POCS) framework. The proposed method employs a validity map and/or a segmentation map. The validity map disables projections based on observations with inaccurate motion information for robust reconstruction in the presence of motion estimation errors; while the segmentation map enables object-based processing where more accurate motion models can be utilized to improve the quality of the reconstructed image. Procedures for the computation of the validity map and segmentation map are presented. Experimental results demonstrate the improvement in image quality that can be achieved by the proposed methods.

Motion segmentation by multistage affine classification.

We present a multistage affine motion segmentation method that combines the benefits of the dominant motion and block-based affine modeling approaches. In particular, we propose two key modifications to a recent motion segmentation algorithm developed by Wang and Adelson (1994). 1) The adaptive k-means clustering step is replaced by a merging step, whereby the affine parameters of a block which has the smallest representation error, rather than the respective cluster center, is used to represent each layer; and 2) we implement it in multiple stages, where pixels belonging to a single motion model are labeled at each stage. Performance improvement due to the proposed modifications is demonstrated on real video frames.

Closed-form connectivity-preserving solutions for motion compensation using 2-D meshes.

Motion compensation using two-dimensional (2-D) mesh models requires computation of the parameters of a spatial transformation for each mesh element (patch). It is well known that the parameters of an affine (bilinear or perspective) mapping can be uniquely estimated from three (four) point correspondences (at the vertices of a triangular or quadrilateral mesh element). On the other hand, overdetermined solutions using more than the required minimum number of point correspondences provide increased robustness against correspondence-estimation errors, however, this necessitates special consideration to preserve mesh-connectivity. This paper presents closed-form, overdetermined solutions for least squares estimation of affine motion parameters for a triangular mesh, which preserve mesh-connectivity using patch-based or node-based connectivity constraints. In particular, four new algorithms are presented: patch-constrained methods using point correspondences or spatio-temporal intensity gradients, and node-constrained methods using point correspondences or spatio-temporal intensity gradients. The methods using point correspondences can be viewed as postprocessing of a dense motion field for best representation in terms of a set of irregularly spaced samples. The methods that are based on spatio-temporal intensity gradients offer closed-form solutions for direct estimation of the best node-point motion vectors (equivalently the best transformation parameters). We show that the performance of the proposed closed-form solutions are comparable to those of the alternative search-based solutions at a fraction of the computational cost.

Occlusion-adaptive, content-based mesh design and forward tracking.

Two-dimensional (2-D) mesh-based motion compensation preserves neighboring relations (through connectivity of the mesh) as well as allowing warping transformations between pairs of frames; thus, it effectively eliminates blocking artifacts that are common in motion compensation by block matching. However, available 2-D mesh models, whether uniform or non-uniform, enforce connectivity everywhere within a frame, which is clearly not suitable across occlusion boundaries. To this effect, we hereby propose an occlusion-adaptive forward-tracking mesh model, where connectivity of the mesh elements (patches) across covered and uncovered region boundaries are broken. This is achieved by allowing no node points within the background to be covered (BTBC) and refining the mesh structure within the model failure (MF) region(s) at each frame. The proposed content-based mesh structure enables better rendition of the motion (compared to a uniform or a hierarchical mesh), while tracking is necessary to avoid transmission of all node locations at each frame. Experimental results show successful motion compensation and tracking.

Simultaneous motion estimation and segmentation.

We present a Bayesian framework that combines motion (optical flow) estimation and segmentation based on a representation of the motion field as the sum of a parametric field and a residual field. The parameters describing the parametric component are found by a least squares procedure given the best estimates of the motion and segmentation fields. The motion field is updated by estimating the minimum-norm residual field given the best estimate of the parametric field, under the constraint that motion field be smooth within each segment. The segmentation field is updated to yield the minimum-norm residual field given the best estimate of the motion field, using Gibbsian priors. The solution to successive optimization problems are obtained using the highest confidence first (HCF) or iterated conditional mode, (ICM) optimization methods. Experimental results on real video are shown.

Out-of-plane motion compensation in multislice spin-echo MRI.

In magnetic resonance imaging (MRI), it is well-known that patient motion plays a significant role in the degradation of image quality. Although the case of translational in-plane motion (x-y-motion) has been studied by several researchers, the effect of rigid, translational out-of-plane motion (z-motion) has not yet been completely analyzed due to its more complex nature. Out-of-plane motion introduces blurring along the slice-selection direction in addition to motion artifacts. Here, the authors present a model to represent the effect of out-of-plane motion on multislice MR data. The inversion of this model not only results in the correction of the artifacts due to out-of-plane motion, but also reduces blurring in the slice-selection direction, yielding higher resolution images. Because of the shift-varying nature of the authors' model, they propose to use a nonlinear postprocessing method, projection onto convex sets (POCS), for its inversion, provided that the motion kernel and the slice-selection profile are known. The proposed method has been tested on simulated data and then applied to actual MR data to demonstrate the feasibility of the technique in real imaging situations.

POCS-based restoration of space-varying blurred images.

We propose a new method for space-varying image restoration using the method of projection onto convex sets (POCS). The formulation allows the use of a different blurring function at each pixel of the image in a computationally efficient manner. We illustrate the performance of the proposed approach by comparing the new results with those of the ROMKF method on simulated images. We also present results on a real-life image with unknown space-varying out-of-focus blur.

An improvement to MBASIC algorithm for 3-D motion and depth estimation.

In model-based coding of facial images, the accuracy of motion and depth parameter estimates strongly affects the coding efficiency. MBASIC (model-based analysis-synthesis image coding) is a simple and effective iterative algorithm recently proposed by Aizawa et el. (see Signal Processing: Image Communication, no.1, p.139-52, 1989) for 3-D motion and depth estimation when the initial depth estimates are relatively accurate. In this correspondence, we analyze its performance in the presence of errors in the initial depth estimates and propose a modification to MBASIC algorithm that significantly improves its robustness to random errors with only a small increase in the computational load.

Flow compensation in MRI using a phase-corrected real reconstruction.

Flow- and motion-related artifacts are problematic in clinical MR imaging. In this paper we discuss the utility of a phase-corrected real reconstruction to reduce flow artifacts. This technique is particularly useful when flow-compensation pulse sequences may not be possible, such as when a very short echo delay or small field-of-view is desired. We will demonstrate that the phase-corrected real reconstruction provides superior results to the magnitude reconstruction either used alone or in conjunction with existing flow-compensation techniques.

Efficient multiframe Wiener restoration of blurred and noisy image sequences.

Computationally efficient multiframe Wiener filtering algorithms that account for both intraframe (spatial) and interframe (temporal) correlations are proposed for restoring image sequences that are degraded by both blur and noise. One is a general computationally efficient multiframe filter, the cross-correlated multiframe (CCMF) Wiener filter, which directly utilizes the power and cross power spectra of only NxN matrices, where N is the number of frames used in the restoration. In certain special cases the CCMF lends itself to a closed-form solution that does not involve any matrix inversion. A special case is the motion-compensated multiframe (MCMF) filter, where each frame is assumed to be a globally shifted version of the previous frame. In this case, the interframe correlations can be implicitly accounted for using the estimated motion information. Thus the MCMF filter requires neither explicit estimation of cross correlations among the frames nor matrix inversion. Performance and robustness results are given.

Maximum likelihood parametric blur identification based on a continuous spatial domain model.

A formulation for maximum-likelihood (ML) blur identification based on parametric modeling of the blur in the continuous spatial coordinates is proposed. Unlike previous ML blur identification methods based on discrete spatial domain blur models, this formulation makes it possible to find the ML estimate of the extent, as well as other parameters, of arbitrary point spread functions that admit a closed-form parametric description in the continuous coordinates. Experimental results are presented for the cases of 1-D uniform motion blur, 2-D out-of-focus blur, and 2-D truncated Gaussian blur at different signal-to-noise ratios.

Correction of periodic motion artifacts along the slice selection axis in MRI.

The effect of periodic motion of a single magnetic resonance imaging (MRI) slice in the direction of the slice selection axis is modeled as amplitude modulation of the raw data with a motion kernel along the phase encoding direction in the Fourier domain. It is shown that this motion can be detected in 1-D projections of the raw data along the frequency encoding direction which in combination with appropriate filtering leads to the recovery of the motion kernel. It is demonstrated by means of simulation examples that significant reduction in the amplitude of ghost artifacts is obtained when the image is filtered by the inverse of the motion kernel. Some issues to be investigated before the technique can be used in a clinical environment are mentioned.