Dynamic Spatial Predicted Background.

We present a novel method for online background modeling for static video cameras - Dynamic Spatial Predicted Background (DSPB). Our unique method employs a small subset of image pixels to predict the whole scene by exploiting pixel correlations (distant and close). DSPB acts as a hybrid model combining successful elements taken from two major approaches: local-adaptive that propose to fit a distribution pixelwise, and global-linear that reconstruct the background by finding a lowrank version of the scene. To our knowledge, this is the first attempt to combine these approaches in a unified system. DSPB models the scene as a superposition of illumination effects and predicts each pixel's value by a linear estimator comprised of only 5 pixels of the scene and can initialize the background starting from the 5th frame. By doing so, we keep the computational load low, allowing our method to be used in many real-time applications using simple hardware. The suggested prediction model of scene appearance is novel, and the scheme is very accurate and efficient computationally. We show the method merits on an application for video FG-BG separation, and how some of the main existing approaches may be challenged and how their drawbacks are less dominant in our model. Experimental results validate our findings, by computation speed and mean F-measure values on several public datasets. We also examine how results may improve by analyzing each video individually according to its content. DSPB can be successfully incorporated in other image processing tasks like change detection, video compression and video inpainting.

Geometry and Radiometry Invariant Matched Manifold Detection.

Consider a set of deformable objects undergoing geometric and radiometric transformations. As a result of the action of these transformations, the set of different realizations of each object is generally a manifold in the space of observations. Assuming the geometric deformations an object undergoes, belong to some finite dimensional family, it has been shown that the universal manifold embedding (UME) provides a set of nonlinear operators that universally maps each of the different manifolds, where each manifold is generated by the set all of possible appearances of a single object, into a distinct linear subspace of an Euclidean space. In this paper, we generalize this framework to the case where the observed object undergoes both an affine geometric transformation, and a monotonic radiometric transformation, and present a novel framework for the detection and recognition of the deformable objects. Applying to each of the observations an operator that makes it invariant to monotonic amplitude transformations, but is geometry-covariant with the affine transformation, the set of all possible observations on that object is mapped by the UME into a single linear subspace-invariant with respect to both the geometric and radiometric transformations. The embedding of the space of observations is independent of the specific observed object; hence it is universal. The invariant representation of the object is the basis of a matched manifold detection and tracking framework of objects that undergo complex geometric and radiometric deformations: the observed surface is tessellated into a set of tiles such that the deformation of each one is well approximated by an affine geometric transformation and a monotonic transformation of the measured intensities. Since each tile is mapped by the radiometry invariant UME to a distinct linear subspace, the detection and tracking problems are solved by evaluating distances between linear subspaces. Classification in this context becomes a problem of determining which labeled subspace in a Grassmannian is closest to a subspace in the same Grassmannian, where the latter has been generated by radiometry invariant UME from an unlabeled observation.

Combined affine geometric transformations and spatially dependent radiometric deformations: a decoupled linear estimation framework.

This paper considers the problem of registering two observations of the same object, where the observations differ due to a combined effect of an affine geometric transformation and nonuniform illumination changes. The problem of deriving new representations of the observations that are both invariant to geometric transformations and linear in the illumination model is analyzed. In this framework, we present a novel method for linear estimation of illumination changes in an affine invariant manner, thus, decoupling the original problem into two simpler ones. The computational complexity of the method is low as it requires no more than solving a linear set of equations. The prior step of illumination estimation is shown to improve the accuracy of state-of-the-art registration techniques by a factor of two.

Decoupled linear estimation of affine geometric deformations and nonlinear intensity transformations of images.

We consider the problem of registering two observations on an arbitrary object, where the two are related by a geometric affine transformation of their coordinate systems, and by a nonlinear mapping of their intensities. More generally, the framework is that of jointly estimating the geometric and radiometric deformations relating two observations on the same object. We show that the original high-dimensional, nonlinear, and nonconvex search problem of simultaneously recovering the geometric and radiometric deformations can be represented by an equivalent sequence of two linear systems. A solution of this sequence yields an exact, explicit, and efficient solution to the joint estimation problem.