Models for motion-based video indexing and retrieval.

With the rapid proliferation of multimedia applications that require video data management, it is becoming more desirable to provide proper video data indexing techniques capable of representing the rich semantics in video data. In real-time applications, the need for efficient query processing is another reason for the use of such techniques. We present models that use the object motion information in order to characterize the events to allow subsequent retrieval. Algorithms for different spatiotemporal search cases in terms of spatial and temporal translation and scale invariance have been developed using various signal and image processing techniques. We have developed a prototype video search engine, PICTURESQUE (pictorial information and content transformation unified retrieval engine for spatiotemporal queries) to verify the proposed methods. Development of such technology will enable true multimedia search engines that will enable indexing and searching of the digital video data based on its true content.

Recursive estimation of images using non-Gaussian autoregressive models.

We consider recursive estimation of images modeled by non-Gaussian autoregressive (AR) models and corrupted by spatially white Gaussian noise. The goal is to find a recursive algorithm to compute a near minimum mean square error (MMSE) estimate of each pixel of the scene using a fixed lookahead of D rows and D columns of the observations. Our method is based on a simple approximation that makes possible the development of a useful suboptimal nonlinear estimator. The algorithm is first developed for a non-Gaussian AR time-series and then generalized to two dimensions. In the process, we draw on the well-known reduced update Kalman filter (KF) technique of Woods and Radewan to circumvent computational load problems. Several examples demonstrate the non-Gaussian nature of residuals for AR image models and that our algorithm compares favorably with the Kalman filtering techniques in such cases.

Bayesian decision feedback for segmentation of binary images.

We present real-time algorithms for the segmentation of binary images modeled by Markov mesh random fields (MMRFs) and corrupted by independent noise. The goal is to find a recursive algorithm to compute the maximum a posteriori (MAP) estimate of each pixel of the scene using a fixed lookahead of D rows and D columns of the observations. First, this MAP fixed-lag estimation problem is set up and the corresponding optimal recursive (but computationally complex) estimator is derived. Then, both hard and soft (conditional) decision feedbacks are introduced at appropriate stages of the optimal estimator to reduce the complexity. The algorithm is applied to several synthetic and real images. The results demonstrate the viability of the algorithm both complexity-wise and performance-wise, and show its subjective relevance to the image segmentation problem.

Using polygons to recognize and locate partially occluded objects.

We present computer vision algorithms that recognize and locate partially occluded objects. The scene may contain unknown objects that may touch or overlap giving rise to partial occlusion. The algorithms revolve around a generate-test paradigm. The paradigm iteratively generates and tests hypotheses for compatibility with the scene until it identifies all the scene objects. Polygon representations of the object's boundary guide the hypothesis generation scheme. Choosing the polygon representation turns out to have powerful consequences in all phases of hypothesis generation and verification. Special vertices of the polygon called ``corners'' help detect and locate the model in the scene. Polygon moment calculations lead to estimates of the dissimilarity between scene and model corners, and determine the model corner location in the scene. An association graph represents the matches and compatibility constraints. Extraction of the largest set of mutually compatible matches from the association graph forms a model hypothesis. Using a coordinate transform that maps the model onto the scene, the hypothesis gives the proposed model's location and orientation. Hypothesis verification requires checking for region consistency. The union of two polygons and other polygon operations combine to measure the consistency of the hypothesis with the scene. Experimental results give examples of all phases of recognizing and locating the objects.

Synthesis and estimation of random fields using long-correlation models.

Random field models have arisen in important applications in many diverse fields, including image synthesis and analysis. The automatic description and processing of images is limited by the types of underlying models that are hypothesized for the image. Hence, new random field models and associated techniques are developed. The new class of models includes the traditional simultaneous autoregressive and moving average models, but extends these to comprise a substantially larger new class. Thus, new image textures are representable with convenient small-order parameterizations. In particular, representations with significant correlations between distant locations are achieved. The algorithms for synthesis and estimation, however, require only about the same amount of time as the classical models. The development is facilitated by a new mathematical formulation suitable for problems involving high-dimensional transformations.

Scale preserving smoothing of polygons.

A smoother version of a polygon ¿ is defined as a polygon which approximates ¿ according to a given criterion and which simultaneously has no more edges than ¿ itself. In this paper, a scale preserving smoothing algorithm is presented. The input to the algorithm is a polygon ¿ and the output is its smoothed version ¿. ¿, which contains all the scale information that ¿ contains, is called the linear minimum perimeter polygon (LMPP) of ¿ within a tolerance of. Using the quantity the degree of with ¿ approximates ¿ can be controlled. From the LMPP a representation for a polygon approximating ¿ can be procured, which is invariant to scale and translation changes. Examples of smoothing maps and characters have been presented.

Optimal Choice of AR and MA Parts in Autoregressive Moving Average Models.

This paper deals with the Bayesian method of choosing the best model for a given one-dimensional series among a finite number of candidates belonging to autoregressive (AR), moving average (MA), ARMA, and other families. The series could be either a sequence of observations in time as in speech applications, or a sequence of pixel intensities of a two-dimensional image. The observation set is not restricted to be Gaussian. We first derive an optimum decision rule for assigning the given observation set to one of the candidate models so as to minimize the average probability of error in the decision. We also derive an optimal decision rule so as to minimize the average value of the loss function. Then we simplify the decision rule when the candidate models are different Gaussian ARMA models of different orders. We discuss the consistency of the optimal decision rule and compare it with the other decision rules in the literature for comparing dynamical models.

A geometrical approach to polygonal dissimilarity and shape matching.

Two geometrical measures have been proposed to quantify the dissimilarity between two irregular polygons. These measures capture the intuitive notion of the dissimilarity between shapes and are related to the minimum value of the intersecting area of the polygons on superposing one on the other in various configurations. A more easily computable measure of dissimilarity, referred to as the minimum integral square error between the polygons, has also been proposed, and using the latter measure pattern classification, has been performed. Experimental results involving the classification of the noisy boundaries of the four Great Lakes, Erie, Huron, Michigan, and Superior, using this measure, have been presented.

Syntactic decision rules for recognition of spoken words and phrases using a stochastic automaton.

This study deals with the design of a syntactic decision rule for recognizing an unknown utterance from a set X. The decision rule is expressed as a function of the character string (CS) derived from the test utterance. To obtain the CS, the waveform of the utterance is divided into a large number of frames of roughly equal duration numbered 1, 2,...,n. The ith symbol in the CS is the phonemic symbol obtained by subjecting the ith frame of the waveform to a relatively simple phoneme decision rule, the number of symbols in the CS being n. All the available nonacoustic information such as the lexicon of words in the set X, the possibility of confusion between different phonemes as seen by the phoneme decision rule, etc. is used in the design of the decision rule. The syntactic decision rule can be implemented by a stochastic finite state automaton involving limited memory and computation. The decision rule can also be interpreted as yielding the phrase x which minimizes a distance measure D(x, z) between the phrase x X and the observed CS z. We wili compare this approach with the other approaches such as the Viterbi methods, the distance approaches involving various types of distances, etc.