Shape and Reflectance Reconstruction Using Concentric Multi-Spectral Light Field.

Recovering the shape and reflectance of non-Lambertian surfaces remains a challenging problem in computer vision since the view-dependent appearance invalidates traditional photo-consistency constraint. In this paper, we introduce a novel concentric multi-spectral light field (CMSLF) design that is able to recover the shape and reflectance of surfaces of various materials in one shot. Our CMSLF system consists of an array of cameras arranged on concentric circles where each ring captures a specific spectrum. Coupled with a multi-spectral ring light, we are able to sample viewpoint and lighting variations in a single shot via spectral multiplexing. We further show that our concentric camera and light source setting results in a unique single-peak pattern in specularity variations across viewpoints. This property enables robust depth estimation for specular points. To estimate depth and multi-spectral reflectance map, we formulate a physics-based reflectance model for the CMSLF under the surface camera (S-Cam) representation. Extensive synthetic and real experiments show that our method outperforms the state-of-the-art shape reconstruction methods, especially for non-Lambertian surfaces.

Hybrid sensing face detection and registration for low-light and unconstrained conditions.

The capability to track, detect, and identify human targets in highly cluttered scenes under extreme conditions, such as in complete darkness or on the battlefield, has been one of the primary tactical advantages in military operations. In this paper, we propose a new collaborative, multi-spectrum sensing method to achieve face detection and registration under low-light and unconstrained conditions. We design and prototype a novel type of hybrid sensor by combining a pair of near-infrared (NIR) cameras and a thermal camera (a long-wave infrared camera). We strategically surround each NIR sensor with a ring of LED IR flashes to capture the "red-eye," or more precisely, the "bright-eye" effect of the target. The "bright-eyes" are used to localize the 3D position of eyes and face. The recovered 3D information is further used to warp the thermal face imagery to a frontal-parallel pose so that additional tasks, such as face recognition, can be reliably conducted, especially with the assistance of accurate eye locations. Experiments on real face images are provided to demonstrate the merit of our method.

Face recognition performance with superresolution.

With the prevalence of surveillance systems, face recognition is crucial to aiding the law enforcement community and homeland security in identifying suspects and suspicious individuals on watch lists. However, face recognition performance is severely affected by the low face resolution of individuals in typical surveillance footage, oftentimes due to the distance of individuals from the cameras as well as the small pixel count of low-cost surveillance systems. Superresolution image reconstruction has the potential to improve face recognition performance by using a sequence of low-resolution images of an individual's face in the same pose to reconstruct a more detailed high-resolution facial image. This work conducts an extensive performance evaluation of superresolution for a face recognition algorithm using a methodology and experimental setup consistent with real world settings at multiple subject-to-camera distances. Results show that superresolution image reconstruction improves face recognition performance considerably at the examined midrange and close range.

Super-resolution for flash ladar imagery.

Flash ladar systems are compact devices with high frame rates that hold promise for robotics applications, but these devices suffer from poor spatial resolution. This work develops a wavelet preprocessing stage to enhance registration of multiple frames and applies super-resolution to improve the resolution of flash ladar range imagery. The triangle orientation discrimination methodology was used for a subjective evaluation of the effectiveness of super-resolution for flash ladar. Results show statistically significant increases in the probability of target discrimination at all target ranges, as well as a reduction in subject response times for super-resolved imagery.

Adaptive deblurring of noisy images.

We propose a practical sensor deblurring filtering method for images that are contaminated with noise. A sensor blurring function is usually modeled via a Gaussian-like function having a bell shape. The straightforward inverse function results in the magnification of noise at high frequencies. To address this issue, we apply a special spectral window to the inverse blurring function. This special window is called the power window, which is a Fourier-based smoothing window that preserves most of the spatial frequency components in the passband and attenuates quickly at the transition band. The power window is differentiable at the transition point, which gives a desired smooth property and limits the ripple effect. Utilizing the properties of the power window, we design the deblurring filter adaptively by estimating the energy of the signal and the noise of the image to determine the passband and the transition band of the filter. The deblurring filter design criteria are (a) the filter magnitude is less than 1 at the frequencies where the noise is stronger than the desired signal (the transition band), and (b) the filter magnitude is greater than 1 at the other frequencies (the passband). Therefore the adaptively designed deblurring filter is able to deblur the image by a desired amount based on the estimated or known blurring function while suppressing the noise in the output image. The deblurring filter performance is demonstrated by a human perception experiment in which 10 observers are to identify 12 military targets with 12 aspect angles. The results of comparing target identification probabilities with blurred and deblurred images and adding two levels of noise to blurred and deblurred noisy images are reported.

Superresolution image reconstruction from a sequence of aliased imagery.

We present a superresolution image reconstruction from a sequence of aliased imagery. The subpixel shifts (displacement) among the images are unknown due to the uncontrolled natural jitter of the imager. A correlation method is utilized to estimate subpixel shifts between each low-resolution aliased image with respect to a reference image. An error-energy reduction algorithm is derived to reconstruct the high-resolution alias-free output image. The main feature of this proposed error-energy reduction algorithm is that we treat the spatial samples from low-resolution images that possess unknown and irregular (uncontrolled) subpixel shifts as a set of constraints to populate an oversampled (sampled above the desired output bandwidth) processing array. The estimated subpixel locations of these samples and their values constitute a spatial domain constraint. Furthermore, the bandwidth of the alias-free image (or the sensor imposed bandwidth) is the criterion used as a spatial frequency domain constraint on the oversampled processing array. The results of testing the proposed algorithm on the simulated low- resolution forward-looking infrared (FLIR) images, real-world FLIR images, and visible images are provided. A comparison of the proposed algorithm with a standard interpolation algorithm for processing the simulated low-resolution FLIR images is also provided.