Absorption-based hyperspectral thermal ranging: performance analyses, optimization, and simulations.

The wavelength dependence of atmospheric absorption creates range cues in hyperspectral measurements that can be exploited for passive ranging using only thermal emissions. In this work, we present fundamental limits on absorption-based ranging under a model of known air temperature and wavelength-dependent attenuation coefficient, with object temperature and emissivity unknown; reflected solar and environmental radiance is omitted from our analysis. Fisher information computations illustrate how performance limits depend on atmospheric conditions such as air temperature and humidity; temperature contrast in the scene; spectral resolution of measurement; and distance. These results should prove valuable in sensor system design.

Learning Time-multiplexed phase-coded apertures for snapshot spectral-depth imaging.

Depth and spectral imaging are essential technologies for a myriad of applications but have been conventionally studied as individual problems. Recent efforts have been made to optically encode spectral-depth (SD) information jointly in a single image sensor measurement, subsequently decoded by a computational algorithm. The performance of single snapshot SD imaging systems mainly depends on the optical modulation function, referred to as codification, and the computational methods used to recover the SD information from the coded measurement. The optical modulation has been conventionally realized using coded apertures (CAs), phase masks, prisms or gratings, active illumination, and many others. In this work, we propose an optical modulation (codification) strategy that employs a color-coded aperture (CCA) in conjunction with a time-varying phase-coded aperture and a spatially-varying pixel shutter, thus yielding an effective time-multiplexed coded aperture (TMCA). We show that the proposed TMCA entails a spatially-variant point spread function (PSF) for a constant depth in a scene, which, in turn, facilitates the distinguishability, and therefore, better recovery of the depth information. Further, the selective filtering of specific spectral bands by the CCA encodes relevant spectral information that is disentangled using a reconstruction algorithm. We leverage the advances of deep learning techniques to jointly learn the optical modulation and the computational decoding algorithm in an end-to-end (E2E) framework. We demonstrate via simulations and with a real testbed prototype that the proposed TMCA strategy outperforms state-of-the-art snapshot SD imaging alternatives in both spectral and depth reconstruction quality.

Non-line-of-sight snapshots and background mapping with an active corner camera.

The ability to form reconstructions beyond line-of-sight view could be transformative in a variety of fields, including search and rescue, autonomous vehicle navigation, and reconnaissance. Most existing active non-line-of-sight (NLOS) imaging methods use data collection steps in which a pulsed laser is directed at several points on a relay surface, one at a time. The prevailing approaches include raster scanning of a rectangular grid on a vertical wall opposite the volume of interest to generate a collection of confocal measurements. These and a recent method that uses a horizontal relay surface are inherently limited by the need for laser scanning. Methods that avoid laser scanning to operate in a snapshot mode are limited to treating the hidden scene of interest as one or two point targets. In this work, based on more complete optical response modeling yet still without multiple illumination positions, we demonstrate accurate reconstructions of foreground objects while also introducing the capability of mapping the stationary scenery behind moving objects. The ability to count, localize, and characterize the sizes of hidden objects, combined with mapping of the stationary hidden scene, could greatly improve indoor situational awareness in a variety of applications.

Compressive spectral image fusion via a single aperture high throughput imaging system.

Spectral image fusion techniques combine the detailed spatial information of a multispectral (MS) image and the rich spectral information of a hyperspectral (HS) image into a high-spatial and high-spectral resolution image. Due to the data deluge entailed by such images, new imaging modalities have exploited their intrinsic correlations in such a way that, a computational algorithm can fuse them from few multiplexed linear projections. The latter has been coined compressive spectral image fusion. State-of-the-art research work have focused mainly on the algorithmic part, simulating instrumentation characteristics and assuming independently registered sensors to conduct compressed MS and HS imaging. In this manuscript, we report on the construction of a unified computational imaging framework that includes a proof-of-concept optical testbed to simultaneously acquire MS and HS compressed projections, and an alternating direction method of multipliers algorithm to reconstruct high-spatial and high-spectral resolution images from the fused compressed measurements. The testbed employs a digital micro-mirror device (DMD) to encode and split the input light towards two compressive imaging arms, which collect MS and HS measurements, respectively. This strategy entails full light throughput sensing since no light is thrown away by the coding process. Further, different resolutions can be dynamically tested by binning the DMD and sensors pixels. Real spectral responses and optical characteristics of the employed equipment are obtained through a per-pixel point spread function calibration approach to enable accurate compressed image fusion performance. The proposed framework is demonstrated through real experiments within the visible spectral range using as few as 5% of the data.

Compressive Spectral Light Field Image Reconstruction via Online Tensor Representation.

In recent years there has been an increasing interest in sensing devices that capture multidimensional information such as the spectral light field (SLF) images, which are 5-dimensional (5D) representations of a scene including 2D spatial, 2D angular and 1D spectral information. Spatio-spectral and angular information plays an important role in modern applications spanning from microscopy to computer vision. However, SLF sensors use expensive beam-splitters or cameras arrays placed in tandem, which split the sensing problem in two time consuming and independent tasks: spectral and light field imaging tasks. This work proposes a compressive spectral light field imaging architecture that builds on the principles of the compressive imaging framework, to capture multiplexed representations of the multidimensional information, so that, less measurements are required to capture the SLF data cube. Alongside, we propose a computational algorithm to recover the 5D information from the compressed measurements, exploiting the inherent high correlations within the SLF by treating them as 3D tensors. Furthermore, exploiting the geometry properties of the proposed optical architecture, the Tucker decomposition is applied to the set of compressed measurements, so that, an ad-hoc dictionary-like image representation basis is calculated online. This in turn, entails a more accurate reconstruction of the SLF since the dictionary fits the specific characteristics of the image itself. We demonstrate through simulations over three SLF datasets captured in our laboratory, and an experimental proof-of-concept implementation, that the proposed compressive imaging device together with the proposed computational algorithm represent an efficient alternative to capture SLF, compared to conventional methods that employ either side-information or multiple sensors. Also, we show that the tensor-based proposed algorithm exhibits a lower computational complexity than the matrix-based state of the art counterparts, thus enabling fast processing of multidimensional images.

Snapshot Compressive ToF+Spectral Imaging via Optimized Color-Coded Apertures.

Compressive multispectral imaging systems comprise a new generation of spectral imagers that capture coded projections of a scene where spectral data cubes are reconstructed computationally. Separately, time-of-flight (ToF) cameras obtain 2D range images where each pixel records the distance from the camera sensor to the target surface. The demand for these imaging modalities is rapidly increasing, and thus, there is strong interest in developing new image sensors that can simultaneously acquire multispectral-color-and-depth imagery (MS+D) using a single aperture. Work in this path has been mainly developed via RGB+D imaging. However, in RGB+D, the multispectral image is limited to three spectral channels, and the imaging system often relies on two image sensors. We recently proposed a compressive MS+D imaging device that used a digital-micromirror-device, requiring a bulky double imaging-and-relay path. To overcome the bulkiness and other difficulties of our previous imaging system, this work presents a more-compact MS+D imaging device with snapshot capabilities. It provides better spectral sensing, relying on a static color-coded-aperture (CCA) and a ToF sensor. To guarantee good quality in the recovery, we develop an optimization method for CCA based-on blue-noise-multitoning, solved via the direct-binary-search algorithm. A testbed-setup is reported along with simulated and real experiments that demonstrate the MS+D capabilities of the proposed system over static and dynamic scenes.

Prediction of the total acid number (TAN) of colombian crude oils via ATR-FTIR spectroscopy and chemometric methods.

Reactions as the attack by naphthenic and hydrogen sulfide have caused corrosion problems in the petroleum industry due to they affect the crude oil heating furnaces and distillation towers at temperatures between 220 and 400 °C. The total acid number (TAN) measurement has been used as a test to quantify the acid compounds in crude oils and has shown to be a reliable indicator of their corrosion degree. However, the standard method for the TAN measurement, ASTM D-644, involves long times, environment unfriendly wastes and high costs for each analysis. A more appropriate method for the TAN determination is implemented in this paper, by correlating Fourier transform infrared spectroscopy (FTIR) spectral data of the samples with the standard method measurements using multivariate regression models. In particular, the intensities and frequencies of their mid-infrared attenuated total reflectance (MIR-ATR) spectra (4000 - 400 cm-1) are used as independent variables of several principal component regression (PCR) and partial least squares regression (PLSR) models. The latter are employed to correlate the spectra with their respective TAN values so as to obtain a suitable prediction model. Twenty-six (26) samples of Colombian crude oils are used for the study with a TAN ranging from 0.1 to 6.8 mg KOH/g crude oil (ASTM D-664). The models are evaluated according to the coefficient of determination (R2), the root mean square error of calibration (RMSEC) and of prediction (RMSEP). The best model is obtained via PLSR using as few as four components (i.e. factors), which attains a calibration R2 of 0.981 and an RMSEC of 0.317 mg KOH/g crude oil, while for prediction it attains an R2 of 0.996 and an RMSEP of 0.160 mg KOH/g crude oil. It is observed that the functional groups COOH, CH3 and CH2 contribute the most to the prediction models. The designed methodology is faster and environmentally friendly since it does not require sample pretreatment and the use of toxic reagents, and of low-cost compared with the standard procedure since FTIR measurements can be easily taken anywhere using a hand-held or portable spectrometer and a laptop.

Sensing Matrix Design for Compressive Spectral Imaging via Binary Principal Component Analysis.

Compressive spectral imaging (CSI) is a framework that captures coded-and-multiplexed low-dimensional projections of spectral data-cubes. In general, the sensing process in many CSI architectures is described using binary matrices, so-called sensing/projection matrices, whose elements can be either random or designed. However, some characteristics of the spectral data, such as the ℓ2-norm or the second moment statistics, can be lost when this dimensionality reduction is performed. Similarly, principal component analysis (PCA) is a data dimensionality reduction technique that minimizes the least-squared error between the spectral data and its low-dimensional projection, but preserving its structure or variance. Thus, PCA can be used to guide the CSI acquisition process by designing the binary sensing matrix. Nonetheless, PCA requires to know the spectral image a-priori, and also, its associated projection matrix is not binary, as required by CSI optical architectures. Therefore, in this paper, an algorithm to design CSI sensing matrices by exploiting the structure-preserving property of the PCA projection is proposed. First, a set of compressive measurements obtained with random sensing matrices is used to rapidly estimate the covariance matrix associated with the spectral data. Then, a new sensing matrix is designed by solving a non-convex optimization problem that finds a set of binary vectors that approximate the principal components of the covariance matrix, thus maximizing the explanation of the data variance. Experimental results show an improvement of up to 3 dB in image reconstruction quality, in terms of the peak signal to noise ratio (PSNR), when the binary PCA-based sensing matrices are used and compared with conventional random sensing matrices and state-of-art designed matrices based on PCA.