Light field camera calibration and point spread function calculation based on differentiable ray tracing.

The imaging process of the light field (LF) camera with a micro-lens array (MLA) may suffer from multiple aberrations. It is thus difficult to precisely calibrate the intrinsic hardware parameters and calculate the corresponding point spread function (PSF). To build an aberration-aware solution with better generalization, we propose an end-to-end imaging model based on the differentiable ray tracing. The input end is the point source location, and the output end is the rendered LF image, namely, PSF. Specially, a projection method is incorporated into the imaging model, eliminating the huge memory overhead induced by a large array of periodic elements. Taking captured PSF images as the ground truth, the LF camera is calibrated with the genetic algorithm initially and then the gradient-based optimization. This method is promising to be used in various LF camera applications, especially in challenging imaging conditions with severe aberrations.

Design and evaluation of a light-field multi-wavelength pyrometer.

This letter describes the design and implementation of a multi-wavelength light-field pyrometer, where six-channel radiation images were captured with one CMOS sensor. Such capability is achieved by placing a 2 × 3 filter array in front of the main lens of an unfocused light-field camera, such that discrete wavelength and radiation intensity can be simultaneously recorded. It demonstrates that through black-body furnace experiments, how multi-channel radiation images can be extracted from one raw light-field multispectral image, and how accurate 2D temperature distribution can be recovered by optimization algorithms.

Light-field multi-spectral radiation thermometry.

This Letter proposes light-field multi-spectral radiation thermometry based on an unfocused light-field camera, which can simultaneously record directions and intensities of incident rays. In this method, the direction information of rays is substituted by radiation spectrums via placing an array of filters in front of camera main lens, such that the image sensor can simultaneously acquire spectrums and intensities of rays. By decoupling a raw multi-spectral light-field (MSLF) image and utilizing traditional multi-wavelength pyrometer algorithms, the scalar field of surface temperature distribution can be achieved. To verify the method, measurement errors of different temperature levels on several typical areas of MSLF images are analyzed. In addition, the validation experiment demonstrates that accurate surface temperature measurement can be achieved with a single lens, single monochromatic image sensor, and just one snapshot in the proposed method.

General model for phase shifting profilometry with an object in motion.

When implementing the phase shifting profilometry to reconstruct an object, the object is always required to be kept stable as multiple fringe patterns are required. Movement during the measurement will cause failed reconstruction. This paper proposes a general model describing the fringe patterns with any three-dimensional movement based on phase shifting profilometry. The object movement is classified as five types and their characteristics are analyzed respectively. Then, by introducing a virtual plane, the influence on the phase value caused by different types of movement is described mathematically and a new model including movement information is proposed. At last, with the help of the movement tracking and least-square algorithm, the moving object is reconstructed with high accuracy. The proposed method can remove the reference plane during the reconstruction of the moving object, which extends the application range of the phase shifting profilometry. The effectiveness of the proposed model is verified by the experiments.

Automated approach for the surface profile measurement of moving objects based on PSP.

Phase shifting profilometry can achieve high accuracy for the 3D shape measurement of static object. Errors will be introduced when the object is moved during the movement. The fundamental reason causing the above issue is: PSP requires multiple fringe patterns but the reconstruction model does not include the object movement information. This paper proposes a new method to automatically measure the 3D shape of the rigid object with arbitrary 2D movement. Firstly, the object movement is tracked by the SIFT algorithm and the rotation matrix and translation vector describing the movement are estimated. Then, with the reconstruction model including movement information, a least-square algorithm is applied to retrieve the correct phase value. The proposed method can significantly reduce the errors caused by the object movement. The whole reconstruction process does not need human intervention and the proposed method has high potential to be applied in industrial applications. Experiments are presented to verify the effectiveness.