Mask-based single-pixel tracking and imaging for moving objects.

Tracking and imaging for high-speed moving objects have a wide range of application prospects in many fields, such as transportation and security monitoring. In this paper, the chrome plated masks are designed to carry geometric moment and random binary encoding patterns, combined with single pixel detectors, to achieve real-time tracking and imaging of fast-moving object. By using the geometric moment principle to obtain the motion trajectory of the object, coding sub-patterns and corresponding detection signals are extracted at different positions to reconstruct the image of the object. Multiple optical paths are established to avoid the side effects of motion error, and a dedicated calibration approach is proposed to improve the accuracy of tracking. The feasibility of the method is demonstrated by simulations and experiments. The proposed scheme, which modulates light with static mask instead of spatial light modulator (SLM), improves the speed and spectral range meanwhile reduces the system cost.

Imaging through a scattering medium via model-driven deep learning.

Imaging through a scattering medium is of great significance in many areas. Especially, speckle correlation imaging has been valued for its noninvasiveness. In this work, we report a deep learning solution that incorporates the physical model and an additional regularization for high-fidelity speckle correlation imaging. Without large-scale data to train, the physical model and regularization prior provide a correct direction for neural network to precisely reconstruct hidden objects from speckle under different scattering scenarios and noise levels. Experimental results demonstrate that the proposed method presents a significant advance in improving generalization and combating the invasion of noise.

High precision 3D imaging with timing corrected single photon LiDAR.

Single photon light detection and ranging (LiDAR) is an important technique for high precision long distance three-dimensional (3D) imaging. However, due to the effects and native limitations of system components, there exists ranging errors when using LiDAR system. For the LiDAR system that requires trigger detector to provide synchronization signals, the fluctuation of laser pulse energy causes the change of the initial time of the constant threshold triggered timing module, and subsequently leads to the ranging error. In this paper, we build a dual SPADs LiDAR system to avoid the ranging error caused by the fluctuation of laser pulse energy. By adding a reference optical path, the flight time of signal photons is corrected by reference photons, so as to realize the correction of ranging. A series of experiments demonstrate that the proposed LiDAR system has the capability of high precision ranging and 3D imaging. The system achieves range of error of 0.15 mm and range resolution of 0.3 mm at a distance of 29 m.

Non-line-of-sight imaging based on an untrained deep decoder network.

In recent years, low-cost high-quality non-line-of-sight (NLOS) imaging by a passive light source has been a significant research dimension. Here, we report a new, to the best of our knowledge, reconstruction method for the well-known "occluder-aided" NLOS imaging configuration based on an untrained deep decoder network. Using the interaction between the neural network and the physical forward model, the network weights can be automatically updated without the need for training data. Completion of the optimization process facilitates high-quality reconstructions of hidden scenes from photographs of a blank wall under high ambient light conditions. Simulations and experiments show the superior performance of the proposed method in terms of the details and the robustness of the reconstructed images. Our method will further promote the practical application of NLOS imaging in real scenes.

New strategy for high-dimensional single-pixel imaging.

Single-pixel imaging (SPI) technique has been studied intensively due to its minimum requirement for the detector resolution and the equipment costs. In this work, we proposed a new strategy of the SPI to explore its capability in high-dimensional imaging, which is the first comprehensive scheme as we know to achieve calibration, color texture and viewpoint expansion of single-pixel three-dimensional imaging. We realized a low-cost single-pixel three-dimensional imaging scheme which employ a raster scanner to provide the structured illumination and a grating to encode the height information. In order to reduce the blocking area, we introduce two single-pixel detectors (SPDs) to detect from two detection angles, a modified total variation based criterion is proposed to fuse the height information from two SPDs and reduce the error of shape fusion. To acquire the information of higher dimension, we introduce the third SPD aims to gain the color texture, three bandpass filter is placed in front of three SPDs, respectively, to collect different color information. Meanwhile a viewpoint switching method inspired by the shape from shading theory is presented to improve the color fidelity. Our study is expected to provide a demonstration for SPI in acquisition, reconstruction, and fusion of high-dimensional image data.

Cryptographic analysis on an optical random-phase-encoding cryptosystem for complex targets based on physics-informed learning.

Optical cryptanalysis based on deep learning (DL) has grabbed more and more attention. However, most DL methods are purely data-driven methods, lacking relevant physical priors, resulting in generalization capabilities restrained and limiting practical applications. In this paper, we demonstrate that the double-random phase encoding (DRPE)-based optical cryptosystems are susceptible to preprocessing ciphertext-only attack (pCOA) based on DL strategies, which can achieve high prediction fidelity for complex targets by using only one random phase mask (RPM) for training. After preprocessing the ciphertext information to procure substantial intrinsic information, the physical knowledge DL method based on physical priors is exploited to further learn the statistical invariants in different ciphertexts. As a result, the generalization ability has been significantly improved by increasing the number of training RPMs. This method also breaks the image size limitation of the traditional COA method. Optical experiments demonstrate the feasibility and the effectiveness of the proposed learning-based pCOA method.

A Multi-Image Encryption Based on Sinusoidal Coding Frequency Multiplexing and Deep Learning.

Multi-image encryption technology is a vital branch of optical encryption technology. The traditional encryption method can only encrypt a small number of images, which greatly restricts its application in practice. In this paper, a new multi-image encryption method based on sinusoidal stripe coding frequency multiplexing and deep learning is proposed to realize the encryption of a greater number of images. In the process of encryption, several images are grouped, and each image in each group is first encoded with a random matrix and then modulated with a specific sinusoidal stripe; therefore, the dominant frequency of each group of images can be separated in the Fourier frequency domain. Each group is superimposed and scrambled to generate the final ciphertext. In the process of decryption, deep learning is used to improve the quality of decrypted image and the decryption speed. Specifically, the obtained ciphertext can be sent into the trained neural network and then the plaintext image can be reconstructed directly. Experimental analysis shows that when 32 images are encrypted, the CC of the decrypted result can reach more than 0.99. The efficiency of the proposed encryption method is proved in terms of histogram analysis, adjacent pixels correlation analysis, anti-noise attack analysis and resistance to occlusion attacks analysis. The encryption method has the advantages of large amount of information, good robustness and fast decryption speed.

Texture mapping based on photogrammetric reconstruction of the coded markers.

Texture mapping is one of the key procedures to generate photorealistic three-dimensional (3D) models. To avoid dependence on the features of the texture and the geometric model, coded markers are introduced as the control points to assist the texture mapping. Multiple texture images containing the markers are captured, and the 3D coordinates of the markers are reconstructed with photogrammetry; meanwhile, the parameters of the texture camera are optimized with the bundle adjustment strategy. Then the pose parameters of the texture mapping can be calculated with the assistance of the marker registration and the iterative closest point (ICP) algorithms. The validity of the proposed algorithm is demonstrated with an experiment on an ordinary object.

Singular value decomposition ghost imaging.

The singular value decomposition ghost imaging (SVDGI) is proposed to enhance the fidelity of computational ghost imaging (GI) by constructing a measurement matrix using singular value decomposition (SVD) transform. After SVD transform on a random matrix, the non-zero elements of singular value matrix are all made equal to 1.0, then the measurement matrix is acquired by inverse SVD transform. Eventually, the original objects can be reconstructed by multiplying the transposition of the matrix by a series of collected intensity. SVDGI enables the reconstruction of an N-pixel image using much less than N measurements, and perfectly reconstructs original object with N measurements. Both the simulated and the optical experimental results show that SVDGI always costs less time to accomplish better works. Firstly, it is at least ten times faster than GI and differential ghost imaging (DGI), and several orders of magnitude faster than pseudo-inverse ghost imaging (PGI). Secondly, in comparison with GI, the clarity of SVDGI can get sharply improved, and it is more robust than the other three methods so that it yields a clearer image in the noisy environment.

Suppression of the nonlinear phase error in phase shifting profilometry: considering non-smooth reflectivity and fractional period.

Hilbert transform (HT) has been employed to compensate phase error arising from the nonlinear effect in phase shifting profilometry (PSP). However, in most common situations, pure HT may lead to a significant system error, which has a negative impact on subsequent phase error compensation. In this paper, system error from HT of non-stationary and non-continuous fringe is analyzed, and then a novel phase error suppression approach is presented. The cosine fringe without direct current (DC) component is reconstructed to eliminate the influence of non-smooth reflectivity, and the fractional periods at both ends of the reconstructed fringe are extended to generate fringe with integer number of periods. And then the HT is applied to the reconstructed and extended fringe. Finally, a revised phase-shifting algorithm is employed to calculate the phase with the fringe after HT. The proposed approach is suitable for PSP of the surface with non-smooth reflectivity (e.g. texture of complex colors), which is demonstrated in a series of experiments.

General model for phase shifting profilometry with an object in motion.

When implementing the phase shifting profilometry to reconstruct an object, the object is always required to be kept stable as multiple fringe patterns are required. Movement during the measurement will cause failed reconstruction. This paper proposes a general model describing the fringe patterns with any three-dimensional movement based on phase shifting profilometry. The object movement is classified as five types and their characteristics are analyzed respectively. Then, by introducing a virtual plane, the influence on the phase value caused by different types of movement is described mathematically and a new model including movement information is proposed. At last, with the help of the movement tracking and least-square algorithm, the moving object is reconstructed with high accuracy. The proposed method can remove the reference plane during the reconstruction of the moving object, which extends the application range of the phase shifting profilometry. The effectiveness of the proposed model is verified by the experiments.

Automated approach for the surface profile measurement of moving objects based on PSP.

Phase shifting profilometry can achieve high accuracy for the 3D shape measurement of static object. Errors will be introduced when the object is moved during the movement. The fundamental reason causing the above issue is: PSP requires multiple fringe patterns but the reconstruction model does not include the object movement information. This paper proposes a new method to automatically measure the 3D shape of the rigid object with arbitrary 2D movement. Firstly, the object movement is tracked by the SIFT algorithm and the rotation matrix and translation vector describing the movement are estimated. Then, with the reconstruction model including movement information, a least-square algorithm is applied to retrieve the correct phase value. The proposed method can significantly reduce the errors caused by the object movement. The whole reconstruction process does not need human intervention and the proposed method has high potential to be applied in industrial applications. Experiments are presented to verify the effectiveness.

Improved performance of multi-view fringe projection 3D microscopy.

Fringe projection 3D microcopy (FP-3DM) plays an increasingly important role in micro manufacturing and measurement. In recent decades, research on FP-3DM has made considerable progress. Nevertheless, some disadvantages arising from the limited depth of field, local specular reflection and occlusion still exist and need to be further addressed. In this paper, a multi-view FP-3DM (MVFP-3DM) is presented. Four imaging branches with the Scheimpflug condition and one vertical projection branch are deployed to establish the system. The system is described with a general imaging model, which is independent of the system configuration. In system calibration, the edge of binary fringe is used to locate the benchmark, which takes advantage of the fact that the edge will keep its position whether it is in focus or out of focus. Furthermore, a group of experiments prove that our proposed MVFP-3DM system can extend measurable range in depth, improve precision in 3D reconstruction and reduce occlusion.

Structured light field 3D imaging.

In this paper, we propose a method by means of light field imaging under structured illumination to deal with high dynamic range 3D imaging. Fringe patterns are projected onto a scene and modulated by the scene depth then a structured light field is detected using light field recording devices. The structured light field contains information about ray direction and phase-encoded depth, via which the scene depth can be estimated from different directions. The multidirectional depth estimation can achieve high dynamic 3D imaging effectively. We analyzed and derived the phase-depth mapping in the structured light field and then proposed a flexible ray-based calibration approach to determine the independent mapping coefficients for each ray. Experimental results demonstrated the validity of the proposed method to perform high-quality 3D imaging for highly and lowly reflective surfaces.

Shadow removal method for phase-shifting profilometry.

In a typical phase-shifting profilometry system for the three-dimensional (3D) shape measurement, shadows often exist in the captured images as the camera and projector probe the object from different directions. The shadow areas do not reflect the fringe patterns which will cause errors in the measurement results. This paper proposed a new method to remove the shadow areas from taking part in the 3D measurement. With the system calibrated and the object reconstructed, the 3D results are mapped on a point-by-point basis into the corresponding positions on the digital micro-mirror device (DMD) of the projector. A set of roles are presented to detect the shadow points based on their mapped positions on the DMD plane. Experimental results are presented to verify the effectiveness of the proposed method.

Fringe projection 3D microscopy with the general imaging model.

Three-dimensional (3D) imaging and metrology of microstructures is a critical task for the design, fabrication, and inspection of microelements. Newly developed fringe projection 3D microscopy is presented in this paper. The system is configured according to camera-projector layout and long working distance lenses. The Scheimpflug principle is employed to make full use of the limited depth of field. For such a specific system, the general imaging model is introduced to reach a full 3D reconstruction. A dedicated calibration procedure is developed to realize quantitative 3D imaging. Experiments with a prototype demonstrate the accessibility of the proposed configuration, model, and calibration approach.

Distortion correction for microscopic fringe projection system with Scheimpflug telecentric lens.

To increase the measurement range of 3D microscopy, Scheimpflug adjustment, in which the imaging plane is tilted with respect to the telecentric lens plane, is often employed. However, the inclined imaging plane will introduce certain distortion to the captured image, which further affects the accuracy of the 3D reconstruction result. In this paper, a distortion model was derived based on the geometric optics theory. With it, the imaging distortion caused by the Scheimpflug condition can be effectively corrected. Experimental results will be presented to demonstrate the feasibility and validity of the proposed method.

Dynamic 3D imaging based on acousto-optic heterodyne fringe interferometry.

An acoustic-optics heterodyne fringe interferometry coupled with a three-camera system is developed for dynamic 3D imaging. In this system, first-order beams with a slight frequency difference diffracted from two acousto-optic deflectors (AODs) form a beat intensity fringe pattern. Setting the frequency of the trigger signal for the CCD cameras into four times the beat frequency, four-step phase-shifting fringe patterns can be obtained, and the wrapped phase map (WPM) can be calculated. Under the epipolar constraint among three cameras, the homologous points can be determined unambiguously with the assistant of a WPM; thus the 3D shape can be reconstructed while skipping the phase unwrapping step. Experimental results are presented to validate this approach.

Calibration of fringe projection profilometry with bundle adjustment strategy.

The measurement accuracy of fringe projection profilometry (FPP) largely depends on the calibration procedure. A more reliable calibration approach based on the stereo vision model of the FPP scheme in conjunction with the bundle adjustment strategy is presented. It can adjust the coordinates of benchmarks and thereby estimate the scheme parameters more accurately even with an imperfect target. The experiment results shows that the proposed approach can reach highly accurate calibration by solely using a printed target pattern, which verifies the proposed approach.

Optical measurement network for large-scale and shell-like objects.

An optical measurement method for large-scale and shell-like objects is proposed and is verified by experiments. The underlying concept is a model-based optical measurement network consisting of multinode three-dimensional (3D) sensors. To achieve this, a synthetic calibration method is presented to enable the measurement. A phase-aided active stereoscopy is thus applied to each node sensor for acquiring partial range images from different viewpoints. The multiple range images are then registered to obtain a 3D reconstructed model, which is compared with the computer-aided design (CAD) model to quantitatively reveal the differences between the two models. Experiment results are also presented to validate the proposed approach.