Parameter selection on a multi-exposure fusion method for measuring surfaces with varying reflectivity in microscope fringe projection profilometry.

As industrial and scientific advancements continue, the demand for precise measurement of three-dimensional (3D) shapes and surfaces is steadily increasing. However, accurate 3D measurement of certain surfaces, especially those with varying reflectivities, has always been a challenging issue. Multi-exposure fusion methods have shown stable, high-quality measurement results, but the selection of parameters for these methods has largely been based on experience. To address this issue, this paper has improved the multi-exposure fusion method and introduced a guided approach for parameter selection, significantly enhancing the completeness of measurement results. Additionally, a comparative model is developed to experimentally validate the specific impacts of Gaussian window variance, optimal grayscale range, and attenuation factor variance on the integrity of 3D reconstruction. The experimental results demonstrate that under the guidance of the parameter adjustment method proposed in this paper, the multi-exposure fusion for measuring the 3D topography of high-dynamic surfaces improves the restoration coverage from the original 86% (bright areas) and 50% (dark areas) to over 99%. This provides a selection strategy for parameter adjustment guidance in precise measurements based on the multi-exposure method.

Rice's trajectory from wild to domesticated in East Asia.

Rice (Oryza sativa) serves as a staple food for more than one-third of the global population. However, its journey from a wild gathered food to domestication remains enigmatic, sparking ongoing debates in the biological and anthropological fields. Here, we present evidence of rice phytoliths sampled from two archaeological sites in China, Shangshan and Hehuashan, near the lower reaches of the Yangtze River. We demonstrate the growth of wild rice at least 100,000 years before present, its initial exploitation as a gathered resource at about 24,000 years before present, its predomestication cultivation at about 13,000 years before present, and eventually its domestication at about 11,000 years before present. These developmental stages illuminate a protracted process of rice domestication in East Asia and extend the continuous records of cereal evolution beyond the Fertile Crescent.

NMSCANet: stereo matching network for speckle variations in single-shot speckle projection profilometry.

In single-shot speckle projection profilometry (SSPP), the projected speckle inevitably undergoes changes in shape and size due to variations such as viewing angles, complex surface modulations of the test object and different projection ratios. These variations introduce randomness and unpredictability to the speckle features, resulting in erroneous or missing feature extraction and subsequently degrading 3D reconstruction accuracy across the tested surface. This work strives to explore the relationship between speckle size variations and feature extraction, and address the issue solely from the perspective of network design by leveraging specific variations in speckle size without expanding the training set. Based on the analysis of the relationship between speckle size variations and feature extraction, we introduce the NMSCANet, enabling the extraction of multi-scale speckle features. Multi-scale spatial attention is employed to enhance the perception of complex and varying speckle features in space, allowing comprehensive feature extraction across different scales. Channel attention is also employed to selectively highlight the most important and representative feature channels in each image, which is able to enhance the detection capability of high-frequency 3D surface profiles. Especially, a real binocular 3D measurement system and its digital twin with the same calibration parameters are established. Experimental results imply that NMSCANet can also exhibit more than 8 times the point cloud reconstruction stability (Std) on the testing set, and the smallest change range in terms of Mean~dis (0.0614 mm - 0.4066 mm) and Std (0.0768 mm - 0.7367 mm) when measuring a standard sphere and plane compared to other methods, faced with the speckle size changes, meanwhile NMSCANet boosts the disparity matching accuracy (EPE) by over 35% while reducing the matching error (N-PER) by over 62%. Ablation studies and validity experiments collectively substantiate that our proposed modules and constructed network have made significant advancements in enhancing network accuracy and robustness against speckle variations.

Flexible and accurate system calibration method in microscopic fringe projection profilometry.

The three-dimensional (3D) measurement task of complex microstructures holds paramount significance in the domains of precision manufacturing and inspection. The calibration of the 3D system heavily determines the final reconstruction accuracy. The widely adopted system calibration method is phase-height mapping (PHM) and stereo vision (SV) based. The former can be applied directly to the calculation without considering the imaging model of the system, but it relies on highly precise and expensive translation stages or standard blocks. The latter's accuracy cannot be guaranteed because it is difficult to accurately calibrate the projector. In this paper, we establish an optically coupled microscopic fringe projection profilometry system that consists of a Scheimpflug pinhole projector and a super-low distortion bi-telecentric camera. We introduce a simplified 3D system calibration approach that combines phase modulation transfer and ray propagation. Our method enables the simultaneous calibration of the system, including the calibration of the projector, camera, and the phase to a 3D coordinates relationship, using only a 2D target. The calibrated projector's external parameters are used to obtain the target's complete poses, and then the direct mapping coefficients of the phase to the 3D coordinates can be obtained through the optical geometry structure and phase labels. Comparable experiments verify the feasibility of the proposed method.

Deep learning-based end-to-end 3D depth recovery from a single-frame fringe pattern with the MSUNet++ network.

Deep learning (DL)-based single-frame fringe pattern to 3D depth reconstruction methods have aroused extensive research interest. The goal is to estimate high-precision 3D shape from a single frame of fringe pattern with limited information. Therefore, the purpose of this work attempts to propose an end-to-end DL-based 3D reconstruction method from the single fringe pattern with excellent capability of achieving high accuracy depth recovery and geometry details preservation of tested objects. We construct a multi-scale feature fusion convolutional neural network (CNN) called MSUNet++, which incorporates discrete wavelet transform (DWT) in data preprocessing for extracting high-frequency signals of fringe patterns as input of the network. Additionally, a loss function that combines structural similarity with edge perception is established. Through these measures, high-frequency geometry details of the reconstruction results can be obviously enhanced, while the geometric shape can be effectively maintained. Ablation experiments are involved in validating the effectiveness of our proposed solution. 3D reconstructed results and analysis of generalization experiments on different tested samples imply that the proposed method in this research enjoys capabilities of higher accuracy, better detail preservation, and robustness in comparison with the compared methods.

Nonlinear optimization considering target feature points for bitelecentric camera calibration: experimental study.

The telecentric camera has found extensive application in microscopy imaging due to its remarkable attributes of maintaining constant magnification and minimal distortion within its depth of field. In telecentric imaging technology, the accuracy of measurements frequently hinges upon the calibration precision of the telecentric camera. In real-world scenarios, the shallow depth of field characteristic of telecentric cameras often leads to out-of-focus targets during the capturing process, which in turn results in the inability to accurately extract pixel coordinates of feature points, making it difficult for optimization algorithms to converge to the optimal value. We propose a nonlinear optimization algorithm based on pixel coordinates of optimized feature points for bitelecentric cameras. Incorporating pixel coordinates into the optimization process yields the theoretically optimal solution based on bitelecentric camera model. The obtained pixel coordinates are used for second initial value estimation, followed by the optimization of all parameters. Compared to existing methods, the proposed approach significantly reduces reprojection errors under both blurry and clear target conditions. Experimental results demonstrate superior performance in processing blurry defocused images.

High dynamic reflection surface 3D reconstruction with sharing phase demodulation mechanism and multi-indicators guided phase domain fusion.

Accurate and complete 3D measurement of complex high dynamic range (HDR) surfaces has been challenging for structured light projection technique. The behavior of spraying a layer of diffuse reflection material, which will inevitably incur additional thickness. Existing methods based on additional facilities will increase the cost of hardware system. The algorithms-based methods are cost-effective and nondestructive, but they generally require redundant patterns for image fusion and model training, which fail to be suitable for practicing automated 3D measurement for complex HDR surfaces. In this paper, a HDR surface 3D reconstruction method based on sharing demodulation phase unwrapping mechanism and multi-indicators guided phase fusion strategy is proposed. The division of the exposure interval is optimized via the image entropy to generate an optimal exposure sequence. The combination of temporal-spatial binary (TSB) encoding fringe patterns with time-integration strategy and the variable exposure mode of digital mirror device (DMD)-based projector with a minimum projection exposure time of 233µs enables the proposed approach to broadly adapt complex HDR surfaces. We propose an efficient phase analysis solution called sharing mechanism that wrapped phase sequences from captured different intensity fringe images are unwrapped through sharing the same group of misaligned Gray code (MGC) decoding result. Finally, a phase sequences fusion model guided by multi-indicators, including exposure quality, phase gradient smoothness and pixel effectiveness, is established to obtain an optimum phase map for final 3D reconstruction. Comparative experiments indicate that the proposed method can completely restore the 3D topography of HDR surfaces with the images reduction of at least 65% and the measurement integrity is maintained at over 98% while preserving the measurement accuracy and excluding the outliers.

Temporal-spatial binary encoding method based on dynamic threshold optimization for 3D shape measurement.

The binary encoding method has been widely used for three-dimensional (3D) shape measurement due to the high-speed projection characteristics of its digital mirror device (DMD)-based projector. However, traditional binary encoding methods require a larger defocus to achieve a good sinusoidality, leading to a reduction in the measurement depth of field and signal-to-noise ratio (SNR) of captured images, which can adversely affect the accuracy of phase extraction, particularly high-frequency fringes for 3D reconstruction. This paper proposes a spatial-temporal binary encoding method based on dynamic threshold optimization for 3D shape measurement. The proposed method decomposes an 8-bit sinusoidal fringe pattern into multiple(K) binary patterns, which can be outlined into two steps: determining the dynamic threshold and then performing temporal-spatial error diffusion encoding. By using an integral imaging strategy, approximate sinusoidal patterns can be obtained under nearly focused projection, which can then be subjected to absolute phase unwrapping and 3D reconstruction. The experiments show that compared to the three comparative algorithms under the same experimental conditions, this proposed method improves the reconstruction error of measuring a plane and an object by at least 13.66% and 12.57% when K=2. The dynamic experimental result on the palm confirms that the proposed method can reliably reconstruct the 3D shape of the moving object.

Surface warming from altitudinal and latitudinal amplification over Antarctica since the International Geophysical Year.

Warming has been and is being enhanced at high latitudes or high elevations, whereas the quantitative estimation for warming from altitude and latitude effects has not been systematically investigated over Antarctic Ice Sheet, which covers more than 27 degrees of latitude and 4000 m altitude ranges. Based on the monthly surface air temperature data (1958-2020) from ERA5 reanalysis, this work aims to explore whether elevation-dependent warming (EDW) and latitude-dependent warming (LDW) exist. Results show that both EDW and LDW have the cooperative effect on Antarctic warming, and the magnitude of EDW is stronger than LDW. The negative EDW appears between 250 m and 2500 m except winter, and is strongest in autumn. The negative LDW occurs between 83 °S and 90 °S except in summer. Moreover, the surface downward long-wave radiation that related to the specific humidity, total cloud cover and cloud base height is a major contributor to the EDW over Antarctica. Further research on EDW and LDW should be anticipated to explore the future Antarctic amplification under different emission scenarios.

Accurate 3D reconstruction of single-frame speckle-encoded textureless surfaces based on densely connected stereo matching network.

Speckle projection profilometry (SPP) determines the global correspondence between stereo images by speckle pattern(s) projection in three-dimensional (3D) vision. However, it is extremely challenging for traditional algorithms to achieve a satisfactory 3D reconstruction accuracy generally via single-frame speckle pattern, which heavily constraints the application in dynamic 3D imaging. Recently some deep learning (DL) based methods have made process in this issue but there exist deficiencies in feature extraction, leading to a limited improvement in accuracy. In this paper, we propose a stereo matching network called Densely Connected Stereo Matching (DCSM) Network that requires only single-frame speckle pattern as input, adopts densely connected feature extraction and incorporates attention weight volume construction. The densely connected multi-scale feature extraction module we constructed in DCSM Network has a positive effect on the combination of global and local information and inhibition of information loss. We also establish a real measurement system and its digital twin through Blender to obtain rich speckle data under SPP framework. Meanwhile, we introduce Fringe Projection Profilometry (FPP) to obtain phase information to assist in generating high-precision disparity as Ground Truth (GT). Experiments with different types of models and models with various perspectives are implemented to prove the effectiveness and generalization of the proposed network compared with classic and the latest DL-based algorithms. Finally, the 0.5-Pixel-Error of our method in the disparity maps is as low as 4.81%, and the accuracy is verified to be improved by up to 33.4%. As for the cloud point, our method has a reduction of 18%∼30% compared with other network-based methods.

New evidence for rice harvesting in the early Neolithic Lower Yangtze River, China.

The Lower Yangtze River of China has been identified as an independent center of rice domestication, but tracing the earliest evidence for rice cultivation practices has been challenging. Here we report the first evidence for rice harvesting, based on use-wear and phytolith residue analyses of 52 flaked stone tools (10000-7000 BP) from the Shangshan and Hehuashan sites. The tools reflect two harvesting methods: reaping the panicles at the top and cutting the stalk near the base. Thus, our research provides a new method for investigating prehistoric cereal cultivation, and the data lend support to the evidence of rice domestication in the early Holocene. The results also show the complexity of rice harvesting strategies several millennia before the emergence of full-fledged agriculture in the Lower Yangtze.

A Wireless Real-Time Continuous Monitoring System for the Internal Movements of Mountain Glaciers Using Sensor Networks.

With the escalation of global warming, the shrinkage of mountain glaciers has accelerated globally, the water volume from glaciers has changed, and relative disasters have increased in intensity and frequency (for example, ice avalanches, surging glaciers, and glacial lake outburst floods). However, the wireless monitoring of glacial movements cannot currently achieve omnidirectional, high-precision, real-time results, since there are some technical bottlenecks. Based on wireless networks and sensor application technologies, this study designed a wireless monitoring system for measuring the internal parameters of mountain glaciers, such as temperature, pressure, humidity, and power voltage, and for wirelessly transmitting real-time measurement data. The system consists of two parts, with a glacier internal monitoring unit as one part and a glacier surface base station as the second part. The former wirelessly transmits the monitoring data to the latter, and the latter processes the received data and then uploads the data to a cloud data platform via 4G or satellite signals. The wireless system can avoid cable constraints and transmission failures due to breaking cables. The system can provide more accurate field-monitoring data for simulating glacier movements and further offers an early warning system for glacial disasters.

Theoretical analysis and experimental investigation of the Floyd-Steinberg-based fringe binary method with offset compensation for accurate 3D measurement.

Digital fringe projection (DFP) with defocused binary fringe patterns has the ability to overcome the projector nonlinearity and achieve a high-speed 3D measurement. The Floyd-Steinberg (FS) dithering technique is one of the most commonly adopted binary fringe coding methods due to its relatively high measurement accuracy. Nevertheless, we found that the FS binary fringe would cause a fixed error in the recovered phase, which is proven to be invariable for various defocusing levels and various phase-shift steps according to the analysis of the phase error based on noise model of phase-shifting profilometry. It means that FS binary fringe would have a certain offset in space, compared with standard sinusoidal fringe, which is verified to be essentially constant for different fringe pitches through simulation and experiment. This offset would distort the 3D geometry of the tested target for monocular systems relying on triangulation, which needs to be compensated to improve 3D measurement accuracy. Experiments are presented to demonstrate the enhanced 3D result after compensation.

Optimal frequency selection for accuracy improvement in binary defocusing fringe projection profilometry.

The binary defocusing fringe projection profilometry (FPP) technique has demonstrated various advantages for high-speed and high-accuracy three-dimensional (3D) surface measurement. However, higher fringe frequency does not necessarily give better measurements in binary defocusing FPP. To improve the 3D geometry measurement accuracy, this paper proposes an optimal frequency selection approach by analyzing the phase error distribution under different defocusing degrees. The phase error is analyzed theoretically based on the multi-frequency temporal phase unwrapping process, and the associated relationship with fringe frequency, system defocusing degree, noise, and other influencing factors is established. Meanwhile, optimal fringe frequency in a specific system is selected by the theoretical model combined with the validation of simulation experiments. Finally, the measurement accuracy could be effectively enhanced by the generated binary fringe patterns of optimal frequency. Both simulations and experiments verify the effectiveness and robustness of the proposed method.

Polar amplification comparison among Earth's three poles under different socioeconomic scenarios from CMIP6 surface air temperature.

The polar amplification (PA) has become the focus of climate change. However, there are seldom comparisons of amplification among Earth's three poles of Arctic (latitude higher than 60 °N), Antarctica (Antarctic Ice Sheet) and the Third Pole (the High Mountain Asia with the elevation higher than 4000 m) under different socioeconomic scenarios. Based on CMIP6 multi-model ensemble, two types of PA index (PAI) have been defined to quantify the PA intensity and variations, and PAI1/PAI2 is defined as the ratio of the absolute value of surface air temperature linear trend over Earth's three poles and that for global mean/over other regions except Earth's three poles. Arctic warms fastest in winter and weakest in summer, followed by the Third Pole, and Antarctica warms least. The similar phenomenon proceeds when global warming of 1.5-2.0 °C, and 2.0-3.0 °C above pre-industrial levels. After removing the Earth's three poles self-influence, all the PAI2s increase much more obviously relative to the PAI1s, especially the Antarctic PAI. Earth's three poles warm faster than the other regions. With the forcing increasing, PA accelerates much more over Antarctica and the Third Pole, but becomes weaker over Arctic. This demonstrates that future warming rate might make a large difference among Earth's three poles under different scenarios.

Constrained nonlinear optimization method for accurate calibration of a bi-telecentric camera in a three-dimensional microtopography system.

Telecentric cameras are widely used in the field of microscopic imaging because of their constant magnification and tiny distortion in the depth of field. Camera calibration has always been a key step in the field of computer vision. Usually, the precise parameters of the telecentric camera are obtained by nonlinear optimization; however, the randomness of the optimization algorithm without proper constraints will cause the results to be inconsistent with reality. Existing studies paid little attention to this issue; therefore, we show a reliable optimization approach for the bi-telecentric camera in a structured illumination three-dimensional microtopography measurement system. In this method, the distortion-free camera parameters are solved through a closed-form solution. Then a nonlinear optimization algorithm with constraining the world coordinates of the precise calibration target is proposed to refine the global parameters, leading to the calibration results being more accurate and authentic. The real experiments are conducted to verify the feasibility of the proposed method. The comparative experiments with the exiting approach are then carried out, manifesting that the proposed method enjoys advantages in terms of both reprojection error and operating efficiency. Additionally, the average offset of the world coordinates on the calibration target derived from the proposed method verifies its effectiveness and reasonability.

Efficient 3D measurement of a HDR surface based on adaptive fringe projection.

3D measurement methods of a high-dynamic-range (HDR) surface based on adaptive fringe projection have aroused extensive research interest. They tend to pixel-wise adjust the fringe projection intensity to ensure full-field phase quality in light or dark regions, which has two problems: (1) traditional image intensity-based temporal phase unwrapping (TPU) is susceptible to noise in dark regions, and (2) it is time-consuming to project orthogonal fringe patterns for coordinate mapping and 3D reconstruction. Aiming to address these issues, we present an efficient adaptive fringe projection method where misaligned Gray code patterns are adopted to remove the phase error induced by low-frequency fringe patterns. Compared with traditional image intensity-based TPU, misaligned Gray-code-based TPU provides a better noise-suppression effect in dark regions, as Gray codes are generally better preserved than image intensity. Moreover, the images captured in the coordinate mapping process are shared for optimal projection intensity calculation and 3D reconstruction to reduce the number of total projection patterns, thus improving measurement efficiency. Extensive contrast experiments are conducted to demonstrate that the proposed method retrieves the 3D shapes of micro-scale HDR surfaces with high accuracy and a minimum number of projection patterns on the premise of high measurement integrity.

3D face imaging with the spatial-temporal correlation method using a rotary speckle projector.

In this paper, a compact, cost-effective, and fast rotary speckle projector (RSP) is designed and manufactured for high-precision three-dimensional (3D) face data acquisition. Compared with the common speckle projectors, RSP uses a simple speckle pattern design method and has a good performance in high-speed projection and compact structure, which allows a flexible balance between measurement accuracy and time cost in a real acquisition task. Using a carefully designed rotation angle of the speckle mask, temporally and spatially non-correlative speckle patterns in the measurement volume can be generated. The rotation angle of the speckle mask is carefully checked and optimally selected via detailed theoretical analysis, simulation, and experiments to ensure 3D reconstruction accuracy across the reconstruction area. Subsequently, a binocular 3D face imaging system composed of the RSP and two cameras is constructed. With captured stereo speckle image pairs, we adopted our previously well-established spatial-temporal correlation method to determine the disparity. The accuracy of the 3D face imaging system was verified by using a real face mask, which is standardized by a certified, high-precision industrial 3D scanner. The real face data collection under various expressions has demonstrated that the proposed system also has a good performance for 3D face imaging in dynamic scenes.

Accurate stereo vision system calibration with chromatic concentric fringe patterns.

Camera calibration is used to determine the intrinsic and extrinsic parameters of a 3D imaging system based on structured light. Traditional methods like chessboard and circular dots usually employ an intensity-based feature point detection procedure, and are susceptible to noise, image contrast, and image blur. To address these issues, we proposed an active calibration method to accurately detect the centers of chromatic concentric fringe patterns (CCFP). Specifically, we first acquired the circular phase using a phase analysis algorithm, then extracted nine phase contours from the circular phase for the corresponding subpixel center coordinates using an ellipse fitting algorithm, and precisely calculated the final center with their weighted sum. We ran a simulation and evaluated the impacts of different degrees of Gaussian blur and noise on the calibrated parameters. The simulation demonstrates that our approach is more robust to noise and blur than previous ones, and our approach yields a higher calibration accuracy. Moreover, we carried out a comparison experiment to evaluate the performance of our method. It showed that the reprojection error can be reduced by at least 10% in the out-of-focus condition (i.e., the target is beyond the working distance of the camera) and the 3D reconstruction accuracy can be improved by nearly 10%.

3-D face registration solution with speckle encoding based spatial-temporal logical correlation algorithm.

3-D information acquisition (registration) of whole face plays a significant role in 3-D human face recognition application. In this paper, we develop a prototype of 3-D system consisting of two binocular measurement units that allows a full 3-D reconstruction by utilizing the advantages of a novel correlation algorithm. In this system, we use optical modulation to produce temporally and spatially varying high-density binary speckle patterns to encode the tested face, then propose a spatial-temporal logical correlation (STLC) stereo matching algorithm to fast determine the accurate disparity with a coarse and refined strategy. Finally the 3-D information of whole face from left- and right ear (~180°) can be obtainable by fusing the data from two measurement units. Comparative researches are performed to test a plastic model and a real human face by simulating real application situations. The results verify the feasibility and good performances of our computational frameworks and experimental configuration in terms of accuracy and time cost, which show a good application prospect in our future 3-D human face recognition research.