Polarimetric Helmholtz Stereopsis.

Helmholtz stereopsis (HS) exploits the reciprocity principle of light propagation (i.e., the Helmholtz reciprocity) for 3D reconstruction of surfaces with arbitrary reflectance. In this paper, we present the polarimetric Helmholtz stereopsis (polar-HS), which extends the classical HS by considering the polarization state of light in the reciprocal paths. With the additional phase information from polarization, polar-HS requires only one reciprocal image pair. We derive the reciprocity relationship of Mueller matrix and formulate new reciprocity constraint that takes polarization state into account. We also utilize polarimetric constraints and extend them to the case of perspective projection. For the recovery of surface depths and normals, we incorporate reciprocity constraint with diffuse/specular polarimetric constraints in a unified optimization framework. For depth estimation, we further propose to utilize the consistency of diffuse angle of polarization. For normal estimation, we develop a normal refinement strategy based on degree of linear polarization. Using a hardware prototype, we show that our approach produces high-quality 3D reconstruction for different types of surfaces, ranging from diffuse to highly specular.

Full-Volume 3D Fluid Flow Reconstruction With Light Field PIV.

Particle Imaging Velocimetry (PIV) is a classical method that estimates fluid flow by analyzing the motion of injected particles. To reconstruct and track the swirling particles is a difficult computer vision problem, as the particles are dense in the fluid volume and have similar appearances. Further, tracking a large number of particles is particularly challenging due to heavy occlusion. Here we present a low-cost PIV solution that uses compact lenslet-based light field cameras as imaging device. We develop novel optimization algorithms for dense particle 3D reconstruction and tracking. As a single light field camera has limited capacity in resolving depth (z-dimension measurement), the resolution of 3D reconstruction on the x-y plane is much higher than along the z-axis. To compensate for the imbalanced resolution in 3D, we use two light field cameras positioned at an orthogonal angle to capture particle images. In this way, we can achieve high-resolution 3D particle reconstruction in the full fluid volume. For each time frame, we first estimate particle depths under a single viewpoint by exploiting the focal stack symmetry of light field. We then fuse the recovered 3D particles in two views by solving a linear assignment problem (LAP). Specifically, we propose an anisotropic point-to-ray distance as matching cost to handle the resolution mismatch. Finally, given a sequence of 3D particle reconstructions over time, we recover the full-volume 3D fluid flow with a physically-constrained optical flow, which enforces local motion rigidity and fluid incompressibility. We perform comprehensive experiments on synthetic and real data for ablation and evaluation. We show that our method recovers full-volume 3D fluid flows of various types. Two-view reconstruction results achieves higher accuracy than those with one view only.

Multi-frequency acoustic hologram generation with a physics-enhanced deep neural network.

Here, a physics-enhanced multi-frequency acoustic hologram deep neural network (PhysNet_MFAH) method is proposed for designing multi-frequency acoustic holograms, which is built by incorporating multiple physical models that represent the physical processes of acoustic waves propagation for a set of design frequencies into a deep neural network. It is demonstrated that one needs only to feed a set of frequency-specific target patterns into the network, the proposed PhysNet_MFAH method can automatically, accurately, and rapidly generate a high-quality multi-frequency acoustic hologram for holographic rendering of different target acoustic fields in the same or distinct regions of the target plane when driven at different frequencies. Remarkably, it is also demonstrated that the proposed PhysNet_MFAH method can achieve a higher quality of the reconstructed acoustic intensity fields than the existing optimization methods IASA and DS for designing multi-frequency acoustic holograms at a relatively fast-computational speed. Furthermore, the performance dependencies of the proposed PhysNet_MFAH method on different design parameters are established, which provide insight into the performance of the reconstructed acoustic intensity fields when subject to different design conditions of the proposed PhysNet_MFAH method. We believe that the proposed PhysNet_MFAH method can facilitate many potential applications of acoustic holograms, ranging from dynamic particle manipulation to volumetric display.

Structure From Motion on XSlit Cameras.

We present a structure-from-motion (SfM) framework based on a special type of multi-perspective camera called the cross-slit or XSlit camera. Traditional perspective camera based SfM suffers from the scale ambiguity which is inherent to the pinhole camera geometry. In contrast, an XSlit camera captures rays passing through two oblique lines in 3D space and we show such ray geometry directly resolves the scale ambiguity when employed for SfM. To accommodate the XSlit cameras, we develop tailored feature matching, camera pose estimation, triangulation, and bundle adjustment techniques. Specifically, we devise a SIFT feature variant using non-uniform Gaussian kernels to handle the distortions in XSlit images for reliable feature matching. Moreover, we demonstrate that the XSlit camera exhibits ambiguities in pose estimation process which can not be handled by existing work. Consequently, we propose a 14 point algorithm to properly handle the XSlit degeneracy and estimate the relative pose between XSlit cameras from feature correspondences. We further exploit the unique depth-dependent aspect ratio (DDAR) property to improve the bundle adjustment for the XSlit camera. Synthetic and real experiments demonstrate that the proposed XSlit SfM can conduct reliable and high fidelity 3D reconstruction at an absolute scale.

Shape and Reflectance Reconstruction Using Concentric Multi-Spectral Light Field.

Recovering the shape and reflectance of non-Lambertian surfaces remains a challenging problem in computer vision since the view-dependent appearance invalidates traditional photo-consistency constraint. In this paper, we introduce a novel concentric multi-spectral light field (CMSLF) design that is able to recover the shape and reflectance of surfaces of various materials in one shot. Our CMSLF system consists of an array of cameras arranged on concentric circles where each ring captures a specific spectrum. Coupled with a multi-spectral ring light, we are able to sample viewpoint and lighting variations in a single shot via spectral multiplexing. We further show that our concentric camera and light source setting results in a unique single-peak pattern in specularity variations across viewpoints. This property enables robust depth estimation for specular points. To estimate depth and multi-spectral reflectance map, we formulate a physics-based reflectance model for the CMSLF under the surface camera (S-Cam) representation. Extensive synthetic and real experiments show that our method outperforms the state-of-the-art shape reconstruction methods, especially for non-Lambertian surfaces.

Content Aware Image Pre-Compensation.

The goal of image pre-compensation is to process an image such that after being convolved with a known kernel, will appear close to the sharp reference image. In a practical setting, the pre-compensated image has significantly higher dynamic range than the latent image. As a result, some form of tone mapping is needed. In this paper, we show how global tone mapping functions affect contrast and ringing in image pre-compensation. We further enhance contrast and reduce ringing by considering the visual saliency. Specifically, we prioritize contrast preservation in salient regions while tolerating more blurriness elsewhere. For quantitative analysis, we design new metrics to measure the contrast of an image with ringing. Specifically, we set out to find its "equivalent ringing-free" image that matches its intensity histogram and uses its contrast as the measure. We illustrate our approach on projector defocus compensation and visual acuity enhancement. Compared with the state-of-the-art, our approach significantly improves the contrast. We also perform user studies to demonstrate that our method can effectively improve the viewing experience for users with impaired vision.

Assessing the impacts of the changes in farming systems on food security and environmental sustainability of a Chinese rural region under different policy scenarios: an agent-based model.

Since China has undergone a series of economic reforms and implemented opening up policies, its farming systems have significantly changed and have dramatically influenced the society, economy, and environment of China. To assess the comprehensive impacts of these changes on food security and environmental sustainability, and establish effective and environment-friendly subsidy policies, this research constructed an agent-based model (ABM). Daligang Town, which is located in the two-season rice region of Southern China, was selected as the case study site. Four different policy scenarios, i.e., "sharply increasing" (SI), "no-increase" (NI), "adjusted-method" (AM), and "trend" (TD) scenarios were investigated from 2015 to 2029. The validation result shows that the relative prediction errors between the simulated and actual values annually ranged from -20 to 20%, indicating the reliability of the proposed model. The scenario analysis revealed that the four scenarios generated different variations in cropping systems, rice yield, and fertilizer and pesticide inputs when the purchase price of rice and the non-agricultural income were assumed to increase annually by 0.1 RMB per kg and 10% per person, respectively. Among the four different policy scenarios in Daligang, the TD scenario was considered the best, because it had a relatively high rice yield, fairly minimal use of fertilizers and pesticides, and a lower level of subsidy. Despite its limitations, ABM could be considered a useful tool in analyzing, exploring, and discussing the comprehensive effects of the changes in farming system on food security and environmental sustainability.

Assessing the effects of rural livelihood transition on non-point source pollution: a coupled ABM-IECM model.

Water pollution caused by anthropogenic activities and driven by changes in rural livelihood strategies in an agricultural system has received increasing attention in recent decades. To simulate the effects of rural household livelihood transition on non-point source (NPS) pollution, a model combining an agent-based model (ABM) and an improved export coefficient model (IECM) was developed. The ABM was adopted to simulate the dynamic process of household livelihood transition, and the IECM was employed to estimate the effects of household livelihood transition on NPS pollution. The coupled model was tested in a small catchment in the Dongting Lake region, China. The simulated results reveal that the transition of household livelihood strategies occurred with the changes in the prices of rice, pig, and labor. Thus, the cropping system, land-use intensity, resident population, and number of pigs changed in the small catchment from 2000 to 2014. As a result of these changes, the total nitrogen load discharged into the river initially increased from 6841.0 kg in 2000 to 8446.3 kg in 2004 and then decreased to 6063.9 kg in 2014. Results also suggest that rural living, livestock, paddy field, and precipitation alternately became the main causes of NPS pollution in the small catchment, and the midstream region of the small catchment was the primary area for NPS pollution from 2000 to 2014. Despite some limitations, the coupled model provides an innovative way to simulate the effects of rural household livelihood transition on NPS pollution with the change of socioeconomic factors, and thereby identify the key factors influencing water pollution to provide valuable suggestions on how agricultural environmental risks can be reduced through the regulation of the behaviors of farming households in the future.

Saliency Detection on Light Field.

Existing saliency detection approaches use images as inputs and are sensitive to foreground/background similarities, complex background textures, and occlusions. We explore the problem of using light fields as input for saliency detection. Our technique is enabled by the availability of commercial plenoptic cameras that capture the light field of a scene in a single shot. We show that the unique refocusing capability of light fields provides useful focusness, depths, and objectness cues. We further develop a new saliency detection algorithm tailored for light fields. To validate our approach, we acquire a light field database of a range of indoor and outdoor scenes and generate the ground truth saliency map. Experiments show that our saliency detection scheme can robustly handle challenging scenarios such as similar foreground and background, cluttered background, complex occlusions, etc., and achieve high accuracy and robustness.