Depth Map Recovery Based on a Unified Depth Boundary Distortion Model.

Depth maps acquired by either physical sensors or learning methods are often seriously distorted due to boundary distortion problems, including missing, fake, and misaligned boundaries (compared with RGB images). An RGB-guided depth map recovery method is proposed in this paper to recover true boundaries in seriously distorted depth maps. Therefore, a unified model is first developed to observe all these kinds of distorted boundaries in depth maps. Observing distorted boundaries is equivalent to identifying erroneous regions in distorted depth maps, because depth boundaries are essentially formed by contiguous regions with different intensities. Then, erroneous regions are identified by separately extracting local structures of RGB image and depth map with Gaussian kernels and comparing their similarity on the basis of the SSIM index. A depth map recovery method is then proposed on the basis of the unified model. This method recovers true depth boundaries by iteratively identifying and correcting erroneous regions in recovered depth map based on the unified model and a weighted median filter. Because RGB image generally includes additional textural contents compared with depth maps, texture-copy artifacts problem is further addressed in the proposed method by restricting the model works around depth boundaries in each iteration. Extensive experiments are conducted on five RGB-depth datasets including depth map recovery, depth super-resolution, depth estimation enhancement, and depth completion enhancement. The results demonstrate that the proposed method considerably improves both the quantitative and visual qualities of recovered depth maps in comparison with fifteen competitive methods. Most object boundaries in recovered depth maps are corrected accurately, and kept sharply and well aligned with the ones in RGB images.

Hole Transport Layer Free Perovskite Light-Emitting Diodes With High-Brightness and Air-Stability Based on Solution-Processed CsPbBr3-Cs4PbBr6 Composites Films.

Recently, perovskite light-emitting diodes (PeLEDs) have drew widespread attention due to their high efficiencies. However, because of the sensitivity to moisture and oxygen, perovskite luminescent layers are usually prepared in high-purity nitrogen environment, which increases the cost and process complexity of device preparation and seriously hindrances its commercialization of PeLED in lighting and display application. Herein, dual-phase all-inorganic composite CsPbBr3-Cs4PbBr6 films are fabricated from CsBr-rich perovskite solutions by a simple one-step spin-coating method in the air with high humidity. Compared with the pure CsPbBr3 film, the composite CsPbBr3-Cs4PbBr6 film has much stronger photoluminescence emission and longer fluorescence lifetime, accompanied by increased photoluminescence quantum yield (33%). As a result, we obtained green PeLED devices without hole transport layer exhibiting a maximum brightness of 72,082 cd/m2 and a maximum external quantum efficiency of about 2.45%, respectively. More importantly, the champion device shows excellent stability with operational half-lifetime exceeding 1,000 min under continuous operation in the air. The dual-phase all-inorganic composite CsPbBr3-Cs4PbBr6 film shows attractive prospect for advanced light emission applications.

Neighborhood Geometric Structure-Preserving Variational Autoencoder for Smooth and Bounded Data Sources.

Many data sources, such as human poses, lie on low-dimensional manifolds that are smooth and bounded. Learning low-dimensional representations for such data is an important problem. One typical solution is to utilize encoder-decoder networks. However, due to the lack of effective regularization in latent space, the learned representations usually do not preserve the essential data relations. For example, adjacent video frames in a sequence may be encoded into very different zones across the latent space with holes in between. This is problematic for many tasks such as denoising because slightly perturbed data have the risk of being encoded into very different latent variables, leaving output unpredictable. To resolve this problem, we first propose a neighborhood geometric structure-preserving variational autoencoder (SP-VAE), which not only maximizes the evidence lower bound but also encourages latent variables to preserve their structures as in ambient space. Then, we learn a set of small surfaces to approximately bound the learned manifold to deal with holes in latent space. We extensively validate the properties of our approach by reconstruction, denoising, and random image generation experiments on a number of data sources, including synthetic Swiss roll, human pose sequences, and facial expression images. The experimental results show that our approach learns more smooth manifolds than the baselines. We also apply our approach to the tasks of human pose refinement and facial expression image interpolation where it gets better results than the baselines.

A multiresolution mixture generative adversarial network for video super-resolution.

Generative adversarial networks (GANs) have been used to obtain super-resolution (SR) videos that have improved visual perception quality and more coherent details. However, the latest methods perform poorly in areas with dense textures. To better recover the areas with dense textures in video frames and improve the visual perception quality and coherence in videos, this paper proposes a multiresolution mixture generative adversarial network for video super-resolution (MRMVSR). We propose a multiresolution mixture network (MRMNet) as the generative network that can simultaneously generate multiresolution feature maps. In MRMNet, the high-resolution (HR) feature maps can continuously extract information from low-resolution (LR) feature maps to supplement information. In addition, we propose a residual fluctuation loss function for video super-resolution. The residual fluctuation loss function is used to reduce the overall residual fluctuation on SR and HR video frames to avoid a scenario where local differences are too large. Experimental results on the public benchmark dataset show that our method outperforms the state-of-the-art methods for the majority of the test sets.

A Joint Label Space for Generalized Zero-Shot Classification.

The fundamental problem of Zero-Shot Learning (ZSL) is that the one-hot label space is discrete, which leads to a complete loss of the relationships between seen and unseen classes. Conventional approaches rely on using semantic auxiliary information, e.g. attributes, to re-encode each class so as to preserve the inter-class associations. However, existing learning algorithms only focus on unifying visual and semantic spaces without jointly considering the label space. More importantly, because the final classification is conducted in the label space through a compatibility function, the gap between attribute and label spaces leads to significant performance degradation. Therefore, this paper proposes a novel pathway that uses the label space to jointly reconcile visual and semantic spaces directly, which is named Attributing Label Space (ALS). In the training phase, one-hot labels of seen classes are directly used as prototypes in a common space, where both images and attributes are mapped. Since mappings can be optimized independently, the computational complexity is extremely low. In addition, the correlation between semantic attributes has less influence on visual embedding training because features are mapped into labels instead of attributes. In the testing phase, the discrete condition of label space is removed, and priori one-hot labels are used to denote seen classes and further compose labels of unseen classes. Therefore, the label space is very discriminative for the Generalized ZSL (GZSL), which is more reasonable and challenging for real-world applications. Extensive experiments on five benchmarks manifest improved performance over all of compared state-of-the-art methods.

Cross-View Person Identification Based on Confidence-Weighted Human Pose Matching.

Cross-view person identification (CVPI) from multiple temporally synchronized videos taken by multiple wearable cameras from different, varying views is a very challenging but important problem, which has attracted more interest recently. Current state-of-the-art performance of CVPI is achieved by matching appearance and motion features across videos, while the matching of pose features does not work effectively given the high inaccuracy of the 3D pose estimation on videos/images collected in the wild. To address this problem, we first introduce a new metric of confidence to the estimated location of each human-body joint in 3D human pose estimation. Then, a mapping function, which can be hand-crafted or learned directly from the datasets, is proposed to combine the inaccurately estimated human pose and the inferred confidence metric to accomplish CVPI. Specifically, the joints with higher confidence are weighted more in the pose matching for CVPI. Finally, the estimated pose information is integrated into the appearance and motion features to boost the CVPI performance. In the experiments, we evaluate the proposed method on three wearable-camera video datasets and compare the performance against several other existing CVPI methods. The experimental results show the effectiveness of the proposed confidence metric, and the integration of pose, appearance, and motion produces a new state-of-the-art CVPI performance.

Robust Stability of Efficient Lead-Free Formamidinium Tin Iodide Perovskite Solar Cells Realized by Structural Regulation.

The instability issue of Pb-free Sn-based perovskite is one of the biggest challenges for its application in optoelectronic devices. Herein, a structural regulation strategy is demonstrated to regulate the geometric symmetry of formamidiniumtin iodide (FASnI3) perovskite. Experimental and theoretical works show that the introduction of cesium cation (Cs+) could improve the geometric symmetry, suppress the oxidation of Sn2+, and enhance the thermodynamical structural stability of FASnI3. As a result, the inverted planar Cs-doped FASnI3-based perovskite solar cell (PSC) is shown to maintain 90% of its initial power-conversion efficiency (PCE) after 2000 h stored in N2, which is the best durability to date for 3D Sn-based PSCs. Most importantly, the air, thermal, and illumination stabilities of the PSCs are all improved after Cs doping. The PCE of the Cs-doped PSC shows a 63% increase compared to that of the control device (from 3.74% to 6.08%) due to the improved quality of the Cs-doped FASnI3 film.

Efficient lossy compression for compressive sensing acquisition of images in compressive sensing imaging systems.

Compressive Sensing Imaging (CSI) is a new framework for image acquisition, which enables the simultaneous acquisition and compression of a scene. Since the characteristics of Compressive Sensing (CS) acquisition are very different from traditional image acquisition, the general image compression solution may not work well. In this paper, we propose an efficient lossy compression solution for CS acquisition of images by considering the distinctive features of the CSI. First, we design an adaptive compressive sensing acquisition method for images according to the sampling rate, which could achieve better CS reconstruction quality for the acquired image. Second, we develop a universal quantization for the obtained CS measurements from CS acquisition without knowing any a priori information about the captured image. Finally, we apply these two methods in the CSI system for efficient lossy compression of CS acquisition. Simulation results demonstrate that the proposed solution improves the rate-distortion performance by 0.4~2 dB comparing with current state-of-the-art, while maintaining a low computational complexity.

Spatio-temporal saliency perception via hypercomplex frequency spectral contrast.

Salient object perception is the process of sensing the salient information from the spatio-temporal visual scenes, which is a rapid pre-attention mechanism for the target location in a visual smart sensor. In recent decades, many successful models of visual saliency perception have been proposed to simulate the pre-attention behavior. Since most of the methods usually need some ad hoc parameters or high-cost preprocessing, they are difficult to rapidly detect salient object or be implemented by computing parallelism in a smart sensor. In this paper, we propose a novel spatio-temporal saliency perception method based on spatio-temporal hypercomplex spectral contrast (HSC). Firstly, the proposed HSC algorithm represent the features in the HSV (hue, saturation and value) color space and features of motion by a hypercomplex number. Secondly, the spatio-temporal salient objects are efficiently detected by hypercomplex Fourier spectral contrast in parallel. Finally, our saliency perception model also incorporates with the non-uniform sampling, which is a common phenomenon of human vision that directs visual attention to the logarithmic center of the image/video in natural scenes. The experimental results on the public saliency perception datasets demonstrate the effectiveness of the proposed approach compared to eleven state-of-the-art approaches. In addition, we extend the proposed model to moving object extraction in dynamic scenes, and the proposed algorithm is superior to the traditional algorithms.