Consistent 3D Hand Reconstruction in Video via Self-Supervised Learning.

We present a method for reconstructing accurate and consistent 3D hands from a monocular video. We observe that the detected 2D hand keypoints and the image texture provide important cues about the geometry and texture of the 3D hand, which can reduce or even eliminate the requirement on 3D hand annotation. Accordingly, in this work, we propose S2HAND, a self-supervised 3D hand reconstruction model, that can jointly estimate pose, shape, texture, and the camera viewpoint from a single RGB input through the supervision of easily accessible 2D detected keypoints. We leverage the continuous hand motion information contained in the unlabeled video data and explore S2HAND(V), which uses a set of weights shared S2HAND to process each frame and exploits additional motion, texture, and shape consistency constrains to obtain more accurate hand poses, and more consistent shapes and textures. Experiments on benchmark datasets demonstrate that our self-supervised method produces comparable hand reconstruction performance compared with the recent full-supervised methods in single-frame as input setup, and notably improves the reconstruction accuracy and consistency when using the video training data.

RoReg: Pairwise Point Cloud Registration With Oriented Descriptors and Local Rotations.

We present RoReg, a novel point cloud registration framework that fully exploits oriented descriptors and estimated local rotations in the whole registration pipeline. Previous methods mainly focus on extracting rotation-invariant descriptors for registration but unanimously neglect the orientations of descriptors. In this paper, we show that the oriented descriptors and the estimated local rotations are very useful in the whole registration pipeline, including feature description, feature detection, feature matching, and transformation estimation. Consequently, we design a novel oriented descriptor RoReg-Desc and apply RoReg-Desc to estimate the local rotations. Such estimated local rotations enable us to develop a rotation-guided detector, a rotation coherence matcher, and a one-shot-estimation RANSAC, all of which greatly improve the registration performance. Extensive experiments demonstrate that RoReg achieves state-of-the-art performance on the widely-used 3DMatch and 3DLoMatch datasets, and also generalizes well to the outdoor ETH dataset. In particular, we also provide in-depth analysis on each component of RoReg, validating the improvements brought by oriented descriptors and the estimated local rotations. Source code and supplementary material are available at https://github.com/HpWang-whu/RoReg.

A Strip Adjustment Method of UAV-Borne LiDAR Point Cloud Based on DEM Features for Mountainous Area.

Due to the trajectory error of the low-precision position and orientation system (POS) used in unmanned aerial laser scanning (ULS), discrepancies usually exist between adjacent LiDAR (Light Detection and Ranging) strips. Strip adjustment is an effective way to eliminate these discrepancies. However, it is difficult to apply existing strip adjustment methods in mountainous areas with few artificial objects. Thus, digital elevation model-iterative closest point (DEM-ICP), a pair-wise registration method that takes topography features into account, is proposed in this paper. First, DEM-ICP filters the point clouds to remove the non-ground points. Second, the ground points are interpolated to generate continuous DEMs. Finally, a point-to-plane ICP algorithm is performed to register the adjacent DEMs with the overlapping area. A graph-based optimization is utilized following DEM-ICP to estimate the correction parameters and achieve global consistency between all strips. Experiments were carried out using eight strips collected by ULS in mountainous areas to evaluate the proposed method. The average root-mean-square error (RMSE) of all data was less than 0.4 m after the proposed strip adjustment, which was only 0.015 m higher than the result of manual registration (ground truth). In addition, the plane fitting accuracy of lateral point clouds was improved 4.2-fold, from 1.565 to 0.375 m, demonstrating the robustness and accuracy of the proposed method.

Novel Adaptive Laser Scanning Method for Point Clouds of Free-Form Objects.

Laser scanners are widely used to collect coordinates, also known as point-clouds, of three-dimensional free-form objects. For creating a solid model from a given point-cloud and transferring the data from the model, features-based optimization of the point-cloud to minimize the number if points in the cloud is required. To solve this problem, existing methods mainly extract significant points based on local surface variation of a predefined level. However, comprehensively describing an object's geometric information using a predefined level is difficult since an object usually has multiple levels of details. Therefore, we propose a simplification method based on a multi-level strategy that adaptively determines the optimal level of points. For each level, significant points are extracted from the point cloud based on point importance measured by both local surface variation and the distribution of neighboring significant points. Furthermore, the degradation of perceptual quality for each level is evaluated by the adjusted mesh structural distortion measurement to select the optimal level. Experiments are performed to evaluate the effectiveness and applicability of the proposed method, demonstrating a reliable solution to optimize the adaptive laser scanning of point clouds for free-forms objects.