An efficient point cloud semantic segmentation network with multiscale super-patch transformer.

Efficient semantic segmentation of large-scale point cloud scenes is a fundamental and essential task for perception or understanding the surrounding 3d environments. However, due to the vast amount of point cloud data, it is always a challenging to train deep neural networks efficiently and also difficult to establish a unified model to represent different shapes effectively due to their variety and occlusions of scene objects. Taking scene super-patch as data representation and guided by its contextual information, we propose a novel multiscale super-patch transformer network (MSSPTNet) for point cloud segmentation, which consists of a multiscale super-patch local aggregation (MSSPLA) module and a super-patch transformer (SPT) module. Given large-scale point cloud data as input, a dynamic region-growing algorithm is first adopted to extract scene super-patches from the sampling points with consistent geometric features. Then, the MSSPLA module aggregates local features and their contextual information of adjacent super-patches at different scales. Owing to the self-attention mechanism, the SPT module exploits the similarity among scene super-patches in high-level feature space. By combining these two modules, our MSSPTNet can effectively learn both local and global features from the input point clouds. Finally, the interpolating upsampling and multi-layer perceptrons are exploited to generate semantic labels for the original point cloud data. Experimental results on the public S3DIS dataset demonstrate its efficiency of the proposed network for segmenting large-scale point cloud scenes, especially for those indoor scenes with a large number of repetitive structures, i.e., the network training of our MSSPTNet is much faster than other segmentation networks by a factor of tens to hundreds.

A coarse-to-fine point completion network with details compensation and structure enhancement.

Point cloud completion, the issue of estimating the complete geometry of objects from partially-scanned point cloud data, becomes a fundamental task in many 3d vision and robotics applications. To address the limitations on inadequate prediction of shape details for traditional methods, a novel coarse-to-fine point completion network (DCSE-PCN) is introduced in this work using the modules of local details compensation and shape structure enhancement for effective geometric learning. The coarse completion stage of our network consists of two branches-a shape structure recovery branch and a local details compensation branch, which can recover the overall shape of the underlying model and the shape details of incomplete point cloud through feature learning and hierarchical feature fusion. The fine completion stage of our network employs the structure enhancement module to reinforce the correlated shape structures of the coarse repaired shape (such as regular arrangement or symmetry), thus obtaining the completed geometric shape with finer-grained details. Extensive experimental results on ShapeNet dataset and ModelNet dataset validate the effectiveness of our completion network, which can recover the shape details of the underlying point cloud whilst maintaining its overall shape. Compared to the existing methods, our coarse-to-fine completion network has shown its competitive performance on both chamfer distance (CD) and earth mover distance (EMD) errors. Such as, the repairing results on the ShapeNet dataset of our completion network are reduced by an average of [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] in terms of CD error, comparing with PCN, FoldingNet, Atlas, and CRN methods, respectively; and also reduced by an average of [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] in terms of EMD error, respectively. Meanwhile, the completion results on the ModelNet dataset of our network have an average reduction of [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] in terms of CD error, comparing to PCN, FoldingNet, Atlas, and CRN methods, respectively; and also an average reduction of [Formula: see text], [Formula: see text], [Formula: see text], and [Formula: see text] in terms of EMD error, respectively. Our proposed point completion network is also robust to different degrees of data incompleteness and model noise.

Local imperceptible adversarial attacks against human pose estimation networks.

Deep neural networks are vulnerable to attacks from adversarial inputs. Corresponding attack research on human pose estimation (HPE), particularly for body joint detection, has been largely unexplored. Transferring classification-based attack methods to body joint regression tasks is not straightforward. Another issue is that the attack effectiveness and imperceptibility contradict each other. To solve these issues, we propose local imperceptible attacks on HPE networks. In particular, we reformulate imperceptible attacks on body joint regression into a constrained maximum allowable attack. Furthermore, we approximate the solution using iterative gradient-based strength refinement and greedy-based pixel selection. Our method crafts effective perceptual adversarial attacks that consider both human perception and attack effectiveness. We conducted a series of imperceptible attacks against state-of-the-art HPE methods, including HigherHRNet, DEKR, and ViTPose. The experimental results demonstrate that the proposed method achieves excellent imperceptibility while maintaining attack effectiveness by significantly reducing the number of perturbed pixels. Approximately 4% of the pixels can achieve sufficient attacks on HPE.

An End-to-End Shape-Preserving Point Completion Network.

Shape completion for 3-D point clouds is an important issue in the literature of computer graphics and computer vision. We propose an end-to-end shape-preserving point completion network through encoder-decoder architecture, which works directly on incomplete 3-D point clouds and can restore their overall shapes and fine-scale structures. To achieve this task, we design a novel encoder that encodes information from neighboring points in different orientations and scales, as well as a decoder that outputs dense and uniform complete point clouds. We augment a 3-D object dataset based on ModelNet40 and validate the effectiveness of our shape-preserving completion network. Experimental results demonstrate that the recovered point clouds lie close to ground truth points. Our method outperforms state-of-the-art approaches in terms of Chamfer distance (CD) error and earth mover's distance (EMD) error. Furthermore, our end-to-end completion network is robust to model noise, the different levels of incomplete data, and can also generalize well to unseen objects and real-world data.