Surface Reconstruction from Point Clouds: A Survey and a Benchmark.

Reconstruction of a continuous surface of two-dimensional manifold from its raw, discrete point cloud observation is a long-standing problem in computer vision and graphics research. The problem is technically ill-posed, and becomes more difficult considering that various sensing imperfections would appear in the point clouds obtained by practical depth scanning. In literature, a rich set of methods has been proposed, and reviews of existing methods are also provided. However, existing reviews are short of thorough investigations on a common benchmark. The present paper aims to review and benchmark existing methods in the new era of deep learning surface reconstruction. To this end, we contribute a large-scale benchmarking dataset consisting of both synthetic and real-scanned data; the benchmark includes object- and scene-level surfaces and takes into account various sensing imperfections that are commonly encountered in practical depth scanning. We conduct thorough empirical studies by comparing existing methods on the constructed benchmark, and pay special attention on robustness of existing methods against various scanning imperfections; we also study how different methods generalize in terms of reconstructing complex surface shapes. Our studies help identity the best conditions under which different methods work, and suggest some empirical findings. For example, while deep learning methods are increasingly popular in the research community, our systematic studies suggest that, surprisingly, a few classical methods perform even better in terms of both robustness and generalization; our studies also suggest that the practical challenges of misalignment of point sets from multi-view scanning, missing of surface points, and point outliers remain unsolved by all the existing surface reconstruction methods. We expect that the benchmark and our studies would be valuable both for practitioners and as a guidance for new innovations in future research. We make the benchmark publicly accessible at https://Gorilla-Lab-SCUT.github.io/SurfaceReconstructionBenchmark.

MSSPA-GC: Multi-Scale Shape Prior Adaptation with 3D Graph Convolutions for Category-Level Object Pose Estimation.

Category-level object pose estimation aims to predict the 6D object pose and size of arbitrary objects from known categories. It remains a challenge due to the large intra-class shape variation. Recently, the introduction of the shape prior adaptation mechanism into the normalized canonical coordinates (i.e., NOCS) reconstruction process has been shown to be effective in mitigating the intra-class shape variation. However, existing shape prior adaptation methods simply map the observed point cloud to the normalized object space, and the extracted object descriptors are not sufficient for the perception of the object pose. As a result, they fail to predict the pose of objects with complex geometric structures (e.g., cameras). To this end, this paper proposes a novel shape prior adaption method named MSSPA-GC for category-level object pose estimation. Specifically, our main network takes the observed instance point cloud converted from the RGB-D image and the prior shape point cloud pre-trained on the object CAD models as inputs. Then, a novel 3D graph convolution network and a PointNet-like MLP network are designed to extract pose-aware object features and shape-aware object features from these two inputs, respectively. After that, the two-stream object features are aggregated through a multi-scale feature propagation mechanism to generate comprehensive 3D object descriptors that maintain both pose-sensitive geometric stability and intra-class shape consistency. Finally, by leveraging object descriptors aware of both object pose and shape when reconstructing the NOCS coordinates, our approach elegantly achieves state-of-the-art performance on the widely used REAL275 and CAMERA25 datasets using only 25% of the parameters compared with existing shape prior adaptation models. Moreover, our method also exhibits decent generalization ability on the unconstrained REDWOOD75 dataset.

MonkeyNet: A robust deep convolutional neural network for monkeypox disease detection and classification.

The monkeypox virus poses a new pandemic threat while we are still recovering from COVID-19. Despite the fact that monkeypox is not as lethal and contagious as COVID-19, new patient cases are recorded every day. If preparations are not made, a global pandemic is likely. Deep learning (DL) techniques are now showing promise in medical imaging for figuring out what diseases a person has. The monkeypox virus-infected human skin and the region of the skin can be used to diagnose the monkeypox early because an image has been used to learn more about the disease. But there is still no reliable Monkeypox database that is available to the public that can be used to train and test DL models. As a result, it is essential to collect images of monkeypox patients. The "MSID" dataset, short form of "Monkeypox Skin Images Dataset", which was developed for this research, is free to use and can be downloaded from the Mendeley Data database by anyone who wants to use it. DL models can be built and used with more confidence using the images in this dataset. These images come from a variety of open-source and online sources and can be used for research purposes without any restrictions. Furthermore, we proposed and evaluated a modified DenseNet-201 deep learning-based CNN model named MonkeyNet. Using the original and augmented datasets, this study suggested a deep convolutional neural network that was able to correctly identify monkeypox disease with an accuracy of 93.19% and 98.91% respectively. This implementation also shows the Grad-CAM which indicates the level of the model's effectiveness and identifies the infected regions in each class image, which will help the clinicians. The proposed model will also help doctors make accurate early diagnoses of monkeypox disease and protect against the spread of the disease.

6D-ViT: Category-Level 6D Object Pose Estimation via Transformer-Based Instance Representation Learning.

This paper presents 6D vision transformer (6D-ViT), a transformer-based instance representation learning network suitable for highly accurate category-level object pose estimation based on RGB-D images. Specifically, a novel two-stream encoder-decoder framework is dedicated to exploring complex and powerful instance representations from RGB images, point clouds, and categorical shape priors. The whole framework consists of two main branches, named Pixelformer and Pointformer. Pixelformer contains a pyramid transformer encoder with an all-multilayer perceptron (MLP) decoder to extract pixelwise appearance representations from RGB images, while Pointformer relies on a cascaded transformer encoder and an all-MLP decoder to acquire the pointwise geometric characteristics from point clouds. Then, dense instance representations (i.e., correspondence matrix and deformation field) for NOCS model reconstruction are obtained from a multisource aggregation (MSA) network with shape prior, appearance and geometric information as inputs. Finally, the instance 6D pose is computed by solving the similarity transformation between the observed point clouds and the reconstructed NOCS representations. Extensive experiments with synthetic and real-world datasets demonstrate that the proposed framework achieves state-of-the-art performance for both datasets. Code is available at https://github.com/luzzou/6D-ViT.

Active contour-based cell segmentation during freezing and its application in cryopreservation.

Water permeability of the plasma membrane plays an important role in making optimal cryopreservation protocols for different types of cells. To quantify water permeability effectively, automated cell volume segmentation during freezing is necessary. Unfortunately, there exists so far no efficient and accurate segmentation method to handle this kind of image processing task gracefully. The existence of extracellular ice and variable background present significant challenges for most traditional segmentation algorithms. In this paper, we propose a novel approach to reliably extract cells from the extracellular ice, which attaches to or surrounds cells. Our method operates on temporal image sequences and is composed of two steps. First, for each image from the sequence, a greedy search strategy is employed to track approximate locations of cells in motion. Second, we utilize a localized competitive active contour model to obtain the contour of each cell. Based on the first step's result, the initial contour for level set evolution can be determined appropriately, thus considerably easing the pain of initialization for an active contour model. Experimental results demonstrate that the proposed method is efficient and effective in segmenting cells during freezing.