Global Clue-Guided Cross-Memory Quaternion Transformer Network for Multisource Remote Sensing Data Classification.

Multisource remote sensing data classification is a challenging research topic, and how to address the inherent heterogeneity between multimodal data while exploring their complementarity is crucial. Existing deep learning models usually directly adopt feature-level fusion designs, most of which, however, fail to overcome the impact of heterogeneity, limiting their performance. As such, a multimodal joint classification framework, called global clue-guided cross-memory quaternion transformer network (GCCQTNet), is proposed for multisource data i.e., hyperspectral image (HSI) and synthetic aperture radar (SAR)/light detection and ranging (LiDAR) classification. First, a three-branch structure is built to extract the local and global features, where an independent squeeze-expansion-like fusion (ISEF) structure is designed to update the local and global representations by considering the global information as an agent, suppressing the negative impact of multimodal heterogeneity layer by layer. A cross-memory quaternion transformer (CMQT) structure is further constructed to model the complex inner relationships between the intramodality and intermodality features to capture more discriminative fusion features that fully characterize multimodal complementarity. Finally, a cross-modality comparative learning (CMCL) structure is developed to impose the consistency constraint on global information learning, which, in conjunction with a classification head, is used to guide the end-to-end training of GCCQTNet. Extensive experiments on three public multisource remote sensing datasets illustrate the superiority of our GCCQTNet with regards to other state-of-the-art methods.

Weakly Supervised Building Semantic Segmentation Based on Spot-Seeds and Refinement Process.

Automatic building semantic segmentation is the most critical and relevant task in several geospatial applications. Methods based on convolutional neural networks (CNNs) are mainly used in current building segmentation. The requirement of huge pixel-level labels is a significant obstacle to achieve the semantic segmentation of building by CNNs. In this paper, we propose a novel weakly supervised framework for building segmentation, which generates high-quality pixel-level annotations and optimizes the segmentation network. A superpixel segmentation algorithm can predict a boundary map for training images. Then, Superpixels-CRF built on the superpixel regions is guided by spot seeds to propagate information from spot seeds to unlabeled regions, resulting in high-quality pixel-level annotations. Using these high-quality pixel-level annotations, we can train a more robust segmentation network and predict segmentation maps. To iteratively optimize the segmentation network, the predicted segmentation maps are refined, and the segmentation network are retrained. Comparative experiments demonstrate that the proposed segmentation framework achieves a marked improvement in the building's segmentation quality while reducing human labeling efforts.

A3 CLNN: Spatial, Spectral and Multiscale Attention ConvLSTM Neural Network for Multisource Remote Sensing Data Classification.

The problem of effectively exploiting the information multiple data sources has become a relevant but challenging research topic in remote sensing. In this article, we propose a new approach to exploit the complementarity of two data sources: hyperspectral images (HSIs) and light detection and ranging (LiDAR) data. Specifically, we develop a new dual-channel spatial, spectral and multiscale attention convolutional long short-term memory neural network (called dual-channel A3 CLNN) for feature extraction and classification of multisource remote sensing data. Spatial, spectral, and multiscale attention mechanisms are first designed for HSI and LiDAR data in order to learn spectral- and spatial-enhanced feature representations and to represent multiscale information for different classes. In the designed fusion network, a novel composite attention learning mechanism (combined with a three-level fusion strategy) is used to fully integrate the features in these two data sources. Finally, inspired by the idea of transfer learning, a novel stepwise training strategy is designed to yield a final classification result. Our experimental results, conducted on several multisource remote sensing data sets, demonstrate that the newly proposed dual-channel A 3 CLNN exhibits better feature representation ability (leading to more competitive classification performance) than other state-of-the-art methods.

An Efficient Supervised Deep Hashing Method for Image Retrieval.

In recent years, searching and retrieving relevant images from large databases has become an emerging challenge for the researcher. Hashing methods that mapped raw data into a short binary code have attracted increasing attention from the researcher. Most existing hashing approaches map samples to a binary vector via a single linear projection, which restricts the flexibility of those methods and leads to optimization problems. We introduce a CNN-based hashing method that uses multiple nonlinear projections to produce additional short-bit binary code to tackle this issue. Further, an end-to-end hashing system is accomplished using a convolutional neural network. Also, we design a loss function that aims to maintain the similarity between images and minimize the quantization error by providing a uniform distribution of the hash bits to illustrate the proposed technique's effectiveness and significance. Extensive experiments conducted on various datasets demonstrate the superiority of the proposed method in comparison with state-of-the-art deep hashing methods.

Variational Textured Dirichlet Process Mixture Model with Pairwise Constraint for Unsupervised Classification of Polarimetric SAR Images.

This paper proposes an unsupervised classification method for multilook polarimetric synthetic aperture radar (Pol-SAR) data. The proposed method simultaneously deals with the heterogeneity and incorporates the local correlation in PolSAR images. Specifically, within the probabilistic framework of the Dirichlet process mixture model (DPMM), an observed PolSAR data point is described by the multiplication of a Wishartdistributed component and a class-dependent random variable (i.e., the textual variable). This modeling scheme leads to the proposed textured DPMM (tDPMM), which possesses more flexibility in characterizing PolSAR data in heterogeneous areas and from high-resolution images due to the introduction of the classdependent texture variable. The proposed tDPMM is learned by solving an optimization problem to achieve its Bayesian inference. With the knowledge of this optimization-based learning, the local correlation is incorporated through the pairwise constraint, which integrates an appropriate penalty term into the objective function so as to encourage the neighboring pixels to fall into the same category and to alleviate the "salt-and-pepper" classification appearance.We develop the learning algorithm with all the closed-form updates. The performance of the proposed method is evaluated with both low-resolution and high-resolution PolSAR images, which involve homogeneous, heterogeneous, and extremely heterogeneous areas. The experimental results reveal that the class-dependent texture variable is beneficial to PolSAR image classification and the pairwise constraint can effectively incorporate the local correlation in PolSAR images.

Evaluation of characteristics and surgical outcomes in cervical spondylotic amyotrophy.

BACKGROUND: Cervical spondylotic amyotrophy (CSA) is a rare clinical syndrome resulting from cervical spondylosis. Surgical treatment includes anterior cervical decompression and fusion (ACDF), and laminoplasty with or without foraminotomy. Some studies indicate that ACDF is an effective method for treating CSA because anterior decompression with or without medial foraminotomy can completely eliminate anterior and/or anterolateral lesions. We retrospectively evaluated outcome of surgical outcome by anterior cervical decompression and fusion (ACDF). MATERIALS AND METHODS: 28 CSA patients, among whom 12 had proximal type CSA and 16 had distal type CSA, treated by ACDF, were evaluated clinicoradiologically. The improvement in atrophic muscle power was assessed by manual muscle testing (MMT) and the recovery rate of the patients was determined on the basis of the Japanese Orthopedic Association (JOA) scores. Patient satisfaction was also examined. RESULTS: The percentage of patients, who gained 1 or more grades of muscle power improvement, as determined by MMT, was 91.7% for those with proximal type CSA and 37.5% for those with distal type CSA (P < 0.01). The JOA score-based recovery rates of patients with proximal type and distal type CSA were 60.8% and 41.8%, respectively (P < 0.05). Patient satisfaction was 8.2 for those with proximal type CSA and 6.9 for those with distal type CSA (P < 0.01). A correlation was observed among the levels of improvement in muscle power, JOA score based recovery rate, patient satisfaction and course of disease (P < 0.05). CONCLUSION: ACDF can effectively improve the clinical function of patients with CSA and result in good patient satisfaction despite the surgical outcomes for distal type CSA being inferior to those for proximal type CSA. Course of disease is the fundamental factor that affects the surgical outcomes for CSA. We recommend that patients with CSA undergo surgical intervention as early as possible.