Multiscale Deep Learning for Detection and Recognition: A Comprehensive Survey.

Recently, the multiscale problem in computer vision has gradually attracted people's attention. This article focuses on multiscale representation for object detection and recognition, comprehensively introduces the development of multiscale deep learning, and constructs an easy-to-understand, but powerful knowledge structure. First, we give the definition of scale, explain the multiscale mechanism of human vision, and then lead to the multiscale problem discussed in computer vision. Second, advanced multiscale representation methods are introduced, including pyramid representation, scale-space representation, and multiscale geometric representation. Third, the theory of multiscale deep learning is presented, which mainly discusses the multiscale modeling in convolutional neural networks (CNNs) and Vision Transformers (ViTs). Fourth, we compare the performance of multiple multiscale methods on different tasks, illustrating the effectiveness of different multiscale structural designs. Finally, based on the in-depth understanding of the existing methods, we point out several open issues and future directions for multiscale deep learning.

Learning Cross-Domain Features With Dual-Path Signal Transformer.

The past decade has witnessed the rapid development of deep neural networks (DNNs) for automatic modulation classification (AMC). However, most of the available works learn signal features from only a single domain via DNNs, which is not reliable enough to work in uncertain and complex electromagnetic environments. In this brief, a new cross-domain signal transformer (CDSiT) is proposed for AMC, to explore the latent association between different domains of signals. By constructing a signal fusion bottleneck (SFB), CDSiT can implicitly fuse and classify signal features with complementary structures in different domains. Extensive experiments are performed on RadioML2016.10A and RadioML2018.01A, and the results show that CDSiT outperforms its counterparts, particularly for some modulation modes that are difficult to classify before. Through ablation experiences, we also verify the effectiveness of each module in CDSiT.

AFT: Adaptive Fusion Transformer for Visible and Infrared Images.

In this paper, an Adaptive Fusion Transformer (AFT) is proposed for unsupervised pixel-level fusion of visible and infrared images. Different from the existing convolutional networks, transformer is adopted to model the relationship of multi-modality images and explore cross-modal interactions in AFT. The encoder of AFT uses a Multi-Head Self-attention (MSA) module and Feed Forward (FF) network for feature extraction. Then, a Multi-head Self-Fusion (MSF) module is designed for the adaptive perceptual fusion of the features. By sequentially stacking the MSF, MSA, and FF, a fusion decoder is constructed to gradually locate complementary features for recovering informative images. In addition, a structure-preserving loss is defined to enhance the visual quality of fused images. Extensive experiments are conducted on several datasets to compare our proposed AFT method with 21 popular approaches. The results show that AFT has state-of-the-art performance in both quantitative metrics and visual perception.

VMFNet: visual-microwave dual-modality real-time target detection model for detecting damage to curved radar absorbing materials.

Damage to radar absorbing materials (RAMs) reduces the stealth capabilities and battlefield survivability of the equipment. Research on RAM damage detection technology is key to outfield equipment maintenance. In this paper, an intelligent RAM damage detection method based on visual and microwave modalities is proposed. A compressed sensing planar-scanning microwave imaging method based on a range migration algorithm (RMA) imaging operator and fast Gaussian gridding nonuniform fast Fourier transform (FGG-NUFFT) is proposed, achieving high imaging quality and speed. A dual-modality, curved RAM dataset (DCR dataset) is constructed, composed of visual images and microwave images showing two kinds of damage: round shedding and strip cracks. A new dual-modality target detection model, the visual-microwave fusion network (VMFNet), is designed to detect RAM damage. Its mean average precision (mAP) reaches 81.87%, and its inference speed reaches 35.91 fps. A visual network (VisNet) and microwave network (MicNet) are designed as the backbone of VMFNet for extracting the visual and microwave features of RAMs. A path aggregation network (PANet) unit is designed to fuse the multiscale features of the two modalities, resulting in good retention of shallow-level features and high detection accuracy. The head contains different receptive fields and outputs three scales of detection results, effectively detecting damage of different sizes.

Infrared and ultrasonic intelligent damage recognition of composite materials based on deep learning.

With the large-scale application of composite materials in military aircraft, various composite material detection technologies with infrared nondestructive and ultrasonic nondestructive testing as the core have played an important role in detecting composite material component damage in military aircraft. At present, the damage of composite materials is mostly recognized manually, which is time-consuming, laborious, and inefficient. It can effectively improve detection efficiency and accuracy by using intelligent detection methods to detect and recognize damage. Moreover, the effect of infrared detection is significantly reduced with increasing detection depth, while ultrasonic detection has shallow-blind areas. A cascade fusion R-CNN network is proposed in order to comprehensively identify composite material damage. This network realizes the intelligent fusion recognition of infrared and ultrasonic damage images of composite materials. The network is based on a cascade R-CNN network, using fusion modules and BiFPN for improvement. For the infrared image and ultrasonic C-scan image data set established in this paper, the algorithm can identify the type and location of damage detected by infrared and ultrasonic testing. Its recognition accuracy is 99.3% and mean average precision (mAP) is 90.4%. In the detection process, the characteristics of infrared and ultrasonic images are used to realize the recognition of damage depth. Compared to SSD, YOLOv4, faster R-CNN and cascade R-CNN, the network proposed in this paper is better and more effective.

A Transfer Learning Approach for Early Diagnosis of Alzheimer's Disease on MRI Images.

Mild cognitive impairment (MCI) detection using magnetic resonance image (MRI), plays a crucial role in the treatment of dementia disease at an early stage. Deep learning architecture produces impressive results in such research. Algorithms require a large number of annotated datasets for training the model. In this study, we overcome this issue by using layer-wise transfer learning as well as tissue segmentation of brain images to diagnose the early stage of Alzheimer's disease (AD). In layer-wise transfer learning, we used the VGG architecture family with pre-trained weights. The proposed model segregates between normal control (NC), the early mild cognitive impairment (EMCI), the late mild cognitive impairment (LMCI), and the AD. In this paper, 85 NC patients, 70 EMCI, 70 LMCI, and 75 AD patients access form the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Tissue segmentation was applied on each subject to extract the gray matter (GM) tissue. In order to check the validity, the proposed method is tested on preprocessing data and achieved the highest rates of the classification accuracy on AD vs NC is 98.73%, also distinguish between EMCI vs LMCI patients testing accuracy 83.72%, whereas remaining classes accuracy is more than 80%. Finally, we provide a comparative analysis with other studies which shows that the proposed model outperformed the state-of-the-art models in terms of testing accuracy.

Learning Dual Geometric Low-Rank Structure for Semisupervised Hyperspectral Image Classification.

Most of the available graph-based semisupervised hyperspectral image classification methods adopt the cluster assumption to construct a Laplacian regularizer. However, they sometimes fail due to the existence of mixed pixels whose recorded spectra are a combination of several materials. In this paper, we propose a geometric low-rank Laplacian regularized semisupervised classifier, by exploring both the global spectral geometric structure and local spatial geometric structure of hyperspectral data. A new geometric regularized Laplacian low-rank representation (GLapLRR)-based graph is developed to evaluate spectral-spatial affinity of mixed pixels. By revealing the global low-rank and local spatial structure of images via GLapLRR, the constructed graph has the characteristics of spatial-spectral geometry description, robustness, and low sparsity, from which a more accurate classification of mixed pixels can be achieved. The proposed method is experimentally evaluated on three real hyperspectral datasets, and the results show that the proposed method outperforms its counterparts, when only a small number of labeled instances are available.

The complete chloroplast genome of Impatiens uliginosa Franch., an endemic species in Southwest China.

The complete chloroplast genome sequence of Impatiens uliginosa Franch., an endemic species in Southwest China, we research genetic and phylogenetic relationship with other species in an effort to provide genomic resources useful for promoting its conservation and utilization. The total chloroplast genome size of I. uliginosa is 152,609 bp, with a typical quadripartite structure including a pair of inverted repeat (IRs, 25,871 bp) regions separated by a small single copy (SSC, 17,502 bp) region and a large single copy (LSC, 83,365 bp) region. The overall GC content of I. uliginosa plastid genome was 36.8%. The whole chloroplast genome contains 136 genes, including 89 protein-coding genes (PCGs), 38 transfer RNA genes (tRNAs), and 8 ribosomal RNA genes (rRNAs). Among these genes, 15 genes have one intron and 2 genes contain two introns. To investigate the evolution status, the phylogenetic tree based on APG III from 12 complete chloroplast plastomes of Ericales supports close relationships. According to the phylogenetic topologies, I. uliginosa was closely related to I. piufanensis.

The complete chloroplast genome sequence of horticultural plant, Impatiens hawkeri (Sect. Balsaminacea, Impatiens).

The complete chloroplast genome sequence of Impatiens hawker, a widely cultivated horticultural species in the world is 151,692 bp, with a typical quadripartite structure including a pair of inverted repeat (IRs, 25,584 bp) regions separated by a small single copy (SSC, 17,494 bp) region and a large single copy (LSC, 83,029 bp) region. The overall GC content of I. hawker plastid genome was 36.8%. The whole chloroplast genome contains 135 genes, including 89 protein-coding genes (PCGs), 38 transfer RNA genes(tRNAs), and 8 ribosomal RNA genes (rRNAs). Among these genes, 15 genes have one intron and 2 genes contain two introns. To investigate its evolution status, the phylogenetic tree based on APGIII reveal that there are close relationships to the same genus species I. uliginosa and I. piufanensis.

Self-Paced Learning-Based Probability Subspace Projection for Hyperspectral Image Classification.

In this paper a self-paced learning-based probability subspace projection (SL-PSP) method is proposed for hyperspectral image classification. First, a probability label is assigned for each pixel, and a risk is assigned for each labeled pixel. Then, two regularizers are developed from a self-paced maximum margin and a probability label graph, respectively. The first regularizer can increase the discriminant ability of features by gradually involving the most confident pixels into the projection to simultaneously push away heterogeneous neighbors and pull inhomogeneous neighbors. The second regularizer adopts a relaxed clustering assumption to make avail of unlabeled samples, thus accurately revealing the affinity between mixed pixels and achieving accurate classification with very few labeled samples. Several hyperspectral data sets are used to verify the effectiveness of SL-PSP, and the experimental results show that it can achieve the state-of-the-art results in terms of accuracy and stability.

Deep Sparse Tensor Filtering Network for Synthetic Aperture Radar Images Classification.

Recognizing scenes from synthetic aperture radar (SAR) images has been a challenging task due to the increasing resolution of SAR data. Extracting discriminative features from SAR images is extremely difficult for their sensitivity to target aspect. Considering the intractability of the available deep neural networks in practical implementations, in this brief, we propose a simple and efficient deep sparse tensor filtering network (DSTFN) for SAR image classification. An SAR image is first organized into a data tensor by an overlapped partition. Then, a set of dimension-inseparable geometric filters is developed from a least squares support vector machine, followed by a learned sparse filtering of tensors. Finally, the constructed sparse tensor filters are cascaded to a deep network to automatically extract the discriminative features of the image for accurate classification. Simulations are carried out to verify the effectiveness of the proposed DSTFN.