Adaptive locating foveated ghost imaging based on affine transformation.

Ghost imaging (GI) has been widely used in the applications including spectral imaging, 3D imaging, and other fields due to its advantages of broad spectrum and anti-interference. Nevertheless, the restricted sampling efficiency of ghost imaging has impeded its extensive application. In this work, we propose a novel foveated pattern affine transformer method based on deep learning for efficient GI. This method enables adaptive selection of the region of interest (ROI) by combining the proposed retina affine transformer (RAT) network with minimal computational and parametric quantities with the foveated speckle pattern. For single-target and multi-target scenarios, we propose RAT and RNN-RAT (recurrent neural network), respectively. The RAT network enables an adaptive alteration of the fovea of the variable foveated patterns spot to different sizes and positions of the target by predicting the affine matrix with a minor number of parameters for efficient GI. In addition, we integrate a recurrent neural network into the proposed RAT to form an RNN-RAT model, which is capable of performing multi-target ROI detection. Simulations and experimental results show that the method can achieve ROI localization and pattern generation in 0.358 ms, which is a 1 × 105 efficiency improvement compared with the previous methods and improving the image quality of ROI by more than 4 dB. This approach not only improves its overall applicability but also enhances the reconstruction quality of ROI. This creates additional opportunities for real-time GI.

Dual-YOLO Architecture from Infrared and Visible Images for Object Detection.

With the development of infrared detection technology and the improvement of military remote sensing needs, infrared object detection networks with low false alarms and high detection accuracy have been a research focus. However, due to the lack of texture information, the false detection rate of infrared object detection is high, resulting in reduced object detection accuracy. To solve these problems, we propose an infrared object detection network named Dual-YOLO, which integrates visible image features. To ensure the speed of model detection, we choose the You Only Look Once v7 (YOLOv7) as the basic framework and design the infrared and visible images dual feature extraction channels. In addition, we develop attention fusion and fusion shuffle modules to reduce the detection error caused by redundant fusion feature information. Moreover, we introduce the Inception and SE modules to enhance the complementary characteristics of infrared and visible images. Furthermore, we design the fusion loss function to make the network converge fast during training. The experimental results show that the proposed Dual-YOLO network reaches 71.8% mean Average Precision (mAP) in the DroneVehicle remote sensing dataset and 73.2% mAP in the KAIST pedestrian dataset. The detection accuracy reaches 84.5% in the FLIR dataset. The proposed architecture is expected to be applied in the fields of military reconnaissance, unmanned driving, and public safety.

DAN-SuperPoint: Self-Supervised Feature Point Detection Algorithm with Dual Attention Network.

In view of the poor performance of traditional feature point detection methods in low-texture situations, we design a new self-supervised feature extraction network that can be applied to the visual odometer (VO) front-end feature extraction module based on the deep learning method. First, the network uses the feature pyramid structure to perform multi-scale feature fusion to obtain a feature map containing multi-scale information. Then, the feature map is passed through the position attention module and the channel attention module to obtain the feature dependency relationship of the spatial dimension and the channel dimension, respectively, and the weighted spatial feature map and the channel feature map are added element by element to enhance the feature representation. Finally, the weighted feature maps are trained for detectors and descriptors respectively. In addition, in order to improve the prediction accuracy of feature point locations and speed up the network convergence, we add a confidence loss term and a tolerance loss term to the loss functions of the detector and descriptor, respectively. The experiments show that our network achieves satisfactory performance under the Hpatches dataset and KITTI dataset, indicating the reliability of the network.