Convolutional MLP orthogonal fusion of multiscale features for visual place recognition.

Visual place recognition (VPR) involves obtaining robust image descriptors to cope with differences in camera viewpoints and drastic external environment changes. Utilizing multiscale features improves the robustness of image descriptors; however, existing methods neither exploit the multiscale features generated during feature extraction nor consider the feature redundancy problem when fusing multiscale information when image descriptors are enhanced. We propose a novel encoding strategy-convolutional multilayer perceptron orthogonal fusion of multiscale features (ConvMLP-OFMS)-for VPR. A ConvMLP is used to obtain robust and generalized global image descriptors and the multiscale features generated during feature extraction are used to enhance the global descriptors to cope with changes in the environment and viewpoints. Additionally, an attention mechanism is used to eliminate noise and redundant information. Compared to traditional methods that use tensor splicing for feature fusion, we introduced matrix orthogonal decomposition to eliminate redundant information. Experiments demonstrated that the proposed architecture outperformed NetVLAD, CosPlace, ConvAP, and other methods. On the Pittsburgh and MSLS datasets, which contained significant viewpoint and illumination variations, our method achieved 92.5% and 86.5% Recall@1, respectively. We also achieved good performances-80.6% and 43.2%-on the SPED and NordLand datasets, respectively, which have more extreme illumination and appearance variations.

Lightweight Cross-Modal Information Mutual Reinforcement Network for RGB-T Salient Object Detection.

RGB-T salient object detection (SOD) has made significant progress in recent years. However, most existing works are based on heavy models, which are not applicable to mobile devices. Additionally, there is still room for improvement in the design of cross-modal feature fusion and cross-level feature fusion. To address these issues, we propose a lightweight cross-modal information mutual reinforcement network for RGB-T SOD. Our network consists of a lightweight encoder, the cross-modal information mutual reinforcement (CMIMR) module, and the semantic-information-guided fusion (SIGF) module. To reduce the computational cost and the number of parameters, we employ the lightweight module in both the encoder and decoder. Furthermore, to fuse the complementary information between two-modal features, we design the CMIMR module to enhance the two-modal features. This module effectively refines the two-modal features by absorbing previous-level semantic information and inter-modal complementary information. In addition, to fuse the cross-level feature and detect multiscale salient objects, we design the SIGF module, which effectively suppresses the background noisy information in low-level features and extracts multiscale information. We conduct extensive experiments on three RGB-T datasets, and our method achieves competitive performance compared to the other 15 state-of-the-art methods.

Development and CT image-domain validation of a computational lung lesion model for use in virtual imaging trials.

PURPOSE: Computational abnormalities (e.g., lesion models) for use in medical imaging simulation studies are frequently generated using data collected from clinical images. Although this approach allows for highly-customizable lesion detectability studies on clinical computed tomography (CT) data, the ground-truth lesion models produced with this method do not provide a sufficiently realistic lesion morphology for use with current anthropomorphic simulation studies. This work is intended to demonstrate that the new anatomically-informed lesion model presented here is not inferior to the previous lesion model under CT imaging, and can therefore provide a more biologically-informed model for use with simulated CT imaging studies. METHODS: The lesion model was simulated initially from a seed cell with 10 µm diameter placed in an anatomical location within segmented lung CT and was allowed to reproduce locally within the available solid angle in discrete time-intervals (corresponding to synchronous cell cycles) up to a size of ∼200 µm in diameter. Daughter cells of generation G were allowed also to reproduce on the next available time-step given sufficient space. At lesion sizes beyond 200 µm in diameter, the health of subregions of cells were tracked with a Markov chain technique, indicating which regions were likely to continue growing, which were likely stable, and which were likely to develop necrosis given their proximity to anatomical features and other lesion cells. For lesion sizes beyond 500 µm, the lesion was represented with three nested, triangulated surfaces (corresponding to proliferating, dormant, and necrotic regions), indicating how discrete volumes of the lesion were behaving at a particular time. Lesions were then assigned smoothly-varying material properties based on their cellular level health in each region, resulting in a multi-material lesion model. The lesions produced with this model were then voxelized and placed into lung CT images for comparison with both prior work and clinical data. This model was subject to an observer study in which cardiothoracic imaging radiologists assessed the realism of both clinical and synthetic lesions in CT images. RESULTS: The useable outputs of this work were voxel- or surface-based, validated, computational lesions, at a scale clearly visible on clinical CT (3-4 cm). Analysis of the observer study results indicated that the computationally-generated lesions were indistinguishable from clinical lesions (AUC = 0.49, 95% CI = [0.36, 0.61]) and non-inferior to an earlier image-based lesion model-indicating the advantage of the model for use in both hybrid CT images and in simulated CT imaging of the lungs. CONCLUSIONS: Results indicated the non-inferiority of this model as compared to previous methods, indicating the utility of the model for use in both hybrid CT images and in simulated CT imaging.

2D-3D cascade network for glioma segmentation in multisequence MRI images using multiscale information.

BACKGROUND AND OBJECTIVE: Glioma segmentation is an important procedure for the treatment plan and follow-up evaluation of patients with glioma. UNet-based networks are widely used in medical image segmentation tasks and have achieved state-of-the-art performance. However, context information along the third dimension is ignored in 2D convolutions, whereas difference between z-axis and in-plane resolutions is large in 3D convolutions. Moreover, an original UNet structure cannot capture fine details because of the reduced resolution of feature maps near bottleneck layers. METHODS: To address these issues, a novel 2D-3D cascade network with multiscale information module is proposed for the multiclass segmentation of gliomas in multisequence MRI images. First, a 2D network is applied to fully exploit potential intra-slice features. A variational autoencoder module is incorporated into 2D DenseUNet to regularize a shared encoder, extract useful information, and represent glioma heterogeneity. Second, we integrated 3D DenseUNet with the 2D network in cascade mode to extract useful inter-slice features and alleviate the influence of large difference between z-axis and in-plane resolutions. Moreover, a multiscale information module is used in the 2D and 3D networks to further capture the fine details of gliomas. Finally, the whole 2D-3D cascade network is trained in an end-to-end manner, where the intra-slice and inter-slice features are fused and optimized jointly to take full advantage of 3D image information. RESULTS: Our method is evaluated on publicly available and clinical datasets and achieves competitive performance in these two datasets. CONCLUSIONS: These results indicate that the proposed method may be a useful tool for glioma segmentation.

Registration of Magnetic Resonance Tomography (MRT) Data with a Low Frequency Adaption of Fourier-Mellin-SOFT (LF-FMS).

Fourier-Mellin-SOFT (FMS) is a rigid 3D registration method, which allows the robust registration of 3 degrees-of-freedom (dof) rotation, 1-dof scale, and 3-dof translation between scans on discrete grids. FMS is based on a spectral decomposition of these 7-dof. This complete spectral representation of the input data enables an adaption to certain frequency ranges. This special property is used here to focus on relevant mutual 3D information between bone structures with a Low Frequency adaptation of FMS (LF-FMS), that is, it is utilized for matching and concurrently determining corresponding transformation parameters. This process is applied on a set of Magnetic Resonance Tomography (MRT) data representing the hand region, in particular the carpal bone area, in a sequence of different hand positions. This data set is available for different probands, which allows a comparison of resulting parameter plots and furthermore matching in between bone structures.

Recognition of Emotional States Using Multiscale Information Analysis of High Frequency EEG Oscillations.

Exploring the manifestation of emotion in electroencephalogram (EEG) signals is helpful for improving the accuracy of emotion recognition. This paper introduced the novel features based on the multiscale information analysis (MIA) of EEG signals for distinguishing emotional states in four dimensions based on Russell's circumplex model. The algorithms were applied to extract features on the DEAP database, which included multiscale EEG complexity index in the time domain, and ensemble empirical mode decomposition enhanced energy and fuzzy entropy in the frequency domain. The support vector machine and cross validation method were applied to assess classification accuracy. The classification performance of MIA methods (accuracy = 62.01%, precision = 62.03%, recall/sensitivity = 60.51%, and specificity = 82.80%) was much higher than classical methods (accuracy = 43.98%, precision = 43.81%, recall/sensitivity = 41.86%, and specificity = 70.50%), which extracted features contain similar energy based on a discrete wavelet transform, fractal dimension, and sample entropy. In this study, we found that emotion recognition is more associated with high frequency oscillations (51-100Hz) of EEG signals rather than low frequency oscillations (0.3-49Hz), and the significance of the frontal and temporal regions are higher than other regions. Such information has predictive power and may provide more insights into analyzing the multiscale information of high frequency oscillations in EEG signals.