Electronic eye and electronic tongue data fusion combined with a GETNet model for the traceability and detection of Astragalus.

BACKGROUND: Astragalus is a widely used traditional Chinese medicine material that is easily confused due to its quality, price and other factors derived from different origins. This article describes a novel method for the rapid tracing and detection of Astragalus via the joint application of an electronic tongue (ET) and an electronic eye (EE) combined with a lightweight convoluted neural network (CNN)-transformer model. First, ET and EE systems were employed to measure the taste fingerprints and appearance images, respectively, of different Astragalus samples. Three spectral transform methods - the Markov transition field, short-time Fourier transform and recurrence plot - were utilized to convert the ET signals into 2D spectrograms. Then, the obtained ET spectrograms were fused with the EE image to obtain multimodal information. A lightweight hybrid model, termed GETNet, was designed to achieve pattern recognition for the Astragalus fusion information. The proposed model employed an improved transformer module and an improved Ghost bottleneck as its backbone network, complementarily utilizing the benefits of CNN and transformer architectures for local and global feature representation. Furthermore, the Ghost bottleneck was further optimized using a channel attention technique, which boosted the model's feature extraction effectiveness. RESULTS: The experiments indicate that the proposed data fusion strategy based on ET and EE devices has better recognition accuracy than that attained with independent sensing devices. CONCLUSION: The proposed method achieved high precision (99.1%) and recall (99.1%) values, providing a novel approach for rapidly identifying the origin of Astragalus, and it holds great promise for applications involving other types of Chinese herbal medicines. © 2024 Society of Chemical Industry.

Real-Time Photometric Calibrated Monocular Direct Visual SLAM.

To solve the illumination sensitivity problems of mobile ground equipment, an enhanced visual SLAM algorithm based on the sparse direct method was proposed in this paper. Firstly, the vignette and response functions of the input sequences were optimized based on the photometric formation of the camera. Secondly, the Shi-Tomasi corners of the input sequence were tracked, and optimization equations were established using the pixel tracking of sparse direct visual odometry (VO). Thirdly, the Levenberg-Marquardt (L-M) method was applied to solve the joint optimization equation, and the photometric calibration parameters in the VO were updated to realize the real-time dynamic compensation of the exposure of the input sequences, which reduced the effects of the light variations on SLAM's (simultaneous localization and mapping) accuracy and robustness. Finally, a Shi-Tomasi corner filtered strategy was designed to reduce the computational complexity of the proposed algorithm, and the loop closure detection was realized based on the oriented FAST and rotated BRIEF (ORB) features. The proposed algorithm was tested using TUM, KITTI, EuRoC, and an actual environment, and the experimental results show that the positioning and mapping performance of the proposed algorithm is promising.