Spiking Transfer Learning From RGB Image to Neuromorphic Event Stream.

Recent advances in bio-inspired vision with event cameras and associated spiking neural networks (SNNs) have provided promising solutions for low-power consumption neuromorphic tasks. However, as the research of event cameras is still in its infancy, the amount of labeled event stream data is much less than that of the RGB database. The traditional method of converting static images into event streams by simulation to increase the sample size cannot simulate the characteristics of event cameras such as high temporal resolution. To take advantage of both the rich knowledge in labeled RGB images and the features of the event camera, we propose a transfer learning method from the RGB to the event domain in this paper. Specifically, we first introduce a transfer learning framework named R2ETL (RGB to Event Transfer Learning), including a novel encoding alignment module and a feature alignment module. Then, we introduce the temporal centered kernel alignment (TCKA) loss function to improve the efficiency of transfer learning. It aligns the distribution of temporal neuron states by adding a temporal learning constraint. Finally, we theoretically analyze the amount of data required by the deep neuromorphic model to prove the necessity of our method. Numerous experiments demonstrate that our proposed framework outperforms the state-of-the-art SNN and artificial neural network (ANN) models trained on event streams, including N-MNIST, CIFAR10-DVS and N-Caltech101. This indicates that the R2ETL framework is able to leverage the knowledge of labeled RGB images to help the training of SNN on event streams.

Diffusion probabilistic model for bike-sharing demand recovery with factual knowledge fusion.

The mining of diverse patterns from bike flow has attracted widespread interest from researchers and practitioners. Prior arts concentrate on forecasting the flow evolution from bike demand records. Nevertheless, a tricky reality is the frequent occurrence of missing bike flow, which hinders us from accurately understanding flow patterns. This study investigates an interesting task, i.e., Bike-sharing demand recovery (Biker). Biker is not a simple time-series imputation problem, rather, it confronts three concerns: observation uncertainty, complex dependencies, and environmental facts. To this end, we present a novel diffusion probabilistic solution with factual knowledge fusion, namely DBiker. Specifically, DBiker is the first attempt to extend the diffusion probabilistic models to the Biker task, along with a conditional Markov decision-making process. In contrast to existing probabilistic solutions, DBiker forecasts missing observations through progressive steps guided by an adaptive prior. Particularly, we introduce a Flow Conditioner with step embedding and a Factual Extractor to explore the complex dependencies and multiple environmental facts, respectively. Additionally, we devise a self-gated fusion layer that adaptively selects valuable knowledge to act as an adaptive prior, guiding the generation of missing observations. Finally, experiments conducted on three real-world bike systems demonstrate the superiority of DBiker against several baselines.

Adjunct Methods for Alzheimer's Disease Detection: A Review of Auditory Evoked Potentials.

The auditory afferent pathway as a clinical marker of Alzheimer's disease (AD) has sparked interest in investigating the relationship between age-related hearing loss (ARHL) and AD. Given the earlier onset of ARHL compared to cognitive impairment caused by AD, there is a growing emphasis on early diagnosis and intervention to postpone or prevent the progression from ARHL to AD. In this context, auditory evoked potentials (AEPs) have emerged as a widely used objective auditory electrophysiological technique for both the clinical diagnosis and animal experimentation in ARHL due to their non-invasive and repeatable nature. This review focuses on the application of AEPs in AD detection and the auditory nerve system corresponding to different latencies of AEPs. Our objective was to establish AEPs as a systematic and non-invasive adjunct method for enhancing the diagnostic accuracy of AD. The success of AEPs in the early detection and prediction of AD in research settings underscores the need for further clinical application and study.

Event-Driven Spiking Learning Algorithm Using Aggregated Labels.

Traditional spiking learning algorithm aims to train neurons to spike at a specific time or on a particular frequency, which requires precise time and frequency labels in the training process. While in reality, usually only aggregated labels of sequential patterns are provided. The aggregate-label (AL) learning is proposed to discover these predictive features in distracting background streams only by aggregated spikes. It has achieved much success recently, but it is still computationally intensive and has limited use in deep networks. To address these issues, we propose an event-driven spiking aggregate learning algorithm (SALA) in this article. Specifically, to reduce the computational complexity, we improve the conventional spike-threshold-surface (STS) calculation in AL learning by analytical calculating voltage peak values in spiking neurons. Then we derive the algorithm to multilayers by event-driven strategy using aggregated spikes. We conduct comprehensive experiments on various tasks including temporal clue recognition, segmented and continuous speech recognition, and neuromorphic image classification. The experimental results demonstrate that the new STS method improves the efficiency of AL learning significantly, and the proposed algorithm outperforms the conventional spiking algorithm in various temporal clue recognition tasks.

Micro SleepNet: efficient deep learning model for mobile terminal real-time sleep staging.

The real-time sleep staging algorithm that can perform inference on mobile devices without burden is a prerequisite for closed-loop sleep modulation. However, current deep learning sleep staging models have poor real-time efficiency and redundant parameters. We propose a lightweight and high-performance sleep staging model named Micro SleepNet, which takes a 30-s electroencephalography (EEG) epoch as input, without relying on contextual signals. The model features a one-dimensional group convolution with a kernel size of 1 × 3 and an Efficient Channel and Spatial Attention (ECSA) module for feature extraction and adaptive recalibration. Moreover, the model efficiently performs feature fusion using dilated convolution module and replaces the conventional fully connected layer with Global Average Pooling (GAP). These design choices significantly reduce the total number of model parameters to 48,226, with only approximately 48.95 Million Floating-point Operations per Second (MFLOPs) computation. The proposed model is conducted subject-independent cross-validation on three publicly available datasets, achieving an overall accuracy of up to 83.3%, and the Cohen Kappa is 0.77. Additionally, we introduce Class Activation Mapping (CAM) to visualize the model's attention to EEG waveforms, which demonstrate the model's ability to accurately capture feature waveforms of EEG at different sleep stages. This provides a strong interpretability foundation for practical applications. Furthermore, the Micro SleepNet model occupies approximately 100 KB of memory on the Android smartphone and takes only 2.8 ms to infer one EEG epoch, meeting the real-time requirements of sleep staging tasks on mobile devices. Consequently, our proposed model has the potential to serve as a foundation for accurate closed-loop sleep modulation.

Effective Active Learning Method for Spiking Neural Networks.

A large quantity of labeled data is required to train high-performance deep spiking neural networks (SNNs), but obtaining labeled data is expensive. Active learning is proposed to reduce the quantity of labeled data required by deep learning models. However, conventional active learning methods in SNNs are not as effective as that in conventional artificial neural networks (ANNs) because of the difference in feature representation and information transmission. To address this issue, we propose an effective active learning method for a deep SNN model in this article. Specifically, a loss prediction module ActiveLossNet is proposed to extract features and select valuable samples for deep SNNs. Then, we derive the corresponding active learning algorithm for deep SNN models. Comprehensive experiments are conducted on CIFAR-10, MNIST, Fashion-MNIST, and SVHN on different SNN frameworks, including seven-layer CIFARNet and 20-layer ResNet-18. The comparison results demonstrate that the proposed active learning algorithm outperforms random selection and conventional ANN active learning methods. In addition, our method converges faster than conventional active learning methods.

Human-Level Control Through Directly Trained Deep Spiking Q-Networks.

As the third-generation neural networks, spiking neural networks (SNNs) have great potential on neuromorphic hardware because of their high energy efficiency. However, deep spiking reinforcement learning (DSRL), that is, the reinforcement learning (RL) based on SNNs, is still in its preliminary stage due to the binary output and the nondifferentiable property of the spiking function. To address these issues, we propose a deep spiking Q -network (DSQN) in this article. Specifically, we propose a directly trained DSRL architecture based on the leaky integrate-and-fire (LIF) neurons and deep Q -network (DQN). Then, we adapt a direct spiking learning algorithm for the DSQN. We further demonstrate the advantages of using LIF neurons in DSQN theoretically. Comprehensive experiments have been conducted on 17 top-performing Atari games to compare our method with the state-of-the-art conversion method. The experimental results demonstrate the superiority of our method in terms of performance, stability, generalization and energy efficiency. To the best of our knowledge, our work is the first one to achieve state-of-the-art performance on multiple Atari games with the directly trained SNN.

Effective Transfer Learning Algorithm in Spiking Neural Networks.

As the third generation of neural networks, spiking neural networks (SNNs) have gained much attention recently because of their high energy efficiency on neuromorphic hardware. However, training deep SNNs requires many labeled data that are expensive to obtain in real-world applications, as traditional artificial neural networks (ANNs). In order to address this issue, transfer learning has been proposed and widely used in traditional ANNs, but it has limited use in SNNs. In this article, we propose an effective transfer learning framework for deep SNNs based on the domain in-variance representation. Specifically, we analyze the rationality of centered kernel alignment (CKA) as a domain distance measurement relative to maximum mean discrepancy (MMD) in deep SNNs. In addition, we study the feature transferability across different layers by testing on the Office-31, Office-Caltech-10, and PACS datasets. The experimental results demonstrate the transferability of SNNs and show the effectiveness of the proposed transfer learning framework by using CKA in SNNs.

Detection of Microaneurysms in Fundus Images Based on an Attention Mechanism.

Microaneurysms (MAs) are the earliest detectable diabetic retinopathy (DR) lesions. Thus, the ability to automatically detect MAs is critical for the early diagnosis of DR. However, achieving the accurate and reliable detection of MAs remains a significant challenge due to the size and complexity of retinal fundus images. Therefore, this paper presents a novel MA detection method based on a deep neural network with a multilayer attention mechanism for retinal fundus images. First, a series of equalization operations are performed to improve the quality of the fundus images. Then, based on the attention mechanism, multiple feature layers with obvious target features are fused to achieve preliminary MA detection. Finally, the spatial relationships between MAs and blood vessels are utilized to perform a secondary screening of the preliminary test results to obtain the final MA detection results. We evaluated the method on the IDRiD_VOC dataset, which was collected from the open IDRiD dataset. The results show that our method effectively improves the average accuracy and sensitivity of MA detection.

Multi-source sequential knowledge regression by using transfer RNN units.

Transfer learning has achieved a lot of success in deep neural networks to reuse useful knowledge from source domains. However, most of the existing transfer learning strategies on neural networks are for classification tasks or based on simple training strategies, which have limited use in multi-source knowledge regression due to the ineffectiveness of learning common latent features and source information loss in regression. In this paper, we propose transferable Recurrent Neural Network (RNN) units on the Long Short-Term Memory (LSTM) and Gated Recurrent Unit (GRU) to adapt source knowledge in multi-source regression scenarios. Specifically, two knowledge adaptation methods are proposed, the first one utilizes similarity weights as the transfer coefficients of each source, and the other defines a transfer-gate to control the flow of source knowledge. By using the proposed methods, useful source knowledge embedded in both internal state and output is adapted. Extensive experiments on both synthetic data and human motion prediction tasks on the Human 3.6M dataset demonstrate the superiority of our transfer RNN units compared with conventional models.

An Efficient Supervised Training Algorithm for Multilayer Spiking Neural Networks.

The spiking neural networks (SNNs) are the third generation of neural networks and perform remarkably well in cognitive tasks such as pattern recognition. The spike emitting and information processing mechanisms found in biological cognitive systems motivate the application of the hierarchical structure and temporal encoding mechanism in spiking neural networks, which have exhibited strong computational capability. However, the hierarchical structure and temporal encoding approach require neurons to process information serially in space and time respectively, which reduce the training efficiency significantly. For training the hierarchical SNNs, most existing methods are based on the traditional back-propagation algorithm, inheriting its drawbacks of the gradient diffusion and the sensitivity on parameters. To keep the powerful computation capability of the hierarchical structure and temporal encoding mechanism, but to overcome the low efficiency of the existing algorithms, a new training algorithm, the Normalized Spiking Error Back Propagation (NSEBP) is proposed in this paper. In the feedforward calculation, the output spike times are calculated by solving the quadratic function in the spike response model instead of detecting postsynaptic voltage states at all time points in traditional algorithms. Besides, in the feedback weight modification, the computational error is propagated to previous layers by the presynaptic spike jitter instead of the gradient decent rule, which realizes the layer-wised training. Furthermore, our algorithm investigates the mathematical relation between the weight variation and voltage error change, which makes the normalization in the weight modification applicable. Adopting these strategies, our algorithm outperforms the traditional SNN multi-layer algorithms in terms of learning efficiency and parameter sensitivity, that are also demonstrated by the comprehensive experimental results in this paper.

Waveband selection of reagent-free determination for thalassemia screening indicators using Fourier transform infrared spectroscopy with attenuated total reflection.

A reagent-free determination method for the thalassemia screening indicators hemoglobin (Hb), mean corpuscular Hb (MCH), and mean corpuscular volume (MCV) was developed based on Fourier transform infrared spectrometers equipped with an attenuated total reflection accessory. A random and stability-dependent rigorous process of calibration, prediction, and validation was conducted. Appropriate wavebands were selected using the improved moving window partial least squares method with stability and equivalence. The obtained optimal wavebands were 1722 to 1504 cm⁻¹ for Hb, 1653 to 901 cm⁻¹ for MCH, and 1562 to 964 cm⁻¹ for MCV. A model set equivalent to the optimal model was proposed for each indicator; the public waveband of Hb equivalent wavebands was 1717 to 1510 cm⁻¹, and the public equivalent waveband for MCH and MCV was 1562 to 901 cm⁻¹. All selected wavebands were within the MIR fingerprint region and achieved high validation effects. The sensitivity and specificity were 100.0% and 96.9% for the optimal wavebands and 100.0% and 95.3% for the equivalent wavebands, respectively. Thus, the spectral prediction was highly accurate for determining negative and positive for thalassemia screening. This technique is rapid and simple in comparison with conventional methods and is a promising tool for thalassemia screening in large populations.

[Vis-NIR spectroscopic pattern recognition combined with SG smoothing applied to breed screening of transgenic sugarcane].

UNLABELLED: Based on Savitzky-Golay (SG) smoothing screening, principal component analysis (PCA) combined with separately supervised linear discriminant analysis (LDA) and unsupervised hierarchical clustering analysis (HCA) were used for non-destructive visible and near-infrared (Vis-NIR) detection for breed screening of transgenic sugarcane. A random and stability-dependent framework of calibration, prediction, and validation was proposed. A total of 456 samples of sugarcane leaves planting in the elongating stage were collected from the field, which was composed of 306 transgenic (positive) samples containing Bt and Bar gene and 150 non-transgenic (negative) samples. A total of 156 samples (negative 50 and positive 106) were randomly selected as the validation set; the remaining samples (negative 100 and positive 200, a total of 300 samples) were used as the modeling set, and then the modeling set was subdivided into calibration (negative 50 and positive 100, a total of 150 samples) and prediction sets (negative 50 and positive 100, a total of 150 samples) for 50 times. The number of SG smoothing points was ex- panded, while some modes of higher derivative were removed because of small absolute value, and a total of 264 smoothing modes were used for screening. The pairwise combinations of first three principal components were used, and then the optimal combination of principal components was selected according to the model effect. Based on all divisions of calibration and prediction sets and all SG smoothing modes, the SG-PCA-LDA and SG-PCA-HCA models were established, the model parameters were optimized based on the average prediction effect for all divisions to produce modeling stability. Finally, the model validation was performed by validation set. With SG smoothing, the modeling accuracy and stability of PCA-LDA, PCA-HCA were signif- icantly improved. For the optimal SG-PCA-LDA model, the recognition rate of positive and negative validation samples were 94.3%, 96.0%; and were 92.5%, 98.0% for the optimal SG-PCA-LDA model, respectively. CONCLUSION: Vis-NIR spectro- scopic pattern recognition combined with SG smoothing could be used for accurate recognition of transgenic sugarcane leaves, and provided a convenient screening method for transgenic sugarcane breeding.

[Fourier transform infrared spectroscopy combined with attenuated total reflection applied to reagent-free quantitative analysis of thalassemia screening indicators].

UNLABELLED: A simultaneous quantitative analysis method for the thalassemia screening indicators mean corpuscular hemoglobin (MCH), mean corpuscular volume (MCV), and hemoglobin (Hb) was developed with Fourier transform infrared (FTIR) spectrometers and attenuated total reflection (ATR) combined with partial least squares (PLS). A total of 380 human peripheral blood samples were collected, which were composed of 180 positive samples and 200 negative samples according to the criteria of hematological indicator screening for thalassemia. One hundred fifty samples (64 negative, 86 positive) were randomly selected from all samples as the validation set, the remaining 230 samples (136 negative, 94 positive) were used as modeling samples; and then the modeling set was further subdivided into calibration set (68 negative, 47 positive, and 115 in total) and prediction set (68 negative, 47 positive, and 115 in total) for 200 times. Comparison of experimental results show that the prediction effect of PLS models in mid-infrared (MIR) fingerprint region (1,600-900 cm(-1)) was significantly better those of PLS models in the full scanning region (4,000-600 cm(-1)), and model complexity is significantly reduced. Based on PLS model in MIR fingerprint region, the optimal numbers of PLS factors for MCH, MCV and Hb were 10, 10 and 6, respectively, and the root mean square error (M_SEP(Ave)) and the correlation coefficient (M_Rp, Ave) of prediction in the modeling set were 2.19 pg, 0.902 for MCH, 5.13 fL, 0.898 for MCV and 8.0 g · L(-1), 0.922 for Hb, respectively. The root mean square error (V_SEP) and the correlation coefficient (V_Rp) of prediction in the validation set were 2.22 pg, 0.900 for MCH, 5.38 fL, 0.895 for MCV and 7.7 g · L(-1), 0.929 for Hb, respectively. The sensitivity and specificity for thalassemia screening achieved 100.0% and 95.3%, respectively. CONCLUSION: FTIR/ATR spectroscopy combined with PLS method could provide a new reagent-free and rapid technique for thalassemia screening for large populations.