RESUMO
Motor imagery (MI) based brain computer interface significantly oriented the development of neuro-rehabilitation, and the crucial issue is how to accurately detect the changes of cerebral cortex for MI decoding. The brain activity can be calculated based on the head model and observed scalp EEG, providing insights regarding cortical dynamics by using equivalent current dipoles with high spatial and temporal resolution. Now, all the dipoles within entire cortex or partial regions of interest are directly applied to data representation, this may make the key information weakened or lost, and it is worth studying how to choose the most important from numerous dipoles. In this paper, we devote to building a simplified distributed dipoles model (SDDM), which is combined with convolutional neural network (CNN), generating a MI decoding method at source level (called SDDM-CNN). First, all channels of raw MI-EEG signals are subdivided by a series of bandpass filters with width of 1 Hz, the average energies associated with any sub-band signals are calculated and ranked in a descending order to screen the top n sub-bands; then, the MI-EEG signals over each selected sub-band are mapped into source space by using EEG source imaging technology, and for each scout of neuroanatomical Desikan-Killiany partition, a centered dipole is selected as the most relevant dipole and put together to build a SDDM to reflect the neuroelectric activity of entire cerebral cortex; finally, the 4 dimensional (4D) magnitude matrix is constructed for each SDDM and fused into a novel data representation, which is further input to a well-designed 3DCNN with n parallel branches (nB3DCNN) to extract and classify the comprehensive features from time-frequency-space dimensions. Experiments are carried out on three public datasets, and the average ten-fold CV decoding accuracies achieve 95.09%, 97.98% and 94.53% respectively, and the statistical analysis is fulfilled by standard deviation, kappa value and confusion matrix. Experiment results suggest that it is beneficial to pick out the most sensitive sub-bands in sensor domain, and SDDM can sufficiently describe the dynamic changing of entire cortex, improving decoding performance while greatly reducing number of source signals. Also, nB3DCNN is capable of exploring spatial-temporal features from multi sub-bands.
RESUMO
Objective. Motor imagery electroencephalography (MI-EEG) produces one of the most commonly used biosignals in intelligent rehabilitation systems. The newly developed 3D convolutional neural network (3DCNN) is gaining increasing attention for its ability to recognize MI tasks. The key to successful identification of movement intention is dependent on whether the data representation can faithfully reflect the cortical activity induced by MI. However, the present data representation, which is often generated from partial source signals with time-frequency analysis, contains incomplete information. Therefore, it would be beneficial to explore a new type of data representation using raw spatiotemporal dipole information as well as the possible development of a matching 3DCNN.Approach.Based on EEG source imaging and 3DCNN, a novel decoding method for identifying MI tasks is proposed, called ESICNND. MI-EEG is mapped to the cerebral cortex by the standardized low resolution electromagnetic tomography algorithm, and the optimal sampling points of the dipoles are selected as the time of interest to best reveal the difference between any two MI tasks. Then, the initial subject coordinate system is converted to a magnetic resonance imaging coordinate system, followed by dipole interpolation and volume down-sampling; the resulting 3D dipole amplitude matrices are merged at the selected sampling points to obtain 4D dipole feature matrices (4DDFMs). These matrices are augmented by sliding window technology and input into a 3DCNN with a cascading architecture of three modules (3M3DCNN) to perform the extraction and classification of comprehensive features.Main results.Experiments are carried out on two public datasets; the average ten-fold CV classification accuracies reach 88.73% and 96.25%, respectively, and the statistical analysis demonstrates outstanding consistency and stability.Significance.The 4DDFMs reveals the variation of cortical activation in a 3D spatial cube with a temporal dimension and matches the 3M3DCNN well, making full use of the high-resolution spatiotemporal information from all dipoles.
