Emotion Recognition Based on EEG Using Generative Adversarial Nets and Convolutional Neural Network.

Emotion recognition plays an important role in the field of human-computer interaction (HCI). Automatic emotion recognition based on EEG is an important topic in brain-computer interface (BCI) applications. Currently, deep learning has been widely used in the field of EEG emotion recognition and has achieved remarkable results. However, due to the cost of data collection, most EEG datasets have only a small amount of EEG data, and the sample categories are unbalanced in these datasets. These problems will make it difficult for the deep learning model to predict the emotional state. In this paper, we propose a new sample generation method using generative adversarial networks to solve the problem of EEG sample shortage and sample category imbalance. In experiments, we explore the performance of emotion recognition with the frequency band correlation and frequency band separation computational models before and after data augmentation on standard EEG-based emotion datasets. Our experimental results show that the method of generative adversarial networks for data augmentation can effectively improve the performance of emotion recognition based on the deep learning model. And we find that the frequency band correlation deep learning model is more conducive to emotion recognition.

Task Transfer Learning for EEG Classification in Motor Imagery-Based BCI System.

The motor-imagery brain-computer interface system (MI-BCI) has a board prospect for development. However, long calibration time and lack of enough MI commands limit its use in practice. In order to enlarge the command set, we add the combinations of traditional MI commands as new commands into the command set. We also design an algorithm based on transfer learning so as to decrease the calibration time for collecting EEG signal and training model. We create feature extractor based on data from traditional commands and transfer patterns through the data from new commands. Through the comparison of the average accuracy between our algorithm and traditional algorithms and the visualization of spatial patterns in our algorithm, we find that the accuracy of our algorithm is much higher than traditional algorithms, especially as for the low-quality datasets. Besides, the visualization of spatial patterns is meaningful. The algorithm based on transfer learning takes the advantage of the information from source data. We enlarge the command set while shortening the calibration time, which is of significant importance to the MI-BCI application.

Instance Transfer Subject-Dependent Strategy for Motor Imagery Signal Classification Using Deep Convolutional Neural Networks.

In the process of brain-computer interface (BCI), variations across sessions/subjects result in differences in the properties of potential of the brain. This issue may lead to variations in feature distribution of electroencephalogram (EEG) across subjects, which greatly reduces the generalization ability of a classifier. Although subject-dependent (SD) strategy provides a promising way to solve the problem of personalized classification, it cannot achieve expected performance due to the limitation of the amount of data especially for a deep neural network (DNN) classification model. Herein, we propose an instance transfer subject-independent (ITSD) framework combined with a convolutional neural network (CNN) to improve the classification accuracy of the model during motor imagery (MI) task. The proposed framework consists of the following steps. Firstly, an instance transfer learning based on the perceptive Hash algorithm is proposed to measure similarity of spectrogram EEG signals between different subjects. Then, we develop a CNN to decode these signals after instance transfer learning. Next, the performance of classifications by different training strategies (subject-independent- (SI-) CNN, SD-CNN, and ITSD-CNN) are compared. To verify the effectiveness of the algorithm, we evaluate it on the dataset of BCI competition IV-2b. Experiments show that the instance transfer learning can achieve positive instance transfer using a CNN classification model. Among the three different training strategies, the average classification accuracy of ITSD-CNN can achieve 94.7 ± 2.6 and obtain obvious improvement compared with a contrast model (p < 0.01). Compared with other methods proposed in previous research, the framework of ITSD-CNN outperforms the state-of-the-art classification methods with a mean kappa value of 0.664.

Exploring differences between self-report and electrophysiological indices of drowsy driving: A usability examination of a personal brain-computer interface device.

INTRODUCTION: Impaired driving has resulted in numerous accidents, fatalities, and costly damage. One particularly concerning type of impairment is driver drowsiness. Despite advancements, modern vehicle safety systems remain ineffective at keeping drowsy drivers alert and aware of their state, even temporarily. Until recently the use of user-centric brain-computer interface (BCI) devices to capture electrophysiological data relating to driver drowsiness has been limited. METHOD: In this study, 25 participants drove on a simulated roadway under drowsy conditions. RESULTS: Neither subjective nor electrophysiological measures differed between individuals who showed overt signs of drowsiness (prolonged eye closure) during the drive. However, the directionality and effect size estimates provided by the BCI device suggested the practicality and feasibility of its future implementation in vehicle safety systems. Practical applications: This research highlights opportunities for future BCI device research for use to assess the state of drowsy drivers in a real-world context.

Spatial-Frequency Feature Learning and Classification of Motor Imagery EEG Based on Deep Convolution Neural Network.

EEG pattern recognition is an important part of motor imagery- (MI-) based brain computer interface (BCI) system. Traditional EEG pattern recognition algorithm usually includes two steps, namely, feature extraction and feature classification. In feature extraction, common spatial pattern (CSP) is one of the most frequently used algorithms. However, in order to extract the optimal CSP features, prior knowledge and complex parameter adjustment are often required. Convolutional neural network (CNN) is one of the most popular deep learning models at present. Within CNN, feature learning and pattern classification are carried out simultaneously during the procedure of iterative updating of network parameters; thus, it can remove the complicated manual feature engineering. In this paper, we propose a novel deep learning methodology which can be used for spatial-frequency feature learning and classification of motor imagery EEG. Specifically, a multilayer CNN model is designed according to the spatial-frequency characteristics of MI EEG signals. An experimental study is carried out on two MI EEG datasets (BCI competition III dataset IVa and a self-collected right index finger MI dataset) to validate the effectiveness of our algorithm in comparison with several closely related competing methods. Superior classification performance indicates that our proposed method is a promising pattern recognition algorithm for MI-based BCI system.

Research and Verification of Convolutional Neural Network Lightweight in BCI.

With the increasing of depth and complexity of the convolutional neural network, parameter dimensionality and volume of computing have greatly restricted its applications. Based on the SqueezeNet network structure, this study introduces a block convolution and uses channel shuffle between blocks to alleviate the information jam. The method is aimed at reducing the dimensionality of parameters of in an original network structure and improving the efficiency of network operation. The verification performance of the ORL dataset shows that the classification accuracy and convergence efficiency are not reduced or even slightly improved when the network parameters are reduced, which supports the validity of block convolution in structure lightweight. Moreover, using a classic CIFAR-10 dataset, this network decreases parameter dimensionality while accelerating computational processing, with excellent convergence stability and efficiency when the network accuracy is only reduced by 1.3%.

A Parallel Algorithm Framework for Feature Extraction of EEG Signals on MPI.

In this paper, we present a parallel framework based on MPI for a large dataset to extract power spectrum features of EEG signals so as to improve the speed of brain signal processing. At present, the Welch method has been wildly used to estimate the power spectrum. However, the traditional Welch method takes a lot of time especially for the large dataset. In view of this, we added the MPI into the traditional Welch method and developed it into a reusable master-slave parallel framework. As long as the EEG data of any format are converted into the text file of a specified format, the power spectrum features can be extracted quickly by this parallel framework. In the proposed parallel framework, the EEG signals recorded by a channel are divided into N overlapping data segments. Then, the PSD of N segments are computed by some nodes in parallel. The results are collected and summarized by the master node. The final PSD results of each channel are saved in the text file, which can be read and analyzed by Microsoft Excel. This framework can be implemented not only on the clusters but also on the desktop computer. In the experiment, we deploy this framework on a desktop computer with a 4-core Intel CPU. It took only a few minutes to extract the power spectrum features from the 2.85 GB EEG dataset, seven times faster than using Python. This framework makes it easy for users, who do not have any parallel programming experience in constructing the parallel algorithms to extract the EEG power spectrum.

Attention Optimization Method for EEG via the TGAM.

Since the 21st century, noninvasive brain-computer interface (BCI) has developed rapidly, and brain-computer devices have gradually moved from the laboratory to the mass market. Among them, the TGAM (ThinkGear Asic Module) and its encapsulate algorithm have been adopted by many research teams and faculty members around the world. However, due to the limited development cost, the effectiveness of the algorithm to calculate data is not satisfactory. This paper proposes an attention optimization algorithm based on the TGAM for EEG data feedback. Considering that the data output of the TGAM encapsulate algorithm fluctuates greatly, the delay is high and the accuracy is low. The experimental results demonstrated that our algorithm can optimize EEG data, so that with the same or even lower delay and without changing the encapsulate algorithm of the module itself, it can significantly improve the performance of attention data, greatly improve the stability and accuracy of data, and achieve better results in practical applications.

Multi-class motor imagery EEG classification using collaborative representation-based semi-supervised extreme learning machine.

Both labeled and unlabeled data have been widely used in electroencephalographic (EEG)-based brain-computer interface (BCI). However, labeled EEG samples are generally scarce and expensive to collect, while unlabeled samples are considered to be abundant in real applications. Although the semi-supervised learning (SSL) allows us to utilize both labeled and unlabeled data to improve the classification performance as against supervised algorithms, it has been reported that unlabeled data occasionally undermine the performance of SSL in some cases. To overcome this challenge, we propose a collaborative representation-based semi-supervised extreme learning machine (CR-SSELM) algorithm to evaluate the risk of unlabeled samples by a new safety-control mechanism. Specifically, the ELM model is firstly used to predict unlabeled samples and then the collaborative representation (CR) approach is employed to reconstruct the unlabeled samples according to the obtained prediction results, from which the risk degree of unlabeled sample is defined. A risk-based regularization term is then constructed accordingly and embedded into the objective function of the SS-ELM. Experiments conducted on benchmark and EEG datasets demonstrate that the proposed method outperforms the ELM and SS-ELM algorithm. Moreover, the proposed CR-SSELM even offers the best performance while SS-ELM yields worse performance compared with its supervised counterpart (ELM). Graphical abstract This paper proposes a collaborative representation-based semi-supervised extreme learning machine (CR-SSELM) algorithm to evaluate the risk of unlabeled samples by a new safety-control mechanism. It is aim to solve the safety problem of SS-ELM method that SS-ELM yields worse performance than ELM. With the help of safety mechanism, the performance of our method is still better than supervised ELM method.

Detection of Solitary Pulmonary Nodules Based on Brain-Computer Interface.

Solitary pulmonary nodules are the main manifestation of pulmonary lesions. Doctors often make diagnosis by observing the lung CT images. In order to further study the brain response structure and construct a brain-computer interface, we propose an isolated pulmonary nodule detection model based on a brain-computer interface. First, a single channel time-frequency feature extraction model is constructed based on the analysis of EEG data. Second, a multilayer fusion model is proposed to establish the brain-computer interface by connecting the brain electrical signal with a computer. Finally, according to image presentation, a three-frame image presentation method with different window widths and window positions is proposed to effectively detect the solitary pulmonary nodules.

The EEG-Based Attention Analysis in Multimedia m-Learning.

In recent years, research on brain-computer interfaces has been increasing in the field of education, and mobile learning has become a very important way of learning. In this study, EEG experiment of a group of iPad-based mobile learners was conducted through algorithm optimization on the TGAM chip. Under the three learning media (text, text + graphic, and video), the researchers analyzed the difference in learners' attention. The study found no significant difference in attention in different media, but learners using text media had the highest attention value. Later, the researchers studied the attention of learners with different learning styles and found that active and reflective learners' attention exhibited significant differences when using video media to learn.

An Informative and Comprehensive Behavioral Characteristics Analysis Methodology of Android Application for Data Security in Brain-Machine Interfacing.

Recently, brain-machine interfacing is very popular that link humans and artificial devices through brain signals which lead to corresponding mobile application as supplementary. The Android platform has developed rapidly because of its good user experience and openness. Meanwhile, these characteristics of this platform, which cause the amazing pace of Android malware, pose a great threat to this platform and data correction during signal transmission of brain-machine interfacing. Many previous works employ various behavioral characteristics to analyze Android application (or app) and detect Android malware to protect signal data secure. However, with the development of Android app, category of Android app tends to be diverse, and the Android malware behavior tends to be complex. This situation makes existing Android malware detections complicated and inefficient. In this paper, we propose a broad analysis, gathering as many behavior characteristics of an app as possible and compare these behavior characteristics in several metrics. First, we extract static and dynamic behavioral characteristic from Android app in an automatic manner. Second, we explain the decision we made in each kind of behavioral characteristic we choose for Android app analysis and Android malware detection. Third, we design a detailed experiment, which compare the efficiency of each kind of behavior characteristic in different aspects. The results of experiment also show Android malware detection performance of these behavior characteristics combine with well-known machine learning algorithms.

Influential Factors of an Asynchronous BCI for Movement Intention Detection.

In recent years, asynchronous brain computer interface (BCI) systems have been utilized in many domains such as robot controlling, assistive technology, and rehabilitation. In such BCI systems, movement intention detection algorithms are used to detect movement desires. In recent years, movement-related cortical potential (MRCP), an electroencephalogram (EEG) pattern representing voluntary movement intention, attracts wide attention in movement intention detection. Unfortunately, low MRCP detection accuracy makes the asynchronous BCI system impractical for real usage. In order to develop an effective MRCP detection algorithm, EEG data have to be properly preprocessed. In this work, we investigate the relationship and effects of three factors including frequency bands, spatial filters, and classifiers on MRCP classification performance to determine best settings. In particular, we performed a systematic performance investigation on combinations of five frequency bands, five spatial filters, and six classifiers. The EEG data were acquired from subjects performing series of self-paced ankle dorsiflexions. Analysis of variance (ANOVA) statistical test was performed on F1 scores to investigate effects of these three factors. The results show that frequency bands and spatial filters depend on each other. The combinations directly affect the F1 scores, so they have to be chosen carefully. The results can be used as guidelines for BCI researchers to effectively design a preprocessing method for an advanced asynchronous BCI system, which can assist the stroke rehabilitation.

Calibration time reduction through local activities estimation in motor imagery-based brain-computer interfaces.

OBJECTIVE: One of the main limitations for the practical use of brain-computer interfaces (BCI) is the calibration phase. Several methods have been suggested for the truncating of this undesirable time, including various variants of the popular CSP method. In this study, we cope with the problem, using local activities estimation (LAE). APPROACH: LAE is a spatial filtering technique that uses the EEG data of all electrodes along with their position information to emphasize the local activities. After spatial filtering by LAE, a few electrodes are selected based on physiological information. Then the features are extracted from the signal using FFT and classified by the support vector machine. In this study, the LAE is compared with CSP, RCSP, FBCSP and FBRCSP in two different electrode configurations of 118 and 64-channel. MAIN RESULTS: The LAE outperforms CSP-based methods in all experiments using the different number of training samples. The LAE method also obtains an average classification accuracy of 84% even with a calibration time of fewer than 2 min Significance: Unlike CSP-based methods, the LAE does not use the covariance matrix, and also uses a priori physiological information. Therefore, LAE can significantly reduce the calibration time while maintaining proper accuracy. It works well even with a few training samples.

Hemodynamic responses during standing and sitting activities: a study toward fNIRS-BCI.

In this paper, we utilized functional near-infrared spectroscopy (fNIRS) technology to examine the hemodynamic responses in the motor cortex for two conditions, namely standing and sitting tasks. Nine subjects performed five trials of standing and sitting (SAS) tasks with both real movements and imagery thinking of SAS. A group level of statistical parametric mapping (SPM) analysis during these tasks showed bilateral activation of oxy-hemoglobin for both real movements and imagery experiments. Interestingly, the SPM analysis clearly revealed that the sitting tasks induced a higher oxy-hemoglobin level activation compared to the standing task. Remarkably, this finding is persistent across the 22 measured channels at the individual and group levels for both experiments. Furthermore, six features were extracted from pre-processed HbO signals and the performance of four different classifiers was examined in order to test the viability of using SAS tasks in future fNIRS-brain-computer interface (fNIRS-BCI) systems. In particular, two features-combination tests revealed that the signal slope with signal variance represents one of the three best two-combined features for its consistency in providing high accuracy results for both real and imagery experiments. This study shows the potential of implementing such tasks into the fNIRS-BCI system. In the future, the results of this work could pave the way towards the application of fNIRS-BCI in lower limb rehabilitation.

The dynamics of motor learning through the formation of internal models.

A medical student learning to perform a laparoscopic procedure or a recently paralyzed user of a powered wheelchair must learn to operate machinery via interfaces that translate their actions into commands for an external device. Since the user's actions are selected from a number of alternatives that would result in the same effect in the control space of the external device, learning to use such interfaces involves dealing with redundancy. Subjects need to learn an externally chosen many-to-one map that transforms their actions into device commands. Mathematically, we describe this type of learning as a deterministic dynamical process, whose state is the evolving forward and inverse internal models of the interface. The forward model predicts the outcomes of actions, while the inverse model generates actions designed to attain desired outcomes. Both the mathematical analysis of the proposed model of learning dynamics and the learning performance observed in a group of subjects demonstrate a first-order exponential convergence of the learning process toward a particular state that depends only on the initial state of the inverse and forward models and on the sequence of targets supplied to the users. Noise is not only present but necessary for the convergence of learning through the minimization of the difference between actual and predicted outcomes.

Computationally efficient fully-automatic online neural spike detection and sorting in presence of multi-unit activity for implantable circuits.

BACKGROUND: Spike sorting is a basic step for implantable neural interfaces. With the growing number of channels, the process should be computationally efficient, automatic,robust and applicable on implantable circuits. NEW METHOD: The proposed method is a combination of fully-automatic offline and online processes. It introduces a novel method for automatically determining a data-aware spike detection threshold, computationally efficient spike feature extraction, automatic optimal cluster number evaluation and verification coupled with Self-Organizing Maps to accurately determine cluster centroids. The system has the ability of unsupervised online operation after initial fully-automatic offline training. The prime focus of this paper is to fully-automate the complete spike detection and sorting pipeline, while keeping the accuracy high. RESULTS: The proposed system is simulated on two well-known datasets. The automatic threshold improves detection accuracies significantly( > 15%) as compared to the most common detector. The system is able to effectively handle background multi-unit activity with improved performance. COMPARISON: Most of the existing methods are not fully-automatic; they require supervision and expert intervention at various stages of the pipeline. Secondly, existing works focus on foreground neural activity. Recent research has highlighted importance of background multi-unit activity, and this work is amongst the first efforts that proposes and verifies an automatic methodology to effectively handle them as well. CONCLUSION: This paper proposes a fully-automatic, computationally efficient system for spike sorting for both single-unit and multi-unit spikes. Although the scope of this work is design and verification through computer simulations, the system has been designed to be easily transferable into an integrated hardware form.

Single-Trial NIRS Data Classification for Brain-Computer Interfaces Using Graph Signal Processing.

Near-infrared spectroscopy (NIRS)-based brain-computer interface (BCI) systems use feature extraction methods relying mainly on the slope characteristics and mean changes of the hemodynamic responses in respect to certain mental tasks. Nevertheless, spatial patterns across the measurement channels have been detected and should be considered during the feature vector extraction stage of the BCI realization. In this paper, a graph signal processing (GSP) approach for feature extraction is adopted in order to capture the aforementioned spatial information of the NIRS signals. The proposed GSP-based methodology for feature extraction in NIRS-based BCI systems, namely graph NIRS (GNIRS), is applied on a publicly available dataset of NIRS recordings during a mental arithmetic task. GNIRS exhibits higher classification rates (CRs), up to 92.52%, as compared to the CRs of two state-of-the-art feature extraction methodologies related to slope and mean values of hemodynamic response, i.e., 90.35% and 82.60%, respectively. In addition, GNIRS leads to the formation of feature vectors with reduced dimensionality in comparison with the baseline approaches. Moreover, it is shown to facilitate high CRs even from the first second after the onset of the mental task, paving the way for faster NIRS-based BCI systems.

Stimuli and Feature Extraction Algorithms for Brain-Computer Interfaces: A Systematic Comparison.

A brain-computer interface (BCI) is a system that allows communication between the central nervous system and an external device. The BCIs developed by various research groups differ in their main features and the comparison across studies is therefore challenging. Here, in the same group of 19 healthy participants, we investigate three different tasks (SSVEP, P300, and hybrid) that allowed four choices to the user without previous neurofeedback training. We used the same 64-channel EEG equipment to acquire data, while participants performed each of the tasks. We systematically compared the participants' offline performance on the following parameters: 1) accuracy; 2) BCI Utility (in bits/min); and 3) inefficiency/illiteracy. In addition, we evaluated the accuracy as a function of the number of electrodes. In this paper, the SSVEP task outperformed the other tasks in bit rate, reaching an average and maximum BCI Utility of 63.4 and 91.3 bits/min, respectively. All participants achieved an accuracy level above70% on both SSVEP and P300 tasks. Furthermore, the average accuracy of all tasks was highest if a reduced subset with 4-12 electrodes was used. These results are relevant for the development of online BCIs intended for the real-life applications.