Music emotion recognition based on temporal convolutional attention network using EEG.

Music is one of the primary ways to evoke human emotions. However, the feeling of music is subjective, making it difficult to determine which emotions music triggers in a given individual. In order to correctly identify emotional problems caused by different types of music, we first created an electroencephalogram (EEG) data set stimulated by four different types of music (fear, happiness, calm, and sadness). Secondly, the differential entropy features of EEG were extracted, and then the emotion recognition model CNN-SA-BiLSTM was established to extract the temporal features of EEG, and the recognition performance of the model was improved by using the global perception ability of the self-attention mechanism. The effectiveness of the model was further verified by the ablation experiment. The classification accuracy of this method in the valence and arousal dimensions is 93.45% and 96.36%, respectively. By applying our method to a publicly available EEG dataset DEAP, we evaluated the generalization and reliability of our method. In addition, we further investigate the effects of different EEG bands and multi-band combinations on music emotion recognition, and the results confirm relevant neuroscience studies. Compared with other representative music emotion recognition works, this method has better classification performance, and provides a promising framework for the future research of emotion recognition system based on brain computer interface.

Affection of facial artifacts caused by micro-expressions on electroencephalography signals.

Macro-expressions are widely used in emotion recognition based on electroencephalography (EEG) because of their use as an intuitive external expression. Similarly, micro-expressions, as suppressed and brief emotional expressions, can also reflect a person's genuine emotional state. Therefore, researchers have started to focus on emotion recognition studies based on micro-expressions and EEG. However, compared to the effect of artifacts generated by macro-expressions on the EEG signal, it is not clear how artifacts generated by micro-expressions affect EEG signals. In this study, we investigated the effects of facial muscle activity caused by micro-expressions in positive emotions on EEG signals. We recorded the participants' facial expression images and EEG signals while they watched positive emotion-inducing videos. We then divided the 13 facial regions and extracted the main directional mean optical flow features as facial micro-expression image features, and the power spectral densities of theta, alpha, beta, and gamma frequency bands as EEG features. Multiple linear regression and Granger causality test analyses were used to determine the extent of the effect of facial muscle activity artifacts on EEG signals. The results showed that the average percentage of EEG signals affected by muscle artifacts caused by micro-expressions was 11.5%, with the frontal and temporal regions being significantly affected. After removing the artifacts from the EEG signal, the average percentage of the affected EEG signal dropped to 3.7%. To the best of our knowledge, this is the first study to investigate the affection of facial artifacts caused by micro-expressions on EEG signals.

Brain Activation Differences of Six Basic Emotions Between 2D Screen and Virtual Reality Modalities.

To our knowledge, it has been widely studied in Screen-2D modality for the six basic emotions proposed by Professor Paul Ekman, but there are only studies on their positive and negative valence in VR-3D modality. In this study, we will investigate whether the six basic emotions have stronger brain activation states in VR-3D modality than in Screen-2D modality. We designed an emotion-inducing experiment with six basic emotions (happiness, surprise, sadness, fear, anger, and disgust) to record the electroencephalogram (EEG) signals during watching VR-3D and Screen-2D videos. The power spectral density (PSD) was calculated to compare the brain activation differences between VR-3D and Screen-2D modalities during the induction of the six basic emotions. The results of statistical analysis of the relative power differences between VR-3D and Screen-2D modalities for each emotion revealed that both happiness and surprise presented greater differences in the α and γ frequency bands, while sad, fear, disgust and anger all presented greater differences in the α and θ frequency bands, which are mainly observed in the frontal and occipital regions. On the other hand, the six emotions all yielded satisfactory classification accuracy (above 85%) by classification from a subset of power feature of the brain activation states in the same emotion between the two modalities. Overall, there are significant differences in the induction of same discrete emotions in VR-3D and Screen-2D modalities, with greater brain activation in VR-3D modalities. These findings provide a better understanding about the neural activity of discrete emotional tasks assessed in VR environments.