Mixed-reality-based human-animal interaction can relieve mental stress.

Introduction: Interacting with animals has been demonstrated to possess the healing benefits to humans. However, there are limitations in physical interaction due to COVID-19 and safety issues. Therefore, as an alternative, we created mixed-reality (MR)-based human-animal interaction (HAI) content and experimentally verified its effect on mental stress reduction. Methods: We created three types of interactive content: observing the movement of a non-reactive virtual cat, interacting with a virtual cat whose responses can be seen, and interacting with a virtual cat whose responses can be both seen and heard. The experiment was performed by 30 healthy young women, and a mental arithmetic task was used to induce mild mental stress before experiencing each content. During the experiment, the subject's electrocardiogram was continuously recorded, and the psychological state was evaluated through a questionnaire. Results: The results showed that MR-based virtual cat content significantly reduces mental stress and induces positive emotions after stressful situations. In particular, when the virtual cat provided audiovisual feedback, the activation amount of the parasympathetic nervous system and the increase of positive emotions were the greatest. Discussion: Based on this encouraging research result, this method should be further investigated to see if it can replace real HAI for human mental health management.

Deep learning-based self-induced emotion recognition using EEG.

Emotion recognition from electroencephalogram (EEG) signals requires accurate and efficient signal processing and feature extraction. Deep learning technology has enabled the automatic extraction of raw EEG signal features that contribute to classifying emotions more accurately. Despite such advances, classification of emotions from EEG signals, especially recorded during recalling specific memories or imagining emotional situations has not yet been investigated. In addition, high-density EEG signal classification using deep neural networks faces challenges, such as high computational complexity, redundant channels, and low accuracy. To address these problems, we evaluate the effects of using a simple channel selection method for classifying self-induced emotions based on deep learning. The experiments demonstrate that selecting key channels based on signal statistics can reduce the computational complexity by 89% without decreasing the classification accuracy. The channel selection method with the highest accuracy was the kurtosis-based method, which achieved accuracies of 79.03% and 79.36% for the valence and arousal scales, respectively. The experimental results show that the proposed framework outperforms conventional methods, even though it uses fewer channels. Our proposed method can be beneficial for the effective use of EEG signals in practical applications.

Predictions of tDCS treatment response in PTSD patients using EEG based classification.

Transcranial direct current stimulation (tDCS) is an emerging therapeutic tool for treating posttraumatic stress disorder (PTSD). Prior studies have shown that tDCS responses are highly individualized, thus necessitating the individualized optimization of treatment configurations. To date, an effective tool for predicting tDCS treatment outcomes in patients with PTSD has not yet been proposed. Therefore, we aimed to build and validate a tool for predicting tDCS treatment outcomes in patients with PTSD. Forty-eight patients with PTSD received 20 min of 2 mA tDCS stimulation in position of the anode over the F3 and cathode over the F4 region. Non-responders were defined as those with less than 50% improvement after reviewing clinical symptoms based on the Clinician-Administered DSM-5 PTSD Scale (before and after stimulation). Resting-state electroencephalograms were recorded for 3 min before and after stimulation. We extracted power spectral densities (PSDs) for five frequency bands. A support vector machine (SVM) model was used to predict responders and non-responders using PSDs obtained before stimulation. We investigated statistical differences in PSDs before and after stimulation and found statistically significant differences in the F8 channel in the theta band (p = 0.01). The SVM model had an area under the ROC curve (AUC) of 0.93 for predicting responders and non-responders using PSDs. To our knowledge, this study provides the first empirical evidence that PSDs can be useful biomarkers for predicting the tDCS treatment response, and that a machine learning model can provide robust prediction performance. Machine learning models based on PSDs can be useful for informing treatment decisions in tDCS treatment for patients with PTSD.

Mixed Reality-Based Interaction between Human and Virtual Cat for Mental Stress Management.

Human-animal interaction (HAI) has been observed to effectively reduce stress and induce positive emotions owing to the process of directly petting and interacting with animals. Interaction with virtual animals has recently emerged as an alternative due to the limitations in general physical interactions, both due to the COVID-19 pandemic and, more generally, due to the difficulties involved in providing adequate care for animals. This study proposes mixed reality (MR)-based human-animal interaction content along with presenting the experimental verification of its effect on the reduction of mental stress. A mental arithmetic task was employed to induce acute mental stress, which was followed by either MR content, in which a participant interacted with virtual animals via gestures and voice commands, or a slide show of animal images. During the experiment, an electrocardiogram (ECG) was continuously recorded with a patch-type, wireless ECG sensor on the chest of the subject, and their psychological state was evaluated with the help of questionnaires after each task. The findings of the study demonstrate that the MR-based interaction with virtual animals significantly reduces mental stress and induces positive emotions. We expect that this study could provide a basis for the widespread use of MR-based content in the field of mental health.

Prefrontal Functional Connectivity During the Verbal Fluency Task in Patients With Major Depressive Disorder: A Functional Near-Infrared Spectroscopy Study.

Deviations in activation patterns and functional connectivity have been observed in patients with major depressive disorder (MDD) with prefrontal hemodynamics of patients compared with healthy individuals. The graph-theoretical approach provides useful network metrics for evaluating functional connectivity. The evaluation of functional connectivity during a cognitive task can be used to explain the neurocognitive mechanism underlying the cognitive impairments caused by depression. Overall, 31 patients with MDD and 43 healthy individuals completed a verbal fluency task (VFT) while wearing a head-mounted functional near-infrared spectroscopy (fNIRS) devices. Hemodynamics and functional connectivity across eight prefrontal subregions in the two groups were analyzed and compared. We observed a reduction in prefrontal activation and weaker overall and interhemispheric subregion-wise correlations in the patient group compared with corresponding values in the control group. Moreover, efficiency, the network measure related to the effectiveness of information transfer, showed a significant between-group difference [t (71.64) = 3.66, corrected p < 0.001] along with a strong negative correlation with depression severity (rho = -0.30, p = 0.009). The patterns of prefrontal functional connectivity differed significantly between the patient and control groups during the VFT. Network measures can quantitatively characterize the reduction in functional connectivity caused by depression. The efficiency of the functional network may play an important role in the understanding of depressive symptoms.

Effect of Daily Stress on Heart-Rate Variability during Stroop Color Word Task.

It is important to care and manage daily stress, even for young healthy individuals. Conventional approach to investigate the effect of mental stress on our body usually utilizes a cognitive task as an induced stressor and compares the response with a resting state. During the process, the effects of daily stress are ignored. We hypothesized that the response to mental stress may differ depending on the daily stress level. As far as our knowledge, the effect of daily stress on cognitive ability and electrophysiological response has not yet been widely investigated. We designed and conducted an experiment to record ECGs while performing a Stroop color word task for 42 young healthy female subjects. We repeated the experiment for two different days with different levels of stress measured by self-report. As a result, we could find significant differences on behavior and heart-rate variability depending on the stress level. These findings may provide a new insight on how to understand the trend of heart-rate variability according to subject's daily stress.

Deep Cerebellar Low-Intensity Focused Ultrasound Stimulation Restores Interhemispheric Balance after Ischemic Stroke in Mice.

Ischemic damage after stroke disrupts the complex balance of inhibitory and excitatory activity within cortical network causing brain functional asymmetry. Cerebellar deep nuclei with its extensive projections to cortical regions could be a prospective target for stimulation to restore inter-hemispheric balance and enhance neural plasticity after stroke. In our study, we repeatedly stimulated the lateral cerebellar nucleus (LCN) by low-intensity focused ultrasound (LIFU) for 3 days to enhance rehabilitation after middle cerebral artery occlusion (MCAO) in a mouse stroke model. The neural activity of the mice sensorimotor cortex was measured using epidural electrodes and analyzed with quantified electroencephalography (qEEG). Pairwise derived Brain Symmetry Index (pdBSI) and delta power were used to assess the neurorehabilitative effect of LIFU stimulation. Compared to the Stroke (non-treated) group, the LIFU group exhibited a decrease in cortical pathological delta activity, significant recovery in pdBSI and enhanced performance on the balance beam walking test. These results suggest that cerebellar LIFU stimulation could be a non-invasive method for stroke rehabilitation through the restoration of interhemispheric balance.

Deep Joint Spatiotemporal Network (DJSTN) for Efficient Facial Expression Recognition.

Understanding a person's feelings is a very important process for the affective computing. People express their emotions in various ways. Among them, facial expression is the most effective way to present human emotional status. We propose efficient deep joint spatiotemporal features for facial expression recognition based on the deep appearance and geometric neural networks. We apply three-dimensional (3D) convolution to extract spatial and temporal features at the same time. For the geometric network, 23 dominant facial landmarks are selected to express the movement of facial muscle through the analysis of energy distribution of whole facial landmarks.We combine these features by the designed joint fusion classifier to complement each other. From the experimental results, we verify the recognition accuracy of 99.21%, 87.88%, and 91.83% for CK+, MMI, and FERA datasets, respectively. Through the comparative analysis, we show that the proposed scheme is able to improve the recognition accuracy by 4% at least.

Ambulatory and Laboratory Stress Detection Based on Raw Electrocardiogram Signals Using a Convolutional Neural Network.

The goals of this study are the suggestion of a better classification method for detecting stressed states based on raw electrocardiogram (ECG) data and a method for training a deep neural network (DNN) with a smaller data set. We suggest an end-to-end architecture to detect stress using raw ECGs. The architecture consists of successive stages that contain convolutional layers. In this study, two kinds of data sets are used to train and validate the model: A driving data set and a mental arithmetic data set, which smaller than the driving data set. We apply a transfer learning method to train a model with a small data set. The proposed model shows better performance, based on receiver operating curves, than conventional methods. Compared with other DNN methods using raw ECGs, the proposed model improves the accuracy from 87.39% to 90.19%. The transfer learning method improves accuracy by 12.01% and 10.06% when 10 s and 60 s of ECG signals, respectively, are used in the model. In conclusion, our model outperforms previous models using raw ECGs from a small data set and, so, we believe that our model can significantly contribute to mobile healthcare for stress management in daily life.

Neural Correlates of Variations in Human Trust in Human-like Machines during Non-reciprocal Interactions.

As intelligent machines have become widespread in various applications, it has become increasingly important to operate them efficiently. Monitoring human operators' trust is required for productive interactions between humans and machines. However, neurocognitive understanding of human trust in machines is limited. In this study, we analysed human behaviours and electroencephalograms (EEGs) obtained during non-reciprocal human-machine interactions. Human subjects supervised their partner agents by monitoring and intervening in the agents' actions in this non-reciprocal interaction, which reflected practical uses of autonomous or smart systems. Furthermore, we diversified the agents with external and internal human-like factors to understand the influence of anthropomorphism of machine agents. Agents' internal human-likenesses were manifested in the way they conducted a task and affected subjects' trust levels. From EEG analysis, we could define brain responses correlated with increase and decrease of trust. The effects of trust variations on brain responses were more pronounced with agents who were externally closer to humans and who elicited greater trust from the subjects. This research provides a theoretical basis for modelling human neural activities indicate trust in partner machines and can thereby contribute to the design of machines to promote efficient interactions with humans.

Stress Resilience Measurement With Heart-Rate Variability During Mental And Physical Stress.

Stress management is particularly important for healthcare of modern people. In stress research, heart-rate variability (HRV), indicating the change of time intervals in successive heart beats, significantly contributed due to its close relationship with autonomic nervous system. However, the adaptive response to stress, also known as stress resilience, has not been studied much yet. We collected electrocardiogram during mental and physical stress, experimentally designed by mental arithmetic tasks and physical activities for 14 healthy subjects. As a result, we found that resting HRV parameters, particularly associated with the parasympathetic activity, had significant positive correlations with reactivity and recovery from mental and physical stress. These HRV parameters can be used as a measure of stress resilience quantitatively. Our findings suggest that these parameters can help one's stress management by enabling to predict the adaptive response to upcoming stressful events.

The Role of Heart-Rate Variability Parameters in Activity Recognition and Energy-Expenditure Estimation Using Wearable Sensors.

Human-activity recognition (HAR) and energy-expenditure (EE) estimation are major functions in the mobile healthcare system. Both functions have been investigated for a long time; however, several challenges remain unsolved, such as the confusion between activities and the recognition of energy-consuming activities involving little or no movement. To solve these problems, we propose a novel approach using an accelerometer and electrocardiogram (ECG). First, we collected a database of six activities (sitting, standing, walking, ascending, resting and running) of 13 voluntary participants. We compared the HAR performances of three models with respect to the input data type (with none, all, or some of the heart-rate variability (HRV) parameters). The best recognition performance was 96.35%, which was obtained with some selected HRV parameters. EE was also estimated for different choices of the input data type (with or without HRV parameters) and the model type (single and activity-specific). The best estimation performance was found in the case of the activity-specific model with HRV parameters. Our findings indicate that the use of human physiological data, obtained by wearable sensors, has a significant impact on both HAR and EE estimation, which are crucial functions in the mobile healthcare system.

EEG-Based Classification of Implicit Intention During Self-Relevant Sentence Reading.

From electroencephalography (EEG) data during self-relevant sentence reading, we were able to discriminate two implicit intentions: 1) "agreement" and 2) "disagreement" to the read sentence. To improve the classification accuracy, discriminant features were selected based on Fisher score among EEG frequency bands and electrodes. Especially, the time-frequency representation with Morlet wavelet transforms showed clear differences in gamma, beta, and alpha band powers at frontocentral area, and theta band power at centroparietal area. The best classification accuracy of 75.5% was obtained by a support vector machine classifier with the gamma band features at frontocentral area. This result may enable a new intelligent user-interface which understands users' implicit intention, i.e., unexpressed or hidden intention.

Implicit agreeing/disagreeing intention while reading self-relevant sentences: A human fMRI study.

The true intentions of humans are sometimes difficult to ascertain exclusively from explicit expressions, such as speech, gestures, or facial expressions. In this experiment, functional magnetic resonance imaging (fMRI) was used to investigate implicit intentions that were generated while a subject was reading self-relevant sentences. Short sentences, which were presented visually, consisted of self-relevant statements and a substantive verb, which indicated sentence polarity as either affirmative or negative. Each sentence was divided into the contents and the sentence ending, and the subjects were asked to respond with either agreement or disagreement after the complete sentence was presented. The overall group analysis suggested that the intention of the sentence response was found even before the reading of the complete sentences. Increased neural activation was found in the left medial prefrontal cortex (MPFC) during feelings of agreement compared to feelings of disagreement during self-relevant decision-making. In addition, according to the sentence ending, the decision of a response activated the frontopolar cortex (FPC) in the switching condition. These findings indicated that the implicit intentions of responses to the given statements were internally generated before an explicit response occurred, and, hence, intentions can be used to predict a subject's future answer.