Visual stimuli in the peripersonal space facilitate the spatial prediction of tactile events-A comparison between approach and nearness effects.

Previous studies reported that an object in one's peripersonal space (PPS) attracts attention and facilitates subsequent processing of stimuli. Recent studies showed that visual stimuli approaching the body facilitated the spatial prediction of subsequent tactile events, even if these stimuli were task-irrelevant. However, it is unclear whether the approach is important for facilitating this prediction or if the simple existence of stimuli within the PPS is what matters. The present study aimed to scrutinize the predictive function of visuo-tactile interaction in the PPS by examining the effects of visual stimuli approaching the hand and of visual stimuli near the hand. For this purpose, we examined electroencephalograms (EEGs) during a simple reaction time task for tactile stimuli when visual stimuli were presented approaching the hand or were presented near the hand, and we analyzed event-related spectral perturbation (ERSP) as an index of prediction and event-related brain potentials (ERPs) as an index of attention and prediction error. The tactile stimulus was presented to the left (or right) wrist with a high probability (80%) and to the opposite wrist with a low probability (20%). In the approach condition, three visual stimuli were presented approaching the hand to which the high-probability tactile stimulus was presented; in the near condition, three visual stimuli were presented repeatedly near the hand with the high-probability tactile stimulus. Beta-band activity at the C3 and C4 electrodes, around the primary somatosensory area, was suppressed before the onset of the tactile stimulus, and this suppression was larger in the approach condition than in the near condition. The P3 amplitude for high-probability stimuli in the approach condition was larger than that in the near condition. These results revealed that the approach of visual stimuli facilitates spatial prediction and processing of subsequent tactile stimuli compared to situations in which visual stimuli just exist within the PPS. This study indicated that approaching visual stimuli facilitates the prediction of subsequent tactile events, even if they are task-irrelevant.

The influence of peripheral information on a proactive process during multitasking.

The aim of this study was to examine whether peripheral information facilitates proactive processes during multitasking. For this purpose, peripheral information was presented regularly during multitasking and its effects on the performance of a tracking task (main task: reactive process) and a discrimination task (sub-task: proactive process) were examined. Experiment 1 presented peripheral information (white circles) in the same sensory modality (visual) as the information used for multitasking and the number of circle presentations was manipulated. In Experiment 2, a pure tone (auditory) was presented as peripheral information. We found that, in both experiments, the difficulty of the tracking task influenced discrimination performance, showing that as the difficulty of the tracking task (reactive process) increased, more cognitive resources were consumed in the tracking task, resulting in a decrease in cognitive resources available for the discrimination task (proactive process). In addition, regular presentation of peripheral information facilitated discrimination task performance in both experiments. Interestingly, this peripheral information also facilitated the tracking task performance (reactive process) even if the tracking task was difficult. Moreover, this promoting effect of the peripheral information occurred regardless of the sensory modality. This study revealed that processing of peripheral information facilitates the proactive process even if more cognitive resources are consumed, and that this facilitating effect does not conflict with multitasking and provides a margin of cognitive resources and also facilitates the reactive process. Our results provide evidence of how peripheral information and cognitive resources are used during multitasking.

Comparison of the effects of indoor and outdoor exercise on creativity: an analysis of EEG alpha power.

Previous research finds that natural environments and exercise enhance creativity. In this within-subjects design study, we examined the influence of outdoor exercise that combined a natural environment with exercise on creativity compared to an indoor exercise control condition by analyzing cognitive activities related to creativity. The participants performed an Alternative Uses Test (AUT), in which ordinary objects are presented to the participants (e.g., a brick), to prompt as many ideas for alternative uses as possible, which are transformed into a creativity score, after indoor running and outdoor running. During the test, brain activity was recorded using electroencephalography (EEG) and a short version flow state scale (FSS) was completed after the experiment. Results showed that while AUT scores did not significantly differ between conditions, alpha band activity at the parietal occipital region involved in divergent creativity increased during the AUT after outdoor exercise while it did not during the AUT after indoor exercise. In addition, FSS scores for positive emotional experience and absorption were higher after outdoor exercise than after indoor exercise. Our results from the FSS suggest that exercise in a natural environment is perceived subjectively differently from indoor exercise, participants report greater experiences of flow compared to indoor exercise, and the EEG measures objectively indicate enhanced cognitive activity in a creativity task after outdoor exercise. This study suggests that outdoor exercise increases neuronal activity in brain regions related to creativity. Further research is needed to understand how this can lead to increased creativity.

Efficient use of peripheral information for temporal prediction.

We adapt to the environment by predicting subsequent events. Generally, intervals between predictions and events make it difficult to predict the events accurately. Previous studies reported that using peripheral information is useful for maintaining predictions of subsequent events; however, it remains unclear how this information maintains the accuracy of the prediction. I presented peripheral visual stimuli in a discrimination task and manipulated the number of times these stimuli were presented while participants were waiting for a task-relevant visual stimulus, and compared participants' response times and event-related brain potentials in Experiment 1. In addition, the influence of the difficulty of predicting the task-relevant visual stimulus was examined in Experiment 2. In both experiments, contingent negative variation (CNV) amplitude immediately before the task-relevant visual stimulus appeared was larger under the condition where many peripheral visual stimuli were presented, and the response time was shorter under this condition. In addition, the largest CNV amplitude under this condition was elicited by the third peripheral visual stimulus, followed in order by the first and second peripheral visual stimuli. These results show that we can predict the timing of events that occur with a delay after the prediction by using peripheral information. Moreover, this peripheral information is processed according to the importance of predicting a task-relevant stimulus, and attentional resources are allocated efficiently. These results provide evidence of the predictive function for temporal prediction of using peripheral information and the allocation of cognitive resources.

Multi-Kernel Temporal and Spatial Convolution for EEG-Based Emotion Classification.

Deep learning using an end-to-end convolutional neural network (ConvNet) has been applied to several electroencephalography (EEG)-based brain-computer interface tasks to extract feature maps and classify the target output. However, the EEG analysis remains challenging since it requires consideration of various architectural design components that influence the representational ability of extracted features. This study proposes an EEG-based emotion classification model called the multi-kernel temporal and spatial convolution network (MultiT-S ConvNet). The multi-scale kernel is used in the model to learn various time resolutions, and separable convolutions are applied to find related spatial patterns. In addition, we enhanced both the temporal and spatial filters with a lightweight gating mechanism. To validate the performance and classification accuracy of MultiT-S ConvNet, we conduct subject-dependent and subject-independent experiments on EEG-based emotion datasets: DEAP and SEED. Compared with existing methods, MultiT-S ConvNet outperforms with higher accuracy results and a few trainable parameters. Moreover, the proposed multi-scale module in temporal filtering enables extracting a wide range of EEG representations, covering short- to long-wavelength components. This module could be further implemented in any model of EEG-based convolution networks, and its ability potentially improves the model's learning capacity.

Effects of the presence of a cell phone and exposure to natural environments on remote associates task performance.

In today's advanced information society, creativity in work is highly valued, and there is growing interest in the kinds of work environments that produce more creative outcomes. Recent researchers have demonstrated that when environmental factors change a worker's attentional state to a diffused state, the worker has access to more information than usual, which can contribute to creativity. Here, we examined whether manipulating environmental factors (the presence of a cell phone and exposure to natural environment) that could affect such attention states would improve performance on the Remote Associates Task, a measure of creativity. Our results showed that the presence of a cell phone increased creative performance regardless of immersion in natural environment. In contrast, exposure to nature did not facilitate creative performance; instead, feelings of pleasure increased, and frustration decreased. These results suggest that the presence of a cell phone can enhance creativity by influencing workers' attentional states. The current study provides a meaningful approach to enhancing creativity by modulating attentional states through environmental factors. It also highlights the essential features of environmental factors that can moderate creative abilities.

Perceptual and cognitive processes in augmented reality - comparison between binocular and monocular presentations.

In the present study, we investigated the difference between monocular augmented reality (AR) and binocular AR in terms of perception and cognition by using a task that combines the flanker task with the oddball task. A right- or left-facing arrowhead was presented as a central stimulus at the central vision, and participants were instructed to press a key only when the direction in which the arrowhead faced was a target. In a small number of trials, arrowheads that were facing in the same or opposite direction (flanker stimuli) were presented beside the central stimulus binocularly or monocularly as an AR image. In the binocular condition, the flanker stimuli were presented to both eyes, and, in the monocular condition, only to the dominant eye. The results revealed that participants could respond faster in the binocular condition than in the monocular one; however, only when the flanker stimuli were in the opposite direction was the response faster in the monocular condition. Moreover, the results of event-related brain potentials (ERPs) showed that all stimuli were processed in both the monocular and the binocular conditions in the perceptual stage; however, the influence of the flanker stimuli was attenuated in the monocular condition in the cognitive stage. The influence of flanker stimuli might be more unstable in the monocular condition than in the binocular condition, but more precise examination should be conducted in a future study.

Brief and Indirect Exposure to Natural Environment Restores the Directed Attention for the Task.

The mental fatigue elicited by working and studying consumed mental resources, thereby eliciting a declined performance and an increased mental stress. The long-term continuous work and study, which is typical for modern workers and students, can increase mental fatigue and health risks. Previous studies reported that the natural environment (i.e., forest and waterside) has a restorative of mental resources (i.e., attention) and reducing stress. However, it is difficult for urban workers and students to take sufficient breaks in real natural environment. We conducted an experiment to examine whether brief and indirect exposure to the natural environment elicits a restorative of attention and reducing stress. Twenty-five undergraduate and graduate students from the university of modern city participated in the experiment. The experiment involved measuring the changes in the task performance of the participants (i.e., sustained attention to response task) and the subjective mental workload (i.e., arousal, valence, and NASA-TLX), while the attention restoration was indexed from physiological response (i.e., skin conductance level, SCL) over time. The participants had two types of resting periods in the middle of the task, i.e., by looking at a blank display (simple break) or by watching a nature video having scenes of, e.g., a forest, small waterfall, and rustling leaves (nature break). The results indicate that the natural environment indirectly depicted through the nature videos does not affect the task performance and the subjective mental workload but decreases the SCL. The results of the physiological response suggest that having rest periods depicting the natural environment, even if indirectly and briefly, can restore the directed attention (i.e., mental resources) for the task. This experiment revealed a useful method of resting for urban workers and students to restore their attention to a task.

Learning Subject-Generalized Topographical EEG Embeddings Using Deep Variational Autoencoders and Domain-Adversarial Regularization.

Two of the biggest challenges in building models for detecting emotions from electroencephalography (EEG) devices are the relatively small amount of labeled samples and the strong variability of signal feature distributions between different subjects. In this study, we propose a context-generalized model that tackles the data constraints and subject variability simultaneously using a deep neural network architecture optimized for normally distributed subject-independent feature embeddings. Variational autoencoders (VAEs) at the input level allow the lower feature layers of the model to be trained on both labeled and unlabeled samples, maximizing the use of the limited data resources. Meanwhile, variational regularization encourages the model to learn Gaussian-distributed feature embeddings, resulting in robustness to small dataset imbalances. Subject-adversarial regularization applied to the bi-lateral features further enforces subject-independence on the final feature embedding used for emotion classification. The results from subject-independent performance experiments on the SEED and DEAP EEG-emotion datasets show that our model generalizes better across subjects than other state-of-the-art feature embeddings when paired with deep learning classifiers. Furthermore, qualitative analysis of the embedding space reveals that our proposed subject-invariant bi-lateral variational domain adversarial neural network (BiVDANN) architecture may improve the subject-independent performance by discovering normally distributed features.

Approach of visual stimuli facilitates the prediction of tactile events and suppresses beta band oscillations around the primary somatosensory area.

The purpose of the present study was to investigate whether the approach of visual stimuli influences prediction of subsequent tactile events. For this purpose, we examined electroencephalograms (EEGs) during the prediction of tactile events when visual stimuli did or did not approach. Tactile stimuli were presented with a high probability (80%) of being applied to the left (or right) index finger and a low probability (20%) of being applied to the opposite index finger. In the approach condition, visual stimuli were presented towards the hand to which the high-probability tactile stimuli were presented; in the neutral condition, visual stimuli did not approach. The result of time-frequency analysis for the EEGs showed that beta band event-related spectral perturbation at the electrodes around the primary somatosensory area (C3 and C4) was suppressed about 300 ms before the presentation of a tactile stimulus and that event-related desynchronization (ERD) occurred in all conditions. Moreover, the beta band ERD of the approach condition was larger than that of the neutral condition. These results provide evidence that the approach of visual stimuli facilitates prediction itself for subsequent tactile events.

Multiple Spatial Coordinates Influence the Prediction of Tactile Events Facilitated by Approaching Visual Stimuli.

Interaction with other sensory information is important for prediction of tactile events. Recent studies have reported that the approach of visual information toward the body facilitates prediction of subsequent tactile events. However, the processing of tactile events is influenced by multiple spatial coordinates, and it remains unclear how this approach effect influences tactile events in different spatial coordinates, i.e., spatial reference frames. We investigated the relationship between the prediction of a tactile stimulus via this approach effect and spatial coordinates by comparing ERPs. Participants were asked to place their arms on a desk and required to respond tactile stimuli which were presented to the left (or right) index finger with a high probability (80%) or to the opposite index finger with a low probability (20%). Before the presentation of each tactile stimulus, visual stimuli approached sequentially toward the hand to which the high-probability tactile stimulus was presented. In the uncrossed condition, each hand was placed on the corresponding side. In the crossed condition, each hand was crossed and placed on the opposite side, i.e., left (right) hand placed on the right (left) side. Thus, the spatial location of the tactile stimulus and hand was consistent in the uncrossed condition and inconsistent in the crossed condition. The results showed that N1 amplitudes elicited by high-probability tactile stimuli only decreased in the uncrossed condition. These results suggest that the prediction of a tactile stimulus facilitated by approaching visual information is influenced by multiple spatial coordinates.

Warmer Environments Increase Implicit Mental Workload Even If Learning Efficiency Is Enhanced.

Climate change is one of the most important issues for humanity. To defuse this problem, it is considered necessary to improve energy efficiency, make energy sources cleaner, and reduce energy consumption in urban areas. The Japanese government has recommended an air conditioner setting of 28°C in summer and 20°C in winter since 2005. The aim of this setting is to save energy by keeping room temperatures constant. However, it is unclear whether this is an appropriate temperature for workers and students. This study examined whether thermal environments influence task performance over time. To examine whether the relationship between task performance and thermal environments influences the psychological states of participants, we recorded their subjective rating of mental workload along with their working memory score, electroencephalogram (EEG), heart rate variability, skin conductance level (SCL), and tympanum temperature during the task and compared the results among different conditions. In this experiment, participants were asked to read some texts and answer questions related to those texts. Room temperature (18, 22, 25, or 29°C) and humidity (50%) were manipulated during the task and participants performed the task at these temperatures. The results of this study showed that the temporal cost of task and theta power of EEG, which is an index for concentration, decreased over time. However, subjective mental workload increased with time. Moreover, the low frequency to high frequency ratio and SCL increased with time and heat (25 and 29°C). These results suggest that mental workload, especially implicit mental workload, increases in warmer environments, even if learning efficiency is facilitated. This study indicates integrated evidence for relationships among task performance, psychological state, and thermal environment by analyzing behavioral, subjective, and physiological indexes multidirectionally.

Correction to: Congruency of intervening events and self-induced action influence prediction of final results.

The article Congruency of intervening events and self-induced action influence prediction of final results written by Tsukasa Kimura and Jun'ichi Katayama was originally published Online First without Open Access.

Congruency of intervening events and self-induced action influence prediction of final results.

Predicting self-induced stimuli is easier than predicting externally produced ones and the amplitude of event-related brain potentials (ERP) elicited by self-induced stimuli is smaller than that elicited by externally produced ones. Previous studies reported that these phenomena occurred strong when stimuli were presented immediately after self-induced action. To be able to adapt to changes, however, it is necessary to predict not only an event that follows a self-induced action but also a subsequent final result. We investigated whether congruency among self-induced actions, intervening events, and final results influences the processing of final results. The congruency of an intervening event with self-induced action was task-irrelevant information for the required response to a final result. The results showed that the P1 amplitude elicited by the final result (i.e., somatosensory stimulus) when an intervening event was congruent with self-induced action was smaller than other elicited amplitudes. This suggests that the congruency of an intervening event and self-induced action may facilitate prediction of a final result, even when this congruency is irrelevant to the ongoing task.

The approach of visual stimuli influences expectations about stimulus types for subsequent somatosensory stimuli.

It is known that perceiving a visual stimulus influences the processing of subsequent somatosensory stimuli. In particular, an emotion-laden visual stimulus influences the processing of types of subsequent somatosensory stimuli. Additionally, visual stimuli approaching the body facilitate spatial and temporal expectations about subsequent somatosensory stimuli even if the visual stimuli do not contain emotional information; however, it remains unclear whether the approach of non-emotional visual stimuli also influences such expectations. To investigate whether the approach of non-emotional visual stimuli influences expectations about types of subsequent somatosensory stimuli, event-related brain potentials (ERPs) during a simple reaction time task using somatosensory stimuli were recorded. Specific colors of visual stimuli and types of somatosensory stimuli were combined to form congruent and incongruent trials. In the congruent trials, specific combinations (e.g., blue color and a single pulse) were presented (80% of the trials), whereas in the incongruent trials, different combinations (e.g., blue color and a train pulse) were presented (20% of the trials). Under the approach condition, the visual stimuli sequentially approached the wrist to which the somatosensory stimulus was presented. In the neutral condition, the visual stimuli did not approach. The results of the ERP analysis showed that incongruence evoked a P3 response with larger amplitude under the approach condition than under the neutral condition. This result suggests that visual stimuli that approach the body function as clues regarding the types of subsequent somatosensory stimuli even if the visual stimuli do not contain emotional information.

Visual stimuli approaching toward the body influence temporal expectations about subsequent somatosensory stimuli.

The present study investigated whether visual stimuli approaching the body influence temporal expectations about subsequent somatosensory stimuli. To examine this question, we recorded event-related brain potentials (ERPs) during a simple reaction time task using somatosensory stimuli. Fourteen participants were asked to place their arms on a desk, and three light-emitting diodes (LEDs) were placed at equal distances between their arms. Each trial was composed of three visual stimuli (i.e., LEDs), and one subsequent electrical stimulus (i.e., somatosensory stimulus) to one wrist. The stimulus onset asynchrony (SOA) between the visual stimuli was set to 1000ms. The SOA between the third visual stimulus and the somatosensory stimulus was set to 1000ms (standard; p=0.75), 500ms (early deviation; p=0.125), and 1500ms (late deviation; p=0.125). In the approach condition, the left, center, and right LEDs (or reverse) were turned sequentially toward the wrist to which the somatosensory stimulus was presented. In the neutral condition, the center LED was flashed three times. The N1 amplitudes for early deviations of stimuli were larger under the approach condition than under the neutral condition. These results show that prior visual stimuli facilitate temporal expectations about subsequent somatosensory stimuli, i.e., visual stimuli approaching toward the body facilitate the processing of early deviant stimuli. The present study indicates the existence of a function of supramodal temporal expectation and detection of deviation from this expectation using the approach of visual stimuli toward the body.

Regularity of approaching visual stimuli influences spatial expectations for subsequent somatosensory stimuli.

This study examines how the regularity of visual stimuli approaching the body influences spatial expectations of subsequent somatosensory stimuli by recording event-related brain potentials (ERPs) during a simple reaction time (RT) task involving responses to somatosensory stimuli. Twenty-one participants were instructed to put their arms on a desk, and three LEDs were placed equidistantly between their arms. Electrical stimuli were presented with a high probability (80%) of being applied to one wrist and a low probability (20%) of being applied to the opposite wrist. One trial was composed of three visual stimuli followed by one electrical stimulus. In the regular approach condition, LEDs flashed sequentially toward the wrist with the high-probability somatosensory stimulus. In the irregular approach condition, the first and second visual stimuli were presented randomly, but the third visual stimulus was invariably presented near the wrist with the high-probability stimulus. In both conditions, RTs for low-probability stimuli were slower than those for high-probability stimuli, and the low-probability stimuli elicited larger P3 amplitudes than the high-probability stimuli. Furthermore, the largest P3 amplitude was elicited by low-probability stimuli under the regular approach condition, whereas the amplitudes of contingent negative variation (CNV) elicited before the presentation of the somatosensory stimuli did not differ between conditions. These results indicate that regularity of visual stimuli approaching the body facilitates an automatic spatial expectation for subsequent somatosensory stimuli.

Approach of visual stimuli modulates spatial expectations for subsequent somatosensory stimuli.

To examine how the approach of visual stimuli toward the body influences expectations regarding subsequent somatosensory stimuli, we recorded event-related brain potentials (ERPs; nose reference) during a simple reaction time to somatosensory stimuli task. Twelve participants were asked to place their arms on a desk, and three LEDs were placed between their arms at equal intervals. Electrical stimuli were presented to the left (or right) wrist at a high probability (80%) or to the opposite wrist at a low probability (20%). Each trial was composed of three visual stimuli followed by one electrical stimulus. In Experiment 1, the right, center, and left (or left, center, and right) LEDs were turned on sequentially toward the wrist to which the high probability somatosensory stimuli was presented (congruent condition), or the center LED were presented three times (neutral condition). Experiment 2 was composed of the congruent condition and the inverse of the congruent condition (incongruent condition). In both experiments, the reaction times to low probability stimuli were longer than those to high probability stimuli. Moreover, the low probability stimuli elicited a larger P3 amplitude than the high probability stimuli. In addition, the P3 amplitude was higher under the visual approach condition (i.e., the congruent condition in each experiment) than under the control condition (i.e., the neutral and incongruent conditions). Furthermore, no effect on the CNV amplitude before the somatosensory stimuli was found. These results suggest that visual stimuli directed toward the body induce an automatic spatial expectation for subsequent somatosensory stimuli.