Meta-analysis of digital game and study characteristics eliciting physiological stress responses.

Digital games have been used as stressors in a range of disciplines for decades. Nonetheless, the underlying characteristics of these stressors and the study in which the stressor was applied are generally not recognized for their moderating effect on the measured physiological stress responses. We have therefore conducted a meta-analysis that analyzes the effects of characteristics of digital game stressors and study design on heart rate, systolic and diastolic blood pressure, in studies carried out from 1976 to 2012. In order to assess the differing quality between study designs, a new scale is developed and presented, coined reliability of effect size. The results show specific and consistent moderating functions of both game and study characteristics, on average accounting for around 43%, and in certain cases up to 57% of the variance found in physiological stress responses. Possible cognitive and physiological processes underlying these moderating functions are discussed, and a new model integrating these processes with the moderating functions is presented. These findings indicate that a digital game stressor does not act as a stressor by virtue of being a game, but rather derives its stressor function from its characteristics and the methodology in which it is used. This finding, together with the size of the associated moderations, indicates the need for a standardization of digital game stressors.

GASICA: generic automated stress induction and control application design of an application for controlling the stress state.

In a multitude of research and therapy paradigms it is relevant to know, and desirably to control, the stress state of a patient or participant. Examples include research paradigms in which the stress state is the dependent or independent variable, or therapy paradigms where this state indicates the boundaries of the therapy. To our knowledge, no application currently exists that focuses specifically on the automated control of the stress state while at the same time being generic enough to be used in various therapy and research purposes. Therefore, we introduce GASICA, an application aimed at the automated control of the stress state in a multitude of therapy and research paradigms. The application consists of three components: a digital stressor game, a set of measurement devices, and a feedback model. These three components form a closed loop (called a biocybernetic loop by Pope et al. (1995) and Fairclough (2009) that continuously presents an acute psychological stressor, measures several physiological responses to this stressor, and adjusts the stressor intensity based on these measurements by means of the feedback model, hereby aiming to control the stress state. In this manner GASICA presents multidimensional and ecological valid stressors, whilst continuously in control of the form and intensity of the presented stressors, aiming at the automated control of the stress state. Furthermore, the application is designed as a modular open-source application to easily implement different therapy and research tasks using a high-level programming interface and configuration file, and allows for the addition of (existing) measurement equipment, making it usable for various paradigms.

Gaze-independent ERP-BCIs: augmenting performance through location-congruent bimodal stimuli.

Gaze-independent event-related potential (ERP) based brain-computer interfaces (BCIs) yield relatively low BCI performance and traditionally employ unimodal stimuli. Bimodal ERP-BCIs may increase BCI performance due to multisensory integration or summation in the brain. An additional advantage of bimodal BCIs may be that the user can choose which modality or modalities to attend to. We studied bimodal, visual-tactile, gaze-independent BCIs and investigated whether or not ERP components' tAUCs and subsequent classification accuracies are increased for (1) bimodal vs. unimodal stimuli; (2) location-congruent vs. location-incongruent bimodal stimuli; and (3) attending to both modalities vs. to either one modality. We observed an enhanced bimodal (compared to unimodal) P300 tAUC, which appeared to be positively affected by location-congruency (p = 0.056) and resulted in higher classification accuracies. Attending either to one or to both modalities of the bimodal location-congruent stimuli resulted in differences between ERP components, but not in classification performance. We conclude that location-congruent bimodal stimuli improve ERP-BCIs, and offer the user the possibility to switch the attended modality without losing performance.

Pre- and post-stimulus EEG patterns associated with the touch-induced illusory flash.

Pairing two brief auditory beeps with a single flash can evoke the percept of a second, illusory, flash. Investigations of the underlying neural mechanisms are limited to post-stimulus effects of this sound-induced illusory flash. We investigated whether touch modulates the visual evoked potential in a similar vein, and also looked at pre-stimulus activity. Electroencephalogram (EEG) was recorded over occipital and parieto-occipital areas of 12 observers. We compared bimodal EEG to its unimodal constituents (i.e., the difference waves) and found significant positive deflections around 110 ms and 200 ms and negative deflections around 330 ms and 390 ms from stimulus onset. These results are similar to those reported for the sound-induced illusion, albeit somewhat later. Furthermore, comparison of the EEG activity between those trials in which the illusion was perceived and those in which it was absent revealed that the phase of pre-stimulus alpha was linked to perceiving the illusion or not. We conclude that touch can modulate activity in the visual cortex and that similar neural mechanisms underlie perception of the sound- and touch-induced illusory flash and that the phase of the alpha wave at the moment of presentation that affects perception.

Observers can reliably identify illusory flashes in the illusory flash paradigm.

In the illusory flash paradigm, a single flash may be experienced as two flashes when accompanied by two beeps or taps, and two flashes may be experienced as a single flash when accompanied by one beep or tap. The classic paradigm restricts responses to '1' and '2' (2-AFC), ignoring possible qualitative differences between real and illusory flashes and implicitly assuming that illusory flashes are indistinguishable from real flashes. We added a third response category 'different from that of either 1 or 2 flashes' (3-AFC). Eight naïve and 6 experienced observers responded to 160 real and 160 illusory flash trials. Experienced observers were exposed to 1,200 trials before the experiment but without receiving feedback on their performance. The third response category was used for only 4 % of the real flash trials and for 44 % of the illusory flash trials. Experienced observers did so more often (78 %) than naïve observers (18 %). This shows that observers can reliably identify illusory flashes and indicates that mere exposure to illusory flash trials (without feedback) is enough to detect and classify potential qualitative differences between real and illusory flashes.

The perception of verticality in lunar and Martian gravity conditions.

Although the mechanisms of neural adaptation to weightlessness and re-adaptation to Earth-gravity have received a lot of attention since the first human space flight, there is as yet little knowledge about how spatial orientation is affected by partial gravity, such as lunar gravity of 0.16 g or Martian gravity of 0.38 g. Up to now twelve astronauts have spent a cumulated time of approximately 80 h on the lunar surface, but no psychophysical experiments were conducted to investigate their perception of verticality. We investigated how the subjective vertical (SV) was affected by reduced gravity levels during the first European Parabolic Flight Campaign of Partial Gravity. In normal and hypergravity, subjects accurately aligned their SV with the gravitational vertical. However, when gravity was below a certain threshold, subjects aligned their SV with their body longitudinal axis. The value of the threshold varied considerably between subjects, ranging from 0.03 to 0.57 g. Despite the small number of subjects, there was a significant positive correlation of the threshold with subject age, which calls for further investigation.

Does bimodal stimulus presentation increase ERP components usable in BCIs?

Event-related potential (ERP)-based brain-computer interfaces (BCIs) employ differences in brain responses to attended and ignored stimuli. Typically, visual stimuli are used. Tactile stimuli have recently been suggested as a gaze-independent alternative. Bimodal stimuli could evoke additional brain activity due to multisensory integration which may be of use in BCIs. We investigated the effect of visual-tactile stimulus presentation on the chain of ERP components, BCI performance (classification accuracies and bitrates) and participants' task performance (counting of targets). Ten participants were instructed to navigate a visual display by attending (spatially) to targets in sequences of either visual, tactile or visual-tactile stimuli. We observe that attending to visual-tactile (compared to either visual or tactile) stimuli results in an enhanced early ERP component (N1). This bimodal N1 may enhance BCI performance, as suggested by a nonsignificant positive trend in offline classification accuracies. A late ERP component (P300) is reduced when attending to visual-tactile compared to visual stimuli, which is consistent with the nonsignificant negative trend of participants' task performance. We discuss these findings in the light of affected spatial attention at high-level compared to low-level stimulus processing. Furthermore, we evaluate bimodal BCIs from a practical perspective and for future applications.

Control-display mapping in brain-computer interfaces.

Event-related potential (ERP) based brain-computer interfaces (BCIs) employ differences in brain responses to attended and ignored stimuli. When using a tactile ERP-BCI for navigation, mapping is required between navigation directions on a visual display and unambiguously corresponding tactile stimuli (tactors) from a tactile control device: control-display mapping (CDM). We investigated the effect of congruent (both display and control horizontal or both vertical) and incongruent (vertical display, horizontal control) CDMs on task performance, the ERP and potential BCI performance. Ten participants attended to a target (determined via CDM), in a stream of sequentially vibrating tactors. We show that congruent CDM yields best task performance, enhanced the P300 and results in increased estimated BCI performance. This suggests a reduced availability of attentional resources when operating an ERP-BCI with incongruent CDM. Additionally, we found an enhanced N2 for incongruent CDM, which indicates a conflict between visual display and tactile control orientations. PRACTITIONER SUMMARY: Incongruency in control-display mapping reduces task performance. In this study, brain responses, task and system performance are related to (in)congruent mapping of command options and the corresponding stimuli in a brain-computer interface (BCI). Directional congruency reduces task errors, increases available attentional resources, improves BCI performance and thus facilitates human-computer interaction.

Human locomotion through a multiple obstacle environment: strategy changes as a result of visual field limitation.

This study investigated how human locomotion through an obstacle environment is influenced by visual field limitation. Participants were asked to walk at a comfortable pace to a target location while avoiding multiple vertical objects. During this task, they wore goggles restricting their visual field to small (S: 40°×25°), medium (M: 80°×60°), large (L: 115°×90°), or unlimited (U) visual field sizes. Full-body motion capture was used to extract for each trial the mean speed, pathlength, mean step width, magnitude of head rotation and head mean angular speed. The results show that compared with the U condition, the M and L conditions caused participants to select a wider path around the obstacles without slowing down or altering step width. However, the S condition did slow down the participants, and increased both their step width and path length. We conclude that only for the S condition, balancing problems were substantial enough to spend more energy associated with increased step width. In all cases, participants choose to optimize safety (collision avoidance) at the cost of spending more energy.

Obstacle crossing with lower visual field restriction: shifts in strategy.

In this study, the authors investigated how restriction of the vertical viewing angle influences obstacle-crossing behavior. Twelve participants stepped over obstacles of different dimensions while wearing visual-field-restricting goggles. Using full-body motion capture, several kinematic measures were extracted and analyzed. Results indicate that both a 40° and 90° vertical viewing angle yielded increased step length and toe clearance as compared to an unrestricted view (i.e., 135°), whereas speed remained unaltered. A further decrease (to 25°) caused participants to slow down in addition to a further increase of step length and toe clearance. These results are discussed in terms of a change in priorities, from conservation of energy and time to safety.

Multisensory effects differ for counting small and large pulse numbers.

In Illusory Flash (IF) experiments, congruent multisensory presentation has no effect on the mean estimate of the number of events, but decreases the variance in comparison with unisensory presentation. In contrast, congruent multisensory presentation in other Temporal Numerosity Judgement (TNJ) tasks affects the mean estimate (i.e., it often results in a reduction in underestimation) and increases the variance. In three experiments, we investigated the differences between both paradigms as possible causes of this discrepancy: the presence or absence of incongruent stimuli (Experiment 1), the instruction to the observer to either count flashes, beeps or multisensory events (Experiment 2), and the range of pulses presented (Experiment 3). We found significant differences between the mean numerosity estimate of multisensory and unisensory series in Experiment 3, but not in 1 and 2. This suggests that the difference in the range of pulses presented in IF (1-3 pulses) and TNJ (1-10 pulses) is the primary cause of the discrepancy. In the discussion we propose that this result may be explained by the use of two different strategies and their susceptibility to multisensory presentation. For small pulse numbers, observers can accurately count both unisensory and multisensory pulses. For larger numbers, observers can no longer count but will estimate the number based on the pulse series duration which is improved for multisensory stimuli.

Integration of visual and inertial cues in perceived heading of self-motion.

In the present study, we investigated whether the perception of heading of linear self-motion can be explained by Maximum Likelihood Integration (MLI) of visual and non-visual sensory cues. MLI predicts smaller variance for multisensory judgments compared to unisensory judgments. Nine participants were exposed to visual, inertial, or visual-inertial motion conditions in a moving base simulator, capable of accelerating along a horizontal linear track with variable heading. Visual random-dot motion stimuli were projected on a display with a 40° horizontal × 32° vertical field of view (FoV). All motion profiles consisted of a raised cosine bell in velocity. Stimulus heading was varied between 0 and 20°. After each stimulus, participants indicated whether perceived self-motion was straight-ahead or not. We fitted cumulative normal distribution functions to the data as a psychometric model and compared this model to a nested model in which the slope of the multisensory condition was subject to the MLI hypothesis. Based on likelihood ratio tests, the MLI model had to be rejected. It seems that the imprecise inertial estimate was weighed relatively more than the precise visual estimate, compared to the MLI predictions. Possibly, this can be attributed to low realism of the visual stimulus. The present results concur with other findings of overweighing of inertial cues in synthetic environments.

Counting visual and tactile events: the effect of attention on multisensory integration.

Irrelevant events in one sensory modality can influence the number of events that are perceived in another modality. Previously, the underlying process of sensory integration was studied in conditions in which participants knew a priori which sensory modality was relevant and which was not. Consequently, (bottom-up) sensory interference and (top-down) selective attention were confounded. We disentangled these effects by measuring the influence of visual flashes on the number of tactile taps that were perceived, and vice versa, in two conditions. In the cue condition, participants were instructed on which modality to report before the bimodal stimulus was presented. In the no-cue condition, they were instructed after stimulus presentation. Participants reported the number of events that they perceived for bimodal combinations of one, two, or three flashes and one, two, or three taps. Our main findings were that (1) in no-cue conditions, the influence of vision on touch was stronger than was the influence of touch on vision; (2) in cue conditions, the integration effects were smaller than those in no-cue conditions; and (3) irrelevant taps were less easily ignored than were irrelevant flashes. This study disentangled previously confounded bottom-up and top-down effects: The bottom-up influence of vision on touch was stronger, but vision was also more easily suppressed by top-down selective attention. We have compared our results qualitatively and quantitatively with recently proposed sensory-integration models.

Multisensory temporal numerosity judgment.

In temporal numerosity judgment, observers systematically underestimate the number of pulses. The strongest underestimations occur when stimuli are presented with a short interstimulus interval (ISI) and are stronger for vision than for audition and touch. We investigated if multisensory presentation leads to a reduction of underestimation. Participants were presented with 2 to 10 (combinations of) auditory beeps, tactile taps to the index finger and visual flashes at different ISIs (20 to 320 ms). For all presentation modes, we found underestimation, except for small number of pulses. A control experiment showed that the latter is due to a (cognitive) range effect. Averaged over conditions, the order of performance of sensory modalities is touch, audition and last vision. Generally, multisensory presentation improves performance over the unisensory presentations. For larger ISIs (160 and 320 ms), we found a tendency toward a reduction in variance for the multisensory presentation modes. For smaller ISIs (20 and 40 ms), we found a reduction in underestimation, but an increase in variance for the multisensory presentation modes. In the discussion, we relate these two findings to Maximum Likelihood Estimation (MLE) models predicting that multisensory integration reduces variance.

Vibro-tactile and visual asynchronies: sensitivity and consistency.

We investigated the consistency between tactually and visually designated empty time intervals. In a forced-choice discrimination task, participants judged whether the second of two intervals was shorter or longer than the first interval. Two pulses defined the intervals. The pulse was either a vibro-tactile burst presented to the fingertip, or a foveally presented white square. The comparisons were made for uni-modal and cross-modal intervals. We used four levels of standard interval durations in the range of 100- 800 ms. The results showed that tactile empty intervals must be 8.5% shorter to be perceived as long as visual intervals. This cross-modal bias is larger for small intervals and decreases with increasing standard intervals. The Weber fractions (the threshold divided by the standard interval) are 20% and are constant over the standard intervals. This indicates that the Weber law holds for the range of interval lengths tested. Furthermore, the Weber fractions are consistent over uni-modal and cross-modal comparisons, which indicates that there is no additional noise involved in the cross-modal comparisons.

Effects of head-slaved navigation and the use of teleports on spatial orientation in virtual environments.

The type of navigation interface in a virtual environment (VE)--head slaved or indirect--determines whether or not proprioceptive feedback stimuli are present during movement. In addition, teleports can be used, which do not provide continuous movement but, rather, discontinuously displace the viewpoint over large distances. A two-part experiment was performed. The first part investigated whether head-slaved navigation provides an advantage for spatial learning in a VE. The second part investigated the role of anticipation when using teleports. The results showed that head-slaved navigation has an advantage over indirect navigation for the acquisition of spatial knowledge in a VE. Anticipating the destination of the teleport prevented disorientation after the displacement to a great extent but not completely. The time that was needed for anticipation increased if the teleport involved a rotation of the viewing direction. This research shows the potential added value of using a head-slaved navigation interface--for example, when using VE for training purposes--and provides practical guidelines for the use of teleports in VE applications.