Using Fixation-Related Potentials for Inspecting Natural Interactions.

Brain-Computer Interfaces (BCI) offer unique windows into the cognitive processes underlying human-machine interaction. Identifying and analyzing the appropriate brain activity to have access to such windows is often difficult due to technical or psycho-physiological constraints. Indeed, studying interactions through this approach frequently requires adapting them to accommodate specific BCI-related paradigms which change the functioning of their interface on both the human-side and the machine-side. The combined examination of Electroencephalography and Eyetracking recordings, mainly by means of studying Fixation-Related Potentials, can help to circumvent the necessity for these adaptations by determining interaction-relevant moments during natural manipulation. In this contribution, we examine how properties contained within the bi-modal recordings can be used to assess valuable information about the interaction. Practically, three properties are studied which can be obtained solely through data obtained from analysis of the recorded biosignals. Namely, these properties consist of relative gaze metrics, being abstractions of the gaze patterns, the amplitude variations in the early brain activity potentials and the brain activity frequency band differences between fixations. Through their observation, information about three different aspects of the explored interface are obtained. Respectively, the properties provide insights about general perceived task difficulty, locate moments of higher attentional effort and discriminate between moments of exploration and moments of active interaction.

Flex-printed forehead EEG sensors (fEEGrid) for long-term EEG acquisition.

OBJECTIVE: In this report we present the fEEGrid, an electrode array applied to the forehead that allows convenient long-term recordings of electroencephalography (EEG) signals over many hours. APPROACH: Twenty young, healthy participants wore the fEEGrid and completed traditional EEG paradigms in two sessions on the same day. The sessions were eight hours apart, participants performed the same tasks in an early and a late session. For the late session fEEGrid data were concurrently recorded with traditional cap EEG data. MAIN RESULTS: Our analyses show that typical event-related potentials responses were captured reliably by the fEEGrid. Single-trial analyses revealed that classification was possible above chance level for auditory and tactile oddball paradigms. We also found that the signal quality remained high and impedances did not deteriorate, but instead improved over the course of the day. Regarding wearing comfort, all participants indicated that the fEEGrid was comfortable to wear and did not cause any pain even after 8 h of wearing it. SIGNIFICANCE: We show in this report, that high quality EEG signals can be captured with the fEEGrid reliably, even in long-term recording scenarios and with a signal quality that may be considered suitable for online brain-computer Interface applications.

Frequency Band Variations Predict EEG Single-Trial Classification Performance in Disorder of Consciousness Patients.

Single-trial classification of EEG data from Disorder of Consciousness patients (DoC) has proved particularly challenging. We present an approach that establishes a measure to relate the performance of single-trial classification of DoC patient EEG data with relational frequency bands and thus with their mental state. We evaluate our approach on 31 patient data sets from two studies, showing that our measure indicates for different data sets a particular likelihood for misclassifying either target or non-target class samples.

Toward FRP-Based Brain-Machine Interfaces-Single-Trial Classification of Fixation-Related Potentials.

The co-registration of eye tracking and electroencephalography provides a holistic measure of ongoing cognitive processes. Recently, fixation-related potentials have been introduced to quantify the neural activity in such bi-modal recordings. Fixation-related potentials are time-locked to fixation onsets, just like event-related potentials are locked to stimulus onsets. Compared to existing electroencephalography-based brain-machine interfaces that depend on visual stimuli, fixation-related potentials have the advantages that they can be used in free, unconstrained viewing conditions and can also be classified on a single-trial level. Thus, fixation-related potentials have the potential to allow for conceptually different brain-machine interfaces that directly interpret cortical activity related to the visual processing of specific objects. However, existing research has investigated fixation-related potentials only with very restricted and highly unnatural stimuli in simple search tasks while participant's body movements were restricted. We present a study where we relieved many of these restrictions while retaining some control by using a gaze-contingent visual search task. In our study, participants had to find a target object out of 12 complex and everyday objects presented on a screen while the electrical activity of the brain and eye movements were recorded simultaneously. Our results show that our proposed method for the classification of fixation-related potentials can clearly discriminate between fixations on relevant, non-relevant and background areas. Furthermore, we show that our classification approach generalizes not only to different test sets from the same participant, but also across participants. These results promise to open novel avenues for exploiting fixation-related potentials in electroencephalography-based brain-machine interfaces and thus providing a novel means for intuitive human-machine interaction.

Using a cVEP-Based Brain-Computer Interface to Control a Virtual Agent.

Brain-computer interfaces provide a means for controlling a device by brain activity alone. One major drawback of noninvasive BCIs is their low information transfer rate, obstructing a wider deployment outside the lab. BCIs based on codebook visually evoked potentials (cVEP) outperform all other state-of-the-art systems in that regard. Previous work investigated cVEPs for spelling applications. We present the first cVEP-based BCI for use in real-world settings to accomplish everyday tasks such as navigation or action selection. To this end, we developed and evaluated a cVEP-based on-line BCI that controls a virtual agent in a simulated, but realistic, 3-D kitchen scenario. We show that cVEPs can be reliably triggered with stimuli in less restricted presentation schemes, such as on dynamic, changing backgrounds. We introduce a novel, dynamic repetition algorithm that allows for optimizing the balance between accuracy and speed individually for each user. Using these novel mechanisms in a 12-command cVEP-BCI in the 3-D simulation results in ITRs of 50 bits/min on average and 68 bits/min maximum. Thus, this work supports the notion of cVEP-BCIs as a particular fast and robust approach suitable for real-world use.

Does machine-mediated interaction induce inter-brain synchrony?--A hyperscanning study.

We present a study in which participants were trained in several sessions to control a (comparatively simple) robot via an EEG-/motor imagery-based Brain-Computer Interface (BCI). In the final (experiment) session pairs of participants were formed and each participant controlled one of two robots in a shared space. EEG data was recorded synchronously from both participants. We performed a joint data analysis on the datasets and found increases of phase-locking in µ- and θ-band. One such phase-locking effect appears to be time-locked to the start of the robotic action.

Hierarchical Codebook Visually Evoked Potentials for fast and flexible BCIs.

Brain-Computer Interfaces provide a direct communication channel from the brain to a technical device. One major problem in state-of-the-art BCIs is their low communication speed. BCIs based on Codebook Visually Evoked Potentials (cVEP) outperform all other non-invasive approaches in terms of information transfer rate. Used only in spelling tasks so far, more flexibility with respect to stimulus structure and properties is needed. We propose using hierarchical codebook vectors together with varying color schemes to increase the stimulus flexibility. An off-line study showed that our novel hcVEP approach is capable of discriminating groups of targets after only 250 ms of stimulus flickering and the final target within the group after 1s. The accuracies are 81% and 67%, respectively. Different color schemes (black/white and green/red) are equally effective.

A hybrid brain interface for a humanoid robot assistant.

We present an advanced approach towards a semi-autonomous, robotic personal assistant for handicapped people. We developed a multi-functional hybrid brain-robot interface that provides a communication channel between humans and a state-of-the-art humanoid robot, Honda's Humanoid Research Robot. Using cortical signals, recorded, processed and translated by an EEG-based brain-machine interface (BMI), human-robot interaction functions independently of users' motor control deficits. By exploiting two distinct cortical activity patterns, P300 and event-related desynchronization (ERD), the interface provides different dimensions for robot control. An empirical study demonstrated the functionality of the BMI guided humanoid robot. All participants could successfully control the robot that accomplished a shopping task.

The MindGame: a P300-based brain-computer interface game.

We present a Brain-Computer Interface (BCI) game, the MindGame, based on the P300 event-related potential. In the MindGame interface P300 events are translated into movements of a character on a three-dimensional game board. A linear feature selection and classification scheme is applied to identify P300 events and calculate gradual feedback features from a scalp electrode array. The classification during the online run of the game is computed on a single-trial basis without averaging over subtrials. We achieve classification rates of 0.65 on single-trials during the online operation of the system while providing gradual feedback to the player.