Feedback Related Potentials for EEG-Based Typing Systems.

Error related potentials (ErrP), which are elicited in the EEG in response to a perceived error, have been used for error correction and adaption in the event related potential (ERP)-based brain computer interfaces designed for typing. In these typing interfaces, ERP evidence is collected in response to a sequence of stimuli presented usually in the visual form and the intended user stimulus is probabilistically inferred (stimulus with highest probability) and presented to the user as the decision. If the inferred stimulus is incorrect, ErrP is expected to be elicited in the EEG. Early approaches to use ErrP in the design of typing interfaces attempt to make hard decisions on the perceived error such that the perceived error is corrected and either the sequence of stimuli are repeated to obtain further ERP evidence, or without further repetition the stimulus with the second highest probability is presented to the user as the decision of the system. Moreover, none of the existing approaches use a language model to increase the performance of typing. In this work, unlike the existing approaches, we study the potential benefits of fusing feedback related potentials (FRP), a form of ErrP, with ERP and context information (language model, LM) in a Bayesian fashion to detect the user intent. We present experimental results based on data from 12 healthy participants using RSVP Keyboard™ to complete a copy-phrase-task. Three paradigms are compared: [P1] uses only ERP/LM Bayesian fusion; [P2] each RSVP sequence is appended with the top candidate in the alphabet according to posterior after ERP evidence fusion; corresponding FRP is then incorporated; and [P3] the top candidate is shown as a prospect to generate FRP evidence only if its posterior exceeds a threshold. Analyses indicate that ERP/LM/FRP evidence fusion during decision making yields significant speed-accuracy benefits for the user.

Language-Model Assisted And Icon-based Communication Through a Brain Computer Interface With Different Presentation Paradigms.

Augmentative and alternative communication (AAC) is typically used by people with severe speech and physical disabilities (SSPI) and is one of the main application areas for brain computer interface (BCI) technology. The target population includes people with cerebral palsy (CP), amyotrophic lateral sclerosis (ALS) and locked-in-syndrome (LIS). Word-based AAC systems are mainly faster than letter-based counterparts and are usually supplemented by icons to aid the users. Those iconbased AAC systems that use binary signaling methods such as single click can convert into a single input BCI systems such as ERP detection. Matrix speller paradigm are typically used to help users identify their target icon on the screen, however it ties screen space to vocabulary size and navigation complexity, which may require users to make repetitive head, neck, or eye movements to visually locate their intended targets on the screen. Rapid serial visual presentation (RSVP) is an alternative interface that minimizes required movement by displaying all icons at a fix location, one at a time. IconMessenger is an icon-based BCI-AAC system that combines ERP signal detection with a unified framework for different presentation paradigms including RSVP, matrix speller row&column presentation (RCP) and matrix speller single character presentation (SCP). Icon- Messenger also take advantage of a unique sem-gram language model, incorporated tightly in the inference engine. In this study, we assess the ERP shape, classification accuracy and typing performance of different presentation paradigms on 10 healthy participants.

EEG-assisted Modulation of Sound Sources in the Auditory Scene.

Noninvasive EEG (electroencephalography) based auditory attention detection could be useful for improved hearing aids in the future. This work is a novel attempt to investigate the feasibility of online modulation of sound sources by probabilistic detection of auditory attention, using a noninvasive EEG-based brain computer interface. Proposed online system modulates the upcoming sound sources through gain adaptation which employs probabilistic decisions (soft decisions) from a classifier trained on offline calibration data. In this work, calibration EEG data were collected in sessions where the participants listened to two sound sources (one attended and one unattended). Cross-correlation coefficients between the EEG measurements and the attended and unattended sound source envelope (estimates) are used to show differences in sharpness and delays of neural responses for attended versus unattended sound source. Salient features to distinguish attended sources from the unattended ones in the correlation patterns have been identified, and later they have been used to train an auditory attention classifier. Using this classifier, we have shown high offline detection performance with single channel EEG measurements compared to the existing approaches in the literature which employ large number of channels. In addition, using the classifier trained offline in the calibration session, we have shown the performance of the online sound source modulation system. We observe that online sound source modulation system is able to keep the level of attended sound source higher than the unattended source.

Decoding complex imagery hand gestures.

Brain computer interfaces (BCIs) offer individuals suffering from major disabilities an alternative method to interact with their environment. Sensorimotor rhythm (SMRs) based BCIs can successfully perform control tasks; however, the traditional SMR paradigms intuitively disconnect the control and real task, making them non-ideal for complex control scenarios. In this study we design a new, intuitively connected motor imagery (MI) paradigm using hierarchical common spatial patterns (HCSP) and context information to effectively predict intended hand grasps from electroencephalogram (EEG) data. Experiments with 5 participants yielded an aggregate classification accuracy-intended grasp prediction probability-of 64.5% for 8 different hand gestures, more than 5 times the chance level.

Context-aware recursive bayesian graph traversal in BCIs.

Noninvasive brain computer interfaces (BCI), and more specifically Electroencephalography (EEG) based systems for intent detection need to compensate for the low signal to noise ratio of EEG signals. In many applications, the temporal dependency information from consecutive decisions and contextual data can be used to provide a prior probability for the upcoming decision. In this study we proposed two probabilistic graphical models (PGMs), using context information and previously observed EEG evidences to estimate a probability distribution over the decision space in graph based decision-making mechanism. In this approach, user moves a pointer to the desired vertex in the graph in which each vertex represents an action. To select a vertex, a "Select" command, or a proposed probabilistic Selection criterion (PSC) can be used to automatically detect the user intended vertex. Performance of different PGMs and Selection criteria combinations are compared over a keyboard based on a graph layout. Based on the simulation results, probabilistic Selection criterion along with the probabilistic graphical model provides the highest performance boost for individuals with pour calibration performance and achieving the same performance for individuals with high calibration performance.

A Graphical Model for Online Auditory Scene Modulation Using EEG Evidence for Attention.

Recent findings indicate that brain interfaces have the potential to enable attention-guided auditory scene analysis and manipulation in applications, such as hearing aids and augmented/virtual environments. Specifically, noninvasively acquired electroencephalography (EEG) signals have been demonstrated to carry some evidence regarding, which of multiple synchronous speech waveforms the subject attends to. In this paper, we demonstrate that: 1) using data- and model-driven cross-correlation features yield competitive binary auditory attention classification results with at most 20 s of EEG from 16 channels or even a single well-positioned channel; 2) a model calibrated using equal-energy speech waveforms competing for attention could perform well on estimating attention in closed-loop unbalanced-energy speech waveform situations, where the speech amplitudes are modulated by the estimated attention posterior probability distribution; 3) such a model would perform even better if it is corrected (linearly, in this instance) based on EEG evidence dependence on speech weights in the mixture; and 4) calibrating a model based on population EEG could result in acceptable performance for new individuals/users; therefore, EEG-based auditory attention classifiers may generalize across individuals, leading to reduced or eliminated calibration time and effort.

An Active RBSE Framework to Generate Optimal Stimulus Sequences in a BCI for Spelling.

A class of brain computer interfaces (BCIs) employs noninvasive recordings of electroencephalography (EEG) signals to enable users with severe speech and motor impairments to interact with their environment and social network. For example, EEG based BCIs for typing popularly utilize event related potentials (ERPs) for inference. Presentation paradigm design in current ERP-based letter by letter typing BCIs typically query the user with an arbitrary subset characters. However, the typing accuracy and also typing speed can potentially be enhanced with more informed subset selection and flash assignment. In this manuscript, we introduce the active recursive Bayesian state estimation (active-RBSE) framework for inference and sequence optimization. Prior to presentation in each iteration, rather than showing a subset of randomly selected characters, the developed framework optimally selects a subset based on a query function. Selected queries are made adaptively specialized for users during each intent detection. Through a simulation-based study, we assess the effect of active-RBSE on the performance of a language-model assisted typing BCI in terms of typing speed and accuracy. To provide a baseline for comparison, we also utilize standard presentation paradigms namely, row and column matrix presentation paradigm and also random rapid serial visual presentation paradigms. The results show that utilization of active-RBSE can enhance the online performance of the system, both in terms of typing accuracy and speed. Moreover, we conduct real time experiments with human participants to study the human-in-the-loop effect on the performance of the proposed active-RBSE framework and consistent with the simulation results, the results of these experiments show improvement both in typing speed and accuracy.

Recursive Bayesian Coding for BCIs.

Brain-Computer Interfaces (BCIs) seek to infer some task symbol, a task relevant instruction, from brain symbols, classifiable physiological states. For example, in a motor imagery robot control task a user would indicate their choice from a dictionary of task symbols (rotate arm left, grasp, etc.) by selecting from a smaller dictionary of brain symbols (imagined left or right hand movements). We examine how a BCI infers a task symbol using selections of brain symbols. We offer a recursive Bayesian decision framework which incorporates context prior distributions (e.g., language model priors in spelling applications), accounts for varying brain symbol accuracy and is robust to single brain symbol query errors. This framework is paired with Maximum Mutual Information (MMI) coding which maximizes a generalization of ITR. Both are applicable to any discrete task and brain phenomena (e.g., P300, SSVEP, MI). To demonstrate the efficacy of our approach we perform SSVEP "Shuffle" Speller experiments and compare our recursive coding scheme with traditional decision tree methods including Huffman coding. MMI coding leverages the asymmetry of the classifier's mistakes across a particular user's SSVEP responses; in doing so it offers a 33% increase in letter accuracy though it is 13% slower in our experiment.

Probabilistic Simulation Framework for EEG-Based BCI Design.

A simulation framework could decrease the burden of attending long and tiring experimental sessions on the potential users of brain computer interface (BCI) systems. Specifically during the initial design of a BCI, a simulation framework that could replicate the operational performance of the system would be a useful tool for designers to make design choices. In this manuscript, we develop a Monte Carlo based probabilistic simulation framework for electroencephalography (EEG) based BCI design. We employ one event related potential (ERP) based typing and one steady state evoked potential (SSVEP) based control interface as testbeds. We compare the results of simulations with real time experiments. Even though over and under estimation of the performance is possible, the statistical results over the Monte Carlo simulations show that the developed framework generally provides a good approximation of the real time system performance.

Language-Model Assisted Brain Computer Interface for Typing: A Comparison of Matrix and Rapid Serial Visual Presentation.

Noninvasive electroencephalography (EEG)-based brain-computer interfaces (BCIs) popularly utilize event-related potential (ERP) for intent detection. Specifically, for EEG-based BCI typing systems, different symbol presentation paradigms have been utilized to induce ERPs. In this manuscript, through an experimental study, we assess the speed, recorded signal quality, and system accuracy of a language-model-assisted BCI typing system using three different presentation paradigms: a 4 × 7 matrix paradigm of a 28-character alphabet with row-column presentation (RCP) and single-character presentation (SCP), and rapid serial visual presentation (RSVP) of the same. Our analyses show that signal quality and classification accuracy are comparable between the two visual stimulus presentation paradigms. In addition, we observe that while the matrix-based paradigm can be generally employed with lower inter-trial-interval (ITI) values, the best presentation paradigm and ITI value configuration is user dependent. This potentially warrants offering both presentation paradigms and variable ITI options to users of BCI typing systems.

Noninvasive brain-computer interfaces for augmentative and alternative communication.

Brain-computer interfaces (BCIs) promise to provide a novel access channel for assistive technologies, including augmentative and alternative communication (AAC) systems, to people with severe speech and physical impairments (SSPI). Research on the subject has been accelerating significantly in the last decade and the research community took great strides toward making BCI-AAC a practical reality to individuals with SSPI. Nevertheless, the end goal has still not been reached and there is much work to be done to produce real-world-worthy systems that can be comfortably, conveniently, and reliably used by individuals with SSPI with help from their families and care givers who will need to maintain, setup, and debug the systems at home. This paper reviews reports in the BCI field that aim at AAC as the application domain with a consideration on both technical and clinical aspects.