SSVEP BCI and Eye Tracking Use by Individuals With Late-Stage ALS and Visual Impairments.

Access to communication is critical for individuals with late-stage amyotrophic lateral sclerosis (ALS) and minimal volitional movement, but they sometimes present with concomitant visual or ocular motility impairments that affect their performance with eye tracking or visual brain-computer interface (BCI) systems. In this study, we explored the use of modified eye tracking and steady state visual evoked potential (SSVEP) BCI, in combination with the Shuffle Speller typing interface, for this population. Two participants with late-stage ALS, visual impairments, and minimal volitional movement completed a single-case experimental research design comparing copy-spelling performance with three different typing systems: (1) commercially available eye tracking communication software, (2) Shuffle Speller with modified eye tracking, and (3) Shuffle Speller with SSVEP BCI. Participant 1 was unable to type any correct characters with the commercial system, but achieved accuracies of up to 50% with Shuffle Speller eye tracking and 89% with Shuffle Speller BCI. Participant 2 also had higher maximum accuracies with Shuffle Speller, typing with up to 63% accuracy with eye tracking and 100% accuracy with BCI. However, participants' typing accuracy for both Shuffle Speller conditions was highly variable, particularly in the BCI condition. Both the Shuffle Speller interface and SSVEP BCI input show promise for improving typing performance for people with late-stage ALS. Further development of innovative BCI systems for this population is needed.

An experimental and computational framework for modeling multi-muscle responses to transcranial magnetic stimulation of the human motor cortex.

Current knowledge of coordinated motor control of multiple muscles is derived primarily from invasive stimulation-recording techniques in animal models. Similar studies are not generally feasible in humans, so a modeling framework is needed to facilitate knowledge transfer from animal studies. We describe such a framework that uses a deep neural network model to map finite element simulation of transcranial magnetic stimulation induced electric fields (E-fields) in motor cortex to recordings of multi-muscle activation. Critically, we show that model generalization is improved when we incorporate empirically derived physiological models for E-field to neuron firing rate and low-dimensional control via muscle synergies.

Hierarchical Graphical Models for Context-Aware Hybrid Brain-Machine Interfaces.

We present a novel hierarchical graphical model based context-aware hybrid brain-machine interface (hBMI) using probabilistic fusion of electroencephalographic (EEG) and electromyographic (EMG) activities. Based on experimental data collected during stationary executions and subsequent imageries of five different hand gestures with both limbs, we demonstrate feasibility of the proposed hBMI system through within session and online across sessions classification analyses. Furthermore, we investigate the context-aware extent of the model by a simulated probabilistic approach and highlight potential implications of our work in the field of neurophysiologically-driven robotic hand prosthetics.

Effects of simulated visual acuity and ocular motility impairments on SSVEP brain-computer interface performance: An experiment with Shuffle Speller.

Individuals with severe speech and physical impairments may have concomitant visual acuity impairments (VAI) or ocular motility impairments (OMI) impacting visual BCI use. We report on the use of the Shuffle Speller typing interface for an SSVEP BCI copy-spelling task under three conditions: simulated VAI, simulated OMI, and unimpaired vision. To mitigate the effect of visual impairments, we introduce a method that adaptively selects a user-specific trial length to maximize expected information transfer rate (ITR); expected ITR is shown to closely approximate the rate of correct letter selections. All participants could type under the unimpaired and simulated VAI conditions, with no significant differences in typing accuracy or speed. Most participants (31 of 37) could not type under the simulated OMI condition; some achieved high accuracy but with slower typing speeds. Reported workload and discomfort were low, and satisfaction high, under the unimpaired and simulated VAI conditions. Implications and future directions to examine effect of visual impairment on BCI use is discussed.

INCORPORATING TEMPORAL DEPENDENCY ON ERP BASED BCI.

In brain computer interface (BCI) systems based on event related potentials (ERPs), a windowed electroencephalography (EEG) signal is taken into consideration for the assumed duration of the ERP potential. In BCI applications inter stimuli interval is shorter than the ERP duration. This causes temporal dependencies over observation potentials thus disallows taking the data into consideration independently. However, conventionally the data is assumed to be independent for decreasing complexity. In this paper we propose a graphical model which covers the temporal dependency into consideration by labeling each time sample. We also propose a formulation to exploit the time series structure of the EEG.

IMPROVED CLASSIFICATION IN TACTILE BCIS USING A NOISY LABEL MODEL.

Tactile BCIs have gained recent popularity in the BCI community due to the advantages of using a stimulation medium which does not inhibit the users visual or auditory senses, is naturally inconspicuous, and can still be used by a person who may be visually or auditorily impaired. While many systems have been proposed which utilize the P300 response elicited through an oddball task, these systems struggle to classify user responses with accuracies comparable to many visual stimulus based systems. In this study, we model the tactile ERP generation as label noise and develop a novel BCI paradigm for binary communication designed to minimize label confusion. The classification model is based on a modified Gaussian mixture and trained using expectation maximization (EM). Finally, we show after testing on multiple subjects that this approach yields cross-validated accuracies for all users which are significantly above chance and suggests that such an approach is robust and reliable for a variety of binary communication-based applications.

Decoding complex imagery hand gestures.

Brain computer interfaces (BCIs) offer individuals suffering from major disabilities an alternative method to interact with their environment. Sensorimotor rhythm (SMRs) based BCIs can successfully perform control tasks; however, the traditional SMR paradigms intuitively disconnect the control and real task, making them non-ideal for complex control scenarios. In this study we design a new, intuitively connected motor imagery (MI) paradigm using hierarchical common spatial patterns (HCSP) and context information to effectively predict intended hand grasps from electroencephalogram (EEG) data. Experiments with 5 participants yielded an aggregate classification accuracy-intended grasp prediction probability-of 64.5% for 8 different hand gestures, more than 5 times the chance level.

INFORMATION THEORETIC FEATURE PROJECTION FOR SINGLE-TRIAL BRAIN-COMPUTER INTERFACES.

Current approaches on optimal spatio-spectral feature extraction for single-trial BCIs exploit mutual information based feature ranking and selection algorithms. In order to overcome potential confounders underlying feature selection by information theoretic criteria, we propose a non-parametric feature projection framework for dimensionality reduction that utilizes mutual information based stochastic gradient descent. We demonstrate the feasibility of the protocol based on analyses of EEG data collected during execution of open and close palm hand gestures. We further discuss the approach in terms of potential insights in the context of neurophysiologically driven prosthetic hand control.

Muscle Synergy-based Grasp Classification for Robotic Hand Prosthetics.

The main goal of this study is analyzing whether muscle synergies based on surface electromyography (EMG) measurements could be used for hand posture classification in the context of robotic prosthetic control. Target grasps were selected according to usefulness in daily activities. Additionally, due to the feasibility constraints of robotic prosthetics, only 14 gestures (13 feasible grasps and 1 resting state) were analyzed. EMG signals of intact-limb subjects were decomposed into base and activation components for muscle activity evaluation. The results demonstrate that features based on muscle synergies derived from non-negative matrix factorization (NMF) outperform the ones derived from principal component analysis (PCA). Moreover, we also examine the robustness of these methods in the absence of electrodes (muscle importance) and show that NMF is able to provide sufficiently accurate results.

Recursive Bayesian Coding for BCIs.

Brain-Computer Interfaces (BCIs) seek to infer some task symbol, a task relevant instruction, from brain symbols, classifiable physiological states. For example, in a motor imagery robot control task a user would indicate their choice from a dictionary of task symbols (rotate arm left, grasp, etc.) by selecting from a smaller dictionary of brain symbols (imagined left or right hand movements). We examine how a BCI infers a task symbol using selections of brain symbols. We offer a recursive Bayesian decision framework which incorporates context prior distributions (e.g., language model priors in spelling applications), accounts for varying brain symbol accuracy and is robust to single brain symbol query errors. This framework is paired with Maximum Mutual Information (MMI) coding which maximizes a generalization of ITR. Both are applicable to any discrete task and brain phenomena (e.g., P300, SSVEP, MI). To demonstrate the efficacy of our approach we perform SSVEP "Shuffle" Speller experiments and compare our recursive coding scheme with traditional decision tree methods including Huffman coding. MMI coding leverages the asymmetry of the classifier's mistakes across a particular user's SSVEP responses; in doing so it offers a 33% increase in letter accuracy though it is 13% slower in our experiment.