Brain-computer interfacing under distraction: an evaluation study.

OBJECTIVE: While motor-imagery based brain-computer interfaces (BCIs) have been studied over many years by now, most of these studies have taken place in controlled lab settings. Bringing BCI technology into everyday life is still one of the main challenges in this field of research. APPROACH: This paper systematically investigates BCI performance under 6 types of distractions that mimic out-of-lab environments. MAIN RESULTS: We report results of 16 participants and show that the performance of the standard common spatial patterns (CSP) + regularized linear discriminant analysis classification pipeline drops significantly in this 'simulated' out-of-lab setting. We then investigate three methods for improving the performance: (1) artifact removal, (2) ensemble classification, and (3) a 2-step classification approach. While artifact removal does not enhance the BCI performance significantly, both ensemble classification and the 2-step classification combined with CSP significantly improve the performance compared to the standard procedure. SIGNIFICANCE: Systematically analyzing out-of-lab scenarios is crucial when bringing BCI into everyday life. Algorithms must be adapted to overcome nonstationary environments in order to tackle real-world challenges.

Analyzing neuroimaging data with subclasses: A shrinkage approach.

Among the numerous methods used to analyze neuroimaging data, Linear Discriminant Analysis (LDA) is commonly applied for binary classification problems. LDAs popularity derives from its simplicity and its competitive classification performance, which has been reported for various types of neuroimaging data. Yet the standard LDA approach proves less than optimal for binary classification problems when additional label information (i.e. subclass labels) is present. Subclass labels allow to model structure in the data, which can be used to facilitate the classification task. In this paper, we illustrate how neuroimaging data exhibit subclass labels that may contain valuable information. We also show that the standard LDA classifier is unable to exploit subclass labels. We introduce a novel method that allows subclass labels to be incorporated efficiently into the classifier. The novel method, which we call Relevance Subclass LDA (RSLDA), computes an individual classification hyperplane for each subclass. It is based on regularized estimators of the subclass mean and uses other subclasses as regularization targets. We demonstrate the applicability and performance of our method on data drawn from two different neuroimaging modalities: (I) EEG data from brain-computer interfacing with event-related potentials, and (II) fMRI data in response to different levels of visual motion. We show that RSLDA outperforms the standard LDA approach for both types of datasets. These findings illustrate the benefits of exploiting subclass structure in neuroimaging data. Finally, we show that our classifier also outputs regularization profiles, enabling researchers to interpret the subclass structure in a meaningful way. RSLDA therefore yields increased classification accuracy as well as a better interpretation of neuroimaging data. Since both results are highly favorable, we suggest to apply RSLDA for various classification problems within neuroimaging and beyond.

Solving the EEG inverse problem based on space-time-frequency structured sparsity constraints.

We introduce STOUT (spatio-temporal unifying tomography), a novel method for the source analysis of electroencephalograpic (EEG) recordings, which is based on a physiologically-motivated source representation. Our method assumes that only a small number of brain sources are active throughout a measurement, where each of the sources exhibits focal (smooth but localized) characteristics in space, time and frequency. This structure is enforced through an expansion of the source current density into appropriate spatio-temporal basis functions in combination with sparsity constraints. This approach combines the main strengths of two existing methods, namely Sparse Basis Field Expansions (Haufe et al., 2011) and Time-Frequency Mixed-Norm Estimates (Gramfort et al., 2013). By adjusting the ratio between two regularization terms, STOUT is capable of trading temporal for spatial reconstruction accuracy and vice versa, depending on the requirements of specific analyses and the provided data. Due to allowing for non-stationary source activations, STOUT is particularly suited for the localization of event-related potentials (ERP) and other evoked brain activity. We demonstrate its performance on simulated ERP data for varying signal-to-noise ratios and numbers of active sources. Our analysis of the generators of visual and auditory evoked N200 potentials reveals that the most active sources originate in the temporal and occipital lobes, in line with the literature on sensory processing.

Wyrm: A Brain-Computer Interface Toolbox in Python.

In the last years Python has gained more and more traction in the scientific community. Projects like NumPy, SciPy, and Matplotlib have created a strong foundation for scientific computing in Python and machine learning packages like scikit-learn or packages for data analysis like Pandas are building on top of it. In this paper we present Wyrm ( https://github.com/bbci/wyrm ), an open source BCI toolbox in Python. Wyrm is applicable to a broad range of neuroscientific problems. It can be used as a toolbox for analysis and visualization of neurophysiological data and in real-time settings, like an online BCI application. In order to prevent software defects, Wyrm makes extensive use of unit testing. We will explain the key aspects of Wyrm's software architecture and design decisions for its data structure, and demonstrate and validate the use of our toolbox by presenting our approach to the classification tasks of two different data sets from the BCI Competition III. Furthermore, we will give a brief analysis of the data sets using our toolbox, and demonstrate how we implemented an online experiment using Wyrm. With Wyrm we add the final piece to our ongoing effort to provide a complete, free and open source BCI system in Python.

Impedance based automatic electrode positioning.

The position of electrodes in electrical imaging and stimulation of the human brain is an important variable with vast influences on the precision in modeling approaches. Nevertheless, the exact position is obscured by many factors. 3-D Digitization devices can measure the distribution over the scalp surface but remain uncomfortable in application and often imprecise. We demonstrate a new approach that uses solely the impedance information between the electrodes to determine the geometric position. The algorithm involves multidimensional scaling to create a 3 dimensional space based on these impedances. The success is demonstrated in a simulation study. An average electrode position error of 1.67cm over all 6 subjects could be achieved.

Exploring combinations of auditory and visual stimuli for gaze-independent brain-computer interfaces.

For Brain-Computer Interface (BCI) systems that are designed for users with severe impairments of the oculomotor system, an appropriate mode of presenting stimuli to the user is crucial. To investigate whether multi-sensory integration can be exploited in the gaze-independent event-related potentials (ERP) speller and to enhance BCI performance, we designed a visual-auditory speller. We investigate the possibility to enhance stimulus presentation by combining visual and auditory stimuli within gaze-independent spellers. In this study with N = 15 healthy users, two different ways of combining the two sensory modalities are proposed: simultaneous redundant streams (Combined-Speller) and interleaved independent streams (Parallel-Speller). Unimodal stimuli were applied as control conditions. The workload, ERP components, classification accuracy and resulting spelling speed were analyzed for each condition. The Combined-speller showed a lower workload than uni-modal paradigms, without the sacrifice of spelling performance. Besides, shorter latencies, lower amplitudes, as well as a shift of the temporal and spatial distribution of discriminative information were observed for Combined-speller. These results are important and are inspirations for future studies to search the reason for these differences. For the more innovative and demanding Parallel-Speller, where the auditory and visual domains are independent from each other, a proof of concept was obtained: fifteen users could spell online with a mean accuracy of 87.7% (chance level <3%) showing a competitive average speed of 1.65 symbols per minute. The fact that it requires only one selection period per symbol makes it a good candidate for a fast communication channel. It brings a new insight into the true multisensory stimuli paradigms. Novel approaches for combining two sensory modalities were designed here, which are valuable for the development of ERP-based BCI paradigms.

Motor imagery for severely motor-impaired patients: evidence for brain-computer interfacing as superior control solution.

Brain-Computer Interfaces (BCIs) strive to decode brain signals into control commands for severely handicapped people with no means of muscular control. These potential users of noninvasive BCIs display a large range of physical and mental conditions. Prior studies have shown the general applicability of BCI with patients, with the conflict of either using many training sessions or studying only moderately restricted patients. We present a BCI system designed to establish external control for severely motor-impaired patients within a very short time. Within only six experimental sessions, three out of four patients were able to gain significant control over the BCI, which was based on motor imagery or attempted execution. For the most affected patient, we found evidence that the BCI could outperform the best assistive technology (AT) of the patient in terms of control accuracy, reaction time and information transfer rate. We credit this success to the applied user-centered design approach and to a highly flexible technical setup. State-of-the art machine learning methods allowed the exploitation and combination of multiple relevant features contained in the EEG, which rapidly enabled the patients to gain substantial BCI control. Thus, we could show the feasibility of a flexible and tailorable BCI application in severely disabled users. This can be considered a significant success for two reasons: Firstly, the results were obtained within a short period of time, matching the tight clinical requirements. Secondly, the participating patients showed, compared to most other studies, very severe communication deficits. They were dependent on everyday use of AT and two patients were in a locked-in state. For the most affected patient a reliable communication was rarely possible with existing AT.

Towards user-friendly spelling with an auditory brain-computer interface: the CharStreamer paradigm.

Realizing the decoding of brain signals into control commands, brain-computer interfaces (BCI) aim to establish an alternative communication pathway for locked-in patients. In contrast to most visual BCI approaches which use event-related potentials (ERP) of the electroencephalogram, auditory BCI systems are challenged with ERP responses, which are less class-discriminant between attended and unattended stimuli. Furthermore, these auditory approaches have more complex interfaces which imposes a substantial workload on their users. Aiming for a maximally user-friendly spelling interface, this study introduces a novel auditory paradigm: "CharStreamer". The speller can be used with an instruction as simple as "please attend to what you want to spell". The stimuli of CharStreamer comprise 30 spoken sounds of letters and actions. As each of them is represented by the sound of itself and not by an artificial substitute, it can be selected in a one-step procedure. The mental mapping effort (sound stimuli to actions) is thus minimized. Usability is further accounted for by an alphabetical stimulus presentation: contrary to random presentation orders, the user can foresee the presentation time of the target letter sound. Healthy, normal hearing users (n = 10) of the CharStreamer paradigm displayed ERP responses that systematically differed between target and non-target sounds. Class-discriminant features, however, varied individually from the typical N1-P2 complex and P3 ERP components found in control conditions with random sequences. To fully exploit the sequential presentation structure of CharStreamer, novel data analysis approaches and classification methods were introduced. The results of online spelling tests showed that a competitive spelling speed can be achieved with CharStreamer. With respect to user rating, it clearly outperforms a control setup with random presentation sequences.

SPoC: a novel framework for relating the amplitude of neuronal oscillations to behaviorally relevant parameters.

Previously, modulations in power of neuronal oscillations have been functionally linked to sensory, motor and cognitive operations. Such links are commonly established by relating the power modulations to specific target variables such as reaction times or task ratings. Consequently, the resulting spatio-spectral representation is subjected to neurophysiological interpretation. As an alternative, independent component analysis (ICA) or alternative decomposition methods can be applied and the power of the components may be related to the target variable. In this paper we show that these standard approaches are suboptimal as the first does not take into account the superposition of many sources due to volume conduction, while the second is unable to exploit available information about the target variable. To improve upon these approaches we introduce a novel (supervised) source separation framework called Source Power Comodulation (SPoC). SPoC makes use of the target variable in the decomposition process in order to give preference to components whose power comodulates with the target variable. We present two algorithms that implement the SPoC approach. Using simulations with a realistic head model, we show that the SPoC algorithms are able extract neuronal components exhibiting high correlation of power with the target variable. In this task, the SPoC algorithms outperform other commonly used techniques that are based on the sensor data or ICA approaches. Furthermore, using real electroencephalography (EEG) recordings during an auditory steady state paradigm, we demonstrate the utility of the SPoC algorithms by extracting neuronal components exhibiting high correlation of power with the intensity of the auditory input. Taking into account the results of the simulations and real EEG recordings, we conclude that SPoC represents an adequate approach for the optimal extraction of neuronal components showing coupling of power with continuously changing behaviorally relevant parameters.

Brain-computer interface controlled gaming: evaluation of usability by severely motor restricted end-users.

OBJECTIVE: Connect-Four, a new sensorimotor rhythm (SMR) based brain-computer interface (BCI) gaming application, was evaluated by four severely motor restricted end-users; two were in the locked-in state and had unreliable eye-movement. METHODS: Following the user-centred approach, usability of the BCI prototype was evaluated in terms of effectiveness (accuracy), efficiency (information transfer rate (ITR) and subjective workload) and users' satisfaction. RESULTS: Online performance varied strongly across users and sessions (median accuracy (%) of end-users: A=.65; B=.60; C=.47; D=.77). Our results thus yielded low to medium effectiveness in three end-users and high effectiveness in one end-user. Consequently, ITR was low (0.05-1.44bits/min). Only two end-users were able to play the game in free-mode. Total workload was moderate but varied strongly across sessions. Main sources of workload were mental and temporal demand. Furthermore, frustration contributed to the subjective workload of two end-users. Nevertheless, most end-users accepted the BCI application well and rated satisfaction medium to high. Sources for dissatisfaction were (1) electrode gel and cap, (2) low effectiveness, (3) time-consuming adjustment and (4) not easy-to-use BCI equipment. All four end-users indicated ease of use as being one of the most important aspect of BCI. CONCLUSION: Effectiveness and efficiency are lower as compared to applications using the event-related potential as input channel. Nevertheless, the SMR-BCI application was satisfactorily accepted by the end-users and two of four could imagine using the BCI application in their daily life. Thus, despite moderate effectiveness and efficiency BCIs might be an option when controlling an application for entertainment.

Optimizing event-related potential based brain-computer interfaces: a systematic evaluation of dynamic stopping methods.

OBJECTIVE: In brain-computer interface (BCI) research, systems based on event-related potentials (ERP) are considered particularly successful and robust. This stems in part from the repeated stimulation which counteracts the low signal-to-noise ratio in electroencephalograms. Repeated stimulation leads to an optimization problem, as more repetitions also cost more time. The optimal number of repetitions thus represents a data-dependent trade-off between the stimulation time and the obtained accuracy. Several methods for dealing with this have been proposed as 'early stopping', 'dynamic stopping' or 'adaptive stimulation'. Despite their high potential for BCI systems at the patient's bedside, those methods are typically ignored in current BCI literature. The goal of the current study is to assess the benefit of these methods. APPROACH: This study assesses for the first time the existing methods on a common benchmark of both artificially generated data and real BCI data of 83 BCI sessions, allowing for a direct comparison between these methods in the context of text entry. MAIN RESULTS: The results clearly show the beneficial effect on the online performance of a BCI system, if the trade-off between the number of stimulus repetitions and accuracy is optimized. All assessed methods work very well for data of good subjects, and worse for data of low-performing subjects. Most methods, however, are robust in the sense that they do not reduce the performance below the baseline of a simple no stopping strategy. SIGNIFICANCE: Since all methods can be realized as a module between the BCI and an application, minimal changes are needed to include these methods into existing BCI software architectures. Furthermore, the hyperparameters of most methods depend to a large extend on only a single variable-the discriminability of the training data. For the convenience of BCI practitioners, the present study proposes linear regression coefficients for directly estimating the hyperparameters from the data based on this discriminability. The data that were used in this publication are made publicly available to benchmark future methods.

Natural stimuli improve auditory BCIs with respect to ergonomics and performance.

Moving from well-controlled, brisk artificial stimuli to natural and less-controlled stimuli seems counter-intuitive for event-related potential (ERP) studies. As natural stimuli typically contain a richer internal structure, they might introduce higher levels of variance and jitter in the ERP responses. Both characteristics are unfavorable for a good single-trial classification of ERPs in the context of a multi-class brain-computer interface (BCI) system, where the class-discriminant information between target stimuli and non-target stimuli must be maximized. For the application in an auditory BCI system, however, the transition from simple artificial tones to natural syllables can be useful despite the variance introduced. In the presented study, healthy users (N = 9) participated in an offline auditory nine-class BCI experiment with artificial and natural stimuli. It is shown that the use of syllables as natural stimuli does not only improve the users' ergonomic ratings; also the classification performance is increased. Moreover, natural stimuli obtain a better balance in multi-class decisions, such that the number of systematic confusions between the nine classes is reduced. Hopefully, our findings may contribute to make auditory BCI paradigms more user friendly and applicable for patients.

How stimulation speed affects Event-Related Potentials and BCI performance.

In most paradigms for Brain-Computer Interfaces (BCIs) that are based on Event-Related Potentials (ERPs), stimuli are presented with a pre-defined and constant speed. In order to boost BCI performance by optimizing the parameters of stimulation, this offline study investigates the impact of the stimulus onset asynchrony (SOA) on ERPs and the resulting classification accuracy. The SOA is defined as the time between the onsets of two consecutive stimuli, which represents a measure for stimulation speed. A simple auditory oddball paradigm was tested in 14 SOA conditions with a SOA between 50 ms and 1000 ms. Based on an offline ERP analysis, the BCI performance (quantified by the Information Transfer Rate, ITR in bits/min) was simulated. A great variability in the simulated BCI performance was observed within subjects (N=11). This indicates a potential increase in BCI performance (≥ 1.6 bits/min) for ERP-based paradigms, if the stimulation speed is specified for each user individually.

No surprise--sequence event-related potentials for brain-computer interfaces.

INTRODUCTION: In the field of Brain-Computer Interfaces (BCI), the original two-class oddball paradigm has been extended to multiple stimuli with balanced probabilities and random presentation sequences. Exploiting the differences between standard and deviant ERP responses, these multi-class paradigms are suitable for communication and control. METHODS: The present study investigates the effect of giving up the randomness of stimulation sequences in favor of a repeated, predictable pattern. Data of healthy subjects (n=10) who performed a single session with a 6-class spatial auditory ERP paradigm were analyzed offline. Their auditory evoked potentials (AEP) resulting from the potentially simpler task (using fixed sequences) are compared with the AEP evoked by pseudo-randomized stimulation sequences. RESULTS: Class-discriminative EEG responses between target and non-target stimuli were observed for both conditions. The binary classification error estimated for standard epochs of was comparable for both conditions (random: 24%, fixed: 25%). Expanding the standard epochs to include pre-stimulus intervals, we found that the regular structure of the fixed sequence can be exploited. Compared to the standard epoch, the MSE improves by 7%, while in the random condition an improvement could not be observed.

A Novel 9-Class Auditory ERP Paradigm Driving a Predictive Text Entry System.

Brain-computer interfaces (BCIs) based on event related potentials (ERPs) strive for offering communication pathways which are independent of muscle activity. While most visual ERP-based BCI paradigms require good control of the user's gaze direction, auditory BCI paradigms overcome this restriction. The present work proposes a novel approach using auditory evoked potentials for the example of a multiclass text spelling application. To control the ERP speller, BCI users focus their attention to two-dimensional auditory stimuli that vary in both, pitch (high/medium/low) and direction (left/middle/right) and that are presented via headphones. The resulting nine different control signals are exploited to drive a predictive text entry system. It enables the user to spell a letter by a single nine-class decision plus two additional decisions to confirm a spelled word. This paradigm - called PASS2D - was investigated in an online study with 12 healthy participants. Users spelled with more than 0.8 characters per minute on average (3.4 bits/min) which makes PASS2D a competitive method. It could enrich the toolbox of existing ERP paradigms for BCI end users like people with amyotrophic lateral sclerosis disease in a late stage.

Performance optimization of ERP-based BCIs using dynamic stopping.

Brain-computer interfaces based on event-related potentials face a trade-off between the speed and accuracy of the system, as both depend on the number of iterations. Increasing the number of iterations leads to a higher accuracy but reduces the speed of the system. This trade-off is generally dealt with by finding a fixed number of iterations that give a good result on the calibration data. We show here that this method is sub optimal and increases the performance significantly in only one out of five datasets. Several alternative methods have been described in literature, and we test the generalization of four of them. One method, called rank diff, significantly increased the performance over all datasets. These findings are important, as they show that 1) one should be cautious when reporting the potential performance of a BCI based on post-hoc offline performance curves and 2) simple methods are available that do boost performance.

Two-dimensional auditory p300 speller with predictive text system.

P300-based Brain Computer Interfaces offer communication pathways which are independent of muscle activity. Mostly visual stimuli, e.g. blinking of different letters are used as a paradigm of interaction. Neural degenerative diseases like amyotrophic lateral sclerosis (ALS) also cause a decrease in sight, but the ability of hearing is usually unaffected. Therefore, the use of the auditory modality might be preferable. This work presents a multiclass BCI paradigm using two-dimensional auditory stimuli: cues are varying in pitch (high/medium/low) and location (left/middle/right). The resulting nine different classes are embedded in a predictive text system, enabling to spell a letter with a 9-class decision. Moreover, an unbalanced subtrial selection is investigated and compared to the well-established sequence-wise paradigm. Twelve healthy subjects participated in an online study to investigate these approaches.

Assessment of transmission distortion on chromosome 6p in healthy individuals using tagSNPs.

The best-documented example for transmission distortion (TD) to normal offspring are the t haplotypes on mouse chromosome 17. In healthy humans, TD has been described for whole chromosomes and for particular loci, but multiple comparisons have presented a statistical obstacle in wide-ranging analyses. Here we provide six high-resolution TD maps of the short arm of human chromosome 6 (Hsa6p), based on single-nucleotide polymorphism (SNP) data from 60 trio families belonging to two ethnicities that are available through the International HapMap Project. We tested all approximately 70,000 previously genotyped SNPs within Hsa6p by the transmission disequilibrium test. TagSNP selection followed by permutation testing was performed to adjust for multiple testing. A statistically significant evidence for TD was observed among male parents of European ancestry, due to strong and wide-ranging skewed segregation in a 730 kb long region containing the transcription factor-encoding genes SUPT3H and RUNX2, as well as the microRNA locus MIRN586. We also observed that this chromosomal segment coincides with pronounced linkage disequilibrium (LD), suggesting a relationship between TD and LD. The fact that TD may be taking place in samples not selected for a genetic disease implies that linkage studies must be assessed with particular caution in chromosomal segments with evidence of TD.