From full calibration to zero training for a code-modulated visual evoked potentials for brain-computer interface.

Objective.Typically, a brain-computer interface (BCI) is calibrated using user- and session-specific data because of the individual idiosyncrasies and the non-stationary signal properties of the electroencephalogram (EEG). Therefore, it is normal for BCIs to undergo a time-consuming passive training stage that prevents users from directly operating them. In this study, we systematically reduce the training data set in a stepwise fashion, to ultimately arrive at a calibration-free method for a code-modulated visually evoked potential (cVEP)-based BCI to fully eliminate the tedious training stage.Approach.In an extensive offline analysis, we compare our sophisticated encoding model with a traditional event-related potential (ERP) technique. We calibrate the encoding model in a standard way, with data limited to a single class while generalizing to all others and without any data. In addition, we investigate the feasibility of the zero-training cVEP BCI in an online setting.Main results.By adopting the encoding model, the training data can be reduced substantially, while maintaining both the classification performance as well as the explained variance of the ERP method. Moreover, with data from only one class or even no data at all, it still shows excellent performance. In addition, the zero-training cVEP BCI achieved high communication rates in an online spelling task, proving its feasibility for practical use.Significance.To date, this is the fastest zero-training cVEP BCI in the field, allowing high communication speeds without calibration while using only a few non-invasive water-based EEG electrodes. This allows us to skip the training stage altogether and spend all the valuable time on direct operation. This minimizes the session time and opens up new exciting directions for practical plug-and-play BCI. Fundamentally, these results validate that the adopted neural encoding model compresses data into event responses without the loss of explanatory power compared to using full ERPs as a template.

The N400 for brain computer interfacing: complexities and opportunities.

The N400 is an event related potential that is evoked in response to conceptually meaningful stimuli. It is for instance more negative in response to incongruent than congruent words in a sentence, and more negative for unrelated than related words following a prime word. This sensitivity to semantic content of a stimulus in relation to the mental context of an individual makes it a signal of interest for Brain Computer Interfaces. A complicating aspect is the number of factors that can affect the N400 amplitude. In this paper, we provide an accessible overview of this range of N400 effects, and survey the three main BCI application areas that currently exploit the N400: (1) exploiting the semantic processing of faces to enhance matrix speller performance, (2) detecting language processing in patients with Disorders of Consciousness, and (3) using semantic stimuli to probe what is on a user's mind. Drawing on studies from these application areas, we illustrate that the N400 can successfully be exploited for BCI purposes, but that the signal-to-noise ratio is a limiting factor, with signal strength also varying strongly across subjects. Furthermore, we put findings in context of the general N400 literature, noting open questions and identifying opportunities for further research.

Low channel count montages using sensor tying for VEP-based BCI.

OBJECTIVE: Brain computer interfaces (BCIs) are slowly making their appearance on the consumer market, accompanied by a higher popularity among the general public. This new group of users requires easy-to-use headsets with robustness to non-precise placement. In this paper, an optimized fixed montage EEG headset for VEP BCIs is proposed. APPROACH: The proposed layout covers only the most relevant area with large sensors to account for slight misplacement. To obtain large sensors, without having them physically available, we tie multiple sensors together and simulate the effect by averaging the signal of multiple sensors. MAIN RESULTS: In simulations based on recorded 256-channel EEG data, it is shown that a circular center-surround configuration with sensor tying, leading to only eight channels covering a large part of the occipital lobe, can provide high performance and good robustness to misplacement. Automatically optimized layouts were unable to achieve better performance, demonstrating the utility of this manual design. Finally, the performance and benefits of sensor tying in the manual design are then validated in a physical experiment. SIGNIFICANCE: The resulting proposed layout fulfills most requirements of an easy to use consumer EEG headset.

Comparing tactile and visual gaze-independent brain-computer interfaces in patients with amyotrophic lateral sclerosis and healthy users.

OBJECTIVE: Brain-computer interfaces (BCI) tested in patients often are gaze-dependent, while these intended users could possibly lose the ability to focus their gaze. Therefore, a visual and a tactile gaze-independent spelling system were investigated. METHODS: Five patients with amyotrophic lateral sclerosis (ALS) tested a visual Hex-o-Spell and a tactile speller. Six healthy participants were also included, mainly to evaluate the tactile stimulators. RESULTS: A significant attentional modulation was seen in the P300 for the Hex-o-Spell and in the N2 for the tactile speller. Average on-line classification performance for selecting a step in the speller was above chance level (17%) for both spellers. However, average performance was higher for the Hex-o-Spell (88% and 85% for healthy participants and patients, respectively) than for the tactile speller (56% and 53%, respectively). Likewise, bitrates were higher for the Hex-o-Spell compared with the tactile speller, and in the subjective usability a preference for the Hex-o-Spell was found. CONCLUSIONS: The Hex-o-Spell outperformed the tactile speller in classification performance, bit rate and subjective usability. SIGNIFICANCE: This is the first study showing the possible use of tactile and visual gaze-independent BCI spelling systems by ALS patients with mild to moderate disabilities.

Feasibility of measuring event related desynchronization with electroencephalography during walking.

Brain Computer Interfaces could be useful in rehabilitation of movement, perhaps also for gait. Until recently, research on movement related brain signals has not included measuring electroencephalography (EEG) during walking, because of the potential artifacts. We investigated if it is possible to measure the event Related Desynchronization (ERD) and event related spectral perturbations (ERSP) during walking. Six subjects walked on a treadmill with a slow speed, while EEG, electromyography (EMG) of the neck muscles and step cycle were measured. A Canonical Correlation Analysis (CCA) was used to remove EMG artifacts from the EEG signals. It was shown that this method correctly deleted EMG components. A strong ERD in the mu band and a somewhat less strong ERD in the beta band were found during walking compared to a baseline period. Furthermore, lateralized ERSPs were found, depending on the phase in the step cycle. It is concluded that this is a promising method to use in BCI research on walking. These results therefore pave the way for using brain signals related to walking in a BCI context.

Sequenced subjective accents for brain-computer interfaces.

Subjective accenting is a cognitive process in which identical auditory pulses at an isochronous rate turn into the percept of an accenting pattern. This process can be voluntarily controlled, making it a candidate for communication from human user to machine in a brain-computer interface (BCI) system. In this study we investigated whether subjective accenting is a feasible paradigm for BCI and how its time-structured nature can be exploited for optimal decoding from non-invasive EEG data. Ten subjects perceived and imagined different metric patterns (two-, three- and four-beat) superimposed on a steady metronome. With an offline classification paradigm, we classified imagined accented from non-accented beats on a single trial (0.5 s) level with an average accuracy of 60.4% over all subjects. We show that decoding of imagined accents is also possible with a classifier trained on perception data. Cyclic patterns of accents and non-accents were successfully decoded with a sequence classification algorithm. Classification performances were compared by means of bit rate. Performance in the best scenario translates into an average bit rate of 4.4 bits min(-1) over subjects, which makes subjective accenting a promising paradigm for an online auditory BCI.

Shared mechanisms in perception and imagery of auditory accents.

OBJECTIVE: An auditory rhythm can be perceived as a sequence of accented (loud) and non-accented (soft) beats or it can be imagined. Subjective rhythmization refers to the induction of accenting patterns during the presentation of identical auditory pulses at an isochronous rate. It can be an automatic process, but it can also be voluntarily controlled. We investigated whether imagined accents can be decoded from brain signals on a single-trial basis, and if there is information shared between perception and imagery in the contrast of accents and non-accents. METHODS: Ten subjects perceived and imagined three different metric patterns (two-, three-, and four-beat) superimposed on a steady metronome while electroencephalography (EEG) measurements were made. Shared information between perception and imagery EEG is investigated by means of principal component analysis and by means of single-trial classification. RESULTS: Classification of accented from non-accented beats was possible with an average accuracy of 70% for perception and 61% for imagery data. Cross-condition classification yielded significant performance above chance level for a classifier trained on perception and tested on imagery data (up to 66%), and vice versa (up to 60%). CONCLUSIONS: Results show that detection of imagined accents is possible and reveal similarity in brain signatures relevant to distinction of accents from non-accents in perception and imagery. SIGNIFICANCE: Our results support the idea of shared mechanisms in perception and imagery for auditory processing. This is relevant for a number of clinical settings, most notably by elucidating the basic mechanisms of rhythmic auditory cuing paradigms, e.g. as used in motor rehabilitation or therapy for Parkinson's disease. As a novel Brain-Computer Interface (BCI) paradigm, our results imply a reduction of the necessary BCI training in healthy subjects and in patients.

P300 audio-visual speller.

The Farwell and Donchin matrix speller is well known as one of the highest performing brain-computer interfaces (BCIs) currently available. However, its use of visual stimulation limits its applicability to users with normal eyesight. Alternative BCI spelling systems which rely on non-visual stimulation, e.g. auditory or tactile, tend to perform much more poorly and/or can be very difficult to use. In this paper we present a novel extension of the matrix speller, based on flipping the letter matrix, which allows us to use the same interface for visual, auditory or simultaneous visual and auditory stimuli. In this way we aim to allow users to utilize the best available input modality for their situation, that is use visual + auditory for best performance and move smoothly to purely auditory when necessary, e.g. when disease causes the user's eyesight to deteriorate. Our results on seven healthy subjects demonstrate the effectiveness of this approach, with our modified visual + auditory stimulation slightly out-performing the classic matrix speller. The purely auditory system performance was lower than for visual stimulation, but comparable to other auditory BCI systems.

Transient and steady-state responses to mechanical stimulation of different fingers reveal interactions based on lateral inhibition.

OBJECTIVE: Simultaneous tactile finger stimulation evokes transient ERP responses that are smaller than the linear summation of ERP responses to individual stimulation. Occlusion and lateral inhibition are two possible mechanisms responsible for this effect. The present study disentangles these two effects using steady-state somatosensory evoked potentials (SSSEP). Simultaneous stimulation on adjacent and distant finger pairs with the same and different stimulation frequencies are compared. METHODS: The index finger (IF), middle finger (MF) and little finger (LF) were mechanically stimulated with a frequency of 18, 22 or 26Hz, respectively. Stimulation was applied for each finger separately, and for the IF (18Hz) in combination with either the MF or LF for 22 and 26Hz, respectively. A measure for interaction (IR) was calculated for the P60 component and the SSSEP amplitude. RESULTS: Significant interactions were found in both the P60 response and in the SSSEP response. Stimulation of adjacent finger combinations caused more interaction than distant finger combinations. No difference was found between stimulation of two fingers with the same or a different frequency. CONCLUSIONS: Our results indicate that lateral inhibition is mainly responsible for the interaction effect. SIGNIFICANCE: These observations provide further insight in the mechanisms behind interaction between somatosensory inputs.

Data processing in music performance research: using structural information to improve score-performance matching.

In order to study aspects of music performance, one has to find correspondences between the performance data and a score. Locating the corresponding score note for every performance note, called matching, is therefore a common task. An algorithm that automates this procedure is called a matcher. Automated matching is difficult because performers make errors, performers use expressive timing, and scores are frequently underspecified. To find the best match, most matchers use information about pitch, temporal order, and the number of matched notes. We show that adding information about the musical structure of the score gives better results. However, we found that even this information was insufficient to identify some types of performance errors and that a definition of best match based only on the number of matched notes is sometimes problematic. We provide some suggestions about how to achieve greater improvements.