Evaluation of Different Types of Stimuli in an Event-Related Potential-Based Brain-Computer Interface Speller under Rapid Serial Visual Presentation.

Rapid serial visual presentation (RSVP) is currently a suitable gaze-independent paradigm for controlling visual brain-computer interfaces (BCIs) based on event-related potentials (ERPs), especially for users with limited eye movement control. However, unlike gaze-dependent paradigms, gaze-independent ones have received less attention concerning the specific choice of visual stimuli that are used. In gaze-dependent BCIs, images of faces-particularly those tinted red-have been shown to be effective stimuli. This study aims to evaluate whether the colour of faces used as visual stimuli influences ERP-BCI performance under RSVP. Fifteen participants tested four conditions that varied only in the visual stimulus used: grey letters (GL), red famous faces with letters (RFF), green famous faces with letters (GFF), and blue famous faces with letters (BFF). The results indicated significant accuracy differences only between the GL and GFF conditions, unlike prior gaze-dependent studies. Additionally, GL achieved higher comfort ratings compared with other face-related conditions. This study highlights that the choice of stimulus type impacts both performance and user comfort, suggesting implications for future ERP-BCI designs for users requiring gaze-independent systems.

Influence of spatial frequency in visual stimuli for cVEP-based BCIs: evaluation of performance and user experience.

Code-modulated visual evoked potentials (c-VEPs) are an innovative control signal utilized in brain-computer interfaces (BCIs) with promising performance. Prior studies on steady-state visual evoked potentials (SSVEPs) have indicated that the spatial frequency of checkerboard-like stimuli influences both performance and user experience. Spatial frequency refers to the dimensions of the individual squares comprising the visual stimulus, quantified in cycles (i.e., number of black-white squares pairs) per degree of visual angle. However, the specific effects of this parameter on c-VEP-based BCIs remain unexplored. Therefore, the objective of this study is to investigate the role of spatial frequency of checkerboard-like visual stimuli in a c-VEP-based BCI. Sixteen participants evaluated selection matrices with eight spatial frequencies: C001 (0 c/°, 1×1 squares), C002 (0.15 c/°, 2×2 squares), C004 (0.3 c/°, 4×4 squares), C008 (0.6 c/°, 8×8 squares), C016 (1.2 c/°, 16×16 squares), C032 (2.4 c/°, 32×32 squares), C064 (4.79 c/°, 64×64 squares), and C128 (9.58 c/°, 128×128 squares). These conditions were tested in an online spelling task, which consisted of 18 trials each conducted on a 3×3 command interface. In addition to accuracy and information transfer rate (ITR), subjective measures regarding comfort, ocular irritation, and satisfaction were collected. Significant differences in performance and comfort were observed based on different stimulus spatial frequencies. Although all conditions achieved mean accuracy over 95% after 2.1 s of trial duration, C016 stood out in terms user experience. The proposed condition not only achieved a mean accuracy of 96.53% and 164.54 bits/min with a trial duration of 1.05s, but also was reported to be significantly more comfortable than the traditional C001 stimulus. Since both features are key for BCI development, higher spatial frequencies than the classical black-to-white stimulus might be more adequate for c-VEP systems. Hence, we assert that the spatial frequency should be carefully considered in the development of future applications for c-VEP-based BCIs.

Evaluation of Single-Trial Classification to Control a Visual ERP-BCI under a Situation Awareness Scenario.

An event-related potential (ERP)-based brain-computer interface (BCI) can be used to monitor a user's cognitive state during a surveillance task in a situational awareness context. The present study explores the use of an ERP-BCI for detecting new planes in an air traffic controller (ATC). Two experiments were conducted to evaluate the impact of different visual factors on target detection. Experiment 1 validated the type of stimulus used and the effect of not knowing its appearance location in an ERP-BCI scenario. Experiment 2 evaluated the effect of the size of the target stimulus appearance area and the stimulus salience in an ATC scenario. The main results demonstrate that the size of the plane appearance area had a negative impact on the detection performance and on the amplitude of the P300 component. Future studies should address this issue to improve the performance of an ATC in stimulus detection using an ERP-BCI.

Comparison of Two Paradigms Based on Stimulation with Images in a Spelling Brain-Computer Interface.

A P300-based speller can be used to control a home automation system via brain activity. Evaluation of the visual stimuli used in a P300-based speller is a common topic in the field of brain-computer interfaces (BCIs). The aim of the present work is to compare, using the usability approach, two types of stimuli that have provided high performance in previous studies. Twelve participants controlled a BCI under two conditions, which varied in terms of the type of stimulus employed: a red famous face surrounded by a white rectangle (RFW) and a range of neutral pictures (NPs). The usability approach included variables related to effectiveness (accuracy and information transfer rate), efficiency (stress and fatigue), and satisfaction (pleasantness and System Usability Scale and Affect Grid questionnaires). The results indicated that there were no significant differences in effectiveness, but the system that used NPs was reported as significantly more pleasant. Hence, since satisfaction variables should also be considered in systems that potential users are likely to employ regularly, the use of different NPs may be a more suitable option than the use of a single RFW for the development of a home automation system based on a visual P300-based speller.

Effect of Stimulus Size in a Visual ERP-Based BCI under RSVP.

Rapid serial visual presentation (RSVP) is currently one of the most suitable paradigms for use with a visual brain-computer interface based on event-related potentials (ERP-BCI) by patients with a lack of ocular motility. However, gaze-independent paradigms have not been studied as closely as gaze-dependent ones, and variables such as the sizes of the stimuli presented have not yet been explored under RSVP. Hence, the aim of the present work is to assess whether stimulus size has an impact on ERP-BCI performance under the RSVP paradigm. Twelve participants tested the ERP-BCI under RSVP using three different stimulus sizes: small (0.1 × 0.1 cm), medium (1.9 × 1.8 cm), and large (20.05 × 19.9 cm) at 60 cm. The results showed significant differences in accuracy between the conditions; the larger the stimulus, the better the accuracy obtained. It was also shown that these differences were not due to incorrect perception of the stimuli since there was no effect from the size in a perceptual discrimination task. The present work therefore shows that stimulus size has an impact on the performance of an ERP-BCI under RSVP. This finding should be considered by future ERP-BCI proposals aimed at users who need gaze-independent systems.

Brain-Computer Interface (BCI) Control of a Virtual Assistant in a Smartphone to Manage Messaging Applications.

Brain-computer interfaces (BCI) are a type of assistive technology that uses the brain signals of users to establish a communication and control channel between them and an external device. BCI systems may be a suitable tool to restore communication skills in severely motor-disabled patients, as BCI do not rely on muscular control. The loss of communication is one of the most negative consequences reported by such patients. This paper presents a BCI system focused on the control of four mainstream messaging applications running in a smartphone: WhatsApp, Telegram, e-mail and short message service (SMS). The control of the BCI is achieved through the well-known visual P300 row-column paradigm (RCP), allowing the user to select control commands as well as spelling characters. For the control of the smartphone, the system sends synthesized voice commands that are interpreted by a virtual assistant running in the smartphone. Four tasks related to the four mentioned messaging services were tested with 15 healthy volunteers, most of whom were able to accomplish the tasks, which included sending free text e-mails to an address proposed by the subjects themselves. The online performance results obtained, as well as the results of subjective questionnaires, support the viability of the proposed system.

Different effects of using pictures as stimuli in a P300 brain-computer interface under rapid serial visual presentation or row-column paradigm.

Previous proposals for controlling a P300-based BCI speller have shown an improvement using alternative images instead of letters as target stimuli under a row-column paradigm (RCP). However, the RCP is not suitable for those patients with a lack of gaze control. To solve that, the rapid serial visual presentation (RSVP) paradigm has been proposed in previous studies. The aim of the present work is to assess if a set of alternative pictures that improved performance in RCP could also improve performance in RSVP. Sixteen participants controlled four conditions in calibration and online tasks: letters in RCP, pictures in RCP, letters in RSVP and pictures in RSVP. The effect given by pictures was greater under RCP than under RSVP, both for performance and event-related potential analyses. Indeed, pictures did not show any improvement under RSVP in comparison to letters. In addition, the condition with pictures under RCP was declared the favourite by most users (68.75%), while the condition with pictures under RSVP was not chosen as favourite by any participant. Therefore, this work shows that the improvement related to the use of pictures as alternative flashing stimuli under RCP may not be transferred to RSVP. Graphical abstract.

Speech stream segregation to control an ERP-based auditory BCI.

Objective. The use of natural sounds in auditory brain-computer interfaces (BCI) has been shown to improve classification results and usability. Some auditory BCIs are based on stream segregation, in which the subjects must attend one audio stream and ignore the other(s); these streams include some kind of stimuli to be detected. In this work we focus on event-related potentials (ERP) and study whether providing intelligible content to each audio stream could help the users to better concentrate on the desired stream and so to better attend the target stimuli and to ignore the non-target ones.Approach. In addition to a control condition, two experimental conditions, based on the selective attention and the cocktail party effect, were tested using two simultaneous and spatialized audio streams: (a) the condition A2 consisted of an overlap of auditory stimuli (single syllables) on a background consisting of natural speech for each stream, (b) in condition A3, brief alterations of the natural flow of each speech were used as stimuli.Main results. The two experimental proposals improved the results of the control condition (single words as stimuli without a speech background) both in a cross validation analysis of the calibration part and in the online test. The analysis of the ERP responses also presented better discriminability for the two proposals in comparison to the control condition. The results of subjective questionnaires support the better usability of the first experimental condition.Significance. The use of natural speech as background improves the stream segregation in an ERP-based auditory BCI (with significant results in the performance metrics, the ERP waveforms, and in the preference parameter in subjective questionnaires). Future work in the field of ERP-based stream segregation should study the use of natural speech in combination with easily perceived but not distracting stimuli.

Effect of Distracting Background Speech in an Auditory Brain-Computer Interface.

Studies so far have analyzed the effect of distractor stimuli in different types of brain-computer interface (BCI). However, the effect of a background speech has not been studied using an auditory event-related potential (ERP-BCI), a convenient option when the visual path cannot be adopted by users. Thus, the aim of the present work is to examine the impact of a background speech on selection performance and user workload in auditory BCI systems. Eleven participants tested three conditions: (i) auditory BCI control condition, (ii) auditory BCI with a background speech to ignore (non-attentional condition), and (iii) auditory BCI while the user has to pay attention to the background speech (attentional condition). The results demonstrated that, despite no significant differences in performance, shared attention to auditory BCI and background speech required a higher cognitive workload. In addition, the P300 target stimuli in the non-attentional condition were significantly higher than those in the attentional condition for several channels. The non-attentional condition was the only condition that showed significant differences in the amplitude of the P300 between target and non-target stimuli. The present study indicates that background speech, especially when it is attended to, is an important interference that should be avoided while using an auditory BCI.

P300-Based Brain-Computer Interface Speller: Usability Evaluation of Three Speller Sizes by Severely Motor-Disabled Patients.

Brain-computer interface (BCI) spellers allow severe motor-disabled patients to communicate using their brain activity without muscular mobility. Different visual configurations of the widely studied P300-based BCI speller had been assessed with healthy and motor-disabled users. However, the speller size (in terms of cm) had only been assessed for healthy subjects. We think that the speller size might be limiting for some severely motor-disabled patients with restricted head and eye movements. The usability of three speller sizes was assessed for seven patients diagnosed with amyotrophic lateral sclerosis (ALS) and a participant diagnosed with Duchenne muscular dystrophy (DMD). This is the first usability evaluation of speller size with severely motor-disabled participants. Effectiveness (in the online results) and efficiency (in the workload test) of the medium speller was remarkably better. Satisfaction was significantly the highest with the medium size speller and the lowest with the small size. These results correlate with previously described findings in healthy subjects. In conclusion, the speller size should be considered when designing a speller paradigm, especially for motor-disabled individuals, since it might affect their performance and user experience while controlling a BCI speller.

Effects of Spatial Stimulus Overlap in a Visual P300-based Brain-computer Interface.

The rapid serial visual presentation (RSVP) paradigm seems to be one of the most appropriate for patients using P300-based brain-computer interface (BCI) applications, since non-ocular movements are required. However, according to previous works, the use of different locations for each stimulus may improve performance. Thus, the aim of the present work is to explore how spatial overlap between stimuli influences performance in controlling a visual P300-based BCI. Nineteen participants were tested using four levels of overlap between two stimuli: 100%, 66.7%, 33.3% and 0%. Significant differences in accuracy were found between the 0% overlapped condition and all the other conditions, and between 33.3% and higher overlap (66.7% and 100%). These results can be explained due to a modulation in the non-target stimulus amplitude signal caused by the overlapping factor. In short, the stimulus overlap provokes a modulation in performance using a P300-based BCI; this should be considered in future BCI proposals in which an optimal surface exploitation is convenient and potential users have only residual ocular movement.

Performance Analysis With Different Types of Visual Stimuli in a BCI-Based Speller Under an RSVP Paradigm.

Brain-Computer Interface (BCI) systems enable an alternative communication channel for severely-motor disabled patients to interact with their environment using no muscular movements. In recent years, the importance of research into non-gaze dependent brain-computer interface paradigms has been increasing, in contrast to the most frequently studied BCI-based speller paradigm (i.e., row-column presentation, RCP). Several visual modifications that have already been validated under the RCP paradigm for communication purposes have not been validated under the most extended non-gaze dependent rapid serial visual presentation (RSVP) paradigm. Thus, in the present study, three different sets of stimuli were assessed under RSVP, with the following communication features: white letters (WL), famous faces (FF), neutral pictures (NP). Eleven healthy subjects participated in this experiment, in which the subjects had to go through a calibration phase, an online phase and, finally, a subjective questionnaire completion phase. The results showed that the FF and NP stimuli promoted better performance in the calibration and online phases, being slightly better in the FF paradigm. Regarding the subjective questionnaires, again both FF and NP were preferred by the participants in contrast to the WL stimuli, but this time the NP stimuli scored slightly higher. These findings suggest that the use of FF and NP for RSVP-based spellers could be beneficial to increase information transfer rate in comparison to the most frequently used letter-based stimuli and could represent a promising communication system for individuals with altered ocular-motor function.

Evaluation of emotional and neutral pictures as flashing stimuli using a P300 brain-computer interface speller.

OBJECTIVE: Previous works have reported that complex emotional and visual stimuli can increase the amplitude of the P300 brain potential. Thus, the aim of the present work is to assess these kinds of images in a P300 brain-computer interface (BCI) speller as flashing stimuli. APPROACH: Twenty-three volunteers controlled four spellers with different sets of flashing stimuli: flashing letters, neutral pictures (NP), emotional pleasant pictures (EPP) and emotional unpleasant pictures (EUP). MAIN RESULTS: The sets of pictures showed a higher performance than the letters in accuracy and information transfer rate. These results were supported by the analysis of the P300 signal, where the picture sets offered the greatest amplitudes. The NP and EPP sets were the best evaluated in the subjective questionnaire. SIGNIFICANCE: In short, despite the fact that the effect of emotional stimuli could not be observed in the performance metrics, picture sets have offered a high performance and should be considered in future proposals for visual P300-based BCI applications.

Evaluation of flashing stimuli shape and colour heterogeneity using a P300 brain-computer interface speller.

Previous works using a visual P300-based speller have reported an improvement modifying the shape or colour of the presented stimulus. However, the effects of both blended factors have not been yet studied. Thus, the aim of the present work was to study both factors and assess the interaction between them. Fifteen naïve participants tested four different spellers in a calibration and online task. All spellers were similar except the employed illumination of the target stimulus: white letters, white blocks, coloured letters, and coloured blocks. Regarding the results, the block-shaped conditions offered an improvement versus the letter-shaped conditions in the calibration (accuracy) and online (accuracy and correct commands per minute) tasks. The analysis of the event-related potential waveforms showed a larger difference between target and no target stimuli waveforms for the block-shaped conditions versus the letter-shaped. The hypothesis regarding the colour heterogeneity of the stimuli was not found at any level of the analysis. Therefore, this first study combining block-shaped and colour factors, and offering an exhaustive evaluation of both, demonstrated the superiority of block-shaped illumination versus the standard letter-shaped flashing stimuli in classification performance.

UMA-BCI Speller: An easily configurable P300 speller tool for end users.

BACKGROUND AND OBJECTIVE: Some neurodegenerative conditions can severely limit patients' capability to communicate because of the loss of muscular control. Brain-computer interfaces may help in the restoration of communication with these patients, bypassing the muscular activity, so that brain signals can be directly interpreted by a computer. There are many studies regarding brain-controlled spellers; however, these systems do not usually leap out of the lab because of technical and economic requirements. As a consequence, the potential end users do not benefit from these scientific advances in their daily life. The objective of this paper is to present a novel brain-controlled speller designed to be used by patients due to its versatility and ease of use. METHODS: The brain-computer interface research group of the University of Málaga (UMA-BCI) has developed a speller application based on the well-known P300 potential which can be easily installed, configured and used. The application supports the common P300 paradigms: the Row-Column Paradigm and the Rapid Serial Visual Presentation Paradigm. The inner core of the application is implemented with a widely used and studied platform, BCI2000, which ensures its reliability and allows other researchers to apply modifications at will in order to test new features. Ten naïve volunteers carried out exercises using the application and completed usability tests for evaluation purposes. RESULTS: New subjects using the application managed to set up and use the proposed speller in less than an hour. The positive results of the evaluation through the usability tests support this application's ease of use. CONCLUSIONS: A new brain-controlled spelling tool has been presented whose aim is to be used by severely paralyzed patients in their daily lives, as well as by researchers to test new spelling features.

A Shaping Procedure to Modulate Two Cognitive Tasks to Improve a Sensorimotor Rhythm-Based Brain-Computer Interface System.

This study aimed to propose an adapted feedback using a psychological learning technique based on Skinner's shaping method to help the users to modulate two cognitive tasks (right-hand motor imagination and relaxed state) and improve better control in a Brain-Computer Interface. In the first experiment, a comparative study between performance in standard feedback (N = 9) and shaping method (N = 10) was conducted. The NASA Task Load Index questionnaire was applied to measure the user's workload. In the second experiment, a single case study was performed (N = 5) to verify the continuous learning by the shaping method. The first experiment showed significant interaction effect between sessions and group (F(1, 17) = 5.565; p = .031) which the shaping paradigm was applied. A second interaction effect demonstrates a higher performance increase in the relax state task with shaping procedure (F(1, 17) = 5. 038; p = .038). In NASA-TXL an interaction effect was obtained between the group and the cognitive task in Mental Demand (F(1, 17) = 6, 809; p = .018), Performance (F(1, 17) = 5, 725; p = .029), and Frustration (F(1, 17) = 9, 735; p = .006), no significance was found in Effort. In the second experiment, a trial-by-trial analysis shows an ascendant trend learning curve for the cognitive task with the lowest initial acquisition (relax state). The results suggest the effectiveness of the shaping procedure to modulate brain rhythms, improving mainly the cognitive task with greater initial difficulty and provide better interaction perception.

Evaluation of Switch and Continuous Navigation Paradigms to Command a Brain-Controlled Wheelchair.

A brain-computer interface (BCI) is a technology allowing patients with severe motor dysfunctions to use their electroencephalographic signals to create a communication channel to control devices. The objective of this paper is to study the feasibility of continuous and switch control modes for a brain-controlled wheelchair (BCW) using sensorimotor rhythms (SMR) modulated through a right-hand motor imagery task. Previous studies, which used a continuous navigation control with SMR, have reported the difficulty of maintaining the motor imagery task for a long time, especially for the forward command. The switch control has been presented as a proposal that may help to solve this issue since this task is only used temporary for either disabling or enabling the movement. Regarding the methodology, 10 of 15 able-bodied users, who had overcome the criterion of 30% error rate in the calibration phase, controlled the BCW using both paradigms. The navigation tasks consisted of a straight path divided in five sections: in three of them the users had to move forward, and in the other two the users had to maintain their position. To assess user performance in the device management, a usability approach was adopted, measuring the factors of effectiveness, efficiency, and satisfaction. Then, variables related to the time employed and commands selected by the user or parameters related to the confusion matrix were applied. In addition, the scores in NASA-TLX and two ad hoc questionnaires were considered to discuss the user experience controlling the wheelchair. Despite the results showed that the best system for a specific user relies on his/her abilities and preferences, the switch control mode obtained better accuracy (0.59 ± 0.17 for continuous and 0.72 ± 0.05 for switch). Furthermore, the switch paradigm can be recommended for the advance sections as with it users could complete the advance sections in less time (42.2 ± 28.7 s for continuous and 15.47 ± 3.43 s for switch), while the continuous mode seems to be better at keeping the wheelchair stopped (42.45 ± 16.01 s for continuous and 24.35 ± 10.94 s for switch).

A shaping procedure to modulate two cognitive tasks to improve a sensorimotor rhythm-based brain-computer interface systems

This study aimed to propose an adapted feedback using a psychological learning technique based on Skinner's shaping method to help the users to modulate two cognitive tasks (right-hand motor imagination and relaxed state) and improve better control in a Brain-Computer Interface. In the first experiment, a comparative study between performance in standard feedback (N = 9) and shaping method (N = 10) was conducted. The NASA Task Load Index questionnaire was applied to measure the user's workload. In the second experiment, a single case study was performed (N = 5) to verify the continuous learning by the shaping method. The first experiment showed significant interaction effect between sessions and group (F(1, 17) = 5.565; p = .031) which the shaping paradigm was applied. A second interaction effect demonstrates a higher performance increase in the relax state task with shaping procedure (F(1, 17) = 5. 038; p = .038). In NASA-TXL an interaction effect was obtained between the group and the cognitive task in Mental Demand (F(1, 17) = 6, 809; p = .018), Performance (F(1, 17) = 5, 725; p = .029), and Frustration (F(1, 17) = 9, 735; p = .006), no significance was found in Effort. In the second experiment, a trial-by-trial analysis shows an ascendant trend learning curve for the cognitive task with the lowest initial acquisition (relax state). The results suggest the effectiveness of the shaping procedure to modulate brain rhythms, improving mainly the cognitive task with greater initial difficulty and provide better interaction perception

No disponible

Brain-Computer Interface application: auditory serial interface to control a two-class motor-imagery-based wheelchair.

BACKGROUND: Certain diseases affect brain areas that control the movements of the patients' body, thereby limiting their autonomy and communication capacity. Research in the field of Brain-Computer Interfaces aims to provide patients with an alternative communication channel not based on muscular activity, but on the processing of brain signals. Through these systems, subjects can control external devices such as spellers to communicate, robotic prostheses to restore limb movements, or domotic systems. The present work focus on the non-muscular control of a robotic wheelchair. METHOD: A proposal to control a wheelchair through a Brain-Computer Interface based on the discrimination of only two mental tasks is presented in this study. The wheelchair displacement is performed with discrete movements. The control signals used are sensorimotor rhythms modulated through a right-hand motor imagery task or mental idle state. The peculiarity of the control system is that it is based on a serial auditory interface that provides the user with four navigation commands. The use of two mental tasks to select commands may facilitate control and reduce error rates compared to other endogenous control systems for wheelchairs. RESULTS: Seventeen subjects initially participated in the study; nine of them completed the three sessions of the proposed protocol. After the first calibration session, seven subjects were discarded due to a low control of their electroencephalographic signals; nine out of ten subjects controlled a virtual wheelchair during the second session; these same nine subjects achieved a medium accuracy level above 0.83 on the real wheelchair control session. CONCLUSION: The results suggest that more extensive training with the proposed control system can be an effective and safe option that will allow the displacement of a wheelchair in a controlled environment for potential users suffering from some types of motor neuron diseases.