Real-time detection and discrimination of visual perception using electrocorticographic signals.

OBJECTIVE: Several neuroimaging studies have demonstrated that the ventral temporal cortex contains specialized regions that process visual stimuli. This study investigated the spatial and temporal dynamics of electrocorticographic (ECoG) responses to different types and colors of visual stimulation that were presented to four human participants, and demonstrated a real-time decoder that detects and discriminates responses to untrained natural images. APPROACH: ECoG signals from the participants were recorded while they were shown colored and greyscale versions of seven types of visual stimuli (images of faces, objects, bodies, line drawings, digits, and kanji and hiragana characters), resulting in 14 classes for discrimination (experiment I). Additionally, a real-time system asynchronously classified ECoG responses to faces, kanji and black screens presented via a monitor (experiment II), or to natural scenes (i.e. the face of an experimenter, natural images of faces and kanji, and a mirror) (experiment III). Outcome measures in all experiments included the discrimination performance across types based on broadband γ activity. MAIN RESULTS: Experiment I demonstrated an offline classification accuracy of 72.9% when discriminating among the seven types (without color separation). Further discrimination of grey versus colored images reached an accuracy of 67.1%. Discriminating all colors and types (14 classes) yielded an accuracy of 52.1%. In experiment II and III, the real-time decoder correctly detected 73.7% responses to face, kanji and black computer stimuli and 74.8% responses to presented natural scenes. SIGNIFICANCE: Seven different types and their color information (either grey or color) could be detected and discriminated using broadband γ activity. Discrimination performance maximized for combined spatial-temporal information. The discrimination of stimulus color information provided the first ECoG-based evidence for color-related population-level cortical broadband γ responses in humans. Stimulus categories can be detected by their ECoG responses in real time within 500 ms with respect to stimulus onset.

Passive functional mapping guides electrical cortical stimulation for efficient determination of eloquent cortex in epilepsy patients.

Electrical cortical stimulation (ECS) is often used in presurgical evaluation procedures for patients suffering from pharmacoresistant epilepsy. Real-time functional mapping (RTFM) is an alternative brain mapping methodology that can accompany traditional functional mapping approaches like ECS. In this paper, we present a combined RTFM/ECS system that aims to exploit the common ground and the advantages of the two procedures for improved time/effort effectiveness, patients' experience and safety. Using the RTFM and ECS data from four patients who suffer epilepsy, we demonstrate that the RTFM-guided ECS procedure hypothetically reduces the number of electrical stimulations necessary for eloquent cortex detection by 40%.

Online control of a humanoid robot through hand movement imagination using CSP and ECoG based features.

Intention recognition through decoding brain activity could lead to a powerful and independent Brain-Computer-Interface (BCI) allowing for intuitive control of devices like robots. A common strategy for realizing such a system is the motor imagery (MI) BCI using electroencephalography (EEG). Changing to invasive recordings like electrocorticography (ECoG) allows extracting very robust features and easy introduction of an idle state, which might simplify the mental task and allow the subject to focus on the environment. Especially for multi-channel recordings like ECoG, common spatial patterns (CSP) provide a powerful tool for feature optimization and dimensionality reduction. This work focuses on an invasive and independent MI BCI that allows triggering from an idle state, and therefore facilitates tele-operation of a humanoid robot. The task was to lift a can with the robot's hand. One subject participated and reached 95.4 % mean online accuracy after six runs of 40 trials. To our knowledge, this is the first online experiment with a MI BCI using CSPs from ECoG signals.

Distinction of individual finger responses in somatosensory cortex using ECoG high-gamma activation mapping.

This study demonstrates the feasibility of high-gamma activity mapping for localization of somatosensory finger areas in the human brain. Identification of functional brain regions is important in surgical planning, such as for resections of epileptic foci or brain tumors. The mapping procedure is done using electrocorticography (ECoG), an invasive technique in which electrical brain signals are acquired from the cortical surface. Two epilepsy patients with implanted electrode grids participated in the study. Data were collected during a vibrotactile finger stimulation paradigm and showed significant cortical activation (p <; 0.001) in the high-gamma range over the contralateral somatosensory cortex. The results are consistent with previous studies that used fMRI in test subjects without implanted electrodes. Therefore, the results suggest that localizing the cortical representations of the fingers in clinical practice using ECoG is feasible, even without the patient's active participation.

Rapid and low-invasive functional brain mapping by realtime visualization of high gamma activity for awake craniotomy.

For neurosurgery with an awake craniotomy, the critical issue is to set aside enough time to identify eloquent cortices by electrocortical stimulation (ECS). High gamma activity (HGA) ranging between 80 and 120 Hz on electrocorticogram (ECoG) is assumed to reflect localized cortical processing. In this report, we used realtime HGA mapping and functional magnetic resonance imaging (fMRI) for rapid and reliable identification of motor and language functions. Three patients with intra-axial tumors in their dominant hemisphere underwent preoperative fMRI and lesion resection with an awake craniotomy. All patients showed significant fMRI activation evoked by motor and language tasks. After the craniotomy, we recorded ECoG activity by placing subdural grids directly on the exposed brain surface. Each patient performed motor and language tasks and demonstrated realtime HGA dynamics in hand motor areas and parts of the inferior frontal gyrus. Sensitivity and specificity of HGA mapping were 100% compared to ECS mapping in the frontal lobe, which suggested HGA mapping precisely indicated eloquent cortices. The investigation times of HGA mapping was significantly shorter than that of ECS mapping. Specificities of the motor and language-fMRI, however, did not reach 85%. The results of HGA mapping was mostly consistent with those of ECS mapping, although fMRI tended to overestimate functional areas. This novel technique enables rapid and accurate functional mapping.

Single trial detection of hand poses in human ECoG using CSP based feature extraction.

Decoding brain activity of corresponding highlevel tasks may lead to an independent and intuitively controlled Brain-Computer Interface (BCI). Most of today's BCI research focuses on analyzing the electroencephalogram (EEG) which provides only limited spatial and temporal resolution. Derived electrocorticographic (ECoG) signals allow the investigation of spatially highly focused task-related activation within the high-gamma frequency band, making the discrimination of individual finger movements or complex grasping tasks possible. Common spatial patterns (CSP) are commonly used for BCI systems and provide a powerful tool for feature optimization and dimensionality reduction. This work focused on the discrimination of (i) three complex hand movements, as well as (ii) hand movement and idle state. Two subjects S1 and S2 performed single `open', `peace' and `fist' hand poses in multiple trials. Signals in the high-gamma frequency range between 100 and 500 Hz were spatially filtered based on a CSP algorithm for (i) and (ii). Additionally, a manual feature selection approach was tested for (i). A multi-class linear discriminant analysis (LDA) showed for (i) an error rate of 13.89 % / 7.22 % and 18.42 % / 1.17 % for S1 and S2 using manually / CSP selected features, where for (ii) a two class LDA lead to a classification error of 13.39 % and 2.33 % for S1 and S2, respectively.

A BCI using VEP for continuous control of a mobile robot.

A brain-computer interface (BCI) translates brain activity into commands to control devices or software. Common approaches are based on visual evoked potentials (VEP), extracted from the electroencephalogram (EEG) during visual stimulation. High information transfer rates (ITR) can be achieved using (i) steady-state VEP (SSVEP) or (ii) code-modulated VEP (c-VEP). This study investigates how applicable such systems are for continuous control of robotic devices and which method performs best. Eleven healthy subjects steered a robot along a track using four BCI controls on a computer screen in combination with feedback video of the movement. The average time to complete the tasks was (i) 573.43 s and (ii) 222.57 s. In a second non-continuous trial-based validation run the maximum achievable online classification accuracy over all subjects was (i) 91.36 % and (ii) 98.18 %. This results show that the c-VEP fits the needs of a continuous system better than the SSVEP implementation.

Prototype of an auto-calibrating, context-aware, hybrid brain-computer interface.

We present the prototype of a context-aware framework that allows users to control smart home devices and to access internet services via a Hybrid BCI system of an auto-calibrating sensorimotor rhythm (SMR) based BCI and another assistive device (Integra Mouse mouth joystick). While there is extensive literature that describes the merit of Hybrid BCIs, auto-calibrating and co-adaptive ERD BCI training paradigms, specialized BCI user interfaces, context-awareness and smart home control, there is up to now, no system that includes all these concepts in one integrated easy-to-use framework that can truly benefit individuals with severe functional disabilities by increasing independence and social inclusion. Here we integrate all these technologies in a prototype framework that does not require expert knowledge or excess time for calibration. In a first pilot-study, 3 healthy volunteers successfully operated the system using input signals from an ERD BCI and an Integra Mouse and reached average positive predictive values (PPV) of 72 and 98% respectively. Based on what we learned here we are planning to improve the system for a test with a larger number of healthy volunteers so we can soon bring the system to benefit individuals with severe functional disability.