Reaching the ceiling? Empirical scaling behaviour for deep EEG pathology classification.

Machine learning techniques, particularly deep convolutional neural networks (ConvNets), are increasingly being used to automate clinical EEG analysis, with the potential to reduce the clinical burden and improve patient care. However, further research is required before they can be used in clinical settings, particularly regarding the impact of the number of training samples and model parameters on their testing error. To address this, we present a comprehensive study of the empirical scaling behaviour of ConvNets for EEG pathology classification. We analysed the testing error with increasing the training samples and model size for four different ConvNet architectures. The focus of our experiments is width scaling, and we have increased the number of parameters to up to 1.8 million. Our evaluation was based on two publicly available datasets: the Temple University Hospital (TUH) Abnormal EEG Corpus and the TUH Abnormal Expansion Balanced EEG Corpus, which together contain 10,707 training samples. The results show that the testing error follows a saturating power-law with both model and dataset size. This pattern is consistent across different datasets and ConvNet architectures. Furthermore, empirically observed accuracies saturate at 85%-87%, which may be due to an imperfect inter-rater agreement on the clinical labels. The empirical scaling behaviour of the test performance with dataset and model size has significant implications for deep EEG pathology classification research and practice. Our findings highlight the potential of deep ConvNets for high-performance EEG pathology classification, and the identified scaling relationships provide valuable recommendations for the advancement of automated EEG diagnostics.

An extended clinical EEG dataset with 15,300 automatically labelled recordings for pathology decoding.

Automated clinical EEG analysis using machine learning (ML) methods is a growing EEG research area. Previous studies on binary EEG pathology decoding have mainly used the Temple University Hospital (TUH) Abnormal EEG Corpus (TUAB) which contains approximately 3,000 manually labelled EEG recordings. To evaluate and eventually even improve the generalisation performance of machine learning methods for EEG pathology, decoding larger, publicly available datasets is required. A number of studies addressed the automatic labelling of large open-source datasets as an approach to create new datasets for EEG pathology decoding, but little is known about the extent to which training on larger, automatically labelled dataset affects decoding performances of established deep neural networks. In this study, we automatically created additional pathology labels for the Temple University Hospital (TUH) EEG Corpus (TUEG) based on the medical reports using a rule-based text classifier. We generated a dataset of 15,300 newly labelled recordings, which we call the TUH Abnormal Expansion EEG Corpus (TUABEX), and which is five times larger than the TUAB. Since the TUABEX contains more pathological (75%) than non-pathological (25%) recordings, we then selected a balanced subset of 8,879 recordings, the TUH Abnormal Expansion Balanced EEG Corpus (TUABEXB). To investigate how training on a larger, automatically labelled dataset affects the decoding performance of deep neural networks, we applied four established deep convolutional neural networks (ConvNets) to the task of pathological versus non-pathological classification and compared the performance of each architecture after training on different datasets. The results show that training on the automatically labelled TUABEXB dataset rather than training on the manually labelled TUAB dataset increases accuracies on TUABEXB and even for TUAB itself for some architectures. We argue that automatically labelling of large open-source datasets can be used to efficiently utilise the massive amount of EEG data stored in clinical archives. We make the proposed TUABEXB available open source and thus offer a new dataset for EEG machine learning research.

A reusable benchmark of brain-age prediction from M/EEG resting-state signals.

Population-level modeling can define quantitative measures of individual aging by applying machine learning to large volumes of brain images. These measures of brain age, obtained from the general population, helped characterize disease severity in neurological populations, improving estimates of diagnosis or prognosis. Magnetoencephalography (MEG) and Electroencephalography (EEG) have the potential to further generalize this approach towards prevention and public health by enabling assessments of brain health at large scales in socioeconomically diverse environments. However, more research is needed to define methods that can handle the complexity and diversity of M/EEG signals across diverse real-world contexts. To catalyse this effort, here we propose reusable benchmarks of competing machine learning approaches for brain age modeling. We benchmarked popular classical machine learning pipelines and deep learning architectures previously used for pathology decoding or brain age estimation in 4 international M/EEG cohorts from diverse countries and cultural contexts, including recordings from more than 2500 participants. Our benchmarks were built on top of the M/EEG adaptations of the BIDS standard, providing tools that can be applied with minimal modification on any M/EEG dataset provided in the BIDS format. Our results suggest that, regardless of whether classical machine learning or deep learning was used, the highest performance was reached by pipelines and architectures involving spatially aware representations of the M/EEG signals, leading to R2 scores between 0.60-0.74. Hand-crafted features paired with random forest regression provided robust benchmarks even in situations in which other approaches failed. Taken together, this set of benchmarks, accompanied by open-source software and high-level Python scripts, can serve as a starting point and quantitative reference for future efforts at developing M/EEG-based measures of brain aging. The generality of the approach renders this benchmark reusable for other related objectives such as modeling specific cognitive variables or clinical endpoints.

A Structured Approach to Test the Signal Quality of Electroencephalography Measurements During Use of Head-Mounted Displays for Virtual Reality Applications.

Joint applications of virtual reality (VR) systems and electroencephalography (EEG) offer numerous new possibilities ranging from behavioral science to therapy. VR systems allow for highly controlled experimental environments, while EEG offers a non-invasive window to brain activity with a millisecond-ranged temporal resolution. However, EEG measurements are highly susceptible to electromagnetic (EM) noise and the influence of EM noise of head-mounted-displays (HMDs) on EEG signal quality has not been conclusively investigated. In this paper, we propose a structured approach to test HMDs for EM noise potentially harmful to EEG measures. The approach verifies the impact of HMDs on the frequency- and time-domain of the EEG signal recorded in healthy subjects. The verification task includes a comparison of conditions with and without an HMD during (i) an eyes-open vs. eyes-closed task, and (ii) with respect to the sensory- evoked brain activity. The approach is developed and tested to derive potential effects of two commercial HMDs, the Oculus Rift and the HTC Vive Pro, on the quality of 64-channel EEG measurements. The results show that the HMDs consistently introduce artifacts, especially at the line hum of 50 Hz and the HMD refresh rate of 90 Hz, respectively, and their harmonics. The frequency range that is typically most important in non-invasive EEG research and applications (<50 Hz) however, remained largely unaffected. Hence, our findings demonstrate that high-quality EEG recordings, at least in the frequency range up to 50 Hz, can be obtained with the two tested HMDs. However, the number of commercially available HMDs is constantly rising. We strongly suggest to thoroughly test such devices upfront since each HMD will most likely have its own EM footprint and this article provides a structured approach to implement such tests with arbitrary devices.

The CLAIRE COVID-19 initiative: approach, experiences and recommendations.

A volunteer effort by Artificial Intelligence (AI) researchers has shown it can deliver significant research outcomes rapidly to help tackle COVID-19. Within two months, CLAIRE's self-organising volunteers delivered the World's first comprehensive curated repository of COVID-19-related datasets useful for drug-repurposing, drafted review papers on the role CT/X-ray scan analysis and robotics could play, and progressed research in other areas. Given the pace required and nature of voluntary efforts, the teams faced a number of challenges. These offer insights in how better to prepare for future volunteer scientific efforts and large scale, data-dependent AI collaborations in general. We offer seven recommendations on how to best leverage such efforts and collaborations in the context of managing future crises.

Blink- and saccade-related suppression effects in early visual areas of the human brain: Intracranial EEG investigations during natural viewing conditions.

Blinks and saccades, both ubiquitous in natural viewing conditions, cause rapid changes of visual inputs that are hardly consciously perceived. The neural dynamics in early visual areas of the human brain underlying this remarkable visual stability are still incompletely understood. We used electrocorticography (ECoG) from electrodes directly implanted on the human early visual areas V1, V2, V3d/v, V4d/v and the fusiform gyrus to investigate blink- and saccade-related neuronal suppression effects during non-experimental, free viewing conditions. We found a characteristic, biphasic, broadband gamma power decrease-increase pattern in all investigated visual areas. During saccades, a decrease in gamma power clearly preceded eye movement onset, at least in V1. This may indicate that cortical information processing is actively suppressed in human early visual areas before and during saccades, which then possibly mediates perceptual visual suppression. The following eye movement offset-related increase in gamma power may indicate the recovery of visual perception and the resumption of visual processing.

A Study of Word Complexity Under Conditions of Non-experimental, Natural Overt Speech Production Using ECoG.

The linguistic complexity of words has largely been studied on the behavioral level and in experimental settings. Only little is known about the neural processes underlying it in uninstructed, spontaneous conversations. We built up a multimodal neurolinguistic corpus composed of synchronized audio, video, and electrocorticographic (ECoG) recordings from the fronto-temporo-parietal cortex to address this phenomenon based on uninstructed, spontaneous speech production. We performed extensive linguistic annotations of the language material and calculated word complexity using several numeric parameters. We orthogonalized the parameters with the help of a linear regression model. Then, we correlated the spectral components of neural activity with the individual linguistic parameters and with the residuals of the linear regression model, and compared the results. The proportional relation between the number of consonants and vowels, which was the most informative parameter with regard to the neural representation of word complexity, showed effects in two areas: the frontal one was at the junction of the premotor cortex, the prefrontal cortex, and Brodmann area 44. The postcentral one lay directly above the lateral sulcus and comprised the ventral central sulcus, the parietal operculum and the adjacent inferior parietal cortex. Beyond the physiological findings summarized here, our methods may be useful for those interested in ways of studying neural effects related to natural language production and in surmounting the intrinsic problem of collinearity between multiple features of spontaneously spoken material.

Machine-learning-based diagnostics of EEG pathology.

Machine learning (ML) methods have the potential to automate clinical EEG analysis. They can be categorized into feature-based (with handcrafted features), and end-to-end approaches (with learned features). Previous studies on EEG pathology decoding have typically analyzed a limited number of features, decoders, or both. For a I) more elaborate feature-based EEG analysis, and II) in-depth comparisons of both approaches, here we first develop a comprehensive feature-based framework, and then compare this framework to state-of-the-art end-to-end methods. To this aim, we apply the proposed feature-based framework and deep neural networks including an EEG-optimized temporal convolutional network (TCN) to the task of pathological versus non-pathological EEG classification. For a robust comparison, we chose the Temple University Hospital (TUH) Abnormal EEG Corpus (v2.0.0), which contains approximately 3000 EEG recordings. The results demonstrate that the proposed feature-based decoding framework can achieve accuracies on the same level as state-of-the-art deep neural networks. We find accuracies across both approaches in an astonishingly narrow range from 81 to 86%. Moreover, visualizations and analyses indicated that both approaches used similar aspects of the data, e.g., delta and theta band power at temporal electrode locations. We argue that the accuracies of current binary EEG pathology decoders could saturate near 90% due to the imperfect inter-rater agreement of the clinical labels, and that such decoders are already clinically useful, such as in areas where clinical EEG experts are rare. We make the proposed feature-based framework available open source and thus offer a new tool for EEG machine learning research.

Modulating dream experience: Noninvasive brain stimulation over the sensorimotor cortex reduces dream movement.

Recently, cortical correlates of specific dream contents have been reported, such as the activation of the sensorimotor cortex during dreamed hand clenching. Yet, despite a close resemblance of such activation patterns to those seen during the corresponding wakeful behaviour, the causal mechanisms underlying specific dream contents remain largely elusive. Here, we aimed to investigate the causal role of the sensorimotor cortex in generating movement and bodily sensations during REM sleep dreaming. Following bihemispheric transcranial direct current stimulation (tDCS) or sham stimulation, guided by functional mapping of the primary motor cortex, naive participants were awakened from REM sleep and responded to a questionnaire on bodily sensations in dreams. Electromyographic (EMG) and electroencephalographic (EEG) recordings were used to quantify physiological changes during the preceding REM period. We found that tDCS, compared to sham stimulation, significantly decreased reports of dream movement, especially of repetitive actions. Other types of bodily experiences, such as tactile or vestibular sensations, were not affected by tDCS, confirming the specificity of stimulation effects to movement sensations. In addition, tDCS reduced EEG interhemispheric coherence in parietal areas and affected the phasic EMG correlation between both arms. These findings show that a complex temporal reorganization of the motor network co-occurred with the reduction of dream movement, revealing a link between central and peripheral motor processes and movement sensations of the dream self. tDCS over the sensorimotor cortex interferes with dream movement during REM sleep, which is consistent with a causal contribution to dream experience and has broader implications for understanding the neural basis of self-experience in dreams.

Large-scale cortical travelling waves predict localized future cortical signals.

Predicting future brain signal is highly sought-after, yet difficult to achieve. To predict the future phase of cortical activity at localized ECoG and MEG recording sites, we exploit its predominant, large-scale, spatiotemporal dynamics. The dynamics are extracted from the brain signal through Fourier analysis and principal components analysis (PCA) only, and cast in a data model that predicts future signal at each site and frequency of interest. The dominant eigenvectors of the PCA that map the large-scale patterns of past cortical phase to future ones take the form of smoothly propagating waves over the entire measurement array. In ECoG data from 3 subjects and MEG data from 20 subjects collected during a self-initiated motor task, mean phase prediction errors were as low as 0.5 radians at local sites, surpassing state-of-the-art methods of within-time-series or event-related models. Prediction accuracy was highest in delta to beta bands, depending on the subject, was more accurate during episodes of high global power, but was not strongly dependent on the time-course of the task. Prediction results did not require past data from the to-be-predicted site. Rather, best accuracy depended on the availability in the model of long wavelength information. The utility of large-scale, low spatial frequency traveling waves in predicting future phase activity at local sites allows estimation of the error introduced by failing to account for irreducible trajectories in the activity dynamics.

Hybrid Brain-Computer-Interfacing for Human-Compliant Robots: Inferring Continuous Subjective Ratings With Deep Regression.

Appropriate robot behavior during human-robot interaction is a key part in the development of human-compliant assistive robotic systems. This study poses the question of how to continuously evaluate the quality of robotic behavior in a hybrid brain-computer interfacing (BCI) task, combining brain and non-brain signals, and how to use the collected information to adapt the robot's behavior accordingly. To this aim, we developed a rating system compatible with EEG recordings, requiring the users to execute only small movements with their thumb on a wireless controller to rate the robot's behavior on a continuous scale. The ratings were recorded together with dry EEG, respiration, ECG, and robotic joint angles in ROS. Pilot experiments were conducted with three users that had different levels of previous experience with robots. The results demonstrate the feasibility to obtain continuous rating data that give insight into the subjective user perception during direct human-robot interaction. The rating data suggests differences in subjective perception for users with no, moderate, or substantial previous robot experience. Furthermore, a variety of regression techniques, including deep CNNs, allowed us to predict the subjective ratings. Performance was better when using the position of the robotic hand than when using EEG, ECG, or respiration. A consistent advantage of features expected to be related to a motor bias could not be found. Across-user predictions showed that the models most likely learned a combination of general and individual features across-users. A transfer of pre-trained regressor to a new user was especially accurate in users with more experience. For future research, studies with more participants will be needed to evaluate the methodology for its use in practice. Data and code to reproduce this study are available at https://github.com/TNTLFreiburg/NiceBot.

Erratum: Author Correction: Human motor cortex relies on sparse and action-specific activation during laughing, smiling and speech production.

[This corrects the article DOI: 10.1038/s42003-019-0360-3.].

Probabilistic neuroanatomical assignment of intracranial electrodes using the ELAS toolbox.

BACKGROUND: Intracranial electroencephalography (iEEG) is increasingly used in neuroscientific research. However, the position of the implanted electrodes varies greatly between patients, which makes group analyses particularly difficult. Therefore, an assignment procedure is needed that enables the neuroanatomical information to be obtained for each individual electrode contact. NEW METHOD: Here, we present a MATLAB-based electrode assignment approach for iEEG electrode contacts, implemented in the open-source toolbox ELAS, that allows a hierarchical probabilistic assignment of individual electrode contacts to cytoarchitectonically-defined brain areas. The here presented ELAS consists of two major steps: (I) a pre-assignment to the cerebral lobes and (II) a following probabilistic assignment based on lobe-specific probability maps of the SPM Anatomy Toolbox. RESULTS: We analyzed iEEG data obtained in 14 epilepsy patients with a total of 783 intracranial electrode contacts. The neuroanatomical assignment to cortical brain areas was possible in 72.5% of the electrode contacts that were located on the lateral cortical convexity. COMPARISON WITH EXISTING METHODS: This assignment procedure is to our knowledge the first approach that combines both individual macro-anatomical and cytoarchitectonic probabilistic information. Due to the integration of information about individual anatomical landmarks, incorrect assignments could be avoided in approx. 7% of electrode contacts. CONCLUSION: The present study demonstrates how probabilistic assignment procedures developed for the analysis of neuroimaging data can be adapted to iEEG, which is especially helpful for group analyses. The presented assignment approach is freely available via the open-source toolbox ELAS, including a 3D visualization and a file export for virtual reality setups.

A transcallosal fibre system between homotopic inferior frontal regions supports complex linguistic processing.

Inferior frontal regions in the left and right hemisphere support different aspects of language processing. In the canonical model, left inferior frontal regions are mostly involved in processing based on phonological, syntactic and semantic features of language, whereas the right inferior frontal regions process paralinguistic aspects like affective prosody. Using diffusion tensor imaging (DTI)-based probabilistic fibre tracking in 20 healthy volunteers, we identify a callosal fibre system connecting left and right inferior frontal regions that are involved in linguistic processing of varying complexity. Anatomically, we show that the interhemispheric fibres are highly aligned and distributed along a rostral to caudal gradient in the body and genu of the corpus callosum to connect homotopic inferior frontal regions. In the light of converging data, taking previous DTI-based tracking studies and clinical case studies into account, our findings suggest that the right inferior frontal cortex not only processes paralinguistic aspects of language (such as affective prosody), as purported by the canonical model, but also supports the computation of linguistic aspects of varying complexity in the human brain. Our model may explain patterns of right-hemispheric contribution to stroke recovery as well as disorders of prosodic processing. Beyond language-related brain function, we discuss how inter-species differences in interhemispheric connectivity and fibre density, including the system we described here may also explain differences in transcallosal information transfer and cognitive abilities across different mammalian species.

Human motor cortex relies on sparse and action-specific activation during laughing, smiling and speech production.

Smiling, laughing, and overt speech production are fundamental to human everyday communication. However, little is known about how the human brain achieves the highly accurate and differentiated control of such orofacial movement during natural conditions. Here, we utilized the high spatiotemporal resolution of subdural recordings to elucidate how human motor cortex is functionally engaged during control of real-life orofacial motor behaviour. For each investigated movement class-lip licking, speech production, laughing and smiling-our findings reveal a characteristic brain activity pattern within the mouth motor cortex with both spatial segregation and overlap between classes. Our findings thus show that motor cortex relies on sparse and action-specific activation during real-life orofacial behaviour, apparently organized in distinct but overlapping subareas that control different types of natural orofacial movements.

Erratum: Author Correction: Human motor cortex relies on sparse and action-specific activation during laughing, smiling and speech production.

[This corrects the article DOI: 10.1038/s42003-019-0360-3.].

Real-life speech production and perception have a shared premotor-cortical substrate.

Motor-cognitive accounts assume that the articulatory cortex is involved in language comprehension, but previous studies may have observed such an involvement as an artefact of experimental procedures. Here, we employed electrocorticography (ECoG) during natural, non-experimental behavior combined with electrocortical stimulation mapping to study the neural basis of real-life human verbal communication. We took advantage of ECoG's ability to capture high-gamma activity (70-350 Hz) as a spatially and temporally precise index of cortical activation during unconstrained, naturalistic speech production and perception conditions. Our findings show that an electrostimulation-defined mouth motor region located in the superior ventral premotor cortex is consistently activated during both conditions. This region became active early relative to the onset of speech production and was recruited during speech perception regardless of acoustic background noise. Our study thus pinpoints a shared ventral premotor substrate for real-life speech production and perception with its basic properties.

Electrophysiological correlates of neurodegeneration in motor and non-motor brain regions in amyotrophic lateral sclerosis-implications for brain-computer interfacing.

OBJECTIVE: For patients with amyotrophic lateral sclerosis (ALS) who are suffering from severe communication or motor problems, brain-computer interfaces (BCIs) can improve the quality of life and patient autonomy. However, current BCI systems are not as widely used as their potential and patient demand would let assume. This underutilization is a result of technological as well as user-based limitations but also of the comparatively poor performance of currently existing BCIs in patients with late-stage ALS, particularly in the locked-in state. APPROACH: Here we review a broad range of electrophysiological studies in ALS patients with the aim to identify electrophysiological correlates of ALS-related neurodegeneration in motor and non-motor brain regions in to better understand potential neurophysiological limitations of current BCI systems for ALS patients. To this end we analyze studies in ALS patients that investigated basic sensory evoked potentials, resting-state and task-based paradigms using electroencephalography or electrocorticography for basic research purposes as well as for brain-computer interfacing. Main results and significance. Our review underscores that, similarly to mounting evidence from neuroimaging and neuropathology, electrophysiological measures too indicate neurodegeneration in non-motor areas in ALS. Furthermore, we identify an unexpected gap of basic and advanced electrophysiological studies in late-stage ALS patients, particularly in the locked-in state. We propose a research strategy on how to fill this gap in order to improve the design and performance of future BCI systems for this patient group.

Spectral bandwidth of interictal fast epileptic activity characterizes the seizure onset zone.

The foremost aim of presurgical epilepsy evaluation is the delineation of the seizure onset zone (SOZ). There is increasing evidence that fast epileptic activity (FEA, 14-250 Hz) occurring interictally, i.e. between seizures, is predominantly localized within the SOZ. Currently it is unknown, which frequency band of FEA performs best in identifying the SOZ, although prior studies suggest highest concordance of spectral changes with the SOZ for high frequency changes. We suspected that FEA reflects dampened oscillations in local cortical excitatory-inhibitory neural networks, and that interictal FEA in the SOZ is a consequence of reduced oscillatory damping. We therefore predict a narrowing of the spectral bandwidth alongside increased amplitudes of spectral peaks during interictal FEA events. To test this hypothesis, we evaluated spectral changes during interictal FEA in invasive EEG (iEEG) recordings of 13 patients with focal epilepsy. In relative spectra of beta and gamma band changes (14-250 Hz) during FEA, we found that spectral peaks within the SOZ indeed were significantly more narrow-banded and their power changes were significantly higher than outside the SOZ. In contrast, the peak frequency did not differ within and outside the SOZ. Our results show that bandwidth and power changes of spectral modulations during FEA both help localizing the SOZ. We propose the spectral bandwidth as new source of information for the evaluation of EEG data.