Optimal processing of surface facial EMG to identify emotional expressions: A data-driven approach.

Surface facial electromyography (EMG) is commonly used to detect emotions from subtle facial expressions. Although there are established procedures for collecting EMG data and some aspects of their processing, there is little agreement among researchers about the optimal way to process the EMG signal, so that the study-unrelated variability (noise) is removed, and the emotion-related variability is best detected. The aim of the current paper was to establish an optimal processing pipeline for EMG data for identifying emotional expressions in facial muscles. We identified the most common processing steps from existing literature and created 72 processing pipelines that represented all the different processing choices. We applied these pipelines to a previously published dataset from a facial mimicry experiment, where 100 adult participants observed happy and sad facial expressions, whilst the activity of their facial muscles, zygomaticus major and corrugator supercilii, was recorded with EMG. We used a resampling approach and subsets of the original data to investigate the effect and robustness of different processing choices on the performance of a logistic regression model that predicted the mimicked emotion (happy/sad) from the EMG signal. In addition, we used a random forest model to identify the most important processing steps for the sensitivity of the logistic regression model. Three processing steps were found to be most impactful: baseline correction, standardisation within muscles, and standardisation within subjects. The chosen feature of interest and the signal averaging had little influence on the sensitivity to the effect. We recommend an optimal processing pipeline, share our code and data, and provide a step-by-step walkthrough for researchers.

Source-reconstruction of the sensorimotor network from resting-state macaque electrocorticography.

The discovery of hemodynamic (BOLD-fMRI) resting-state networks (RSNs) has brought about a fundamental shift in our thinking about the role of intrinsic brain activity. The electrophysiological underpinnings of RSNs remain largely elusive and it has been shown only recently that electric cortical rhythms are organized into the same RSNs as hemodynamic signals. Most electrophysiological studies into RSNs use magnetoencephalography (MEG) or scalp electroencephalography (EEG), which limits the spatial resolution with which electrophysiological RSNs can be observed. Due to their close proximity to the cortical surface, electrocorticographic (ECoG) recordings can potentially provide a more detailed picture of the functional organization of resting-state cortical rhythms, albeit at the expense of spatial coverage. In this study we propose using source-space spatial independent component analysis (spatial ICA) for identifying generators of resting-state cortical rhythms as recorded with ECoG and for reconstructing their functional connectivity. Network structure is assessed by two kinds of connectivity measures: instantaneous correlations between band-limited amplitude envelopes and oscillatory phase-locking. By simulating rhythmic cortical generators, we find that the reconstruction of oscillatory phase-locking is more challenging than that of amplitude correlations, particularly for low signal-to-noise levels. Specifically, phase-lags can both be over- and underestimated, which troubles the interpretation of lag-based connectivity measures. We illustrate the methodology on somatosensory beta rhythms recorded from a macaque monkey using ECoG. The methodology decomposes the resting-state sensorimotor network into three cortical generators, distributed across primary somatosensory and primary and higher-order motor areas. The generators display significant and reproducible amplitude correlations and phase-locking values with non-zero lags. Our findings illustrate the level of spatial detail attainable with source-projected ECoG and motivates wider use of the methodology for studying resting-state as well as event-related cortical dynamics in macaque and human.

Alpha power indexes task-related networks on large and small scales: A multimodal ECoG study in humans and a non-human primate.

Performing different tasks, such as generating motor movements or processing sensory input, requires the recruitment of specific networks of neuronal populations. Previous studies suggested that power variations in the alpha band (8-12Hz) may implement such recruitment of task-specific populations by increasing cortical excitability in task-related areas while inhibiting population-level cortical activity in task-unrelated areas (Klimesch et al., 2007; Jensen and Mazaheri, 2010). However, the precise temporal and spatial relationships between the modulatory function implemented by alpha oscillations and population-level cortical activity remained undefined. Furthermore, while several studies suggested that alpha power indexes task-related populations across large and spatially separated cortical areas, it was largely unclear whether alpha power also differentially indexes smaller networks of task-related neuronal populations. Here we addressed these questions by investigating the temporal and spatial relationships of electrocorticographic (ECoG) power modulations in the alpha band and in the broadband gamma range (70-170Hz, indexing population-level activity) during auditory and motor tasks in five human subjects and one macaque monkey. In line with previous research, our results confirm that broadband gamma power accurately tracks task-related behavior and that alpha power decreases in task-related areas. More importantly, they demonstrate that alpha power suppression lags population-level activity in auditory areas during the auditory task, but precedes it in motor areas during the motor task. This suppression of alpha power in task-related areas was accompanied by an increase in areas not related to the task. In addition, we show for the first time that these differential modulations of alpha power could be observed not only across widely distributed systems (e.g., motor vs. auditory system), but also within the auditory system. Specifically, alpha power was suppressed in the locations within the auditory system that most robustly responded to particular sound stimuli. Altogether, our results provide experimental evidence for a mechanism that preferentially recruits task-related neuronal populations by increasing cortical excitability in task-related cortical areas and decreasing cortical excitability in task-unrelated areas. This mechanism is implemented by variations in alpha power and is common to humans and the non-human primate under study. These results contribute to an increasingly refined understanding of the mechanisms underlying the selection of the specific neuronal populations required for task execution.

Adding dynamics to the Human Connectome Project with MEG.

The Human Connectome Project (HCP) seeks to map the structural and functional connections between network elements in the human brain. Magnetoencephalography (MEG) provides a temporally rich source of information on brain network dynamics and represents one source of functional connectivity data to be provided by the HCP. High quality MEG data will be collected from 50 twin pairs both in the resting state and during performance of motor, working memory and language tasks. These data will be available to the general community. Additionally, using the cortical parcellation scheme common to all imaging modalities, the HCP will provide processing pipelines for calculating connection matrices as a function of time and frequency. Together with structural and functional data generated using magnetic resonance imaging methods, these data represent a unique opportunity to investigate brain network connectivity in a large cohort of normal adult human subjects. The analysis pipeline software and the dynamic connectivity matrices that it generates will all be made freely available to the research community.

The Human Connectome Project: a data acquisition perspective.

The Human Connectome Project (HCP) is an ambitious 5-year effort to characterize brain connectivity and function and their variability in healthy adults. This review summarizes the data acquisition plans being implemented by a consortium of HCP investigators who will study a population of 1200 subjects (twins and their non-twin siblings) using multiple imaging modalities along with extensive behavioral and genetic data. The imaging modalities will include diffusion imaging (dMRI), resting-state fMRI (R-fMRI), task-evoked fMRI (T-fMRI), T1- and T2-weighted MRI for structural and myelin mapping, plus combined magnetoencephalography and electroencephalography (MEG/EEG). Given the importance of obtaining the best possible data quality, we discuss the efforts underway during the first two years of the grant (Phase I) to refine and optimize many aspects of HCP data acquisition, including a new 7T scanner, a customized 3T scanner, and improved MR pulse sequences.

Successful declarative memory formation is associated with ongoing activity during encoding in a distributed neocortical network related to working memory: a magnetoencephalography study.

The aim of the present study was to investigate the spatio-temporal characteristics of the neural correlates of declarative memory formation as assessed by the subsequent memory effect, i.e. the difference in encoding activity between subsequently remembered and subsequently forgotten items. Different operations could account for these effects. In particular, it has been proposed that successful memory formation depends on the organization of the information at the time of encoding, an operation accomplished by the working memory system. Consequently, functional magnetic resonance imaging studies have already shown that the very same regions that are involved in certain working memory processes are also involved in declarative memory formation. Here, we used magnetoencephalography to investigate whether the subsequent memory effects in these regions are present throughout picture stimulus presentation, postulating ongoing working memory operations as an effective factor. The results showed that subsequent memory effects began to appear after about 300 ms post stimulus onset over bilateral temporal areas and left parietal regions and were sustained throughout the recording epoch (1000 ms). Roughly parallel to these effects, we identified a left frontal subsequent memory effect, which, however, was less sustained than the other effects. In addition, we revealed a late subsequent memory effect over the right occipital region, which has not been described previously in the event-related potential literature. These sustained subsequent memory effects are suggestive of working memory processes that may enable deep semantic and perceptual processing. Additionally, contextually constrained visual perception after top-down modulation may account for a more efficient encoding of the complex scene.

Attention and movement-related motor cortex activation: a high-density EEG study of spatial stimulus-response compatibility.

Visual spatial attentional activation of motor areas has been documented in single cell neurophysiology and functional imaging studies of the brain. Here, we investigate a candidate event-related brain potential representing visuospatial attentional activity in motor areas of the cortex. The investigation aimed to elucidate the neural origin and the functional characteristics of this brain potential, which has been labelled N2cc and is typically observed in spatial stimulus-response compatibility tasks. High-density EEG was recorded in 10 subjects while they performed a Simon-type spatial stimulus-response compatibility task and a control task where the same stimuli were assigned to Go-Nogo response alternatives. The N2cc showed a time course parallel to the posteriorly distributed N2pc, associated with visuospatial selection. Scalp distribution and current source density reconstructions allowed a spatial separation of N2pc and centrally distributed N2cc and were compatible with a source for the N2cc in the lateral premotor cortex. Comparisons across tasks demonstrated that the N2cc depends on bilateral response readiness, ruling out an exclusively attentional interpretation. Instead, the activity appears associated with visuospatial attentional processes that serve the selection and suppression of competing responses, in accord with a function of the dorsal premotor cortex in response selection. Together, the results consolidate the N2cc as a new ERP component relevant to the investigation of visuospatial motor processes.

Proprioception-related evoked potentials: origin and sensitivity to movement parameters.

Reafferent electroencephalography (EEG) potentials evoked by active or passive movement are largely dependent on muscle spindle input, which projects to postrolandic sensory areas as well as the precentral motor cortex. The origin of these proprioception-related evoked potentials has previously been studied by using N20-P20 source locations of the median nerve somatosensory evoked potential as an landmark for postcentral area 3b. As this approach has yielded contradictory findings, likely due to spatial undersampling, we applied dipole source analysis on two independently collected sets of high-density EEG data, containing the proprioception-related N90 elicited by passive finger movement, and the N20-P20 elicited by median nerve stimulation. In addition, the influence of movement parameters on the N90 was explored by varying amplitude/duration and direction of passive movements. The results showed that the proprioceptive N90 component was not influenced by movement direction, but had a duration that covaried with the duration of the movement. Sources were localized in the precentral cortex, located on average 10 mm anterior to the N20-P20 sources. The latter result supports earlier claims that the motor cortex is involved in the generation of proprioception-related EEG potentials.

The five percent electrode system for high-resolution EEG and ERP measurements.

OBJECTIVE: A system for electrode placement is described. It is designed for studies on topography and source analysis of spontaneous and evoked EEG activity. METHOD: The proposed system is based on the extended International 10-20 system which contains 74 electrodes, and extends this system up to 345 electrode locations. RESULTS: The positioning and nomenclature of the electrode system is described, and a subset of locations is proposed as especially useful for modern EEG/ERP systems, often having 128 channels available. CONCLUSION: Similar to the extension of the 10-20 system to the 10-10 system ("10% system"), proposed in 1985, the goal of this new extension to a 10-5 system is to further promote standardization in high-resolution EEG studies.

Overlap of attention and movement-related activity in lateralized event-related brain potentials.

OBJECTIVE: In tasks that involve lateralized visuospatial attention and a lateralized motor response, the associated brain electrical potentials, i.e. the attention-related N2pc and the lateralized readiness potential, typically overlap at central scalp sites. The manifestation of the N2pc at central electrode sites is commonly attributed to electric volume conduction effects, assuming the N2pc to be generated in occipito-temporal brain areas. We evaluated this explanation in a simulation study. METHODS: Using a forward modeling approach with a realistically shaped volume conduction model, we calculated the range of amplitude ratios between occipital and central electrode sites when a single source is assumed in area V4 or in area TO, at the temporo-occipital convexity. RESULTS: A comparison of the simulated amplitude ratios with reported data indicates that volume conduction effects from the investigated source origins in the occipito-temporal region are insufficient to explain the experimental data. CONCLUSIONS: We conclude that the anterior spread of the N2pc from its occipito-temporal maximum to central electrode sites is probably due to simultaneous attention-related activity in posterior and central brain areas.

Magnetic stimulation-induced modulations of motor unit firings extracted from multi-channel surface EMG.

The noninvasive assessment of motor unit (MU) firing patterns on the basis of topographical information from 128-channel high-density surface electromyography (SEMG) is reported. First, multi-channel MU action potential (MUAP) templates are obtained by clustering detected firing events according to the surface topography of the MUAP. Second, a template-matching algorithm is used to find all firings of a MU, including the superimpositions of MUAPs. From a single recording, the firing pattern of up to five MUs could be derived. The modulation of MU firing by transcranial magnetic stimulation was analyzed in peri-stimulus time histograms. The results are similar to previous results of transcranial magnetic stimulation (TMS) obtained by needle electromyographic (EMG) recordings. The method can be used to investigate MU firing patterns in patients with central motor disorders. An additional advantage of the technique, apart from its noninvasiveness, is the structural and functional information that it provides on the MUs, which is not obtained by needle EMG.

Increased auditory cortical representation in musicians.

Acoustic stimuli are processed throughout the auditory projection pathway, including the neocortex, by neurons that are aggregated into 'tonotopic' maps according to their specific frequency tunings. Research on animals has shown that tonotopic representations are not statically fixed in the adult organism but can reorganize after damage to the cochlea or after training the intact subject to discriminate between auditory stimuli. Here we used functional magnetic source imaging (single dipole model) to measure cortical representations in highly skilled musicians. Dipole moments for piano tones, but not for pure tones of similar fundamental frequency (matched in loudness), were found to be enlarged by about 25% in musicians compared with control subjects who had never played an instrument. Enlargement was correlated with the age at which musicians began to practise and did not differ between musicians with absolute or relative pitch. These results, when interpreted with evidence for modified somatosensory representations of the fingering digits in skilled violinists, suggest that use-dependent functional reorganization extends across the sensory cortices to reflect the pattern of sensory input processed by the subject during development of musical skill.