Simultaneous multimodal fNIRS-EEG recordings reveal new insights in neural activity during motor execution, observation, and imagery.

Motor execution, observation, and imagery are important skills used in motor learning and rehabilitation. The neural mechanisms underlying these cognitive-motor processes are still poorly understood. We used a simultaneous recording of functional near-infrared spectroscopy (fNIRS) and electroencephalogram (EEG) to elucidate the differences in neural activity across three conditions requiring these processes. Additionally, we used a new method called structured sparse multiset Canonical Correlation Analysis (ssmCCA) to fuse the fNIRS and EEG data and determine the brain regions of neural activity consistently detected by both modalities. Unimodal analyses revealed differentiated activation between conditions; however, the activated regions did not fully overlap across the two modalities (fNIRS: left angular gyrus, right supramarginal gyrus, as well as right superior and inferior parietal lobes; EEG: bilateral central, right frontal, and parietal). These discrepancies might be because fNIRS and EEG detect different signals. Using fused fNIRS-EEG data, we consistently found activation over the left inferior parietal lobe, superior marginal gyrus, and post-central gyrus during all three conditions, suggesting that our multimodal approach identifies a shared neural region associated with the Action Observation Network (AON). This study highlights the strengths of using the multimodal fNIRS-EEG fusion technique for studying AON. Neural researchers should consider using the multimodal approach to validate their findings.

Structured sparse multiset canonical correlation analysis of simultaneous fNIRS and EEG provides new insights into the human action-observation network.

The action observation network (AON) is a network of brain regions involved in the execution and observation of a given action. The AON has been investigated in humans using mostly electroencephalogram (EEG) and functional magnetic resonance imaging (fMRI), but shared neural correlates of action observation and action execution are still unclear due to lack of ecologically valid neuroimaging measures. In this study, we used concurrent EEG and functional Near Infrared Spectroscopy (fNIRS) to examine the AON during a live-action observation and execution paradigm. We developed structured sparse multiset canonical correlation analysis (ssmCCA) to perform EEG-fNIRS data fusion. MCCA is a generalization of CCA to more than two sets of variables and is commonly used in medical multimodal data fusion. However, mCCA suffers from multi-collinearity, high dimensionality, unimodal feature selection, and loss of spatial information in interpreting the results. A limited number of participants (small sample size) is another problem in mCCA, which leads to overfitted models. Here, we adopted graph-guided (structured) fused least absolute shrinkage and selection operator (LASSO) penalty to mCCA to conduct feature selection, incorporating structural information amongst the variables (i.e., brain regions). Benefitting from concurrent recordings of brain hemodynamic and electrophysiological responses, the proposed ssmCCA finds linear transforms of each modality such that the correlation between their projections is maximized. Our analysis of 21 right-handed participants indicated that the left inferior parietal region was active during both action execution and action observation. Our findings provide new insights into the neural correlates of AON which are more fine-tuned than the results from each individual EEG or fNIRS analysis and validate the use of ssmCCA to fuse EEG and fNIRS datasets.

Cerebral hemodynamic response during a live action-observation and action-execution task: A fNIRS study.

Although many studies have examined the location of the action observation network (AON) in human adults, the shared neural correlates of action-observation and action-execution are still unclear partially due to lack of ecologically valid neuroimaging measures. In this study, we aim to demonstrate the feasibility of using functional near infrared spectroscopy (fNIRS) to measure the neural correlates of action-observation and action execution regions during a live task. Thirty adults reached for objects or observed an experimenter reaching for objects while their cerebral hemodynamic responses including oxy-hemoglobin (HbO) and deoxy-hemoglobin (HbR) were recorded in the sensorimotor and parietal regions. Our results indicated that the parietal regions, including bilateral superior parietal lobule (SPL), bilateral inferior parietal lobule (IPL), right supra-marginal region (SMG) and right angular gyrus (AG) share neural activity during action-observation and action-execution. Our findings confirm the applicability of fNIRS for the study of the AON and lay the foundation for future work with developmental and clinical populations.

Characterizing the Action-Observation Network Through Functional Near-Infrared Spectroscopy: A Review.

Functional near-infrared spectroscopy (fNIRS) is a neuroimaging technique that has undergone tremendous growth over the last decade due to methodological advantages over other measures of brain activation. The action-observation network (AON), a system of brain structures proposed to have "mirroring" abilities (e.g., active when an individual completes an action or when they observe another complete that action), has been studied in humans through neural measures such as fMRI and electroencephalogram (EEG); however, limitations of these methods are problematic for AON paradigms. For this reason, fNIRS is proposed as a solution to investigating the AON in humans. The present review article briefly summarizes previous neural findings in the AON and examines the state of AON research using fNIRS in adults. A total of 14 fNIRS articles are discussed, paying particular attention to methodological choices and considerations while summarizing the general findings to aid in developing better protocols to study the AON through fNIRS. Additionally, future directions of this work are discussed, specifically in relation to researching AON development and potential multimodal imaging applications.