Evaluating Speaker-Listener Cognitive Effort in Speech Communication Through Brain-to-Brain Synchrony: A Pilot Functional Near-Infrared Spectroscopy Investigation.

PURPOSE: We explore a new approach to the study of cognitive effort involved in listening to speech by measuring the brain activity in a listener in relation to the brain activity in a speaker. We hypothesize that the strength of this brain-to-brain synchrony (coupling) reflects the magnitude of cognitive effort involved in verbal communication and includes both listening effort and speaking effort. We investigate whether interbrain synchrony is greater in native-to-native versus native-to-nonnative communication using functional near-infrared spectroscopy (fNIRS). METHOD: Two speakers participated, a native speaker of American English and a native speaker of Korean who spoke English as a second language. Each speaker was fitted with the fNIRS cap and told short stories. The native English speaker provided the English narratives, and the Korean speaker provided both the nonnative (accented) English and Korean narratives. In separate sessions, fNIRS data were obtained from seven English monolingual participants ages 20-24 years who listened to each speaker's stories. After listening to each story in native and nonnative English, they retold the content, and their transcripts and audio recordings were analyzed for comprehension and discourse fluency, measured in the number of hesitations and articulation rate. No story retellings were obtained for narratives in Korean (an incomprehensible language for English listeners). Utilizing fNIRS technique termed sequential scanning, we quantified the brain-to-brain synchronization in each speaker-listener dyad. RESULTS: For native-to-native dyads, multiple brain regions associated with various linguistic and executive functions were activated. There was a weaker coupling for native-to-nonnative dyads, and only the brain regions associated with higher order cognitive processes and functions were synchronized. All listeners understood the content of all stories, but they hesitated significantly more when retelling stories told in accented English. The nonnative speaker hesitated significantly more often than the native speaker and had a significantly slower articulation rate. There was no brain-to-brain coupling during listening to Korean, indicating a break in communication when listeners failed to comprehend the speaker. CONCLUSIONS: We found that effortful speech processing decreased interbrain synchrony and delayed comprehension processes. The obtained brain-based and behavioral patterns are consistent with our proposal that cognitive effort in verbal communication pertains to both the listener and the speaker and that brain-to-brain synchrony can be an indicator of differences in their cumulative communicative effort. SUPPLEMENTAL MATERIAL: https://doi.org/10.23641/asha.25452142.

Editorial: Signal Processing for Brain-Computer Interfaces-Special Issue.

With the astounding ability to capture a wealth of brain signals, Brain-Computer Interfaces (BCIs) have the potential to revolutionize humans' quality of life [...].

Brain space image reconstruction of functional near-infrared spectroscopy using a Bayesian adaptive fused sparse overlapping group lasso model.

Significance: Functional near-infrared spectroscopy (fNIRS) is a noninvasive technology that uses low levels of nonionizing light in the range of red and near-infrared to record changes in the optical absorption and scattering of the underlying tissue that can be used to infer blood flow and oxygen changes during brain activity. The challenges and difficulties of reconstructing spatial images of hemoglobin changes from fNIRS data are mainly caused by the illposed nature of the optical inverse model. Aim: We describe a Bayesian approach combining several lasso-based regularizations to apply anatomy-prior information to solving the inverse model. Approach: We built a Bayesian hierarchical model to solve the Bayesian adaptive fused sparse overlapping group lasso (Ba-FSOGL) model. The method is evaluated and validated using simulation and experimental datasets. Results: We apply this approach to the simulation and experimental datasets to reconstruct a known brain activity. The reconstructed images and statistical plots are shown. Conclusion: We discuss the adaptation of this method to fNIRS data and demonstrate that this approach provides accurate image reconstruction with a low false-positive rate, through numerical simulations and application to experimental data collected during motor and sensory tasks.

Mother-child neural synchronization is time linked to mother-child positive affective state matching.

In the first years of life, in which self-regulation occurs via external means, mother-child synchronization of positive affect (PA) facilitates regulation of child homeostatic systems. Mother-child affective synchrony may contribute to mother-child synchronization of neural systems, but limited research has explored this possibility. Participants were 41 healthy mother-child dyads (56% girls; Mage = 24.76 months; s.d. = 8.77 months, Range = 10-42 months). Mothers' and children's brain activities were assessed simultaneously using near-infrared spectroscopy while engaging in dyadic play. Mother and child PA during play were coded separately to characterize periods in which mothers and children (i) matched on high PA, (ii) matched on low/no PA or (iii) showed a mismatch in PA. Models evaluated moment-to-moment correlations between affective matching and neural synchrony in mother-child dyads. Greater positive affective synchrony, in which mother and child showed similarly high levels of PA but not similarly low levels of PA, was related to greater synchrony in medial and lateral frontal and temporoparietal regions. Age moderated associations between mother and child neural activities but only during moments of high PA state matching. Positive, synchronous mother-child interactions may foster greater neural responding in affective and social regions important for self-regulation and interpersonal bonds.

Assessing the relationship between maternal risk for attention deficit hyperactivity disorder and functional connectivity in their biological toddlers.

BACKGROUND: Attention deficit hyperactivity disorder (ADHD) is a neurodevelopmental disorder associated with increased risk for poor educational attainment and compromised social integration. Currently, clinical diagnosis rarely occurs before school-age, despite behavioral signs of ADHD in very early childhood. There is no known brain biomarker for ADHD risk in children ages 2-3 years-old. METHODS: The current study aimed to investigate the functional connectivity (FC) associated with ADHD risk in 70 children aged 2.5 and 3.5 years via functional near-infrared spectroscopy (fNIRS) in bilateral frontal and parietal cortices; regions involved in attentional and goal-directed cognition. Children were instructed to passively watch videos for approximately 5 min. Risk for ADHD in each child was assessed via maternal symptoms of ADHD, and brain data was evaluated for FC. RESULTS: Higher risk for maternal ADHD was associated with lower FC in a left-sided parieto-frontal network. Further, the interaction between sex and risk for ADHD was significant, where FC reduction in a widespread bilateral parieto-frontal network was associated with higher risk in male, but not female, participants. CONCLUSIONS: These findings suggest functional organization differences in the parietal-frontal network in toddlers at risk for ADHD; potentially advancing the understanding of the neural mechanisms underlying the development of ADHD.

To snack or not to snack: Using fNIRS to link inhibitory control to functional connectivity in the toddler brain.

Inhibitory control (IC) emerges in infancy, continues to develop throughout childhood and is linked to later life outcomes such as school achievement, prosocial behavior, and psychopathology. Little, however, is known about the neural processes underpinning IC, especially in 2-year-olds. In this study, we examine functional connectivity (FC) in 2.5-year-olds while recording hemodynamic responses via functional infrared spectroscopy (fNIRS) during a traditional snack delay task. We found that functional connectivity strength between left frontal and parietal cortex and bilateral parietal cortex were positively associated with performance on this task. The current findings present the first neural data for toddlers during this IC task. Further, these data are the first to link this self-regulatory process to differences in brain development within this population. Implications for future directions and work with clinical populations are discussed.

Offset analgesia is associated with opposing modulation of medial versus dorsolateral prefrontal cortex activations: A functional near-infrared spectroscopy study.

Offset analgesia is defined by a dramatic drop in perceived pain intensity with a relatively small decrease in noxious input. Although functional magnetic resonance imaging studies implicate subcortical descending inhibitory circuits during offset analgesia, the role of cortical areas remains unclear. The current study identifies cortical correlates of offset analgesia using functional near infrared spectroscopy (fNIRS). Twenty-four healthy volunteers underwent fNIRS scanning during offset (OS) and control (Con) heat stimuli applied to the forearm. After controlling for non-neural hemodynamic responses in superficial tissues, widespread increases in cortical oxygenated hemoglobin concentration were observed, reflecting cortical activation during heat pain. OS-Con contrasts revealed deactivations in bilateral medial prefrontal cortex (mPFC) and bilateral somatosensory cortex (SSC) associated with offset analgesia. Right dorsolateral prefrontal cortex (dlPFC) showed activation only during OS. These data demonstrate opposing cortical activation patterns during offset analgesia and support a model in which right dlPFC underlies ongoing evaluation of pain intensity change. With predictions of decreasing pain intensity, right dlPFC activation likely inhibits ascending noxious input via subcortical pathways resulting in SSC and mPFC deactivation. This study identifies cortical circuitry underlying offset analgesia and introduces the use of fNIRS to study pain modulation in an outpatient clinical environment.

Postpartum Depression Is Associated With Altered Neural Connectivity Between Affective and Mentalizing Regions During Mother-Infant Interactions.

Although there has been growing interest in mood-related neural alterations in women in the initial weeks postpartum, recent work has demonstrated that postpartum depression often lingers for months or years following birth. However, research evaluating the impact of depression on maternal brain function during mother-infant interactions in the late postpartum period is lacking. The current study tested the hypothesis that depressive symptoms at 12-months postpartum are associated with neural alterations in affective and social neural regions, using near-infrared spectroscopy during in vivo mother-infant interactions. Participants were 23 birth mothers of 12-month-old infants (60% boys). While undergoing near-infrared spectroscopy, mothers engaged in an ecologically valid interactive task in which they looked at an age-appropriate book with their infants. Mothers also reported on their depressive symptoms in the past week and were rated on their observed levels of maternal sensitivity during mother-infant play. Greater depressive severity at 12-months postpartum was related to lower connectivity between the right temporoparietal junction and the lateral prefrontal cortex, but greater connectivity between the right temporoparietal junction and anterior medial prefrontal cortex during mother-infant interaction. Given the putative functions of these neural regions within the maternal brain network, our findings suggest that in the context of depression, postpartum mothers' mentalizing about her infants' thoughts and feelings may be related to lower ability to express and regulate her own emotions, but greater ability to engage in emotional bonding with her infant. Future work should explore how connectivity among these regions is associated with longitudinal changes in maternal behavior, especially in the context of changes in mothers' depressive symptoms (e.g., with treatment) over time.

Decoding Multiple Sound-Categories in the Auditory Cortex by Neural Networks: An fNIRS Study.

This study aims to decode the hemodynamic responses (HRs) evoked by multiple sound-categories using functional near-infrared spectroscopy (fNIRS). The six different sounds were given as stimuli (English, non-English, annoying, nature, music, and gunshot). The oxy-hemoglobin (HbO) concentration changes are measured in both hemispheres of the auditory cortex while 18 healthy subjects listen to 10-s blocks of six sound-categories. Long short-term memory (LSTM) networks were used as a classifier. The classification accuracy was 20.38 ± 4.63% with six class classification. Though LSTM networks' performance was a little higher than chance levels, it is noteworthy that we could classify the data subject-wise without feature selections.

Using anatomically defined regions-of-interest to adjust for head-size and probe alignment in functional near-infrared spectroscopy.

Significance: Functional near-infrared spectroscopy (fNIRS) uses surface-placed light sources and detectors to record underlying changes in the brain due to fluctuations in hemoglobin levels and oxygenation. Since these measurements are recorded from the surface of the scalp, the mapping from underlying regions-of-interest (ROIs) in the brain space to the fNIRS channel space measurements depends on the registration of the sensors, the anatomy of the head/brain, and the sensitivity of these diffuse measurements through the tissue. However, small displacements in the probe position can change the distribution of recorded brain activity across the fNIRS measurements. Aim: We propose an approach using either individual or atlas-based brain-space anatomical information to define ROI-based statistical hypotheses to test the null involvement of specific regions, which allows us to test the analogous ROI across subjects while adjusting for fNIRS probe placement and sensitivity differences due to head size variations without a localizer task. Approach: We use the optical forward model to project the underlying brain-space ROI into a tapered contrast vector, which defines the relative weighting of the fNIRS channels contributing to the ROI and allows us to test the null hypothesis of no brain activity in this region during a functional task. We demonstrate this method through simulation and compare the sensitivity-specificity of this approach to other conventional methods. Results: We examine the performance of this method in the scenario where head size and probe registration are both an accurately known parameters and where this is subject to unknown experimental errors. This method is compared with the performance of the conventional method using 364 different simulation parameter combinations. Conclusion: The proposed method is always recommended in ROI-based analysis, since it significantly improves the analysis performance without a localizer task, wherever the fNIRS probe registration is known or unknown.

Quantitative comparison of correction techniques for removing systemic physiological signal in functional near-infrared spectroscopy studies.

Significance: Isolating task-evoked brain signals from background physiological noise (e.g., cardiac, respiratory, and blood pressure fluctuations) poses a major challenge for the analysis of functional near-infrared spectroscopy (fNIRS) data. Aim: The performance of several analytic methods to separate background physiological noise from brain activity including spatial and temporal filtering, regression, component analysis, and the use of short-separation (SS) measurements were quantitatively compared. Approach: Using experimentally recorded background signals (breath-hold task), receiver operating characteristics simulations were performed by adding various levels of additive synthetic "brain" responses in order to examine the sensitivity and specificity of several previously proposed analytic approaches. Results: We found that the use of SS fNIRS channels as regressors of no-interest within a linear regression model was the best performing approach examined. Furthermore, we found that the addition of all available SS data, including all recorded channels and both hemoglobin species, improved the method performance despite the additional degrees-of-freedom of the models. When SS data were not available, we found that principal component filtering using a separate baseline scan was the best alternative. Conclusions: The use of multiple SS measurements as regressors of no interest implemented in a robust, iteratively prewhitened, general linear model has the best performance of the tested existing methods.

Investigation of the sensitivity-specificity of canonical- and deconvolution-based linear models in evoked functional near-infrared spectroscopy.

Functional near-infrared spectroscopy (fNIRS) is a noninvasive brain imaging technique to measure evoked changes in cerebral blood oxygenation. In many evoked-task studies, the analysis of fNIRS experiments is based on a temporal linear regression model, which includes block-averaging, deconvolution, and canonical analysis models. The statistical parameters of this model are then spatially mapped across fNIRS measurement channels to infer brain activity. The trade-offs in sensitivity and specificity of using variations of canonical or deconvolution/block-average models are unclear. We quantitatively investigate how the choice of basis set for the regression linear model affects the sensitivity and specificity of fNIRS analysis in the presence of variability or systematic bias in underlying evoked response. For statistical parametric mapping of amplitude-based hypotheses, we conclude that these models are fairly insensitive to the parameters of the regression basis for task durations > 10 s and we report the highest sensitivity-specificity results using a low degree-of-freedom canonical model under these conditions. For shorter duration task ( < 10 s ), the signal-to-noise ratio of the data is also important in this decision and we find that deconvolution or block-averaging models outperform the canonical models at high signal-to-noise ratio but not at lower levels.

Investigation of the sensitivity of functional near-infrared spectroscopy brain imaging to anatomical variations in 5- to 11-year-old children.

Functional near-infrared spectroscopy (fNIRS) is a noninvasive brain imaging technique that uses scalp-placed light sensors to measure evoked changes in cerebral blood oxygenation. The portability, low overhead cost, and ability to use this technology under a wide range of experimental environments make fNIRS well-suited for studies involving infants and children. However, since fNIRS does not directly provide anatomical or structural information, these measurements may be sensitive to individual or group level differences associated with variations in head size, depth of the brain from the scalp, or other anatomical factors affecting the penetration of light into the head. This information is generally not available in pediatric populations, which are often the target of study for fNIRS. Anatomical magnetic resonance imaging information from 90 school-age children (5 to 11 years old) was used to quantify the expected effect on fNIRS measures of variations in cerebral and extracerebral structure. Monte Carlo simulations of light transport in tissue were used to estimate differential and partial optical pathlengths at 690, 780, 808, 830, and 850 nm and their variations with age, sex, and head size. This work provides look-up tables of these values and general guidance for future investigations using fNIRS sans anatomical information in this child population.

The NIRS Brain AnalyzIR Toolbox.

Functional near-infrared spectroscopy (fNIRS) is a noninvasive neuroimaging technique that uses low-levels of light (650-900 nm) to measure changes in cerebral blood volume and oxygenation. Over the last several decades, this technique has been utilized in a growing number of functional and resting-state brain studies. The lower operation cost, portability, and versatility of this method make it an alternative to methods such as functional magnetic resonance imaging for studies in pediatric and special populations and for studies without the confining limitations of a supine and motionless acquisition setup. However, the analysis of fNIRS data poses several challenges stemming from the unique physics of the technique, the unique statistical properties of data, and the growing diversity of non-traditional experimental designs being utilized in studies due to the flexibility of this technology. For these reasons, specific analysis methods for this technology must be developed. In this paper, we introduce the NIRS Brain AnalyzIR toolbox as an open-source Matlab-based analysis package for fNIRS data management, pre-processing, and first- and second-level (i.e., single subject and group-level) statistical analysis. Here, we describe the basic architectural format of this toolbox, which is based on the object-oriented programming paradigm. We also detail the algorithms for several of the major components of the toolbox including statistical analysis, probe registration, image reconstruction, and region-of-interest based statistics.

Characterization and correction of the false-discovery rates in resting state connectivity using functional near-infrared spectroscopy.

Functional near-infrared spectroscopy (fNIRS) is a noninvasive neuroimaging technique that uses low levels of red to near-infrared light to measure changes in cerebral blood oxygenation. Spontaneous (resting state) functional connectivity (sFC) has become a critical tool for cognitive neuroscience for understanding task-independent neural networks, revealing pertinent details differentiating healthy from disordered brain function, and discovering fluctuations in the synchronization of interacting individuals during hyperscanning paradigms. Two of the main challenges to sFC-NIRS analysis are (i) the slow temporal structure of both systemic physiology and the response of blood vessels, which introduces false spurious correlations, and (ii) motion-related artifacts that result from movement of the fNIRS sensors on the participants' head and can introduce non-normal and heavy-tailed noise structures. In this work, we systematically examine the false-discovery rates of several time- and frequency-domain metrics of functional connectivity for characterizing sFC-NIRS. Specifically, we detail the modifications to the statistical models of these methods needed to avoid high levels of false-discovery related to these two sources of noise in fNIRS. We compare these analysis procedures using both simulated and experimental resting-state fNIRS data. Our proposed robust correlation method has better performance in terms of being more reliable to the noise outliers due to the motion artifacts.

Decoding four different sound-categories in the auditory cortex using functional near-infrared spectroscopy.

The ability of the auditory cortex in the brain to distinguish different sounds is important in daily life. This study investigated whether activations in the auditory cortex caused by different sounds can be distinguished using functional near-infrared spectroscopy (fNIRS). The hemodynamic responses (HRs) in both hemispheres using fNIRS were measured in 18 subjects while exposing them to four sound categories (English-speech, non-English-speech, annoying sounds, and nature sounds). As features for classifying the different signals, the mean, slope, and skewness of the oxy-hemoglobin (HbO) signal were used. With regard to the language-related stimuli, the HRs evoked by understandable speech (English) were observed in a broader brain region than were those evoked by non-English speech. Also, the magnitudes of the HbO signals evoked by English-speech were higher than those of non-English speech. The ratio of the peak values of non-English and English speech was 72.5%. Also, the brain region evoked by annoying sounds was wider than that by nature sounds. However, the signal strength for nature sounds was stronger than that for annoying sounds. Finally, for brain-computer interface (BCI) purposes, the linear discriminant analysis (LDA) and support vector machine (SVM) classifiers were applied to the four sound categories. The overall classification performance for the left hemisphere was higher than that for the right hemisphere. Therefore, for decoding of auditory commands, the left hemisphere is recommended. Also, in two-class classification, the annoying vs. nature sounds comparison provides a higher classification accuracy than the English vs. non-English speech comparison. Finally, LDA performs better than SVM.

Lateralization of music processing with noises in the auditory cortex: an fNIRS study.

The present study is to determine the effects of background noise on the hemispheric lateralization in music processing by exposing 14 subjects to four different auditory environments: music segments only, noise segments only, music + noise segments, and the entire music interfered by noise segments. The hemodynamic responses in both hemispheres caused by the perception of music in 10 different conditions were measured using functional near-infrared spectroscopy. As a feature to distinguish stimulus-evoked hemodynamics, the difference between the mean and the minimum value of the hemodynamic response for a given stimulus was used. The right-hemispheric lateralization in music processing was about 75% (instead of continuous music, only music segments were heard). If the stimuli were only noises, the lateralization was about 65%. But, if the music was mixed with noises, the right-hemispheric lateralization has increased. Particularly, if the noise was a little bit lower than the music (i.e., music level 10~15%, noise level 10%), the entire subjects showed the right-hemispheric lateralization: This is due to the subjects' effort to hear the music in the presence of noises. However, too much noise has reduced the subjects' discerning efforts.

Asymmetry brain function in auditory cortex: a functional near-infrared spectroscopy study.

Functional near-infrared spectroscopy (fNIRS) can measure the change of hemodynamic response, it enables to determine the concentration changes of oxy-hemoglobin and deoxy-hemoglobin. The aim in this paper is to investigate the forms of lateralization or asymmetry brain function in auditory cortex using fNIRS. This technique shows good promise for assessment of asymmetry functions in the auditory cortex.

Noise reduction in functional near-infrared spectroscopy signals by independent component analysis.

Functional near-infrared spectroscopy (fNIRS) is used to detect concentration changes of oxy-hemoglobin and deoxy-hemoglobin in the human brain. The main difficulty entailed in the analysis of fNIRS signals is the fact that the hemodynamic response to a specific neuronal activation is contaminated by physiological and instrument noises, motion artifacts, and other interferences. This paper proposes independent component analysis (ICA) as a means of identifying the original hemodynamic response in the presence of noises. The original hemodynamic response was reconstructed using the primary independent component (IC) and other, less-weighting-coefficient ICs. In order to generate experimental brain stimuli, arithmetic tasks were administered to eight volunteer subjects. The t-value of the reconstructed hemodynamic response was improved by using the ICs found in the measured data. The best t-value out of 16 low-pass-filtered signals was 37, and that of the reconstructed one was 51. Also, the average t-value of the eight subjects' reconstructed signals was 40, whereas that of all of their low-pass-filtered signals was only 20. Overall, the results showed the applicability of the ICA-based method to noise-contamination reduction in brain mapping.