It takes a village: A multi-brain approach to studying multigenerational family communication.

Grandparents play a critical role in child rearing across the globe. Yet, there is a shortage of neurobiological research examining the relationship between grandparents and their grandchildren. We employ multi-brain neurocomputational models to simulate how changes in neurophysiological processes in both development and healthy aging affect multigenerational inter-brain coupling - a neural marker that has been linked to a range of socio-emotional and cognitive outcomes. The simulations suggest that grandparent-child interactions may be paired with higher inter-brain coupling than parent-child interactions, raising the possibility that the former may be more advantageous under certain conditions. Critically, this enhancement of inter-brain coupling for grandparent-child interactions is more pronounced in tri-generational interactions that also include a parent, which may speak to findings that grandparent involvement in childrearing is most beneficial if the parent is also an active household member. Together, these findings underscore that a better understanding of the neurobiological basis of cross-generational interactions is vital, and that such knowledge can be helpful in guiding interventions that consider the whole family. We advocate for a community neuroscience approach in developmental social neuroscience to capture the diversity of child-caregiver relationships in real-world settings.

The Temporal Dynamics of Brain-to-Brain Synchrony Between Students and Teachers Predict Learning Outcomes.

Much of human learning happens through interaction with other people, but little is known about how this process is reflected in the brains of students and teachers. Here, we concurrently recorded electroencephalography (EEG) data from nine groups, each of which contained four students and a teacher. All participants were young adults from the northeast United States. Alpha-band (8-12 Hz) brain-to-brain synchrony between students predicted both immediate and delayed posttest performance. Further, brain-to-brain synchrony was higher in specific lecture segments associated with questions that students answered correctly. Brain-to-brain synchrony between students and teachers predicted learning outcomes at an approximately 300-ms lag in the students' brain activity relative to the teacher's brain activity, which is consistent with the time course of spoken-language comprehension. These findings provide key new evidence for the importance of collecting brain data simultaneously from groups of learners in ecologically valid settings.

Trait mindful awareness predicts inter-brain coupling but not individual brain responses during naturalistic face-to-face interactions.

In recent years, the possible benefits of mindfulness meditation have sparked much public and academic interest. Mindfulness emphasizes cultivating awareness of our immediate experience and has been associated with compassion, empathy, and various other prosocial traits. However, neurobiological evidence pertaining to the prosocial benefits of mindfulness in social settings is sparse. In this study, we investigate neural correlates of trait mindful awareness during naturalistic dyadic interactions, using both intra-brain and inter-brain measures. We used the Muse headset, a portable electroencephalogram (EEG) device often used to support mindfulness meditation, to record brain activity from dyads as they engaged in naturalistic face-to-face interactions in a museum setting. While we did not replicate prior laboratory-based findings linking trait mindfulness to individual brain responses (N = 379 individuals), self-reported mindful awareness did predict dyadic inter-brain synchrony, in theta (~5-8 Hz) and beta frequencies (~26-27 Hz; N = 62 dyads). These findings underscore the importance of conducting social neuroscience research in ecological settings to enrich our understanding of how (multi-brain) neural correlates of social traits such as mindful awareness manifest during social interaction, while raising critical practical considerations regarding the viability of commercially available EEG systems.

Instructor-learner body coupling reflects instruction and learning.

It is widely accepted that nonverbal communication is crucial for learning, but the exact functions of interpersonal coordination between instructors and learners remain unclear. Specifically, it is unknown what role instructional approaches play in the coupling of physical motion between instructors and learners, and crucially, how such instruction-mediated Body-to-Body Coupling (BtBC) might affect learning. We used a video-based, computer-vision Motion Energy Analysis (MEA) to quantify BtBC between learners and instructors who used two different instructional approaches to teach psychological concepts. BtBC was significantly greater when the instructor employed a scaffolding approach than when an explanation approach was used. The importance of the instructional approach was further underscored by the fact that an increase in motion in the instructor was associated with boosted BtBC, but only during scaffolding; no such relationship between the instructor movements and BtBC was found during explanation interactions. Finally, leveraging machine learning approaches (i.e., support vector and logistic regression models), we demonstrated that both learning outcome and instructional approaches could be decoded based on BtBC. Collectively, these results show that the real-time interaction of teaching and learning bodies is important for learning and that the instructional approach matters, with possible implications for both in-person and online learning.

Resonance as a Design Strategy for AI and Social Robots.

Resonance, a powerful and pervasive phenomenon, appears to play a major role in human interactions. This article investigates the relationship between the physical mechanism of resonance and the human experience of resonance, and considers possibilities for enhancing the experience of resonance within human-robot interactions. We first introduce resonance as a widespread cultural and scientific metaphor. Then, we review the nature of "sympathetic resonance" as a physical mechanism. Following this introduction, the remainder of the article is organized in two parts. In part one, we review the role of resonance (including synchronization and rhythmic entrainment) in human cognition and social interactions. Then, in part two, we review resonance-related phenomena in robotics and artificial intelligence (AI). These two reviews serve as ground for the introduction of a design strategy and combinatorial design space for shaping resonant interactions with robots and AI. We conclude by posing hypotheses and research questions for future empirical studies and discuss a range of ethical and aesthetic issues associated with resonance in human-robot interactions.

Cooperative Behavior Evokes Interbrain Synchrony in the Prefrontal and Temporoparietal Cortex: A Systematic Review and Meta-Analysis of fNIRS Hyperscanning Studies.

Single-brain neuroimaging studies have shown that human cooperation is associated with neural activity in frontal and temporoparietal regions. However, it remains unclear whether single-brain studies are informative about cooperation in real life, where people interact dynamically. Such dynamic interactions have become the focus of interbrain studies. An advantageous technique in this regard is functional near-infrared spectroscopy (fNIRS) because it is less susceptible to movement artifacts than more conventional techniques like electroencephalography (EEG) or functional magnetic resonance imaging (fMRI). We conducted a systematic review and the first quantitative meta-analysis of fNIRS hyperscanning of cooperation, based on thirteen studies with 890 human participants. Overall, the meta-analysis revealed evidence of statistically significant interbrain synchrony while people were cooperating, with large overall effect sizes in both frontal and temporoparietal areas. All thirteen studies observed significant interbrain synchrony in the prefrontal cortex (PFC), suggesting that this region is particularly relevant for cooperative behavior. The consistency in these findings is unlikely to be because of task-related activations, given that the relevant studies used diverse cooperation tasks. Together, the present findings support the importance of interbrain synchronization of frontal and temporoparietal regions in interpersonal cooperation. Moreover, the present article highlights the usefulness of meta-analyses as a tool for discerning patterns in interbrain dynamics.

Opportunities and Limitations of Mobile Neuroimaging Technologies in Educational Neuroscience.

As the field of educational neuroscience continues to grow, questions have emerged regarding the ecological validity and applicability of this research to educational practice. Recent advances in mobile neuroimaging technologies have made it possible to conduct neuroscientific studies directly in naturalistic learning environments. We propose that embedding mobile neuroimaging research in a cycle (Matusz, Dikker, Huth, & Perrodin, 2019), involving lab-based, seminaturalistic, and fully naturalistic experiments, is well suited for addressing educational questions. With this review, we take a cautious approach, by discussing the valuable insights that can be gained from mobile neuroimaging technology, including electroencephalography and functional near-infrared spectroscopy, as well as the challenges posed by bringing neuroscientific methods into the classroom. Research paradigms used alongside mobile neuroimaging technology vary considerably. To illustrate this point, studies are discussed with increasingly naturalistic designs. We conclude with several ethical considerations that should be taken into account in this unique area of research.

Crowdsourcing neuroscience: Inter-brain coupling during face-to-face interactions outside the laboratory.

When we feel connected or engaged during social behavior, are our brains in fact "in sync" in a formal, quantifiable sense? Most studies addressing this question use highly controlled tasks with homogenous subject pools. In an effort to take a more naturalistic approach, we collaborated with art institutions to crowdsource neuroscience data: Over the course of 5 years, we collected electroencephalogram (EEG) data from thousands of museum and festival visitors who volunteered to engage in a 10-min face-to-face interaction. Pairs of participants with various levels of familiarity sat inside the Mutual Wave Machine-an artistic neurofeedback installation that translates real-time correlations of each pair's EEG activity into light patterns. Because such inter-participant EEG correlations are prone to noise contamination, in subsequent offline analyses we computed inter-brain coupling using Imaginary Coherence and Projected Power Correlations, two synchrony metrics that are largely immune to instantaneous, noise-driven correlations. When applying these methods to two subsets of recorded data with the most consistent protocols, we found that pairs' trait empathy, social closeness, engagement, and social behavior (joint action and eye contact) consistently predicted the extent to which their brain activity became synchronized, most prominently in low alpha (~7-10 Hz) and beta (~20-22 Hz) oscillations. These findings support an account where shared engagement and joint action drive coupled neural activity and behavior during dynamic, naturalistic social interactions. To our knowledge, this work constitutes a first demonstration that an interdisciplinary, real-world, crowdsourcing neuroscience approach may provide a promising method to collect large, rich datasets pertaining to real-life face-to-face interactions. Additionally, it is a demonstration of how the general public can participate and engage in the scientific process outside of the laboratory. Institutions such as museums, galleries, or any other organization where the public actively engages out of self-motivation, can help facilitate this type of citizen science research, and support the collection of large datasets under scientifically controlled experimental conditions. To further enhance the public interest for the out-of-the-lab experimental approach, the data and results of this study are disseminated through a website tailored to the general public (wp.nyu.edu/mutualwavemachine).

HyPyP: a Hyperscanning Python Pipeline for inter-brain connectivity analysis.

The bulk of social neuroscience takes a 'stimulus-brain' approach, typically comparing brain responses to different types of social stimuli, but most of the time in the absence of direct social interaction. Over the last two decades, a growing number of researchers have adopted a 'brain-to-brain' approach, exploring similarities between brain patterns across participants as a novel way to gain insight into the social brain. This methodological shift has facilitated the introduction of naturalistic social stimuli into the study design (e.g. movies) and, crucially, has spurred the development of new tools to directly study social interaction, both in controlled experimental settings and in more ecologically valid environments. Specifically, 'hyperscanning' setups, which allow the simultaneous recording of brain activity from two or more individuals during social tasks, has gained popularity in recent years. However, currently, there is no agreed-upon approach to carry out such 'inter-brain connectivity analysis', resulting in a scattered landscape of analysis techniques. To accommodate a growing demand to standardize analysis approaches in this fast-growing research field, we have developed Hyperscanning Python Pipeline, a comprehensive and easy open-source software package that allows (social) neuroscientists to carry-out and to interpret inter-brain connectivity analyses.

Inter-brain synchrony in teams predicts collective performance.

Despite decades of research in economics and psychology attempting to identify ingredients that make up successful teams, neuroscientists have only just begun to study how multiple brains interact. Recent research has shown that people's brain activity becomes synchronized with others' (inter-brain synchrony) during social engagement. However, little is known as to whether inter-brain synchrony relates to collective behavior within teams. Here, we merge the nascent field of group neuroscience with the extant literature of team dynamics and collective performance. We recruited 174 participants in groups of 4 and randomly assigned them to complete a series of problem-solving tasks either independently or as a team, while simultaneously recording each person's brain activity using an electroencephalography hyperscanning setup. This design allowed us to examine the relationship between group identification and inter-brain synchrony in explaining collective performance. As expected, teammates identified more strongly with one another, cooperated more on an economic game, and outperformed the average individual on most problem-solving tasks. Crucially, inter-brain synchrony, but not self-reported group identification, predicted collective performance among teams. These results suggest that inter-brain synchrony can be informative in understanding collective performance among teams where self-report measures may fail to capture behavior.

Hybrid Harmony: A Multi-Person Neurofeedback Application for Interpersonal Synchrony.

Recent years have seen a dramatic increase in studies measuring brain activity, physiological responses, and/or movement data from multiple individuals during social interaction. For example, so-called "hyperscanning" research has demonstrated that brain activity may become synchronized across people as a function of a range of factors. Such findings not only underscore the potential of hyperscanning techniques to capture meaningful aspects of naturalistic interactions, but also raise the possibility that hyperscanning can be leveraged as a tool to help improve such naturalistic interactions. Building on our previous work showing that exposing dyads to real-time inter-brain synchrony neurofeedback may help boost their interpersonal connectedness, we describe the biofeedback application Hybrid Harmony, a Brain-Computer Interface (BCI) that supports the simultaneous recording of multiple neurophysiological datastreams and the real-time visualization and sonification of inter-subject synchrony. We report results from 236 dyads experiencing synchrony neurofeedback during naturalistic face-to-face interactions, and show that pairs' social closeness and affective personality traits can be reliably captured with the inter-brain synchrony neurofeedback protocol, which incorporates several different online inter-subject connectivity analyses that can be applied interchangeably. Hybrid Harmony can be used by researchers who wish to study the effects of synchrony biofeedback, and by biofeedback artists and serious game developers who wish to incorporate multiplayer situations into their practice.

Morning brain: real-world neural evidence that high school class times matter.

Researchers, parents and educators consistently observe a stark mismatch between biologically preferred and socially imposed sleep-wake hours in adolescents, fueling debate about high school start times. We contribute neural evidence to this debate with electroencephalogram data collected from high school students during their regular morning, mid-morning and afternoon classes. Overall, student alpha power was lower when class content was taught via videos than through lectures. Students' resting state alpha brain activity decreased as the day progressed, consistent with adolescents being least attentive early in the morning. During the lessons, students showed consistently worse performance and higher alpha power for early morning classes than for mid-morning classes, while afternoon quiz scores and alpha levels varied. Together, our findings demonstrate that both class activity and class time are reflected in adolescents' brain states in a real-world setting, and corroborate educational research suggesting that mid-morning may be the best time to learn.

Magnetoencephalography and Language.

This article provides an overview of research that uses magnetoencephalography to understand the brain basis of human language. The cognitive processes and brain networks that have been implicated in written and spoken language comprehension and production are discussed in relation to different methodologies: we review event-related brain responses, research on the coupling of neural oscillations to speech, oscillatory coupling between brain regions (eg, auditory-motor coupling), and neural decoding approaches in naturalistic language comprehension.

Instructor-learner brain coupling discriminates between instructional approaches and predicts learning.

The neural mechanisms that support naturalistic learning via effective pedagogical approaches remain elusive. Here we used functional near-infrared spectroscopy to measure brain activity from instructor-learner dyads simultaneously during dynamic conceptual learning. Results revealed that brain-to-brain coupling was correlated with learning outcomes, and, crucially, appeared to be driven by specific scaffolding behaviors on the part of the instructors (e.g., asking guiding questions or providing hints). Brain-to-brain coupling enhancement was absent when instructors used an explanation approach (e.g., providing definitions or clarifications). Finally, we found that machine-learning techniques were more successful when decoding instructional approaches (scaffolding vs. explanation) from brain-to-brain coupling data than when using a single-brain method. These findings suggest that brain-to-brain coupling as a pedagogically relevant measure tracks the naturalistic instructional process during instructor-learner interaction throughout constructive engagement, but not information clarification.

Brain-to-Brain Synchrony and Learning Outcomes Vary by Student-Teacher Dynamics: Evidence from a Real-world Classroom Electroencephalography Study.

How does the human brain support real-world learning? We used wireless electroencephalography to collect neurophysiological data from a group of 12 senior high school students and their teacher during regular biology lessons. Six scheduled classes over the course of the semester were organized such that class materials were presented using different teaching styles (videos and lectures), and students completed a multiple-choice quiz after each class to measure their retention of that lesson's content. Both students' brain-to-brain synchrony and their content retention were higher for videos than lectures across the six classes. Brain-to-brain synchrony between the teacher and students varied as a function of student engagement as well as teacher likeability: Students who reported greater social closeness to the teacher showed higher brain-to-brain synchrony with the teacher, but this was only the case for lectures-that is, when the teacher is an integral part of the content presentation. Furthermore, students' retention of the class content correlated with student-teacher closeness, but not with brain-to-brain synchrony. These findings expand on existing social neuroscience research by showing that social factors such as perceived closeness are reflected in brain-to-brain synchrony in real-world group settings and can predict cognitive outcomes such as students' academic performance.

Are We Ready for Real-world Neuroscience?

Real-world environments are typically dynamic, complex, and multisensory in nature and require the support of top-down attention and memory mechanisms for us to be able to drive a car, make a shopping list, or pour a cup of coffee. Fundamental principles of perception and functional brain organization have been established by research utilizing well-controlled but simplified paradigms with basic stimuli. The last 30 years ushered a revolution in computational power, brain mapping, and signal processing techniques. Drawing on those theoretical and methodological advances, over the years, research has departed more and more from traditional, rigorous, and well-understood paradigms to directly investigate cognitive functions and their underlying brain mechanisms in real-world environments. These investigations typically address the role of one or, more recently, multiple attributes of real-world environments. Fundamental assumptions about perception, attention, or brain functional organization have been challenged-by studies adapting the traditional paradigms to emulate, for example, the multisensory nature or varying relevance of stimulation or dynamically changing task demands. Here, we present the state of the field within the emerging heterogeneous domain of real-world neuroscience. To be precise, the aim of this Special Focus is to bring together a variety of the emerging "real-world neuroscientific" approaches. These approaches differ in their principal aims, assumptions, or even definitions of "real-world neuroscience" research. Here, we showcase the commonalities and distinctive features of the different "real-world neuroscience" approaches. To do so, four early-career researchers and the speakers of the Cognitive Neuroscience Society 2017 Meeting symposium under the same title answer questions pertaining to the added value of such approaches in bringing us closer to accurate models of functional brain organization and cognitive functions.

Brain-to-Brain Synchrony Tracks Real-World Dynamic Group Interactions in the Classroom.

The human brain has evolved for group living [1]. Yet we know so little about how it supports dynamic group interactions that the study of real-world social exchanges has been dubbed the "dark matter of social neuroscience" [2]. Recently, various studies have begun to approach this question by comparing brain responses of multiple individuals during a variety of (semi-naturalistic) tasks [3-15]. These experiments reveal how stimulus properties [13], individual differences [14], and contextual factors [15] may underpin similarities and differences in neural activity across people. However, most studies to date suffer from various limitations: they often lack direct face-to-face interaction between participants, are typically limited to dyads, do not investigate social dynamics across time, and, crucially, they rarely study social behavior under naturalistic circumstances. Here we extend such experimentation drastically, beyond dyads and beyond laboratory walls, to identify neural markers of group engagement during dynamic real-world group interactions. We used portable electroencephalogram (EEG) to simultaneously record brain activity from a class of 12 high school students over the course of a semester (11 classes) during regular classroom activities (Figures 1A-1C; Supplemental Experimental Procedures, section S1). A novel analysis technique to assess group-based neural coherence demonstrates that the extent to which brain activity is synchronized across students predicts both student class engagement and social dynamics. This suggests that brain-to-brain synchrony is a possible neural marker for dynamic social interactions, likely driven by shared attention mechanisms. This study validates a promising new method to investigate the neuroscience of group interactions in ecologically natural settings.

On the same wavelength: predictable language enhances speaker-listener brain-to-brain synchrony in posterior superior temporal gyrus.

Recent research has shown that the degree to which speakers and listeners exhibit similar brain activity patterns during human linguistic interaction is correlated with communicative success. Here, we used an intersubject correlation approach in fMRI to test the hypothesis that a listener's ability to predict a speaker's utterance increases such neural coupling between speakers and listeners. Nine subjects listened to recordings of a speaker describing visual scenes that varied in the degree to which they permitted specific linguistic predictions. In line with our hypothesis, the temporal profile of listeners' brain activity was significantly more synchronous with the speaker's brain activity for highly predictive contexts in left posterior superior temporal gyrus (pSTG), an area previously associated with predictive auditory language processing. In this region, predictability differentially affected the temporal profiles of brain responses in the speaker and listeners respectively, in turn affecting correlated activity between the two: whereas pSTG activation increased with predictability in the speaker, listeners' pSTG activity instead decreased for more predictable sentences. Listeners additionally showed stronger BOLD responses for predictive images before sentence onset, suggesting that highly predictable contexts lead comprehenders to preactivate predicted words.

Predicting language: MEG evidence for lexical preactivation.

It is widely assumed that prediction plays a substantial role in language processing. However, despite numerous studies demonstrating that contextual information facilitates both syntactic and lexical-semantic processing, there exists no direct evidence pertaining to the neural correlates of the prediction process itself. Using magnetoencephalography (MEG), this study found that brain activity was modulated by whether or not a specific noun could be predicted, given a picture prime. Specifically, before the noun was presented, predictive contexts triggered enhanced activation in left mid-temporal cortex (implicated in lexical access), ventro-medial prefrontal cortex (previously associated with top-down processing), and visual cortex (hypothesized to index the preactivation of predicted form features), successively. This finding suggests that predictive language processing recruits a top-down network where predicted words are activated at different levels of representation, from more 'abstract' lexical-semantic representations in temporal cortex, all the way down to visual word form features. The same brain regions that exhibited enhanced activation for predictive contexts before the onset of the noun showed effects of congruence during the target word. To our knowledge, this study is one of the first to directly investigate the anticipatory stage of predictive language processing.

Before the N400: effects of lexical-semantic violations in visual cortex.

There exists an increasing body of research demonstrating that language processing is aided by context-based predictions. Recent findings suggest that the brain generates estimates about the likely physical appearance of upcoming words based on syntactic predictions: words that do not physically look like the expected syntactic category show increased amplitudes in the visual M100 component, the first salient MEG response to visual stimulation. This research asks whether violations of predictions based on lexical-semantic information might similarly generate early visual effects. In a picture-noun matching task, we found early visual effects for words that did not accurately describe the preceding pictures. These results demonstrate that, just like syntactic predictions, lexical-semantic predictions can affect early visual processing around ∼100ms, suggesting that the M100 response is not exclusively tuned to recognizing visual features relevant to syntactic category analysis. Rather, the brain might generate predictions about upcoming visual input whenever it can. However, visual effects of lexical-semantic violations only occurred when a single lexical item could be predicted. We argue that this may be due to the fact that in natural language processing, there is typically no straightforward mapping between lexical-semantic fields (e.g., flowers) and visual or auditory forms (e.g., tulip, rose, magnolia). For syntactic categories, in contrast, certain form features do reliably correlate with category membership. This difference may, in part, explain why certain syntactic effects typically occur much earlier than lexical-semantic effects.