Spontaneous dyadic behavior predicts the emergence of interpersonal neural synchrony.

Synchronization of neural activity across brains - Interpersonal Neural Synchrony (INS) - is emerging as a powerful marker of social interaction that predicts success of multi-person coordination, communication, and cooperation. As the origins of INS are poorly understood, we tested whether and how INS might emerge from spontaneous dyadic behavior. We recorded neural activity (EEG) and human behavior (full-body kinematics, eye movements, and facial expressions) while dyads of participants were instructed to look at each other without speaking or making co-verbal gestures. We made four fundamental observations. First, despite the absence of a structured social task, INS emerged spontaneously only when participants were able to see each other. Second, we show that such spontaneous INS, comprising specific spectral and topographic profiles, did not merely reflect intra-personal modulations of neural activity, but it rather reflected real-time and dyad-specific coupling of neural activities. Third, using state-of-art video-image processing and deep learning, we extracted the temporal unfolding of three notable social behavioral cues - body movement, eye contact, and smiling - and demonstrated that these behaviors also spontaneously synchronized within dyads. Fourth, we probed the correlates of INS in such synchronized social behaviors. Using cross-correlation and Granger causality analyses, we show that synchronized social behaviors anticipate and in fact Granger cause INS. These results provide proof-of-concept evidence for studying interpersonal neural and behavioral synchrony under natural and unconstrained conditions. Most importantly, the results suggest that INS could be conceptualized as an emergent property of two coupled neural systems: an entrainment phenomenon, promoted by real-time dyadic behavior.

Interpersonal synchronization of spontaneously generated body movements.

Interpersonal movement synchrony (IMS) is central to social behavior in several species. In humans, IMS is typically studied using structured tasks requiring participants to produce specific body movements. Instead, spontaneously generated (i.e., not instructed) movements have received less attention. To test whether spontaneous movements synchronize interpersonally, we recorded full-body kinematics from dyads of participants who were only asked to sit face-to-face and to look at each other. We manipulated interpersonal (i) visual contact and (ii) spatial proximity. We found that spontaneous movements synchronized across participants only when they could see each other and regardless of interpersonal spatial proximity. This synchronization emerged very rapidly and did not selectively entail homologous body parts (as in mimicry); rather, the synchrony generalized to nearly all possible combinations of body parts. Hence, spontaneous behavior alone can lead to IMS. More generally, our results highlight that IMS can be studied under natural and unconstrained conditions.

Transient Disruption of the Inferior Parietal Lobule Impairs the Ability to Attribute Intention to Action.

Although it is well established that fronto-parietal regions are active during action observation, whether they play a causal role in the ability to infer others' intentions from visual kinematics remains undetermined. In the experiments reported here, we combined offline continuous theta burst stimulation (cTBS) with computational modeling to reveal and causally probe single-trial computations in the inferior parietal lobule (IPL) and inferior frontal gyrus (IFG). Participants received cTBS over the left anterior IPL and the left IFG pars orbitalis in separate sessions before completing an intention discrimination task (discriminate intention of observed reach-to-grasp acts) or a kinematic discrimination task unrelated to intention (discriminate peak wrist height of the same acts). We targeted intention-sensitive regions whose fMRI activity, recorded when observing the same reach-to-grasp acts, could accurately discriminate intention. We found that transient disruption of activity of the left IPL, but not the IFG, impaired the observer's ability to attribute intention to action. Kinematic discrimination unrelated to intention, in contrast, was largely unaffected. Computational analyses of how encoding (mapping of intention to movement kinematics) and readout (mapping of kinematics to intention choices) intersect at the single-trial level revealed that IPL cTBS did not diminish the overall sensitivity of intention readout to movement kinematics. Rather, it selectively misaligned intention readout with respect to encoding, deteriorating mapping from informative kinematic features to intention choices. These results provide causal evidence of how the left anterior IPL computes mapping from kinematics to intentions.

A kind of magic: Enhanced detection of pantomimed grasps in professional magicians.

Professional magicians regularly use pantomimed grasps (i.e., movements towards imagined objects) to deceive audiences. To do so, they learn to shape their hands similarly for real and pantomimed grasps. Here we tested whether this form of motor expertise provides them a significant benefit when processing pantomimed grasps. To this aim, in a one-interval discrimination design, we asked 17 professional magicians and 17 naïve controls to watch video clips of reach-to-grasp movements recorded from naïve participants and judge whether the observed movement was real or pantomimed. All video clips were edited to spatially occlude the grasped object (either present or imagined). Data were analysed within a drift diffusion model approach. Fitting different models showed that, whereas magicians and naïve performed similarly when observing real grasps, magicians had a specific advantage compared with naïve at discriminating pantomimed grasps. These findings suggest that motor expertise may be crucial for detecting relevant cues from hand movement during the discrimination of pantomimed grasps. Results are discussed in terms of motor recalibration.

Modulation of corticospinal output during goal-directed actions: Evidence for a contingent coding hypothesis.

Seeing a person perform an action activates the observer's motor system. The present study aimed at investigating the temporal relationship between execution and observation of goal-directed actions. One possibility is that the corticospinal excitability (CSE) follows the dynamic evolution of the pattern of muscle activity in the executed action. Alternatively, CSE may anticipate the future course of the observed action, prospectively extrapolating future states. Our study was designed to test these alternative hypotheses by directly comparing the time course of muscle recruitment during the execution and observation of reach-to-grasp movements. We found that the time course of CSE during action observation followed the time course of the EMG signal during action execution. This contingent coding was observed despite the outcome of the observed motor act being predictable from the earliest phases of the movement. These findings challenge the view that CSE serves to predict the target of an observed action.

Tracking the Leader: Gaze Behavior in Group Interactions.

Can social gaze behavior reveal the leader during real-world group interactions? To answer this question, we developed a novel tripartite approach combining (1) computer vision methods for remote gaze estimation, (2) a detailed taxonomy to encode the implicit semantics of multi-party gaze features, and (3) machine learning methods to establish dependencies between leadership and visual behaviors. We found that social gaze behavior distinctively identified group leaders. Crucially, the relationship between leadership and gaze behavior generalized across democratic and autocratic leadership styles under conditions of low and high time-pressure, suggesting that gaze can serve as a general marker of leadership. These findings provide the first direct evidence that group visual patterns can reveal leadership across different social behaviors and validate a new promising method for monitoring natural group interactions.

Increased functional coupling of the left amygdala and medial prefrontal cortex during the perception of communicative point-light stimuli.

Interpersonal predictive coding (IPPC) describes the behavioral phenomenon whereby seeing a communicative rather than an individual action helps to discern a masked second agent. As little is known, yet, about the neural correlates of IPPC, we conducted a functional magnetic resonance imaging study in a group of 27 healthy participants using point-light displays of moving agents embedded in distractors. We discovered that seeing communicative compared to individual actions was associated with higher activation of right superior frontal gyrus, whereas the reversed contrast elicited increased neural activation in an action observation network that was activated during all trials. Our findings, therefore, potentially indicate the formation of action predictions and a reduced demand for executive control in response to communicative actions. Further, in a regression analysis, we revealed that increased perceptual sensitivity was associated with a deactivation of the left amygdala during the perceptual task. A consecutive psychophysiological interaction analysis showed increased connectivity of the amygdala with medial prefrontal cortex in the context of communicative compared to individual actions. Thus, whereas increased amygdala signaling might interfere with task-relevant processes, increased co-activation of the amygdala and the medial prefrontal cortex in a communicative context might represent the integration of mentalizing computations.

The kinematics that you do not expect: Integrating prior information and kinematics to understand intentions.

Expectations facilitate perception of expected stimuli but may hinder perception of unexpected alternatives. Here, we consider how prior expectations about others' intentions are integrated with visual kinematics over time in detecting the intention of an observed motor act (grasp-to-pour vs. grasp-to-drink). Using rigorous psychophysics methods, we find that the processes of ascribing intentions to others are well described by drift diffusion models in which evidence from observed movements is accumulated over time until a decision threshold is reached. Testing of competing models revealed that when kinematics contained no discriminative intention information, prior expectations predicted the intention choice of the observer. When kinematics contained intention information, kinematics predicted the intention choice. These findings provide evidence for a diffusion process in which the influence of expectations is modulated by movement informativeness and informative kinematics can override initial expectations.

Action Observation Areas Represent Intentions From Subtle Kinematic Features.

Mirror neurons have been proposed to underlie humans' ability to understand others' actions and intentions. Despite 2 decades of research, however, the exact computational and neuronal mechanisms implied in this ability remain unclear. In the current study, we investigated whether, in the absence of contextual cues, regions considered to be part of the human mirror neuron system represent intention from movement kinematics. A total of 21 participants observed reach-to-grasp movements, performed with either the intention to drink or to pour while undergoing functional magnetic resonance imaging. Multivoxel pattern analysis revealed successful decoding of intentions from distributed patterns of activity in a network of structures comprising the inferior parietal lobule, the superior parietal lobule, the inferior frontal gyrus, and the middle frontal gyrus. Consistent with the proposal that parietal regions play a key role in intention understanding, classifier weights were higher in the inferior parietal region. These results provide the first demonstration that putative mirror neuron regions represent subtle differences in movement kinematics to read the intention of an observed motor act.

Seeing mental states: An experimental strategy for measuring the observability of other minds.

Is it possible to perceive others' mental states? Are mental states visible in others' behavior? In contrast to the traditional view that mental states are hidden and not directly accessible to perception, in recent years a phenomenologically-motivated account of social cognition has emerged: direct social perception. However, despite numerous published articles that both defend and critique direct perception, researchers have made little progress in articulating the conditions under which direct perception of others' mental states is possible. This paper proposes an empirically anchored approach to the observability of others' mentality - not just in the weak sense of discussing relevant empirical evidence for and against the phenomenon of interest, but also, and more specifically, in the stronger sense of identifying an experimental strategy for measuring the observability of mental states and articulating the conditions under which mental states are observable. We conclude this article by reframing the problem of direct perception in terms of establishing a definable and measurable relationship between movement features and perceived mental states.

PredPsych: A toolbox for predictive machine learning-based approach in experimental psychology research.

Recent years have seen an increased interest in machine learning-based predictive methods for analyzing quantitative behavioral data in experimental psychology. While these methods can achieve relatively greater sensitivity compared to conventional univariate techniques, they still lack an established and accessible implementation. The aim of current work was to build an open-source R toolbox - "PredPsych" - that could make these methods readily available to all psychologists. PredPsych is a user-friendly, R toolbox based on machine-learning predictive algorithms. In this paper, we present the framework of PredPsych via the analysis of a recently published multiple-subject motion capture dataset. In addition, we discuss examples of possible research questions that can be addressed with the machine-learning algorithms implemented in PredPsych and cannot be easily addressed with univariate statistical analysis. We anticipate that PredPsych will be of use to researchers with limited programming experience not only in the field of psychology, but also in that of clinical neuroscience, enabling computational assessment of putative bio-behavioral markers for both prognosis and diagnosis.

Decoding intentions from movement kinematics.

How do we understand the intentions of other people? There has been a longstanding controversy over whether it is possible to understand others' intentions by simply observing their movements. Here, we show that indeed movement kinematics can form the basis for intention detection. By combining kinematics and psychophysical methods with classification and regression tree (CART) modeling, we found that observers utilized a subset of discriminant kinematic features over the total kinematic pattern in order to detect intention from observation of simple motor acts. Intention discriminability covaried with movement kinematics on a trial-by-trial basis, and was directly related to the expression of discriminative features in the observed movements. These findings demonstrate a definable and measurable relationship between the specific features of observed movements and the ability to discriminate intention, providing quantitative evidence of the significance of movement kinematics for anticipating others' intentional actions.

Doing It Your Way: How Individual Movement Styles Affect Action Prediction.

Individuals show significant variations in performing a motor act. Previous studies in the action observation literature have largely ignored this ubiquitous, if often unwanted, characteristic of motor performance, assuming movement patterns to be highly similar across repetitions and individuals. In the present study, we examined the possibility that individual variations in motor style directly influence the ability to understand and predict others' actions. To this end, we first recorded grasping movements performed with different intents and used a two-step cluster analysis to identify quantitatively 'clusters' of movements performed with similar movement styles (Experiment 1). Next, using videos of the same movements, we proceeded to examine the influence of these styles on the ability to judge intention from action observation (Experiments 2 and 3). We found that motor styles directly influenced observers' ability to 'read' others' intention, with some styles always being less 'readable' than others. These results provide experimental support for the significance of motor variability for action prediction, suggesting that the ability to predict what another person is likely to do next directly depends on her individual movement style.

Are We Real When We Fake? Attunement to Object Weight in Natural and Pantomimed Grasping Movements.

Behavioral and neuropsychological studies suggest that real actions and pantomimed actions tap, at least in part, different neural systems. Inspired by studies showing weight-attunement in real grasps, here we asked whether (and to what extent) kinematics of pantomimed reach-to-grasp movement can reveal the weight of the pretended target. To address this question, we instructed participants (n = 15) either to grasp or pretend to grasp toward two differently weighted objects, i.e., a light object and heavy object. Using linear discriminant analysis, we then proceeded to classify the weight of the target - either real or pretended - on the basis of the recorded movement patterns. Classification analysis revealed that pantomimed reach-to-grasp movements retained information about object weight, although to a lesser extent than real grasp movements. These results are discussed in relation to the mechanisms underlying the control of real and pantomimed grasping movements.

Grasping others' movements: Rapid discrimination of object size from observed hand movements.

During reach-to-grasp movements, the hand is gradually molded to conform to the size and shape of the object to be grasped. Yet the ability to glean information about object properties by observing grasping movements is poorly understood. In this study, we capitalized on the effect of object size to investigate the ability to discriminate the size of an invisible object from movement kinematics. The study consisted of 2 phases. In the first action execution phase, to assess grip scaling, we recorded and analyzed reach-to-grasp movements performed toward differently sized objects. In the second action observation phase, video clips of the corresponding movements were presented to participants in a two-alternative forced-choice task. To probe discrimination performance over time, videos were edited to provide selective vision of different periods from 2 viewpoints. Separate analyses were conducted to determine how the participants' ability to discriminate between stimulus alternatives (Type I sensitivity) and their metacognitive ability to discriminate between correct and incorrect responses (Type II sensitivity) varied over time and viewpoint. We found that as early as 80 ms after movement onset, participants were able to discriminate object size from the observation of grasping movements delivered from the lateral viewpoint. For both viewpoints, information pickup closely matched the evolution of the hand's kinematics, reaching an almost perfect performance well before the fingers made contact with the object (60% of movement duration). These findings suggest that observers are able to decode object size from kinematic sources specified early on in the movement. (PsycINFO Database Record

Predicting object size from hand kinematics: a temporal perspective.

Research on reach-to-grasp movements generally concentrates on kinematics values that are expression of maxima, in particular the maximum aperture of the hand and the peak of wrist velocity. These parameters provide a snapshot description of movement kinematics at a specific time point during reach, i.e., the maximum within a set of value, but do not allow to investigate how hand kinematics gradually conform to target properties. The present study was designed to extend the characterization of object size effects to the temporal domain. Thus, we computed the wrist velocity and the grip aperture throughout reach-to-grasp movements aimed at large versus small objects. To provide a deeper understanding of how joint movements varied over time, we also considered the time course of finger motion relative to hand motion. Results revealed that movement parameters evolved in parallel but at different rates in relation to object size. Furthermore, a classification analysis performed using a Support Vector Machine (SVM) approach showed that kinematic features taken as a group predicted the correct target size well before contact with the object. Interestingly, some kinematics features exhibited a higher ability to discriminate the target size than others did. These findings reinforce our knowledge about the relationship between kinematics and object properties and shed new light on the quantity and quality of information available in the kinematics of a reach-to-grasp movement over time. This might have important implications for our understanding of the action-perception coupling mechanism.

Notational usage modulates attention networks in binumerates.

Multicultural environments require learning multiple number notations wherein some are encountered more frequently than others. This leads to differences in exposure and consequently differences in usage between notations. We find that differential notational usage imposes a significant neurocognitive load on number processing. Despite simultaneous acquisition, twenty four adult binumerates, familiar with two positional writing systems namely Hindu Nagari digits and Hindu Arabic digits, reported significantly lower preference and usage for Nagari as compared to Arabic. Twenty-four participants showed significantly increased reaction times and reduced accuracy while performing magnitude comparison tasks in Nagari with respect to Arabic. Functional magnetic resonance imaging revealed that processing Nagari elicited significantly greater activity in number processing and attention networks. A direct subtraction of networks for Nagari and Arabic notations revealed a neural circuit comprising of bilateral Intra-parietal Sulcus (IPS), Inferior and Mid Frontal Gyri, Fusiform Gyrus and the Anterior Cingulate Cortex (FDR p < 0.005). Additionally, whole brain correlation analysis showed that activity in the left inferior parietal region was modulated by task performance in Nagari. We attribute the increased activation in Nagari to increased task difficulty due to infrequent exposure and usage. Our results reiterate the role of left IPS in modulating performance in numeric tasks and highlight the role of the attention network for monitoring symbolic notation mode in binumerates.