Perceptual encoding of emotions in interactive bodily expressions.

For social species, e.g., primates, the perceptual analysis of social interactions is an essential skill for survival, emerging already early during development. While real-life emotional behavior includes predominantly interactions between conspecifics, research on the perception of emotional body expressions has primarily focused on perception of single individuals. While previous studies using point-light or video stimuli of interacting people suggest an influence of social context on the perception and neural encoding of interacting bodies, it remains entirely unknown how emotions of multiple interacting agents are perceptually integrated. We studied this question using computer animation by creating scenes with two interacting avatars whose emotional style was independently controlled. While participants had to report the emotional style of a single agent, we found a systematic influence of the emotion expressed by the other, which was consistent with the social interaction context. The emotional styles of interacting individuals are thus jointly encoded.

Bodies in motion: Unraveling the distinct roles of motion and shape in dynamic body responses in the temporal cortex.

The temporal cortex represents social stimuli, including bodies. We examine and compare the contributions of dynamic and static features to the single-unit responses to moving monkey bodies in and between a patch in the anterior dorsal bank of the superior temporal sulcus (dorsal patch [DP]) and patches in the anterior inferotemporal cortex (ventral patch [VP]), using fMRI guidance in macaques. The response to dynamics varies within both regions, being higher in DP. The dynamic body selectivity of VP neurons correlates with static features derived from convolutional neural networks and motion. DP neurons' dynamic body selectivity is not predicted by static features but is dominated by motion. Whereas these data support the dominance of motion in the newly proposed "dynamic social perception" stream, they challenge the traditional view that distinguishes DP and VP processing in terms of motion versus static features, underscoring the role of inferotemporal neurons in representing body dynamics.

A large-scale brain network of species-specific dynamic human body perception.

This ultrahigh field 7 T fMRI study addressed the question of whether there exists a core network of brain areas at the service of different aspects of body perception. Participants viewed naturalistic videos of monkey and human faces, bodies, and objects along with mosaic-scrambled videos for control of low-level features. Independent component analysis (ICA) based network analysis was conducted to find body and species modulations at both the voxel and the network levels. Among the body areas, the highest species selectivity was found in the middle frontal gyrus and amygdala. Two large-scale networks were highly selective to bodies, dominated by the lateral occipital cortex and right superior temporal sulcus (STS) respectively. The right STS network showed high species selectivity, and its significant human body-induced node connectivity was focused around the extrastriate body area (EBA), STS, temporoparietal junction (TPJ), premotor cortex, and inferior frontal gyrus (IFG). The human body-specific network discovered here may serve as a brain-wide internal model of the human body serving as an entry point for a variety of processes relying on body descriptions as part of their more specific categorization, action, or expression recognition functions.

Shape-invariant encoding of dynamic primate facial expressions in human perception.

Dynamic facial expressions are crucial for communication in primates. Due to the difficulty to control shape and dynamics of facial expressions across species, it is unknown how species-specific facial expressions are perceptually encoded and interact with the representation of facial shape. While popular neural network models predict a joint encoding of facial shape and dynamics, the neuromuscular control of faces evolved more slowly than facial shape, suggesting a separate encoding. To investigate these alternative hypotheses, we developed photo-realistic human and monkey heads that were animated with motion capture data from monkeys and humans. Exact control of expression dynamics was accomplished by a Bayesian machine-learning technique. Consistent with our hypothesis, we found that human observers learned cross-species expressions very quickly, where face dynamics was represented largely independently of facial shape. This result supports the co-evolution of the visual processing and motor control of facial expressions, while it challenges appearance-based neural network theories of dynamic expression recognition.

A Naturalistic Dynamic Monkey Head Avatar Elicits Species-Typical Reactions and Overcomes the Uncanny Valley.

Research on social perception in monkeys may benefit from standardized, controllable, and ethologically valid renditions of conspecifics offered by monkey avatars. However, previous work has cautioned that monkeys, like humans, show an adverse reaction toward realistic synthetic stimuli, known as the "uncanny valley" effect. We developed an improved naturalistic rhesus monkey face avatar capable of producing facial expressions (fear grin, lip smack and threat), animated by motion capture data of real monkeys. For validation, we additionally created decreasingly naturalistic avatar variants. Eight rhesus macaques were tested on the various videos and avoided looking at less naturalistic avatar variants, but not at the most naturalistic or the most unnaturalistic avatar, indicating an uncanny valley effect for the less naturalistic avatar versions. The avoidance was deepened by motion and accompanied by physiological arousal. Only the most naturalistic avatar evoked facial expressions comparable to those toward the real monkey videos. Hence, our findings demonstrate that the uncanny valley reaction in monkeys can be overcome by a highly naturalistic avatar.

Prejudiced interactions: implicit racial bias reduces predictive simulation during joint action with an out-group avatar.

During social interactions people automatically apply stereotypes in order to rapidly categorize others. Racial differences are among the most powerful cues that drive these categorizations and modulate our emotional and cognitive reactivity to others. We investigated whether implicit racial bias may also shape hand kinematics during the execution of realistic joint actions with virtual in- and out-group partners. Caucasian participants were required to perform synchronous imitative or complementary reach-to-grasp movements with avatars that had different skin color (white and black) but showed identical action kinematics. Results demonstrate that stronger visuo-motor interference (indexed here as hand kinematics differences between complementary and imitative actions) emerged: i) when participants were required to predict the partner's action goal in order to on-line adapt their own movements accordingly; ii) during interactions with the in-group partner, indicating the partner's racial membership modulates interactive behaviors. Importantly, the in-group/out-group effect positively correlated with the implicit racial bias of each participant. Thus visuo-motor interference during joint action, likely reflecting predictive embodied simulation of the partner's movements, is affected by cultural inter-individual differences.