Perceptual encoding of emotions in interactive bodily expressions.

For social species, e.g., primates, the perceptual analysis of social interactions is an essential skill for survival, emerging already early during development. While real-life emotional behavior includes predominantly interactions between conspecifics, research on the perception of emotional body expressions has primarily focused on perception of single individuals. While previous studies using point-light or video stimuli of interacting people suggest an influence of social context on the perception and neural encoding of interacting bodies, it remains entirely unknown how emotions of multiple interacting agents are perceptually integrated. We studied this question using computer animation by creating scenes with two interacting avatars whose emotional style was independently controlled. While participants had to report the emotional style of a single agent, we found a systematic influence of the emotion expressed by the other, which was consistent with the social interaction context. The emotional styles of interacting individuals are thus jointly encoded.

Apraxia of object-related action does not depend on visual feedback.

Pantomime of tool use is typically affected in neurological patients with apraxia, while at the same time these patients are able to perform the use of the actual tool with less or no errors. This discrepancy is commonly explained by differences in afferent input, in particular a lack of visual online feedback from the object in pantomime. The present study investigated the role of visual feedback in apraxia of pantomime by testing neurological patients with apraxia and healthy controls in a task requiring the pantomime of tool use as well as real tool use. Visual feedback was systematically removed at different phases of the action using shutter glasses that were controlled online based on real-time motion-capturing. Data analyses revealed more errors in pantomime than in real tool use. These differences were similar in patients as well as in controls. Removal of visual feedback did not affect apractic errors specifically; it neither increased patients' apractic errors during pantomime of tool use nor transformed the patients' normal movements with a real tool into movements with apractic errors. Our findings contradict the hypothesis that apraxia patients pathologically over-rely on visual feedback. Instead, we propose that pantomime of tool use requires cognitive processes that are not necessary for real tool use and independent of visual online feedback.

Brain activity correlates with emotional perception induced by dynamic avatars.

An accurate judgment of the emotional state of others is a prerequisite for successful social interaction and hence survival. Thus, it is not surprising that we are highly skilled at recognizing the emotions of others. Here we aimed to examine the neuronal correlates of emotion recognition from gait. To this end we created highly controlled dynamic body-movement stimuli based on real human motion-capture data (Roether et al., 2009). These animated avatars displayed gait in four emotional (happy, angry, fearful, and sad) and speed-matched neutral styles. For each emotional gait and its equivalent neutral gait, avatars were displayed at five morphing levels between the two. Subjects underwent fMRI scanning while classifying the emotions and the emotional intensity levels expressed by the avatars. Our results revealed robust brain selectivity to emotional compared to neutral gait stimuli in brain regions which are involved in emotion and biological motion processing, such as the extrastriate body area (EBA), fusiform body area (FBA), superior temporal sulcus (STS), and the amygdala (AMG). Brain activity in the amygdala reflected emotional awareness: for visually identical stimuli it showed amplified stronger response when the stimulus was perceived as emotional. Notably, in avatars gradually morphed along an emotional expression axis there was a parametric correlation between amygdala activity and emotional intensity. This study extends the mapping of emotional decoding in the human brain to the domain of highly controlled dynamic biological motion. Our results highlight an extensive level of brain processing of emotional information related to body language, which relies mostly on body kinematics.

Prejudiced interactions: implicit racial bias reduces predictive simulation during joint action with an out-group avatar.

During social interactions people automatically apply stereotypes in order to rapidly categorize others. Racial differences are among the most powerful cues that drive these categorizations and modulate our emotional and cognitive reactivity to others. We investigated whether implicit racial bias may also shape hand kinematics during the execution of realistic joint actions with virtual in- and out-group partners. Caucasian participants were required to perform synchronous imitative or complementary reach-to-grasp movements with avatars that had different skin color (white and black) but showed identical action kinematics. Results demonstrate that stronger visuo-motor interference (indexed here as hand kinematics differences between complementary and imitative actions) emerged: i) when participants were required to predict the partner's action goal in order to on-line adapt their own movements accordingly; ii) during interactions with the in-group partner, indicating the partner's racial membership modulates interactive behaviors. Importantly, the in-group/out-group effect positively correlated with the implicit racial bias of each participant. Thus visuo-motor interference during joint action, likely reflecting predictive embodied simulation of the partner's movements, is affected by cultural inter-individual differences.

An intact action-perception coupling depends on the integrity of the cerebellum.

It is widely accepted that action and perception in humans functionally interact on multiple levels. Moreover, areas originally suggested to be predominantly motor-related, as the cerebellum, are also involved in action observation. However, as yet, few studies provided unequivocal evidence that the cerebellum is involved in the action perception coupling (APC), specifically in the integration of motor and multisensory information for perception. We addressed this question studying patients with focal cerebellar lesions in a virtual-reality paradigm measuring the effect of action execution on action perception presenting self-generated movements as point lights. We measured the visual sensitivity to the point light stimuli based on signal detection theory. Compared with healthy controls cerebellar patients showed no beneficial influence of action execution on perception indicating deficits in APC. Applying lesion symptom mapping, we identified distinct areas in the dentate nucleus and the lateral cerebellum of both hemispheres that are causally involved in APC. Lesions of the right ventral dentate, the ipsilateral motor representations (lobules V/VI), and most interestingly the contralateral posterior cerebellum (lobule VII) impede the benefits of motor execution on perception. We conclude that the cerebellum establishes time-dependent multisensory representations on different levels, relevant for motor control as well as supporting action perception. Ipsilateral cerebellar motor representations are thought to support the somatosensory state estimate of ongoing movements, whereas the ventral dentate and the contralateral posterior cerebellum likely support sensorimotor integration in the cerebellar-parietal loops. Both the correct somatosensory as well as the multisensory state representations are vital for an intact APC.

Show me how you walk and I tell you how you feel - a functional near-infrared spectroscopy study on emotion perception based on human gait.

The ability to recognize and adequately interpret emotional states in others plays a fundamental role in regulating social interaction. Body language presents an essential element of nonverbal communication which is often perceived prior to mimic expression. However, the neural networks that underlie the processing of emotionally expressive body movement and body posture are poorly understood. 33 healthy subjects have been investigated using the optically based imaging method functional near-infrared spectroscopy (fNIRS) during the performance of a newly developed emotion discrimination paradigm consisting of faceless avatars expressing fearful, angry, sad, happy or neutral gait patterns. Participants were instructed to judge (a) the presented emotional state (emotion task) and (b) the observed walking speed of the respective avatar (speed task). We measured increases in cortical oxygenated haemoglobin (O2HB) in response to visual stimulation during emotion discrimination. These O2HB concentration changes were enhanced for negative emotions in contrast to neutral gait sequences in right occipito-temporal and left temporal and temporo-parietal brain regions. Moreover, fearful and angry bodies elicited higher activation increases during the emotion task compared to the speed task. Haemodynamic responses were correlated with a number of behavioural measures, whereby a positive relationship between emotion regulation strategy preference and O2HB concentration increases after sad walks was mediated by the ability to accurately categorize sad walks. Our results support the idea of a distributed brain network involved in the recognition of bodily emotion expression that comprises visual association areas as well as body/movement perception specific cortical regions that are also sensitive to emotion. This network is activated less when the emotion is not intentionally processed (i.e. during the speed task). Furthermore, activity of this perceptive network is, mediated by the ability to correctly recognize emotions, indirectly connected to active emotion regulation processes. We conclude that a full understanding of emotion perception and its neural substrate requires the investigation of dynamic representations and means of expression other than the face.

Effects of cerebellar lesions on working memory interacting with motor tasks of different complexities.

We examined the influence of focal cerebellar lesions on working memory (n-back task), gait, and the interaction between working memory and different gait tasks in a dual-task paradigm. The analysis included 17 young patients with chronic focal lesions after cerebellar tumor resection and 17 age-matched controls. Patients have shown mild to moderate ataxia. Lesion sites were examined on the basis of structural magnetic resonance imaging. N-back tasks were executed with different levels of difficulty (n = 1-4) during sitting (baseline), treadmill walking, and treadmill tandem walking (dual-task conditions). Patients exhibited decreased n-back performance particularly at difficult n-back levels and in dual-task conditions. Voxel-based lesion-symptom mapping revealed that decreased baseline n-back performance was associated with lesions of the posterolateral cerebellar hemisphere and the dentate nucleus. By contrast, decreased n-back performance in dual-task conditions was more associated with motor-related areas including dorsal portions of the dentate and the interposed nucleus, suggesting a prioritization of the motor task. During baseline walking, increased gait variability was associated with lesions in medial and intermediate regions, whereas for baseline tandem gait, lesions in the posterolateral hemispheres and the dentate nucleus became important. Posterolateral regions overlapped with regions related to baseline n-back performance. Consistently, we observed increased tandem gait variability with growing n-back difficulty in the dual-task condition. These findings suggest that dual-task effects in cerebellar patients are at least partially caused by a common involvement of posterolateral cerebellar regions in working memory and complex motor tasks.

Effects of pictorial cues on reaching depend on the distinctiveness of target objects.

There is an ongoing debate under what conditions learned object sizes influence visuomotor control under preserved stereovision. Using meaningful objects (matchboxes of locally well-known brands in the UK) a previous study has nicely shown that the recognition of these objects influences action programming by means of reach amplitude and grasp pre-shaping even under binocular vision. Using the same paradigm, we demonstrated that short-term learning of colour-size associations was not sufficient to induce any visuomotor effects under binocular viewing conditions. Now we used the same matchboxes, for which the familiarity effect was shown in the UK, with German participants who have never seen these objects before. We addressed the question whether simply a high degree of distinctness, or whether instead actual prior familiarity of these objects, are required to affect motor computations. We found that under monocular and binocular viewing conditions the learned size and location influenced the amplitude of the reaching component significantly. In contrast, the maximum grip aperture remained unaffected for binocular vision. We conclude that visual distinctness is sufficient to form reliable associations in short-term learning to influence reaching even for preserved stereovision. Grasp pre-shaping instead seems to be less susceptible to such perceptual effects.

Neural theory for the perception of causal actions.

The efficient prediction of the behavior of others requires the recognition of their actions and an understanding of their action goals. In humans, this process is fast and extremely robust, as demonstrated by classical experiments showing that human observers reliably judge causal relationships and attribute interactive social behavior to strongly simplified stimuli consisting of simple moving geometrical shapes. While psychophysical experiments have identified critical visual features that determine the perception of causality and agency from such stimuli, the underlying detailed neural mechanisms remain largely unclear, and it is an open question why humans developed this advanced visual capability at all. We created pairs of naturalistic and abstract stimuli of hand actions that were exactly matched in terms of their motion parameters. We show that varying critical stimulus parameters for both stimulus types leads to very similar modulations of the perception of causality. However, the additional form information about the hand shape and its relationship with the object supports more fine-grained distinctions for the naturalistic stimuli. Moreover, we show that a physiologically plausible model for the recognition of goal-directed hand actions reproduces the observed dependencies of causality perception on critical stimulus parameters. These results support the hypothesis that selectivity for abstract action stimuli might emerge from the same neural mechanisms that underlie the visual processing of natural goal-directed action stimuli. Furthermore, the model proposes specific detailed neural circuits underlying this visual function, which can be evaluated in future experiments.

Spatiotemporal tuning of the facilitation of biological motion perception by concurrent motor execution.

The execution of motor behavior influences concurrent visual action observation and especially the perception of biological motion. The neural mechanisms underlying this interaction between perception and motor execution are not exactly known. In addition, the available experimental evidence is partially inconsistent because previous studies have reported facilitation as well as impairments of action perception by concurrent execution. Exploiting a novel virtual reality paradigm, we investigated the spatiotemporal tuning of the influence of motor execution on the perception of biological motion within a signal-detection task. Human observers were presented with point-light stimuli that were controlled by their own movements. Participants had to detect a point-light arm in a scrambled mask, either while executing waving movements or without concurrent motor execution (baseline). The temporal and spatial coherence between the observed and executed movements was parametrically varied. We found a systematic tuning of the facilitatory versus inhibitory influences of motor execution on biological motion detection with respect to the temporal and the spatial congruency between observed and executed movements. Specifically, we found a gradual transition between facilitatory and inhibitory interactions for decreasing temporal synchrony and spatial congruency. This result provides evidence for a spatiotemporally highly selective coupling between dynamic motor representations and neural structures involved in the visual processing of biological motion. In addition, our study offers a unifying explanation that reconciles contradicting results about modulatory effects of motor execution on biological motion perception in previous studies.

Visual action control does not rely on strangers--effects of pictorial cues under monocular and binocular vision.

Fast goal-directed actions are supposed to be controlled almost exclusively by bottom-up visual control. This mode of processing has been identified with the so-called dorsal visual stream. It is generally accepted that object recognition, mediated by the ventral stream, must be important for deciding what action to execute depending on the specific object to be grasped and the particular purpose. In contrast, the kinematic parameters of the actual movement itself are supposed to be unaffected by recognition processes. This view was recently challenged by the demonstration of a significant impact of object familiarity on grasping kinematics under binocular visual control (McIntosh & Lashley, 2008). This effect was observed for very well known everyday objects. However, it remained unclear whether the effect was really due to long-term, everyday familiarity of the target objects or whether it was simply mediated by short-term learning during the experiment. Therefore, we examined whether the same effect could also be found with objects that were geometrically identical to the ones used by McIntosh and Lashley (2008) and could be distinguished by a pictorial cue but were not associated with long-term, everyday experience. We only found an effect of familiarity under monocular but not under binocular control. Our observation suggests that indeed familiarity exerts an effect on movements under binocular control only if explicit knowledge about the objects is very stable and salient, e.g. after long-term experience.

Critical features for the perception of emotion from gait.

Human observers readily recognize emotions expressed in body movement. Their perceptual judgments are based on simple movement features, such as overall speed, but also on more intricate posture and dynamic cues. The systematic analysis of such features is complicated due to the difficulty of considering the large number of potentially relevant kinematic and dynamic parameters. To identify emotion-specific features we motion-captured the neutral and emotionally expressive (anger, happiness, sadness, fear) gaits of 25 individuals. Body posture was characterized by average flexion angles, and a low-dimensional parameterization of the spatio-temporal structure of joint trajectories was obtained by approximation with a nonlinear mixture model. Applying sparse regression, we extracted critical emotion-specific posture and movement features, which typically depended only on a small number of joints. The features we extracted from the motor behavior closely resembled features that were critical for the perception of emotion from gait, determined by a statistical analysis of classification and rating judgments of 21 observers presented with avatars animated with the recorded movements. The perceptual relevance of these features was further supported by another experiment showing that artificial walkers containing only the critical features induced high-level after-effects matching those induced by adaptation with natural emotional walkers.