Predictive processing account of action perception: Evidence from effective connectivity in the action observation network.

Visual perception of actions is supported by a network of brain regions in the occipito-temporal, parietal, and premotor cortex in the primate brain, known as the Action Observation Network (AON). Although there is a growing body of research that characterizes the functional properties of each node of this network, the communication and direction of information flow between the nodes is unclear. According to the predictive coding account of action perception (Kilner, Friston, & Frith, 2007a; 2007b), this network is not a purely feedforward system but has backward connections through which prediction error signals are communicated between the regions of the AON. In the present study, we investigated the effective connectivity of the AON in an experimental setting where the human subjects' predictions about the observed agent were violated, using fMRI and Dynamical Causal Modeling (DCM). We specifically examined the influence of the lowest and highest nodes in the AON hierarchy, pSTS and ventral premotor cortex, respectively, on the middle node, inferior parietal cortex during prediction violation. Our DCM results suggest that the influence on the inferior parietal node is through a feedback connection from ventral premotor cortex during perception of actions that violate people's predictions.

Visual tests predict dementia risk in Parkinson disease.

OBJECTIVE: To assess the role of visual measures and retinal volume to predict the risk of Parkinson disease (PD) dementia. METHODS: In this cohort study, we collected visual, cognitive, and motor data in people with PD. Participants underwent ophthalmic examination, retinal imaging using optical coherence tomography, and visual assessment including acuity and contrast sensitivity and high-level visuoperception measures of skew tolerance and biological motion. We assessed the risk of PD dementia using a recently described algorithm that combines age at onset, sex, depression, motor scores, and baseline cognition. RESULTS: One hundred forty-six people were included in the study (112 with PD and 34 age-matched controls). The mean disease duration was 4.1 (±2·5) years. None of these participants had dementia. Higher risk of dementia was associated with poorer performance in visual measures (acuity: ρ = 0.29, p = 0.0024; contrast sensitivity: ρ = -0.37, p < 0.0001; skew tolerance: ρ = -0.25, p = 0.0073; and biological motion: ρ = -0.26, p = 0.0054). In addition, higher risk of PD dementia was associated with thinner retinal structure in layers containing dopaminergic cells, measured as ganglion cell layer (GCL) and inner plexiform layer (IPL) thinning (ρ = -0.29, p = 0.0021; ρ = -0.33, p = 0.00044). These relationships were not seen for the retinal nerve fiber layer that does not contain dopaminergic cells and were not seen in unaffected controls. CONCLUSION: Visual measures and retinal structure in dopaminergic layers were related to risk of PD dementia. Our findings suggest that visual measures and retinal GCL and IPL volumes may be useful to predict the risk of dementia in PD.

Tool Use Modulates Somatosensory Cortical Processing in Humans.

Tool use leads to plastic changes in sensorimotor body representations underlying tactile perception. The neural correlates of this tool-induced plasticity in humans have not been adequately characterized. This study used ERPs to investigate the stage of sensory processing modulated by tool use. Somatosensory evoked potentials, elicited by median nerve stimulation, were recorded before and after two forms of object interaction: tool use and hand use. Compared with baseline, tool use-but not use of the hand alone-modulated the amplitude of the P100. The P100 is a mid-latency component that indexes the construction of multisensory models of the body and has generators in secondary somatosensory and posterior parietal cortices. These results mark one of the first demonstrations of the neural correlates of tool-induced plasticity in humans and suggest that tool use modulates relatively late stages of somatosensory processing outside primary somatosensory cortex. This finding is consistent with what has been observed in tool-trained monkeys and suggests that the mechanisms underlying tool-induced plasticity have been preserved across primate evolution.

Distinct representations in occipito-temporal, parietal, and premotor cortex during action perception revealed by fMRI and computational modeling.

Visual processing of actions is supported by a network consisting of occipito-temporal, parietal, and premotor regions in the human brain, known as the Action Observation Network (AON). In the present study, we investigate what aspects of visually perceived actions are represented in this network using fMRI and computational modeling. Human subjects performed an action perception task during scanning. We characterized the different aspects of the stimuli starting from purely visual properties such as form and motion to higher-aspects such as intention using computer vision and categorical modeling. We then linked the models of the stimuli to the three nodes of the AON with representational similarity analysis. Our results show that different nodes of the network represent different aspects of actions. While occipito-temporal cortex performs visual analysis of actions by means of integrating form and motion information, parietal cortex builds on these visual representations and transforms them into more abstract and semantic representations coding target of the action, action type and intention. Taken together, these results shed light on the neuro-computational mechanisms that support visual perception of actions and provide support that AON is a hierarchical system in which increasing levels of the cortex code increasingly complex features.

Uncanny valley as a window into predictive processing in the social brain.

Uncanny valley refers to humans' negative reaction to almost-but-not-quite-human agents. Theoretical work proposes prediction violation as an explanation for uncanny valley but no empirical work has directly tested it. Here, we provide evidence that supports this theory using event-related brain potential recordings from the human scalp. Human subjects were presented images and videos of three agents as EEG was recorded: a real human, a mechanical robot, and a realistic robot in between. The real human and the mechanical robot had congruent appearance and motion whereas the realistic robot had incongruent appearance and motion. We hypothesize that the appearance of the agent would provide a context to predict her movement, and accordingly the perception of the realistic robot would elicit an N400 effect indicating the violation of predictions, whereas the human and the mechanical robot would not. Our data confirmed this hypothesis suggesting that uncanny valley could be explained by violation of one's predictions about human norms when encountered with realistic but artificial human forms. Importantly, our results implicate that the mechanisms underlying perception of other individuals in our environment are predictive in nature.

Assessing cognitive dysfunction in Parkinson's disease: An online tool to detect visuo-perceptual deficits.

BACKGROUND: People with Parkinson's disease (PD) who develop visuo-perceptual deficits are at higher risk of dementia, but we lack tests that detect subtle visuo-perceptual deficits and can be performed by untrained personnel. Hallucinations are associated with cognitive impairment and typically involve perception of complex objects. Changes in object perception may therefore be a sensitive marker of visuo-perceptual deficits in PD. OBJECTIVE: We developed an online platform to test visuo-perceptual function. We hypothesised that (1) visuo-perceptual deficits in PD could be detected using online tests, (2) object perception would be preferentially affected, and (3) these deficits would be caused by changes in perception rather than response bias. METHODS: We assessed 91 people with PD and 275 controls. Performance was compared using classical frequentist statistics. We then fitted a hierarchical Bayesian signal detection theory model to a subset of tasks. RESULTS: People with PD were worse than controls at object recognition, showing no deficits in other visuo-perceptual tests. Specifically, they were worse at identifying skewed images (P < .0001); at detecting hidden objects (P = .0039); at identifying objects in peripheral vision (P < .0001); and at detecting biological motion (P = .0065). In contrast, people with PD were not worse at mental rotation or subjective size perception. Using signal detection modelling, we found this effect was driven by change in perceptual sensitivity rather than response bias. CONCLUSIONS: Online tests can detect visuo-perceptual deficits in people with PD, with object recognition particularly affected. Ultimately, visuo-perceptual tests may be developed to identify at-risk patients for clinical trials to slow PD dementia. © 2018 The Authors. Movement Disorders published by Wiley Periodicals, Inc. on behalf of International Parkinson and Movement Disorder Society.

The recalibration of tactile perception during tool use is body-part specific.

Two decades of research have demonstrated that using a tool modulates spatial representations of the body. Whether this embodiment is specific to representations of the tool-using limb or extends to representations of other body parts has received little attention. Several studies of other perceptual phenomena have found that modulations to the primary somatosensory representation of the hand transfers to the face, due in part to their close proximity in primary somatosensory cortex. In the present study, we investigated whether tool-induced recalibration of tactile perception on the hand transfers to the cheek. Participants verbally estimated the distance between two tactile points applied to either their hand or face, before and after using a hand-shaped tool. Tool use recalibrated tactile distance perception on the hand-in line with previous findings-but left perception on the cheek unchanged. This finding provides support for the idea that embodiment is body-part specific. Furthermore, it suggests that tool-induced perceptual recalibration occurs at a level of somatosensory processing, where representations of the hand and face have become functionally disentangled.

Visual illusion of tool use recalibrates tactile perception.

Brief use of a tool recalibrates multisensory representations of the user's body, a phenomenon called tool embodiment. Despite two decades of research, little is known about its boundary conditions. It has been widely argued that embodiment requires active tool use, suggesting a critical role for somatosensory and motor feedback. The present study used a visual illusion to cast doubt on this view. We used a mirror-based setup to induce a visual experience of tool use with an arm that was in fact stationary. Following illusory tool use, tactile perception was recalibrated on this stationary arm, and with equal magnitude as physical use. Recalibration was not found following illusory passive tool holding, and could not be accounted for by sensory conflict or general interhemispheric plasticity. These results suggest visual tool-use signals play a critical role in driving tool embodiment.

Is that a human? Categorization (dis)fluency drives evaluations of agents ambiguous on human-likeness.

A fundamental and seemingly unbridgeable psychological boundary divides humans and nonhumans. Essentialism theories suggest that mixing these categories violates "natural kinds." Perceptual theories propose that such mixing creates incompatible cues. Most theories suggest that mixed agents, with both human and nonhuman features, obligatorily elicit discomfort. In contrast, we demonstrate top-down, cognitive control of these effects-such that the discomfort with mixed agents is partially driven by disfluent categorization of ambiguous features that are pertinent to the agent. Three experiments tested this idea. Participants classified 3 different agents (humans, androids, and robots) either on the human-likeness or control dimension and then evaluated them. Classifying on the human-likeness dimensions made the mixed agent (android) more disfluent, and in turn, more disliked. Disfluency also mediated the negative affective reaction. Critically, devaluation only resulted from disfluency on human-likeness-and not from an equally disfluent color dimension. We argue that negative consequences on evaluations of mixed agents arise from integral disfluency (on features that are relevant to the judgment at-hand, like ambiguous human-likeness). In contrast, no negative effects stem from incidental disfluency (on features that do not bear on the current judgment, like ambiguous color backgrounds). Overall, these findings support a top-down account of why, when, and how mixed agents elicit conflict and discomfort. (PsycINFO Database Record

Mental body representations retain homuncular shape distortions: Evidence from Weber's illusion.

Mental body representations underlying tactile perception do not accurately reflect the body's true morphology. For example, perceived tactile distance is dependent on both the body part being touched and the stimulus orientation, a phenomenon called Weber's illusion. These findings suggest the presence of size and shape distortions, respectively. However, whereas each morphological feature is typically measured in isolation, a complete morphological characterization requires the concurrent measurement of both size and shape. We did so in three experiments, manipulating both the stimulated body parts (hand; forearm) and stimulus orientation while requiring participants to make tactile distance judgments. We found that the forearm was significantly more distorted than the hand lengthwise but not widthwise. Effects of stimulus orientation are thought to reflect receptive field anisotropies in primary somatosensory cortex. The results of the present study therefore suggest that mental body representations retain homuncular shape distortions that characterize early stages of somatosensory processing.

Observation and imitation of actions performed by humans, androids, and robots: an EMG study.

Understanding others' actions is essential for functioning in the physical and social world. In the past two decades research has shown that action perception involves the motor system, supporting theories that we understand others' behavior via embodied motor simulation. Recently, empirical approach to action perception has been facilitated by using well-controlled artificial stimuli, such as robots. One broad question this approach can address is what aspects of similarity between the observer and the observed agent facilitate motor simulation. Since humans have evolved among other humans and animals, using artificial stimuli such as robots allows us to probe whether our social perceptual systems are specifically tuned to process other biological entities. In this study, we used humanoid robots with different degrees of human-likeness in appearance and motion along with electromyography (EMG) to measure muscle activity in participants' arms while they either observed or imitated videos of three agents produce actions with their right arm. The agents were a Human (biological appearance and motion), a Robot (mechanical appearance and motion), and an Android (biological appearance and mechanical motion). Right arm muscle activity increased when participants imitated all agents. Increased muscle activation was found also in the stationary arm both during imitation and observation. Furthermore, muscle activity was sensitive to motion dynamics: activity was significantly stronger for imitation of the human than both mechanical agents. There was also a relationship between the dynamics of the muscle activity and motion dynamics in stimuli. Overall our data indicate that motor simulation is not limited to observation and imitation of agents with a biological appearance, but is also found for robots. However we also found sensitivity to human motion in the EMG responses. Combining data from multiple methods allows us to obtain a more complete picture of action understanding and the underlying neural computations.

Action verbs are processed differently in metaphorical and literal sentences depending on the semantic match of visual primes.

Language comprehension requires rapid and flexible access to information stored in long-term memory, likely influenced by activation of rich world knowledge and by brain systems that support the processing of sensorimotor content. We hypothesized that while literal language about biological motion might rely on neurocognitive representations of biological motion specific to the details of the actions described, metaphors rely on more generic representations of motion. In a priming and self-paced reading paradigm, participants saw video clips or images of (a) an intact point-light walker or (b) a scrambled control and read sentences containing literal or metaphoric uses of biological motion verbs either closely or distantly related to the depicted action (walking). We predicted that reading times for literal and metaphorical sentences would show differential sensitivity to the match between the verb and the visual prime. In Experiment 1, we observed interactions between the prime type (walker or scrambled video) and the verb type (close or distant match) for both literal and metaphorical sentences, but with strikingly different patterns. We found no difference in the verb region of literal sentences for Close-Match verbs after walker or scrambled motion primes, but Distant-Match verbs were read more quickly following walker primes. For metaphorical sentences, the results were roughly reversed, with Distant-Match verbs being read more slowly following a walker compared to scrambled motion. In Experiment 2, we observed a similar pattern following still image primes, though critical interactions emerged later in the sentence. We interpret these findings as evidence for shared recruitment of cognitive and neural mechanisms for processing visual and verbal biological motion information. Metaphoric language using biological motion verbs may recruit neurocognitive mechanisms similar to those used in processing literal language but be represented in a less-specific way.

Tool morphology constrains the effects of tool use on body representations.

What factors constrain whether tool use modulates the user's body representations? To date, studies on representational plasticity following tool use have primarily focused on the act of using the tool. Here, we investigated whether the tool's morphology also serves to constrain plasticity. In 2 experiments, we varied whether the tool was morphologically similar to a target body part (Experiment 1, hand; Experiment 2, arm). Participants judged the tactile distance between pairs of points applied to their tool-using target body surface and forehead (control surface) before and after tool use. We applied touch in 2 orientations, allowing us to quantify how tool use modulates the representation's shape. Significant representational plasticity in hand shape (increase in width, decrease in length) was found when the tool was morphologically similar to a hand (Experiment 1A), but not when the tool was arm-shaped (Experiment 1B). Conversely, significant representational plasticity was found on the arm when the tool was arm-shaped (Experiment 2B), but not when hand-shaped (Experiment 2A). Taken together, our results indicate that morphological similarity between the tool and the effector constrains tool-induced representational plasticity. The embodiment of tools may thus depend on a match-to-template process between tool morphology and representation of the body.

The emergence of mirror-like response properties from domain-general principles in vision and audition.

Like Cook et al., we suggest that mirror neurons are a fascinating product of cross-modal learning. As predicted by an associative account, responses in motor regions are observed for novel and/or abstract visual stimuli such as point-light and android movements. Domain-specific mirror responses also emerge as a function of audiomotor expertise that is slowly acquired over years of intensive training.

EEG theta and Mu oscillations during perception of human and robot actions.

The perception of others' actions supports important skills such as communication, intention understanding, and empathy. Are mechanisms of action processing in the human brain specifically tuned to process biological agents? Humanoid robots can perform recognizable actions, but can look and move differently from humans, and as such, can be used in experiments to address such questions. Here, we recorded EEG as participants viewed actions performed by three agents. In the Human condition, the agent had biological appearance and motion. The other two conditions featured a state-of-the-art robot in two different appearances: Android, which had biological appearance but mechanical motion, and Robot, which had mechanical appearance and motion. We explored whether sensorimotor mu (8-13 Hz) and frontal theta (4-8 Hz) activity exhibited selectivity for biological entities, in particular for whether the visual appearance and/or the motion of the observed agent was biological. Sensorimotor mu suppression has been linked to the motor simulation aspect of action processing (and the human mirror neuron system, MNS), and frontal theta to semantic and memory-related aspects. For all three agents, action observation induced significant attenuation in the power of mu oscillations, with no difference between agents. Thus, mu suppression, considered an index of MNS activity, does not appear to be selective for biological agents. Observation of the Robot resulted in greater frontal theta activity compared to the Android and the Human, whereas the latter two did not differ from each other. Frontal theta thus appears to be sensitive to visual appearance, suggesting agents that are not sufficiently biological in appearance may result in greater memory processing demands for the observer. Studies combining robotics and neuroscience such as this one can allow us to explore neural basis of action processing on the one hand, and inform the design of social robots on the other.

The role of human ventral visual cortex in motion perception.

Visual motion perception is fundamental to many aspects of visual perception. Visual motion perception has long been associated with the dorsal (parietal) pathway and the involvement of the ventral 'form' (temporal) visual pathway has not been considered critical for normal motion perception. Here, we evaluated this view by examining whether circumscribed damage to ventral visual cortex impaired motion perception. The perception of motion in basic, non-form tasks (motion coherence and motion detection) and complex structure-from-motion, for a wide range of motion speeds, all centrally displayed, was assessed in five patients with a circumscribed lesion to either the right or left ventral visual pathway. Patients with a right, but not with a left, ventral visual lesion displayed widespread impairments in central motion perception even for non-form motion, for both slow and for fast speeds, and this held true independent of the integrity of areas MT/V5, V3A or parietal regions. In contrast with the traditional view in which only the dorsal visual stream is critical for motion perception, these novel findings implicate a more distributed circuit in which the integrity of the right ventral visual pathway is also necessary even for the perception of non-form motion.

Individual differences in the perception of biological motion: links to social cognition and motor imagery.

Biological motion perception is often claimed to support social cognition, and to rely upon embodied representations and motor imagery. Are people with higher levels of social traits or more vivid motor imagery better at biological motion perception? We administered four experiments measuring sensitivity in using (global) form and (local) motion cues in biological motion, plus well-established measures of social cognition (e.g., empathy) and motor imagery (e.g., kinesthetic motor imagery). This first systematic investigation of individual variability in biological motion processing demonstrated significant relationships between these domains, along with a dissociation. Sensitivity for using form cues in biological motion processing was correlated with social (and not the imagery) measures; sensitivity for using motion cues was correlated with motor imagery (and not the social) measures. These results could not be explained by performance on non-biological control stimuli. We thus show that although both social cognition and motor imagery predict sensitivity to biological motion, these skills likely tap into different aspects of perception.

Neuroanatomical correlates of biological motion detection.

Biological motion detection is both commonplace and important, but there is great inter-individual variability in this ability, the neural basis of which is currently unknown. Here we examined whether the behavioral variability in biological motion detection is reflected in brain anatomy. Perceptual thresholds for detection of biological motion and control conditions (non-biological object motion detection and motion coherence) were determined in a group of healthy human adults (n=31) together with structural magnetic resonance images of the brain. Voxel based morphometry analyzes revealed that gray matter volumes of left posterior superior temporal sulcus (pSTS) and left ventral premotor cortex (vPMC) significantly predicted individual differences in biological motion detection, but showed no significant relationship with performance on the control tasks. Our study reveals a neural basis associated with the inter-individual variability in biological motion detection, reliably linking the neuroanatomical structure of left pSTS and vPMC with biological motion detection performance.

Investigating the status of biological stimuli as objects of attention in multiple object tracking.

BACKGROUND: Humans are able to track multiple simultaneously moving objects. A number of factors have been identified that can influence the ease with which objects can be attended and tracked. Here, we explored the possibility that object tracking abilities may be specialized for tracking biological targets such as people. METHODOLOGY/PRINCIPAL FINDINGS: We used the Multiple Object Tracking (MOT) paradigm to explore whether the high-level biological status of the targets affects the efficiency of attentional selection and tracking. In Experiment 1, we assessed the tracking of point-light biological motion figures. As controls, we used either the same stimuli or point-light letters, presented in upright, inverted or scrambled configurations. While scrambling significantly affected performance for both letters and point-light figures, there was an effect of inversion restricted to biological motion, inverted figures being harder to track. In Experiment 2, we found that tracking performance was equivalent for natural point-light walkers and 'moon-walkers', whose implied direction was incongruent with their actual direction of motion. In Experiment 3, we found higher tracking accuracy for inverted faces compared with upright faces. Thus, there was a double dissociation between inversion effects for biological motion and faces, with no inversion effect for our non-biological stimuli (letters, houses). CONCLUSIONS/SIGNIFICANCE: MOT is sensitive to some, but not all naturalistic aspects of biological stimuli. There does not appear to be a highly specialized role for tracking people. However, MOT appears constrained by principles of object segmentation and grouping, where effectively grouped, coherent objects, but not necessarily biological objects, are tracked most successfully.