Differences in functional brain organization during gesture recognition between autistic and neurotypical individuals.

Persons with and without autism process sensory information differently. Differences in sensory processing are directly relevant to social functioning and communicative abilities, which are known to be hampered in persons with autism. We collected functional magnetic resonance imaging data from 25 autistic individuals and 25 neurotypical individuals while they performed a silent gesture recognition task. We exploited brain network topology, a holistic quantification of how networks within the brain are organized to provide new insights into how visual communicative signals are processed in autistic and neurotypical individuals. Performing graph theoretical analysis, we calculated two network properties of the action observation network: 'local efficiency', as a measure of network segregation, and 'global efficiency', as a measure of network integration. We found that persons with autism and neurotypical persons differ in how the action observation network is organized. Persons with autism utilize a more clustered, local-processing-oriented network configuration (i.e. higher local efficiency) rather than the more integrative network organization seen in neurotypicals (i.e. higher global efficiency). These results shed new light on the complex interplay between social and sensory processing in autism.

Observed and Performed Error Signals in Auditory Lexical Decisions.

This study investigates the error processing components in the EEG signal of Performers and Observers using an auditory lexical decision task, in which participants heard spoken items and decided for each item if it was a real word or not. Pairs of participants were tested in both the role of the Performer and the Observer. In the literature, an Error Related Negativity (ERN)-Error Positivity (Pe) complex has been identified for performed (ERN-Pe) and observed (oERN-oPe) errors. While these effects have been widely studied for performance errors in speeded decision tasks relying on visual input, relatively little is known about the performance monitoring signatures in observed language processing based on auditory input. In the lexical decision task, native Dutch speakers listened to real Dutch Words, Non-Words, and crucially, long Pseudowords that resembled words until the final syllable and were shown to be error-prone in a pilot study, because they were responded to too soon. We hypothesised that the errors in the task would result in a response locked ERN-Pe pattern both for the Performer and for the Observer. Our hypothesis regarding the ERN was not supported, however a Pe-like effect, as well as a P300 were present. Analyses to disentangle lexical and error processing similarly indicated a P300 for errors, and the results furthermore pointed to differences between responses before and after word offset. The findings are interpreted as marking attention during error processing during auditory word recognition.

Differences in the production and perception of communicative kinematics in autism.

In human communication, social intentions and meaning are often revealed in the way we move. In this study, we investigate the flexibility of human communication in terms of kinematic modulation in a clinical population, namely, autistic individuals. The aim of this study was twofold: to assess (a) whether communicatively relevant kinematic features of gestures differ between autistic and neurotypical individuals, and (b) if autistic individuals use communicative kinematic modulation to support gesture recognition. We tested autistic and neurotypical individuals on a silent gesture production task and a gesture comprehension task. We measured movement during the gesture production task using a Kinect motion tracking device in order to determine if autistic individuals differed from neurotypical individuals in their gesture kinematics. For the gesture comprehension task, we assessed whether autistic individuals used communicatively relevant kinematic cues to support recognition. This was done by using stick-light figures as stimuli and testing for a correlation between the kinematics of these videos and recognition performance. We found that (a) silent gestures produced by autistic and neurotypical individuals differ in communicatively relevant kinematic features, such as the number of meaningful holds between movements, and (b) while autistic individuals are overall unimpaired at recognizing gestures, they processed repetition and complexity, measured as the amount of submovements perceived, differently than neurotypicals do. These findings highlight how subtle aspects of neurotypical behavior can be experienced differently by autistic individuals. They further demonstrate the relationship between movement kinematics and social interaction in high-functioning autistic individuals. LAY SUMMARY: Hand gestures are an important part of how we communicate, and the way that we move when gesturing can influence how easy a gesture is to understand. We studied how autistic and typical individuals produce and recognize hand gestures, and how this relates to movement characteristics. We found that autistic individuals moved differently when gesturing compared to typical individuals. In addition, while autistic individuals were not worse at recognizing gestures, they differed from typical individuals in how they interpreted certain movement characteristics.

Constructing Others' Beliefs from One's Own Using Medial Frontal Cortex.

Many daily choices are based on one's own knowledge. However, when predicting other people's behavior, we need to consider the differences between our knowledge and other people's presumed knowledge. Social agents need a mechanism to use privileged information for their own behavior but exclude it from predictions of others. Using fMRI, we investigated the neural implementation of such social and personal predictions in healthy human volunteers of both sexes by manipulating privileged and shared information. The medial frontal cortex appeared to have an important role in flexibly making decisions using privileged information for oneself or predicting others' behavior. Specifically, we show that ventromedial PFC tracked the state of the world independent of the type of decision (personal, social), whereas dorsomedial regions adjusted their frame of reference to the use of privileged or shared information. Sampling privileged evidence not available to another person also relied on specific interactions between temporoparietal junction area and frontal pole.SIGNIFICANCE STATEMENT What we know about the minds of others and how we use that information is crucial to understanding social interaction. Mentalizing, or reading the minds of others, is argued to be particularly well developed in the human and crucially affected in some disorders. However, the intractable nature of human interactions makes it very difficult to study these processes. Here, we present a way to objectively quantify the information people have about others and to investigate how their brain deals with this information. This shows that people use similar areas in the brain related to nonsocial decision-making when making decisions in social situations and modify this information processing by the knowledge about others use these to modify their information processing according to the knowledge of others.

Contrasting Similar Words Facilitates Second Language Vocabulary Learning in Children by Sharpening Lexical Representations.

This study considers one of the cognitive mechanisms underlying the development of second language (L2) vocabulary in children: The differentiation and sharpening of lexical representations. We propose that sharpening is triggered by an implicit comparison of similar representations, a process we call contrasting. We investigate whether integrating contrasting in a learning method in which children contrast orthographically and semantically similar L2 words facilitates learning of those words by sharpening their new lexical representations. In our study, 48 Dutch-speaking children learned unfamiliar orthographically and semantically similar English words in a multiple-choice learning task. One half of the group learned the similar words by contrasting them, while the other half did not contrast them. Their word knowledge was measured immediately after learning as well as 1 week later. Contrasting was found to facilitate learning by leading to more precise lexical representations. However, only highly skilled readers benefitted from contrasting. Our findings offer novel insights into the development of L2 lexical representations from fuzzy to more precise, and have potential implications for education.

Action predictability is reflected in beta power attenuation and predictive eye movements in adolescents with and without autism.

Most theoretical accounts of autism posit difficulties in predicting others' actions, and this difficulty has been proposed to be at the root of autistic individuals' social communication differences. Empirical results are mixed, however, with autistic individuals showing reduced action prediction in some studies but not in others. It has recently been proposed that this effect might be observed primarily when observed actions are less predictable, but this idea has yet to be tested. To assess the influence of predictability on neural and behavioural action prediction, the current study employed an action observation paradigm with multi-step actions that become gradually more predictable. Autistic and non-autistic adolescents showed similar patterns of motor system activation during observation, as seen in attenuated mu and beta power compared to baseline, with beta power further modulated by predictability in both groups. Bayesian statistics confirmed that action predictability influenced beta power similarly in both groups. The groups also made similar behavioural predictions, as seen in three eye-movement measures. We found no evidence that autistic adolescents responded differently than non-autistic adolescents to the predictability of an observed action. These findings show that autistic adolescents do spontaneously predict others' actions, both neurally and behaviourally, which calls into question the role of action prediction as a key mechanism underlying autism.

Neural mechanisms of predicting individual preferences based on group membership.

Successful social interaction requires humans to predict others' behavior. To do so, internal models of others are generated based on previous observations. When predicting others' preferences for objects, for example, observations are made at an individual level (5-year-old Rosie often chooses a pencil) or at a group level (kids often choose pencils). But previous research has focused either on already established group knowledge, i.e. stereotypes, or on the neural correlates of predicting traits and preferences of individuals. We identified the neural mechanisms underlying predicting individual behavior based on learned group knowledge using fMRI. We show that applying learned group knowledge hinges on both a network of regions commonly referred to as the mentalizing network, and a network of regions implicated in representing social knowledge. Additionally, we provide evidence for the presence of a gradient in the posterior temporal cortex and the medial frontal cortex, catering to different functions while applying learned group knowledge. This process is characterized by an increased connectivity between medial prefrontal cortex and other mentalizing network regions and increased connectivity between anterior temporal lobe and other social knowledge regions. Our study provides insights into the neural mechanisms underlying the application of learned group knowledge.

Predicting Choice Behavior of Group Members.

Meaningful social interactions rest upon our ability to accurately infer and predict other people's preferences. Ireferen doing so, we can separate two sources of information: knowledge we have about the particular individual (individual knowledge) and knowledge we have about the social group to which that individual belongs (categorical knowledge). However, it is yet unclear how these two types of knowledge contribute to making predictions about other people's choice behavior. To fill this gap, we had participants learn probabilistic preferences by predicting object choices of agents with and without a common logo printed on their shirt. The logo thereby served as a visual cue to increase perceptions of groupness. We quantified how similar predictions for a specific agent are relative to the objective individual-level preferences of that agent and how close these predictions are relative to the objective group-level preferences to which that agent belongs. We found that the logo influenced how close participants' predictions were to the individual-level preferences of an agent relative to the preferences of the group the agent belongs to. We interpret this pattern of results as indicative of a differential weighting of individual and categorical group knowledge when making predictions about individuals that are perceived as forming a social group. The results are interpreted in an assimilation account of categorization and stress the importance of group knowledge during daily social interactions.

The communicative advantage: how kinematic signaling supports semantic comprehension.

Humans are unique in their ability to communicate information through representational gestures which visually simulate an action (eg. moving hands as if opening a jar). Previous research indicates that the intention to communicate modulates the kinematics (e.g., velocity, size) of such gestures. If and how this modulation influences addressees' comprehension of gestures have not been investigated. Here we ask whether communicative kinematic modulation enhances semantic comprehension (i.e., identification) of gestures. We additionally investigate whether any comprehension advantage is due to enhanced early identification or late identification. Participants (n = 20) watched videos of representational gestures produced in a more- (n = 60) or less-communicative (n = 60) context and performed a forced-choice recognition task. We tested the isolated role of kinematics by removing visibility of actor's faces in Experiment I, and by reducing the stimuli to stick-light figures in Experiment II. Three video lengths were used to disentangle early identification from late identification. Accuracy and response time quantified main effects. Kinematic modulation was tested for correlations with task performance. We found higher gesture identification performance in more- compared to less-communicative gestures. However, early identification was only enhanced within a full visual context, while late identification occurred even when viewing isolated kinematics. Additionally, temporally segmented acts with more post-stroke holds were associated with higher accuracy. Our results demonstrate that communicative signaling, interacting with other visual cues, generally supports gesture identification, while kinematic modulation specifically enhances late identification in the absence of other cues. Results provide insights into mutual understanding processes as well as creating artificial communicative agents.

Seeing the Unexpected: How Brains Read Communicative Intent through Kinematics.

Social interaction requires us to recognize subtle cues in behavior, such as kinematic differences in actions and gestures produced with different social intentions. Neuroscientific studies indicate that the putative mirror neuron system (pMNS) in the premotor cortex and mentalizing system (MS) in the medial prefrontal cortex support inferences about contextually unusual actions. However, little is known regarding the brain dynamics of these systems when viewing communicatively exaggerated kinematics. In an event-related functional magnetic resonance imaging experiment, 28 participants viewed stick-light videos of pantomime gestures, recorded in a previous study, which contained varying degrees of communicative exaggeration. Participants made either social or nonsocial classifications of the videos. Using participant responses and pantomime kinematics, we modeled the probability of each video being classified as communicative. Interregion connectivity and activity were modulated by kinematic exaggeration, depending on the task. In the Social Task, communicativeness of the gesture increased activation of several pMNS and MS regions and modulated top-down coupling from the MS to the pMNS, but engagement of the pMNS and MS was not found in the nonsocial task. Our results suggest that expectation violations can be a key cue for inferring communicative intention, extending previous findings from wholly unexpected actions to more subtle social signaling.

Preview of partial stimulus information in search prioritizes features and conjunctions, not locations.

Visual search often requires combining information on distinct visual features such as color and orientation, but how the visual system does this is not fully understood. To better understand this, we showed observers a brief preview of part of a search stimulus-either its color or orientation-before they performed a conjunction search task. Our experimental questions were (1) whether observers would use such previews to prioritize either potential target locations or features, and (2) which neural mechanisms might underlie the observed effects. In two experiments, participants searched for a prespecified target in a display consisting of bar elements, each combining one of two possible colors and one of two possible orientations. Participants responded by making an eye movement to the selected bar. In our first experiment, we found that a preview consisting of colored bars with identical orientation improved saccadic target selection performance, while a preview of oriented gray bars substantially decreased performance. In a follow-up experiment, we found that previews consisting of discs of the same color as the bars (and thus without orientation information) hardly affected performance. Thus, performance improved only when the preview combined color and (noninformative) orientation information. Previews apparently result in a prioritization of features and conjunctions rather than of spatial locations (in the latter case, all previews should have had similar effects). Our results thus also indicate that search for, and prioritization of, combinations involve conjunctively tuned neural mechanisms. These probably reside at the level of the primary visual cortex.

Young children integrate current observations, priors and agent information to predict others' actions.

From early on in life, children are able to use information from their environment to form predictions about events. For instance, they can use statistical information about a population to predict the sample drawn from that population and infer an agent's preferences from systematic violations of random sampling. We investigated whether and how young children infer an agent's sampling biases. Moreover, we examined whether pupil data of toddlers follow the predictions of a computational model based on the causal Bayesian network formalization of predictive processing. We formalized three hypotheses about how different explanatory variables (i.e., prior probabilities, current observations, and agent characteristics) are used to predict others' actions. We measured pupillary responses as a behavioral marker of 'prediction errors' (i.e., the perceived mismatch between what one's model of an agent predicts and what the agent actually does). Pupillary responses of 24-month-olds, but not 18-month-olds, showed that young children integrated information about current observations, priors and agents to make predictions about agents and their actions. These findings shed light on the mechanisms behind toddlers' inferences about agent-caused events. To our knowledge, this is the first study in which young children's pupillary responses are used as markers of prediction errors, which were qualitatively compared to the predictions by a computational model based on the causal Bayesian network formalization of predictive processing.

Processing of Prediction Errors in Mentalizing Areas.

When seeing people perform actions, we are able to quickly predict the action's outcomes. These predictions are not solely based on the observed actions themselves but utilize our prior knowledge of others. It has been suggested that observed outcomes that are not in line with these predictions result in prediction errors, which require additional processing to be integrated or updated. However, there is no consensus on whether this is indeed the case for the kind of high-level social-cognitive processes involved in action observation. In this fMRI study, we investigated whether observation of unexpected outcomes causes additional activation in line with the processing of prediction errors and, if so, whether this activation overlaps with activation in brain areas typically associated with social-cognitive processes. In the first part of the experiment, participants watched animated movies of two people playing a bowling game, one experienced and one novice player. In cases where the player's score was higher or lower than expected based on their skill level, there was increased BOLD activity in areas that were also activated during a theory of mind task that participants performed in the second part of the experiment. These findings are discussed in the light of different theoretical accounts of human social-cognitive processing.

The processing of task-irrelevant emotion and colour in the Approach-Avoidance Task.

When processing information about human faces, we have to integrate different sources of information like skin colour and emotional expression. In 3 experiments, we investigated how these features are processed in a top-down manner when task instructions determine the relevance of features, and in a bottom-up manner when the stimulus features themselves determine process priority. In Experiment 1, participants learned to respond with approach-avoidance movements to faces that presented both emotion and colour features (e.g. happy faces printed in greyscale). For each participant, only one of these two features was task-relevant while the other one could be ignored. In contrast to our predictions, we found better learning of task-irrelevant colour when emotion was task-relevant than vice versa. Experiment 2 showed that the learning of task-irrelevant emotional information was improved in general when participants' awareness was increased by adding NoGo-trials. Experiment 3 replicated these results for faces and emotional words. We conclude that during the processing of faces, both bottom-up and top-down processes are involved, such that task instructions and feature characteristics play a role. Ecologically significant features like emotions are not necessarily processed with high priority. The findings are discussed in the light of theories of attention and cognitive biases.

Communicative intent modulates production and comprehension of actions and gestures: A Kinect study.

Actions may be used to directly act on the world around us, or as a means of communication. Effective communication requires the addressee to recognize the act as being communicative. Humans are sensitive to ostensive communicative cues, such as direct eye gaze (Csibra & Gergely, 2009). However, there may be additional cues present in the action or gesture itself. Here we investigate features that characterize the initiation of a communicative interaction in both production and comprehension. We asked 40 participants to perform 31 pairs of object-directed actions and representational gestures in more- or less- communicative contexts. Data were collected using motion capture technology for kinematics and video recording for eye-gaze. With these data, we focused on two issues. First, if and how actions and gestures are systematically modulated when performed in a communicative context. Second, if observers exploit such kinematic information to classify an act as communicative. Our study showed that during production the communicative context modulates space-time dimensions of kinematics and elicits an increase in addressee-directed eye-gaze. Naïve participants detected communicative intent in actions and gestures preferentially using eye-gaze information, only utilizing kinematic information when eye-gaze was unavailable. Our study highlights the general communicative modulation of action and gesture kinematics during production but also shows that addressees only exploit this modulation to recognize communicative intention in the absence of eye-gaze. We discuss these findings in terms of distinctive but potentially overlapping functions of addressee directed eye-gaze and kinematic modulations within the wider context of human communication and learning.

Effects of stimulus response compatibility on covert imitation of vowels.

When we observe someone else speaking, we tend to automatically activate the corresponding speech motor patterns. When listening, we therefore covertly imitate the observed speech. Simulation theories of speech perception propose that covert imitation of speech motor patterns supports speech perception. Covert imitation of speech has been studied with interference paradigms, including the stimulus-response compatibility paradigm (SRC). The SRC paradigm measures covert imitation by comparing articulation of a prompt following exposure to a distracter. Responses tend to be faster for congruent than for incongruent distracters; thus, showing evidence of covert imitation. Simulation accounts propose a key role for covert imitation in speech perception. However, covert imitation has thus far only been demonstrated for a select class of speech sounds, namely consonants, and it is unclear whether covert imitation extends to vowels. We aimed to demonstrate that covert imitation effects as measured with the SRC paradigm extend to vowels, in two experiments. We examined whether covert imitation occurs for vowels in a consonant-vowel-consonant context in visual, audio, and audiovisual modalities. We presented the prompt at four time points to examine how covert imitation varied over the distracter's duration. The results of both experiments clearly demonstrated covert imitation effects for vowels, thus supporting simulation theories of speech perception. Covert imitation was not affected by stimulus modality and was maximal for later time points.

One wouldn't expect an expert bowler to hit only two pins: Hierarchical predictive processing of agent-caused events.

Evidence is accumulating that our brains process incoming information using top-down predictions. If lower level representations are correctly predicted by higher level representations, this enhances processing. However, if they are incorrectly predicted, additional processing is required at higher levels to "explain away" prediction errors. Here, we explored the potential nature of the models generating such predictions. More specifically, we investigated whether a predictive processing model with a hierarchical structure and causal relations between its levels is able to account for the processing of agent-caused events. In Experiment 1, participants watched animated movies of "experienced" and "novice" bowlers. The results are in line with the idea that prediction errors at a lower level of the hierarchy (i.e., the outcome of how many pins fell down) slow down reporting of information at a higher level (i.e., which agent was throwing the ball). Experiments 2 and 3 suggest that this effect is specific to situations in which the predictor is causally related to the outcome. Overall, the study supports the idea that a hierarchical predictive processing model can account for the processing of observed action outcomes and that the predictions involved are specific to cases where action outcomes can be predicted based on causal knowledge.

Frontal cortex electrophysiology in reward- and punishment-related feedback processing during advice-guided decision making: An interleaved EEG-DC stimulation study.

During decision making, individuals are prone to rely on external cues such as expert advice when the outcome is not known. However, the electrophysiological correlates associated with outcome uncertainty and the use of expert advice are not completely understood. The feedback-related negativity (FRN), P3a, and P3b are event-related brain potentials (ERPs) linked to dissociable stages of feedback and attentional processing during decision making. Even though these ERPs are influenced by both reward- and punishment-related feedback, it remains unclear how extrinsic information during uncertainty modulates these brain potentials. In this study, the effects of advice cues on decision making were investigated in two separate experiments. In the first experiment, electroencephalography (EEG) was recorded in healthy volunteers during a decision-making task in which the participants received reward or punishment feedback preceded by novice, amateur, or expert advice. The results showed that the P3a component was significantly influenced by the subjective predictive value of an advice cue, whereas the FRN and P3b were unaffected by the advice cues. In the second, sham-controlled experiment, cathodal transcranial direct current stimulation (ctDCS) was administered in conjunction with EEG in order to explore the direct contributions of the frontal cortex to these brain potentials. Results showed no significant change in either advice-following behavior or decision times. However, ctDCS did decrease FRN amplitudes as compared to sham, with no effect on the P3a or P3b. Together, these findings suggest that advice information may act primarily on attention allocation during feedback processing, whereas the electrophysiological correlates of the detection and updating of internal prediction models are not affected.

Lateral frontal pole and relational processing: Activation patterns and connectivity profile.

The functional contribution of the lateral frontal cortex to behavior has been discussed with reference to several higher-order cognitive domains. In a separate line of research, recent studies have focused on the anatomical organization of this part of the brain. These different approaches are rarely combined. Here, we combine previous work using anatomical connectivity that identified a lateral subdivision of the human frontal pole and work that suggested a general role for rostrolateral prefrontal cortex in processing higher-order relations, irrespective of the type of information. We asked healthy human volunteers to judge the relationship between pairs of stimuli, a task previously suggested to engage the lateral frontal pole. Presenting both shape and face stimuli, we indeed observed overlapping activation of the lateral prefrontal cortex when subjects judged relations between pairs. Using resting state functional MRI, we confirmed that the activated region's whole-brain connectivity most strongly resembles that of the lateral frontal pole. Using diffusion MRI, we showed that the pattern of connections of this region with the main association fibers again is most similar to that of the lateral frontal pole, consistent with the observation that it is this anatomical region that is involved in relational processing.