The prediction potential indexes the meaning and communicative function of upcoming utterances.

Prediction has a fundamental role in language processing. However, predictions can be made at different levels, and it is not always clear whether speech sounds, morphemes, words, meanings, or communicative functions are anticipated during dialogues. Previous studies reported specific brain signatures of communicative pragmatic function, in particular enhanced brain responses immediately after encountering an utterance used to request an object from a partner, but relatively smaller ones when the same utterance was used for naming the object. The present experiment now investigates whether similar neuropragmatic signatures emerge in recipients before the onset of upcoming utterances carrying different predictable communicative functions. Trials started with a context question and object pictures displayed on the screen, raising the participant's expectation that words from a specific semantic category (food or tool) would subsequently be used to either name or request one of the objects. Already 600 msec before utterance onset, a larger prediction potential was observed when a request was anticipated relative to naming expectation. As this result is congruent with the neurophysiological difference previously observed right after the critical utterance, the anticipatory brain activity may index predictions about the social-communicative function of upcoming utterances. In addition, we also found that the predictable semantic category of the upcoming word was likewise reflected in the anticipatory brain potential. Thus, the neurophysiological characteristics of the prediction potential can capture different types of upcoming linguistic information, including semantic and pragmatic aspects of an upcoming utterance and communicative action.

Predictive Brain Activity Shows Congruent Semantic Specificity in Language Comprehension and Production.

Sentence fragments strongly predicting a specific subsequent meaningful word elicit larger preword slow waves, prediction potentials (PPs), than unpredictive contexts. To test the current predictive processing models, 128-channel EEG data were collected from both sexes to examine whether (1) different semantic PPs are elicited in language comprehension and production and (2) whether these PPs originate from the same specific "prediction area(s)" or rather from widely distributed category-specific neuronal circuits reflecting the meaning of the predicted item. Slow waves larger after predictable than unpredictable contexts were present both before subjects heard the sentence-final word in the comprehension experiment and before they pronounced the sentence-final word in the production experiment. Crucially, cortical sources underlying the semantic PP were distributed across several cortical areas and differed between the semantic categories of the expected words. In both production and comprehension, the anticipation of animal words was reflected by sources in posterior visual areas, whereas predictable tool words were preceded by sources in the frontocentral sensorimotor cortex. For both modalities, PP size increased with higher cloze probability, thus further confirming that it reflects semantic prediction, and with shorter latencies with which participants completed sentence fragments. These results sit well with theories viewing distributed semantic category-specific circuits as the mechanistic basis of semantic prediction in the two modalities.

Predicting and coding sound into action translation in spinal cord injured people.

Motor activation in response to perception of action-related stimuli may depend on a resonance mechanism subserving action understanding. The extent to which this mechanism is innate or learned from sensorimotor experience is still unclear. Here, we recorded EEG while people with paraplegia or tetraplegia consequent to spinal cord injury (SCI) and healthy control participants were presented with action sounds produced by body parts (mouth, hands or feet) that were or were not affected by SCI. Non-action sounds were used as further control. We observed reduced brain activation in subjects affected by SCI at both pre- and post-stimulus latencies specifically for those actions whose effector was disconnected by the spinal lesion (i.e., hand sound for tetraplegia and leg sound for both paraplegia and tetraplegia). Correlation analyses showed that these modulations were functionally linked with the chronicity of the lesion, indicating that the longer the time the lesion- EEG data acquisition interval and/or the more the lesion occurred at a young age, the weaker was the cortical activity in response to these action sounds. Tellingly, source estimations confirmed that these modulations originated from a deficit in the motor resonance mechanism, by showing diminished activity in premotor (during prediction and perception) and near the primary motor (during perception) areas. Such dissociation along the cortical hierarchy is consistent with both previous reports in healthy subjects and with hierarchical predictive coding accounts. Overall, these data expand on the notion that sensorimotor experience maintains the cortical representations relevant to anticipate and perceive action-related stimuli.

Predictive and perceptual phonemic processing in articulatory motor areas: A prediction potential & mismatch negativity study.

The recent finding of predictive brain signals preceding anticipated perceptual and linguistic stimuli opens new questions for experimental research. Here, we address the possible brain basis of phonological predictions regarding the features of specific speech sounds and their relationship to phonological priming. To this end, we recorded EEG correlates of both pre- and post-stimulus brain responses in a phonological priming study. Redundant spoken sounds induced stimulus expectations, which manifested as a slow-wave anticipatory activity (the Prediction Potential, PP), whereas articulatory-congruent (e.g.,/bƏ/in the context of expected/pƏ/) pairs elicited weaker post-stimulus MMN-like responses as compared to the articulatory-incongruent (e.g.,/bƏ/in the context of expected/dƏ/) pairs, a pattern reminiscent of perceptual priming mediated by articulatory-motor areas. Source analysis reveal clusters of activation in lateral prefrontal, temporal and ventral motor areas, thus providing the proof of the relevance of multimodal representation units subserving predictive and perceptual phonemic processing.

Instantaneous neural processing of communicative functions conveyed by speech prosody.

During conversations, speech prosody provides important clues about the speaker's communicative intentions. In many languages, a rising vocal pitch at the end of a sentence typically expresses a question function, whereas a falling pitch suggests a statement. Here, the neurophysiological basis of intonation and speech act understanding were investigated with high-density electroencephalography (EEG) to determine whether prosodic features are reflected at the neurophysiological level. Already approximately 100 ms after the sentence-final word differing in prosody, questions, and statements expressed with the same sentences led to different neurophysiological activity recorded in the event-related potential. Interestingly, low-pass filtered sentences and acoustically matched nonvocal musical signals failed to show any neurophysiological dissociations, thus suggesting that the physical intonation alone cannot explain this modulation. Our results show rapid neurophysiological indexes of prosodic communicative information processing that emerge only when pragmatic and lexico-semantic information are fully expressed. The early enhancement of question-related activity compared with statements was due to sources in the articulatory-motor region, which may reflect the richer action knowledge immanent to questions, namely the expectation of the partner action of answering the question. The present findings demonstrate a neurophysiological correlate of prosodic communicative information processing, which enables humans to rapidly detect and understand speaker intentions in linguistic interactions.

Brain signatures predict communicative function of speech production in interaction.

People normally know what they want to communicate before they start speaking. However, brain indicators of communication are typically observed only after speech act onset, and it is unclear when any anticipatory brain activity prior to speaking might first emerge, along with the communicative intentions it possibly reflects. Here, we investigated brain activity prior to the production of different speech act types, request and naming actions performed by uttering single words embedded into language games with a partner, similar to natural communication. Starting ca. 600 msec before speech onset, an event-related potential maximal at fronto-central electrodes, which resembled the Readiness Potential, was larger when preparing requests compared to naming actions. Analysis of the cortical sources of this anticipatory brain potential suggests a relatively stronger involvement of fronto-central motor regions for requests, which may reflect the speaker's expectation of the partner actions typically following requests, e.g., the handing over of a requested object. Our results indicate that different neuronal circuits underlying the processing of different speech act types activate already before speaking. Results are discussed in light of previous work addressing the neural basis of speech act understanding and predictive brain indexes of language comprehension.

Correlated Brain Indexes of Semantic Prediction and Prediction Error: Brain Localization and Category Specificity.

With strong and valid predictions, grasping a message is easy, whereas more demanding processing is required in the absence of robust expectations. We here demonstrate that brain correlates of the interplay between prediction and perception mechanisms in the understanding of meaningful sentences. Sentence fragments that strongly predict subsequent words induced anticipatory brain activity preceding the expected words; this potential was absent if context did not strongly predict subsequent words. Subjective reports of certainty about upcoming words and objective corpus-based measures correlated with the size of the anticipatory signal, thus establishing its status as a semantic prediction potential (SPP). Crucially, there was an inverse correlation between the SPP and the N400 brain response. The main cortical generators of SPP and N400 were found in inferior prefrontal cortex and posterior temporal cortex, respectively. Interestingly, sentence meaning was reflected by both measures, with additional category-specific sources of SPPs and N400s falling into parieto-temporo-occipital (visual) and frontocentral (sensorimotor) areas for animal- and tool-related words, respectively. These results show that the well-known brain index of semantic comprehension, N400, has an antecedent with different brain localization but similar semantic discriminatory function. We discuss whether N400 dynamics may causally depend on mechanisms underlying SPP size and sources.

Semantic Prediction in Brain and Mind.

We highlight a novel brain correlate of prediction, the prediction potential (or PP), a slow negative-going potential shift preceding visual, acoustic, and spoken or written verbal stimuli that can be predicted from their context. The cortical sources underlying the prediction potential reflect perceptual and semantic features of anticipated stimuli before these appear.

Prediction and Mismatch Negativity Responses Reflect Impairments in Action Semantic Processing in Adults With Autism Spectrum Disorders.

The neurophysiological mechanisms underlying motor and language difficulties in autism spectrum disorders (ASD) are still largely unclear. The present work investigates biological indicators of sound processing, (action-) semantic understanding and predictive coding and their correlation with clinical symptoms of ASD. Twenty-two adults with high-functioning ASD and 25 typically developed (TD) participants engaged in an auditory, passive listening, Mismatch Negativity (MMN) task while high-density electroencephalography (EEG) was recorded. Action and non-action words were presented in the context of sounds, which were either semantically congruent with regard to the body part they relate to or semantically incongruent or unrelated. The anticipatory activity before sound onset, the Prediction Potential (PP), was significantly reduced in the ASD group specifically for action, but not for non-action sounds. The early-MMN-like responses to words (latency: 120 ms) were differentially modulated across groups: controls showed larger amplitudes for words in action-sound compared to non-action contexts, whereas ASD participants demonstrated enlarged early-MMN-like responses only in a pure tone context, with no other modulation dependent on action sound context. Late-MMN-like responses around 560 ms post-stimulus onset revealed body-part-congruent action-semantic priming for words in control participants, but not in the ASD group. Importantly, neurophysiological indices of semantic priming in ASD participants correlated with the extent of autistic traits as revealed by the Autism Spectrum Quotient (AQ). The data suggest that high-functioning adults with ASD show a specific deficit in semantic processing and predictive coding of sounds and words related to action, which is absent for neutral, non-action, sounds.

Neurophysiological evidence for rapid processing of verbal and gestural information in understanding communicative actions.

During everyday social interaction, gestures are a fundamental part of human communication. The communicative pragmatic role of hand gestures and their interaction with spoken language has been documented at the earliest stage of language development, in which two types of indexical gestures are most prominent: the pointing gesture for directing attention to objects and the give-me gesture for making requests. Here we study, in adult human participants, the neurophysiological signatures of gestural-linguistic acts of communicating the pragmatic intentions of naming and requesting by simultaneously presenting written words and gestures. Already at ~150 ms, brain responses diverged between naming and request actions expressed by word-gesture combination, whereas the same gestures presented in isolation elicited their earliest neurophysiological dissociations significantly later (at ~210 ms). There was an early enhancement of request-evoked brain activity as compared with naming, which was due to sources in the frontocentral cortex, consistent with access to action knowledge in request understanding. In addition, an enhanced N400-like response indicated late semantic integration of gesture-language interaction. The present study demonstrates that word-gesture combinations used to express communicative pragmatic intentions speed up the brain correlates of comprehension processes - compared with gesture-only understanding - thereby calling into question current serial linguistic models viewing pragmatic function decoding at the end of a language comprehension cascade. Instead, information about the social-interactive role of communicative acts is processed instantaneously.

Prediction mechanisms in motor and auditory areas and their role in sound perception and language understanding.

Is the meaning of an expected stimulus manifest in brain activity even before it appears? Although theories of predictive coding see anticipatory activity as crucial for the understanding of brain function, few studies have explored neurophysiologically manifest semantic predictions. Here, we report predictive negative-going potentials before the onset of action (i.e. whistle and hand clap) and non-action (i.e. pure tone, water drop) sounds. These prediction potentials (PP) indexed the meaning of action-related sounds. Dependent on the body-part-relationship of sound stimuli, neural sources were relatively more prominent in dorsal or ventral motor areas. In contrast, meaningless sounds (pure tones) activated predictive sources in temporal areas close to the auditory cortex; complex environmental sounds induced an anticipatory positivity broadly distributed over the scalp. We also found a systematic relationship between predictive activity and a Mismatch Negativity (MMN) like response to unexpected meaningful words which were presented as rare deviant stimuli amongst frequently repeated sounds. This deviant-elicited potential indexed semantic priming between action sounds and action-related words and semantic mismatch (prediction error). These results suggest a systematic link between perceptual/semantic prediction and matching mechanisms in the processing of sounds and words.

Neural Correlates of Semantic Prediction and Resolution in Sentence Processing.

Most brain-imaging studies of language comprehension focus on activity following meaningful stimuli. Testing adult human participants with high-density EEG, we show that, already before the presentation of a critical word, context-induced semantic predictions are reflected by a neurophysiological index, which we therefore call the semantic readiness potential (SRP). The SRP precedes critical words if a previous sentence context constrains the upcoming semantic content (high-constraint contexts), but not in unpredictable (low-constraint) contexts. Specific semantic predictions were indexed by SRP sources within the motor system-in dorsolateral hand motor areas for expected hand-related words (e.g., "write"), but in ventral motor cortex for face-related words ("talk"). Compared with affirmative sentences, negated ones led to medial prefrontal and more widespread motor source activation, the latter being consistent with predictive semantic computation of alternatives to the negated expected concept. Predictive processing of semantic alternatives in negated sentences is further supported by a negative-going event-related potential at ∼400 ms (N400), which showed the typical enhancement to semantically incongruent sentence endings only in high-constraint affirmative contexts, but not to high-constraint negated ones. These brain dynamics reveal the interplay between semantic prediction and resolution (match vs error) processing in sentence understanding.SIGNIFICANCE STATEMENT Most neuroscientists agree on the eminent importance of predictive mechanisms for understanding basic as well as higher brain functions. This contrasts with a sparseness of brain measures that directly reflects specific aspects of prediction, as they are relevant in the processing of language and thought. Here we show that when critical words are strongly expected in their sentence context, a predictive brain response reflects meaning features of these anticipated symbols already before they appear. The granularity of the semantic predictions was so fine grained that the cortical sources in sensorimotor and medial prefrontal cortex even distinguished between predicted face- or hand-related action words (e.g., the words "lick" or "pick") and between affirmative and negated sentence meanings.

Somatotopic Semantic Priming and Prediction in the Motor System.

The recognition of action-related sounds and words activates motor regions, reflecting the semantic grounding of these symbols in action information; in addition, motor cortex exerts causal influences on sound perception and language comprehension. However, proponents of classic symbolic theories still dispute the role of modality-preferential systems such as the motor cortex in the semantic processing of meaningful stimuli. To clarify whether the motor system carries semantic processes, we investigated neurophysiological indexes of semantic relationships between action-related sounds and words. Event-related potentials revealed that action-related words produced significantly larger stimulus-evoked (Mismatch Negativity-like) and predictive brain responses (Readiness Potentials) when presented in body-part-incongruent sound contexts (e.g., "kiss" in footstep sound context; "kick" in whistle context) than in body-part-congruent contexts, a pattern reminiscent of neurophysiological correlates of semantic priming. Cortical generators of the semantic relatedness effect were localized in areas traditionally associated with semantic memory, including left inferior frontal cortex and temporal pole, and, crucially, in motor areas, where body-part congruency of action sound-word relationships was indexed by a somatotopic pattern of activation. As our results show neurophysiological manifestations of action-semantic priming in the motor cortex, they prove semantic processing in the motor system and thus in a modality-preferential system of the human brain.

Embodying Others in Immersive Virtual Reality: Electro-Cortical Signatures of Monitoring the Errors in the Actions of an Avatar Seen from a First-Person Perspective.

Brain monitoring of errors in one's own and other's actions is crucial for a variety of processes, ranging from the fine-tuning of motor skill learning to important social functions, such as reading out and anticipating the intentions of others. Here, we combined immersive virtual reality and EEG recording to explore whether embodying the errors of an avatar by seeing it from a first-person perspective may activate the error monitoring system in the brain of an onlooker. We asked healthy participants to observe, from a first- or third-person perspective, an avatar performing a correct or an incorrect reach-to-grasp movement toward one of two virtual mugs placed on a table. At the end of each trial, participants reported verbally how much they embodied the avatar's arm. Ratings were maximal in first-person perspective, indicating that immersive virtual reality can be a powerful tool to induce embodiment of an artificial agent, even through mere visual perception and in the absence of any cross-modal boosting. Observation of erroneous grasping from a first-person perspective enhanced error-related negativity and medial-frontal theta power in the trials where human onlookers embodied the virtual character, hinting at the tight link between early, automatic coding of error detection and sense of embodiment. Error positivity was similar in 1PP and 3PP, suggesting that conscious coding of errors is similar for self and other. Thus, embodiment plays an important role in activating specific components of the action monitoring system when others' errors are coded as if they are one's own errors. SIGNIFICANCE STATEMENT: Detecting errors in other's actions is crucial for social functions, such as reading out and anticipating the intentions of others. Using immersive virtual reality and EEG recording, we explored how the brain of an onlooker reacted to the errors of an avatar seen from a first-person perspective. We found that mere observation of erroneous actions enhances electrocortical markers of error detection in the trials where human onlookers embodied the virtual character. Thus, the cerebral system for action monitoring is maximally activated when others' errors are coded as if they are one's own errors. The results have important implications for understanding how the brain can control the external world and thus creating new brain-computer interfaces.

Rubber hand illusion highlights massive visual capture and sensorimotor face-hand remapping in a tetraplegic man.

PURPOSE: The illusory ownership of a fake hand as part the body follows synchronous tactile stimulation over a visible rubber hand and a covered hand. Whether brain plasticity mechanisms after sensory and motor disconnection modulates this illusion remain unexplored. METHODS: We tested a tetraplegic man after synchronous and asynchronous stimulation of the hand and face. RESULTS: The illusory ownership of the fake hand was tested four times in separate days and always reported. To verify whether this ownership feeling generalized also to object not resembling the human body we tested this illusion with a plastic bottle and a rubber hand. The illusionary perception of owning an external object using the rubber hand paradigm showed that the temporally matched tactile stimulation on a fake hand and visual capture mechanism create the illusionary feeling that the rubber hand was part of his body. CONCLUSIONS: Despite lesions that dramatically disconnect the access to sensory inputs and motor outputs our data suggests a strong visual capture of a rubber hand and a possible remapping of hand-face representations after the spinal lesion. We suggest that vision and brain plasticity may represent a supportive tool for motor rehabilitation in patients with sensory deficits.

The motor cost of telling lies: electrocortical signatures and personality foundations of spontaneous deception.

Although universal, lying is generally considered immoral behavior. Most neuroscience studies on lying sanction or instruct deceptive behaviors and thus might fail to acknowledge the significance of lie-related moral conflicts. By combining electroencephalogram (EEG) recordings with a novel paradigm in which participants decided freely whether to deceive another person, we have generated indices of the cognitive (reaction times and stimulus-locked event-related components) and moral (readiness potential and its correlations with deception-related personality traits) cost of spontaneous deception. Our data fail to support the consensus that deception is cognitively more demanding than truth telling, suggesting that spontaneous deception, as opposed to lying out of requirement, might not mandate additional cognitive workload. Interestingly, lying was associated with decreased motor readiness, an event-related potential (ERP) component that is linked to motor preparation of self-determined actions and modulated when we face moral dilemmas. Notably, this reduction was less extensive in manipulative participants and greater in those who cared highly about their impression management. Our study expands on previous findings on deception by associating a cortical marker of reduced preparation to act with individual differences in moral cognition.