Predictive language processing: integrating comprehension and production, and what atypical populations can tell us.

Predictive processing, a crucial aspect of human cognition, is also relevant for language comprehension. In everyday situations, we exploit various sources of information to anticipate and therefore facilitate processing of upcoming linguistic input. In the literature, there are a variety of models that aim at accounting for such ability. One group of models propose a strict relationship between prediction and language production mechanisms. In this review, we first introduce very briefly the concept of predictive processing during language comprehension. Secondly, we focus on models that attribute a prominent role to language production and sensorimotor processing in language prediction ("prediction-by-production" models). Contextually, we provide a summary of studies that investigated the role of speech production and auditory perception on language comprehension/prediction tasks in healthy, typical participants. Then, we provide an overview of the limited existing literature on specific atypical/clinical populations that may represent suitable testing ground for such models-i.e., populations with impaired speech production and auditory perception mechanisms. Ultimately, we suggest a more widely and in-depth testing of prediction-by-production accounts, and the involvement of atypical populations both for model testing and as targets for possible novel speech/language treatment approaches.

I know how you'll say it: evidence of speaker-specific speech prediction.

Most models of language comprehension assume that the linguistic system is able to pre-activate phonological information. However, the evidence for phonological prediction is mixed and controversial. In this study, we implement a paradigm that capitalizes on the fact that foreign speakers usually make phonological errors. We investigate whether speaker identity (native vs. foreign) is used to make specific phonological predictions. Fifty-two participants were recruited to read sentence frames followed by a last spoken word which was uttered by either a native or a foreign speaker. They were required to perform a lexical decision on the last spoken word, which could be either semantically predictable or not. Speaker identity (native vs. foreign) may or may not be cued by the face of the speaker. We observed that the face cue is effective in speeding up the lexical decision when the word is predictable, but it is not effective when the word is not predictable. This result shows that speech prediction takes into account the phonological variability between speakers, suggesting that it is possible to pre-activate in a detailed and specific way the phonological representation of a predictable word.

Pragmatic and knowledge range lenience towards foreigners.

Foreign-accented speech categorizes the speaker as an outgroup individual with a lower linguistic competence and a different knowledge heritage from a native speaker. Here we explore whether the identification of an individual as a native or a foreign speaker has an impact on trivia statement judgments, regardless of her foreign-accented speech. Italian native participants first read a bio description of a native and of a foreign speaker and then rate to what degree a series of statements associated with each of the speakers makes sense (Studies 1 and 2) or are true (Study 3). Importantly, the fluency processing between native and foreign speakers was kept constant by using a written presentation of the materials. Under-informative statements such as 'Some frogs are amphibians' were tested in Study 1. The results of Study 1 show more acceptable judgments when the sentences were associated with the foreign speaker. Unknown facts about world knowledge such as 'Butterflies do not see gray' were tested in Studies 2 and 3. The results show more acceptable (Study 2) and more true (Study 3) judgments when the sentences were associated with the foreign speaker. In addition, in Study 3 the foreign speaker was considered more trustworthy than the native speaker in a rating test at the end of the main judgment-sentence task. Our findings show that linguistic identity per se has an impact on evaluation judgments, suggesting that message interpretation cannot be dissociated from who is communicating the message.

Allophonic familiarity differentiates word representations in the brain of native speakers of regional linguistic varieties.

This study aims to shed light on the issue whether familiar allophonic variation is encoded in word representations. Both Italian speakers born in Trentino and speakers born in the Central-Southern regions of Italy took part in the experiment. We tested the MMN elicited by the same word encompassing two different allophones, one of which was more familiar to one group of participants than to the other, depending on their regional variety of Italian. The Trentino group showed an enhanced MMN for the word embedding the familiar variant while Central-Southern speakers showed no difference. The amplitude of the MMN for the unfamiliar word variant in Trentino speakers showed an inverse correlation with the passive exposure to the Trentino dialect. We conclude that words embedding familiar and unfamiliar allophones are differently represented in the brain of native speakers of regional language and the degree of differentiation is modulated by individual experience.

The unique role of novel linguistic labels on the disengagement of visual attention.

Do novel linguistic labels have privileged access to attentional resources compared to non-linguistic labels? This study explores this possibility through two experiments with a training and an attentional overlap task. Experiment 1 investigates how novel label and object-only stimuli influence resource allocation and disengagement of visual attention. Experiment 2 tests the impact of linguistic information on visual attention by comparing novel tones and labels. Because disengagement of attention is affected both by the salience of the perceptual stimulus and by the degree of familiarity with the stimulus to be disengaged from, we compared pupil size variations and saccade latency under different test conditions: (a) consistent with (i.e., identical to) the training; (b) inconsistent with the training (i.e., with an altered feature), and (c) deprived of one feature only in Experiment 1. Experiment 1 indicated a general consistency advantage (and deprived disadvantage) driven by linguistic label-object pairs compared to object-only stimuli. Experiment 2 revealed that tone-object pairs led to higher pupil dilation and longer saccade latency than linguistic label-object pairs. Our results suggest that novel linguistic labels preferentially impact the early orienting of attention.

Distinct Neural Processes for Memorizing Form and Meaning Within Sentences.

In order to memorize sentences we use both processes of language comprehension during encoding and processes of language production during maintenance. While the former processes are easily testable via controlled presentation of the input, the latter are more difficult to assess directly as language production is typically initiated and controlled internally. In the present event-related potential (ERP) study we track subvocal rehearsal of sentences, with the goal of studying the concomitant planning processes with the help of a silent cued-production task. Native German participants read different types of sentences word-by-word, then were prompted by a visual cue to silently repeat each individual word, in a rehearsal phase. In order to assess both local and global effects of sentence planning, we presented correct sentences, syntactically or semantically violated sentences, or random word order sequences. Semantic violations during reading elicited an N400 effect at the noun violating the selectional restrictions of the preceding verb. Syntactic violations, induced by a gender incongruency between determiner and noun, led to a P600 effect at the same position. Different ERP patterns occurred during the silent production phase. Here, semantically violated sentences elicited an early fronto-central negativity at the verb, while syntactically violated sentences elicited a late right-frontal positivity at the determiner. Random word order was accompanied by long-lasting slow waves during the production phase. The findings are consistent with models of hierarchical sentence planning and further indicate that the ongoing working memory processes are qualitatively distinct from comprehension mechanisms and neurophysiologically specific for syntactic and lexical-semantic level planning. In conclusion, active working memory maintenance of sentences is likely to comprise specific stages of sentence production that are indicated by ERP correlates of syntactic and semantic planning at the phrasal and clausal level respectively.

Attachment and Concord of Temporal Adverbs: Evidence From Eye Movements.

The present study examined the processing of temporal adverbial phrases such as "last week," which must agree in temporal features with the verb they modify. We investigated readers' sensitivity to this feature match or mismatch in two eye-tracking studies. The main aim of this study was to expand the range of concord phenomena which have been investigated in real-time processing in order to understand how linguistic dependencies are formed during sentence comprehension (Felser et al., 2017). Under a cue-based perspective, linguistic dependency formation relies on an associative cue-based retrieval mechanism (Lewis et al., 2006; McElree, 2006), but how such a mechanism is deployed over diverse linguistic dependencies remains a matter of debate. Are all linguistic features candidate cues that guide retrieval? Are all cues given similar weight? Are different cues differently weighted based on the dependency being processed? To address these questions, we implemented a mismatch paradigm (Sturt, 2003) adapted for temporal concord dependencies. This paradigm tested whether readers were sensitive to a temporal agreement between a temporal adverb like last week and a linearly distant, but structurally accessible verb, as well as a linearly proximate but structurally inaccessible verb. We found clear evidence that readers were sensitive to feature match between the adverb and the linearly distant, structurally accessible verb. We found no clear evidence on whether feature match with the inaccessible verb impacted the processing of a temporal adverb. Our results suggest syntactic positional information plays an important role during the processing of the temporal concord relation.

Seeing emotions, reading emotions: Behavioral and ERPs evidence of the regulation of pictures and words.

BACKGROUND: Whilst there has been extensive study of the mechanisms underlying the regulation for pictures, the ability and the mechanisms beyond the regulation of words remains to be clarified. Similarly, the effect of strategy when applying a regulatory process is still poorly explored. The present study seeks to elucidate these issues comparing the effect of regulation and of strategy to both neutral and emotional words and pictures. METHODOLOGY/PRINCIPAL FINDINGS: Thirty young adults applied the strategy of distancing to the emotions elicited by unpleasant and neutral pictures and words while their subjective ratings and ERPs were recorded. At a behavioral level, participants successfully regulated the arousal and the valence of both pictures and words. At a neural level, unpleasant pictures produced an increase in the late positive potential modulated during the regulate condition. Unpleasant linguistic stimuli elicited a posterior negativity as compared to neutral stimuli, but no effect of regulation on ERP was detectable. More importantly, the effect of strategy independently of stimulus type, produced a significant larger Stimulus Preceding Negativity. Dipole reconstruction localized this effect in the middle frontal areas of the brain. CONCLUSIONS: As such, these new psychophysiological findings might help to understand how pictures and words can be regulated by distancing in daily life and clinical contexts, and the neural bases of the effect of strategy for which we suggest an integrative model.

Basic composition and enriched integration in idiom processing: An EEG study.

We investigated the extent to which the literal meanings of the words forming literally plausible idioms (e.g., break the ice) are semantically composed and how the idiomatic meaning is integrated in the unfolding sentence representation. Participants read ambiguous idiom strings embedded in highly predictable, literal, and idiomatic contexts while their EEG was recorded. Control sentences only contained the idiom-final word whose cloze values were as high as in literal and idiomatic contexts. Event-related potentials data showed that differences in the amplitude of a frontal positivity (PNP) emerged at the beginning and at the end of the idiom strings, with the idiomatic context condition associated with more positive voltages. The time frequency analysis of the EEG showed an increase in power of the middle gamma frequency band only in the literal context condition. These findings suggest that sentence revision mechanisms, associated with the frontal PNP, are involved in idiom meaning integration, and that the literal semantic composition of the idiomatic constituents, associated with changes in gamma frequency, is not carried out after idiom recognition. (PsycINFO Database Record

First Language Attrition Induces Changes in Online Morphosyntactic Processing and Re-Analysis: An ERP Study of Number Agreement in Complex Italian Sentences.

First language (L1) attrition in adulthood offers new insight on neuroplasticity and the role of language experience in shaping neurocognitive responses to language. Attriters are multilinguals for whom advancing L2 proficiency comes at the cost of the L1, as they experience a shift in exposure and dominance (e.g., due to immigration). To date, the neurocognitive mechanisms underlying L1 attrition are largely unexplored. Using event-related potentials (ERPs), we examined L1-Italian grammatical processing in 24 attriters and 30 Italian native-controls. We assessed whether (a) attriters differed from non-attriting native speakers in their online detection and re-analysis/repair of number agreement violations, and whether (b) differences in processing were modulated by L1-proficiency. To test both local and non-local agreement violations, we manipulated agreement between three inflected constituents and examined ERP responses on two of these (subject, verb, modifier). Our findings revealed group differences in amplitude, scalp distribution, and duration of LAN/N400 + P600 effects. We discuss these differences as reflecting influence of attriters' L2-English, as well as shallower online sentence repair processes than in non-attriting native speakers. ERP responses were also predicted by L1-Italian proficiency scores, with smaller N400/P600 amplitudes in lower proficiency individuals. Proficiency only modulated P600 amplitude between 650 and 900 ms, whereas the late P600 (beyond 900 ms) depended on group membership and amount of L1 exposure within attriters. Our study is the first to show qualitative and quantitative differences in ERP responses in attriters compared to non-attriting native speakers. Our results also emphasize that proficiency predicts language processing profiles, even in native-speakers, and that the P600 should not be considered a monolithic component.

On the time course of lexical stress priming in speech production: Behavioral and ERPs evidence from a free-stress language.

The goal of the present research was to study the time course of lexical stress encoding in a free-stress language with unpredictable stress. To this aim we measured event-related brain potentials (ERPs) during lexical priming. Participants named pictures bearing either the dominant or non-dominant stress pattern, and preceded by either a congruent or an incongruent word prime (e.g., CInema-FRAgola'cinema-strawberry' vs. benZIna-FRAgola'petrol-strawberry'). Behavioral results show that participants were slower in naming targets that had the same stress pattern as the prime, and were also faster in producing words with the dominant stress pattern in the language. The electrophysiological results show that both the effects are compatible with the time course of phonological encoding in speech production. Surprisingly, a dominant stress effect occurred in the ERPs elicited by the primes, with a larger positivity for non-dominant stress words in a 150-250ms time-window. The pattern of results indicates that during speech production: a) the system is sensitive to the stress patterns distribution; b) the automatic pre-activation of a metrical frame may interfere with the phonological encoding of a to-be-uttered word.

The Sound of Voice: Voice-Based Categorization of Speakers' Sexual Orientation within and across Languages.

Empirical research had initially shown that English listeners are able to identify the speakers' sexual orientation based on voice cues alone. However, the accuracy of this voice-based categorization, as well as its generalizability to other languages (language-dependency) and to non-native speakers (language-specificity), has been questioned recently. Consequently, we address these open issues in 5 experiments: First, we tested whether Italian and German listeners are able to correctly identify sexual orientation of same-language male speakers. Then, participants of both nationalities listened to voice samples and rated the sexual orientation of both Italian and German male speakers. We found that listeners were unable to identify the speakers' sexual orientation correctly. However, speakers were consistently categorized as either heterosexual or gay on the basis of how they sounded. Moreover, a similar pattern of results emerged when listeners judged the sexual orientation of speakers of their own and of the foreign language. Overall, this research suggests that voice-based categorization of sexual orientation reflects the listeners' expectations of how gay voices sound rather than being an accurate detector of the speakers' actual sexual identity. Results are discussed with regard to accuracy, acoustic features of voices, language dependency and language specificity.

Asymmetric activation spreading in the multiplication associative network due to asymmetric overlap between numerosities semantic representations?

Simple multiplication facts are thought to be organised in a network structure in which problems and solutions are associated. Converging evidence suggests that the ability for solving symbolic arithmetic problems is based on an approximate number system (ANS). Most theoretical stances concerning the metric underlying the ANS converge on the assumption that the representational overlap between two adjacent numbers increases as the numerical magnitude of the numbers increases. Given a number N, the overlap between N and N+1 is larger than the overlap between N and N-1. Here, we test whether this asymmetric overlap influences the activation spreading within the multiplication associative network (MAN). When verifying simple multiplication problems such as 8×4 participants were slower in rejecting false but related outcomes that were larger than the actual outcome (e.g., 8×4=36) than rejecting smaller related outcomes (e.g., 8×4=28), despite comparable numerical distance from the correct result (here: 4). This effect was absent for outcomes which are not part of either operands table (e.g., 8×4=35). These results suggest that the metric of the ANS influences the activation spreading within the MAN, further substantiating the notion that symbolic arithmetic is grounded in the ANS.

Is the noun ending a cue to grammatical gender processing? An ERP study on sentences in Italian.

Gender-to-ending consistency has been shown to influence grammatical gender retrieval in isolated word presentation. Notwithstanding the wealth of evidence, the exact role and the time course of processing of this distributional information remain unclear. This ERP study investigated if and when the brain detects gender-to-ending consistency in sentences containing Italian determiner-noun pairs. Determiners either agreed or disagreed in gender with the nouns whose endings were reliable or misleading cues to gender (transparent and irregular nouns). Transparent nouns elicited an increased frontal negativity and a late posterior positivity compared to irregular nouns (350-950 ms), suggesting that the system is sensitive to gender-to-ending consistency from relatively early stages of processing. Gender agreement violations evoked a similar LAN-P600 pattern for both types of nouns. The present findings provide evidence for an early detection of reliable gender-related endings during sentence reading.

Operand-order effect in multiplication and addition: the long-term effects of reorganization processand acquisition sequence.

Butterworth, Marchesini, and Girelli (2003) showed that children solved multiplications faster when the larger operand was first (e.g., 5 · 2) than when the smaller operand was first (e.g., 2 · 5). This result was interpreted according to the reorganization hypothesis, which states that, as children begin to switch from counting-based strategies (e.g., repeated additions) to direct retrieval, non-retrieval strategies generate an advantage for the larger-operand-first order. In two experiments we showed that order preferences also persist into adulthood. With additions, the larger-operand-first order was solved faster than the inverse order. With multiplications we obtained a novel result: Largeroperand-first problems were solved faster when at least one operand was smaller than 5, whereas smaller-operand-first problems were solved faster when both operands were larger than 5. Since the reorganization process alone cannot explain our results, we propose that order preferences are also influenced by the sequence in which the members of a commuted pair are acquired.

Predictive mechanisms in idiom comprehension.

Prediction is pervasive in human cognition and plays a central role in language comprehension. At an electrophysiological level, this cognitive function contributes substantially in determining the amplitude of the N400. In fact, the amplitude of the N400 to words within a sentence has been shown to depend on how predictable those words are: The more predictable a word, the smaller the N400 elicited. However, predictive processing can be based on different sources of information that allow anticipation of upcoming constituents and integration in context. In this study, we investigated the ERPs elicited during the comprehension of idioms, that is, prefabricated multiword strings stored in semantic memory. When a reader recognizes a string of words as an idiom before the idiom ends, she or he can develop expectations concerning the incoming idiomatic constituents. We hypothesized that the expectations driven by the activation of an idiom might differ from those driven by discourse-based constraints. To this aim, we compared the ERP waveforms elicited by idioms and two literal control conditions. The results showed that, in both cases, the literal conditions exhibited a more negative potential than the idiomatic condition. Our analyses suggest that before idiom recognition the effect is due to modulation of the N400 amplitude, whereas after idiom recognition a P300 for the idiomatic sentence has a fundamental role in the composition of the effect. These results suggest that two distinct predictive mechanisms are at work during language comprehension, based respectively on probabilistic information and on categorical template matching.

Brain MRI diffusion-weighted imaging in patients with classical phenylketonuria.

INTRODUCTION: The aim of this study was to grade magnetic resonance white matter abnormalities (WMAs) of classical phenylketonuria (cPKU) patients treated from birth and to compare sensitivity and specificity of T2-weighted and diffusion-weighted images (DWI). METHODS: Twenty early-treated cPKU patients still on a low-phenylalanine diet (12 males; mean age 21.2 years) and 26 normal subjects (ten males; mean age 25.1 years) were enrolled. Typical T2- and diffusion-weighted WMAs were semiquantitatively graded according to Thompson score (TS). Besides, a regional magnetic resonance imaging (MRI) score (mTS) was developed according to extension and intensity of WMAs. Phenylalanine and tyrosine plasma concentrations before performing MRI and the amino acid mean levels collected the year before MRI (Tyr(year) and Phe(year)) were measured. RESULTS: No patient with Phe(year) concentration below 460 mumol/L showed WMAs. In cPKU patients, TS and mTS were significantly higher on DWI than on T2 images (3.50 vs 2.65 and 23.65 vs 15.85, respectively, p < 0.002, Wilcoxon test). All controls were scored 0 on DWI, while in T2 images, TS and mTS were 0.19 and 1.70. DWI evaluated by mTS disclosed a frontotemporal, occipital, and parietal WM progressive involvement. TS and mTS, both on T2 images and on DWI, showed no correlation with tyrosine while they proved to have a strong correlation with phenylalaninemia and an excellent one with Phe(year) levels. CONCLUSION: Among the different MR sequences, DWI seems to be the most sensitive and reliable in detecting and grading the typical WMAs of cPKU patients.

Cloze probability does not only affect N400 amplitude: the case of complex prepositions.

Cloze-probability levels are inversely correlated with N400 amplitude, indicating an easier integration for expected words in semantic-pragmatic contexts. Here we exploited the prespecified standard order of complex prepositions and measured the ERPs time-locked to the last preposition in sentences in which complex prepositions were presented in their standard form or with the last preposition changed. The expected preposition elicited an N280 followed by an N400-700, two ERP components previously associated to the processing of closed-class words. The unexpected preposition elicited only an N280, and the N400-700 was reduced. These results reflect the specificity of the contextual constraints linked to the complex preposition word sequence.

A deeper reanalysis of a superficial feature: an ERP study on agreement violations.

A morphosyntactic agreement violation during reading elicits a well-documented biphasic ERP pattern (LAN+P600). The cognitive variables that affect both the amplitude of the two components and the topography of the anterior negativity are still debated. We studied the ERP correlates of the violation of a specific agreement feature based on the phonology of the critical word. This was compared with the violation of a lexical feature, namely grammatical gender. These two features are different both in the level of representation involved in the agreement computation and in terms of their role in establishing structural relations with possible following constituents. The ERP pattern elicited by the two agreement violations showed interesting dissociations. The LAN was distributed ventrally for both types of violation, but showed a central extension for the gender violation. The P600 showed an amplitude modulation: this component was larger for phonotactic violations in its late time window (700-900 ms). The former result is indicative of a difference in the brain structures recruited for the processing of violations at different levels of representation. The P600 effect is interpreted assuming a hierarchical relation among features that forces a deeper reanalysis of the violation involving a word form property. Finally the two features elicit distinct end-of-sentence wrap-up effects, consistent with the different roles they play in the processing of the whole sentence.

Anaphoric agreement violation: an ERP analysis of its interpretation.

In the present study we analyzed the processing of grammatically anomalous sentences like "The famous dancer were nervously preparing herself/themselves to face the crowd.", which contains two anomalies, one early and one late. We investigated how processing of the later anomaly (at the pronoun 'herself' or 'themselves') was affected by the processing of the early anomaly (at 'were'). We considered two processing scenarios involving the first anomaly: (1) The representation of the subject-verb number agreement error at the first verb is coerced to match the verb, rendering 'herself' anomalous; (2) The representation of the subject-verb agreement error is coerced to match the subject noun, rendering 'themselves' anomalous. Our dependent measure was event-related scalp potentials (ERPs). When the pronoun disagreed with the verb (and agreed with the subject), a P600 was recorded, while the opposite condition elicited no reliable effect. Our data suggest that interpretation of reflexive pronouns involves the reactivation of multiple lexical items, verbs included.