Exploring the Interplay Between Language Comprehension and Cortical Tracking: The Bilingual Test Case.

Cortical tracking, the synchronization of brain activity to linguistic rhythms is a well-established phenomenon. However, its nature has been heavily contested: Is it purely epiphenomenal or does it play a fundamental role in speech comprehension? Previous research has used intelligibility manipulations to examine this topic. Here, we instead varied listeners' language comprehension skills while keeping the auditory stimulus constant. To do so, we tested 22 native English speakers and 22 Spanish/Catalan bilinguals learning English as a second language (SL) in an EEG cortical entrainment experiment and correlated the responses with the magnitude of the N400 component of a semantic comprehension task. As expected, native listeners effectively tracked sentential, phrasal, and syllabic linguistic structures. In contrast, SL listeners exhibited limitations in tracking sentential structures but successfully tracked phrasal and syllabic rhythms. Importantly, the amplitude of the neural entrainment correlated with the amplitude of the detection of semantic incongruities in SLs, showing a direct connection between tracking and the ability to understand speech. Together, these findings shed light on the interplay between language comprehension and cortical tracking, to identify neural entrainment as a fundamental principle for speech comprehension.

Inflection across Categories: Tracking Abstract Morphological Processing in Language Production with MEG.

Coherent language production requires that speakers adapt words to their grammatical contexts. A fundamental challenge in establishing a functional delineation of this process in the brain is that each linguistic process tends to correlate with numerous others. Our work investigated the neural basis of morphological inflection by measuring magnetoencephalography during the planning of inflected and uninflected utterances that varied across several linguistic dimensions. Results reveal increased activity in the left lateral frontotemporal cortex when inflection is planned, irrespective of phonological specification, syntactic context, or semantic type. Additional findings from univariate and connectivity analyses suggest that the brain distinguishes between different types of inflection. Specifically, planning noun and verb utterances requiring the addition of the suffix -s elicited increased activity in the ventral prefrontal cortex. A broadly distributed effect of syntactic context (verb vs. noun) was also identified. Results from representational similarity analysis indicate that this effect cannot be explained in terms of word meaning. Together, these results 1) offer evidence for a neural representation of abstract inflection that separates from other stimulus properties and 2) challenge theories that emphasize semantic content as a source of verb/noun processing differences.

Natural vs forced language switching: Free selection and consistent language use eliminate significant performance costs and cognitive demands in the brain.

Bilinguals are known to switch language spontaneously in everyday conversations, even if there are no external requirements to do so. However, in the laboratory setting, language control is often investigated using forced switching tasks, which result in significant performance costs. The present study assessed whether switching would be less costly when performed in a more natural fashion, and what factors might account for this. Mandarin-English bilinguals engaged in language switching under three different contexts with varied task demands. We examined two factors which may be characteristic of natural switching: (i) freedom of language selection; (ii) consistency of language used to name each item. Participants' brain activities were recorded using magnetoencephalography (MEG), along with behavioural measures of reaction speed and accuracy. The natural context (with both free selection and consistent language use for each item) produced better performance overall, showing reduced mixing cost and no significant switch cost. The neural effect of language mixing was also reversed in this context, suggesting that freely mixing two languages was easier than staying in a single language. Further, while switching in the forced context elicited increased brain activity in the right inferior frontal gyrus, this switch effect disappeared when the language used to name each item was consistent. Together, these findings demonstrate that the two factors above conjointly contribute to eliminating significant performance costs and cognitive demands associated with language switching and mixing. Such evidence aligns with lexical selection models which do not assume bilingual production to be inherently effortful.

Neural representation of words within phrases: Temporal evolution of color-adjectives and object-nouns during simple composition.

In language, stored semantic representations of lexical items combine into an infinitude of complex expressions. While the neuroscience of composition has begun to mature, we do not yet understand how the stored representations evolve and morph during composition. New decoding techniques allow us to crack open this very hard question: we can train a model to recognize a representation in one context or time-point and assess its accuracy in another. We combined the decoding approach with magnetoencephalography recorded during a picture naming task to investigate the temporal evolution of noun and adjective representations during speech planning. We tracked semantic representations as they combined into simple two-word phrases, using single words and two-word lists as non-combinatory controls. We found that nouns were generally more decodable than adjectives, suggesting that noun representations were stronger and/or more consistent across trials than those of adjectives. When training and testing across contexts and times, the representations of isolated nouns were recoverable when those nouns were embedded in phrases, but not so if they were embedded in lists. Adjective representations did not show a similar consistency across isolated and phrasal contexts. Noun representations in phrases also sustained over time in a way that was not observed for any other pairing of word class and context. These findings offer a new window into the temporal evolution and context sensitivity of word representations during composition, revealing a clear asymmetry between adjectives and nouns. The impact of phrasal contexts on the decodability of nouns may be due to the nouns' status as head of phrase-an intriguing hypothesis for future research.

A common selection mechanism at each linguistic level in bilingual and monolingual language production.

The primary goal of research on the functional and neural architecture of bilingualism is to elucidate how bilingual individuals' language architecture is organized such that they can both speak in a single language without accidental insertions of the other, but also flexibly switch between their two languages if the context allows/demands them to. Here we review the principles under which any proposed architecture could operate, and present a framework where the selection mechanism for individual elements strictly operates on the basis of the highest level of activation and does not require suppressing representations in the non-target language. We specify the conjunction of parameters and factors that jointly determine these levels of activation and develop a theory of bilingual language organization that extends beyond the lexical level to other levels of representation (i.e., semantics, morphology, syntax and phonology). The proposed architecture assumes a common selection principle at each linguistic level to account for attested features of bilingual speech in, but crucially also out, of experimental settings.

Adaptation to mis-pronounced speech: evidence for a prefrontal-cortex repair mechanism.

Speech is a complex and ambiguous acoustic signal that varies significantly within and across speakers. Despite the processing challenge that such variability poses, humans adapt to systematic variations in pronunciation rapidly. The goal of this study is to uncover the neurobiological bases of the attunement process that enables such fluent comprehension. Twenty-four native English participants listened to words spoken by a "canonical" American speaker and two non-canonical speakers, and performed a word-picture matching task, while magnetoencephalography was recorded. Non-canonical speech was created by including systematic phonological substitutions within the word (e.g. [s] → [sh]). Activity in the auditory cortex (superior temporal gyrus) was greater in response to substituted phonemes, and, critically, this was not attenuated by exposure. By contrast, prefrontal regions showed an interaction between the presence of a substitution and the amount of exposure: activity decreased for canonical speech over time, whereas responses to non-canonical speech remained consistently elevated. Grainger causality analyses further revealed that prefrontal responses serve to modulate activity in auditory regions, suggesting the recruitment of top-down processing to decode non-canonical pronunciations. In sum, our results suggest that the behavioural deficit in processing mispronounced phonemes may be due to a disruption to the typical exchange of information between the prefrontal and auditory cortices as observed for canonical speech.

On the Need for Theoretically Guided Approaches to Possible Bilingual Advantages: An Evaluation of the Potential Loci in the Language and Executive Control Systems.

Whether a cognitive advantage exists for bilingual individuals has been the source of heated debate in the last decade. While empirical evidence putatively in favor of or against this alleged advantage has been frequently discussed, the potential sources of enhanced cognitive control in bilinguals have only been broadly declared, with no mechanistic elaboration of where, why, and how this purported link between bilingualism and enhanced language control develops, and how this enhancement transfers to, and subsequently improves, general executive function. Here, we evaluate different potential sources for a bilingual advantage and develop the assumptions one would have to make about the language processing system to be consistent with each of these notions. Subsequently, we delineate the limitations in the generalizations from language to overall executive function, and characterize where these advantages could be identified if there were to be any. Ultimately, we conclude that in order to make significant progress in this area, it is necessary to look for advantages in theoretically motivated areas, and that in the absence of clear theories as to the source, transfer, and target processes that could lead to potential advantages, an inconsistent body of results will follow, making the whole pursuit of a bilingual advantage moot.

Understanding Requires Tracking: Noise and Knowledge Interact in Bilingual Comprehension.

Understanding speech in noise is a fundamental challenge for speech comprehension. This perceptual demand is amplified in a second language: It is a common experience in bars, train stations, and other noisy environments that degraded signal quality severely compromises second language comprehension. Through a novel design, paired with a carefully selected participant profile, we independently assessed signal-driven and knowledge-driven contributions to the brain bases of first versus second language processing. We were able to dissociate the neural processes driven by the speech signal from the processes that come from speakers' knowledge of their first versus second languages. The neurophysiological data show that, in combination with impaired access to top-down linguistic information in the second language, the locus of bilinguals' difficulty in understanding second language speech in noisy conditions arises from a failure to successfully perform a basic, low-level process: cortical entrainment to speech signals above the syllabic level.

The role of proactive control on subcomponents of language control: Evidence from trilinguals.

Language control in bilingual individuals has been the source of thorough study in the last decade. However, the characterization of the subcomponents of this cognitive process remains shallow. In this experiment we tested Chinese, English, and Japanese trilinguals who completed a modified language-switching task. Participants named pictures in one of three conditions: Repeat language, Switch-away from a language (target language undetermined) and Switch-to a particular language (target language determined). Brain activity was recorded by electroencephalogram (EEG) and general proactive control ability was measured independently by the AX-Continuous Performance Test (AX-CPT). Switch-to and Switch-away processes elicited distinct neural signatures. Both at the cue and stimulus stage, Switching away elicited more negative activity at an early time window (N2); and less positive activity at the later time window (LPC). Further, at the cue stage this amplitude was negatively correlated with the proactive control index. These results show that the different subcomponents of cued-switching are dissociable and that there is a direct relation between the online signatures elicited by some of these processes and the general proactive control abilities of individuals.

Ecological Validity in Bilingualism Research and the Bilingual Advantage.

Traditional research in bilingualism has consistently found that switching languages is effortful, placing demands on neural systems of cognitive control. This finding runs counter to most bilinguals' intuitive experience. We review a body of recent work showing that, in fact, when bilinguals switch languages voluntarily, both the behavioral cost of switching and the associated recruitment of cognitive control areas are greatly reduced or completely eliminated. This suggests that switching languages is not inherently effortful, but rather, particular communicative demands may make it costly. The new evidence also challenges the basic premise underlying the bilingual advantage hypothesis. We articulate a more nuanced version of it, in which the advantage is limited to bilinguals who frequently switch languages based on external constraints.

Language switching decomposed through MEG and evidence from bimodal bilinguals.

A defining feature of human cognition is the ability to quickly and accurately alternate between complex behaviors. One striking example of such an ability is bilinguals' capacity to rapidly switch between languages. This switching process minimally comprises disengagement from the previous language and engagement in a new language. Previous studies have associated language switching with increased prefrontal activity. However, it is unknown how the subcomputations of language switching individually contribute to these activities, because few natural situations enable full separation of disengagement and engagement processes during switching. We recorded magnetoencephalography (MEG) from American Sign Language-English bilinguals who often sign and speak simultaneously, which allows to dissociate engagement and disengagement. MEG data showed that turning a language "off" (switching from simultaneous to single language production) led to increased activity in the anterior cingulate cortex (ACC) and dorsolateral prefrontal cortex (dlPFC), while turning a language "on" (switching from one language to two simultaneously) did not. The distinct representational nature of these on and off processes was also supported by multivariate decoding analyses. Additionally, Granger causality analyses revealed that (i) compared with "turning on" a language, "turning off" required stronger connectivity between left and right dlPFC, and (ii) dlPFC activity predicted ACC activity, consistent with models in which the dlPFC is a top-down modulator of the ACC. These results suggest that the burden of language switching lies in disengagement from the previous language as opposed to engaging a new language and that, in the absence of motor constraints, producing two languages simultaneously is not necessarily more cognitively costly than producing one.

Shared neural correlates for building phrases in signed and spoken language.

Research on the mental representation of human language has convincingly shown that sign languages are structured similarly to spoken languages. However, whether the same neurobiology underlies the online construction of complex linguistic structures in sign and speech remains unknown. To investigate this question with maximally controlled stimuli, we studied the production of minimal two-word phrases in sign and speech. Signers and speakers viewed the same pictures during magnetoencephalography recording and named them with semantically identical expressions. For both signers and speakers, phrase building engaged left anterior temporal and ventromedial cortices with similar timing, despite different linguistic articulators. Thus the neurobiological similarity of sign and speech goes beyond gross measures such as lateralization: the same fronto-temporal network achieves the planning of structured linguistic expressions.

The priming of basic combinatory responses in MEG.

Priming has been a powerful tool for the study of human memory and especially the memory representations relevant for language. However, although it is well established that lexical access can be primed, we do not know exactly what types of computations can be primed above the word level. This work took a neurobiological approach and assessed the ways in which the complex representation of a minimal combinatory phrase, such as red boat, can be primed, as evidenced by the spatiotemporal profiles of magnetoencephalography (MEG) signals. Specifically, we built upon recent progress on the neural signatures of phrasal composition and tested whether the brain activities implicated for the basic combination of two words could be primed. In two experiments, MEG was recorded during a picture naming task where the prime trials were designed to replicate previously reported combinatory effects and the target trials to test whether those combinatory effects could be primed. The manipulation of the primes was successful in eliciting larger activity for adjective-noun combinations than single nouns in left anterior temporal and ventromedial prefrontal cortices, replicating prior MEG studies on parallel contrasts. Priming of similarly timed activity was observed during target trials in anterior temporal cortex, but only when the prime and target shared an adjective. No priming in temporal cortex was observed for single word repetition and two control tasks showed that the priming effect was not elicited if the prime pictures were simply viewed but not named. In sum, this work provides evidence that very basic combinatory operations can be primed, with the necessity for some lexical overlap between prime and target suggesting combinatory conceptual, as opposed to syntactic processing. Both our combinatory and priming effects were early, onsetting between 100 and 150ms after picture onset and thus are likely to reflect the very earliest planning stages of a combinatory message. Thus our findings suggest that at the earliest stages of combinatory planning in production, a combinatory memory representation is formed that affects the planning of a relevantly similar combination on a subsequent trial.

Bilingual Language Switching in the Laboratory versus in the Wild: The Spatiotemporal Dynamics of Adaptive Language Control.

For a bilingual human, every utterance requires a choice about which language to use. This choice is commonly regarded as part of general executive control, engaging prefrontal and anterior cingulate cortices similarly to many types of effortful task switching. However, although language control within artificial switching paradigms has been heavily studied, the neurobiology of natural switching within socially cued situations has not been characterized. Additionally, although theoretical models address how language control mechanisms adapt to the distinct demands of different interactional contexts, these predictions have not been empirically tested. We used MEG (RRID: NIFINV:nlx_inv_090918) to investigate language switching in multiple contexts ranging from completely artificial to the comprehension of a fully natural bilingual conversation recorded "in the wild." Our results showed less anterior cingulate and prefrontal cortex involvement for more natural switching. In production, voluntary switching did not engage the prefrontal cortex or elicit behavioral switch costs. In comprehension, while laboratory switches recruited executive control areas, fully natural switching within a conversation only engaged auditory cortices. Multivariate pattern analyses revealed that, in production, interlocutor identity was represented in a sustained fashion throughout the different stages of language planning until speech onset. In comprehension, however, a biphasic pattern was observed: interlocutor identity was first represented at the presentation of the interlocutor and then again at the presentation of the auditory word. In all, our findings underscore the importance of ecologically valid experimental paradigms and offer the first neurophysiological characterization of language control in a range of situations simulating real life to various degrees.SIGNIFICANCE STATEMENT Bilingualism is an inherently social phenomenon, interactional context fully determining language choice. This research addresses the neural mechanisms underlying multilingual individuals' ability to successfully adapt to varying conversational contexts both while speaking and listening. Our results showed that interactional context critically determines language control networks' engagement: switching under external constraints heavily recruited prefrontal control regions, whereas natural, voluntary switching did not. These findings challenge conclusions derived from artificial switching paradigms, which suggested that language switching is intrinsically effortful. Further, our results predict that the so-called bilingual advantage should be limited to individuals who need to control their languages according to external cues and thus would not occur by virtue of an experience in which switching is fully free.

Bilingual Language Control in Perception versus Action: MEG Reveals Comprehension Control Mechanisms in Anterior Cingulate Cortex and Domain-General Control of Production in Dorsolateral Prefrontal Cortex.

For multilingual individuals, adaptive goal-directed behavior as enabled by cognitive control includes the management of two or more languages. This work used magnetoencephalography (MEG) to investigate the degree of neural overlap between language control and domain-general cognitive control both in action and perception. Highly proficient Arabic-English bilingual individuals participated in maximally parallel language-switching tasks in production and comprehension as well as in analogous tasks in which, instead of the used language, the semantic category of the comprehended/produced word changed. Our results indicated a clear dissociation of language control mechanisms in production versus comprehension. Language-switching in production recruited dorsolateral prefrontal regions bilaterally and, importantly, these regions were similarly recruited by category-switching. Conversely, effects of language-switching in comprehension were observed in the anterior cingulate cortex and were not shared by category-switching. These results suggest that bilingual individuals rely on adaptive language control strategies and that the neural involvement during language-switching could be extensively influenced by whether the switch is active (e.g., in production) or passive (e.g., in comprehension). In addition, these results support that humans require high-level cognitive control to switch languages in production, but the comprehension of language switches recruits a distinct neural circuitry. The use of MEG enabled us to obtain the first characterization of the spatiotemporal profile of these effects, establishing that switching processes begin ∼ 400 ms after stimulus presentation. SIGNIFICANCE STATEMENT: This research addresses the neural mechanisms underlying multilingual individuals' ability to successfully manage two or more languages, critically targeting whether language control is uniform across linguistic domains (production and comprehension) and whether it is a subdomain of general cognitive control. The results showed that language production and comprehension rely on different networks: whereas language control in production recruited domain-general networks, the brain bases of switching during comprehension seemed language specific. Therefore, the crucial assumption of the bilingual advantage hypothesis, that there is a close relationship between language control and general cognitive control, seems to only hold during production.

Composition of complex numbers: Delineating the computational role of the left anterior temporal lobe.

What is the neurobiological basis of our ability to create complex messages with language? Results from multiple methodologies have converged on a set of brain regions as relevant for this general process, but the computational details of these areas remain to be characterized. The left anterior temporal lobe (LATL) has been a consistent node within this network, with results suggesting that although it rather systematically shows increased activation for semantically complex structured stimuli, this effect does not extend to number phrases such as 'three books.' In the present work we used magnetoencephalography to investigate whether numbers in general are an invalid input to the combinatory operations housed in the LATL or whether the lack of LATL engagement for stimuli such as 'three books' is due to the quantificational nature of such phrases. As a relevant test case, we employed complex number terms such as 'twenty-three', where one number term is not a quantifier of the other but rather, the two terms form a type of complex concept. In a number naming paradigm, participants viewed rows of numbers and depending on task instruction, named them as complex number terms ('twenty-three'), numerical quantifications ('two threes'), adjectival modifications ('blue threes') or non-combinatory lists (e.g., 'two, three'). While quantificational phrases failed to engage the LATL as compared to non-combinatory controls, both complex number terms and adjectival modifications elicited a reliable activity increase in the LATL. Our results show that while the LATL does not participate in the enumeration of tokens within a set, exemplified by the quantificational phrases, it does support conceptual combination, including the composition of complex number concepts.

Brain bases of language selection: MEG evidence from Arabic-English bilingual language production.

Much of the world's population is bilingual, hence, language selection is a core component of language processing in a significant proportion of individuals. Though language selection has been investigated using artificial cues to language choice such as color, little is known about more ecologically valid cues. We examined with MEG the neurophysiological and behavioral effects of two natural cues: script and cultural context, hypothesizing the former to trigger more automatic language selection. Twenty Arabic-English bilinguals performed a number-naming task with a Match condition, where the cue and target language of response matched, and a Mismatch condition, with opposite instruction. The latter addressed the mechanisms responsible for overriding natural cue-language associations. Early visual responses patterned according to predictions from prior object recognition literature, while at 150-300 ms, the anterior cingulate cortex showed robust sensitivity to cue-type, with enhanced amplitudes to culture trials. In contrast, a mismatch effect for both cue-types was observed at 300-400 ms in the left inferior prefrontal cortex. Our findings provide the first characterization of the spatio-temporal profile of naturally cued language selection and demonstrate that natural but less automatic language-choice, elicited by cultural cues, does not engage the same mechanisms as the clearly unnatural language-choice of our mismatch tasks.