Acoustic and language-specific sources for phonemic abstraction from speech.

Spoken language comprehension requires abstraction of linguistic information from speech, but the interaction between auditory and linguistic processing of speech remains poorly understood. Here, we investigate the nature of this abstraction using neural responses recorded intracranially while participants listened to conversational English speech. Capitalizing on multiple, language-specific patterns where phonological and acoustic information diverge, we demonstrate the causal efficacy of the phoneme as a unit of analysis and dissociate the unique contributions of phonemic and spectrographic information to neural responses. Quantitive higher-order response models also reveal that unique contributions of phonological information are carried in the covariance structure of the stimulus-response relationship. This suggests that linguistic abstraction is shaped by neurobiological mechanisms that involve integration across multiple spectro-temporal features and prior phonological information. These results link speech acoustics to phonology and morphosyntax, substantiating predictions about abstractness in linguistic theory and providing evidence for the acoustic features that support that abstraction.

The nested hierarchy of overt, mouthed, and imagined speech activity evident in intracranial recordings.

Recent studies have demonstrated that it is possible to decode and synthesize various aspects of acoustic speech directly from intracranial measurements of electrophysiological brain activity. In order to continue progressing toward the development of a practical speech neuroprosthesis for the individuals with speech impairments, better understanding and modeling of imagined speech processes are required. The present study uses intracranial brain recordings from participants that performed a speaking task with trials consisting of overt, mouthed, and imagined speech modes, representing various degrees of decreasing behavioral output. Speech activity detection models are constructed using spatial, spectral, and temporal brain activity features, and the features and model performances are characterized and compared across the three degrees of behavioral output. The results indicate the existence of a hierarchy in which the relevant channels for the lower behavioral output modes form nested subsets of the relevant channels from the higher behavioral output modes. This provides important insights for the elusive goal of developing more effective imagined speech decoding models with respect to the better-established overt speech decoding counterparts.

Taxonomic and thematic semantic relationships in picture naming as revealed by Laplacian-transformed event-related potentials.

Semantically related concepts co-activate when we speak. Prior research reported both behavioral interference and facilitation due to co-activation during picture naming. Different word relationships may account for some of this discrepancy. Taxonomically related words (e.g., WOLF-DOG) have been associated with semantic interference; thematically related words (e.g., BONE-DOG) have been associated with facilitation. Although these different semantic relationships have been associated with opposite behavioral outcomes, electrophysiological studies have found inconsistent effects on event-related potentials. We conducted a picture-word interference electroencephalography experiment to examine word retrieval dynamics in these different semantic relationships. Importantly, we used traditional monopolar analysis as well as Laplacian transformation allowing us to examine spatially deblurred event-related components. Both analyses revealed greater negativity (150-250 ms) for unrelated than related taxonomic pairs, though more restricted in space for thematic pairs. Critically, Laplacian analyses revealed a larger negative-going component in the 300 to 500 ms time window in taxonomically related versus unrelated pairs which were restricted to a left frontal recording site. In parallel, an opposite effect was found in the same time window but localized to a left parietal site. Finding these opposite effects in the same time window was feasible thanks to the use of the Laplacian transformation and suggests that frontal control processes are concurrently engaged with cascading effects of the spread of activation through semantically related representations.

Gender bias in academia: A lifetime problem that needs solutions.

Despite increased awareness of the lack of gender equity in academia and a growing number of initiatives to address issues of diversity, change is slow, and inequalities remain. A major source of inequity is gender bias, which has a substantial negative impact on the careers, work-life balance, and mental health of underrepresented groups in science. Here, we argue that gender bias is not a single problem but manifests as a collection of distinct issues that impact researchers' lives. We disentangle these facets and propose concrete solutions that can be adopted by individuals, academic institutions, and society.

Neural Underpinnings of Proactive Interference in Working Memory: Evidence From Patients With Unilateral Lesions.

Proactive interference in working memory refers to the fact that memory of past experiences can interfere with the ability to hold new information in working memory. The left inferior frontal gyrus (LIFG) has been proposed to play an important role in resolving proactive interference in working memory. However, the role of white matter pathways and other cortical regions has been less investigated. Here we investigated proactive interference in working memory using the Recent Probes Test (RPT) in 15 stroke patients with unilateral chronic lesions in left (n = 7) or right (n = 2) prefrontal cortex (PFC), or left temporal cortex (n = 6). We examined the impact of lesions in both gray and white matter regions on the size of the proactive interference effect. We found that patients with left PFC lesions performed worse overall, but the proactive interference effect in this patient group was comparable to that of patients with right PFC lesions, temporal lobe lesions, and controls. Interestingly, the size of the interference effect was significantly correlated with the degree of damage in the extreme/external capsule and marginally correlated with the degree of damage in the inferior frontal occipital fasciculus (IFOF). These findings suggests that ventral white matter pathways connecting the LIFG to left posterior regions play a role in resolving proactive interference in working memory. This effect was particularly evident in one patient with a very large interference effect (>3 SDs above controls) who had mostly spared LIFG, but virtually absent ventral white matter pathways (i.e., passing through the extreme/external capsules and IFOF). This case study further supports the idea that the role of the LIFG in resolving interference in working memory is dependent on connectivity with posterior regions via ventral white matter pathways.

Characterizing multi-word speech production using event-related potentials.

Event-related potentials (ERPs) derived from electroencephalography (EEG) have proven useful for understanding linguistic processes during language perception and production. Words are commonly produced in sequences, yet most ERP studies have used single-word experimental designs. Single-word designs reduce potential ERP overlap in word sequence production. However, word sequence production engages brain mechanisms in different ways than single word production. In particular, speech monitoring and planning mechanisms are more engaged than for single words since several words must be produced in a short period of time. This study evaluates the feasibility of recording ERP components in the context of word sequence production, and whether separate components could be isolated for each word. Scalp EEG data were acquired, while participants recited word sequences from memory at a regular pace, using a tongue-twister paradigm. The results revealed fronto-central error-related negativity, previously associated with speech monitoring, which could be distinguished for each word. Its peak amplitude was sensitive to Cycle and Phonological Similarity. However, an effect of sequential production was also observable on baseline measures, indicating baseline shifts throughout the word sequence due to concurrent sustained medial-frontal EEG activity. We also report a late left anterior negativity (LLAN), associated with verbal response planning and execution, onsetting around 100 ms before the first word in each cycle and sustained throughout the rest of the cycle. This work underlines the importance of considering the contribution of transient and sustained EEG activity on ERPs, and provides evidence that ERPs can be used to study sequential word production.

Investigating the Link Between Linguistic and Non-Linguistic Cognitive Control in Bilinguals Using Laplacian-Transformed Event Related Potentials.

Bilinguals' need to suppress the activation of their other language while speaking has been proposed to result in enhanced cognitive control abilities outside of language. Several studies therefore suggest shared cognitive control processes across linguistic and non-linguistic tasks. Here we investigate this potential overlap using scalp electroencephalographic recordings and the Laplacian transformation, providing an estimation of the current source density and enabling the separation of EEG components in space. Fourteen Spanish-English bilinguals performed a picture-word matching task contrasting incongruent trials using cross-linguistic false cognates (e.g., a picture - foot, overlaid with distractor text: the English word PIE, i.e., the false cognate for the Spanish pie meaning "foot") with congruent trials (matching English picture names and words, i.e., a picture - foot, with overlaid text: the English word FOOT), and an unrelated control condition. In addition, participants performed an arrow-version of the Eriksen flanker task. Worse behavioral performance was observed in incongruent compared to congruent trials in both tasks. In the non-linguistic task, we replicated the previously observed congruency effect on a medial-frontal event-related potential (ERP) peaking around 50 ms before electromyography (EMG) onset. A similar ERP was present in the linguistic task, was sensitive to congruency, and peaked earlier, around 150 ms before EMG onset. In addition, another component was found in the linguistic task at a left lateralized anterior frontal site peaking around 200 ms before EMG onset, but was absent in the non-linguistic task. Our results suggest a partial overlap between linguistic and non-linguistic cognitive control processes and that linguistic conflict resolution may engage additional left anterior frontal control processes.

Pre-output Language Monitoring in Sign Production.

A domain-general monitoring mechanism is proposed to be involved in overt speech monitoring. This mechanism is reflected in a medial frontal component, the error negativity (Ne), present in both errors and correct trials (Ne-like wave) but larger in errors than correct trials. In overt speech production, this negativity starts to rise before speech onset and is therefore associated with inner speech monitoring. Here, we investigate whether the same monitoring mechanism is involved in sign language production. Twenty deaf signers (American Sign Language [ASL] dominant) and 16 hearing signers (English dominant) participated in a picture-word interference paradigm in ASL. As in previous studies, ASL naming latencies were measured using the keyboard release time. EEG results revealed a medial frontal negativity peaking within 15 msec after keyboard release in the deaf signers. This negativity was larger in errors than correct trials, as previously observed in spoken language production. No clear negativity was present in the hearing signers. In addition, the slope of the Ne was correlated with ASL proficiency (measured by the ASL Sentence Repetition Task) across signers. Our results indicate that a similar medial frontal mechanism is engaged in preoutput language monitoring in sign and spoken language production. These results suggest that the monitoring mechanism reflected by the Ne/Ne-like wave is independent of output modality (i.e., spoken or signed) and likely monitors prearticulatory representations of language. Differences between groups may be linked to several factors including differences in language proficiency or more variable lexical access to motor programming latencies for hearing than deaf signers.

Lesions to the left lateral prefrontal cortex impair decision threshold adjustment for lexical selection.

Patients with lesions in the left prefrontal cortex (PFC) have been shown to be impaired in lexical selection, especially when interference between semantically related alternatives is increased. To more deeply investigate which computational mechanisms may be impaired following left PFC damage due to stroke, a psychometric modelling approach is employed in which we assess the cognitive parameters of the patients from an evidence accumulation (sequential information sampling) modelling of their response data. We also compare the results to healthy speakers. Analysis of the cognitive parameters indicates an impairment of the PFC patients to appropriately adjust their decision threshold, in order to handle the increased item difficulty that is introduced by semantic interference. Also, the modelling contributes to other topics in psycholinguistic theory, in which specific effects are observed on the cognitive parameters according to item familiarization, and the opposing effects of priming (lower threshold) and semantic interference (lower drift) which are found to depend on repetition. These results are developed for the blocked-cyclic picture naming paradigm, in which pictures are presented within semantically homogeneous (HOM) or heterogeneous (HET) blocks, and are repeated several times per block. Overall, the results are in agreement with a role of the left PFC in adjusting the decision threshold for lexical selection in language production.

Spatiotemporal dynamics of word retrieval in speech production revealed by cortical high-frequency band activity.

Word retrieval is core to language production and relies on complementary processes: the rapid activation of lexical and conceptual representations and word selection, which chooses the correct word among semantically related competitors. Lexical and conceptual activation is measured by semantic priming. In contrast, word selection is indexed by semantic interference and is hampered in semantically homogeneous (HOM) contexts. We examined the spatiotemporal dynamics of these complementary processes in a picture naming task with blocks of semantically heterogeneous (HET) or HOM stimuli. We used electrocorticography data obtained from frontal and temporal cortices, permitting detailed spatiotemporal analysis of word retrieval processes. A semantic interference effect was observed with naming latencies longer in HOM versus HET blocks. Cortical response strength as indexed by high-frequency band (HFB) activity (70-150 Hz) amplitude revealed effects linked to lexical-semantic activation and word selection observed in widespread regions of the cortical mantle. Depending on the subsecond timing and cortical region, HFB indexed semantic interference (i.e., more activity in HOM than HET blocks) or semantic priming effects (i.e., more activity in HET than HOM blocks). These effects overlapped in time and space in the left posterior inferior temporal gyrus and the left prefrontal cortex. The data do not support a modular view of word retrieval in speech production but rather support substantial overlap of lexical-semantic activation and word selection mechanisms in the brain.

The time course of visual influences in letter recognition.

This study builds on a specific characteristic of letters of the Roman alphabet-namely, that each letter name is associated with two visual formats, corresponding to their uppercase and lowercase versions. Participants had to read aloud the names of single letters, and event-related potentials (ERPs) for six pairs of visually dissimilar upper- and lowercase letters were recorded. Assuming that the end product of processing is the same for upper- and lowercase letters sharing the same vocal response, ERPs were compared backward, starting from the onset of articulatory responses, and the first significant divergence was observed 120 ms before response onset. Given that naming responses were produced at around 414 ms, on average, these results suggest that letter processing is influenced by visual information until 294 ms after stimulus onset. This therefore provides new empirical evidence regarding the time course and interactive nature of visual letter perception processes.

Choosing words: left hemisphere, right hemisphere, or both? Perspective on the lateralization of word retrieval.

Language is considered to be one of the most lateralized human brain functions. Left hemisphere dominance for language has been consistently confirmed in clinical and experimental settings and constitutes one of the main axioms of neurology and neuroscience. However, functional neuroimaging studies are finding that the right hemisphere also plays a role in diverse language functions. Critically, the right hemisphere may also compensate for the loss or degradation of language functions following extensive stroke-induced damage to the left hemisphere. Here, we review studies that focus on our ability to choose words as we speak. Although fluidly performed in individuals with intact language, this process is routinely compromised in aphasic patients. We suggest that parceling word retrieval into its subprocesses-lexical activation and lexical selection-and examining which of these can be compensated for after left hemisphere stroke can advance the understanding of the lateralization of word retrieval in speech production. In particular, the domain-general nature of the brain regions associated with each process may be a helpful indicator of the right hemisphere's propensity for compensation.

Early and Late Electrophysiological Effects of Distractor Frequency in Picture Naming: Reconciling Input and Output Accounts.

The "distractor-frequency effect" refers to the finding that high-frequency (HF) distractor words slow picture naming less than low-frequency distractors in the picture-word interference paradigm. Rival input and output accounts of this effect have been proposed. The former attributes the effect to attentional selection mechanisms operating during distractor recognition, whereas the latter attributes it to monitoring/decision mechanisms operating on distractor and target responses in an articulatory buffer. Using high-density (128-channel) EEG, we tested hypotheses from these rival accounts. In addition to conducting stimulus- and response-locked whole-brain corrected analyses, we investigated the correct-related negativity, an ERP observed on correct trials at fronto-central electrodes proposed to reflect the involvement of domain general monitoring. The whole-brain ERP analysis revealed a significant effect of distractor frequency at inferior right frontal and temporal sites between 100 and 300-msec post-stimulus onset, during which lexical access is thought to occur. Response-locked, region of interest (ROI) analyses of fronto-central electrodes revealed a correct-related negativity starting 121 msec before and peaking 125 msec after vocal onset on the grand averages. Slope analysis of this component revealed a significant difference between HF and low-frequency distractor words, with the former associated with a steeper slope on the time window spanning from 100 msec before to 100 msec after vocal onset. The finding of ERP effects in time windows and components corresponding to both lexical processing and monitoring suggests the distractor frequency effect is most likely associated with more than one physiological mechanism.

Lesions to Lateral Prefrontal Cortex Impair Lexical Interference Control in Word Production.

Speaking is an action that requires control, for example, to prevent interference from distracting or competing information present in the speaker's environment. Control over task performance is thought to depend on the lateral prefrontal cortex (PFC). However, the neuroimaging literature does not show a consistent relation between left PFC and interference control in word production. Here, we examined the role of left PFC in interference control in word production by testing six patients with lesions to left PFC (centered around the ventrolateral PFC) on a control-demanding task. Patients and age-matched controls named pictures presented along with distractor words, inducing within-trial interference effects. We varied the degree of competing information from distractors to increase the need for interference control. Distractors were semantically related, phonologically related, unrelated to the picture name, or neutral (XXX). Both groups showed lexical interference (slower responses with unrelated than neutral distractors), reflecting naming difficulty in the presence of competing linguistic information. Relative to controls, all six left PFC patients had larger lexical interference effects. By contrast, patients did not show a consistent semantic interference effect (reflecting difficulty in selecting amongst semantic competitors) whereas the controls did. This suggests different control mechanisms may be engaged in semantic compared to lexical interference resolution in this paradigm. Finally, phonological facilitation (faster responses with phonological than unrelated distractors) was larger in patients than in controls. These findings suggest that the lateral PFC is a necessary structure in providing control over lexical interference in word production, possibly through an early attentional blocking mechanism. By contrast, the left PFC does not seem critical in semantic interference resolution in the picture-word interference paradigm.

Corrigendum to "Why does picture naming take longer than word naming? The contribution of articulatory processes".

In a previous article, (Riès, Legou, Burle, Alario, & Malfait, 2012), we reported that articulatory processes contribute to the well-established finding that response latencies are longer for picture naming than for word reading. We based this conclusion on the observation that picture naming, as compared with word reading, lengthened not only the interval between stimulus onset and the initiation of lip muscle activation (premotor time), but also the interval between lip muscle activation and vocal response onset (motor time). However, on the basis of our subsequent work in this area, we believe that our original definition of premotor time (and, consequently, of motor time) was suboptimal. On a sizable number of trials, this led to the detection of lip muscle activation (as inferred from surface EMG) that was apparently unrelated to the articulation of the vocal response. Therefore, we believe it is preferable to operationalize premotor time as the interval between stimulus onset and the muscle activation that occurred closest in time to vocal response onset. After reestimating premotor times according to this new definition, we no longer found an effect of our task contrast on the motor time interval. The present article explains the caveats regarding our previous analysis.

A comparison of two procedures for verbal response time fractionation.

To describe the mental architecture between stimulus and response, cognitive models often divide the stimulus-response (SR) interval into stages or modules. Predictions derived from such models are typically tested by focusing on the moment of response emission, through the analysis of response time (RT) distributions. To go beyond the single response event, we recently proposed a method to fractionate verbal RTs into two physiologically defined intervals that are assumed to reflect different processing stages. The analysis of the durations of these intervals can be used to study the interaction between cognitive and motor processing during speech production. Our method is inspired by studies on decision making that used manual responses, in which RTs were fractionated into a premotor time (PMT), assumed to reflect cognitive processing, and a motor time (MT), assumed to reflect motor processing. In these studies, surface EMG activity was recorded from participants' response fingers. EMG onsets, reflecting the initiation of a motor response, were used as the point of fractionation. We adapted this method to speech-production research by measuring verbal responses in combination with EMG activity from facial muscles involved in articulation. However, in contrast to button-press tasks, the complex task of producing speech often resulted in multiple EMG bursts within the SR interval. This observation forced us to decide how to operationalize the point of fractionation: as the first EMG burst after stimulus onset (the stimulus-locked approach), or as the EMG burst that is coupled to the vocal response (the response-locked approach). The point of fractionation has direct consequences on how much of the overall task effect is captured by either interval. Therefore, the purpose of the current paper was to compare both onset-detection procedures in order to make an informed decision about which of the two is preferable. We concluded in favor or the response-locked approach.

How familiarization and repetition modulate the picture naming network.

A common strategy to reveal the components of the speech production network is to use psycholinguistic manipulations previously tested in behavioral protocols. This often disregards how implementation aspects that are nonessential for interpreting behavior may affect the neural response. We compared the electrophysiological (EEG) signature of two popular picture naming protocols involving either unfamiliar pictures without repetitions or repeated familiar pictures. We observed significant semantic interference effects in behavior but not in the EEG, contrary to some previous findings. Remarkably, the two protocols elicited clearly distinct EEG responses. These were not due to naming latency differences nor did they reflect a homogeneous modulation of amplitude over the trial time-window. The effect of protocol is attributed to the familiarization induced by the first encounter with the materials. Picture naming processes can be substantially modulated by specific protocol requirements controlled by familiarity and, to a much lesser degree, the repetition of materials.

Role of the lateral prefrontal cortex in speech monitoring.

The role of lateral prefrontal cortex (LPFC) in speech monitoring has not been delineated. Recent work suggests that medial frontal cortex (MFC) is involved in overt speech monitoring initiated before auditory feedback. This mechanism is reflected in an event-related potential (ERP), the error negativity (Ne), peaking within 100 ms after vocal-onset. Critically, in healthy individuals the Ne is sensitive to the accuracy of the response; it is larger for error than correct trials. By contrast, patients with LPFC damage are impaired in non-verbal monitoring tasks showing no amplitude difference between the Ne measured in correct vs. error trials. Interactions between the LPFC and the MFC are assumed to play a necessary role for normal action monitoring. We investigated whether the LPFC was involved in speech monitoring to the same extent as in non-linguistic actions by comparing performance and EEG activity in patients with LPFC damage and in aged-matched controls performing linguistic (Picture Naming) and non-linguistic (Simon) tasks. Controls did not produce enough errors to allow the comparison of the Ne or other ERP in error vs. correct trials. PFC patients had worse performance than controls in both tasks, but their Ne was larger for error than correct trials only in Naming. This task-dependent pattern can be explained by LPFC-dependent working-memory requirements present in non-linguistic tasks used to study action monitoring but absent in picture naming. This suggests that LPFC may not be necessary for speech monitoring as assessed by simple picture naming. In addition, bilateral temporal cortex activity starting before and peaking around vocal-onset was observed in LPFC and control groups in both tasks but was larger for error than correct trials only in Naming, suggesting the temporal cortex is associated with on-line monitoring of speech specifically when access to lexical representations is necessary.

Response-locked brain dynamics of word production.

The cortical regions involved in the different stages of speech production are relatively well-established, but their spatio-temporal dynamics remain poorly understood. In particular, the available studies have characterized neural events with respect to the onset of the stimulus triggering a verbal response. The core aspect of language production, however, is not perception but action. In this context, the most relevant question may not be how long after a stimulus brain events happen, but rather how long before the production act do they occur. We investigated speech production-related brain activity time-locked to vocal onset, in addition to the common stimulus-locked approach. We report the detailed temporal interplay between medial and left frontal activities occurring shortly before vocal onset. We interpret those as reflections of, respectively, word selection and word production processes. This medial-lateral organization is in line with that described in non-linguistic action control, suggesting that similar processes are at play in word production and non-linguistic action production. This novel view of the brain dynamics underlying word production provides a useful background for future investigations of the spatio-temporal brain dynamics that lead to the production of verbal responses.