The P600 as a continuous index of integration effort.

The integration of word meaning into an unfolding utterance representation is a core operation of incremental language comprehension. There is considerable debate, however, as to which component of the ERP signal-the N400 or the P600-directly reflects integrative processes, with far reaching consequences for the temporal organization and architecture of the comprehension system. Multi-stream models maintaining the N400 as integration crucially rely on the presence of a semantically attractive plausible alternative interpretation to account for the absence of an N400 effect in response to certain semantic anomalies, as reported in previous studies. The single-stream Retrieval-Integration account posits the P600 as an index of integration, further predicting that its amplitude varies continuously with integrative effort. Here, we directly test these competing hypotheses using a context manipulation design in which a semantically attractive alternative is either available or not, and target word plausibility is varied across three levels. An initial self-paced reading study revealed graded reading times for plausibility, suggesting differential integration effort. A subsequent ERP study showed no N400 differences across conditions, and that P600 amplitude is graded for plausibility. These findings are inconsistent with the interpretation of the N400 as an index of integration, as no N400 effect emerged even in the absence of a semantically attractive alternative. By contrast, the link between plausibility, reading times, and P600 amplitude supports the view that the P600 is a continuous index of integration effort. More generally, our results support a single-stream architecture and eschew the need for multi-stream accounts.

Retrieval (N400) and integration (P600) in expectation-based comprehension.

Expectation-based theories of language processing, such as Surprisal theory, are supported by evidence of anticipation effects in both behavioural and neurophysiological measures. Online measures of language processing, however, are known to be influenced by factors such as lexical association that are distinct from-but often confounded with-expectancy. An open question therefore is whether a specific locus of expectancy related effects can be established in neural and behavioral processing correlates. We address this question in an event-related potential experiment and a self-paced reading experiment that independently cross expectancy and lexical association in a context manipulation design. We find that event-related potentials reveal that the N400 is sensitive to both expectancy and lexical association, while the P600 is modulated only by expectancy. Reading times, in turn, reveal effects of both association and expectancy in the first spillover region, followed by effects of expectancy alone in the second spillover region. These findings are consistent with the Retrieval-Integration account of language comprehension, according to which lexical retrieval (N400) is facilitated for words that are both expected and associated, whereas integration difficulty (P600) will be greater for unexpected words alone. Further, an exploratory analysis suggests that the P600 is not merely sensitive to expectancy violations, but rather, that there is a continuous relation. Taken together, these results suggest that the P600, like reading times, may reflect a meaning-centric notion of Surprisal in language comprehension.

When components collide: Spatiotemporal overlap of the N400 and P600 in language comprehension.

The problem of spatiotemporal overlap between event-related potential (ERP) components is generally acknowledged in language research. However, its implications for the interpretation of experimental results are often overlooked. In a previous experiment on the functional interpretation of the N400 and P600, it was argued that a P600 effect to implausible words was largely obscured - in one of two implausible conditions - by an overlapping N400 effect of semantic association. In the present ERP study, we show that the P600 effect of implausibility is uncovered when the critical condition is tested against a proper baseline condition which elicits a similar N400 amplitude, while it is obscured when tested against a baseline condition producing an N400 effect. Our findings reveal that component overlap can result in the apparent absence or presence of an effect in the surface signal and should therefore be carefully considered when interpreting ERP patterns. Importantly, we show that, by factoring in the effects of spatiotemporal overlap between the N400 and P600 on the surface signal, which we reveal using rERP analysis, apparent inconsistencies in previous findings are easily reconciled, enabling us to draw unambiguous conclusions about the functional interpretation of the N400 and P600 components. Overall, our results provide compelling evidence that the N400 reflects lexical retrieval processes, while the P600 indexes compositional integration of word meaning into the unfolding utterance interpretation.

Neurobehavioral Correlates of Surprisal in Language Comprehension: A Neurocomputational Model.

Expectation-based theories of language comprehension, in particular Surprisal Theory, go a long way in accounting for the behavioral correlates of word-by-word processing difficulty, such as reading times. An open question, however, is in which component(s) of the Event-Related brain Potential (ERP) signal Surprisal is reflected, and how these electrophysiological correlates relate to behavioral processing indices. Here, we address this question by instantiating an explicit neurocomputational model of incremental, word-by-word language comprehension that produces estimates of the N400 and the P600-the two most salient ERP components for language processing-as well as estimates of "comprehension-centric" Surprisal for each word in a sentence. We derive model predictions for a recent experimental design that directly investigates "world-knowledge"-induced Surprisal. By relating these predictions to both empirical electrophysiological and behavioral results, we establish a close link between Surprisal, as indexed by reading times, and the P600 component of the ERP signal. The resultant model thus offers an integrated neurobehavioral account of processing difficulty in language comprehension.

Splitting event-related potentials: Modeling latent components using regression-based waveform estimation.

Event-related potentials (ERPs) provide a multidimensional and real-time window into neurocognitive processing. The typical Waveform-based Component Structure (WCS) approach to ERPs assesses the modulation pattern of components-systematic, reoccurring voltage fluctuations reflecting specific computational operations-by looking at mean amplitude in predetermined time-windows. This WCS approach, however, often leads to inconsistent results within as well as across studies. It has been argued that at least some inconsistencies may be reconciled by considering spatiotemporal overlap between components; that is, components may overlap in both space and time, and given their additive nature, this means that the WCS may fail to accurately represent its underlying latent component structure (LCS). We employ regression-based ERP (rERP) estimation to extend traditional approaches with an additional layer of analysis, which enables the explicit modeling of the LCS underlying WCS. To demonstrate its utility, we incrementally derive an rERP analysis of a recent study on language comprehension with seemingly inconsistent WCS-derived results. Analysis of the resultant regression models allows one to derive an explanation for the WCS in terms of how relevant regression predictors combine in space and time, and crucially, how individual predictors may be mapped onto unique components in LCS, revealing how these spatiotemporally overlap in the WCS. We conclude that rERP estimation allows for investigating how scalp-recorded voltages derive from the spatiotemporal combination of experimentally manipulated factors. Moreover, when factors can be uniquely mapped onto components, rERPs may offer explanations for seemingly inconsistent ERP waveforms at the level of their underlying latent component structure.

Event-related potentials index lexical retrieval (N400) and integration (P600) during language comprehension.

The functional interpretation of two salient language-sensitive ERP components - the N400 and the P600 - remains a matter of debate. Prominent alternative accounts link the N400 to processes related to lexical retrieval, semantic integration, or both, while the P600 has been associated with syntactic reanalysis or, alternatively, to semantic integration. The often overlapping predictions of these competing accounts in extant experimental designs, however, has meant that previous findings have failed to clearly decide among them. Here, we present an experiment that directly tests the competing hypotheses using a design that clearly teases apart the retrieval versus integration view of the N400, while also dissociating a syntactic reanalysis/reprocessing account of the P600 from semantic integration. Our findings provide support for an integrated functional interpretation according to which the N400 reflects context-sensitive lexical retrieval - but not integration - processes. While the observed P600 effects were not predicted by any account, we argue that they can be reconciled with the integration view, if spatio-temporal overlap of ERP components is taken into consideration.

A new insight into sentence comprehension: The impact of word associations in sentence processing as shown by invasive EEG recording.

The effect of word association on sentence processing is still a matter of debate. Some studies observe no effect while others found a dependency on sentence congruity or an independent effect. In an attempt to separate the effects of sentence congruity and word association in the spatio-temporal domain, we jointly recorded scalp- and invasive-EEG (iEEG). The latter provides highly localized spatial (unlike scalp-EEG) and high temporal (unlike fMRI) resolutions. We recorded scalp- and iEEG in three patients with refractory epilepsy. The stimuli consisted of 280 sentences with crossed factors of sentence congruity and within sentence word-association. We mapped semantic retrieval processes involved in sentence comprehension onto the left temporal cortex and both hippocampi, and showed for the first time that certain localized regions participate in the processing of word-association in sentence context. Furthermore, simultaneous recording of scalp- and iEEG gave us a direct overview of signal change due to its propagation across the head tissues.

A Neurocomputational Model of the N400 and the P600 in Language Processing.

Ten years ago, researchers using event-related brain potentials (ERPs) to study language comprehension were puzzled by what looked like a Semantic Illusion: Semantically anomalous, but structurally well-formed sentences did not affect the N400 component-traditionally taken to reflect semantic integration-but instead produced a P600 effect, which is generally linked to syntactic processing. This finding led to a considerable amount of debate, and a number of complex processing models have been proposed as an explanation. What these models have in common is that they postulate two or more separate processing streams, in order to reconcile the Semantic Illusion and other semantically induced P600 effects with the traditional interpretations of the N400 and the P600. Recently, however, these multi-stream models have been called into question, and a simpler single-stream model has been proposed. According to this alternative model, the N400 component reflects the retrieval of word meaning from semantic memory, and the P600 component indexes the integration of this meaning into the unfolding utterance interpretation. In the present paper, we provide support for this "Retrieval-Integration (RI)" account by instantiating it as a neurocomputational model. This neurocomputational model is the first to successfully simulate the N400 and P600 amplitude in language comprehension, and simulations with this model provide a proof of concept of the single-stream RI account of semantically induced patterns of N400 and P600 modulations.

A time and place for language comprehension: mapping the N400 and the P600 to a minimal cortical network.

We propose a new functional-anatomical mapping of the N400 and the P600 to a minimal cortical network for language comprehension. Our work is an example of a recent research strategy in cognitive neuroscience, where researchers attempt to align data regarding the nature and time-course of cognitive processing (from ERPs) with data on the cortical organization underlying it (from fMRI). The success of this "alignment" approach critically depends on the functional interpretation of relevant ERP components. Models of language processing that have been proposed thus far do not agree on these interpretations, and present a variety of complicated functional architectures. We put forward a very basic functional-anatomical mapping based on the recently developed Retrieval-Integration account of language comprehension (Brouwer et al., 2012). In this mapping, the left posterior part of the Middle Temporal Gyrus (BA 21) serves as an epicenter (or hub) in a neurocognitive network for the retrieval of word meaning, the ease of which is reflected in N400 amplitude. The left Inferior Frontal Gyrus (BA 44/45/47), in turn, serves a network epicenter for the integration of this retrieved meaning with the word's preceding context, into a mental representation of what is being communicated; these semantic and pragmatic integrative processes are reflected in P600 amplitude. We propose that our mapping describes the core of the language comprehension network, a view that is parsimonious, has broad empirical coverage, and can serve as the starting point for a more focused investigation into the coupling of brain anatomy and electrophysiology.

Questions left unanswered: how the brain responds to missing information.

It sometimes happens that when someone asks a question, the addressee does not give an adequate answer, for instance by leaving out part of the required information. The person who posed the question may wonder why the information was omitted, and engage in extensive processing to find out what the partial answer actually means. The present study looks at the neural correlates of the pragmatic processes invoked by partial answers to questions. Two experiments are presented in which participants read mini-dialogues while their Event-Related brain Potentials (ERPs) are being measured. In both experiments, violating the dependency between questions and answers was found to lead to an increase in the amplitude of the P600 component. We interpret these P600-effects as reflecting the increased effort in creating a coherent representation of what is communicated. This effortful processing might include the computation of what the dialogue participant meant to communicate by withholding information. Our study is one of few investigating language processing in conversation, be it that our participants were 'eavesdroppers' instead of real interactants. Our results contribute to the as of yet small range of pragmatic phenomena that modulate the processes underlying the P600 component, and suggest that people immediately attempt to regain cohesion if a question-answer dependency is violated in an ongoing conversation.

Getting real about semantic illusions: rethinking the functional role of the P600 in language comprehension.

In traditional theories of language comprehension, syntactic and semantic processing are inextricably linked. This assumption has been challenged by the 'semantic illusion effect' found in studies using event related brain potentials. Semantically anomalous sentences did not produce the expected increase in N400 amplitude but rather one in P600 amplitude. To explain these findings, complex models have been devised in which an independent semantic processing stream can arrive at a sentence interpretation that may differ from the interpretation prescribed by the syntactic structure of the sentence. We review five such multi-stream models and argue that they do not account for the full range of relevant results because they assume that the amplitude of the N400 indexes some form of semantic integration. Based on recent evidence we argue that N400 amplitude might reflect the retrieval of lexical information from memory. On this view, the absence of an N400-effect in semantic illusion sentences can be explained in terms of priming. Furthermore, we suggest that semantic integration, which has previously been linked to the N400 component, might be reflected in the P600 instead. When combined, these functional interpretations result in a single-stream account of language processing that can explain all of the Semantic Illusion data.