How statistical correlations influence discourse-level processing: Clause type as a cue for discourse relations.

Linguistic phenomena (e.g., words and syntactic structure) co-occur with a wide variety of meanings. These systematic correlations can help readers to interpret a text and create predictions about upcoming material. However, to what extent these correlations influence discourse processing is still unknown. We address this question by examining whether clause type serves as a cue for discourse relations. We found that the co-occurrence of gerund-free adjuncts and specific discourse relations found in natural language is also reflected in readers' offline expectations for discourse relations. However, we also found that clause structure did not facilitate the online processing of these discourse relations, nor that readers have a preference for these relations in a paraphrase selection task. The present research extends previous research on discourse relation processing, which mostly focused on lexical cues, by examining the role of non-semantic cues. We show that readers are aware of correlations between clause structure and discourse relations in natural language, but that, unlike what has been found for lexical cues, this information does not seem to influence online processing and discourse interpretation. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Perspective Taking Reflects Beliefs About Partner Sophistication: Modern Computer Partners Versus Basic Computer and Human Partners.

We investigate partner effects on spatial perspective taking behavior in listeners, comparing behavior with a human versus a computer partner (Experiments 1 and 2), and with computer partners of different perceived capabilities (Experiment 3). Participants responded to spoken instructions from their partner which could be interpreted egocentrically (from their own perspective) or othercentrically (from their partner's perspective). In contrast to earlier work, we found that participants were more egocentric with a computer than a human partner. Participants were also more egocentric with a computer partner that appeared more modern and capable, compared to one that appeared outdated and limited in ability. Our results show that perspective taking behavior is sensitive to information about one's partner; in particular, listeners consider their partner's potential ability to collaborate, adjusting their egocentric tendencies accordingly. Moreover, we highlight what appears to be a shift in listeners' expectations regarding computers' collaborative capabilities, leading to greater willingness to push the burden of perspective taking onto a computer partner.

Individual Differences in Spatial Orientation Modulate Perspective Taking in Listeners.

Previous research suggests that individuals exhibit consistent tendencies towards taking their own (an egocentric) or their partner's (an othercentric) spatial perspective. In addition, several factors such as spatial orientation ability, inhibitory control, and social preferences, have been found to mediate these perspective taking tendencies. However, these factors have not been studied together in the context of a single task. The present study explores these individual differences together in spatial perspective taking, using a task of simulated interaction in which listeners can choose to interpret an ambiguous spatial utterance egocentrically or othercentrically. We use a data-driven approach of latent profile analysis to classify participants into subgroups based on their spatial perspective taking tendencies. Our results show that stable subgroups of participants can be identified who differ in their perspective taking tendencies. This behaviour also correlates with a measure of listeners' spatial orientation ability, but not their inhibitory control or social preferences. Our results can be interpreted within a framework that views spatial perspective taking as an embodied cognitive process of a mental reorientation of the self relative to the environment, providing insight on the nature of the mechanisms underlying this operation.

High Performance on a Pragmatic Task May Not Be the Result of Successful Reasoning: On the Importance of Eliciting Participants' Reasoning Strategies.

Formal probabilistic models, such as the Rational Speech Act model, are widely used for formalizing the reasoning involved in various pragmatic phenomena, and when a model achieves good fit to experimental data, that is interpreted as evidence that the model successfully captures some of the underlying processes. Yet how can we be sure that participants' performance on the task is the result of successful reasoning and not of some feature of experimental setup? In this study, we carefully manipulate the properties of the stimuli that have been used in several pragmatics studies and elicit participants' reasoning strategies. We show that certain biases in experimental design inflate participants' performance on the task. We then repeat the experiment with a new version of stimuli which is less susceptible to the identified biases, obtaining a somewhat smaller effect size and more reliable estimates of individual-level performance.

A systematic evaluation of factors affecting referring expression choice in passage completion tasks.

There is a long-standing controversy around the question of whether referent predictability affects pronominalization: while there are good theoretical reasons for this prediction (e.g., Arnold, 2008), the experimental evidence has been rather mixed. We here report on three highly powered studies that manipulate a range of factors that have differed between previous studies, in order to determine more exactly under which conditions a predictability effect on pronominalization can be found. We use a constrained as well as a free reference task, and manipulate verb type, antecedent ambiguity, length of NP and whether the stimuli are presented within a story context or not. Our results find the story context to be the single important factor that allows to elicit an effect of predictability on pronoun choice, in line with (Rosa and Arnold, 2017; Weatherford and Arnold, 2021). We also propose a parametrization for a rational speech act model, that reconciles the findings between many of the experiments in the literature.

Informationally redundant utterances elicit pragmatic inferences.

Most theories of pragmatics and language processing predict that speakers avoid excessive informational redundancy. Informationally redundant utterances are, however, quite common in natural dialogue. From a comprehension standpoint, it remains unclear how comprehenders interpret these utterances, and whether they make attempts to reconcile the 'dips' in informational utility with expectations of 'appropriate' or 'rational' speaker informativity. We show that informationally redundant (overinformative) utterances can trigger pragmatic inferences that increase utterance utility in line with comprehender expectations. In a series of three studies, we look at utterances which refer to stereotyped event sequences describing common activities (scripts). When comprehenders encounter utterances describing events that can be easily inferred from prior context, they interpret them as signifying that the event conveys new, unstated information (i.e. an event otherwise assumed to be habitual, such as paying the cashier when shopping, is reinterpreted as non-habitual). We call these inferences atypicality inferences. Further, we show that the degree to which these atypicality inferences are triggered depends on the framing of the utterance. In the absence of an exclamation mark or a discourse marker indicating the speaker's specific intent to communicate the given information, such inferences are far less likely to arise. Overall, the results demonstrate that excessive conceptual redundancy leads to comprehenders revising the conversational common ground, in an effort to accommodate unexpected dips in informational utility.

Rational speech comprehension: Interaction between predictability, acoustic signal, and noise.

Introduction: During speech comprehension, multiple sources of information are available to listeners, which are combined to guide the recognition process. Models of speech comprehension posit that when the acoustic speech signal is obscured, listeners rely more on information from other sources. However, these models take into account only word frequency information and local contexts (surrounding syllables), but not sentence-level information. To date, empirical studies investigating predictability effects in noise did not carefully control the tested speech sounds, while the literature investigating the effect of background noise on the recognition of speech sounds does not manipulate sentence predictability. Additionally, studies on the effect of background noise show conflicting results regarding which noise type affects speech comprehension most. We address this in the present experiment. Methods: We investigate how listeners combine information from different sources when listening to sentences embedded in background noise. We manipulate top-down predictability, type of noise, and characteristics of the acoustic signal, thus creating conditions which differ in the extent to which a specific speech sound is masked in a way that is grounded in prior work on the confusability of speech sounds in noise. Participants complete an online word recognition experiment. Results and discussion: The results show that participants rely more on the provided sentence context when the acoustic signal is harder to process. This is the case even when interactions of the background noise and speech sounds lead to small differences in intelligibility. Listeners probabilistically combine top-down predictions based on context with noisy bottom-up information from the acoustic signal, leading to a trade-off between the different types of information that is dependent on the combination of a specific type of background noise and speech sound.

Mishearing as a Side Effect of Rational Language Comprehension in Noise.

Language comprehension in noise can sometimes lead to mishearing, due to the noise disrupting the speech signal. Some of the difficulties in dealing with the noisy signal can be alleviated by drawing on the context - indeed, top-down predictability has shown to facilitate speech comprehension in noise. Previous studies have furthermore shown that strong reliance on the top-down predictions can lead to increased rates of mishearing, especially in older adults, which are attributed to general deficits in cognitive control in older adults. We here propose that the observed mishearing may be a simple consequence of rational language processing in noise. It should not be related to failure on the side of the older comprehenders, but instead would be predicted by rational processing accounts. To test this hypothesis, we extend earlier studies by running an online listening experiment with younger and older adults, carefully controlling the target and direct competitor in our stimuli. We show that mishearing is directly related to the perceptibility of the signal. We furthermore add an analysis of wrong responses, which shows that results are at odds with the idea that participants overly strongly rely on context in this task, as most false answers are indeed close to the speech signal, and not to the semantics of the context.

Semantic Predictability Facilitates Comprehension of Degraded Speech in a Graded Manner.

Previous studies have shown that at moderate levels of spectral degradation, semantic predictability facilitates language comprehension. It is argued that when speech is degraded, listeners have narrowed expectations about the sentence endings; i.e., semantic prediction may be limited to only most highly predictable sentence completions. The main objectives of this study were to (i) examine whether listeners form narrowed expectations or whether they form predictions across a wide range of probable sentence endings, (ii) assess whether the facilitatory effect of semantic predictability is modulated by perceptual adaptation to degraded speech, and (iii) use and establish a sensitive metric for the measurement of language comprehension. For this, we created 360 German Subject-Verb-Object sentences that varied in semantic predictability of a sentence-final target word in a graded manner (high, medium, and low) and levels of spectral degradation (1, 4, 6, and 8 channels noise-vocoding). These sentences were presented auditorily to two groups: One group (n =48) performed a listening task in an unpredictable channel context in which the degraded speech levels were randomized, while the other group (n =50) performed the task in a predictable channel context in which the degraded speech levels were blocked. The results showed that at 4 channels noise-vocoding, response accuracy was higher in high-predictability sentences than in the medium-predictability sentences, which in turn was higher than in the low-predictability sentences. This suggests that, in contrast to the narrowed expectations view, comprehension of moderately degraded speech, ranging from low- to high- including medium-predictability sentences, is facilitated in a graded manner; listeners probabilistically preactivate upcoming words from a wide range of semantic space, not limiting only to highly probable sentence endings. Additionally, in both channel contexts, we did not observe learning effects; i.e., response accuracy did not increase over the course of experiment, and response accuracy was higher in the predictable than in the unpredictable channel context. We speculate from these observations that when there is no trial-by-trial variation of the levels of speech degradation, listeners adapt to speech quality at a long timescale; however, when there is a trial-by-trial variation of the high-level semantic feature (e.g., sentence predictability), listeners do not adapt to low-level perceptual property (e.g., speech quality) at a short timescale.

Limits to the Rational Production of Discourse Connectives.

Rational accounts of language use such as the uniform information density hypothesis, which asserts that speakers distribute information uniformly across their utterances, and the rational speech act (RSA) model, which suggests that speakers optimize the formulation of their message by reasoning about what the comprehender would understand, have been hypothesized to account for a wide range of language use phenomena. We here specifically focus on the production of discourse connectives. While there is some prior work indicating that discourse connective production may be governed by RSA, that work uses a strongly gamified experimental setting. In this study, we aim to explore whether speakers reason about the interpretation of their conversational partner also in more realistic settings. We thereby systematically vary the task setup to tease apart effects of task instructions and effects of the speaker explicitly seeing the interpretation alternatives for the listener. Our results show that the RSA-predicted effect of connective choice based on reasoning about the listener is only found in the original setting where explicit interpretation alternatives of the listener are available for the speaker. The effect disappears when the speaker has to reason about listener interpretations. We furthermore find that rational effects are amplified by the gamified task setting, indicating that meta-reasoning about the specific task may play an important role and potentially limit the generalizability of the found effects to more naturalistic every-day language use.

Effects of Aging and Dual-Task Demands on the Comprehension of Less Expected Sentence Continuations: Evidence From Pupillometry.

Prior studies on language processing in aging have shown that older adults experience integration difficulties for contextually unpredictable target words (as indicated by low cloze probabilities in prior ratings), and that such comprehension difficulties are more likely to occur under more demanding processing conditions (e.g., dual-task situations). However, these effects have primarily been demonstrated for conditions when cloze probability of the linguistic stimuli was very low. The question we asked here was do dual-task demands also impair comprehension when target words provide a good, but not perfect, match with prior context? We used a dual-task design, consisting of a sentence comprehension and secondary motor tracking task. Critical target words were those which were not perfectly predictable based on context (words with a cloze probability of 0.7), as opposed to words that were near perfectly predictable based on context (cloze probabilities of 0.99). As a measure to index online processing difficulty for less expected target words, we took into account participants' pupil size. Separate mixed effects models were fit for language comprehension, motor tracking, and pupil size, showing the following: (1) dual-task demands led to age-related comprehension difficulties when target words were less expected (as opposed to very highly expected), (2) integration difficulty in older adults was related to cognitive overload as less expected sentence continuations progressed over time, resulting in behavioral trade-offs between language comprehension and motor tracking, and (3) lower levels of working memory were predictive of whether or not older adults experienced cognitive overload when processing less expected words. In sum, more demanding processing conditions lead to comprehension impairments when words are highly unpredictable based on context, as many prior studies showed. Comprehension impairments among older individuals also occur for conditions when words provide a good, but not perfect, match with prior context. Higher working memory capacity can alleviate such impairments in older adults, thereby suggesting that only high-WM older adults have sufficient cognitive resources to pre-activate words that complete a sentence context plausibly, but not perfectly.

The Index of Cognitive Activity as a Measure of Cognitive Processing Load in Dual Task Settings.

Increases in pupil size have long been used as an indicator of cognitive load. Recently, the Index of Cognitive Activity (ICA), a novel pupillometric measure has received increased attention. The ICA measures the frequency of rapid pupil dilations, and is an interesting complementary measure to overall pupil size because it disentangles the pupil response to cognitive activity from effects of light input. As such, it has been evaluated as a useful measure of processing load in dual task settings coordinating language comprehension and driving. However, the cognitive underpinnings of pupillometry, and any differences between rapid small dilations as measured by the ICA and overall effects on pupil size are still poorly understood. Earlier work has observed that the ICA and overall pupil size may not always behave in the same way, reporting an increase in overall pupil size but decrease in ICA in a dual task setting. To further investigate this, we systematically tested two new dual-task combinations, combining both language comprehension and simulated driving with a memory task. Our findings confirm that more difficult linguistic processing is reflected in a larger ICA. More importantly, however, the dual task settings did not result in an increase in the ICA as compared to the single task, and, consistent with earlier findings, showed a significant decrease with a more difficult secondary task. This contrasts with our findings for pupil size, which showed an increase with greater secondary task difficulty in both tasks. Our results are compatible with the idea that although both pupillometry measures are indicators of cognitive load, they reflect different cognitive and neuronal processes in dual task situations.

Surprisal modulates dual-task performance in older adults: Pupillometry shows age-related trade-offs in task performance and time-course of language processing.

Even though older adults are known to have difficulty at language processing when a secondary task has to be performed simultaneously, few studies have addressed how older adults process language in dual-task demands when linguistic load is systematically varied. Here, we manipulated surprisal, an information theoretic measure that quantifies the amount of new information conveyed by a word, to investigate how linguistic load affects younger and older adults during early and late stages of sentence processing under conditions when attention is split between two tasks. In high-surprisal sentences, target words were implausible and mismatched with semantic expectancies based on context, thereby causing integration difficulty. Participants performed semantic meaningfulness judgments on sentences that were presented in isolation (single task) or while performing a secondary tracking task (dual task). Cognitive load was measured by means of pupillometry. Mixed-effects models were fit to the data, showing the following: (a) During the dual task, younger but not older adults demonstrated early sensitivity to surprisal (higher levels of cognitive load, indexed by pupil size) as sentences were heard online; (b) Older adults showed no immediate reaction to surprisal, but a delayed response, where their meaningfulness judgments to high-surprisal words remained stable in accuracy, while secondary tracking performance declined. Findings are discussed in relation to age-related trade-offs in dual tasking and differences in the allocation of attentional resources during language processing. Collectively, our data show that higher linguistic load leads to task trade-offs in older adults and differently affects the time course of online language processing in aging. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Salience and Attention in Surprisal-Based Accounts of Language Processing.

The notion of salience has been singled out as the explanatory factor for a diverse range of linguistic phenomena. In particular, perceptual salience (e.g., visual salience of objects in the world, acoustic prominence of linguistic sounds) and semantic-pragmatic salience (e.g., prominence of recently mentioned or topical referents) have been shown to influence language comprehension and production. A different line of research has sought to account for behavioral correlates of cognitive load during comprehension as well as for certain patterns in language usage using information-theoretic notions, such as surprisal. Surprisal and salience both affect language processing at different levels, but the relationship between the two has not been adequately elucidated, and the question of whether salience can be reduced to surprisal / predictability is still open. Our review identifies two main challenges in addressing this question: terminological inconsistency and lack of integration between high and low levels of representations in salience-based accounts and surprisal-based accounts. We capitalize upon work in visual cognition in order to orient ourselves in surveying the different facets of the notion of salience in linguistics and their relation with models of surprisal. We find that work on salience highlights aspects of linguistic communication that models of surprisal tend to overlook, namely the role of attention and relevance to current goals, and we argue that the Predictive Coding framework provides a unified view which can account for the role played by attention and predictability at different levels of processing and which can clarify the interplay between low and high levels of processes and between predictability-driven expectation and attention-driven focus.

The Frequency of Rapid Pupil Dilations as a Measure of Linguistic Processing Difficulty.

While it has long been known that the pupil reacts to cognitive load, pupil size has received little attention in cognitive research because of its long latency and the difficulty of separating effects of cognitive load from the light reflex or effects due to eye movements. A novel measure, the Index of Cognitive Activity (ICA), relates cognitive effort to the frequency of small rapid dilations of the pupil. We report here on a total of seven experiments which test whether the ICA reliably indexes linguistically induced cognitive load: three experiments in reading (a manipulation of grammatical gender match/mismatch, an experiment of semantic fit, and an experiment comparing locally ambiguous subject versus object relative clauses, all in German), three dual-task experiments with simultaneous driving and spoken language comprehension (using the same manipulations as in the single-task reading experiments), and a visual world experiment comparing the processing of causal versus concessive discourse markers. These experiments are the first to investigate the effect and time course of the ICA in language processing. All of our experiments support the idea that the ICA indexes linguistic processing difficulty. The effects of our linguistic manipulations on the ICA are consistent for reading and auditory presentation. Furthermore, our experiments show that the ICA allows for usage within a multi-task paradigm. Its robustness with respect to eye movements means that it is a valid measure of processing difficulty for usage within the visual world paradigm, which will allow researchers to assess both visual attention and processing difficulty at the same time, using an eye-tracker. We argue that the ICA is indicative of activity in the locus caeruleus area of the brain stem, which has recently also been linked to P600 effects observed in psycholinguistic EEG experiments.

Data from eye-tracking corpora as evidence for theories of syntactic processing complexity.

We evaluate the predictions of two theories of syntactic processing complexity, dependency locality theory (DLT) and surprisal, against the Dundee Corpus, which contains the eye-tracking record of 10 participants reading 51,000 words of newspaper text. Our results show that DLT integration cost is not a significant predictor of reading times for arbitrary words in the corpus. However, DLT successfully predicts reading times for nouns. We also find evidence for integration cost effects at auxiliaries, not predicted by DLT. For surprisal, we demonstrate that an unlexicalized formulation of surprisal can predict reading times for arbitrary words in the corpus. Comparing DLT integration cost and surprisal, we find that the two measures are uncorrelated, which suggests that a complete theory will need to incorporate both aspects of processing complexity. We conclude that eye-tracking corpora, which provide reading time data for naturally occurring, contextualized sentences, can complement experimental evidence as a basis for theories of processing complexity.