Dynamically predicting comprehension difficulties through physiological data and intelligent wearables.

Comprehending digital content written in natural language online is vital for many aspects of life, including learning, professional tasks, and decision-making. However, facing comprehension difficulties can have negative consequences for learning outcomes, critical thinking skills, decision-making, error rate, and productivity. This paper introduces an innovative approach to predict comprehension difficulties at the local content level (e.g., paragraphs). Using affordable wearable devices, we acquire physiological responses non-intrusively from the autonomous nervous system, specifically pulse rate variability, and electrodermal activity. Additionally, we integrate data from a cost-effective eye-tracker. Our machine learning algorithms identify 'hotspots' within the content and regions corresponding to a high cognitive load. These hotspots represent real-time predictors of comprehension difficulties. By integrating physiological data with contextual information (such as the levels of experience of individuals), our approach achieves an accuracy of 72.11% ± 2.21, a precision of 0.77, a recall of 0.70, and an f1 score of 0.73. This study opens possibilities for developing intelligent, cognitive-aware interfaces. Such interfaces can provide immediate contextual support, mitigating comprehension challenges within content. Whether through translation, content generation, or content summarization using available Large Language Models, this approach has the potential to enhance language comprehension.

The role of embodied cognition in action language comprehension in L1 and L2.

In this study we carried out a behavioral experiment comparing action language comprehension in L1 (Italian) and L2 (English). Participants were Italian native speakers who had acquired the second language late (after the age of 10). They performed semantic judgments on L1 and L2 literal, idiomatic and metaphorical action sentences after viewing a video of a hand performing an action that was related or unrelated to the verb used in the sentence. Results showed that responses to literal and metaphorical L1 sentences were faster when the action depicted was related to the verb used rather than when the action depicted was unrelated to the verb used. No differences were found for the idiomatic condition. In L2 we found that all responses to the three conditions were facilitated when the action depicted was related to the verb used. Moreover, we found that the difference between the unrelated and the related modalities was greater in L2 than in L1 for the literal and the idiomatic condition but not for the metaphorical condition. These findings are consistent with the embodied cognition hypothesis of language comprehension.

The effects of question previewing on response accuracy and text processing: An eye-movement study.

The present study investigated the impact of manipulating reading strategies (i.e., reading the questions first [QF] or reading the passage first [PF]) during a reading comprehension test where we explored how reading strategy was related to student characteristics (i.e., reading achievement and working memory capacity). Participants' eye movements were monitored as they read 12 passages and answered multiple-choice questions. We examined differences in (a) response accuracy, (b) average total time on words in the text, (c) total task reading time, and (d) time reading text relevant to questions as a function of PF and QF strategies. Analyses were conducted to examine whether findings varied as a function of student characteristics (i.e., reading achievement and working memory capacity) and grade level (Grades 3, 5, and 8). Several interesting findings emerged from our study, including a limited effect of reading strategy use on response accuracy, with only eighth graders demonstrating better accuracy in the QF condition, and several demonstrations of PF leading to more efficient test-taking processes, including (a) longer average total reading times on words in the passage in the PF condition that could be associated with creating a better mental model of the text, (b) often being associated with less total-task time, and (c) being associated with more successful search strategies. Implications for providing teachers and students with strategies are discussed.

Infants who are rarely spoken to nevertheless understand many words.

Theories of language development-informed largely by studies of Western, middleclass infants-have highlighted the language that caregivers direct to children as a key driver of language learning. However, some have argued that language development unfolds similarly across environmental contexts, including those in which childdirected language is scarce. This raises the possibility that children are able to learn from other sources of language in their environments, particularly the language directed to others in their environment. We explore this hypothesis with infants in an indigenous Tseltal-speaking community in Southern Mexico who are rarely spoken to, yet have the opportunity to overhear a great deal of other-directed language by virtue of being carried on their mothers' backs. Adapting a previously established gaze-tracking method for detecting early word knowledge to our field setting, we find that Tseltal infants exhibit implicit knowledge of common nouns (Exp. 1), analogous to their US peers who are frequently spoken to. Moreover, they exhibit comprehension of Tseltal honorific terms that are exclusively used to greet adults in the community (Exp. 2), representing language that could only have been learned through overhearing. In so doing, Tseltal infants demonstrate an ability to discriminate words with similar meanings and perceptually similar referents at an earlier age than has been shown among Western children. Together, these results suggest that for some infants, learning from overhearing may be an important path toward developing language.

The spontaneous activities of the multiple demand network are related to individual differences in indirect replies comprehension.

This study aims to investigate the brain networks engaged in the comprehension of indirect language, as well as the individual difference in this capacity. Specially, we aim to determine whether the difference is solely influenced by the difference in individuals' default network (DN)/language network or whether it also relies on the networks associated with processing of complex cognitive tasks, particularly the multiple demand network (MDN). Conversational indirectness scale (CIS) scores in the interpretation dimension were used as a behavioral indicator of the indirect comprehension tendency. Reading time difference between indirect replies and direct replies collected through a self-paced reading experiment was deemed as a behavioral indicator of comprehension speed of indirect replies comprehension. The two behavioral indicators were combined with resting-state functional magnetic resonance imaging (rs-fMRI). The behaviour-rfMRI analysis showed that ALFF value of right SPL and the functional connectivity (FC) between the right SPL and right IPL/SMA/ITG/Precuneus/bilateral IFG were positively correlated with the interpretation dimension of CIS scores. In addition, the ALFF value of right fusiform gyrus, the FC between the right fusiform gyrus and right precuneus, and the FCs between right SPL and right IPL/Precuneus/IFG were negatively correlated with indirect replies comprehension speed. Overlapping of these regions with large-scale brain network revealed that the right SPL was mainly located in the MDN, and the right fusiform gyrus was mainly located in the language network. Additionally, the areas showing functional connectivity with these regions were primarily located in the MDN, with a smaller subset located in the DN. Our findings suggest that the ability of individuals to actively and rapidly acquire indirect meaning relies not only on the support of the DN and the language network, but also requires collective support from the MDN.

The effects of restrictiveness on relative clause processing in Farsi.

With an eye-tracking experiment, we investigated the processing of Farsi object and subject relative clauses. Since restrictive relative clauses in Farsi are marked and distinguished clearly by the enclitic particle ی /-i/ attached to the head noun, we also compared the processing of restrictive and non-restrictive relative clauses. Seifi (2021) conducted a corpus analysis that showed that object relative clauses are in general less frequent than subject relative clauses. However, while non-restrictive relative clauses are predominantly subject relative clauses, restrictive relative clauses are more balanced in the corpus. In an eye-tracking experiment, Farsi speakers processed restrictive and non-restrictive relative clauses differently. In non-restrictive relative clauses, the effect is similar to that found in most other languages: a clear processing delay in object relative clauses, compared to subject relative clauses. This effect was visible both at the relative clause verb and at the end of the matrix sentence. In restrictive relative clauses, on the other hand, the picture is different: Just as for the non-restrictive relative clauses object relative clauses had long reading times in the relative clause, but at the end of the sentence a reverse effect was found. Thus, the processing data reflected the pattern found in the corpus. We discuss these findings in terms of the distinct functions of restrictive and non-restrictive relative clauses.

Segmentals weigh much more in comprehensibility than nuclear stress in read speech.

This study investigates the impact of segmental accuracy and nucleus placement on the comprehensibility of English as an International Language (EIL), with the aim of informing phonological norms and teaching models. Speech samples from 59 EIL speakers with varying levels of segmental accuracy were collected during a reading task, involving reading a passage in three different versions of speech, each version lasting approximately 30 to 40 s. To directly compare the impact of nuclear stress placement on comprehensibility, based on these samples, two versions of stimuli were created, each differing only in their placement of nuclear stress - either correct or incorrect. The correctness of placements was determined by seven native speakers of English. Eight native English speakers, aged 19-24, and eight EIL speakers, aged 20-24 with an upper-intermediate to advanced proficiency level, rated the comprehensibility of the two versions of speech. Results suggest that while correct nucleus placement enhances comprehensibility for native English listeners, it has little influence on EIL listeners. Segmental accuracy in EIL speech impacts comprehensibility substantially more than nucleus placement on both native and EIL listeners, indicating that English language teaching should focus on minimizing segmental errors to improve comprehensibility for EIL speakers, despite the benefits of correct nucleus placement.

Middle-schoolers' reading and lexical-semantic processing depth in response to digital and print media: An N400 study.

We report the first use of ERP measures to identify text engagement differences when reading digitally or in print. Depth of semantic encoding is key for reading comprehension, and we predicted that deeper reading of expository texts would facilitate stronger associations with subsequently-presented related words, resulting in enhanced N400 responses to unrelated probe words and a graded attenuation of the N400 to related and moderately related words. In contrast, shallow reading would produce weaker associations between probe words and text passages, resulting in enhanced N400 responses to both moderately related and unrelated words, and an attenuated response to related words. Behavioral research has shown deeper semantic encoding of text from paper than from a screen. Hence, we predicted that the N400 would index deeper reading of text passages that were presented in print, and shallower reading of texts presented digitally. Middle-school students (n = 59) read passages in digital and print formats and high-density EEG was recorded while participants completed single-word semantic judgment tasks after each passage. Following digital text presentation, the N400 response pattern to moderately-related words indicated shallow reading, tracking with responses to words that were unrelated to the text. Following print reading, the N400 responses to moderately-related words patterned instead with responses to related words, interpreted as an index of deeper reading. These findings provide evidence of differences in brain responses to texts presented in print and digital media, including deeper semantic encoding for print than digital texts.

The Relationships of Oral Reading Fluency at Word, Sentence, and Passage Levels and Reading Comprehension in Chinese.

This study aims to expand our understanding of the relations of oral reading fluency at word, sentence, and passage levels to reading comprehension in Chinese-speaking secondary school-aged students. In total, 80 participants (46 males and 34 females) ranging from 13 to 15 years old joined this study and were tested on tasks of oral reading fluency at three levels, reading comprehension, and nonverbal IQ. Our results showed a clear relationship from fluency at the level of the word to the sentence and then the passage in oral reading fluency as well as both the direct and indirect importance of word-level oral reading fluency in reading comprehension. Only the indirect effect from word-level oral reading fluency to reading comprehension through passage-level oral reading fluency was significant. Our findings suggest that sentence-level oral reading fluency is the crucial component to reading comprehension in Chinese. Additionally, recognition of the potential value of unique features, such as syntactic awareness and word segment accuracy, that happen at the sentence level should be integrated into instructional activities for reading.

Predicting the next sentence (not word) in large language models: What model-brain alignment tells us about discourse comprehension.

Current large language models (LLMs) rely on word prediction as their backbone pretraining task. Although word prediction is an important mechanism underlying language processing, human language comprehension occurs at multiple levels, involving the integration of words and sentences to achieve a full understanding of discourse. This study models language comprehension by using the next sentence prediction (NSP) task to investigate mechanisms of discourse-level comprehension. We show that NSP pretraining enhanced a model's alignment with brain data especially in the right hemisphere and in the multiple demand network, highlighting the contributions of nonclassical language regions to high-level language understanding. Our results also suggest that NSP can enable the model to better capture human comprehension performance and to better encode contextual information. Our study demonstrates that the inclusion of diverse learning objectives in a model leads to more human-like representations, and investigating the neurocognitive plausibility of pretraining tasks in LLMs can shed light on outstanding questions in language neuroscience.

Animal cognition: Dogs build semantic expectations between spoken words and objects.

A recent study has used scalp-recorded electroencephalography to obtain evidence of semantic processing of human speech and objects by domesticated dogs. The results suggest that dogs do comprehend the meaning of familiar spoken words, in that a word can evoke the mental representation of the object to which it refers.

Greater utilization of executive functions networks when listening to stories with visual stimulation is related to lower reading abilities in children.

Narrative comprehension relies on basic sensory processing abilities, such as visual and auditory processing, with recent evidence for utilizing executive functions (EF), which are also engaged during reading. EF was previously related to the "supporter" of engaging the auditory and visual modalities in different cognitive tasks, with evidence of lower efficiency in this process among those with reading difficulties in the absence of a visual stimulus (i.e. while listening to stories). The current study aims to fill out the gap related to the level of reliance on these neural circuits while visual aids (pictures) are involved during story listening in relation to reading skills. Functional MRI data were collected from 44 Hebrew-speaking children aged 8-12 years while listening to stories with vs without visual stimuli (i.e., pictures). Functional connectivity of networks supporting reading was defined in each condition and compared between the conditions against behavioral reading measures. Lower reading skills were related to greater functional connectivity values between EF networks (default mode and memory networks), and between the auditory and memory networks for the stories with vs without the visual stimulation. A greater difference in functional connectivity between the conditions was related to lower reading scores. We conclude that lower reading skills in children may be related to a need for greater scaffolding, i.e., visual stimulation such as pictures describing the narratives when listening to stories, which may guide future intervention approaches.

Neural processing of speech comprehension in noise predicts individual age using fNIRS-based brain-behavior models.

Speech comprehension in noise depends on complex interactions between peripheral sensory and central cognitive systems. Despite having normal peripheral hearing, older adults show difficulties in speech comprehension. It remains unclear whether the brain's neural responses could indicate aging. The current study examined whether individual brain activation during speech perception in different listening environments could predict age. We applied functional near-infrared spectroscopy to 93 normal-hearing human adults (20 to 70 years old) during a sentence listening task, which contained a quiet condition and 4 different signal-to-noise ratios (SNR = 10, 5, 0, -5 dB) noisy conditions. A data-driven approach, the region-based brain-age predictive modeling was adopted. We observed a significant behavioral decrease with age under the 4 noisy conditions, but not under the quiet condition. Brain activations in SNR = 10 dB listening condition could successfully predict individual's age. Moreover, we found that the bilateral visual sensory cortex, left dorsal speech pathway, left cerebellum, right temporal-parietal junction area, right homolog Wernicke's area, and right middle temporal gyrus contributed most to prediction performance. These results demonstrate that the activations of regions about sensory-motor mapping of sound, especially in noisy conditions, could be sensitive measures for age prediction than external behavior measures.

Mapping lower secondary school students' conceptions of three aspects critical for understanding the nervous system.

Understanding the nervous system is an important but perhaps ambitious goal, particularly for students in lower secondary education. It is important because of its' direct role in both mental and physical health, and it is ambitious because instruction focuses on the human nervous system, which is extremely complex, and subject to numerous misconceptions. Despite its' complexity, the science curricula, both nationally and internationally, emphasize an understanding of the system, and not just knowledge of isolated facts. But what does it mean to understand this system, and what content knowledge is critical for understanding it? Unfortunately, the curricula are usually too general to answer these questions, therefore other sources of information are needed. Using the science literature, the present study defines the system level of the nervous system and proposes three basic aspects necessary to understand it: 1) neural circuit architecture, 2) synaptic action, and 3) nerve signal origin. With this background, the aim of the present study is to identify lower secondary school students' conceptions of these three aspects, and to determine how they impact students' understanding of the system. To reach this aim, the study used a questionary which allowed for a mixed method design, and the results show that many students have an immediate conception of the brain as the origin of nerve signals. In addition, many students hold the alternative conceptions that 1) synaptic action is exclusively excitatory, and that 2) neural circuits consists of neurons connected in a chain, one single neuron after another. These alternative conceptions prevent students from understanding the system. Implications for instruction are discussed in the context of conceptual learning theories, and teaching strategies are proposed. Since similar curricula goals and textbook content exist in several countries, the present results may be representative across nations.

Linguistic modulation of the neural encoding of phonemes.

Speech comprehension entails the neural mapping of the acoustic speech signal onto learned linguistic units. This acousto-linguistic transformation is bi-directional, whereby higher-level linguistic processes (e.g. semantics) modulate the acoustic analysis of individual linguistic units. Here, we investigated the cortical topography and linguistic modulation of the most fundamental linguistic unit, the phoneme. We presented natural speech and "phoneme quilts" (pseudo-randomly shuffled phonemes) in either a familiar (English) or unfamiliar (Korean) language to native English speakers while recording functional magnetic resonance imaging. This allowed us to dissociate the contribution of acoustic vs. linguistic processes toward phoneme analysis. We show that (i) the acoustic analysis of phonemes is modulated by linguistic analysis and (ii) that for this modulation, both of acoustic and phonetic information need to be incorporated. These results suggest that the linguistic modulation of cortical sensitivity to phoneme classes minimizes prediction error during natural speech perception, thereby aiding speech comprehension in challenging listening situations.

Indirect and direct contributions of executive functions to reading comprehension.

In the current study, we investigated the role of executive functions in explaining how word recognition and language comprehension jointly predict reading comprehension in multilingual and monolingual students (Grades 1 and 2). Specifically, mediation and moderation models were tested and compared to offer a more nuanced understanding of the role of executive functions in reading comprehension. The results provided support for the mediation model in which executive functions indirectly contribute to reading comprehension via word recognition and language comprehension in both language groups. In addition, executive functions directly predicted reading comprehension (i.e., partial mediation). These findings suggest that executive functions serve as general cognitive processes that support word recognition, language comprehension, and reading comprehension (i.e., direct contribution) as well as facilitate connecting word recognition and language comprehension in support for reading comprehension (i.e., indirect contribution). These findings are consistent with prominent models of reading comprehension.

Evidence for an event-integration window: A cognitive temporal window supports flexible integration of multimodal events.

Just as the perception of simple events such as clapping hands requires a linkage of sound with movements that produce the sound, the integration of more complex events such as describing how to give an injection requires a linkage between the instructor's utterances and their actions. However, the mechanism for integrating these complex multimodal events is unclear. For example, it is possible that predictive temporal relationships are important for multimodal event understanding, but it is also possible that this form of understanding arises more from meaningful causal between-event links that are temporally unspecified. This latter approach might be supported by a cognitive temporal window within which multimodal event information integrates flexibly with few default commitments about specific temporal relationships. To test this hypothesis, we assessed the consequences of disrupting temporal relationships between instructors' actions and their speech in both narrated screen-capture instructional videos (Experiment 1) and live-action instructional videos (Experiment 2) by displacing the audio channel forward or backward relative to the video by 0, 1, 3, or 7 s. We assessed learning, event segmentation, disruption awareness, segmentation uncertainty, and perceived workload. Across two experiments, 7-s temporal disruptions consistently increased uncertainty and workload and decreased learning in Experiment 2. None of these effects appeared for 3-s disruptions, which were barely detectable. One-second disruptions produced no effects and were undetectable, even though much intraevent information falls within this range. Our results suggest the presence of an event-integration window that supports the integration of events independent of constraining temporal relationships between subevents. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Delta-band Activity Underlies Referential Meaning Representation during Pronoun Resolution.

Human language offers a variety of ways to create meaning, one of which is referring to entities, objects, or events in the world. One such meaning maker is understanding to whom or to what a pronoun in a discourse refers to. To understand a pronoun, the brain must access matching entities or concepts that have been encoded in memory from previous linguistic context. Models of language processing propose that internally stored linguistic concepts, accessed via exogenous cues such as phonological input of a word, are represented as (a)synchronous activities across a population of neurons active at specific frequency bands. Converging evidence suggests that delta band activity (1-3 Hz) is involved in temporal and representational integration during sentence processing. Moreover, recent advances in the neurobiology of memory suggest that recollection engages neural dynamics similar to those which occurred during memory encoding. Integrating from these two research lines, we here tested the hypothesis that neural dynamic patterns, especially in delta frequency range, underlying referential meaning representation, would be reinstated during pronoun resolution. By leveraging neural decoding techniques (i.e., representational similarity analysis) on a magnetoencephalogram data set acquired during a naturalistic story-listening task, we provide evidence that delta-band activity underlies referential meaning representation. Our findings suggest that, during spoken language comprehension, endogenous linguistic representations such as referential concepts may be proactively retrieved and represented via activation of their underlying dynamic neural patterns.

Effect of contextual prediction on emotional word processing: an evidence from ERPNR-D-24-00189.

OBJECTIVE: This study examined the effect of context on the prediction of emotional words with varying valences. It investigated the neural mechanisms underlying the processing differences of emotion words with different valences in both predictable and unpredictable contexts. Additionally, it aimed to address the conflicting results regarding the processing time in predictive contexts reported in previous studies. METHODS: Participants were instructed to carefully read the text that included the specified emotion words. Event-related potentials elicited by emotional words were measured. To ensure that the participants can read the text carefully, 33% of the texts are followed by comprehension problems. After reading the text, the comprehension questions were answered based on the text content. RESULTS: The study revealed that the N400 amplitude elicited by an unpredictable context was greater than that elicited by a predictable context. Additionally, the N400 amplitude triggered by positive emotion words was larger than that triggered by negative emotion words. However, there was no significant difference in late positive component amplitude observed between contextual prediction and emotional word valence. CONCLUSION: The present study suggests that predictive processing takes place at an intermediate stage of speech processing, approximately 400 ms after stimulus onset. Furthermore, the presence of a predictive context enhances the processing of emotional information. Notably, brain activity is more pronounced during the processing of positive emotional stimuli compared to negative emotional stimuli. Additionally, the facilitative effect of a predictable context diminishes in the advanced phase of Chinese speech comprehension.

What eye and hand movements tell us about expectations towards argument order: An eye- and mouse-tracking study in German.

Previous research on real-time sentence processing in German has shown that listeners use the morphological marking of accusative case on a sentence-initial noun phrase to not only interpret the current argument as the object and patient, but also to predict a plausible agent. So far, less is known about the use of case marking to predict the semantic role of upcoming arguments after the subject/agent has been encountered. In the present study, we examined the use of case marking for argument interpretation in transitive as well as ditransitive structures. We aimed to control for multiple factors that could have influenced processing in previous studies, including the animacy of arguments, world knowledge, and the perceptibility of the case cue. Our results from eye- and mouse-tracking indicate that the exploitation of the first case cue that enables the interpretation of the unfolding sentence is influenced by (i) the strength of argument order expectation and (ii) the perceptual salience of the case cue. PsycINFO code: 2720 Linguistics & Language & Speech.