Dataset for multimodal fake news detection and verification tasks.

The proliferation of online disinformation and fake news, particularly in the context of breaking news events, demands the development of effective detection mechanisms. While textual content remains the predominant medium for disseminating misleading information, the contribution of other modalities is increasingly emerging within online outlets and social media platforms. However, multimodal datasets, which incorporate diverse modalities such as texts and images, are not very common yet, especially in low-resource languages. This study addresses this gap by releasing a dataset tailored for multimodal fake news detection in the Italian language. This dataset was originally employed in a shared task on the Italian language. The dataset is divided into two data subsets, each corresponding to a distinct sub-task. In sub-task 1, the goal is to assess the effectiveness of multimodal fake news detection systems. Sub-task 2 aims to delve into the interplay between text and images, specifically analyzing how these modalities mutually influence the interpretation of content when distinguishing between fake and real news. Both sub-tasks were managed as classification problems. The dataset consists of social media posts and news articles. After collecting it, it was labeled via crowdsourcing. Annotators were provided with external knowledge about the topic of the news to be labeled, enhancing their ability to discriminate between fake and real news. The data subsets for sub-task 1 and sub-task 2 consist of 913 and 1350 items, respectively, encompassing newspaper articles and tweets.

Neural Generative Models and the Parallel Architecture of Language: A Critical Review and Outlook.

According to the parallel architecture, syntactic and semantic information processing are two separate streams that interact selectively during language comprehension. While considerable effort is put into psycho- and neurolinguistics to understand the interchange of processing mechanisms in human comprehension, the nature of this interaction in recent neural Large Language Models remains elusive. In this article, we revisit influential linguistic and behavioral experiments and evaluate the ability of a large language model, GPT-3, to perform these tasks. The model can solve semantic tasks autonomously from syntactic realization in a manner that resembles human behavior. However, the outcomes present a complex and variegated picture, leaving open the question of how Language Models could learn structured conceptual representations.

Event Knowledge in Large Language Models: The Gap Between the Impossible and the Unlikely.

Word co-occurrence patterns in language corpora contain a surprising amount of conceptual knowledge. Large language models (LLMs), trained to predict words in context, leverage these patterns to achieve impressive performance on diverse semantic tasks requiring world knowledge. An important but understudied question about LLMs' semantic abilities is whether they acquire generalized knowledge of common events. Here, we test whether five pretrained LLMs (from 2018's BERT to 2023's MPT) assign a higher likelihood to plausible descriptions of agent-patient interactions than to minimally different implausible versions of the same event. Using three curated sets of minimal sentence pairs (total n = 1215), we found that pretrained LLMs possess substantial event knowledge, outperforming other distributional language models. In particular, they almost always assign a higher likelihood to possible versus impossible events (The teacher bought the laptop vs. The laptop bought the teacher). However, LLMs show less consistent preferences for likely versus unlikely events (The nanny tutored the boy vs. The boy tutored the nanny). In follow-up analyses, we show that (i) LLM scores are driven by both plausibility and surface-level sentence features, (ii) LLM scores generalize well across syntactic variants (active vs. passive constructions) but less well across semantic variants (synonymous sentences), (iii) some LLM errors mirror human judgment ambiguity, and (iv) sentence plausibility serves as an organizing dimension in internal LLM representations. Overall, our results show that important aspects of event knowledge naturally emerge from distributional linguistic patterns, but also highlight a gap between representations of possible/impossible and likely/unlikely events.

A study on surprisal and semantic relatedness for eye-tracking data prediction.

Previous research in computational linguistics dedicated a lot of effort to using language modeling and/or distributional semantic models to predict metrics extracted from eye-tracking data. However, it is not clear whether the two components have a distinct contribution, with recent studies claiming that surprisal scores estimated with large-scale, deep learning-based language models subsume the semantic relatedness component. In our study, we propose a regression experiment for estimating different eye-tracking metrics on two English corpora, contrasting the quality of the predictions with and without the surprisal and the relatedness components. Different types of relatedness scores derived from both static and contextual models have also been tested. Our results suggest that both components play a role in the prediction, with semantic relatedness surprisingly contributing also to the prediction of function words. Moreover, they show that when the metric is computed with the contextual embeddings of the BERT model, it is able to explain a higher amount of variance.

Eliciting semantic properties: methods and applications.

Asking subjects to list semantic properties for concepts is essential for predicting performance in several linguistic and non-linguistic tasks and for creating carefully controlled stimuli for experiments. The property elicitation task and the ensuing norms are widely used across the field, to investigate the organization of semantic memory and design computational models thereof. The contributions of the current Special Topic discuss several core issues concerning how semantic property norms are constructed and how they may be used for research aiming at understanding cognitive processing.

Learning abstract words and concepts: insights from developmental language disorder.

Some explanations of abstract word learning suggest that these words are learnt primarily from the linguistic input, using statistical co-occurrences of words in language, whereas concrete words can also rely on non-linguistic, experiential information. According to this hypothesis, we expect that, if the learner is not able to fully exploit the information in the linguistic input, abstract words should be affected more than concrete ones. Embodied approaches instead argue that both abstract and concrete words can rely on experiential information and, therefore, there might not be any linguistic primacy. Here, we test the role of linguistic input in the development of abstract knowledge with children with developmental language disorder (DLD) and typically developing children aged 8-13. We show that DLD children, who by definition have impoverished language, do not show a disproportionate impairment for abstract words in lexical decision and definition tasks. These results indicate that linguistic information does not have a primary role in the learning of abstract concepts and words; rather, it would play a significant role in semantic development across all domains of knowledge.This article is part of the theme issue 'Varieties of abstract concepts: development, use and representation in the brain'.

The Emotions of Abstract Words: A Distributional Semantic Analysis.

Recent psycholinguistic and neuroscientific research has emphasized the crucial role of emotions for abstract words, which would be grounded by affective experience, instead of a sensorimotor one. The hypothesis of affective embodiment has been proposed as an alternative to the idea that abstract words are linguistically coded and that linguistic processing plays a key role in their acquisition and processing. In this paper, we use distributional semantic models to explore the complex interplay between linguistic and affective information in the representation of abstract words. Distributional analyses on Italian norming data show that abstract words have more affective content and tend to co-occur with contexts with higher emotive values, according to affective statistical indices estimated in terms of distributional similarity with a restricted number of seed words strongly associated with a set of basic emotions. Therefore, the strong affective content of abstract words might just be an indirect byproduct of co-occurrence statistics. This is consistent with a version of representational pluralism in which concepts that are fully embodied either at the sensorimotor or at the affective level live side-by-side with concepts only indirectly embodied via their linguistic associations with other embodied words.

Complement Coercion: The Joint Effects of Type and Typicality.

Complement coercion (begin a book →reading) involves a type clash between an event-selecting verb and an entity-denoting object, triggering a covert event (reading). Two main factors involved in complement coercion have been investigated: the semantic type of the object (event vs. entity), and the typicality of the covert event (the author began a book →writing). In previous research, reading times have been measured at the object. However, the influence of the typicality of the subject-object combination on processing an aspectual verb such as begin has not been studied. Using a self-paced reading study, we manipulated semantic type and subject-object typicality, exploiting German word order to measure reading times at the aspectual verb. These variables interacted at the target verb. We conclude that both type and typicality probabilistically guide expectations about upcoming input. These results are compatible with an expectation-based view of complement coercion and language comprehension more generally in which there is rapid interaction between what is typically viewed as linguistic knowledge, and what is typically viewed as domain general knowledge about how the world works.

Modality-independent encoding of individual concepts in the left parietal cortex.

The organization of semantic information in the brain has been mainly explored through category-based models, on the assumption that categories broadly reflect the organization of conceptual knowledge. However, the analysis of concepts as individual entities, rather than as items belonging to distinct superordinate categories, may represent a significant advancement in the comprehension of how conceptual knowledge is encoded in the human brain. Here, we studied the individual representation of thirty concrete nouns from six different categories, across different sensory modalities (i.e., auditory and visual) and groups (i.e., sighted and congenitally blind individuals) in a core hub of the semantic network, the left angular gyrus, and in its neighboring regions within the lateral parietal cortex. Four models based on either perceptual or semantic features at different levels of complexity (i.e., low- or high-level) were used to predict fMRI brain activity using representational similarity encoding analysis. When controlling for the superordinate component, high-level models based on semantic and shape information led to significant encoding accuracies in the intraparietal sulcus only. This region is involved in feature binding and combination of concepts across multiple sensory modalities, suggesting its role in high-level representation of conceptual knowledge. Moreover, when the information regarding superordinate categories is retained, a large extent of parietal cortex is engaged. This result indicates the need to control for the coarse-level categorial organization when performing studies on higher-level processes related to the retrieval of semantic information.

How concepts are encoded in the human brain: A modality independent, category-based cortical organization of semantic knowledge.

How conceptual knowledge is represented in the human brain remains to be determined. To address the differential role of low-level sensory-based and high-level abstract features in semantic processing, we combined behavioral studies of linguistic production and brain activity measures by functional magnetic resonance imaging in sighted and congenitally blind individuals while they performed a property-generation task with concrete nouns from eight categories, presented through visual and/or auditory modalities. Patterns of neural activity within a large semantic cortical network that comprised parahippocampal, lateral occipital, temporo-parieto-occipital and inferior parietal cortices correlated with linguistic production and were independent both from the modality of stimulus presentation (either visual or auditory) and the (lack of) visual experience. In contrast, selected modality-dependent differences were observed only when the analysis was limited to the individual regions within the semantic cortical network. We conclude that conceptual knowledge in the human brain relies on a distributed, modality-independent cortical representation that integrates the partial category and modality specific information retained at a regional level.

Logical metonymy resolution in a words-as-cues framework: evidence from self-paced reading and probe recognition.

Logical metonymy resolution (begin a book → begin reading a book or begin writing a book) has traditionally been explained either through complex lexical entries (qualia structures) or through the integration of the implicit event via post-lexical access to world knowledge. We propose that recent work within the words-as-cues paradigm can provide a more dynamic model of logical metonymy, accounting for early and dynamic integration of complex event information depending on previous contextual cues (agent and patient). We first present a self-paced reading experiment on German subordinate sentences, where metonymic sentences and their paraphrased version differ only in the presence or absence of the clause-final target verb (Der Konditor begann die Glasur → Der Konditor begann, die Glasur aufzutragen/The baker began the icing → The baker began spreading the icing). Longer reading times at the target verb position in a high-typicality condition (baker + icing → spread ) compared to a low-typicality (but still plausible) condition (child + icing → spread) suggest that we make use of knowledge activated by lexical cues to build expectations about events. The early and dynamic integration of event knowledge in metonymy interpretation is bolstered by further evidence from a second experiment using the probe recognition paradigm. Presenting covert events as probes following a high-typicality or a low-typicality metonymic sentence (Der Konditor begann die Glasur → AUFTRAGEN/The baker began the icing → SPREAD), we obtain an analogous effect of typicality at 100 ms interstimulus interval.

Predication drives verb cortical signatures.

Verbs and nouns are fundamental units of language, but their neural instantiation remains poorly understood. Neuropsychological research has shown that nouns and verbs can be damaged independently of each other, and neuroimaging research has found that several brain regions respond differentially to the two word classes. However, the semantic-lexical properties of verbs and nouns that drive these effects remain unknown. Here we show that the most likely candidate is predication: a core lexical feature involved in binding constituent arguments (boy, candies) into a unified syntactic-semantic structure expressing a proposition (the boy likes the candies). We used functional neuroimaging to test whether the intrinsic "predication-building" function of verbs is what drives the verb-noun distinction in the brain. We first identified verb-preferring regions with a localizer experiment including verbs and nouns. Then, we examined whether these regions are sensitive to transitivity--an index measuring its tendency to select for a direct object. Transitivity is a verb-specific property lying at the core of its predication function. Neural activity in the left posterior middle temporal and inferior frontal gyri correlates with transitivity, indicating sensitivity to predication. This represents the first evidence that grammatical class preference in the brain is driven by a word's function to build predication structures.

BLIND: a set of semantic feature norms from the congenitally blind.

Feature-based descriptions of concepts produced by subjects in a property generation task are widely used in cognitive science to develop empirically grounded concept representations and to study systematic trends in such representations. This article introduces BLIND, a collection of parallel semantic norms collected from a group of congenitally blind Italian subjects and comparable sighted subjects. The BLIND norms comprise descriptions of 50 nouns and 20 verbs. All the materials have been semantically annotated and translated into English, to make them easily accessible to the scientific community. The article also presents a preliminary analysis of the BLIND data that highlights both the large degree of overlap between the groups and interesting differences. The complete BLIND norms are freely available and can be downloaded from http://sesia.humnet.unipi.it/blind_data .