Asymmetries in encoding event roles: Evidence from language and cognition.

It has long been hypothesized that the linguistic structure of events, including event participants and their relative prominence, draws on the non-linguistic nature of events and the roles that these events license. However, the precise relation between the prominence of event participants in language and cognition has not been tested experimentally in a systematic way. Here we address this gap. In four experiments, we investigate the relative prominence of (animate) Agents, Patients, Goals and Instruments in the linguistic encoding of complex events and the prominence of these event roles in cognition as measured by visual search and change blindness tasks. The relative prominence of these event roles was largely similar-though not identical-across linguistic and non-linguistic measures. Across linguistic and non-linguistic tasks, Patients were more salient than Goals, which were more salient than Instruments. (Animate) Agents were more salient than Patients in linguistic descriptions and visual search; however, this asymmetrical pattern did not emerge in change detection. Overall, our results reveal homologies between the linguistic and non-linguistic prominence of individual event participants, thereby lending support to the claim that the linguistic structure of events builds on underlying conceptual event representations. We discuss implications of these findings for linguistic theory and theories of event cognition.

It's not just what we don't know: The mapping problem in the acquisition of negation.

How do learners learn what no and not mean when they are only presented with what is? Given its complexity, abstractness, and roles in logic, truth-functional negation might be a conceptual accomplishment. As a result, young children's gradual acquisition of negation words might be due to their undergoing a gradual conceptual change that is necessary to represent those words' logical meaning. However, it's also possible that linguistic expressions of negation take time to learn because of children's gradually increasing grasp of their language. To understand what no and not mean, children might first need to understand the rest of the sentences in which those words are used. We provide experimental evidence that conceptually equipped learners (adults) face the same acquisition challenges that children do when their access to linguistic information is restricted, which simulates how much language children understand at different points in acquisition. When watching a silenced video of naturalistic uses of negators by parents speaking to their children, adults could tell when the parent was prohibiting the child and struggled with inferring that negators were used to express logical negation. However, when provided with additional information about what else the parent said, guessing that the parent had expressed logical negation became easy for adults. Though our findings do not rule out that young learners also undergo conceptual change, they show that increasing understanding of language alone, with no accompanying conceptual change, can account for the gradual acquisition of negation words.

Keeping quantifier meaning in mind: Connecting semantics, cognition, and pragmatics.

A complete theory of the meaning of linguistic expressions needs to explain their semantic properties, their links to non-linguistic cognition, and their use in communication. Even though in principle interconnected, these areas are generally not pursued in tandem. We present a novel take on the semantics-cognition-pragmatics interface. We propose that formal semantic differences in expressions' meanings lead those meanings to activate distinct cognitive systems, which in turn have downstream effects on when speakers prefer to use those expressions. As a case study, we focus on the quantifiers "each" and "every", which can be used to talk about the same state of the world, but have been argued to differ in meaning. In particular, we adopt a mentalistic proposal about these quantifiers on which "each" has a purely individualistic meaning that interfaces with the psychological system for representing object-files, whereas "every" has a meaning that implicates a group and interfaces with the psychological system for representing ensembles. In seven experiments, we demonstrate that this account correctly predicts both known and newly-observed constraints on how "each" and "every" are pragmatically used. More generally, this integrated approach to semantics, cognition, and pragmatics suggests that canonical patterns of language use can be affected in predictable ways by fine-grained differences in semantic meanings and the cognitive systems to which those meanings connect.

Where word and world meet: Language and vision share an abstract representation of symmetry.

Symmetry is ubiquitous in nature, in logic and mathematics, and in perception, language, and thought. Although humans are exquisitely sensitive to visual symmetry (e.g., of a butterfly), symmetry in natural language goes beyond visuospatial properties: many words point to abstract concepts with symmetrical content (e.g., equal, marry). For example, if Mark marries Bill, then Bill marries Mark. In both cases (vision and language), symmetry may be formally characterized as invariance under transformation. Is this a coincidence, or is there some deeper psychological resemblance? Here we asked whether representations of symmetry correspond across language and vision. To do so, we developed a novel cross-modal matching paradigm. On each trial, participants observed a visual stimulus (either symmetrical or nonsymmetrical) and had to choose between a symmetrical and nonsymmetrical English predicate unrelated to the stimulus (e.g., "negotiate" vs. "propose"). In a first study with visual events (symmetrical collision or asymmetrical launch), participants reliably chose the predicate matching the event's symmetry. A second study showed that this "language-vision correspondence" generalized to objects and was weakened when the stimuli's binary nature was made less apparent (i.e., for one object, rather than two inward-facing objects). A final study showed the same effect when nonsigners guessed English translations of signs from American Sign Language, which expresses many symmetrical concepts spatially. Taken together, our findings support the existence of an abstract representation of symmetry which humans access via both perceptual and linguistic means. More broadly, this work sheds light on the rich, structured nature of the language-cognition interface. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Being suspicious of suspicious coincidences: The case of learning subordinate word meanings.

How do learners acquire subordinate terms (such as Dalmatian) and overcome the bias that words have basic-level meanings (such as dog)? Xu and Tenenbaum [Xu, F., & Tenenbaum, J. B. (2007a). Word learning as Bayesian inference. Psychological Review, 114(2), 245-272] found that both children and adults can learn the subordinate meaning of a novel word when it is used ostensively to label multiple exemplars: learners appear to reason about sampling statistics, detecting the suspicious coincidence that, e.g., a random sample of dogs all happen to be Dalmatians. Crucially though, their experimental support did not come from cross-situational ostensive labeling contexts, but from single instances that presented all exemplars at once and included a co-present test array that likely highlighted the relevant semantic contrast. Here we find that adults do not use suspicious coincidences during cross-situational word learning. We only find effects of suspicious coincidences in adults under specific testing conditions similar to those used by Xu and Tenenbaum. We find that adults show a basic-level meaning preference even after encountering five subordinate-level exemplars cross-situationally, even when the first three exemplars were presented simultaneously and labeled ostensively. Instead, participants arrived at subordinate meanings only within settings that highlighted the relevant semantic contrast, i.e., when the target words had referents that belonged to the same basic-level category (e.g., two words referring to dogs, with one referring to Dalmatians and the other to non-Dalmatian dogs). Our findings are consistent with a "semantic contrast" account of word learning, in which learners evaluate which semantic contrasts are relevant in the local learning context and use that information to constrain word meaning.

Encoding Motion Events During Language Production: Effects of Audience Design and Conceptual Salience.

We investigate the extent to which pragmatic versus conceptual factors can affect a speaker's decision to mention or omit different components of an event. In the two experiments, we demonstrate the special role of pragmatic factors related to audience design in speakers' decisions to mention conceptually "peripheral" event components, such as sources (i.e., starting points) in source-goal motion events (e.g., a baby crawling from a crib to a toybox). In particular, we found that pragmatic factors related to audience design could not only drive the decision to omit sources from mention, but could also motivate speakers to mention sources more often than needed. By contrast, speaker's decisions to talk about goals did not appear to be fundamentally driven by pragmatic factors in communication. We also manipulated the animacy of the figure in motion and found that participants in our studies treated both animate and inanimate source-goal motion events in the same way, both linguistically and in memory. We discuss the implications of our work for message generation across different communicative contexts and for future work on the topic of audience design.

Representing agents, patients, goals and instruments in causative events: A cross-linguistic investigation of early language and cognition.

Although it is widely assumed that the linguistic description of events is based on a structured representation of event components at the perceptual/conceptual level, little empirical work has tested this assumption directly. Here, we test the connection between language and perception/cognition cross-linguistically, focusing on the relative salience of causative event components in language and cognition. We draw on evidence from preschoolers speaking English or Turkish. In a picture description task, Turkish-speaking 3-5-year-olds mentioned Agents less than their English-speaking peers (Turkish allows subject drop); furthermore, both language groups mentioned Patients more frequently than Goals, and Instruments less frequently than either Patients or Goals. In a change blindness task, both language groups were equally accurate at detecting changes to Agents (despite surface differences in Agent mentions). The remaining components also behaved similarly: both language groups were less accurate in detecting changes to Instruments than either Patients or Goals (even though Turkish-speaking preschoolers were less accurate overall than their English-speaking peers). To our knowledge, this is the first study offering evidence for a strong-even though not strict-homology between linguistic and conceptual event roles in young learners cross-linguistically.

Now You Hear Me, Later You Don't: The Immediacy of Linguistic Computation and the Representation of Speech.

What happens to an acoustic signal after it enters the mind of a listener? Previous work has demonstrated that listeners maintain intermediate representations over time. However, the internal structure of such representations-be they the acoustic-phonetic signal or more general information about the probability of possible categories-remains underspecified. We present two experiments using a novel speaker-adaptation paradigm aimed at uncovering the format of speech representations. We exposed adult listeners (N = 297) to a speaker whose utterances contained acoustically ambiguous information concerning phones (and thus words), and we manipulated the temporal availability of disambiguating cues via visually presented text (presented before or after each utterance). Results from a traditional phoneme-categorization task showed that listeners adapted to a modified acoustic distribution when disambiguating text was provided before but not after the audio. These results support the position that speech representations consist of activation over categories and are inconsistent with direct maintenance of the acoustic-phonetic signal.

"Look! It is not a bamoule!": 18- and 24-month-olds can use negative sentences to constrain their interpretation of novel word meanings.

Two word-learning experiments were conducted to investigate the understanding of negative sentences in 18- and 24-month-old children. In Experiment 1, after learning that bamoule means "penguin" and pirdaling means "cartwheeling," 18-month-olds (n = 48) increased their looking times when listening to negative sentences rendered false by their visual context ("Look! It is not a bamoule!" while watching a video showing a penguin cartwheeling); however, they did not change their looking behavior when negative sentences were rendered true by their context ("Look! It is not pirdaling!" while watching a penguin spinning). In Experiment 2, 24-month-olds (n = 48) were first exposed to a teaching phase in which they saw a new cartoon character on a television (e.g., a blue monster). Participants in the affirmative condition listened to sentences like "It's a bamoule!" and participants in the negative condition listened to sentences like "It's not a bamoule!." At test, all participants were asked to find the bamoule while viewing two images: the familiar character from the teaching phase versus a novel character (e.g., a red monster). Results showed that participants in the affirmative condition looked more to the familiar character (i.e., they learned the familiar character was a bamoule) than participants in the negative condition. Together, these studies provide the first evidence for the understanding of negative sentences during the second year of life. The ability to understand negative sentences so early might support language acquisition, providing infants with a tool to constrain the space of possibilities for word meanings.

Familiar words can serve as a semantic seed for syntactic bootstrapping.

Young children can exploit the syntactic context of a novel word to narrow down its probable meaning. But how do they learn which contexts are linked to which semantic features in the first place? We investigate if 3- to 4-year-old children (n = 60) can learn about a syntactic context from tracking its use with only a few familiar words. After watching a 5-min training video in which a novel function word (i.e., 'ko') replaced either personal pronouns or articles, children were able to infer semantic properties for novel words co-occurring with the newly learned function word (i.e., objects vs. actions). These findings implicate a mechanism by which a distributional analysis, associated with a small vocabulary of known words, could be sufficient to identify some properties associated with specific syntactic contexts.

Cognitive and pragmatic factors in language production: Evidence from source-goal motion events.

In order to talk about an event they see in the world, speakers have to build a conceptual representation of that event and generate a message that selects the pragmatically appropriate (e.g., informative) parts of that event that they want to talk about. To further understand the relationship between a speaker's conceptual representations and the pragmatic factors that influence message generation, this work investigates the extent to which different aspects of an event could be affected by pragmatic constraints. We focus specifically on source-goal motion events (e.g., a butterfly flying from a lamppost to a chair) because the conceptual structure of these events is well-understood, but the role that those representations play in message generation is yet unclear. In Experiment 1, we manipulated the pragmatic status of the source (e.g., the lamppost) - in particular, whether starting points of motion were or were not already known to an addressee. We found that sources were mentioned significantly more in the latter case, where they provided new, previously unknown information to the addressee. In Experiment 2, we investigated whether the same pragmatic factors could affect goals, or endpoints of motion events (e.g., the chair), in the same way; results showed that they could not. We conclude that conceptually peripheral elements of an event (i.e., sources) are more susceptible to communicative factors than those elements which are conceptually privileged (i.e., goals). We consider the implications of our findings for the relationship between event cognition and pragmatics and discuss how theories of event cognition can be integrated into current models of language production. We also discuss the implications of our work for open issues in the domain of event cognition.

Top-down grouping affects adjacent dependency learning.

A large body of research has demonstrated that humans attend to adjacent co-occurrence statistics when processing sequential information, and bottom-up prosodic information can influence learning. In this study, we investigated how top-down grouping cues can influence statistical learning. Specifically, we presented English sentences that were structurally equivalent to each other, which induced top-down expectations of grouping in the artificial language sequences that immediately followed. We show that adjacent dependencies in the artificial language are learnable when these entrained boundaries bracket the adjacent dependencies into the same sub-sequence, but are not learnable when the elements cross an induced boundary, even though that boundary is not present in the bottom-up sensory input. We argue that when there is top-down bracketing information in the learning sequence, statistical learning takes place for elements bracketed within sub-sequences rather than all the elements in the continuous sequence. This limits the amount of linguistic computations that need to be performed, providing a domain over which statistical learning can operate.

Easy Words: Reference Resolution in a Malevolent Referent World.

This article describes early stages in the acquisition of a first vocabulary by infants and young children. It distinguishes two major stages, the first of which operates by a stand-alone word-to-world pairing procedure and the second of which, using the evidence so acquired, builds a domain-specific syntax-sensitive structure-to-world pairing procedure. As we show, the first stage of learning is slow, restricted in character, and to some extent errorful, whereas the second procedure is determinative, rapid, and essentially errorless. Our central claim here is that the early, referentially based learning procedure succeeds at all because it is reined in by attention-focusing properties of word-to-world timing and related indicants of referential intent.

Motion verbs and memory for motion events.

Language is assumed to affect memory by offering an additional medium of encoding visual stimuli. Given that natural languages differ, cross-linguistic differences might impact memory processes. We investigate the role of motion verbs on memory for motion events in speakers of English, which preferentially encodes manner in motion verbs (e.g., driving), and Greek, which tends to encode path of motion in verbs (e.g., entering). Participants viewed a series of motion events and we later assessed their memory of the path and manner of the original events. There were no effects of language-specific biases on memory when participants watched events in silence; both English and Greek speakers remembered paths better than manners of motion. Moreover, even when motion verbs were available (either produced by or heard by the participants), they affected memory similarly regardless of the participants' language: path verbs attenuated memory for manners of motion, but the reverse did not occur. We conclude that overt language affects motion memory, but these effects interact with underlying, shared biases in how viewers represent motion events.

Spotting Dalmatians: Children's ability to discover subordinate-level word meanings cross-situationally.

Even when children encounter a novel word in the situation of a clear and unique referent, they are nevertheless faced with the problem of semantic uncertainty: when "puziv" refers to a co-present spotted dog, does the word mean Fido, Dalmatian, dog, animal, or entity? Here we explored the extent to which children (3-5 years of age) can reason about a novel word's meaning from information they have gathered cross-situationally, from a series of simple ostensive labeling events ("I see a puziv!"). Of particular interest were the conditions under which children arrive at a subordinate level meaning (e.g., Dalmatian) rather than a basic level meaning (e.g., dog). Experiment 1 showed that children (N = 32) were capable of using lexical contrast and/or mutual exclusivity cross-situationally, such that they arrived at subordinate level meanings only when the words being learned contrasted at the subordinate level, otherwise they strongly preferred basic level meanings (e.g., dog) even when the word had previously referred to subordinate level exemplars (always Dalmatians). Experiment 2 showed that some children in this same age range (N = 20) can also arrive at subordinate level meanings cross-situationally when offered relatively minimal linguistic support ("It's a kind of dog."). The findings are interpreted with respect to current theories of cross-situational word learning, and suggest that word meanings rather than sets of referential exemplars are tracked and used for cross-situational comparison.

Studying the Real-Time Interpretation of Novel Noun and Verb Meanings in Young Children.

Decades of research show that children rely on the linguistic context in which novel words occur to infer their meanings. However, because learning in these studies was assessed after children had heard numerous occurrences of a novel word in informative linguistic contexts, it is impossible to determine how much exposure would be needed for a child to learn from such information. This study investigated the speed with which French 20-month-olds and 3-to-4-year-olds exploit function words to determine the syntactic category of novel words and therefore infer their meanings. In a real-time preferential looking task, participants saw two videos side-by-side on a TV-screen: one showing a person performing a novel action, and the other a person passively holding a novel object. At the same time, participants heard only three occurrences of a novel word preceded either by a determiner (e.g., "Regarde! Une dase! - "Look! A dase!") or a pronoun (e.g., "Regarde! Elle dase!" - "Look! She's dasing!"). 3-to-4-year-olds exploited function words to categorize novel words and infer their meanings: they looked more to the novel action in the verb condition, while participants in the noun condition looked more to the novel object. 20-month-olds, however, did not show this difference. We discuss possible reasons for why 20-month-olds may have found it difficult to infer novel word meanings in our task. Given that 20-month-olds can use function words to learn word meanings in experiments providing many repetitions, we suspect that more repetitions might be needed to observe positive effects of learning in this age range in our task. Our study establishes nevertheless that before age 4, young children become able to exploit function words to infer the meanings of unknown words as soon as they occur. This ability to interpret speech in real-time and build interpretations about novel word meanings might be extremely useful for young children to map words to their possible referents and to boost their acquisition of word meanings.

Encoding of event roles from visual scenes is rapid, spontaneous, and interacts with higher-level visual processing.

A crucial component of event recognition is understanding event roles, i.e. who acted on whom: boy hitting girl is different from girl hitting boy. We often categorize Agents (i.e. the actor) and Patients (i.e. the one acted upon) from visual input, but do we rapidly and spontaneously encode such roles even when our attention is otherwise occupied? In three experiments, participants observed a continuous sequence of two-person scenes and had to search for a target actor in each (the male/female or red/blue-shirted actor) by indicating with a button press whether the target appeared on the left or the right. Critically, although role was orthogonal to gender and shirt color, and was never explicitly mentioned, participants responded more slowly when the target's role switched from trial to trial (e.g., the male went from being the Patient to the Agent). In a final experiment, we demonstrated that this effect cannot be fully explained by differences in posture associated with Agents and Patients. Our results suggest that extraction of event structure from visual scenes is rapid and spontaneous.

Auditory word recognition of verbs: Effects of verb argument structure on referent identification.

Word recognition includes the activation of a range of syntactic and semantic knowledge that is relevant to language interpretation and reference. Here we explored whether or not the number of arguments a verb takes impinges negatively on verb processing time. In this study, three experiments compared the dynamics of spoken word recognition for verbs with different preferred argument structure. Listeners' eye movements were recorded as they searched an array of pictures in response to hearing a verb. Results were similar in all the experiments. The time to identify the referent increased as a function of the number of arguments, above and beyond any effects of label appropriateness (and other controlled variables, such as letter, phoneme and syllable length, phonological neighborhood, oral and written lexical frequencies, imageability and rated age of acquisition). The findings indicate that the number of arguments a verb takes, influences referent identification during spoken word recognition. Representational complexity and amount of information generated by the lexical item that aids target identification are discussed as possible sources of this finding.

Neural Representations of Observed Actions Generalize across Static and Dynamic Visual Input.

People interact with entities in the environment in distinct and categorizable ways (e.g., kicking is making contact with foot). We can recognize these action categories across variations in actors, objects, and settings; moreover, we can recognize them from both dynamic and static visual input. However, the neural systems that support action recognition across these perceptual differences are unclear. Here, we used multivoxel pattern analysis of fMRI data to identify brain regions that support visual action categorization in a format-independent way. Human participants were scanned while viewing eight categories of interactions (e.g., pulling) depicted in two visual formats: (1) visually controlled videos of two interacting actors and (2) visually varied photographs selected from the internet involving different actors, objects, and settings. Action category was decodable across visual formats in bilateral inferior parietal, bilateral occipitotemporal, left premotor, and left middle frontal cortex. In most of these regions, the representational similarity of action categories was consistent across subjects and visual formats, a property that can contribute to a common understanding of actions among individuals. These results suggest that the identified brain regions support action category codes that are important for action recognition and action understanding.SIGNIFICANCE STATEMENT Humans tend to interpret the observed actions of others in terms of categories that are invariant to incidental features: whether a girl pushes a boy or a button and whether we see it in real-time or in a single snapshot, it is still pushing Here, we investigated the brain systems that facilitate the visual recognition of these action categories across such differences. Using fMRI, we identified several areas of parietal, occipitotemporal, and frontal cortex that exhibit action category codes that are similar across viewing of dynamic videos and still photographs. Our results provide strong evidence for the involvement of these brain regions in recognizing the way that people interact physically with objects and other people.