Crossmodal Correspondence Mediates Crossmodal Transfer from Visual to Auditory Stimuli in Category Learning.

This article investigated whether crossmodal correspondence, as a sensory translation phenomenon, can mediate crossmodal transfer from visual to auditory stimuli in category learning and whether multimodal category learning can influence the crossmodal correspondence between auditory and visual stimuli. Experiment 1 showed that the category knowledge acquired from elevation stimuli affected the categorization of pitch stimuli when there were robust crossmodal correspondence effects between elevation and size, indicating that crossmodal transfer occurred between elevation and pitch stimuli. Experiments 2 and 3 revealed that the size category knowledge could not be transferred to the categorization of pitches, but interestingly, size and pitch category learning determined the direction of the pitch-size correspondence, suggesting that the pitch-size correspondence was not stable and could be determined using multimodal category learning. Experiment 4 provided further evidence that there was no crossmodal transfer between size and pitch, due to the absence of a robust pitch-size correspondence. These results demonstrated that crossmodal transfer can occur between audio-visual stimuli with crossmodal correspondence, and multisensory category learning can change the corresponding relationship between audio-visual stimuli. These findings suggest that crossmodal transfer and crossmodal correspondence share similar abstract representations, which can be mediated by semantic content such as category labels.

Exploration, Distributed Attention, and Development of Category Learning.

Category learning is a crucial aspect of cognition that involves organizing entities into equivalence classes. Whereas adults tend to focus on category-relevant features, young children often distribute attention between relevant and irrelevant ones. The reasons for children's distributed attention are not fully understood. In two category-learning experiments with adults and with children aged 4, 5, and 6 (N = 201), we examined potential drivers of distributed attention, including (a) immature filtering of distractors and (b) the general tendency for exploration or broad information sampling. By eliminating distractor competition, we reduced filtering demands. Despite identifying the features critical for accurate categorization, children, regardless of their categorization performance, continued sampling more information than was necessary. These results indicate that the tendency to sample information extensively contributes to distributed attention in young children. We identify candidate drivers of this tendency that need to be examined in future research.

Exploring the effectiveness of reward-based learning strategies for second-language speech sounds.

Adults struggle to learn non-native speech categories in many experimental settings (Goto, Neuropsychologia, 9(3), 317-323 1971), but learn efficiently in a video game paradigm where non-native speech sounds have functional significance (Lim & Holt, Cognitive Science, 35(7), 1390-1405 2011). Behavioral and neural evidence from this and other paradigms point toward the involvement of reinforcement learning mechanisms in speech category learning (Harmon, Idemaru, & Kapatsinski, Cognition, 189, 76-88 2019; Lim, Fiez, & Holt, Proceedings of the National Academy of Sciences, 116, 201811992 2019). We formalize this hypothesis computationally and implement a deep reinforcement learning network to map between environmental input and actions. Comparing to a supervised model of learning, we show that the reinforcement network closely matches aspects of human behavior in two experiments - learning of synthesized auditory noise tokens and improvement in speech sound discrimination. Both models perform comparably and the similarity in the output of each model leads us to believe that there is little inherent computational benefit to a reward-based learning mechanism. We suggest that the specific neural circuitry engaged by the paradigm and links between striatum and superior temporal areas play a critical role in effective learning.

Feedback and observational learning differ in effectiveness during category learning in early school aged children and adults.

When learning new categories, do children benefit from the same types of training as adults? We compared the effects of feedback-based training with observational training in young adults (ages 18-25) and early school aged children (ages 6-7) across two different multimodal category learning tasks: conjunctive rule based and information integration. We used multimodal stimuli that varied across a visual feature (rotation speed of the "planet" stimulus) and an auditory feature (pitch frequency of a pure tone stimulus). We found an interaction between age and training type for the rule-based category task, such that adults performed better in feedback training than in observational training, whereas training type had no significant effect on children's category learning performance. Overall adults performed better than children in learning both the rule based and information integration category structures. In information integration category learning, feedback versus observational training did not have a significant effect on either adults' or children's category learning. Computational modelling revealed that children defaulted to univariate rules in both tasks. The finding that children do not benefit from feedback training and can learn successfully via observational learning has implications for the design of educational interventions appropriate for children.

Dissociable Roles of the Dorsolateral and Ventromedial Prefrontal Cortex in Human Categorization.

Models of human categorization predict the prefrontal cortex (PFC) serves a central role in category learning. The dorsolateral prefrontal cortex (dlPFC) and ventromedial prefrontal cortex (vmPFC) have been implicated in categorization; however, it is unclear whether both are critical for categorization and whether they support unique functions. We administered three categorization tasks to patients with PFC lesions (mean age, 69.6 years; 5 men, 5 women) to examine how the prefrontal subregions contribute to categorization. These included a rule-based (RB) task that was solved via a unidimensional rule, an information integration (II) task that was solved by combining information from two stimulus dimensions, and a deterministic/probabilistic (DP) task with stimulus features that had varying amounts of category-predictive information. Compared with healthy comparison participants, both patient groups had impaired performance. Impairments in the dlPFC patients were largest during the RB task, whereas impairments in the vmPFC patients were largest during the DP task. A hierarchical model was fit to the participants' data to assess learning deficits in the patient groups. PFC damage was correlated with a regularization term that limited updates to attention after each trial. Our results suggest that the PFC, as a whole, is important for learning to orient attention to relevant stimulus information. The dlPFC may be especially important for rule-based learning, whereas the vmPFC may be important for focusing attention on deterministic (highly diagnostic) features and ignoring less predictive features. These results support overarching functions of the dlPFC in executive functioning and the vmPFC in value-based decision-making.

Task imprinting: Another mechanism of representational change?

Research from several areas suggests that mental representations adapt to the specific tasks we carry out in our environment. In this study, we propose a mechanism of adaptive representational change, task imprinting. Thereby, we introduce a computational model, which portrays task imprinting as an adaptation to specific task goals via selective storage of helpful representations in long-term memory. We test the main qualitative prediction of the model in four behavioral experiments using healthy young adults as participants. In each experiment, we assess participants' baseline representations in the beginning of the experiment, then expose participants to one of two tasks intended to shape representations differently according to our model, and finally assess any potential change in representations. Crucially, the tasks used to measure representations differ in the amount that strategic, judgmental processes play a role. The results of Experiments 1 and 2 allow us to exclude the option that representations used in more perceptual tasks become biased categorically. The results of Experiment 4 make it likely that people strategically decide given the specific task context whether they use categorical information or not. One signature of representational change was however observed: category learning practice increased the perceptual sensitivity over and above mere exposure to the same stimuli.

The effect of feedback timing on category learning and feedback processing in younger and older adults.

Introduction: Corrective feedback can be received immediately after an action or with a temporal delay. Neuroimaging studies suggest that immediate and delayed feedback are processed by the striatum and medial temporal lobes (MTL), respectively. Age-related changes in the striatum and MTL may influence the efficiency of feedback-based learning in older adults. The current study leverages event-related potentials (ERPs) to evaluate age-related differences in immediate and delayed feedback processing and consequences for learning. The feedback-related negativity (FRN) captures activity in the frontostriatal circuit while the N170 is hypothesized to reflect MTL activation. Methods: 18 younger (Myears = 24.4) and 20 older (Myears = 65.5) adults completed learning tasks with immediate and delayed feedback. For each group, learning outcomes and ERP magnitudes were evaluated across timing conditions. Results: Younger adults learned better than older adults in the immediate timing condition. This performance difference was associated with a typical FRN signature in younger but not older adults. For older adults, impaired processing of immediate feedback in the striatum may have negatively impacted learning. Conversely, learning was comparable across groups when feedback was delayed. For both groups, delayed feedback was associated with a larger magnitude N170 relative to immediate feedback, suggesting greater MTL activation. Discussion and conclusion: Delaying feedback may increase MTL involvement and, for older adults, improve category learning. Age-related neural changes may differentially affect MTL- and striatal-dependent learning. Future research can evaluate the locus of age-related learning differences and how feedback can be manipulated to optimize learning across the lifespan.

Prototype or Exemplar Representations in the 5/5 Category Learning Task.

Theories of category learning have typically focused on how the underlying category structure affects the category representations acquired by learners. However, there is limited research as to how other factors affect what representations are learned and utilized and how representations might change across the time course of learning. We used a novel "5/5" categorization task developed from the well-studied 5/4 task with the addition of one more stimulus to clarify an ambiguity in the 5/4 prototypes. We used multiple methods including computational modeling to identify whether participants categorized on the basis of exemplar or prototype representations. We found that, overall, for the stimuli we used (schematic robot-like stimuli), learning was best characterized by the use of prototypes. Most importantly, we found that relative use of prototype and exemplar strategies changed across learning, with use of exemplar representations decreasing and prototype representations increasing across blocks.

Discrimination sensitivity of visual shapes sharpens in autistic adults but only after explicit category learning.

BACKGROUND: Categorization and its influence on perceptual discrimination are essential processes to organize information efficiently. Individuals with Autism Spectrum Condition (ASC) are suggested to display enhanced discrimination on the one hand, but also to experience difficulties with generalization and ignoring irrelevant differences on the other, which underlie categorization. Studies on categorization and discrimination in ASC have mainly focused on one process at a time, however, and typically only used either behavioral or neural measures in isolation. Here, we aim to investigate the interrelationships between these perceptual processes using novel stimuli sampled from a well-controlled artificial stimulus space. In addition, we complement standard behavioral psychophysical tasks with frequency-tagging EEG (FT-EEG) to obtain a direct, non-task related neural index of discrimination and categorization. METHODS: The study was completed by 38 adults with ASC and 38 matched neurotypical (NT) individuals. First, we assessed baseline discrimination sensitivity by administering FT-EEG measures and a complementary behavioral task. Second, participants were trained to categorize the stimuli into two groups. Finally, participants again completed the neural and behavioral discrimination sensitivity measures. RESULTS: Before training, NT participants immediately revealed a categorical tuning of discrimination, unlike ASC participants who showed largely similar discrimination sensitivity across the stimuli. During training, both autistic and non-autistic participants were able to categorize the stimuli into two groups. However, in the initial training phase, ASC participants were less accurate and showed more variability, as compared to their non-autistic peers. After training, ASC participants showed significantly enhanced neural and behavioral discrimination sensitivity across the category boundary. Behavioral indices of a reduced categorical processing and perception were related to the presence of more severe autistic traits. Bayesian analyses confirmed overall results. LIMITATIONS: Data-collection occurred during the COVID-19 pandemic. CONCLUSIONS: Our behavioral and neural findings indicate that adults with and without ASC are able to categorize highly similar stimuli. However, while categorical tuning of discrimination sensitivity was spontaneously present in the NT group, it only emerged in the autistic group after explicit categorization training. Additionally, during training, adults with autism were slower at category learning. Finally, this multi-level approach sheds light on the mechanisms underlying sensory and information processing issues in ASC.

The tail of the caudate is sensitive to both gain and loss feedback during information integration categorization.

Although most category learning studies use feedback for training, little attention has been paid to how individuals utilize feedback implemented as gains or losses during categorization. We compared skilled categorization under three different conditions: Gain (earn points for correct answers), Gain and Loss (earn points for correct answers and lose points for wrong answers) and Correct or Wrong (accuracy feedback only). We also manipulated difficulty and point value, with near boundary stimuli having the highest number of points to win or lose, and stimuli far from the boundary having the lowest point value. We found that the tail of the caudate was sensitive to feedback condition, with highest activity when both Gain and Loss feedback were present and least activity when only Gain or accuracy feedback was present. We also found that activity across the caudate was affected by distance from the decision bound, with greatest activity for the near boundary high value stimuli, and lowest for far low value stimuli. Overall these results indicate that the tail of the caudate is sensitive not only to positive rewards but also to loss and punishment, consistent with recent animal research finding tail of the caudate activity in aversive learning.

Reconciling category exceptions through representational shifts.

Real-world categories often contain exceptions that disobey the perceptual regularities followed by other members. Prominent psychological and neurobiological theories indicate that exception learning relies on the flexible modulation of object representations, but the specific representational shifts key to learning remain poorly understood. Here, we leveraged behavioral and computational approaches to elucidate the representational dynamics during the acquisition of exceptions that violate established regularity knowledge. In our study, participants (n = 42) learned novel categories in which regular and exceptional items were introduced successively; we then fitted a computational model to individuals' categorization performance to infer latent stimulus representations before and after exception learning. We found that in the representational space, exception learning not only drove confusable exceptions to be differentiated from regular items, but also led exceptions within the same category to be integrated based on shared characteristics. These shifts resulted in distinct representational clusters of regular items and exceptions that constituted hierarchically structured category representations, and the distinct clustering of exceptions from regular items was associated with a high ability to generalize and reconcile knowledge of regularities and exceptions. Moreover, by having a second group of participants (n = 42) to judge stimuli's similarity before and after exception learning, we revealed misalignment between representational similarity and behavioral similarity judgments, which further highlights the hierarchical layouts of categories with regularities and exceptions. Altogether, our findings elucidate the representational dynamics giving rise to generalizable category structures that reconcile perceptually inconsistent category members, thereby advancing the understanding of knowledge formation.

Memory Reactivation during Sleep Does Not Act Holistically on Object Memory.

Memory reactivation during sleep is thought to facilitate memory consolidation. Most sleep reactivation research has examined how reactivation of specific facts, objects, and associations benefits their overall retention. However, our memories are not unitary, and not all features of a memory persist in tandem over time. Instead, our memories are transformed, with some features strengthened and others weakened. Does sleep reactivation drive memory transformation? We leveraged the Targeted Memory Reactivation technique in an object category learning paradigm to examine this question. Participants (20 female, 14 male) learned three categories of novel objects, where each object had unique, distinguishing features as well as features shared with other members of its category. We used a real-time EEG protocol to cue the reactivation of these objects during sleep at moments optimized to generate reactivation events. We found that reactivation improved memory for distinguishing features while worsening memory for shared features, suggesting a differentiation process. The results indicate that sleep reactivation does not act holistically on object memories, instead supporting a transformation where some features are enhanced over others.

Brief category learning distorts perceptual space for complex scenes.

The formation of categories is known to distort perceptual space: representations are pushed away from category boundaries and pulled toward categorical prototypes. This phenomenon has been studied with artificially constructed objects, whose feature dimensions are easily defined and manipulated. How such category-induced perceptual distortions arise for complex, real-world scenes, however, remains largely unknown due to the technical challenge of measuring and controlling scene features. We address this question by generating realistic scene images from a high-dimensional continuous space using generative adversarial networks and using the images as stimuli in a novel learning task. Participants learned to categorize the scene images along arbitrary category boundaries and later reconstructed the same scenes from memory. Systematic biases in reconstruction errors closely tracked each participant's subjective category boundaries. These findings suggest that the perception of global scene properties is warped to align with a newly learned category structure after only a brief learning experience.

Reconciling categorization and memory via environmental statistics.

How people represent categories and how those representations change over time is a basic question about human cognition. Previous research has demonstrated that people categorize objects by comparing them to category prototypes in early stages of learning but consider the individual exemplars within each category in later stages. However, these results do not seem consistent with findings in the memory literature showing that it becomes increasingly easier to access representations of general knowledge than representations of specific items over time. Why would one rely more on exemplar-based representations in later stages of categorization when it is more difficult to access these exemplars in memory? To reconcile these incongruities, our study proposed that previous findings on categorization are a result of human participants adapting to a specific experimental environment, in which the probability of encountering an object stays uniform over time. In a more realistic environment, however, one would be less likely to encounter the same object if a long time has passed. Confirming our hypothesis, we demonstrated that under environmental statistics identical to typical categorization experiments the advantage of exemplar-based categorization over prototype-based categorization increases over time, replicating previous research in categorization. In contrast, under realistic environmental statistics simulated by our experiments the advantage of exemplar-based categorization over prototype-based categorization decreases over time. A second set of experiments replicated our results, while additionally demonstrating that human categorization is sensitive to the category structure presented to the participants. These results provide converging evidence that human categorization adapts appropriately to environmental statistics.

Bayesian Semiparametric Longitudinal Inverse-Probit Mixed Models for Category Learning.

Understanding how the adult human brain learns novel categories is an important problem in neuroscience. Drift-diffusion models are popular in such contexts for their ability to mimic the underlying neural mechanisms. One such model for gradual longitudinal learning was recently developed in Paulon et al. (J Am Stat Assoc 116:1114-1127, 2021). In practice, category response accuracies are often the only reliable measure recorded by behavioral scientists to describe human learning. Category response accuracies are, however, often the only reliable measure recorded by behavioral scientists to describe human learning. To our knowledge, however, drift-diffusion models for such scenarios have never been considered in the literature before. To address this gap, in this article, we build carefully on Paulon et al. (J Am Stat Assoc 116:1114-1127, 2021), but now with latent response times integrated out, to derive a novel biologically interpretable class of 'inverse-probit' categorical probability models for observed categories alone. However, this new marginal model presents significant identifiability and inferential challenges not encountered originally for the joint model in Paulon et al. (J Am Stat Assoc 116:1114-1127, 2021). We address these new challenges using a novel projection-based approach with a symmetry-preserving identifiability constraint that allows us to work with conjugate priors in an unconstrained space. We adapt the model for group and individual-level inference in longitudinal settings. Building again on the model's latent variable representation, we design an efficient Markov chain Monte Carlo algorithm for posterior computation. We evaluate the empirical performance of the method through simulation experiments. The practical efficacy of the method is illustrated in applications to longitudinal tone learning studies.

Post-training flexibility in category learning.

Exemplar models of categorization, which assume that people make classification decisions based on item information stored in memory, typically assume that all of the exemplars are available and inform decision-making. However, in this study, we hypothesized that people may selectively emphasize subsets of exemplars, giving rise to individual differences in categorization. To verify this hypothesis, we adopted the partial-XOR category structure in Conaway and Kurtz (Psychonomic Bulletin & Review, 24, 1312-1323 2017), which has been evident to be able to induce two major response patterns in the transfer phase: the Proximity and XOR patterns. "Experiment 1" confirmed that these two patterns could be generated if participants were trained with only the exemplars of one category or the other. In "Experiment 2", participants were asked to not only learn the category labels of all exemplars but also memorize the exemplars of only Category A (Condition A), only Category B (Condition B), or two categories (Condition AB) for a recognition test after the training phase of the categorization task. As expected, in the transfer phase, the participants tended to perform the XOR and Proximity patterns, when the exemplars of Category A and Category B were respectively targeted for the recognition test. The parameters of the SDGCM estimated by Bayesian inference for modeling the data of "Experiment 2" showed that the exemplar accessibility of Category A was larger than that of Category B for performing the XOR pattern and vice versa for performing the proximity pattern, hence verifying our hypothesis.

Mechanisms underlying category learning in the human ventral occipito-temporal cortex.

The human ventral occipito-temporal cortex (VOTC) has evolved into specialized regions that process specific categories, such as words, tools, and animals. The formation of these areas is driven by bottom-up visual and top-down nonvisual experiences. However, the specific mechanisms through which top-down nonvisual experiences modulate category-specific regions in the VOTC are still unknown. To address this question, we conducted a study in which participants were trained for approximately 13 h to associate three sets of novel meaningless figures with different top-down nonvisual features: the wordlike category with word features, the non-wordlike category with nonword features, and the visual familiarity condition with no nonvisual features. Pre- and post-training functional MRI (fMRI) experiments were used to measure brain activity during stimulus presentation. Our results revealed that training induced a categorical preference for the two training categories within the VOTC. Moreover, the locations of two training category-specific regions exhibited a notable overlap. Remarkably, within the overlapping category-specific region, training resulted in a dissociation in activation intensity and pattern between the two training categories. These findings provide important insights into how different nonvisual categorical information is encoded in the human VOTC.

Causal Influence of Linguistic Learning on Perceptual and Conceptual Processing: A Brain-Constrained Deep Neural Network Study of Proper Names and Category Terms.

Language influences cognitive and conceptual processing, but the mechanisms through which such causal effects are realized in the human brain remain unknown. Here, we use a brain-constrained deep neural network model of category formation and symbol learning and analyze the emergent model's internal mechanisms at the neural circuit level. In one set of simulations, the network was presented with similar patterns of neural activity indexing instances of objects and actions belonging to the same categories. Biologically realistic Hebbian learning led to the formation of instance-specific neurons distributed across multiple areas of the network, and, in addition, to cell assembly circuits of "shared" neurons responding to all category instances-the network correlates of conceptual categories. In two separate sets of simulations, the network learned the same patterns together with symbols for individual instances ["proper names" (PN)] or symbols related to classes of instances sharing common features ["category terms" (CT)]. Learning CT remarkably increased the number of shared neurons in the network, thereby making category representations more robust while reducing the number of neurons of instance-specific ones. In contrast, proper name learning prevented a substantial reduction of instance-specific neurons and blocked the overgrowth of category general cells. Representational similarity analysis further confirmed that the neural activity patterns of category instances became more similar to each other after category-term learning, relative to both learning with PN and without any symbols. These network-based mechanisms for concepts, PN, and CT explain why and how symbol learning changes object perception and memory, as revealed by experimental studies.

Modeling Similarity and Psychological Space.

Similarity and categorization are fundamental processes in human cognition that help complex organisms make sense of the cacophony of information in their environment. These processes are critical for tasks such as recognizing objects, making decisions, and forming memories. In this review, we provide an overview of the current state of knowledge on similarity and psychological spaces, discussing the theories, methods, and empirical findings that have been generated over the years. Although the concept of similarity has important limitations, it plays a key role in cognitive modeling. The review surfaces three key themes. First, similarity and mental representations are merely two sides of the same coin, existing as a similarity-representation duality that defines a psychological space. Second, both the brain's mental representations and the study of mental representations are made possible by exploiting second-order isomorphism. Third, similarity analysis has near-universal applicability across all levels of cognition, providing a common research language.

Category Learning Selectively Enhances Representations of Boundary-Adjacent Exemplars in Early Visual Cortex.

Category learning and visual perception are fundamentally interactive processes, such that successful categorization often depends on the ability to make fine visual discriminations between stimuli that vary on continuously valued dimensions. Research suggests that category learning can improve perceptual discrimination along the stimulus dimensions that predict category membership and that these perceptual enhancements are a byproduct of functional plasticity in the visual system. However, the precise mechanisms underlying learning-dependent sensory modulation in categorization are not well understood. We hypothesized that category learning leads to a representational sharpening of underlying sensory populations tuned to values at or near the category boundary. Furthermore, such sharpening should occur largely during active learning of new categories. These hypotheses were tested using fMRI and a theoretically constrained model of vision to quantify changes in the shape of orientation representations while human adult subjects learned to categorize physically identical stimuli based on either an orientation rule (N = 12) or an orthogonal spatial frequency rule (N = 13). Consistent with our predictions, modeling results revealed relatively enhanced reconstructed representations of stimulus orientation in visual cortex (V1-V3) only for orientation rule learners. Moreover, these reconstructed representations varied as a function of distance from the category boundary, such that representations for challenging stimuli near the boundary were significantly sharper than those for stimuli at the category centers. These results support an efficient model of plasticity wherein only the sensory populations tuned to the most behaviorally relevant regions of feature space are enhanced during category learning.