Systematic Human Learning and Generalization From a Brief Tutorial With Explanatory Feedback.

We investigate human adults' ability to learn an abstract reasoning task quickly and to generalize outside of the range of training examples. Using a task based on a solution strategy in Sudoku, we provide Sudoku-naive participants with a brief instructional tutorial with explanatory feedback using a narrow range of training examples. We find that most participants who master the task do so within 10 practice trials and generalize well to puzzles outside of the training range. We also find that most of those who master the task can describe a valid solution strategy, and such participants perform better on transfer puzzles than those whose strategy descriptions are vague or incomplete. Interestingly, fewer than half of our human participants were successful in acquiring a valid solution strategy, and this ability was associated with completion of high school algebra and geometry. We consider the implications of these findings for understanding human systematic reasoning, as well as the challenges these findings pose for building computational models that capture all aspects of our findings, and we point toward a role for learning from instructions and explanations to support rapid learning and generalization.

Capturing advanced human cognitive abilities with deep neural networks.

How can artificial neural networks capture the advanced cognitive abilities of pioneering scientists? I suggest they must learn to exploit human-invented tools of thought and human-like ways of using them, and must engage in explicit goal-directed problem solving as exemplified in the activities of scientists and mathematicians and taught in advanced educational settings.

A weighted constraint satisfaction approach to human goal-directed decision making.

When we plan for long-range goals, proximal information cannot be exploited in a blindly myopic way, as relevant future information must also be considered. But when a subgoal must be resolved first, irrelevant future information should not interfere with the processing of more proximal, subgoal-relevant information. We explore the idea that decision making in both situations relies on the flexible modulation of the degree to which different pieces of information under consideration are weighted, rather than explicitly decomposing a problem into smaller parts and solving each part independently. We asked participants to find the shortest goal-reaching paths in mazes and modeled their initial path choices as a noisy, weighted information integration process. In a base task where choosing the optimal initial path required weighting starting-point and goal-proximal factors equally, participants did take both constraints into account, with participants who made more accurate choices tending to exhibit more balanced weighting. The base task was then embedded as an initial subtask in a larger maze, where the same two factors constrained the optimal path to a subgoal, and the final goal position was irrelevant to the initial path choice. In this more complex task, participants' choices reflected predominant consideration of the subgoal-relevant constraints, but also some influence of the initially-irrelevant final goal. More accurate participants placed much less weight on the optimality-irrelevant goal and again tended to weight the two initially-relevant constraints more equally. These findings suggest that humans may rely on a graded, task-sensitive weighting of multiple constraints to generate approximately optimal decision outcomes in both hierarchical and non-hierarchical goal-directed tasks.

Do estimates of numerosity really adhere to Weber's law? A reexamination of two case studies.

Both humans and nonhuman animals can exhibit sensitivity to the approximate number of items in a visual array or events in a sequence, and across various paradigms, uncertainty in numerosity judgments increases with the number estimated or produced. The pattern of increase is usually described as exhibiting approximate adherence to Weber's law, such that uncertainty increases proportionally to the mean estimate, resulting in a constant coefficient of variation. Such a pattern has been proposed to be a signature characteristic of an innate "number sense." We reexamine published behavioral data from two studies that have been cited as prototypical evidence of adherence to Weber's law and observe that in both cases variability increases less than this account would predict, as indicated by a decreasing coefficient of variation with an increase in number. We also consider evidence from numerosity discrimination studies that show deviations from the constant coefficient of variation pattern. Though behavioral data can sometimes exhibit approximate adherence to Weber's law, our findings suggest that such adherence is not a fixed characteristic of the mechanisms whereby humans and animals estimate numerosity. We suggest instead that the observed pattern of increase in variability with number depends on the circumstances of the task and stimuli, and reflects an adaptive ensemble of mechanisms composed to optimize performance under these circumstances.

Transforming task representations to perform novel tasks.

An important aspect of intelligence is the ability to adapt to a novel task without any direct experience (zero shot), based on its relationship to previous tasks. Humans can exhibit this cognitive flexibility. By contrast, models that achieve superhuman performance in specific tasks often fail to adapt to even slight task alterations. To address this, we propose a general computational framework for adapting to novel tasks based on their relationship to prior tasks. We begin by learning vector representations of tasks. To adapt to new tasks, we propose metamappings, higher-order tasks that transform basic task representations. We demonstrate the effectiveness of this framework across a wide variety of tasks and computational paradigms, ranging from regression to image classification and reinforcement learning. We compare to both human adaptability and language-based approaches to zero-shot learning. Across these domains, metamapping is successful, often achieving 80 to 90% performance, without any data, on a novel task, even when the new task directly contradicts prior experience. We further show that metamapping can not only generalize to new tasks via learned relationships, but can also generalize using novel relationships unseen during training. Finally, using metamapping as a starting point can dramatically accelerate later learning on a new task and reduce learning time and cumulative error substantially. Our results provide insight into a possible computational basis of intelligent adaptability and offer a possible framework for modeling cognitive flexibility and building more flexible artificial intelligence systems.

Placing language in an integrated understanding system: Next steps toward human-level performance in neural language models.

Language is crucial for human intelligence, but what exactly is its role? We take language to be a part of a system for understanding and communicating about situations. In humans, these abilities emerge gradually from experience and depend on domain-general principles of biological neural networks: connection-based learning, distributed representation, and context-sensitive, mutual constraint satisfaction-based processing. Current artificial language processing systems rely on the same domain general principles, embodied in artificial neural networks. Indeed, recent progress in this field depends on query-based attention, which extends the ability of these systems to exploit context and has contributed to remarkable breakthroughs. Nevertheless, most current models focus exclusively on language-internal tasks, limiting their ability to perform tasks that depend on understanding situations. These systems also lack memory for the contents of prior situations outside of a fixed contextual span. We describe the organization of the brain's distributed understanding system, which includes a fast learning system that addresses the memory problem. We sketch a framework for future models of understanding drawing equally on cognitive neuroscience and artificial intelligence and exploiting query-based attention. We highlight relevant current directions and consider further developments needed to fully capture human-level language understanding in a computational system.

Integration of new information in memory: new insights from a complementary learning systems perspective.

According to complementary learning systems theory, integrating new memories into the neocortex of the brain without interfering with what is already known depends on a gradual learning process, interleaving new items with previously learned items. However, empirical studies show that information consistent with prior knowledge can sometimes be integrated very quickly. We use artificial neural networks with properties like those we attribute to the neocortex to develop an understanding of the role of consistency with prior knowledge in putatively neocortex-like learning systems, providing new insights into when integration will be fast or slow and how integration might be made more efficient when the items to be learned are hierarchically structured. The work relies on deep linear networks that capture the qualitative aspects of the learning dynamics of the more complex nonlinear networks used in previous work. The time course of learning in these networks can be linked to the hierarchical structure in the training data, captured mathematically as a set of dimensions that correspond to the branches in the hierarchy. In this context, a new item to be learned can be characterized as having aspects that project onto previously known dimensions, and others that require adding a new branch/dimension. The projection onto the known dimensions can be learned rapidly without interleaving, but learning the new dimension requires gradual interleaved learning. When a new item only overlaps with items within one branch of a hierarchy, interleaving can focus on the previously known items within this branch, resulting in faster integration with less interleaving overall. The discussion considers how the brain might exploit these facts to make learning more efficient and highlights predictions about what aspects of new information might be hard or easy to learn. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.

Intrusions into the shadow of attention: A new take on illusory conjunctions.

We present new evidence about illusory conjunctions (ICs) suggesting that their current explanation requires revision. According to Feature Integration Theory (FIT; Treisman & Gelade Cognitive Psychology, 12, 97-136, 1980), focal attention to a single stimulus is required to bind its features into an integrated percept. FIT predicts that if attention is spread over multiple stimuli, features of these different stimuli can be combined into a single percept and produce ICs. Treisman and Schmidt (Cognitive Psychology, 14, 107-141, 1982) and Cohen & Ivry (Journal of Experimental Psychology: Human Perception and Performance, 15(4), 650-663, 1989) supported this prediction. In the latter study, participants viewed brief displays containing two digits and two colored letters. Digit locations were pre-cued, and participants were instructed to prioritize the digits and to spread their attention across the region encompassed by the digits. Cohen & Ivry found that reports of one letter (the 'target') produced ICs when both letters appeared between the digits. Expanding on Cohen & Ivry's paradigm, we find that both letters do not need to appear between the digits to produce ICs. While the target letter was highly susceptible to ICs if the target appeared inside the position of a nearby digit, the position of the other letter was largely irrelevant. Our experimental results also argue that these ICs were not due to mnemonic errors occurring while the digits are being reported. Based on our findings, we propose that attention to the digits casts an attentional 'shadow' projecting towards fixation, interfering with processing of target letters in that shadow and allowing color information from elsewhere in the display to be included in the resulting percept.

Numerosity discrimination in deep neural networks: Initial competence, developmental refinement and experience statistics.

Both humans and non-human animals exhibit sensitivity to the approximate number of items in a visual array, as indexed by their performance in numerosity discrimination tasks, and even neonates can detect changes in numerosity. These findings are often interpreted as evidence for an innate 'number sense'. However, recent simulation work has challenged this view by showing that human-like sensitivity to numerosity can emerge in deep neural networks that build an internal model of the sensory data. This emergentist perspective posits a central role for experience in shaping our number sense and might explain why numerical acuity progressively increases over the course of development. Here we substantiate this hypothesis by introducing a progressive unsupervised deep learning algorithm, which allows us to model the development of numerical acuity through experience. We also investigate how the statistical distribution of numerical and non-numerical features in natural environments affects the emergence of numerosity representations in the computational model. Our simulations show that deep networks can exhibit numerosity sensitivity prior to any training, as well as a progressive developmental refinement that is modulated by the statistical structure of the learning environment. To validate our simulations, we offer a refinement to the quantitative characterization of the developmental patterns observed in human children. Overall, our findings suggest that it may not be necessary to assume that animals are endowed with a dedicated system for processing numerosity, since domain-general learning mechanisms can capture key characteristics others have attributed to an evolutionarily specialized number system.

Value-based decision making: An interactive activation perspective.

Prominent theories of value-based decision making have assumed that choices are made via the maximization of some objective function (e.g., expected value) and that the process of decision making is serial and unfolds across modular subprocesses (e.g., perception, valuation, and action selection). However, the influence of a large number of contextual variables that are not related to expected value in any direct way and the ubiquitous reciprocity among variables thought to belong to different subprocesses suggest that these assumptions may not always hold. Here, we propose an interactive activation framework for value-based decision making that does not assume that objective function maximization is the only consideration affecting choice or that processing is modular or serial. Our framework holds that processing takes place via the interactive propagation of activation in a set of simple, interconnected processing elements. We use our framework to simulate a broad range of well-known empirical phenomena-primarily focusing on decision contexts that feature nonoptimal decision making and/or interactive (i.e., not serial or modular) processing. Our approach is constrained at Marr's (1982) algorithmic and implementational levels rather than focusing strictly on considerations of optimality at the computational theory level. It invites consideration of the possibility that choice is emergent and that its computation is distributed. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Quasi-compositional mapping from form to meaning: a neural network-based approach to capturing neural responses during human language comprehension.

We argue that natural language can be usefully described as quasi-compositional and we suggest that deep learning-based neural language models bear long-term promise to capture how language conveys meaning. We also note that a successful account of human language processing should explain both the outcome of the comprehension process and the continuous internal processes underlying this performance. These points motivate our discussion of a neural network model of sentence comprehension, the Sentence Gestalt model, which we have used to account for the N400 component of the event-related brain potential (ERP), which tracks meaning processing as it happens in real time. The model, which shares features with recent deep learning-based language models, simulates N400 amplitude as the automatic update of a probabilistic representation of the situation or event described by the sentence, corresponding to a temporal difference learning signal at the level of meaning. We suggest that this process happens relatively automatically, and that sometimes a more-controlled attention-dependent process is necessary for successful comprehension, which may be reflected in the subsequent P600 ERP component. We relate this account to current deep learning models as well as classic linguistic theory, and use it to illustrate a domain general perspective on some specific linguistic operations postulated based on compositional analyses of natural language. This article is part of the theme issue 'Towards mechanistic models of meaning composition'.

A mathematical theory of semantic development in deep neural networks.

An extensive body of empirical research has revealed remarkable regularities in the acquisition, organization, deployment, and neural representation of human semantic knowledge, thereby raising a fundamental conceptual question: What are the theoretical principles governing the ability of neural networks to acquire, organize, and deploy abstract knowledge by integrating across many individual experiences? We address this question by mathematically analyzing the nonlinear dynamics of learning in deep linear networks. We find exact solutions to this learning dynamics that yield a conceptual explanation for the prevalence of many disparate phenomena in semantic cognition, including the hierarchical differentiation of concepts through rapid developmental transitions, the ubiquity of semantic illusions between such transitions, the emergence of item typicality and category coherence as factors controlling the speed of semantic processing, changing patterns of inductive projection over development, and the conservation of semantic similarity in neural representations across species. Thus, surprisingly, our simple neural model qualitatively recapitulates many diverse regularities underlying semantic development, while providing analytic insight into how the statistical structure of an environment can interact with nonlinear deep-learning dynamics to give rise to these regularities.

Concepts, control, and context: A connectionist account of normal and disordered semantic cognition.

Semantic cognition requires conceptual representations shaped by verbal and nonverbal experience and executive control processes that regulate activation of knowledge to meet current situational demands. A complete model must also account for the representation of concrete and abstract words, of taxonomic and associative relationships, and for the role of context in shaping meaning. We present the first major attempt to assimilate all of these elements within a unified, implemented computational framework. Our model combines a hub-and-spoke architecture with a buffer that allows its state to be influenced by prior context. This hybrid structure integrates the view, from cognitive neuroscience, that concepts are grounded in sensory-motor representation with the view, from computational linguistics, that knowledge is shaped by patterns of lexical co-occurrence. The model successfully codes knowledge for abstract and concrete words, associative and taxonomic relationships, and the multiple meanings of homonyms, within a single representational space. Knowledge of abstract words is acquired through (a) their patterns of co-occurrence with other words and (b) acquired embodiment, whereby they become indirectly associated with the perceptual features of co-occurring concrete words. The model accounts for executive influences on semantics by including a controlled retrieval mechanism that provides top-down input to amplify weak semantic relationships. The representational and control elements of the model can be damaged independently, and the consequences of such damage closely replicate effects seen in neuropsychological patients with loss of semantic representation versus control processes. Thus, the model provides a wide-ranging and neurally plausible account of normal and impaired semantic cognition. (PsycINFO Database Record

Modelling the N400 brain potential as change in a probabilistic representation of meaning.

The N400 component of the event-related brain potential has aroused much interest because it is thought to provide an online measure of meaning processing in the brain. However, the underlying process remains incompletely understood and actively debated. Here we present a computationally explicit account of this process and the emerging representation of sentence meaning. We simulate N400 amplitudes as the change induced by an incoming stimulus in an implicit and probabilistic representation of meaning captured by the hidden unit activation pattern in a neural network model of sentence comprehension, and we propose that the process underlying the N400 also drives implicit learning in the network. The model provides a unified account of 16 distinct findings from the N400 literature and connects human language comprehension with recent deep learning approaches to language processing.

Distinct Representations of Magnitude and Spatial Position within Parietal Cortex during Number-Space Mapping.

Mapping numbers onto space is foundational to mathematical cognition. These cognitive operations are often conceptualized in the context of a "mental number line" and involve multiple brain regions in or near the intraparietal sulcus (IPS) that have been implicated both in numeral and spatial cognition. Here we examine possible differentiation of function within these brain areas in relating numbers to spatial positions. By isolating the planning phase of a number line task and introducing spatiotopic mapping tools from fMRI into mental number line task research, we are able to focus our analysis on the neural activity of areas in anterior IPS (aIPS) previously associated with number processing and on spatiotopically organized areas in and around posterior IPS (pIPS), while participants prepare to place a number on a number line. Our results support the view that the nonpositional magnitude of a numerical symbol is coded in aIPS, whereas the position of a number in space is coded in posterior areas of IPS. By focusing on the planning phase, we are able to isolate activation related to the cognitive, rather than the sensory-motor, aspects of the task. Also, to allow the separation of spatial position from magnitude, we tested both a standard positive number line (0 to 100) and a zero-centered mixed number line (-100 to 100). We found evidence of a functional dissociation between aIPS and pIPS: Activity in aIPS was associated with a landmark distance effect not modulated by spatial position, whereas activity in pIPS revealed a contralateral preference effect.

Bayesian statistics to test Bayes optimality.

We agree with the authors that putting forward specific models and examining their agreement with experimental data are the best approach for understanding the nature of decision making. Although the authors only consider the likelihood function, prior, cost function, and decision rule (LPCD) framework, other choices are available. Bayesian statistics can be used to estimate essential parameters and assess the degree of optimality.

Building on prior knowledge without building it in.

Lake et al. propose that people rely on "start-up software," "causal models," and "intuitive theories" built using compositional representations to learn new tasks more efficiently than some deep neural network models. We highlight the many drawbacks of a commitment to compositional representations and describe our continuing effort to explore how the ability to build on prior knowledge and to learn new tasks efficiently could arise through learning in deep neural networks.

What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated.

We update complementary learning systems (CLS) theory, which holds that intelligent agents must possess two learning systems, instantiated in mammalians in neocortex and hippocampus. The first gradually acquires structured knowledge representations while the second quickly learns the specifics of individual experiences. We broaden the role of replay of hippocampal memories in the theory, noting that replay allows goal-dependent weighting of experience statistics. We also address recent challenges to the theory and extend it by showing that recurrent activation of hippocampal traces can support some forms of generalization and that neocortical learning can be rapid for information that is consistent with known structure. Finally, we note the relevance of the theory to the design of artificial intelligent agents, highlighting connections between neuroscience and machine learning.

Payoff Information Biases a Fast Guess Process in Perceptual Decision Making under Deadline Pressure: Evidence from Behavior, Evoked Potentials, and Quantitative Model Comparison.

We used electroencephalography (EEG) and behavior to examine the role of payoff bias in a difficult two-alternative perceptual decision under deadline pressure in humans. The findings suggest that a fast guess process, biased by payoff and triggered by stimulus onset, occurred on a subset of trials and raced with an evidence accumulation process informed by stimulus information. On each trial, the participant judged whether a rectangle was shifted to the right or left and responded by squeezing a right- or left-hand dynamometer. The payoff for each alternative (which could be biased or unbiased) was signaled 1.5 s before stimulus onset. The choice response was assigned to the first hand reaching a squeeze force criterion and reaction time was defined as time to criterion. Consistent with a fast guess account, fast responses were strongly biased toward the higher-paying alternative and the EEG exhibited an abrupt rise in the lateralized readiness potential (LRP) on a subset of biased payoff trials contralateral to the higher-paying alternative ∼ 150 ms after stimulus onset and 50 ms before stimulus information influenced the LRP. This rise was associated with poststimulus dynamometer activity favoring the higher-paying alternative and predicted choice and response time. Quantitative modeling supported the fast guess account over accounts of payoff effects supported in other studies. Our findings, taken with previous studies, support the idea that payoff and prior probability manipulations produce flexible adaptations to task structure and do not reflect a fixed policy for the integration of payoff and stimulus information. SIGNIFICANCE STATEMENT: Humans and other animals often face situations in which they must make choices based on uncertain sensory information together with information about expected outcomes (gains or losses) about each choice. We investigated how differences in payoffs between available alternatives affect neural activity, overt choice, and the timing of choice responses. In our experiment, in which participants were under strong time pressure, neural and behavioral findings together with model fitting suggested that our human participants often made a fast guess toward the higher reward rather than integrating stimulus and payoff information. Our findings, taken with findings from other studies, support the idea that payoff and prior probability manipulations produce flexible adaptations to task structure and do not reflect a fixed policy.

Connectionist perspectives on language learning, representation and processing.

The field of formal linguistics was founded on the premise that language is mentally represented as a deterministic symbolic grammar. While this approach has captured many important characteristics of the world's languages, it has also led to a tendency to focus theoretical questions on the correct formalization of grammatical rules while also de-emphasizing the role of learning and statistics in language development and processing. In this review we present a different approach to language research that has emerged from the parallel distributed processing or 'connectionist' enterprise. In the connectionist framework, mental operations are studied by simulating learning and processing within networks of artificial neurons. With that in mind, we discuss recent progress in connectionist models of auditory word recognition, reading, morphology, and syntactic processing. We argue that connectionist models can capture many important characteristics of how language is learned, represented, and processed, as well as providing new insights about the source of these behavioral patterns. Just as importantly, the networks naturally capture irregular (non-rule-like) patterns that are common within languages, something that has been difficult to reconcile with rule-based accounts of language without positing separate mechanisms for rules and exceptions.