How does the primate brain combine generative and discriminative computations in vision?

Vision is widely understood as an inference problem. However, two contrasting conceptions of the inference process have each been influential in research on biological vision as well as the engineering of machine vision. The first emphasizes bottom-up signal flow, describing vision as a largely feedforward, discriminative inference process that filters and transforms the visual information to remove irrelevant variation and represent behaviorally relevant information in a format suitable for downstream functions of cognition and behavioral control. In this conception, vision is driven by the sensory data, and perception is direct because the processing proceeds from the data to the latent variables of interest. The notion of "inference" in this conception is that of the engineering literature on neural networks, where feedforward convolutional neural networks processing images are said to perform inference. The alternative conception is that of vision as an inference process in Helmholtz's sense, where the sensory evidence is evaluated in the context of a generative model of the causal processes that give rise to it. In this conception, vision inverts a generative model through an interrogation of the sensory evidence in a process often thought to involve top-down predictions of sensory data to evaluate the likelihood of alternative hypotheses. The authors include scientists rooted in roughly equal numbers in each of the conceptions and motivated to overcome what might be a false dichotomy between them and engage the other perspective in the realm of theory and experiment. The primate brain employs an unknown algorithm that may combine the advantages of both conceptions. We explain and clarify the terminology, review the key empirical evidence, and propose an empirical research program that transcends the dichotomy and sets the stage for revealing the mysterious hybrid algorithm of primate vision.

Identifying causal subsequent memory effects.

Over 40 y of accumulated research has detailed associations between neuroimaging signals measured during a memory encoding task and later memory performance, across a variety of brain regions, measurement tools, statistical approaches, and behavioral tasks. But the interpretation of these subsequent memory effects (SMEs) remains unclear: if the identified signals reflect cognitive and neural mechanisms of memory encoding, then the underlying neural activity must be causally related to future memory. However, almost all previous SME analyses do not control for potential confounders of this causal interpretation, such as serial position and item effects. We collect a large fMRI dataset and use an experimental design and analysis approach that allows us to statistically adjust for nearly all known exogenous confounding variables. We find that, using standard approaches without adjustment, we replicate several univariate and multivariate subsequent memory effects and are able to predict memory performance across people. However, we are unable to identify any signal that reliably predicts subsequent memory after adjusting for confounding variables, bringing into doubt the causal status of these effects. We apply the same approach to subjects' judgments of learning collected following an encoding period and show that these behavioral measures of mnemonic status do predict memory after adjustments, suggesting that it is possible to measure signals near the time of encoding that reflect causal mechanisms but that existing neuroimaging measures, at least in our data, may not have the precision and specificity to do so.

Children's failure to control variables may reflect adaptive decision-making.

Changing one variable at a time while controlling others is a key aspect of scientific experimentation and a central component of STEM curricula. However, children reportedly struggle to learn and implement this strategy. Why do children's intuitions about how best to intervene on a causal system conflict with scientific practices? Mathematical analyses have shown that controlling variables is not always the most efficient learning strategy, and that its effectiveness depends on the "causal sparsity" of the problem, i.e., how many variables are likely to impact the outcome. We tested the degree to which 7- to 13-year-old children (n = 104) adapt their learning strategies based on expectations about causal sparsity. We report new evidence demonstrating that some previous work may have undersold children's causal learning skills: Children can perform and interpret controlled experiments, are sensitive to causal sparsity, and use this information to tailor their testing strategies, demonstrating adaptive decision-making.

Expectations about future learning influence moment-to-moment feelings of suspense.

Suspense is a cognitive and affective state that is often experienced in the anticipation of information and contributes to the enjoyment and consumption of entertainment such as movies or sports. Ely et al. proposed a formal definition of suspense which relies upon predictions about future belief updates. In order to empirically evaluate this theory, we designed a task based on the casino card game Blackjack where a variety of suspense dynamics can be experimentally induced. Our behavioural data confirmed the explanatory power of this theory. We further compared this formulation with other heuristic models inspired by studies in other domains such as narratives and found that most heuristic models cannot well account for the specific temporal dynamics of suspense across wide range of game variants. We additionally propose a way to test whether experiencing greater levels of suspense motivates more game-playing. In summary, this work is an initial attempt to link formal models of information and uncertainty with affective cognitive states and motivation.

Limits on simulation approaches in intuitive physics.

A popular explanation of the human ability for physical reasoning is that it depends on a sophisticated ability to perform mental simulations. According to this perspective, physical reasoning problems are approached by repeatedly simulating relevant aspects of a scenario, with noise, and making judgments based on aggregation over these simulations. In this paper, we describe three core tenets of simulation approaches, theoretical commitments that must be present in order for a simulation approach to be viable. The identification of these tenets threatens the plausibility of simulation as a theory of physical reasoning, because they appear to be incompatible with what we know about cognition more generally. To investigate this apparent contradiction, we describe three experiments involving simple physical judgments and predictions, and argue their results challenge these core predictions of theories of mental simulation.

Behavioral, Physiological, and Neural Signatures of Surprise during Naturalistic Sports Viewing.

Surprise signals a discrepancy between past and current beliefs. It is theorized to be linked to affective experiences, the creation of particularly resilient memories, and segmentation of the flow of experience into discrete perceived events. However, the ability to precisely measure naturalistic surprise has remained elusive. We used advanced basketball analytics to derive a quantitative measure of surprise and characterized its behavioral, physiological, and neural correlates in human subjects observing basketball games. We found that surprise was associated with segmentation of ongoing experiences, as reflected by subjectively perceived event boundaries and shifts in neocortical patterns underlying belief states. Interestingly, these effects differed by whether surprising moments contradicted or bolstered current predominant beliefs. Surprise also positively correlated with pupil dilation, activation in subcortical regions associated with dopamine, game enjoyment, and long-term memory. These investigations support key predictions from event segmentation theory and extend theoretical conceptualizations of surprise to real-world contexts.

Broken Physics: A Conjunction-Fallacy Effect in Intuitive Physical Reasoning.

One remarkable aspect of human cognition is our ability to reason about physical events. This article provides novel evidence that intuitive physics is subject to a peculiar error, the classic conjunction fallacy, in which people rate the probability of a conjunction of two events as more likely than one constituent (a logical impossibility). Participants viewed videos of physical scenarios and judged the probability that either a single event or a conjunction of two events would occur. In Experiment 1 (n = 60), participants consistently rated conjunction events as more likely than single events for the same scenes. Experiment 2 (n = 180) extended these results to rule out several alternative explanations. Experiment 3 (n = 100) generalized the finding to different scenes. This demonstration of conjunction errors contradicts claims that such errors should not appear in intuitive physics and presents a serious challenge to current theories of mental simulation in physical reasoning.

Causal Information-Seeking Strategies Change Across Childhood and Adolescence.

Intervening on causal systems can illuminate their underlying structures. Past work has shown that, relative to adults, young children often make intervention decisions that appear to confirm a single hypothesis rather than those that optimally discriminate alternative hypotheses. Here, we investigated how the ability to make informative causal interventions changes across development. Ninety participants between the ages of 7 and 25 completed 40 different puzzles in which they had to intervene on various causal systems to determine their underlying structures. Each puzzle comprised a three- or four-node computer chip with hidden wires. On each trial, participants viewed two possible arrangements of the chip's hidden wires and had to select a single node to activate. After observing the outcome of their intervention, participants selected a wire configuration and rated their confidence in their selection. We characterized participant choices with a Bayesian measurement model that indexed the extent to which participants selected nodes that would best disambiguate the two possible causal structures versus those that had high causal centrality in one of the two causal hypotheses but did not necessarily discriminate between them. Our model estimates revealed that the use of a discriminatory strategy increased through early adolescence. Further, developmental improvements in intervention strategy were related to changes in the ability to accurately judge the strength of evidence that interventions revealed, as indexed by participants' confidence in their selections. Our results suggest that improvements in causal information-seeking extend into adolescence and may be driven by metacognitive sensitivity to the efficacy of previous interventions in discriminating competing ideas.

Testing one or multiple: How beliefs about sparsity affect causal experimentation.

What is the best way of discovering the underlying structure of a causal system composed of multiple variables? One prominent idea is that learners should manipulate each candidate variable in isolation to avoid confounds (sometimes known as the control of variables [CV] strategy). We demonstrate that CV is not always the most efficient method for learning. Using an optimal actor model, which aims to minimize the average number of tests, we show that when a causal system is sparse (i.e., when the outcome of interest has few or even just one actual cause among the candidate variables), it is more efficient to test multiple variables at once. Across a series of behavioral experiments, we then show that people are sensitive to causal sparsity and adapt their strategies accordingly. When interacting with a dense causal system (high proportion of actual causes among candidate variables), they use a CV strategy, changing one variable at a time. When interacting with a sparse causal system, they are more likely to test multiple variables at once. However, we also find that people sometimes use a CV strategy even when a system is sparse. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Memory enhancements from active control of learning emerge across development.

This paper investigates whether active control of study leads to enhanced learning in 5- to 11-year-old children. In Experiments 1 and 2, participants played a simple memory game with the instruction to try to remember and later recognize a set of 64 objects. In Experiment 3, the goal was to learn the French names for the same objects. For half of the materials presented, participants could decide the order and pacing of study (Active condition). For the other half, they passively observed the study decisions of a previous participant (Yoked condition). Recognition memory was more accurate for objects studied in the active as compared to the yoked condition. However, the active learning advantage was relatively small among 5-year-olds and increased with age, becoming comparable to adults' by age 8. Our results show that the ability to actively control study develops during early childhood and results in memory benefits that last over a week-long delay. We discuss possible interpretations for the observed developmental change, as well as the implications of these results for educational implementations.

Asking the right questions about the psychology of human inquiry: Nine open challenges.

The ability to act on the world with the goal of gaining information is core to human adaptability and intelligence. Perhaps the most successful and influential account of such abilities is the Optimal Experiment Design (OED) hypothesis, which argues that humans intuitively perform experiments on the world similar to the way an effective scientist plans an experiment. The widespread application of this theory within many areas of psychology calls for a critical evaluation of the theory's core claims. Despite many successes, we argue that the OED hypothesis remains lacking as a theory of human inquiry and that research in the area often fails to confront some of the most interesting and important questions. In this critical review, we raise and discuss nine open questions about the psychology of human inquiry.

The limits of learning: Exploration, generalization, and the development of learning traps.

Learning usually improves the accuracy of beliefs through the accumulation of experience. But are there limits to learning that prevent us from accurately understanding our world? In this article we investigate the concept of a "learning trap"-the formation of a stable false belief even with extensive experience. Our review highlights how these traps develop through the interaction of learning and decision making in unknown environments. We further document a particularly pernicious learning trap driven by selective attention, a mechanism often assumed to facilitate learning in complex environments. Using computer simulation, we demonstrate the key attributes of the agent and environment that lead to this new type of learning trap. Then, in a series of experiments we present evidence that people robustly fall into this trap, even in the presence of various interventions predicted to meliorate it. These results highlight a fundamental limit to learning and adaptive behavior that impacts individuals, organizations, animals, and machines. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Intuitive experimentation in the physical world.

Many aspects of our physical environment are hidden. For example, it is hard to estimate how heavy an object is from visual observation alone. In this paper we examine how people actively "experiment" within the physical world to discover such latent properties. In the first part of the paper, we develop a novel framework for the quantitative analysis of the information produced by physical interactions. We then describe two experiments that present participants with moving objects in "microworlds" that operate according to continuous spatiotemporal dynamics similar to everyday physics (i.e., forces of gravity, friction, etc.). Participants were asked to interact with objects in the microworlds in order to identify their masses, or the forces of attraction/repulsion that governed their movement. Using our modeling framework, we find that learners who freely interacted with the physical system selectively produced evidence that revealed the physical property consistent with their inquiry goal. As a result, their inferences were more accurate than for passive observers and, in some contexts, for yoked participants who watched video replays of an active learner's interactions. We characterize active learners' actions into a range of micro-experiment strategies and discuss how these might be learned or generalized from past experience. The technical contribution of this work is the development of a novel analytic framework and methodology for the study of interactively learning about the physical world. Its empirical contribution is the demonstration of sophisticated goal directed human active learning in a naturalistic context.

Desirable difficulties during the development of active inquiry skills.

This study explores developmental changes in the ability to ask informative questions, hypothesizing a link between the ability to update beliefs in light of evidence and the ability to ask informative questions. Five- to ten-year-old children played an iPad game asking them to identify a hidden insect. Learners could either ask about individual insects, or make a series of feature queries (e.g., "Does the hidden insect have antenna?") that could more efficiently narrow the hypothesis space. Critically, the task display either helped children integrate evidence with the hypothesis space or required them to perform this operation themselves. Our prediction was that assisting children with belief updating would help them formulate more informative queries. This assistance improved some aspects of children's active inquiry behavior; however, despite making some updating mistakes, children required to update their own beliefs asked questions that were more context-sensitive and thus informative. The results show how making a task more difficult can improve some aspects of children's active inquiry skills, thus illustrating a type of "desirable difficulty" for reasoning.

Using precise word timing information improves decoding accuracy in a multiband-accelerated multimodal reading experiment.

The blood-oxygen-level-dependent (BOLD) signal measured in functional magnetic resonance imaging (fMRI) experiments is generally regarded as sluggish and poorly suited for probing neural function at the rapid timescales involved in sentence comprehension. However, recent studies have shown the value of acquiring data with very short repetition times (TRs), not merely in terms of improvements in contrast to noise ratio (CNR) through averaging, but also in terms of additional fine-grained temporal information. Using multiband-accelerated fMRI, we achieved whole-brain scans at 3-mm resolution with a TR of just 500 ms at both 3T and 7T field strengths. By taking advantage of word timing information, we found that word decoding accuracy across two separate sets of scan sessions improved significantly, with better overall performance at 7T than at 3T. The effect of TR was also investigated; we found that substantial word timing information can be extracted using fast TRs, with diminishing benefits beyond TRs of 1000 ms.

Information sampling behavior with explicit sampling costs.

The decision to gather information should take into account both the value of information and its accrual costs in time, energy and money. Here we explore how people balance the monetary costs and benefits of gathering additional information in a perceptual-motor estimation task. Participants were rewarded for touching a hidden circular target on a touch-screen display. The target's center coincided with the mean of a circular Gaussian distribution from which participants could sample repeatedly. Each "cue" - sampled one at a time - was plotted as a dot on the display. Participants had to repeatedly decide, after sampling each cue, whether to stop sampling and attempt to touch the hidden target or continue sampling. Each additional cue increased the participants' probability of successfully touching the hidden target but reduced their potential reward. Two experimental conditions differed in the initial reward associated with touching the hidden target and the fixed cost per cue. For each condition we computed the optimal number of cues that participants should sample, before taking action, to maximize expected gain. Contrary to recent claims that people gather less information than they objectively should before taking action, we found that participants over-sampled in one experimental condition, and did not significantly under- or over-sample in the other. Additionally, while the ideal observer model ignores the current sample dispersion, we found that participants used it to decide whether to stop sampling and take action or continue sampling, a possible consequence of imperfect learning of the underlying population dispersion across trials.

Self-Directed Learning Favors Local, Rather Than Global, Uncertainty.

Collecting (or "sampling") information that one expects to be useful is a powerful way to facilitate learning. However, relatively little is known about how people decide which information is worth sampling over the course of learning. We describe several alternative models of how people might decide to collect a piece of information inspired by "active learning" research in machine learning. We additionally provide a theoretical analysis demonstrating the situations under which these models are empirically distinguishable, and we report a novel empirical study that exploits these insights. Our model-based analysis of participants' information gathering decisions reveals that people prefer to select items which resolve uncertainty between two possibilities at a time rather than items that have high uncertainty across all relevant possibilities simultaneously. Rather than adhering to strictly normative or confirmatory conceptions of information search, people appear to prefer a "local" sampling strategy, which may reflect cognitive constraints on the process of information gathering.

psiTurk: An open-source framework for conducting replicable behavioral experiments online.

Online data collection has begun to revolutionize the behavioral sciences. However, conducting carefully controlled behavioral experiments online introduces a number of new of technical and scientific challenges. The project described in this paper, psiTurk, is an open-source platform which helps researchers develop experiment designs which can be conducted over the Internet. The tool primarily interfaces with Amazon's Mechanical Turk, a popular crowd-sourcing labor market. This paper describes the basic architecture of the system and introduces new users to the overall goals. psiTurk aims to reduce the technical hurdles for researchers developing online experiments while improving the transparency and collaborative nature of the behavioral sciences.

Strategies to intervene on causal systems are adaptively selected.

How do people choose interventions to learn about causal systems? Here, we considered two possibilities. First, we test an information sampling model, information gain, which values interventions that can discriminate between a learner's hypotheses (i.e. possible causal structures). We compare this discriminatory model to a positive testing strategy that instead aims to confirm individual hypotheses. Experiment 1 shows that individual behavior is described best by a mixture of these two alternatives. In Experiment 2 we find that people are able to adaptively alter their behavior and adopt the discriminatory model more often after experiencing that the confirmatory strategy leads to a subjective performance decrement. In Experiment 3, time pressure leads to the opposite effect of inducing a change towards the simpler positive testing strategy. These findings suggest that there is no single strategy that describes how intervention decisions are made. Instead, people select strategies in an adaptive fashion that trades off their expected performance and cognitive effort.

Deconstructing the effect of self-directed study on episodic memory.

Self-directed learning is often associated with better long-term memory retention; however, the mechanisms that underlie this advantage remain poorly understood. This series of experiments was designed to "deconstruct" the notion of self-directed learning, in order to better identify the factors most responsible for these improvements to memory. In particular, we isolated the memory advantage that comes from controlling the content of study episodes from the advantage that comes from controlling the timing of those episodes. Across four experiments, self-directed learning significantly enhanced recognition memory, relative to passive observation. However, the advantage for self-directed learning was found to be present even under extremely minimal conditions of volitional control (simply pressing a button when a participant was ready to advance to the next item). Our results suggest that improvements to memory following self-directed encoding may be related to the ability to coordinate stimulus presentation with the learner's current preparatory or attentional state, and they highlight the need to consider the range of cognitive control processes involved in and influenced by self-directed study.