The Confidence Database.

Understanding how people rate their confidence is critical for the characterization of a wide range of perceptual, memory, motor and cognitive processes. To enable the continued exploration of these processes, we created a large database of confidence studies spanning a broad set of paradigms, participant populations and fields of study. The data from each study are structured in a common, easy-to-use format that can be easily imported and analysed using multiple software packages. Each dataset is accompanied by an explanation regarding the nature of the collected data. At the time of publication, the Confidence Database (which is available at https://osf.io/s46pr/) contained 145 datasets with data from more than 8,700 participants and almost 4 million trials. The database will remain open for new submissions indefinitely and is expected to continue to grow. Here we show the usefulness of this large collection of datasets in four different analyses that provide precise estimations of several foundational confidence-related effects.

Comparing Bayesian and non-Bayesian accounts of human confidence reports.

Humans can meaningfully report their confidence in a perceptual or cognitive decision. It is widely believed that these reports reflect the Bayesian probability that the decision is correct, but this hypothesis has not been rigorously tested against non-Bayesian alternatives. We use two perceptual categorization tasks in which Bayesian confidence reporting requires subjects to take sensory uncertainty into account in a specific way. We find that subjects do take sensory uncertainty into account when reporting confidence, suggesting that brain areas involved in reporting confidence can access low-level representations of sensory uncertainty, a prerequisite of Bayesian inference. However, behavior is not fully consistent with the Bayesian hypothesis and is better described by simple heuristic models that use uncertainty in a non-Bayesian way. Both conclusions are robust to changes in the uncertainty manipulation, task, response modality, model comparison metric, and additional flexibility in the Bayesian model. Our results suggest that adhering to a rational account of confidence behavior may require incorporating implementational constraints.

Humans incorporate attention-dependent uncertainty into perceptual decisions and confidence.

Perceptual decisions are better when they take uncertainty into account. Uncertainty arises not only from the properties of sensory input but also from cognitive sources, such as different levels of attention. However, it is unknown whether humans appropriately adjust for such cognitive sources of uncertainty during perceptual decision-making. Here we show that, in a task in which uncertainty is relevant for performance, human categorization and confidence decisions take into account uncertainty related to attention. We manipulated uncertainty in an orientation categorization task from trial to trial using only an attentional cue. The categorization task was designed to disambiguate decision rules that did or did not depend on attention. Using formal model comparison to evaluate decision behavior, we found that category and confidence decision boundaries shifted as a function of attention in an approximately Bayesian fashion. This means that the observer's attentional state on each trial contributed probabilistically to the decision computation. This responsiveness of an observer's decisions to attention-dependent uncertainty should improve perceptual decisions in natural vision, in which attention is unevenly distributed across a scene.

Limitations of Proposed Signatures of Bayesian Confidence.

The Bayesian model of confidence posits that confidence reflects the observer's posterior probability that the decision is correct. Hangya, Sanders, and Kepecs (2016) have proposed that researchers can test the Bayesian model by deriving qualitative signatures of Bayesian confidence (i.e., patterns that one would expect to see if an observer were Bayesian) and looking for those signatures in human or animal data. We examine two proposed signatures, showing that their derivations contain hidden assumptions that limit their applicability and that they are neither necessary nor sufficient conditions for Bayesian confidence. One signature is an average confidence of 0.75 on trials with neutral evidence. This signature holds only when class-conditioned stimulus distributions do not overlap and when internal noise is very low. Another signature is that as stimulus magnitude increases, confidence increases on correct trials but decreases on incorrect trials. This divergence signature holds only when stimulus distributions do not overlap or when noise is high. Navajas et al. (2017) have proposed an alternative form of this signature; we find no indication that this alternative form is expected under Bayesian confidence. Our observations give us pause about the usefulness of the qualitative signatures of Bayesian confidence. To determine the nature of the computations underlying confidence reports, there may be no shortcut to quantitative model comparison.

Position of neocortical neurons transfected at different gestational ages with shRNA targeted against candidate dyslexia susceptibility genes.

Developmental dyslexia is a language learning disorder that affects approximately 4-10% of the population. A number of candidate dyslexia susceptibility genes have been identified, including DCDC2 and KIAA0319 on Chromosome (Chr) 6p22.2 and DYX1C1 on Chr 15q21. Embryonic knockdown of the function of homologs of these genes in rat neocortical projection cell progenitors by in utero electroporation of plasmids encoding small hairpin RNA (shRNA) revealed that all three genes disrupted neuronal migration to the neocortex. Specifically, this disruption would result in heterotopia formation (Dyx1c1 and Kiaa0319) and/or overmigration past their expected laminar location (Dyx1c1 and Dcdc2). In these experiments, neurons normally destined for the upper neocortical laminæ were transfected on embryonic day (E) 15.5, and we designed experiments to test whether these migration phenotypes were the result of targeting a specific type of projection neuron. We transfected litters with Dcdc2 shRNA, Dyx1c1 shRNA, Kiaa0319 shRNA, or fluorescent protein (as a control) at each of three gestational ages (E14.5, E15.5, or E16.5). Pups were allowed to come to term, and their brains were examined at 3 weeks of age for the position of transfected cells. We found that age of transfection did not affect the percentage of unmigrated neurons--transfection with Kiaa0319 shRNA resulted in heterotopia formation at all three ages. Overmigration of neurons transfected with Dcdc2 shRNA, while present following transfections at the later ages, did not occur following E14.5 transfections. These results are considered in light of the known functions of each of these candidate dyslexia susceptibility genes.