Objectively quantifying subjective phenomena: Measuring the flashed face distortion effect.

Objectively quantifying subjective phenomena like visual illusions is challenging. We address this issue in the context of the Flashed Face Distortion Effect (FFDE), where faces presented in succession appear distorted and grotesque. We first show that the traditional method of quantifying FFDE - via subjective ratings of the level of distortion - is subject to substantial biases. Motivated by this finding, we develop an objective method for quantifying FFDE by introducing two design innovations. First, we create artificially distorted faces and ask subjects to discriminate between undistorted and objectively distorted faces. Second, we employ both an illusion condition, which includes a succession of 15 face flashes, and a control condition, which includes a single face flash and does not induce an illusion. Using these innovations, we quantify the strength of the face distortion illusion by comparing the response bias for identifying distorted faces between the illusion and control conditions. We find that our method successfully quantifies the face distortion, with subjects exhibiting a more liberal response bias in the illusion condition. Finally, we apply our new method to evaluate how the face distortion illusion is modulated by face eccentricity, face inversion, the temporal frequency of the face flashes, and presence of temporal gaps between consecutive faces. Our results demonstrate the utility of our objective method in quantifying the subjective illusion of face distortion. Critically, the method is general and can be applied to other phenomena that are inherently subjective.

Behavior can be decoded across the cortex when individual differences are considered.

Group-level analyses have typically associated behavioral signatures with a constrained set of brain areas. Here we show that two behavioral metrics - reaction time (RT) and confidence - can be decoded across the cortex when each individual is considered separately. Subjects (N=50) completed a perceptual decision-making task with confidence. We built models decoding trial-level RT and confidence separately for each subject using the activation patterns in one brain area at a time after splitting the entire cortex into 200 regions of interest (ROIs). At the group level, we replicated previous results by showing that both RT and confidence could be decoded from a small number of ROIs (12.0% and 3.5%, respectively). Critically, at the level of the individual, both RT and confidence could be decoded from most brain regions even after Bonferroni correction (90.0% and 72.5%, respectively). Surprisingly, we observed that many brain regions exhibited opposite brain-behavior relationships across individuals, such that, for example, higher activations predicted fast RTs in some subjects but slow RTs in others. These results were further replicated in a second dataset. Lastly, we developed a simple test to determine the robustness of decoding performance, which showed that several hundred trials per subject are required for robust decoding. These results show that behavioral signatures can be decoded from a much broader range of cortical areas than previously recognized and suggest the need to study the brain-behavior relationship at both the group and the individual level.

Threat Expectation Does Not Improve Perceptual Discrimination despite Causing Heightened Priority Processing in the Frontoparietal Network.

Threat cues have been widely shown to elicit increased sensory and attentional neural processing. However, whether this enhanced recruitment leads to measurable behavioral improvements in perception is still in question. Here, we adjudicate between two opposing theories: that threat cues do or do not enhance perceptual sensitivity. We created threat stimuli by pairing one direction of motion in a random dot kinematogram with an aversive sound. While in the MRI scanner, 46 subjects (both men and women) completed a cued (threat/safe/neutral) perceptual decision-making task where they indicated the perceived motion direction of each moving dot stimulus. We found strong evidence that threat cues did not increase perceptual sensitivity compared with safe and neutral cues. This lack of improvement in perceptual decision-making ability occurred despite the threat cue resulting in widespread increases in frontoparietal BOLD activity, as well as increased connectivity between the right insula and the frontoparietal network. These results call into question the intuitive claim that expectation automatically enhances our perception of threat and highlight the role of the frontoparietal network in prioritizing the processing of threat-related environmental cues.

Human-like dissociations between confidence and accuracy in convolutional neural networks.

Prior research has shown that manipulating stimulus energy by changing both stimulus contrast and variability results in confidence-accuracy dissociations in humans. Specifically, even when performance is matched, higher stimulus energy leads to higher confidence. The most common explanation for this effect is the positive evidence heuristic where confidence neglects evidence that disconfirms the choice. However, an alternative explanation is the signal-and-variance-increase hypothesis, according to which these dissociations arise from low-level changes in the separation and variance of perceptual representations. Because artificial neural networks lack built-in confidence heuristics, they can serve as a test for the necessity of confidence heuristics in explaining confidence-accuracy dissociations. Therefore, we tested whether confidence-accuracy dissociations induced by stimulus energy manipulations emerge naturally in convolutional neural networks (CNNs). We found that, across three different energy manipulations, CNNs produced confidence-accuracy dissociations similar to those found in humans. This effect was present for a range of CNN architectures from shallow 4-layer networks to very deep ones, such as VGG-19 and ResNet -50 pretrained on ImageNet. Further, we traced back the reason for the confidence-accuracy dissociations in all CNNs to the same signal-and-variance increase that has been proposed for humans: higher stimulus energy increased the separation and variance of the CNNs' internal representations leading to higher confidence even for matched accuracy. These findings cast doubt on the necessity of the positive evidence heuristic to explain human confidence and establish CNNs as promising models for adjudicating between low-level, stimulus-driven and high-level, cognitive explanations of human behavior.

Task learning is subserved by a domain-general brain network.

One of the most important human faculties is the ability to acquire not just new memories but the capacity to perform entirely new tasks. However, little is known about the brain mechanisms underlying the learning of novel tasks. Specifically, it is unclear to what extent learning of different tasks depends on domain-general and/or domain-specific brain mechanisms. Here human subjects (n = 45) learned to perform 6 new tasks while undergoing functional MRI. The different tasks required the engagement of perceptual, motor, and various cognitive processes related to attention, expectation, speed-accuracy tradeoff, and metacognition. We found that a bilateral frontoparietal network was more active during the initial compared with the later stages of task learning, and that this effect was stronger for task variants requiring more new learning. Critically, the same frontoparietal network was engaged by all 6 tasks, demonstrating its domain generality. Finally, although task learning decreased the overall activity in the frontoparietal network, it increased the connectivity strength between the different nodes of that network. These results demonstrate the existence of a domain-general brain network whose activity and connectivity reflect learning for a variety of new tasks, and thus may underlie the human capacity for acquiring new abilities.

Multiple brain activation patterns for the same task.

Meaningful variation in internal states that impacts cognition and behavior remains challenging to discover and characterize. Here we leveraged trial-to-trial fluctuations in the brain-wide signal recorded using functional MRI to test if distinct sets of brain regions are activated on different trials when accomplishing the same task. Across three different perceptual decision-making experiments, we estimated the brain activations for each trial. We then clustered the trials based on their similarity using modularity-maximization, a data-driven classification method. In each experiment, we found multiple distinct but stable subtypes of trials, suggesting that the same task can be accomplished in the presence of widely varying brain activation patterns. Surprisingly, in all experiments, one of the subtypes exhibited strong activation in the default mode network, which is typically thought to decrease in activity during tasks that require externally focused attention. The remaining subtypes were characterized by activations in different task-positive areas. The default mode network subtype was characterized by behavioral signatures that were similar to the other subtypes exhibiting activation with task-positive regions. Finally, in a fourth experiment, we tested whether multiple activation patterns would also appear for a qualitatively different, working memory task. We again found multiple subtypes of trials with differential activation in frontoparietal control, dorsal attention, and ventral attention networks. Overall, these findings demonstrate that the same cognitive tasks are accomplished through multiple brain activation patterns.

How do humans give confidence? A comprehensive comparison of process models of perceptual metacognition.

Humans have the metacognitive ability to assess the accuracy of their decisions via confidence judgments. Several computational models of confidence have been developed but not enough has been done to compare these models, making it difficult to adjudicate between them. Here, we compare 14 popular models of confidence that make various assumptions, such as confidence being derived from postdecisional evidence, from positive (decision-congruent) evidence, from posterior probability computations, or from a separate decision-making system for metacognitive judgments. We fit all models to three large experiments in which subjects completed a basic perceptual task with confidence ratings. In Experiments 1 and 2, the best-fitting model was the lognormal meta noise (LogN) model, which postulates that confidence is selectively corrupted by signal-dependent noise. However, in Experiment 3, the positive evidence (PE) model provided the best fits. We evaluated a new model combining the two consistently best-performing models-LogN and the weighted evidence and visibility (WEV). The resulting model, which we call logWEV, outperformed its individual counterparts and the PE model across all data sets, offering a better, more generalizable explanation for these data. Parameter and model recovery analyses showed mostly good recoverability but with important exceptions carrying implications for our ability to discriminate between models. Finally, we evaluated each model's ability to explain different patterns in the data, which led to additional insight into their performances. These results comprehensively characterize the relative adequacy of current confidence models to fit data from basic perceptual tasks and highlight the most plausible mechanisms underlying confidence generation. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Think twice: Re-assessing confidence improves visual metacognition.

Metacognition is a fundamental feature of human behavior that has adaptive functional value. Current understanding of the factors that influence metacognition remains incomplete, and we lack protocols to improve metacognition. Here, we introduce a two-step confidence choice paradigm to test whether metacognitive performance may improve by asking subjects to reassess their initial confidence. Previous work on perceptual and mnemonic decision-making has shown that (type 1) perceptual sensitivity benefits from reassessing the primary choice, however, it is not clear whether such an effect occurs for type 2 confidence choices. To test this hypothesis, we ran two separate online experiments, in which participants completed a type 1 task followed by two consecutive confidence choices. The results of the two experiments indicated that metacognitive sensitivity improved after re-evaluation. Since post-decisional evidence accumulation following the first confidence choice is likely to be minimal, this metacognitive improvement is better accounted for by an attenuation of metacognitive noise during the process of confidence generation. Thus, here we argue that metacognitive noise may be filtered out by additional post-decisional processing, thereby improving metacognitive sensitivity. We discuss the ramifications of these findings for models of metacognition and for developing protocols to train and manipulate metacognitive processes.

Common computations for metacognition and meta-metacognition.

Recent evidence shows that people have the meta-metacognitive ability to evaluate their metacognitive judgments of confidence. However, it is unclear whether meta-metacognitive judgments are made by a different system and rely on a separate set of computations compared to metacognitive judgments. To address this question, we asked participants (N = 36) to perform a perceptual decision-making task and provide (i) an object-level, Type-1 response about the identity of the stimulus; (ii) a metacognitive, Type-2 response (low/high) regarding their confidence in their Type-1 decision; and (iii) a meta-metacognitive, Type-3 response (low/high) regarding the quality of their Type-2 rating. We found strong evidence for the existence of Type-3, meta-metacognitive ability. In a separate condition, participants performed an identical task with only a Type-1 response followed by a Type-2 response given on a 4-point scale. We found that the two conditions produced equivalent results such that the combination of binary Type-2 and binary Type-3 responses acts similar to a 4-point Type-2 response. Critically, while Type-2 evaluations were subject to metacognitive noise, Type-3 judgments were made at no additional cost. These results suggest that it is unlikely that there is a distinction between Type-2 and Type-3 systems (metacognition and meta-metacognition) in perceptual decision-making and, instead, a single system can be flexibly adapted to produce both Type-2 and Type-3 evaluations recursively.

Brain signatures indexing variation in internal processing during perceptual decision-making.

Brain activity is highly variable during a task. Discovering, characterizing, and linking variability in brain activity to internal processes has primarily relied on experimental manipulations. However, changes in internal processing could arise from many factors independent of experimental conditions. Here we utilize a data-driven clustering method based on modularity-maximation to identify consistent spatial-temporal EEG activity patterns across individual trials. Subjects (N = 25) performed a motion discrimination task with six interleaved levels of coherence. Clustering identified two discrete subtypes of trials with different patterns of activity. Surprisingly, Subtype 1 occurred more frequently in trials with lower motion coherence but was associated with faster response times. Computational modeling suggests that Subtype 1 was characterized by a lower threshold for reaching a decision. These results highlight across-trial variability in decision processes traditionally hidden to experimenters and provide a method for identifying endogenous brain state variability relevant to cognition and behavior.

The timing of confidence computations in human prefrontal cortex.

Knowing when confidence computations take place is critical for building a mechanistic understanding of the neural and computational bases of metacognition. Yet, even though a substantial amount of research has focused on revealing the neural correlates and computations underlying human confidence judgments, very little is known about the timing of confidence computations. To understand when confidence is computed, we delivered single pulses of transcranial magnetic stimulation (TMS) at different times after stimulus presentation while subjects judged the orientation of a briefly presented visual stimulus and provided a confidence rating. TMS was delivered to either the right dorsolateral prefrontal cortex (DLPFC) in the experimental group or to vertex in the control group. We found that TMS to right DLPFC, but not to vertex, led to increased confidence in the absence of changes to accuracy or metacognitive efficiency. Critically, equivalent levels of confidence increase occurred for TMS delivered between 200 and 500 msec after stimulus presentation. These results suggest that confidence computations occur during a broad window that begins before the perceptual decision has been fully made and thus provide important constraints for theories of confidence generation.

Quantifying the contribution of subject and group factors in brain activation.

Research in neuroscience often assumes universal neural mechanisms, but increasing evidence points toward sizeable individual differences in brain activations. What remains unclear is the extent of the idiosyncrasy and whether different types of analyses are associated with different levels of idiosyncrasy. Here we develop a new method for addressing these questions. The method consists of computing the within-subject reliability and subject-to-group similarity of brain activations and submitting these values to a computational model that quantifies the relative strength of group- and subject-level factors. We apply this method to a perceptual decision-making task (n = 50) and find that activations related to task, reaction time, and confidence are influenced equally strongly by group- and subject-level factors. Both group- and subject-level factors are dwarfed by a noise factor, though higher levels of smoothing increases their contributions relative to noise. Overall, our method allows for the quantification of group- and subject-level factors of brain activations and thus provides a more detailed understanding of the idiosyncrasy levels in brain activations.

Stimulation along the anterior-posterior axis of lateral frontal cortex reduces visual serial dependence.

Serial dependence is an attractive pull that recent perceptual history exerts on current judgments. Theory suggests that this bias is due to a form of short-term plasticity prevalent specifically in the frontal lobe. We sought to test the importance of the frontal lobe to serial dependence by disrupting neural activity along its lateral surface during two tasks with distinct perceptual and motor demands. In our first experiment, stimulation of the lateral prefrontal cortex (LPFC) during an oculomotor delayed response task decreased serial dependence only in the first saccade to the target, whereas stimulation posterior to the LPFC decreased serial dependence only in adjustments to eye position after the first saccade. In our second experiment, which used an orientation discrimination task, stimulation anterior to, in, and posterior to the LPFC all caused equivalent decreases in serial dependence. In this experiment, serial dependence occurred only between stimuli at the same location; an alternation bias was observed across hemifields. Frontal stimulation had no effect on the alternation bias. Transcranial magnetic stimulation to parietal cortex had no effect on serial dependence in either experiment. In summary, our experiments provide evidence for both functional differentiation (Experiment 1) and redundancy (Experiment 2) in frontal cortex with respect to serial dependence.

Threat expectation does not improve perceptual discrimination despite causing heightened priority processing in the frontoparietal network.

Threat cues have been widely shown to elicit increased sensory and attentional neural processing. However, whether this enhanced recruitment leads to measurable behavioral improvements in perception is still in question. Here we adjudicate between two opposing theories: that threat cues do or do not enhance perceptual sensitivity. We created threat stimuli by pairing one direction of motion in a random dot kinematogram with an aversive sound. While in the MRI scanner, 46 subjects (both men and women) completed a cued (threat/safe/neutral) perceptual decision-making task where they indicated the perceived motion direction of each moving dots stimulus. We found strong evidence that threat cues did not increase perceptual sensitivity compared to safe and neutral cues. This lack of improvement in perceptual decision-making ability occurred despite the threat cue resulting in widespread increases in frontoparietal BOLD activity, as well as increased connectivity between the right insula and the frontoparietal network. These results call into question the intuitive claim that expectation automatically enhances our perception of threat, and highlight the role of the frontoparietal network in prioritizing the processing of threat-related environmental cues.

Confidence response times: Challenging postdecisional models of confidence.

Even though the nature of confidence computations has been the topic of intense interest, little attention has been paid to what confidence response times (cRTs) reveal about the underlying confidence computations. Several previous studies found cRTs to be negatively correlated with confidence in the group as a whole and consequently hypothesized the existence of an intrinsic relationship of cRT with confidence for all subjects. This hypothesis was further used to support postdecisional models of confidence that predict that cRT and confidence should always be negatively correlated. Here we test the alternative hypothesis that cRT is driven by the frequency of confidence responses such that the most frequent confidence ratings are inherently made faster regardless of whether they are high or low. We examined cRTs in three large data sets from the Confidence Database and found that the lowest cRTs occurred for the most frequent confidence rating. In other words, subjects who gave high confidence ratings most frequently had negative confidence-cRT relationships, whereas subjects who gave low confidence ratings most frequently had positive confidence-cRT relationships. In addition, we found a strong across-subject correlation between response time and cRT, suggesting that response speed for both the decision and the confidence rating is influenced by a common factor. Our results show that cRT is not intrinsically linked to confidence and strongly challenge several postdecisional models of confidence.

Common computations in automatic cue combination and metacognitive confidence reports.

Appropriate perceptual decision making necessitates the accurate estimation and use of sensory uncertainty. Such estimation has been studied in the context of both low-level multisensory cue combination and metacognitive estimation of confidence, but it remains unclear whether the same computations underlie both sets of uncertainty estimation. We created visual stimuli with low vs. high overall motion energy, such that the high-energy stimuli led to higher confidence but lower accuracy in a visual-only task. Importantly, we tested the impact of the low- and high-energy visual stimuli on auditory motion perception in a separate task. Despite being irrelevant to the auditory task, both visual stimuli impacted auditory judgments presumably via automatic low-level mechanisms. Critically, we found that the high-energy visual stimuli influenced the auditory judgments more strongly than the low-energy visual stimuli. This effect was in line with the confidence but contrary to the accuracy differences between the high- and low-energy stimuli in the visual-only task. These effects were captured by a simple computational model that assumes common computational principles underlying both confidence reports and multisensory cue combination. Our results reveal a deep link between automatic sensory processing and metacognitive confidence reports, and suggest that vastly different stages of perceptual decision making rely on common computational principles.

Using serial dependence to predict confidence across observers and cognitive domains.

Our perceptual system appears hardwired to exploit regularities of input features across space and time in seemingly stable environments. This can lead to serial dependence effects whereby recent perceptual representations bias current perception. Serial dependence has also been demonstrated for more abstract representations, such as perceptual confidence. Here, we ask whether temporal patterns in the generation of confidence judgments across trials generalize across observers and different cognitive domains. Data from the Confidence Database across perceptual, memory, and cognitive paradigms was reanalyzed. Machine learning classifiers were used to predict the confidence on the current trial based on the history of confidence judgments on the previous trials. Cross-observer and cross-domain decoding results showed that a model trained to predict confidence in the perceptual domain generalized across observers to predict confidence across the different cognitive domains. The recent history of confidence was the most critical factor. The history of accuracy or Type 1 reaction time alone, or in combination with confidence, did not improve the prediction of the current confidence. We also observed that confidence predictions generalized across correct and incorrect trials, indicating that serial dependence effects in confidence generation are uncoupled to metacognition (i.e., how we evaluate the precision of our own behavior). We discuss the ramifications of these findings for the ongoing debate on domain-generality versus domain-specificity of metacognition.

The timing of confidence computations in human prefrontal cortex.

Knowing when confidence computations take place is critical for building mechanistic understanding of the neural and computational bases of metacognition. Yet, even though substantial amount of research has focused on revealing the neural correlates and computations underlying human confidence judgments, very little is known about the timing of confidence computations. Subjects judged the orientation of a briefly presented visual stimulus and provided a confidence rating regarding the accuracy of their decision. We delivered single pulses of transcranial magnetic stimulation (TMS) at different times after stimulus presentation. TMS was delivered to either dorsolateral prefrontal cortex (DLPFC) in the experimental group or to vertex in the control group. We found that TMS to DLPFC, but not to vertex, led to increased confidence in the absence of changes to accuracy or metacognitive ability. Critically, equivalent levels of confidence increase occurred for TMS delivered between 200 and 500 ms after stimulus presentation. These results suggest that confidence computations occur during a broad window that begins before the perceptual decision has been fully made and thus provide important constraints for theories of confidence generation.

Brain signatures indexing variation in internal processing during perceptual decision-making.

Brain activity is highly variable even while performing the same cognitive task with consequences for performance. Discovering, characterizing, and linking variability in brain activity to internal processes has primarily relied on experimentally inducing changes (e.g., via attention manipulation) to identify neuronal and behavioral consequences or studying spontaneous changes in ongoing brain dynamics. However, changes in internal processing could arise from many factors, such as variation in strategy or arousal, that are independent of experimental conditions. Here we utilize a data-driven clustering method based on modularity-maximation to identify consistent spatial-temporal EEG activity patterns across individual trials and relate this activity to behavioral performance. Subjects (N = 25) performed a motion direction discrimination task with six interleaved levels of motion coherence. Modularity-maximization based clustering identified two discrete spatial-temporal clusters, or subtypes, of trials with different patterns of brain activity. Surprisingly, even though Subtype 1 occurred more frequently with lower motion coherence, it was nonetheless associated with faster response times. Computational modeling suggests that Subtype 1 was characterized by a lower threshold for reaching a decision. These results highlight trial-to-trial variability in decision processes usually masked to experimenters and provide a method for identifying endogenous brain state variability relevant to cognition and behavior.