Sensitivity to the Instrumental Value of Choice Increases Across Development.

Across development, people tend to demonstrate a preference for contexts in which they have the opportunity to make choices. However, it is not clear how children, adolescents, and adults learn to calibrate this preference based on the costs and benefits of agentic choice. Here, in both a primary, in-person, reinforcement-learning experiment (N = 92; age range = 10-25 years) and a preregistered online replication study (N = 150; age range = 8-25 years), we found that participants overvalued agentic choice but also calibrated their agency decisions to the reward structure of the environment, increasingly selecting agentic choice when choice had greater instrumental value. Regression analyses and computational modeling of participant choices revealed that participants' bias toward agentic choice-reflecting its intrinsic value-remained consistent across age, whereas sensitivity to the instrumental value of agentic choice increased from childhood to early adulthood.

Adolescents flexibly adapt action selection based on controllability inferences.

From early in life, we encounter both controllable environments, in which our actions can causally influence the reward outcomes we experience, and uncontrollable environments, in which they cannot. Environmental controllability is theoretically proposed to organize our behavior. In controllable contexts, we can learn to proactively select instrumental actions that bring about desired outcomes. In uncontrollable environments, Pavlovian learning enables hard-wired, reflexive reactions to anticipated, motivationally salient events, providing "default" behavioral responses. Previous studies characterizing the balance between Pavlovian and instrumental learning systems across development have yielded divergent findings, with some studies observing heightened expression of Pavlovian learning during adolescence and others observing a reduced influence of Pavlovian learning during this developmental stage. In this study, we aimed to investigate whether a theoretical model of controllability-dependent arbitration between learning systems might explain these seemingly divergent findings in the developmental literature, with the specific hypothesis that adolescents' action selection might be particularly sensitive to environmental controllability. To test this hypothesis, 90 participants, aged 8-27, performed a probabilistic-learning task that enables estimation of Pavlovian influence on instrumental learning, across both controllable and uncontrollable conditions. We fit participants' data with a reinforcement-learning model in which controllability inferences adaptively modulate the dominance of Pavlovian versus instrumental control. Relative to children and adults, adolescents exhibited greater flexibility in calibrating the expression of Pavlovian bias to the degree of environmental controllability. These findings suggest that sensitivity to environmental reward statistics that organize motivated behavior may be heightened during adolescence.

Novelty and uncertainty differentially drive exploration across development.

Across the lifespan, individuals frequently choose between exploiting known rewarding options or exploring unknown alternatives. A large body of work has suggested that children may explore more than adults. However, because novelty and reward uncertainty are often correlated, it is unclear how they differentially influence decision-making across development. Here, children, adolescents, and adults (ages 8-27 years, N = 122) completed an adapted version of a recently developed value-guided decision-making task that decouples novelty and uncertainty. In line with prior studies, we found that exploration decreased with increasing age. Critically, participants of all ages demonstrated a similar bias to select choice options with greater novelty, whereas aversion to reward uncertainty increased into adulthood. Computational modeling of participant choices revealed that whereas adolescents and adults demonstrated attenuated uncertainty aversion for more novel choice options, children's choices were not influenced by reward uncertainty.

Consensus design of a calibration experiment for human fear conditioning.

Fear conditioning is a widely used laboratory model to investigate learning, memory, and psychopathology across species. The quantification of learning in this paradigm is heterogeneous in humans and psychometric properties of different quantification methods can be difficult to establish. To overcome this obstacle, calibration is a standard metrological procedure in which well-defined values of a latent variable are generated in an established experimental paradigm. These intended values then serve as validity criterion to rank methods. Here, we develop a calibration protocol for human fear conditioning. Based on a literature review, series of workshops, and survey of N = 96 experts, we propose a calibration experiment and settings for 25 design variables to calibrate the measurement of fear conditioning. Design variables were chosen to be as theory-free as possible and allow wide applicability in different experimental contexts. Besides establishing a specific calibration procedure, the general calibration process we outline may serve as a blueprint for calibration efforts in other subfields of behavioral neuroscience that need measurement refinement.

Computational Mechanisms of Addiction and Anxiety: A Developmental Perspective.

A central goal of computational psychiatry is to identify systematic relationships between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. Most psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.

Neural effects of controllability as a key dimension of stress exposure.

Cross-species evidence suggests that the ability to exert control over a stressor is a key dimension of stress exposure that may sensitize frontostriatal-amygdala circuitry to promote more adaptive responses to subsequent stressors. The present study examined neural correlates of stressor controllability in young adults. Participants (N = 56; Mage = 23.74, range = 18-30 years) completed either the controllable or uncontrollable stress condition of the first of two novel stressor controllability tasks during functional magnetic resonance imaging (fMRI) acquisition. Participants in the uncontrollable stress condition were yoked to age- and sex-matched participants in the controllable stress condition. All participants were subsequently exposed to uncontrollable stress in the second task, which is the focus of fMRI analyses reported here. A whole-brain searchlight classification analysis revealed that patterns of activity in the right dorsal anterior insula (dAI) during subsequent exposure to uncontrollable stress could be used to classify participants' initial exposure to either controllable or uncontrollable stress with a peak of 73% accuracy. Previous experience of exerting control over a stressor may change the computations performed within the right dAI during subsequent stress exposure, shedding further light on the neural underpinnings of stressor controllability.

Real-World Exploration Increases Across Adolescence and Relates to Affect, Risk Taking, and Social Connectivity.

Cross-species research suggests that exploratory behaviors increase during adolescence and relate to the social, affective, and risky behaviors characteristic of this developmental stage. However, how these typical adolescent behaviors manifest and relate in real-world settings remains unclear. Using geolocation tracking to quantify exploration-variability in daily movement patterns-over a 3-month period in 58 adolescents and adults (ages 13-27) in New York City, we investigated whether daily exploration varied with age and whether exploration related to social connectivity, risk taking, and momentary positive affect. In our cross-sectional sample, we found an association between daily exploration and age, with individuals near the transition to legal adulthood exhibiting the highest exploration levels. Days of higher exploration were associated with greater positive affect irrespective of age. Higher mean exploration was associated with greater social connectivity in all participants but was linked to higher risk taking selectively among adolescents. Our results highlight the interplay of exploration and socioemotional behaviors across development and suggest that societal norms may modulate their expression in naturalistic contexts.

Reward-motivated memories influence new learning across development.

Previously rewarding experiences can influence choices in new situations. Past work has demonstrated that existing reward associations can either help or hinder future behaviors and that there is substantial individual variability in the transfer of value across contexts. Developmental changes in reward sensitivity may also modulate the impact of prior reward associations on later goal-directed behavior. The current study aimed to characterize how reward associations formed in the past affected learning in the present from childhood to adulthood. Participants completed a reinforcement learning paradigm using high- and low-reward stimuli from a task completed 24 h earlier, as well as novel stimuli, as choice options. We found that prior high-reward associations impeded learning across all ages. We then assessed how individual differences in the prioritization of high- versus low-reward associations in memory impacted new learning. Greater high-reward memory prioritization was associated with worse learning performance for previously high-reward relative to low-reward stimuli across age. Adolescents also showed impeded early learning regardless of individual differences in high-reward memory prioritization. Detrimental effects of previous reward on choice behavior did not persist beyond learning. These findings indicate that prior reward associations proactively interfere with future learning from childhood to adulthood and that individual differences in reward-related memory prioritization influence new learning across age.

How do natural environments shape adaptive cognition across the lifespan?

How does human cognition adapt to idiosyncratic features of our real-world experiences across our lifetimes? The dynamic interaction between individuals and their natural environments is rarely the focus of study within cognitive science, but I argue that a more ecological approach will be critical for advancing developmental science and revealing the adaptive nature of cognition.

Reward Enhances Memory via Age-Varying Online and Offline Neural Mechanisms across Development.

Reward motivation enhances memory through interactions between mesolimbic, hippocampal, and cortical systems, both during and after encoding. Developmental changes in these distributed neural circuits may lead to age-related differences in reward-motivated memory and the underlying neural mechanisms. Converging evidence from cross-species studies suggests that subcortical dopamine signaling is increased during adolescence, which may lead to stronger memory representations of rewarding, relative to mundane, events and changes in the contributions of underlying subcortical and cortical brain mechanisms across age. Here, we used fMRI to examine how reward motivation influences the "online" encoding and "offline" postencoding brain mechanisms that support long-term associative memory from childhood to adulthood in human participants of both sexes. We found that reward motivation led to both age-invariant enhancements and nonlinear age-related differences in associative memory after 24 h. Furthermore, reward-related memory benefits were linked to age-varying neural mechanisms. During encoding, interactions between the prefrontal cortex (PFC) and ventral tegmental area (VTA) were associated with better high-reward memory to a greater degree with increasing age. Preencoding to postencoding changes in functional connectivity between the anterior hippocampus and VTA were also associated with better high-reward memory, but more so at younger ages. Our findings suggest that there may be developmental differences in the contributions of offline subcortical and online cortical brain mechanisms supporting reward-motivated memory.SIGNIFICANCE STATEMENT A substantial body of research has examined the neural mechanisms through which reward influences memory formation in adults. However, despite extensive evidence that both reward processing and associative memory undergo dynamic change across development, few studies have examined age-related changes in these processes. We found both age-invariant and nonlinear age-related differences in reward-motivated memory. Moreover, our findings point to developmental differences in the processes through which reward modulates the prioritization of information in long-term memory, with greater early reliance on offline subcortical consolidation mechanisms and increased contribution of systems-level online encoding circuitry with increasing age. These results highlight dynamic developmental changes in the cognitive and neural mechanisms through which motivationally salient information is prioritized in memory from childhood to adulthood.

Reinforcement learning with associative or discriminative generalization across states and actions: fMRI at 3 T and 7 T.

The model-free algorithms of "reinforcement learning" (RL) have gained clout across disciplines, but so too have model-based alternatives. The present study emphasizes other dimensions of this model space in consideration of associative or discriminative generalization across states and actions. This "generalized reinforcement learning" (GRL) model, a frugal extension of RL, parsimoniously retains the single reward-prediction error (RPE), but the scope of learning goes beyond the experienced state and action. Instead, the generalized RPE is efficiently relayed for bidirectional counterfactual updating of value estimates for other representations. Aided by structural information but as an implicit rather than explicit cognitive map, GRL provided the most precise account of human behavior and individual differences in a reversal-learning task with hierarchical structure that encouraged inverse generalization across both states and actions. Reflecting inference that could be true, false (i.e., overgeneralization), or absent (i.e., undergeneralization), state generalization distinguished those who learned well more so than action generalization. With high-resolution high-field fMRI targeting the dopaminergic midbrain, the GRL model's RPE signals (alongside value and decision signals) were localized within not only the striatum but also the substantia nigra and the ventral tegmental area, including specific effects of generalization that also extend to the hippocampus. Factoring in generalization as a multidimensional process in value-based learning, these findings shed light on complexities that, while challenging classic RL, can still be resolved within the bounds of its core computations.

Developmental shifts in computations used to detect environmental controllability.

Accurate assessment of environmental controllability enables individuals to adaptively adjust their behavior-exploiting rewards when desirable outcomes are contingent upon their actions and minimizing costly deliberation when their actions are inconsequential. However, it remains unclear how estimation of environmental controllability changes from childhood to adulthood. Ninety participants (ages 8-25) completed a task that covertly alternated between controllable and uncontrollable conditions, requiring them to explore different actions to discover the current degree of environmental controllability. We found that while children were able to distinguish controllable and uncontrollable conditions, accuracy of controllability assessments improved with age. Computational modeling revealed that whereas younger participants' controllability assessments relied on evidence gleaned through random exploration, older participants more effectively recruited their task structure knowledge to make highly informative interventions. Age-related improvements in working memory mediated this qualitative shift toward increased use of an inferential strategy. Collectively, these findings reveal an age-related shift in the cognitive processes engaged to assess environmental controllability. Improved detection of environmental controllability may foster increasingly adaptive behavior over development by revealing when actions can be leveraged for one's benefit.

Flexibility in valenced reinforcement learning computations across development.

Optimal integration of positive and negative outcomes during learning varies depending on an environment's reward statistics. The present study investigated the extent to which children, adolescents, and adults (N = 142 8-25 year-olds, 55% female, 42% White, 31% Asian, 17% mixed race, and 8% Black; data collected in 2021) adapt their weighting of better-than-expected and worse-than-expected outcomes when learning from reinforcement. Participants made choices across two contexts: one in which weighting positive outcomes more heavily than negative outcomes led to better performance, and one in which the reverse was true. Reinforcement learning modeling revealed that across age, participants shifted their valence biases in accordance with environmental structure. Exploratory analyses revealed strengthening of context-dependent flexibility with increasing age.

Skin conductance levels and responses in Asian and White participants during fear conditioning✰.

Fear conditioning paradigms are frequently used in the translational study of anxiety and fear-related disorders. Accordingly, it is important to understand whether the measurement of fear conditioning responses is systematically influenced by an individual's race. Studies have found increased pain sensitivity and smaller physiological startle responses in Asian individuals, compared to White individuals; to our knowledge, no studies have evaluated whether skin conductance response (SCR) outcomes differ between Asian and White individuals. In a series of secondary data analyses, we investigated potential differences in skin conductance level (SCL), orienting SCR, unconditioned SCR, SCR to CS+ and CS-, differential SCR, and differential SCR non-responder status. In sample 1, Asian participants (n = 97) demonstrated a significantly smaller mean differential SCR compared to White participants (n = 86). No other between group differences were observed. In sample 2, there was no difference in mean differential SCR between Asian (n = 52) and White (n = 62) participants, although more Asian participants failed to show adequate skin conductance levels for study entry. To our knowledge, this is the first study to evaluate differences between Asian and White samples using skin conductance outcomes in a fear conditioning paradigm. We detected only subtle evidence for SCR differences between Asian and White samples, unlikely to reach significance outside large studies.

Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory.

As individuals learn through trial and error, some are more influenced by good outcomes, while others weight bad outcomes more heavily. Such valence biases may also influence memory for past experiences. Here, we examined whether valence asymmetries in reinforcement learning change across adolescence, and whether individual learning asymmetries bias the content of subsequent memory. Participants ages 8-27 learned the values of 'point machines,' after which their memory for trial-unique images presented with choice outcomes was assessed. Relative to children and adults, adolescents overweighted worse-than-expected outcomes during learning. Individuals' valence biases modulated incidental memory, such that those who prioritized worse- (or better-) than-expected outcomes during learning were also more likely to remember images paired with these outcomes, an effect reproduced in an independent dataset. Collectively, these results highlight age-related changes in the computation of subjective value and demonstrate that a valence-asymmetric valuation process influences how information is prioritized in episodic memory.

Developmental change in prefrontal cortex recruitment supports the emergence of value-guided memory.

Prioritizing memory for valuable information can promote adaptive behavior across the lifespan, but it is unclear how the neurocognitive mechanisms that enable the selective acquisition of useful knowledge develop. Here, using a novel task coupled with functional magnetic resonance imaging, we examined how children, adolescents, and adults (N = 90) learn from experience what information is likely to be rewarding, and modulate encoding and retrieval processes accordingly. We found that the ability to use learned value signals to selectively enhance memory for useful information strengthened throughout childhood and into adolescence. Encoding and retrieval of high- vs. low-value information was associated with increased activation in striatal and prefrontal regions implicated in value processing and cognitive control. Age-related increases in value-based lateral prefrontal cortex modulation mediated the relation between age and memory selectivity. Our findings demonstrate that developmental increases in the strategic engagement of the prefrontal cortex support the emergence of adaptive memory.

Associative memory persistence in 3- to 5-year-olds.

Adults struggle to recollect episodic memories from early life. This phenomenon-referred to as "infantile" and "childhood amnesia"-has been widely observed across species and is characterized by rapid forgetting from birth until early childhood. While a number of studies have focused on infancy, few studies have examined the persistence of memory for newly learned associations during the putative period of childhood amnesia. In this study, we investigated forgetting in 137 children ages 3-5 years old by using an interactive storybook task. We assessed associative memory between subjects after 5-min, 24-h, and 1-week delay periods. Across all delays, we observed a significant increase in memory performance with age. While all ages demonstrated above-chance memory performance after 5-min and 24-h delays, we observed chance-level memory accuracy in 3-year-olds following a 1-week delay. The observed age differences in associative memory support the proposal that hippocampal-dependent memory systems undergo rapid development during the preschool years. These data have the potential to inform future work translating memory persistence and malleability research from rodent models to humans by establishing timescales at which we expect young children to forget newly learned associations.

The rational use of causal inference to guide reinforcement learning strengthens with age.

Beliefs about the controllability of positive or negative events in the environment can shape learning throughout the lifespan. Previous research has shown that adults' learning is modulated by beliefs about the causal structure of the environment such that they update their value estimates to a lesser extent when the outcomes can be attributed to hidden causes. This study examined whether external causes similarly influenced outcome attributions and learning across development. Ninety participants, ages 7 to 25 years, completed a reinforcement learning task in which they chose between two options with fixed reward probabilities. Choices were made in three distinct environments in which different hidden agents occasionally intervened to generate positive, negative, or random outcomes. Participants' beliefs about hidden-agent intervention aligned with the true probabilities of the positive, negative, or random outcome manipulation in each of the three environments. Computational modeling of the learning data revealed that while the choices made by both adults (ages 18-25) and adolescents (ages 13-17) were best fit by Bayesian reinforcement learning models that incorporate beliefs about hidden-agent intervention, those of children (ages 7-12) were best fit by a one learning rate model that updates value estimates based on choice outcomes alone. Together, these results suggest that while children demonstrate explicit awareness of the causal structure of the task environment, they do not implicitly use beliefs about the causal structure of the environment to guide reinforcement learning in the same manner as adolescents and adults.

Beyond the Stimulus: A Neurohumanities Approach to Language, Music, and Emotion.

The brain basis of language, music, and emotion can be studied from the perspective of the psychological and cognitive sciences. Does this approach link to concerns of the humanities meaningfully? We outline prospects of developing a genuine neurohumanities research program.

Causal Information-Seeking Strategies Change Across Childhood and Adolescence.

Intervening on causal systems can illuminate their underlying structures. Past work has shown that, relative to adults, young children often make intervention decisions that appear to confirm a single hypothesis rather than those that optimally discriminate alternative hypotheses. Here, we investigated how the ability to make informative causal interventions changes across development. Ninety participants between the ages of 7 and 25 completed 40 different puzzles in which they had to intervene on various causal systems to determine their underlying structures. Each puzzle comprised a three- or four-node computer chip with hidden wires. On each trial, participants viewed two possible arrangements of the chip's hidden wires and had to select a single node to activate. After observing the outcome of their intervention, participants selected a wire configuration and rated their confidence in their selection. We characterized participant choices with a Bayesian measurement model that indexed the extent to which participants selected nodes that would best disambiguate the two possible causal structures versus those that had high causal centrality in one of the two causal hypotheses but did not necessarily discriminate between them. Our model estimates revealed that the use of a discriminatory strategy increased through early adolescence. Further, developmental improvements in intervention strategy were related to changes in the ability to accurately judge the strength of evidence that interventions revealed, as indexed by participants' confidence in their selections. Our results suggest that improvements in causal information-seeking extend into adolescence and may be driven by metacognitive sensitivity to the efficacy of previous interventions in discriminating competing ideas.