Why do valence asymmetries emerge in value learning? A reinforcement learning account.

The Value Learning Task (VLT; e.g., Raymond & O'Brien, 2009) is widely used to investigate how acquired value impacts how we perceive and process stimuli. The task consists of a series of trials in which participants attempt to maximize accumulated winnings as they make choices from a pair of presented images associated with probabilistic win, loss, or no-change outcomes. The probabilities and outcomes are initially unknown to the participant and thus the task involves decision making and learning under uncertainty. Despite the symmetric outcome structure for win and loss pairs, people learn win associations better than loss associations (Lin, Cabrera-Haro, & Reuter-Lorenz, 2020). This learning asymmetry could lead to differences when the stimuli are probed in subsequent tasks, compromising inferences about how acquired value affects downstream processing. We investigate the nature of the asymmetry using a standard error-driven reinforcement learning model with a softmax choice rule. Despite having no special role for valence, the model yields the learning asymmetry observed in human behavior, whether the model parameters are set to maximize empirical fit, or task payoff. The asymmetry arises from an interaction between a neutral initial value estimate and a choice policy that exploits while exploring, leading to more poorly discriminated value estimates for loss stimuli. We also show how differences in estimated individual learning rates help to explain individual differences in the observed win-loss asymmetries, and how the final value estimates produced by the model provide a simple account of a post-learning explicit value categorization task.

Rapid adaptation of predictive models during language comprehension: Aperiodic EEG slope, individual alpha frequency and idea density modulate individual differences in real-time model updating.

Predictive coding provides a compelling, unified theory of neural information processing, including for language. However, there is insufficient understanding of how predictive models adapt to changing contextual and environmental demands and the extent to which such adaptive processes differ between individuals. Here, we used electroencephalography (EEG) to track prediction error responses during a naturalistic language processing paradigm. In Experiment 1, 45 native speakers of English listened to a series of short passages. Via a speaker manipulation, we introduced changing intra-experimental adjective order probabilities for two-adjective noun phrases embedded within the passages and investigated whether prediction error responses adapt to reflect these intra-experimental predictive contingencies. To this end, we calculated a novel measure of speaker-based, intra-experimental surprisal ("speaker-based surprisal") as defined on a trial-by-trial basis and by clustering together adjectives with a similar meaning. N400 amplitude at the position of the critical second adjective was used as an outcome measure of prediction error. Results showed that N400 responses attuned to speaker-based surprisal over the course of the experiment, thus indicating that listeners rapidly adapt their predictive models to reflect local environmental contingencies (here: the probability of one type of adjective following another when uttered by a particular speaker). Strikingly, this occurs in spite of the wealth of prior linguistic experience that participants bring to the laboratory. Model adaptation effects were strongest for participants with a steep aperiodic (1/f) slope in resting EEG and low individual alpha frequency (IAF), with idea density (ID) showing a more complex pattern. These results were replicated in a separate sample of 40 participants in Experiment 2, which employed a highly similar design to Experiment 1. Overall, our results suggest that individuals with a steep aperiodic slope adapt their predictive models most strongly to context-specific probabilistic information. Steep aperiodic slope is thought to reflect low neural noise, which in turn may be associated with higher neural gain control and better cognitive control. Individuals with a steep aperiodic slope may thus be able to more effectively and dynamically reconfigure their prediction-related neural networks to meet current task demands. We conclude that predictive mechanisms in language are highly malleable and dynamic, reflecting both the affordances of the present environment as well as intrinsic information processing capabilities of the individual.

Icon arrays reduce concern over COVID-19 vaccine side effects: a randomized control study.

On April 13, 2021, the CDC announced that the administration of Johnson and Johnson's COVID-19 vaccine would be paused due to a rare blood clotting side effect in ~ 0.0001% of people given the vaccine. Most people who are hesitant to get a COVID-19 vaccine list potential side effects as their main concern (PEW, 2021); thus, it is likely that this announcement increased vaccine hesitancy among the American public. Two days after the CDC's announcement, we administered a survey to a group of 2,046 Americans to assess their changes in attitudes toward COVID-19 vaccines. The aim of this study was to investigate whether viewing icon arrays of side effect risk would prevent increases in COVID-19 vaccine hesitancy due to the announcement. We found that using icon arrays to illustrate the small chance of experiencing the blood clotting side effect significantly prevented increases in aversion toward the Johnson and Johnson vaccine as well as all other COVID-19 vaccines.

How well do ordinary Americans forecast the growth of COVID-19?

Across three experiments (N = 1565), we investigated how forecasts about the spread of COVID 19 are impacted by data trends, and whether patterns of misestimation predict adherence to social-distancing guidelines. We also investigated how mode of data presentation influences forecasting of future cases by showing participants data on the number of COVID-19 cases from a 5-week period in either graphical, tabular, or text-only form. We consistently found that people shown tables produced more accurate forecasts compared to people shown line-graphs of the same data; yet people shown line-graphs were more confident in their estimates. These findings suggest that graphs engender false-confidence in the accuracy of forecasts, that people's forecasts of future cases have important implications for their attitudes concerning social distancing, and that tables may be better than graphs for informing the public about the trajectory of COVID-19.

Rational use of episodic and working memory: A normative account of prospective memory.

Humans often simultaneously pursue multiple plans at different time scales, a capacity known as prospective memory (PM). The successful realization of non-immediate plans (e.g., post package after work) requires keeping track of a future plan while accomplishing other intermediate tasks (e.g., write a paper). Prospective memory capacity requires the integration of noisy evidence from perceptual input with evidence from both short-term working memory (WM) and long-term or episodic memory (LTM/EM). Here we formulate a set of empirical studies of prospective memory, all dual-task problems, as problems of computational rationality. We ask how a rational model should integrate noisy perceptual evidence and memory to maximize payoffs in these PM studies. The model combines reinforcement learning (optimal action selection) with evidence accumulation (optimal inference) in order to derive good decision parameters for optimal task performance (i.e., performing an ongoing task while monitoring for a cue that triggers executing a second prospective task). We compare model behavior to human behavioral evidence of key accuracy and reaction time phenomena in PM. Notably, our normative approach to theorizing and modeling these phenomena makes no assumptions about mechanisms of attention or retrieval. This approach can be extended to study the learning and use of meta-parameters governing the boundedly rational use of memory in planned action in health and disease. A computational psychiatry extension of the model can capture compensatory mnemonic strategies in neuropsychiatric disorders that may be rational responses to disturbances of inference, memory, and action selection.

Cognitively bounded rational analyses and the crucial role of theories of subjective utility.

We agree that combining rational analysis with cognitive bounds, what we previously introduced as Cognitively Bounded Rational Analysis, is a promising and under-used methodology in psychology. We further situate the framework in the literature, and highlight the important issue of a theory of subjective utility, which is not addressed sufficiently clearly in the framework or related previous work.

Grammatical predictions in Spanish-English bilinguals and Spanish-language learners.

Although bilingual individuals know 2 languages, research suggests that the languages are not separate in the mind. This is especially evident when a bilingual individual switches languages midsentence, indicating that mental representations are, to some degree, overlapping or integrated across the 2 languages. In 2 eye-tracking experiments, we investigated the nature of this integration during reading to examine whether incremental grammatical predictions generated by Spanish-English bilinguals (Experiment 1, N = 50) and Spanish-as-a-second-language learners (Experiment 2, N = 50) are language-specific or language-independent. As participants in same-language and mixed-language pairs performed a 2-string lexical-decision task, we measured eye fixation times on nouns in grammatical (determiner-noun) and ungrammatical (adverb-noun) contexts. In Experiment 1, bilingual participants read nouns faster following determiners than they read adverbs in both same- and mixed-language pairs, indicating that grammatical predictability in this context is language-independent. Surface-string bigram frequencies are unlikely to account for the results because the grammatical predictability effect was just as large for mixed-language (very low bigram frequency) as same-language (higher bigram frequency) pairs, and the effect was not modulated by the code-switching experience of participants. Experiment 2 found a similar, though nonsignificant, pattern for Spanish-language learners. When the data for Experiments 1 and 2 were combined, the effect of grammaticality did not interact with language congruency, participant group, or language proficiency, suggesting that both bilingual participants and language learners generated language-independent predictions. Our results support a bilingual model in which language-independent syntactic representations are involved in word-by-word, incremental syntactic processing, even within the most basic grammatical constituents. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

In Search of the Factors Behind Naive Sentence Judgments: A State Trace Analysis of Grammaticality and Acceptability Ratings.

We present a state-trace analysis of sentence ratings elicited by asking participants to evaluate the overall acceptability of a sentence and those elicited by asking participants to focus on structural well-formedness only. Appealing to literature on "grammatical illusion" sentences, we anticipated that a simple instruction manipulation might prompt people to apply qualitatively different kinds of judgment in the two conditions. Although differences consistent with the subjective experience of grammatical illusion dissociations were observed, the state trace analysis of the rating data indicates that responses were still consistent with both judgment types accessing a single underlying factor. These results add to the existing comparisons between analytic and probabilistic modeling approaches to predicting rating judgments.

The role of (bounded) optimization in theory testing and prediction.

We argue that a radically increased emphasis on (bounded) optimality can contribute to cognitive science by supporting prediction. Bounded optimality (computational rationality), an idea that borrowed from artificial intelligence, supports a priori behavioral prediction from constrained generative models of cognition. Bounded optimality thereby addresses serious failings with the logic and testing of descriptive models of perception and action.

Retrieval Interference in Syntactic Processing: The Case of Reflexive Binding in English.

It has been proposed that in online sentence comprehension the dependency between a reflexive pronoun such as himself/herself and its antecedent is resolved using exclusively syntactic constraints. Under this strictly syntactic search account, Principle A of the binding theory-which requires that the antecedent c-command the reflexive within the same clause that the reflexive occurs in-constrains the parser's search for an antecedent. The parser thus ignores candidate antecedents that might match agreement features of the reflexive (e.g., gender) but are ineligible as potential antecedents because they are in structurally illicit positions. An alternative possibility accords no special status to structural constraints: in addition to using Principle A, the parser also uses non-structural cues such as gender to access the antecedent. According to cue-based retrieval theories of memory (e.g., Lewis and Vasishth, 2005), the use of non-structural cues should result in increased retrieval times and occasional errors when candidates partially match the cues, even if the candidates are in structurally illicit positions. In this paper, we first show how the retrieval processes that underlie the reflexive binding are naturally realized in the Lewis and Vasishth (2005) model. We present the predictions of the model under the assumption that both structural and non-structural cues are used during retrieval, and provide a critical analysis of previous empirical studies that failed to find evidence for the use of non-structural cues, suggesting that these failures may be Type II errors. We use this analysis and the results of further modeling to motivate a new empirical design that we use in an eye tracking study. The results of this study confirm the key predictions of the model concerning the use of non-structural cues, and are inconsistent with the strictly syntactic search account. These results present a challenge for theories advocating the infallibility of the human parser in the case of reflexive resolution, and provide support for the inclusion of agreement features such as gender in the set of retrieval cues.

Why contextual preference reversals maximize expected value.

Contextual preference reversals occur when a preference for one option over another is reversed by the addition of further options. It has been argued that the occurrence of preference reversals in human behavior shows that people violate the axioms of rational choice and that people are not, therefore, expected value maximizers. In contrast, we demonstrate that if a person is only able to make noisy calculations of expected value and noisy observations of the ordinal relations among option features, then the expected value maximizing choice is influenced by the addition of new options and does give rise to apparent preference reversals. We explore the implications of expected value maximizing choice, conditioned on noisy observations, for a range of contextual preference reversal types-including attraction, compromise, similarity, and phantom effects. These preference reversal types have played a key role in the development of models of human choice. We conclude that experiments demonstrating contextual preference reversals are not evidence for irrationality. They are, however, a consequence of expected value maximization given noisy observations. (PsycINFO Database Record

Predicting Short-Term Remembering as Boundedly Optimal Strategy Choice.

It is known that, on average, people adapt their choice of memory strategy to the subjective utility of interaction. What is not known is whether an individual's choices are boundedly optimal. Two experiments are reported that test the hypothesis that an individual's decisions about the distribution of remembering between internal and external resources are boundedly optimal where optimality is defined relative to experience, cognitive constraints, and reward. The theory makes predictions that are tested against data, not fitted to it. The experiments use a no-choice/choice utility learning paradigm where the no-choice phase is used to elicit a profile of each participant's performance across the strategy space and the choice phase is used to test predicted choices within this space. They show that the majority of individuals select strategies that are boundedly optimal. Further, individual differences in what people choose to do are successfully predicted by the analysis. Two issues are discussed: (a) the performance of the minority of participants who did not find boundedly optimal adaptations, and (b) the possibility that individuals anticipate what, with practice, will become a bounded optimal strategy, rather than what is boundedly optimal during training.

Utility maximization and bounds on human information processing.

Utility maximization is a key element of a number of theoretical approaches to explaining human behavior. Among these approaches are rational analysis, ideal observer theory, and signal detection theory. While some examples of these approaches define the utility maximization problem with little reference to the bounds imposed by the organism, others start with, and emphasize approaches in which bounds imposed by the information processing architecture are considered as an explicit part of the utility maximization problem. These latter approaches are the topic of this issue of the journal.

Computational rationality: linking mechanism and behavior through bounded utility maximization.

We propose a framework for including information-processing bounds in rational analyses. It is an application of bounded optimality (Russell & Subramanian, 1995) to the challenges of developing theories of mechanism and behavior. The framework is based on the idea that behaviors are generated by cognitive mechanisms that are adapted to the structure of not only the environment but also the mind and brain itself. We call the framework computational rationality to emphasize the incorporation of computational mechanism into the definition of rational action. Theories are specified as optimal program problems, defined by an adaptation environment, a bounded machine, and a utility function. Such theories yield different classes of explanation, depending on the extent to which they emphasize adaptation to bounds, and adaptation to some ecology that differs from the immediate local environment. We illustrate this variation with examples from three domains: visual attention in a linguistic task, manual response ordering, and reasoning. We explore the relation of this framework to existing "levels" approaches to explanation, and to other optimality-based modeling approaches.

The adaptive nature of eye movements in linguistic tasks: how payoff and architecture shape speed-accuracy trade-offs.

We explore the idea that eye-movement strategies in reading are precisely adapted to the joint constraints of task structure, task payoff, and processing architecture. We present a model of saccadic control that separates a parametric control policy space from a parametric machine architecture, the latter based on a small set of assumptions derived from research on eye movements in reading (Engbert, Nuthmann, Richter, & Kliegl, 2005; Reichle, Warren, & McConnell, 2009). The eye-control model is embedded in a decision architecture (a machine and policy space) that is capable of performing a simple linguistic task integrating information across saccades. Model predictions are derived by jointly optimizing the control of eye movements and task decisions under payoffs that quantitatively express different desired speed-accuracy trade-offs. The model yields distinct eye-movement predictions for the same task under different payoffs, including single-fixation durations, frequency effects, accuracy effects, and list position effects, and their modulation by task payoff. The predictions are compared to-and found to accord with-eye-movement data obtained from human participants performing the same task under the same payoffs, but they are found not to accord as well when the assumptions concerning payoff optimization and processing architecture are varied. These results extend work on rational analysis of oculomotor control and adaptation of reading strategy (Bicknell & Levy, ; McConkie, Rayner, & Wilson, 1973; Norris, 2009; Wotschack, 2009) by providing evidence for adaptation at low levels of saccadic control that is shaped by quantitatively varying task demands and the dynamics of processing architecture.

In search of on-line locality effects in sentence comprehension.

Many comprehension theories assert that increasing the distance between elements participating in a linguistic relation (e.g., a verb and a noun phrase argument) increases the difficulty of establishing that relation during on-line comprehension. Such locality effects are expected to increase reading times and are thought to reveal properties and limitations of the short-term memory system that supports comprehension. Despite their theoretical importance and putative ubiquity, however, evidence for on-line locality effects is quite narrow linguistically and methodologically: It is restricted almost exclusively to self-paced reading of complex structures involving a particular class of syntactic relation. We present 4 experiments (2 self-paced reading and 2 eyetracking experiments) that demonstrate locality effects in the course of establishing subject-verb dependencies; locality effects are seen even in materials that can be read quickly and easily. These locality effects are observable in the earliest possible eye-movement measures and are of much shorter duration than previously reported effects. To account for the observed empirical patterns, we outline a processing model of the adaptive control of button pressing and eye movements. This model makes progress toward the goal of eliminating linking assumptions between memory constructs and empirical measures in favor of explicit theories of the coordinated control of motor responses and parsing.

Resolving semantic and proactive interference in memory over the short-term.

Interference is a major source of short-term errors of memory. The present investigation explores the relationship between two important forms of interference: proactive interference (PI), induced by the need to reject recently studied items no longer relevant to task performance, and semantic interference (SI), induced by the need to reject lures sharing a meaningful relationship with current memoranda. We explore the possibility that shared cognitive control processes are recruited to resolve both forms of interference. In Experiment 1, we find that the requirement to engage in articulatory suppression during the retention interval of tasks that induce either PI or SI increases both forms of interference similarly and selectively. In Experiment 2, we develop a task to examine PI and SI within the same experimental context. The results show interactive effects between factors that lead to the two forms of interference. Taken together, these findings support contextual-cuing models of short-term remembering (Nairne, Annual Review of Psychology, 53, 53-81 2002), where the context in which retrieval occurs can influence susceptibility to interference. Lastly, we discuss several theoretical hypotheses concerning the cognitive control processes that are recruited to resolve SI and PI in short-term remembering.

Divergent effects of different positive emotions on moral judgment.

Positive emotions are often treated as relatively similar in their cognitive-behavioral effects, and as having unambiguously beneficial consequences. For example, Valdesolo and DeSteno (2006) reported that a humorous video made people more prone to choose a utilitarian solution to a moral dilemma. They attributed this finding to increased positive affect. To determine whether such results actually stem in general from positive affect or from other more specific properties of humor, we conducted an experiment with moral dilemmas presented during an interleaved emotion-induction procedure involving mirth and another positive emotion, elevation. Mirth increased permissiveness for deontological violations, whereas elevation had the opposite effect. Furthermore, affective valence had no apparent independent influence on these judgments. Our results suggest that mirth and elevation have distinct cognitive consequences whose properties reflect their respective social functions, not their shared positive valence.

Rational adaptation under task and processing constraints: implications for testing theories of cognition and action.

The authors assume that individuals adapt rationally to a utility function given constraints imposed by their cognitive architecture and the local task environment. This assumption underlies a new approach to modeling and understanding cognition-cognitively bounded rational analysis-that sharpens the predictive acuity of general, integrated theories of cognition and action. Such theories provide the necessary computational means to explain the flexible nature of human behavior but in doing so introduce extreme degrees of freedom in accounting for data. The new approach narrows the space of predicted behaviors through analysis of the payoff achieved by alternative strategies, rather than through fitting strategies and theoretical parameters to data. It extends and complements established approaches, including computational cognitive architectures, rational analysis, optimal motor control, bounded rationality, and signal detection theory. The authors illustrate the approach with a reanalysis of an existing account of psychological refractory period (PRP) dual-task performance and the development and analysis of a new theory of ordered dual-task responses. These analyses yield several novel results, including a new understanding of the role of strategic variation in existing accounts of PRP and the first predictive, quantitative account showing how the details of ordered dual-task phenomena emerge from the rational control of a cognitive system subject to the combined constraints of internal variance, motor interference, and a response selection bottleneck.

In search of decay in verbal short-term memory.

Is forgetting in the short term due to decay with the mere passage of time, interference from other memoranda, or both? Past research on short-term memory has revealed some evidence for decay and a plethora of evidence showing that short-term memory is worsened by interference. However, none of these studies has directly contrasted decay and interference in short-term memory in a task that rules out the use of rehearsal processes. In this article the authors present a series of studies using a novel paradigm to address this problem directly, by interrogating the operation of decay and interference in short-term memory without rehearsal confounds. The results of these studies indicate that short-term memories are subject to very small decay effects with the mere passage of time but that interference plays a much larger role in their degradation. The authors discuss the implications of these results for existing models of memory decay and interference.