Exploring variability in risk taking with large language models.

What are the sources of individual-level differences in risk taking, and how do they depend on the domain or situation in which the decision is being made? Psychologists currently answer such questions with psychometric methods, which analyze correlations across participant responses in survey data sets. In this article, we analyze the preferences that give rise to these correlations. Our approach uses (a) large language models (LLMs) to quantify everyday risky behaviors in terms of the attributes or reasons that may describe those behaviors, and (b) decision models to map these attributes and reasons onto participant responses. We show that LLM-based decision models can explain observed correlations between behaviors in terms of the reasons different behaviors elicit and explain observed correlations between individuals in terms of the weights different individuals place on reasons, thereby providing a decision theoretic foundation for psychometric findings. Since LLMs can generate quantitative representations for nearly any naturalistic decision, they can be used to make accurate out-of-sample predictions for hundreds of everyday behaviors, predict the reasons why people may or may not want to engage in these behaviors, and interpret these reasons in terms of core psychological constructs. Our approach has important theoretical and practical implications for the study of heterogeneity in everyday behavior. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Memory modeling of counterfactual generation.

We use a computational model of memory search to study how people generate counterfactual outcomes in response to an established target outcome. Hierarchical Bayesian model fitting to data from six experiments reveals that counterfactual outcomes that are perceived as more desirable and more likely to occur are also more likely to come to mind and are generated earlier than other outcomes. Additionally, core memory mechanisms such as semantic clustering and word frequency biases have a strong influence on retrieval dynamics in counterfactual thinking. Finally, we find that the set of counterfactuals that come to mind can be manipulated by modifying the total number of counterfactuals that participants are prompted to generate, and our model can predict these effects. Overall, our findings demonstrate how computational memory search models can be integrated with current theories of counterfactual thinking to provide novel insights into the process of generating counterfactual thoughts. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Limitations to optimal search in naturalistic active learning.

Optimality in active learning is under intense debate in numerous disciplines. We introduce a new empirical paradigm for studying naturalistic active learning, as well as new computational tools for jointly modeling algorithmic and rational theories of information search. Participants in our task can ask questions and learn about hundreds of everyday items but must retrieve queried items from memory. To maximize information gain, participants need to retrieve sequences of dissimilar items. In eight experiments (N = 795), we find that participants are unable to do this. Instead, associative memory mechanisms lead to the successive retrieval of similar items, an established memory effect known as semantic congruence. The extent of semantic congruence (and thus suboptimality in question asking) is unaffected by task instructions and incentives, though participants can identify efficient query sequences when given a choice between query sequences. Overall, our results indicate that participants can distinguish between optimal and suboptimal search if explicitly asked to do so, but have difficulty implementing optimal search from memory. We conclude that associative memory processes may place critical restrictions on people's ability to ask good questions in naturalistic active learning tasks. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

A deep learning approach to personality assessment: Generalizing across items and expanding the reach of survey-based research.

Traditional methods of personality assessment, and survey-based research in general, cannot make inferences about new items that have not been surveyed previously. This limits the amount of information that can be obtained from a given survey. In this article, we tackle this problem by leveraging recent advances in statistical natural language processing. Specifically, we extract "embedding" representations of questionnaire items from deep neural networks, trained on large-scale English language data. These embeddings allow us to construct a high-dimensional space of items, in which linguistically similar items are located near each other. We combine item embeddings with machine learning algorithms to extrapolate participant ratings of personality items to completely new items that have not been rated by any participants. The accuracy of our approach is on par with incentivized human judges given an identical task, indicating that it predicts ratings of new personality items as accurately as people do. Our approach is also capable of identifying psychological constructs associated with questionnaire items and can accurately cluster items into their constructs based only on their language content. Overall, our results show how representations of linguistic personality descriptors obtained from deep language models can be used to model and predict a large variety of traits, scales, and constructs. In doing so, they showcase a new scalable and cost-effective method for psychological measurement. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Inductive reasoning in minds and machines.

Induction-the ability to generalize from existing knowledge-is the cornerstone of intelligence. Cognitive models of human induction are largely limited to toy problems and cannot make quantitative predictions for the thousands of different induction arguments that have been studied by researchers, or to the countless induction arguments that could be encountered in everyday life. Leading large language models (LLMs) go beyond toy problems but fail to mimic observed patterns of human induction. In this article, we combine rich knowledge representations obtained from LLMs with theories of human inductive reasoning developed by cognitive psychologists. We show that this integrative approach can capture several benchmark empirical findings on human induction and generate human-like responses to natural language arguments with thousands of common categories and properties. These findings shed light on the cognitive mechanisms at play in human induction and show how existing theories in psychology and cognitive science can be integrated with new methods in artificial intelligence, to successfully model high-level human cognition. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Semantic determinants of memorability.

We examine why some words are more memorable than others by using predictive machine learning models applied to word recognition and recall datasets. Our approach provides more accurate out-of-sample predictions for recognition and recall than previous psychological models, and outperforms human participants in new studies of memorability prediction. Our approach's predictive power stems from its ability to capture the semantic determinants of memorability in a data-driven manner. We identify which semantic categories are important for memorability and show that, unlike features such as word frequency that influence recognition and recall differently, the memorability of semantic categories is consistent across recognition and recall. Our paper sheds light on the complex psychological drivers of memorability, and in doing so illustrates the power of machine learning methods for psychological theory development.

Predicting implicit attitudes with natural language data.

Large-scale language datasets and advances in natural language processing offer opportunities for studying people's cognitions and behaviors. We show how representations derived from language can be combined with laboratory-based word norms to predict implicit attitudes for diverse concepts. Our approach achieves substantially higher correlations than existing methods. We also show that our approach is more predictive of implicit attitudes than are explicit attitudes, and that it captures variance in implicit attitudes that is largely unexplained by explicit attitudes. Overall, our results shed light on how implicit attitudes can be measured by combining standard psychological data with large-scale language data. In doing so, we pave the way for highly accurate computational modeling of what people think and feel about the world around them.

Information acquisition and decision strategies in intertemporal choice.

Intertemporal decision models describe choices between outcomes with different delays. While these models mainly focus on predicting choices, they make implicit assumptions about how people acquire and process information. A link between information processing and choice model predictions is necessary for a complete mechanistic account of decision making. We establish this link by fitting 18 intertemporal choice models to experimental datasets with both choice and information acquisition data. First, we show that choice models have highly correlated fits: people that behave according to one model also behave according to other models that make similar information processing assumptions. Second, we develop and fit an attention model to information acquisition data. Critically, the attention model parameters predict which type of intertemporal choice models best describes a participant's choices. Overall, our results relate attentional processes to models of intertemporal choice, providing a stepping stone towards a complete mechanistic account of intertemporal decision making.

Complex economic decisions from simple neurocognitive processes: the role of interactive attention.

Neurocognitive theories of value-based choice propose that people additively accumulate choice attributes when making decisions. These theories cannot explain the emergence of complex multiplicative preferences such as those assumed by prospect theory and other economic models. We investigate an interactive attention mechanism, according to which attention to attributes (like payoffs) depends on other attributes (like probabilities) attended to previously. We formalize this mechanism using a Markov attention model combined with an accumulator decision process, and test our model on eye-tracking and mouse-tracking data in risky choice. Our tests show that interactive attention is necessary to make good choices, that most participants display interactive attention and that allowing for interactive attention in accumulation-based decision models improves their predictions. By equipping established decision models with sophisticated attentional dynamics, we extend these models to describe complex economic choice, and in the process, we unify two prominent theoretical approaches to studying value-based decision making.

Free association in a neural network.

Free association among words is a fundamental and ubiquitous memory task. Although distributed semantics (DS) models can predict the association between pairs of words, and semantic network (SN) models can describe transition probabilities in free association data, there have been few attempts to apply established cognitive process models of memory search to free association data. Thus, researchers are currently unable to explain the dynamics of free association using memory mechanisms known to be at play in other retrieval tasks, such as free recall from lists. We address this issue using a popular neural network model of free recall, the context maintenance and retrieval (CMR) model, which we fit using stochastic gradient descent on a large data set of free association norms. Special cases of CMR mimic existing DS and SN models of free association, and we find that CMR outperforms these models on out-of-sample free association data. We also show that training CMR on free association data generates improved predictions for free recall from lists, demonstrating the value of free association for the study of many different types of memory phenomena. Overall, our analysis provides a new account of the dynamics of free association, predicts free association with increased accuracy, integrates theories of free association with established models of memory, and shows how large data sets and neural network training methods can be used to model complex cognitive processes that operate over thousands of representations. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Transformer networks of human conceptual knowledge.

We present a computational model capable of simulating aspects of human knowledge for thousands of real-world concepts. Our approach involves a pretrained transformer network that is further fine-tuned on large data sets of participant-generated feature norms. We show that such a model can successfully extrapolate from its training data, and predict human knowledge for new concepts and features. We apply our model to stimuli from 25 previous experiments in semantic cognition research and show that it reproduces many findings on semantic verification, concept typicality, feature distribution, and semantic similarity. We also compare our model against several variants, and by doing so, establish the model properties that are necessary for good prediction. The success of our approach shows how a combination of language data and (laboratory-based) psychological data can be used to build models with rich world knowledge. Such models can be used in the service of new psychological applications, such as the modeling of naturalistic semantic verification and knowledge retrieval, as well as the modeling of real-world categorization, decision-making, and reasoning. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Computational mechanisms for context-based behavioral interventions: A large-scale analysis.

Choice context influences decision processes and is one of the primary determinants of what people choose. This insight has been used by academics and practitioners to study decision biases and to design behavioral interventions to influence and improve choices. We analyzed the effects of context-based behavioral interventions on the computational mechanisms underlying decision-making. We collected data from two large laboratory studies involving 19 prominent behavioral interventions, and we modeled the influence of each intervention using a leading computational model of choice in psychology and neuroscience. This allowed us to parametrize the biases induced by each intervention, to interpret these biases in terms of underlying decision mechanisms and their properties, to quantify similarities between interventions, and to predict how different interventions alter key choice outcomes. In doing so, we offer researchers and practitioners a theoretically principled approach to understanding and manipulating choice context in decision-making.

Computational Methods for Predicting and Understanding Food Judgment.

People make subjective judgments about the healthiness of different foods every day, and these judgments in turn influence their food choices and health outcomes. Despite the importance of such judgments, there are few quantitative theories about their psychological underpinnings. This article introduces a novel computational approach that can approximate people's knowledge representations for thousands of common foods. We used these representations to predict how both lay decision-makers (the general population) and experts judge the healthiness of individual foods. We also applied our method to predict the impact of behavioral interventions, such as the provision of front-of-pack nutrient and calorie information. Across multiple studies with data from 846 adults, our models achieved very high accuracy rates (r2 = .65-.77) and significantly outperformed competing models based on factual nutritional content. These results illustrate how new computational methods applied to established psychological theory can be used to better predict, understand, and influence health behavior.

An ontology of decision models.

Decision models are essential theoretical tools in the study of choice behavior, but there is little consensus about the best model for describing choice, with different fields and different research programs favoring their own idiosyncratic sets of models. Even within a given field, decision models are seldom studied alongside each other, and insights obtained using 1 model are not typically generalized to others. We present the results of a large-scale computational analysis that uses landscaping techniques to generate a representational structure for describing decision models. Our analysis includes 89 prominent models of risky and intertemporal choice, and results in an ontology of decision models, interpretable in terms of model spaces, clusters, hierarchies, and graphs. We use this ontology to measure the properties of individual models and quantify the relationships between different models. Our results show how decades of quantitative research on human choice behavior can be synthesized within a single representational framework. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Process and content in decisions from memory.

Information stored in memory influences the formation of preferences and beliefs in most everyday decision tasks. The richness of this information, and the complexity inherent in interacting memory and decision processes, makes the quantitative model-driven analysis of such decisions very difficult. In this article we present a general framework that can address the theoretical and methodological barriers to building formal models of naturalistic memory-based decision making. Our framework implements established theories of memory search and decision making within a single integrated cognitive system, and uses computational language models to quantify the thoughts over which memory and decision processes operate. It can thus describe both the content of the information that is sampled from memory, as well as the processes involved in retrieving and evaluating this information in order to make a decision. Furthermore, our framework is tractable, and the parameters that characterize memory-based decisions can be recovered using thought listing and choice data from existing experimental tasks, and in turn be used to make quantitative predictions regarding choice probability, length of deliberation, retrieved thoughts, and the effects of decision context. We showcase the power and generality of our framework by applying it to naturalistic binary choices from domains such as risk perception, consumer behavior, financial decision making, ethical decision making, legal decision making, food choice, and social judgment. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The elusiveness of context effects in decision making.

Contextual features influence human and non-human decision making, giving rise to preference reversals. Decades of research have documented the species and situations in which these effects are observed. More recently, however, researchers have focused on boundary conditions, that is, settings in which established effects disappear or reverse. This work is scattered across academic disciplines and some results appear to contradict each other. We synthesize recent findings and resolve apparent contradictions by considering them in terms of three core categories of decision context: spatial arrangement, attribute concreteness, and deliberation time. We suggest that these categories could be understood using theories of choice representation, which specify how context shapes the information over which deliberation processes operate.

Similarity Judgment Within and Across Categories: A Comprehensive Model Comparison.

Similarity is one of the most important relations humans perceive, arguably subserving category learning and categorization, generalization and discrimination, judgment and decision making, and other cognitive functions. Researchers have proposed a wide range of representations and metrics that could be at play in similarity judgment, yet have not comprehensively compared the power of these representations and metrics for predicting similarity within and across different semantic categories. We performed such a comparison by pairing nine prominent vector semantic representations with seven established similarity metrics that could operate on these representations, as well as supervised methods for dimensional weighting in the similarity function. This approach yields a factorial model structure with 126 distinct representation-metric pairs, which we tested on a novel dataset of similarity judgments between pairs of cohyponymic words in eight categories. We found that cosine similarity and Pearson correlation were the overall best performing unweighted similarity functions, and that word vectors derived from free association norms often outperformed word vectors derived from text (including those specialized for similarity). Importantly, models that used human similarity judgments to learn category-specific weights on dimensions yielded substantially better predictions than all unweighted approaches across all types of similarity functions and representations, although dimension weights did not generalize well across semantic categories, suggesting strong category context effects in similarity judgment. We discuss implications of these results for cognitive modeling and natural language processing, as well as for theories of the representations and metrics involved in similarity.

What I like is what I remember: Memory modulation and preferential choice.

Memory is a crucial component of everyday decision making, yet little is known about how memory and choice processes interact and whether or not established memory regularities persist during memory-based decision making. In this paper, we introduce a novel experimental paradigm to study the differences between memory processes at play in standard list recall versus in preferential choice. Using computational memory models, fit to data from 2 preregistered experiments, we find that some established memory regularities (primacy, recency, semantic clustering) emerge in preferential choice, whereas others (temporal clustering) are significantly weakened relative to standard list recall. Notably, decision-relevant features, such as item desirability, play a stronger role in guiding retrieval in choice. Our results suggest memory processes differ across preferential choice and standard memory tasks, and that choice modulates memory by differentially activating decision-relevant features such as what we like. (PsycInfo Database Record (c) 2021 APA, all rights reserved).