Evaluating the robustness of parameter estimates in cognitive models: A meta-analytic review of multinomial processing tree models across the multiverse of estimation methods.

Researchers have become increasingly aware that data-analysis decisions affect results. Here, we examine this issue systematically for multinomial processing tree (MPT) models, a popular class of cognitive models for categorical data. Specifically, we examine the robustness of MPT model parameter estimates that arise from two important decisions: the level of data aggregation (complete-pooling, no-pooling, or partial-pooling) and the statistical framework (frequentist or Bayesian). These decisions span a multiverse of estimation methods. We synthesized the data from 13,956 participants (164 published data sets) with a meta-analytic strategy and analyzed the magnitude of divergence between estimation methods for the parameters of nine popular MPT models in psychology (e.g., process-dissociation, source monitoring). We further examined moderators as potential sources of divergence. We found that the absolute divergence between estimation methods was small on average (<.04; with MPT parameters ranging between 0 and 1); in some cases, however, divergence amounted to nearly the maximum possible range (.97). Divergence was partly explained by few moderators (e.g., the specific MPT model parameter, uncertainty in parameter estimation), but not by other plausible candidate moderators (e.g., parameter trade-offs, parameter correlations) or their interactions. Partial-pooling methods showed the smallest divergence within and across levels of pooling and thus seem to be an appropriate default method. Using MPT models as an example, we show how transparency and robustness can be increased in the field of cognitive modeling. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Associations Between Mental Health Problems in Adolescence and Educational Attainment in Early Adulthood: Results of the German Longitudinal BELLA Study.

Mental health problems (MHP) in adolescence are a major public health concern of the 21st century. Global prevalence estimates range between 10 and 20%. Most MHP manifest by adolescence and persistence rates are high, often accumulating further impairment in early adulthood and beyond. We analyzed data of N = 433 participants from the German longitudinal BELLA study to examine whether MHP in adolescence negatively affect educational attainment in early adulthood. Externalizing and internalizing MHP among adolescents aged 11-17 years were assessed at baseline using the Strengths and Difficulties Questionnaire. Educational attainment was assessed at the 6-year follow-up based on level of education, failure to attain the expected level of education, and dropout from vocational or academic training. Findings from logistic regression analyses suggest that more pronounced externalizing MHP in adolescence predict a lower level of education in early adulthood. We did not find a corresponding effect for internalizing MHP. Adolescents with higher-educated parents were less likely to attain a lower level of education themselves and less likely to fail in attaining their expected level of education. Our findings support that educational attainment presents a central channel for intergenerational reproduction of education and forms an important pathway for upward, but also downward social mobility. The current study emphasizes school as a central setting to implement measures to prevent onset and persistence of MHP and to foster equal opportunities in education.

The Development of Clustering in Episodic Memory: A Cognitive-Modeling Approach.

Younger children's free recall from episodic memory is typically less organized than recall by older children. To investigate if and how repeated learning opportunities help children use organizational strategies that improve recall, the authors analyzed category clustering across four study-test cycles. Seven-year-olds, 10-year-olds, and young adults (N = 150) studied categorically related words for a free-recall task. The cognitive processes underlying recall and clustering were measured with a multinomial model. The modeling revealed that developmental differences emerged particularly in the rate of learning to encode words as categorical clusters. The learning curves showed a common pattern across age groups, indicating developmental invariance. Memory for individual items also contributed to developmental differences and was the only factor driving 7-year-olds' moderate improvements in recall.

Use of the recognition heuristic depends on the domain's recognition validity, not on the recognition validity of selected sets of objects.

According to the recognition-heuristic theory, decision makers solve paired comparisons in which one object is recognized and the other not by recognition alone, inferring that recognized objects have higher criterion values than unrecognized ones. However, success-and thus usefulness-of this heuristic depends on the validity of recognition as a cue, and adaptive decision making, in turn, requires that decision makers are sensitive to it. To this end, decision makers could base their evaluation of the recognition validity either on the selected set of objects (the set's recognition validity), or on the underlying domain from which the objects were drawn (the domain's recognition validity). In two experiments, we manipulated the recognition validity both in the selected set of objects and between domains from which the sets were drawn. The results clearly show that use of the recognition heuristic depends on the domain's recognition validity, not on the set's recognition validity. In other words, participants treat all sets as roughly representative of the underlying domain and adjust their decision strategy adaptively (only) with respect to the more general environment rather than the specific items they are faced with.

The limited use of the fluency heuristic: Converging evidence across different procedures.

In paired comparisons based on which of two objects has the larger criterion value, decision makers could use the subjectively experienced difference in retrieval fluency of the objects as a cue. According to the fluency heuristic (FH) theory, decision makers use fluency-as indexed by recognition speed-as the only cue for pairs of recognized objects, and infer that the object retrieved more speedily has the larger criterion value (ignoring all other cues and information). Model-based analyses, however, have previously revealed that only a small portion of such inferences are indeed based on fluency alone. In the majority of cases, other information enters the decision process. However, due to the specific experimental procedures, the estimates of FH use are potentially biased: Some procedures may have led to an overestimated and others to an underestimated, or even to actually reduced, FH use. In the present article, we discuss and test the impacts of such procedural variations by reanalyzing 21 data sets. The results show noteworthy consistency across the procedural variations revealing low FH use. We discuss potential explanations and implications of this finding.

Individual differences in use of the recognition heuristic are stable across time, choice objects, domains, and presentation formats.

The recognition heuristic (RH) is a simple decision strategy that performs surprisingly well in many domains. According to the RH, people decide on the basis of recognition alone and ignore further knowledge when faced with a recognized and an unrecognized choice object. Previous research has revealed noteworthy individual differences in RH use, suggesting that people have preferences for using versus avoiding this strategy that might be causally linked to cognitive or personality traits. However, trying to explain differences in RH use in terms of traits presupposes temporal and cross-situational stability in use of the RH, an important prerequisite that has not been scrutinized so far. In a series of four experiments, we therefore assessed the stability in RH use across (1) time, (2) choice objects, (3) domains, and (4) presentation formats of the choice objects. In Experiment 1, participants worked on the same inference task and choice objects twice, separated by a delay of either one day or one week. Experiment 2 replicated Experiment 1 using two different object sets from the same domain, whereas Experiment 3 assessed the stability of RH use across two different domains. Finally, in Experiment 4 we investigated stability across verbal and pictorial presentation formats of the choice objects. For all measures of RH use proposed so far, we found strong evidence for both temporal and cross-situational stability in use of the RH. Thus, RH use at least partly reflects a person-specific style of decision making whose determinants await further research.

Whatever the cost? Information integration in memory-based inferences depends on cognitive effort.

One of the most prominent models of probabilistic inferences from memory is the simple recognition heuristic (RH). The RH theory assumes that judgments are based on recognition in isolation, such that other information is ignored. However, some prior research has shown that available knowledge is not generally ignored. In line with the notion of adaptive strategy selection--and, thus, a trade-off between accuracy and effort--we hypothesized that information integration crucially depends on how easily accessible information beyond recognition is, how much confidence decision makers have in this information, and how (cognitively) costly it is to acquire it. In three experiments, we thus manipulated (a) the availability of information beyond recognition, (b) the subjective usefulness of this information, and (c) the cognitive costs associated with acquiring this information. In line with the predictions, we found that RH use decreased substantially, the more easily and confidently information beyond recognition could be integrated, and increased substantially with increasing cognitive costs.