We don't know what you did last summer. On the importance of transparent reporting of reaction time data pre-processing.

In behavioral, cognitive, and social sciences, reaction time measures are an important source of information. However, analyses on reaction time data are affected by researchers' analytical choices and the order in which these choices are applied. The results of a systematic literature review, presented in this paper, revealed that the justification for and order in which analytical choices are conducted are rarely reported, leading to difficulty in reproducing results and interpreting mixed findings. To address this methodological shortcoming, we created a checklist on reporting reaction time pre-processing to make these decisions more explicit, improve transparency, and thus, promote best practices within the field. The importance of the pre-processing checklist was additionally supported by an expert consensus survey and a multiverse analysis. Consequently, we appeal for maximal transparency on all methods applied and offer a checklist to improve replicability and reproducibility of studies that use reaction time measures.

Need for Cognition is associated with a preference for higher task load in effort discounting.

When individuals set goals, they consider the subjective value (SV) of the anticipated reward and the required effort, a trade-off that is of great interest to psychological research. One approach to quantify the SVs of levels of difficulty of a cognitive task is the Cognitive Effort Discounting Paradigm by Westbrook and colleagues (2013). However, it fails to acknowledge the highly individual nature of effort, as it assumes a unidirectional, inverse relationship between task load and SVs. Therefore, it cannot map differences in effort perception that arise from traits like Need for Cognition, since individuals who enjoy effortful cognitive activities likely do not prefer the easiest level. We replicated the analysis of Westbrook and colleagues with an adapted version, the Cognitive and Affective Discounting (CAD) Paradigm. It quantifies SVs without assuming that the easiest level is preferred, thereby enabling the assessment of SVs for tasks without objective order of task load. Results show that many of the 116 participants preferred a more or the most difficult level. Variance in SVs was best explained by a declining logistic contrast of the [Formula: see text]-back levels and by the accuracy of responses, while reaction time as a predictor was highly volatile depending on the preprocessing pipeline. Participants with higher Need for Cognition scores perceived higher [Formula: see text]-back levels as less effortful and found them less aversive. Effects of Need for Cognition on SVs in lower levels did not reach significance, as group differences only emerged in higher levels. The CAD Paradigm appears to be well suited for assessing and analysing task preferences independent of the supposed objective task difficulty.Protocol registrationThe stage 1 protocol for this Registered Report was accepted in principle on August 19, 2022. The protocol, as accepted by the journal, can be found at: https://doi.org/10.17605/OSF.IO/CPXTH .

Cognitive effort investment: Does disposition become action?

Contrary to the law of less work, individuals with high levels of need for cognition and self-control tend to choose harder tasks more often. While both traits can be integrated into a core construct of dispositional cognitive effort investment, its relation to actual cognitive effort investment remains unclear. As individuals with high levels of cognitive effort investment are characterized by a high intrinsic motivation towards effortful cognition, they would be less likely to increase their effort based on expected payoff, but rather based on increasing demand. In the present study, we measured actual effort investment on multiple dimensions, i.e., subjective load, reaction time, accuracy, early and late frontal midline theta power, N2 and P3 amplitude, and pupil dilation. In a sample of N = 148 participants, we examined the relationship of dispositional cognitive effort investment and effort indices during a flanker and an n-back task with varying demand and payoff. Exploratorily, we examined this relationship for the two subdimensions cognitive motivation and effortful-self-control as well. In both tasks, effort indices were sensitive to demand and partly to payoff. The analyses revealed a main effect of cognitive effort investment for accuracy (n-back task), interaction effects with payoff for reaction time (n-back and flanker task) and P3 amplitude (n-back task) and demand for early frontal midline theta power (flanker task). Taken together, our results partly support the notion that individuals with high levels of cognitive effort investment exert effort more efficiently. Moreover, the notion that these individuals exert effort regardless of payoff is partly supported, too. This may further our understanding of the conditions under which person-situation interactions occur, i.e. the conditions under which situations determine effort investment in goal-directed behavior more than personality, and vice versa.

Dispositional individual differences in cognitive effort investment: establishing the core construct.

BACKGROUND: Individuals tend to avoid effortful tasks, regardless of whether they are physical or mental in nature. Recent experimental evidence is suggestive of individual differences in the dispositional willingness to invest cognitive effort in goal-directed behavior. The traits need for cognition (NFC) and self-control are related to behavioral measures of cognitive effort discounting and demand avoidance, respectively. Given that these traits are only moderately related, the question arises whether they reflect a common core factor underlying cognitive effort investment. If so, the common core of both traits might be related to behavioral measures of effort discounting in a more systematic fashion. To address this question, we aimed at specifying a core construct of cognitive effort investment that reflects dispositional differences in the willingness and tendency to exert effortful control. METHODS: We conducted two studies (N = 613 and N = 244) with questionnaires related to cognitive motivation and effort investment including assessment of NFC, intellect, self-control and effortful control. We first calculated Pearson correlations followed by two mediation models regarding intellect and its separate aspects, seek and conquer, as mediators. Next, we performed a confirmatory factor analysis of a hierarchical model of cognitive effort investment as second-order latent variable. First-order latent variables were cognitive motivation reflecting NFC and intellect, and effortful self-control reflecting self-control and effortful control. Finally, we calculated Pearson correlations between factor scores of the latent variables and general self-efficacy as well as traits of the Five Factor Model of Personality for validation purposes. RESULTS: Our findings support the hypothesized correlations between the assessed traits, where the relationship of NFC and self-control is specifically mediated via goal-directedness. We established and replicated a hierarchical factor model of cognitive motivation and effortful self-control that explains the shared variance of the first-order factors by a second-order factor of cognitive effort investment. CONCLUSIONS: Taken together, our results integrate disparate literatures on cognitive motivation and self-control and provide a basis for further experimental research on the role of dispositional individual differences in goal-directed behavior and cost-benefit-models.

Dispositional cognitive effort investment and behavioral demand avoidance: Are they related?

Individuals tend to avoid cognitive demand, yet, individual differences appear to exist. Recent evidence from two studies suggests that individuals high in the personality traits Self-Control and Need for Cognition that are related to the broader construct Cognitive Effort Investment are less prone to avoid cognitive demand and show less effort discounting. These findings suggest that cost-benefit models of decision-making that integrate the costs due to effort should consider individual differences in the willingness to exert mental effort. However, to date, there are almost no replication attempts of the above findings. For the present conceptual replication, we concentrated on the avoidance of cognitive demand and used a longitudinal design and latent state-trait modeling. This approach enabled us to separate the trait-specific variance in our measures of Cognitive Effort Investment and Demand Avoidance that is due to stable, individual differences from the variance that is due to the measurement occasion, the methods used, and measurement error. Doing so allowed us to test the assumption that self-reported Cognitive Effort Investment is related to behavioral Demand Avoidance more directly by relating their trait-like features to each other. In a sample of N = 217 participants, we observed both self-reported Cognitive Effort Investment and behavioral Demand Avoidance to exhibit considerable portions of trait variance. However, these trait variances were not significantly related to each other. Thus, our results call into question previous findings of a relationship between self-reported effort investment and demand avoidance. We suggest that novel paradigms are needed to emulate real-world effortful situations and enable better mapping between self-reported measures and behavioral markers of the willingness to exert cognitive effort.