Better Accuracy for Better Science . . . Through Random Conclusions.

Conducting research with human subjects can be difficult because of limited sample sizes and small empirical effects. We demonstrate that this problem can yield patterns of results that are practically indistinguishable from flipping a coin to determine the direction of treatment effects. We use this idea of random conclusions to establish a baseline for interpreting effect-size estimates, in turn producing more stringent thresholds for hypothesis testing and for statistical-power calculations. An examination of recent meta-analyses in psychology, neuroscience, and medicine confirms that, even if all considered effects are real, results involving small effects are indeed indistinguishable from random conclusions.

The Strengths and Weaknesses of Crowds to Address Global Problems.

Global climate change, the COVID-19 pandemic, and the spread of misinformation on social media are just a handful of highly consequential problems affecting society. We argue that the rough contours of many societal problems can be framed within a "wisdom of crowds" perspective. Such a framing allows researchers to recast complex problems within a simple conceptual framework and leverage known results on crowd wisdom. To this end, we present a simple "toy" model of the strengths and weaknesses of crowd wisdom that easily maps to many societal problems. Our model treats the judgments of individuals as random draws from a distribution intended to represent a heterogeneous population. We use a weighted mean of these individuals to represent the crowd's collective judgment. Using this setup, we show that subgroups have the potential to produce substantively different judgments and we investigate their effect on a crowd's ability to generate accurate judgments about societal problems. We argue that future work on societal problems can benefit from more sophisticated, domain-specific theory and models based on the wisdom of crowds.

A Bayesian nonparametric approach for handling item and examinee heterogeneity in assessment data.

We propose a novel nonparametric Bayesian item response theory model that estimates clusters at the question level, while simultaneously allowing for heterogeneity at the examinee level under each question cluster, characterized by a mixture of binomial distributions. The main contribution of this work is threefold. First, we present our new model and demonstrate that it is identifiable under a set of conditions. Second, we show that our model can correctly identify question-level clusters asymptotically, and the parameters of interest that measure the proficiency of examinees in solving certain questions can be estimated at a n rate (up to a log term). Third, we present a tractable sampling algorithm to obtain valid posterior samples from our proposed model. Compared to the existing methods, our model manages to reveal the multi-dimensionality of the examinees' proficiency level in handling different types of questions parsimoniously by imposing a nested clustering structure. The proposed model is evaluated via a series of simulations as well as apply it to an English proficiency assessment data set. This data analysis example nicely illustrates how our model can be used by test makers to distinguish different types of students and aid in the design of future tests.

Retrospective self-reports of sensitivity to the effects of alcohol: Trait-like stability and concomitant changes with alcohol involvement.

OBJECTIVE: Lower sensitivity to the acute effects of alcohol is known to confer risk for the development of alcohol use disorder. Alcohol sensitivity, or level of response to alcohol's subjective effects, is heritable but also can change as a result of persistent alcohol exposure (i.e., acquired tolerance). Here, we examined how changes over time in four indices of alcohol involvement affected scores on two validated, retrospective self-report measures of alcohol response-the Self-Rating of the Effects of Alcohol (SRE) form and the Alcohol Sensitivity Questionnaire (ASQ)-in a sample of emerging adult drinkers. METHOD: Participants (N = 173; Mage = 19.5 years; 60% assigned female at birth) completed the ASQ, SRE, and measures of alcohol use and problems at two time points separated by a median of 0.77 years (range: 0.30-2.54 years). RESULTS: Multiple linear regression showed that increases in drinking over this period accounted for increases in SRE and ASQ scores (i.e., in reported numbers of drinks needed to experience subjective effects of alcohol). Increased drinking accounted for more variance in the number of drinks needed to experience lighter drinking versus heavier drinking effects, and increases in the number of drinks consumed per occasion had a larger effect than did changes in total numbers of drinks consumed, number of binge-drinking occasions, or drinking-related problems. CONCLUSIONS: Findings suggest that both SRE and ASQ capture some stable, trait-like variability in alcohol response as well as some state-dependent, within-person variability in alcohol response acquired through increases in alcohol involvement. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A trauma-informed approach to understanding firearm decision-making among Black adolescents: Implications for prevention.

Firearm violence remains a public health crisis in marginalized, urban communities, with Black adolescents bearing the burden of firearm homicides and injuries. As such, the prevention of firearm violence among adolescents has moved to a high priority of the U.S. public health agenda. The current paper reviews recent literature to highlight the heterogeneity in firearm behavior among Black adolescents and underscore the need for additional research on decision-making and firearm behavior to better understand how adolescents make decisions to acquire, carry, and use firearms. Through a discussion of the disproportionate levels of trauma exposure and trauma symptoms experienced by Black adolescents, the current paper also proposes a trauma-informed approach to understanding decision-making for risky firearm behavior. We discuss the broader impacts of this approach, including the development of a more comprehensive and contextually relevant understanding of the variability in risky firearm behavior and improvements in risk screening capabilities and preventive intervention strategies.

Decision strategies while intoxicated relate to alcohol-impaired driving attitudes and intentions.

OBJECTIVE: Approximately 28 million individuals engage in alcohol-impaired driving (AID) every year. This study investigated individuals' AID decision making strategies under intoxication, their variability across the breath alcohol concentration (BrAC) curve, and the association between strategy and AID attitudes, intentions, and behavior. METHOD: Seventy-nine adults (mean 23.9 years, 57% female) who drank alcohol ≥2 days per week and lived >2 miles away from their typical drinking locations completed an alcohol administration protocol and AID decision making task. AID attitudes, intentions, and behaviors were assessed repeatedly across the BrAC curve. Bayesian cognitive modeling identified decision strategies used by individuals on the AID decision making task, revealing whether alcohol consumption level and/or ride service cost factored into individuals' decisions to drive while impaired or obtain a ride. Additional analyses tested whether AID attitudes and intentions were related to individuals' decision strategies. RESULTS: Two decision strategies were examined on the ascending and descending limbs of the BrAC curve: compensatory (both consumption level and ride service cost factored into AID decisions) and non-compensatory (only consumption level factored into AID decisions). Switching to a compensatory strategy on the descending limb was associated with lower perceived intoxication, perceiving AID as less dangerous, and being willing to drive above the legal BrAC limit. CONCLUSIONS: Results suggest that risk for engaging in AID is higher for those using a cost-sensitive, compensatory strategy when making AID decisions under intoxication. Future research is needed to test whether AID countermeasures (e.g., subsidized ride services) are differentially effective according to decision strategy type. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The Problem of Coordination and the Pursuit of Structural Constraints in Psychology.

Paul Meehl's famous critique detailed many of the problematic practices and conceptual confusions that stand in the way of meaningful theoretical progress in psychological science. By integrating many of Meehl's points, we argue that one of the reasons for the slow progress in psychology is the failure to acknowledge the problem of coordination. This problem arises whenever we attempt to measure quantities that are not directly observable but can be inferred from observable variables. The solution to this problem is far from trivial, as demonstrated by a historical analysis of thermometry. The key challenge is the specification of a functional relationship between theoretical concepts and observations. As we demonstrate, empirical means alone cannot determine this relationship. In the case of psychology, the problem of coordination has dramatic implications in the sense that it severely constrains our ability to make meaningful theoretical claims. We discuss several examples and outline some of the solutions that are currently available.

Applying Bayesian cognitive models to decisions to drive after drinking.

BACKGROUND AND AIMS: Despite widespread negative perceptions, the prevalence of alcohol-impaired driving (AID) in the United States remains unacceptably high. This study used a novel decision task to evaluate whether individuals considered both ride service cost and alcohol consumption level when deciding whether or not to drive, and whether the resulting strategy was associated with engagement in AID. DESIGN: A two-sample study, where sample 1 developed a novel AID decision task to classify participants by decision strategy. Sample 2 was used to cross-validate the task and examine whether decision strategy classifications were predictive of prior reported AID behavior. SETTING: A laboratory setting at the University of Missouri, USA. PARTICIPANTS: Sample 1 included 38 student participants from introductory psychology classes at the University of Missouri. Sample 2 included 67 young adult participants recruited from the local community. MEASUREMENTS: We developed a decision task that presented hypothetical drinking scenarios that varied in quantity of alcohol consumption (one to six drinks) and the cost of a ride service ($5-25). We applied a Bayesian computational model to classify choices as consistent with either: integrating both ride cost and consumption level (compensatory) or considering only consumption level (non-compensatory) when making hypothetical AID decisions. In sample 2, we assessed established AID risk factors (sex, recent alcohol consumption, perceived safe limit) and recent (past 3 months) engagement in AID. FINDINGS: In sample 1, the majority of participants were classified as using decision strategies consistent with either a compensatory or non-compensatory process. Results from sample 2 replicated the overall classification rate and demonstrated that participants who used a compensatory strategy were more likely to report recent AID, even after accounting for study covariates. CONCLUSIONS: In a hypothetical alcohol-impaired driving (AID) decision task, individuals who considered both consumption level and ride service cost were more likely to report recent AID than those who made decisions based entirely on consumption level.

Evaluations of CRC2631 toxicity, tumor colonization, and genetic stability in the TRAMP prostate cancer model.

Conventional cancer chemotherapies are not fully efficacious and do not target tumors, leading to significant treatment-related morbidities. A number of genetically attenuated cancer-targeting bacteria are being developed to safely target tumors in vivo. Here we report the toxicological, tumor-targeting, and efficacy profiles of Salmonella enterica serovar Typhimurium CRC2631 in a syngeneic and autochthonous TRAMP model of aggressive prostate cancer. CRC2631 preferentially colonize primary and metastatic tumors in the TRAMP animals. In addition, longitudinal whole genome sequencing studies of CRC2631 recovered from prostate tumor tissues demonstrate that CRC2631 is genetically stable. Moreover, tumor-targeted CRC2631 generates an anti-tumor immune response. Combination of CRC2631 with checkpoint blockade reduces metastasis burden. Collectively, these findings demonstrate a potential for CRC2631 in cancer immunotherapy strategies.

Young Adults Make Rational Sexual Decisions.

We examined risky sexual choice under the lens of rational decision-making. Participants (N = 257) completed a novel sexual-choice task in which they selected from among hypothetical sexual partners varying in physical attractiveness and in the probability that one would contract a sexually transmitted infection (STI) from a one-time sexual encounter with them. We found that nearly all participants evaluated the sexual-choice alternatives in a coherent fashion consistent with utility-based theories of rational choice. In subsequent analyses, we classified participants' responses according to whether their sexual preferences were based on maximizing attractiveness or minimizing the risk of STIs. Finally, we established an association between sexual choice in our task and reported real-world sexual risk-taking.

Context, acute tolerance, and subjective response affect alcohol-impaired driving decisions.

RATIONALE: Alcohol intoxication produces effects that can impair judgment and increase engagement in risky behaviors, including alcohol-impaired driving (AID). Real-world AID decisions are informed by contextual circumstances and judgments of associated risk. How individuals vary in their AID decision-making across contexts and whether subjective alcohol responses (stimulation, sedation, acute tolerance) differentially affect AID decisions are critical, but under-studied research questions. OBJECTIVES: We systematically investigated predictors of AID decisions at different hypothetical driving distances across the blood alcohol concentration (BAC) curve. METHODS: Young adults (n = 40; 55% female) completed two laboratory sessions in a within-subjects alcohol/placebo design. At multiple points along the BAC curve (M peak BAC = 0.101 g%), participants rated their subjective intoxication, stimulation, sedation, and perceived dangerousness of driving prior to indicating their willingness to drive distances of 1, 3, and 10 miles. Multilevel mixed models assessed within- and between-person predictors of the maximum distance participants were willing to drive at matched BACs on the ascending and descending limb. RESULTS: Under intoxication (but not placebo), participants were willing to drive greater distances on the descending versus ascending limb. At the momentary level, participants were willing to drive further when they felt less intoxicated, stimulated, and sedated, and perceived driving as less dangerous. CONCLUSIONS: Individuals differed in the distance they were willing to drive as a function of indicators of intoxication, implicating driving distance as an important contextual factor relevant to AID decisions. Individuals may simultaneously perceive themselves as "unsafe" to drive, but "safe enough" to drive short distances, particularly when BAC is falling.

Modeling choice paradoxes under risk: From prospect theories to sampling-based accounts.

Important developments in the study of decision making have been based on the establishment and testing of choice paradoxes (e.g., Allais') that reject different theories (e.g., Expected Utility Theory). One of the most popular and celebrated models in the literature, Cumulative Prospect Theory (CPT), has managed to retain its status despite a growing body of empirical evidence stemming from a collection of choice paradoxes that reject it. Two alternative models, Transfer of Attention Exchange (TAX) and an extension of Decision Field Theory (DFTe), have been proposed as possible alternatives to CPT. To date, no study has directly compared these three models within the context of a large set of lottery problems that tests different choice paradoxes. The present study accomplishes this by using a large and diverse set of lottery problems, involving both potential gains and losses. Our results support the presence and robustness of a set of 'strong' choice paradoxes that reject CPT irrespective of its parametric form. Model comparison results show that DFTe provides the best account for the present set of lottery problems, as it is able to accommodate the choice data at large in a parsimonious fashion. The success of DFTe shows that many behavioral phenomena, including paradoxes that CPT cannot account for, can be successfully captured by a simple noisy-sampling process. Overall, our results suggest that researchers should move away from CPT, and focus their efforts on alternative models such as DFTe.

Cognitive Aging and Tests of Rationality.

We investigated whether older adults are more likely than younger adults to violate a foundational property of rational decision making, the axiom of transitive preference. Our experiment consisted of two groups, older (ages 60-75; 21 participants) and younger (ages 18-30; 20 participants) adults. We used Bayesian model selection to investigate whether individuals were better described via (transitive) weak order-based decision strategies or (possibly intransitive) lexicographic semiorder decision strategies. We found weak evidence for the hypothesis that older adults violate transitivity at a higher rate than younger adults. At the same time, a hierarchical Bayesian analysis suggests that, in this study, the distribution of decision strategies across individuals is similar for both older and younger adults.

Beyond overall effects: A Bayesian approach to finding constraints in meta-analysis.

Most meta-analyses focus on the behavior of meta-analytic means. In many cases, however, this mean is difficult to defend as a construct because the underlying distribution of studies reflects many factors, including how we as researchers choose to design studies. We present an alternative goal for meta-analysis. The analyst may ask about relations that are stable across all the studies. In a typical meta-analysis, there is a hypothesized direction (e.g., that violent video games increase, rather than decrease, aggressive behavior). We ask whether all studies in a meta-analysis have true effects in the hypothesized direction. If so, this is an example of a stable relation across all the studies. We propose 4 models: (a) all studies are truly null; (b) all studies share a single true nonzero effect; (c) studies differ, but all true effects are in the same direction; and (d) some study effects are truly positive, whereas others are truly negative. We develop Bayes factor model comparison for these models and apply them to 4 extant meta-analyses to show their usefulness. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The 'paradox' of converging evidence.

We explore the implication of viewing a psychological theory as the logical conjunction of all its predictions. Even if several predictions derived from a theory are descriptive of behavior in separate studies, the theory as a whole may fail to be descriptive of any single individual. We discuss what proportion of a population satisfies a theory's joint predictions as a function of the true effect sizes and the proportion of variance attributable to individual differences. Unless there are no individual differences, even very well replicated effects may fail to establish that the combination of predictions that have been tested accurately describes even one person. Every additional study that contributes another effect, rather than strengthening support for the theory, may further limit its scope. Using four illustrative examples from cognitive and social psychology, we show how, in particular, small effect sizes dramatically limit the scope of psychological theories unless every small effect coincides with little to no individual differences. In some cases, this 'paradox' can be overcome by casting theories in such a way that they apply to everyone in a target population, without exception. Rather than relegating heterogeneity to a nuisance component of statistical models and data analysis, explicitly keeping track of heterogeneity in hypothetical constructs makes it possible to understand and quantify theoretical scope. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Multinomial Models with Linear Inequality Constraints: Overview and Improvements of Computational Methods for Bayesian Inference.

Many psychological theories can be operationalized as linear inequality constraints on the parameters of multinomial distributions (e.g., discrete choice analysis). These constraints can be described in two equivalent ways: Either as the solution set to a system of linear inequalities or as the convex hull of a set of extremal points (vertices). For both representations, we describe a general Gibbs sampler for drawing posterior samples in order to carry out Bayesian analyses. We also summarize alternative sampling methods for estimating Bayes factors for these model representations using the encompassing Bayes factor method. We introduce the R package multinomineq, which provides an easily-accessible interface to a computationally efficient implementation of these techniques.

Correction: Estimation accuracy in the psychological sciences.

[This corrects the article DOI: 10.1371/journal.pone.0207239.].

Is Cognitive Impairment Related to Violations of Rationality? A Laboratory Alcohol Intoxication Study Testing Transitivity of Preference.

Alcohol intoxication is well known to impair a number of cognitive abilities required for sound decision making. We tested whether an intoxicating dose of alcohol altered whether individuals satisfied a basic property of rational decision making, transitivity of preference. Our study was within-subjects in design and our analysis teased apart stable, yet error-prone, preferences from variable, error-free preferences. We find that alcohol intoxication does not appear to play a major role in determining whether subjects violate transitivity. For a minority of individuals, we find that alcohol intoxication does impact how they select among and/or perceive lotteries with similar attribute values. This, in turn, can cause them to alter various aspects of their preference structure.

Decision Making and Alcohol: Health Policy Implications.

Many negative outcomes associated with alcohol consumption stem from individual decision making, such as the decision whether or not to drive after drinking. Understanding biases in decision making as they relate to alcohol, and measuring variability in decision strategies across individuals, can be a tool for improving policy. For alcohol-related policy, both addiction and the acute effects of intoxication are potential sources of bias or heterogeneity in decision strategy, and we provide a brief primer on how they can affect decision making. We then discuss two alcohol policy domains, alcohol impaired driving and excessive consumption. We review potential implications of major findings in decision making research for each of these domains, and point out how policy could take account of biases and variability in decision making.

A General Approach to Prior Transformation.

We present a general method for setting prior distributions in Bayesian models where parameters of interest are re-parameterized via a functional relationship. We generalize the results of Heck and Wagenmakers (2016) by considering the case where the dimension of the auxiliary parameter space does not equal that of the primary parameter space. We present numerical methods for carrying out prior specification for statistical models that do not admit closed-form solutions. Taken together, these results provide researchers a more complete set of tools for setting prior distributions that could be applied to many cognitive and decision making models. We illustrate our approach by reanalyzing data under the Selective Integration model of Tsetsos et al. (2016). We find, via a Bayes factor analysis, that the selective integration model with all four parameters generally outperforms both the three-parameter variant (omitting early cognitive noise) and the w = 1 variant (omitting selective gating), as well as an unconstrained competitor model. By contrast, Tsetsos et al. found the three parameter variant to be the best performing in a BIC analysis (in the absence of a competitor). Finally, we also include a pedagogical treatment of the mathematical tools necessary to formulate our results, including a simple "toy" example that illustrates our more general points.