Severity influences categorical likelihood communications: A case study with Southeast Asian weather forecasters.

Risk assessments are common in multiple domains, from finance to medicine. They require evaluating an event's potential severity and likelihood. We investigate the possible dependence of likelihood and severity within the domain of impact-based weather forecasting (IBF), following predictions derived from considering asymmetric loss functions. In a collaboration between UK psychologists and partners from four meteorological organisations in Southeast Asia, we conducted two studies (N = 363) eliciting weather warnings from forecasters. Forecasters provided warnings denoting higher likelihoods for high severity impacts than low severity impacts, despite these impacts being described as having the same explicit numerical likelihood of occurrence. This 'Severity effect' is pervasive, and we find it can have a continued influence even for an updated forecast. It is additionally observed when translating warnings made on a risk matrix to numerical probabilities.

Exploring open science practices in behavioural public policy research.

In their book 'Nudge: Improving Decisions About Health, Wealth and Happiness', Thaler & Sunstein (2009) argue that choice architectures are promising public policy interventions. This research programme motivated the creation of 'nudge units', government agencies which aim to apply insights from behavioural science to improve public policy. We closely examine a meta-analysis of the evidence gathered by two of the largest and most influential nudge units (DellaVigna & Linos (2022 Econometrica 90, 81-116 (doi:10.3982/ECTA18709))) and use statistical techniques to detect reporting biases. Our analysis shows evidence suggestive of selective reporting. We additionally evaluate the public pre-analysis plans from one of the two nudge units (Office of Evaluation Sciences). We identify several instances of excellent practice; however, we also find that the analysis plans and reporting often lack sufficient detail to evaluate (unintentional) reporting biases. We highlight several improvements that would enhance the effectiveness of the pre-analysis plans and reports as a means to combat reporting biases. Our findings and suggestions can further improve the evidence base for policy decisions.

An online randomised controlled trial of prognosticating imminent death in advanced cancer patients: Clinicians give greater weight to advice from a prognostic algorithm than from another clinician with a different profession.

BACKGROUND: A second opinion or a prognostic algorithm may increase prognostic accuracy. This study assessed the level to which clinicians integrate advice perceived to be coming from another clinician or a prognostic algorithm into their prognostic estimates, and how participant characteristics and nature of advice received affect this. METHODS: An online double-blind randomised controlled trial was conducted. Palliative doctors, nurses and other types of healthcare professionals were randomised into study arms differing by perceived source of advice (algorithm or another clinician). In fact, the advice was the same in both arms (emanating from the PiPS-B14 prognostic model). Each participant reviewed five patient summaries. For each summary, participants: (1) provided an initial probability estimate of two-week survival (0% 'certain death'-100% 'certain survival'); (2) received advice (another estimate); (3) provided a final estimate. Weight of Advice (WOA) was calculated for each summary (0 '100% advice discounting' - 1 '0% discounting') and multilevel linear regression analyses were conducted. CLINICAL TRIAL REGISTRATION NUMBER: NCT04568629. RESULTS: A total of 283 clinicians were included in the analysis. Clinicians integrated advice from the algorithm more than advice from another clinician (WOA difference = -0.12 [95% CI -0.18, -0.07], p < 0.001). There was no interaction between study arm and participant profession, years of palliative care or overall experience. Advice of intermediate strength (75%) was given a lower WOA (0.31) than advice received at either the 50% (WOA 0.40) or 90% level (WOA 0.43). The overall interaction between strength of advice and study arm on WOA was significant (p < 0.001). CONCLUSION: Clinicians adjusted their prognostic estimates more when advice was perceived to come from a prognostic algorithm than from another clinician. Research is needed to understand how clinicians make prognostic decisions and how algorithms are used in clinical practice.

An appropriate verbal probability lexicon for communicating surgical risks is unlikely to exist.

Effective risk communication about medical procedures is critical to ethical shared decision-making. Here, we explore the potential for development of an evidence-based lexicon for verbal communication of surgical risk. We found that Ear, Nose and Throat (ENT) surgeons expressed a preference for communicating such risks using verbal probability expressions (VPEs; e.g., "high risk"). However, there was considerable heterogeneity in the expressions they reported using (Study 1). Study 2 compared ENT surgeons' and laypeople's (i.e., potential patients) interpretations of the ten most frequent VPEs listed in Study 1. While both groups displayed considerable variability in interpretations, lay participants demonstrated more, as well as providing systematically higher interpretations than those of surgeons. Study 3 found that lay participants were typically unable to provide unique VPEs to differentiate between the ranges of (low) probabilities required. Taken together, these results add to arguments that reliance on VPEs for surgical risk communication is ill-advised. Not only are there systematic interpretational differences between surgeons and potential patients, but the coarse granularity of VPEs raises severe challenges for developing an appropriate evidence-based lexicon for surgical risk communication. We caution against the use of VPEs in any risk context characterized by low, but very different, probabilities. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Plausibility matters: A challenge to Gilbert's "Spinozan" account of belief formation.

Most of the claims we encounter in real life can be assigned some degree of plausibility, even if they are new to us. On Gilbert's (1991) influential account of belief formation, whereby understanding a sentence implies representing it as true, all new propositions are initially accepted, before any assessment of their veracity. As a result, plausibility cannot have any role in initial belief formation on this account. In order to isolate belief formation experimentally, Gilbert, Krull, and Malone (1990) employed a dual-task design: if a secondary task disrupts participants' evaluation of novel claims presented to them, then the initial encoding should be all there is, and if that initial encoding consistently renders claims 'true' (even where participants were told in the learning phase that the claims they had seen were false), then Gilbert's account is confirmed. In this pre-registered study, we replicate one of Gilbert et al.'s (1990) seminal studies ("The Hopi Language Experiment") while additionally introducing a plausibility variable. Our results show that Gilbert's 'truth bias' does not hold for implausible statements - instead, initial encoding seemingly renders implausible statements 'false'. As alternative explanations of this finding that would be compatible with Gilbert's account can be ruled out, it questions Gilbert's account.

Optimism where there is none: Asymmetric belief updating observed with valence-neutral life events.

How people update their beliefs when faced with new information is integral to everyday life. A sizeable body of literature suggests that people's belief updating is optimistically biased, such that their beliefs are updated more in response to good news than bad news. However, recent research demonstrates that findings previously interpreted as evidence of optimistic belief updating may be the result of flaws in experimental design, rather than motivated reasoning. In light of this controversy, we conduct three pre-registered variations of the standard belief updating paradigm (combined N = 300) in which we test for asymmetric belief updating with neutral, non-valenced stimuli using analytic approaches found in previous research. We find evidence of seemingly biased belief updating with neutral stimuli - results that cannot be attributed to a motivational, valence-based, optimism account - and further show that there is uninterpretable variability across samples and analytic techniques. Jointly, these results serve to highlight the methodological flaws in current optimistic belief updating research.

Testing the adaptability of people's use of attribute frame information.

The informational leakage account of attribute framing effects proposes that a communicator's choice of frame provides informational value, such that different frames are not informationally equivalent. Across five studies communicating food risks, we investigated the adaptability of communication recipients' (our participants) use of frame information by manipulating the degree to which the communicator ostensibly had a choice over how the information was framed. Within-participants framing effects were observed across all conditions of all studies. Only in Study 4 (the only study in which communicator choice was manipulated within-participants) was there any evidence for an attenuation of framing effects where the communicator was not responsible for how the information was framed. Overall, regardless of whether or not framing effects are driven by the informational value contained in a communicator's choice of frame, people show little sensitivity to situations where that choice is removed.

Failures to replicate a key result of the selective accessibility theory of anchoring.

Numerical anchoring effects describe the assimilative effect of a previously presented number on subsequent numerical estimates. Such effects are robust and consequential. A number of different accounts have been proposed to explain these effects. What is currently unclear is under which situations different mechanisms play more or less critical roles. An extant test from the literature is proposed as a "signature test" for the operation of selective accessibility mechanisms. Four experiments were conducted to ascertain the evidence for selective accessibility with this test, evidence which was not obtained. A fifth experiment employed a different methodology, and again failed to show evidence for selective accessibility. Subsequent discussion suggests that the robustness of anchoring effects is remarkable, but the theoretical basis for some previous tests of the selective accessibility account of anchoring is shaky, and we advise against their use in this capacity. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The (un)availability of prognostic information in the last days of life: a prospective observational study.

OBJECTIVES: The aims of this study were (1) to document the clinical condition of patients considered to be in the last 2 weeks of life and (2) to compare patients who did or did not survive for 72 hours. DESIGN: A prospective observational study. SETTING: Two sites in London, UK (a hospice and a hospital palliative care team). PARTICIPANTS: Any inpatient, over 18 years old, English speaking, who was identified by the palliative care team as at risk of dying within the next 2 weeks was eligible. OUTCOME MEASURES: Prognostic signs and symptoms were documented at a one off assessment and patients were followed up 7 days later to determine whether or not they had died. RESULTS: Fifty participants were recruited and 24/50 (48%) died within 72 hours of assessment. The most prevalent prognostic features observed were a decrease in oral food intake (60%) and a rapid decline of the participant's global health status (56%). Participants who died within 72 hours had a lower level of consciousness and had more care needs than those who lived longer. A large portion of data was unavailable, particularly that relating to the psychological and spiritual well-being of the patient, due to the decreased consciousness of the patient. CONCLUSIONS: The prevalence of prognostic signs and symptoms in the final days of life has been documented between those predicted to die and those who did not. How doctors make decisions with missing information is an area for future research, in addition to understanding the best way to use the available information to make more accurate predictions.

Underlying wishes and nudged choices.

Is the inferred preference of a deceased relative to donate his or her organs stronger when the choice was made under a mandated rather than under an automatic default (i.e., nudged choice) legislative system? The answer to this is particularly important, because families can, and do, veto the choices of their deceased relatives. In three studies, we asked American and European participants from countries that have either a default opt-in or a default opt-out system to take on the role of a third party to judge the likelihood that an individual's "true wish" was to actually donate his or her organs, given that the decedent was registered to donate on the organ donation register. In each study participants were randomly assigned to one of four organ donation legislative systems (default opt-in, default opt-out, mandated choice, mandatory). Overall, regardless of which country participants came from, they perceived the donor's underlying preference to donate as stronger under the default opt-in and mandated choice systems as compared with the default opt-out and mandatory donor systems. We discuss the practical issues that result from using default systems in the domain of organ donation and propose potential ways to ameliorate the uncertainty around inferences of underlying preference from a nudged choice. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Unrealistic comparative optimism: An unsuccessful search for evidence of a genuinely motivational bias.

One of the most accepted findings across psychology is that people are unrealistically optimistic in their judgments of comparative risk concerning future life events-they judge negative events as less likely to happen to themselves than to the average person. Harris and Hahn (2011), however, demonstrated how unbiased (non-optimistic) responses can result in data patterns commonly interpreted as indicative of optimism due to statistical artifacts. In the current paper, we report the results of 5 studies that control for these statistical confounds and observe no evidence for residual unrealistic optimism, even observing a 'severity effect' whereby severe outcomes were overestimated relative to neutral ones (Studies 3 & 4). We conclude that there is no evidence supporting an optimism interpretation of previous results using the prevalent comparison method.

Understanding the coherence of the severity effect and optimism phenomena: Lessons from attention.

Claims that optimism is a near-universal characteristic of human judgment seem to be at odds with recent results from the judgment and decision making literature suggesting that the likelihood of negative outcomes are overestimated relative to neutral outcomes. In an attempt to reconcile these seemingly contrasting phenomena, inspiration is drawn from the attention literature in which there is evidence that both positive and negative stimuli can have attentional privilege relative to neutral stimuli. This result provides a framework within which I consider three example phenomena that purport to demonstrate that people's likelihood estimates are optimistic: Wishful thinking; Unrealistic comparative optimism and Asymmetric belief updating. The framework clarifies the relationships between these phenomena and stimulates future research questions. Generally, whilst results from the first two phenomena appear reconcilable in this conceptualisation, further research is required in reconciling the third.

A pessimistic view of optimistic belief updating.

Received academic wisdom holds that human judgment is characterized by unrealistic optimism, the tendency to underestimate the likelihood of negative events and overestimate the likelihood of positive events. With recent questions being raised over the degree to which the majority of this research genuinely demonstrates optimism, attention to possible mechanisms generating such a bias becomes ever more important. New studies have now claimed that unrealistic optimism emerges as a result of biased belief updating with distinctive neural correlates in the brain. On a behavioral level, these studies suggest that, for negative events, desirable information is incorporated into personal risk estimates to a greater degree than undesirable information (resulting in a more optimistic outlook). However, using task analyses, simulations, and experiments we demonstrate that this pattern of results is a statistical artifact. In contrast with previous work, we examined participants' use of new information with reference to the normative, Bayesian standard. Simulations reveal the fundamental difficulties that would need to be overcome by any robust test of optimistic updating. No such test presently exists, so that the best one can presently do is perform analyses with a number of techniques, all of which have important weaknesses. Applying these analyses to five experiments shows no evidence of optimistic updating. These results clarify the difficulties involved in studying human 'bias' and cast additional doubt over the status of optimism as a fundamental characteristic of healthy cognition.

Conceptual and direct replications fail to support the stake-likelihood hypothesis as an explanation for the interdependence of utility and likelihood judgments.

Previous research suggests that people systematically overestimate the occurrence of both positive and negative events, compared with neutral future events, and that these biases are due to a misattribution of arousal elicited by utility (stake-likelihood hypothesis; SLH; Vosgerau, 2010). However, extant research has provided only indirect support for these arousal misattribution processes. In the present research, we initially aimed to provide a direct test of the SLH by measuring arousal with galvanic skin responses to examine the mediating role of arousal. We observed no evidence that measured arousal mediated the impact of utility on probability estimates. Given the lack of direct support for the SLH in Experiment 1, Experiments 2-5 aimed to assess the SLH by replicating some of the original findings that provided support for arousal misattribution as a mechanism. Despite our best efforts to create experimental conditions under which we would be able to demonstrate the stake-likelihood effect, we were unable to replicate previous results, with a Bayesian meta-analysis demonstrating support for the null hypothesis. We propose that accounts based on imaginability and loss function asymmetry are currently better candidate explanations for the influence of outcome utility on probability estimates.

Public Reception of Climate Science: Coherence, Reliability, and Independence.

Possible measures to mitigate climate change require global collective actions whose impacts will be felt by many, if not all. Implementing such actions requires successful communication of the reasons for them, and hence the underlying climate science, to a degree that far exceeds typical scientific issues which do not require large-scale societal response. Empirical studies have identified factors, such as the perceived level of consensus in scientific opinion and the perceived reliability of scientists, that can limit people's trust in science communicators and their subsequent acceptance of climate change claims. Little consideration has been given, however, to recent formal results within philosophy concerning the relationship between truth, the reliability of evidence sources, the coherence of multiple pieces of evidence/testimonies, and the impact of (non-)independence between sources of evidence. This study draws on these results to evaluate exactly what has (and, more important, has not yet) been established in the empirical literature about the factors that bias the public's reception of scientific communications about climate change.

The Appeal to Expert Opinion: Quantitative Support for a Bayesian Network Approach.

The appeal to expert opinion is an argument form that uses the verdict of an expert to support a position or hypothesis. A previous scheme-based treatment of the argument form is formalized within a Bayesian network that is able to capture the critical aspects of the argument form, including the central considerations of the expert's expertise and trustworthiness. We propose this as an appropriate normative framework for the argument form, enabling the development and testing of quantitative predictions as to how people evaluate this argument, suggesting that such an approach might be beneficial to argumentation research generally. We subsequently present two experiments as an example of the potential for future research in this vein, demonstrating that participants' quantitative ratings of the convincingness of a proposition that has been supported with an appeal to expert opinion were broadly consistent with the predictions of the Bayesian model.

Fingermark submission decision-making within a UK fingerprint laboratory: Do experts get the marks that they need?

Within UK policing it is routinely the responsibility of fingerprint laboratory practitioners to chemically develop areas of latent fingerprint ridge detail on evidential items and to determine which areas of ridge detail are of sufficient quality to be submitted to fingerprint experts for search or comparison against persons of interest. This study assessed the effectiveness of the fingermark submission process within the Evidence Recovery Unit Fingerprint Laboratory of the Metropolitan Police Service. Laboratory practitioners were presented with known source fingermark images previously deemed identifiable or insufficient by fingerprint experts, and were asked to state which of the marks they would forward to the Fingerprint Bureau. The results indicated that practitioners forwarded a higher percentage of insufficient fingermarks than is acceptable according to current laboratory guidelines, and discarded a number of marks that were of sufficient quality for analysis. Practitioners forwarded more insufficient fingermarks when considering fingermarks thought to be related to a murder and discarded more sufficient fingermarks when considering those thought to be related to a crime of 'theft from vehicle'. The results highlight the need for fingerprint laboratories to work alongside fingerprint experts to ensure that a consistent approach to decision-making is, as far as possible, achieved, and that appropriate thresholds are adopted so as to prevent the loss of valuable evidence and improve the efficiency of the fingerprint filtering process.

Agency affects adults', but not children's, guessing preferences in a game of chance.

Adults and children have recently been shown to prefer guessing the outcome of a die roll after the die has been rolled (but remained out of sight) rather than before it has been rolled. This result is contrary to the predictions of the competence hypothesis (Heath & Tversky, 1991 ), which proposes that people are sensitive to the degree of their relative ignorance and therefore prefer to guess about an outcome it is impossible to know, rather than one that they could know, but do not. We investigated the potential role of agency in guessing preferences about a novel game of chance. When the experimenter controlled the outcome, we replicated the finding that adults and 5- to 6-year-old children preferred to make their guess after the outcome had been determined. For adults only, this preference reversed when they exerted control over the outcome about which they were guessing. The adult data appear best explained by a modified version of the competence hypothesis that highlights the notion of control or responsibility. It is proposed that potential attributions of blame are related to the guesser's role in determining the outcome. The child data were consistent with an imagination-based account of guessing preferences.

Communicating environmental risks: Clarifying the severity effect in interpretations of verbal probability expressions.

Verbal probability expressions are frequently used to communicate risk and uncertainty. The Intergovernmental Panel on Climate Change (IPCC), for example, uses them to convey risks associated with climate change. Given the potential for human action to mitigate future environmental risks, it is important to understand how people respond to these expressions. In 3 studies employing a novel manipulation of event severity (so as to avoid any confound with event base rate), we demonstrated a systematic effect of event severity on the interpretation of verbal probability expressions. Challenging a previous finding in the literature, expressions referring to a severe event were interpreted as indicating a higher probability than those referring to a more neutral event. The finding was demonstrated in scenarios communicating risks relating to climate change (Studies 1 and 2) and replicated in scenarios involving nanotechnology and nuclear materials (Study 3). This is the first direct demonstration of an effect of outcome severity on the interpretation of verbal probability expressions, correcting a previous (potentially problematic) conclusion attributable to a flawed experimental design.