What Affects Perceived Trustworthiness of Online Medical Information and Subsequent Treatment Decision Making? Randomized Trials on the Role of Uncertainty and Institutional Cues.

Background. Online, algorithmically driven prognostic tools are increasingly important in medical decision making. Institutions developing such tools need to be able to communicate the precision and accuracy of the information in a trustworthy manner, and so many attempt to communicate uncertainties but also use institutional logos to underscore their trustworthiness. Bringing together theories on trust, uncertainty, and psychological distance in a novel way, we tested whether and how the communication of uncertainty and the presence of institutional logos together affected trust in medical information, the prognostic tool itself, and treatment decisions. Methods. A pilot and 2 online experiments in which UK (experiment 1) and worldwide (experiment 2) participants (Ntotal = 4,724) were randomized to 1 of 12 arms in a 3 (uncertainty cue) × 4 (institutional cue) between-subjects design. The stimulus was based on an existing medical prognostic tool. Results. Institutional trust was consistently associated with trust in the prognostic tool itself, while uncertainty information had no consistent effect. Institutional trust predicted the amount of weight participants reported placing on institutional endorsements in their decision making and the likelihood of switching from passive to active treatment in a hypothetical scenario. There was also a significant effect of psychological distance to (perceived hypotheticality of) the scenario. Conclusions/Implications. These results underline the importance of institutions demonstrating trustworthiness and building trust with their users. They also suggest that users tend to be insensitive to communications of uncertainty and that communicators may need to be highly explicit when attempting to warn of low precision or quality of evidence. The effect of the perceived hypotheticality of the scenario underscores the importance of realistic decision-making scenarios for studies and the role of familiarity with the decision dilemma generally. Highlights: In a world where information for medical decision making is increasingly going to be provided through digital, online tools, institutions providing such tools need guidance on how best to communicate about their trustworthiness and precision.We find that people are fairly insensitive to cues designed to communicate uncertainty around the outputs of such tools. Even putting "ATTENTION" in bold font or explicitly pointing out the weaknesses in the data did not appear to affect people's decision making using the tool's outputs. Institutions should take note, and further work is required to determine how best to communicate uncertainty in a way that elicits appropriate caution in lay users.People were much more sensitive to institutional logos associated with the outputs. Generalized institutional trust (rather than trust in the specific institution whose logo was shown) was associated with how trustworthy, accurate, and reliable the tool, its algorithm, and the numbers it produced were perceived to be. This underscores the role of societal trust in institutions at large.Finally, as a note to researchers, we found a significant effect of how hypothetical or believable participants felt the experimental scenario was. This is a variable that seems rarely controlled for in studies and yet played as much of a role as some of our variables of interest, so we suggest that it is measured in future experiments.

Do colored cells in risk matrices affect decision-making and risk perception? Insights from randomized controlled studies.

Risk matrices communicate the likelihood and potential impact of risks and are often used to inform decision-making around risk mitigations. The merits and demerits of risk matrices in general have been discussed extensively, yet little attention has been paid to the potential influence of color in risk matrices on their users. We draw from fuzzy-trace theory and hypothesize that when color is present, individuals are likely to place greater value on reducing risks that cross color boundaries (i.e., the boundary-crossing effect), leading to sub-optimal decision making. In two randomized controlled studies, employing forced-choice and willingness-to-pay measures to investigate the boundary-crossing effect in two different color formats for risk matrices, we find preliminary evidence to support our hypotheses that color can influence decision making. The evidence also suggests that the boundary-crossing effect is only present in, or is stronger for, higher numeracy individuals. We therefore recommend that designers should consider avoiding color in risk matrices, particularly in situations where these are likely to be used by highly numerate individuals, if the communication goal is to inform in an unbiased way.

Investigating the presentation of uncertainty in an icon array: A randomized trial.

Background: Clinicians are often advised to use pictographs to communicate risk, but whether they offer benefits when communicating risk imprecision (e.g., 65%-79%) is unknown. Purpose: To test whether any of three approaches to visualizing imprecision would more effectively communicate breast and ovarian cancer risk for BRCA1 pathogenic variant carriers. Methods: 1,300 UK residents were presented with a genetic report with information about BRCA1-related risks, with random assignment to one of four formats: no visualization (text alone), or a pictograph using shaded icons, a gradient, or arrows marking range endpoints. We also tested pictographs in two layouts. Analysis of variance (ANOVA) and regression was employed. Results: There was no effect of format. Participants shown pictographs vs. text alone had better uptake of breast cancer risk messages (p < .05, η 2 = 0.003). Pictographs facilitated memory for the specific amount of risk (p < 0.001, η 2 = 0.019), as did the tabular layout. Individuals not having completed upper secondary education may benefit most. Conclusions: We found weak evidence in favor of using simple pictographs with ranges to communicate BRCA risk (versus text alone), and of the tabular layout. Innovation: Testing different ways of communicating imprecision within pictographs is a novel and promising line of research.

Making BRCA1 genetic test reports easier to understand through user-centered design: A randomized trial.

PURPOSE: Our objective was to apply a user-centered design process to identify phrases, graphics, and ways of communicating numerical risks that could be used to help patients understand their cancer risk and next steps on receiving BRCA1 genetic test results (positive, negative, and variants of uncertain significance). METHODS: The first phase of the study, a user-centered design process, consisted of 4 rounds of interviews (N = 42, including 13 health care professionals and 16 patients having undergone BRCA testing). The second was a randomized, between-participants experimental study of 456 United Kingdom residents that compared the resulting reports to reports used in a United Kingdom national genomic laboratory hub. Outcomes were subjective and objective comprehension, communication efficacy, actionability, and perceived risk. RESULTS: Subjective comprehension, communication efficacy, and actionability were all higher for the user-centered reports, with no difference in perceived risk. Comprehension of participants viewing user-centered reports was significantly better on 2 items, directionally (but not significantly) better on 6 items, and directionally (but not significantly) worse on 2 items. CONCLUSION: Our results imply that user-centered design is a promising approach for developing materials about complex genetic risks. We suggest wordings that are likely to lead to improved comprehension when communicating BRCA-associated risks.

How People Understand Risk Matrices, and How Matrix Design Can Improve their Use: Findings from Randomized Controlled Studies.

Risk matrices are a common way to communicate the likelihood and potential impacts of a variety of risks. Until now, there has been little empirical work on their effectiveness in supporting understanding and decision making, and on how different design choices affect these. In this pair of online experiments (total n = 2699), we show that risk matrices are not always superior to text for the presentation of risk information, and that a nonlinear/geometric labeling scheme helps matrix comprehension (when the likelihood/impact scales are nonlinear). To a lesser degree, results suggested that changing the shape of the matrix so that cells increase in size nonlinearly facilitates comprehension as compared to text alone, and that comprehension might be enhanced by integrating further details about the likelihood and impact onto the axes of the matrix rather than putting them in a separate key. These changes did not affect participants' preference for reducing impact over reducing likelihood when making decisions about risk mitigation. We recommend that designers of risk matrices consider these changes to facilitate better understanding of relationships among risks.

Clinical Impact of the Predict Prostate Risk Communication Tool in Men Newly Diagnosed with Nonmetastatic Prostate Cancer: A Multicentre Randomised Controlled Trial.

BACKGROUND: Predict Prostate is a freely available online personalised risk communication tool for men with nonmetastatic prostate cancer. Its accuracy has been assessed in multiple validation studies, but its clinical impact among patients has not hitherto been assessed. OBJECTIVE: To assess the impact of the tool on patient decision-making and disease perception. DESIGN, SETTING, AND PARTICIPANTS: A multicentre randomised controlled trial was performed across eight UK centres among newly diagnosed men considering either active surveillance or radical treatment. A total of 145 patients were included between 2018 and 2020, with median age 67 yr (interquartile range [IQR] 61-72) and prostate-specific antigen 6.8 ng/ml (IQR 5.1-8.8). INTERVENTION: Participants were randomised to either standard of care (SOC) information or SOC and a structured presentation of the Predict Prostate tool. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Validated questionnaires were completed by assessing the impact of the tool on decisional conflict, uncertainty, anxiety, and perception of survival. RESULTS AND LIMITATIONS: Mean Decisional Conflict Scale scores were 26% lower in the Predict Prostate group (mean = 16.1) than in the SOC group (mean = 21.7; p = 0.027). Scores on the "support", "uncertainty", and "value clarity" subscales all favoured Predict Prostate (all p < 0.05). There was no significant difference in anxiety scores or final treatment selection between the two groups. Patient perception of 15-yr prostate cancer-specific mortality (PCSM) and overall survival benefit from radical treatment were considerably lower and more accurate among men in the Predict Prostate group (p < 0.001). In total, 57% of men reported that the Predict Prostate estimates for PCSM were lower than expected, and 36% reported being less likely to select radical treatment. Over 90% of patients in the intervention group found it useful and 94% would recommend it to others. CONCLUSIONS: Predict Prostate reduces decisional conflict and uncertainty, and shifts patient perception around prognosis to be more realistic. This randomised trial demonstrates that Predict Prostate can directly inform the complex decision-making process in prostate cancer and is felt to be useful by patients. Future larger trials are warranted to test its impact upon final treatment decisions. PATIENT SUMMARY: In this national study, we assessed the impact of an individualised risk communication tool, called Predict Prostate, on patient decision-making after a diagnosis of localised prostate cancer. Men were randomly assigned to two groups, which received either standard counselling and information, or this in addition to a structured presentation of the Predict Prostate tool. Men who saw the tool were less conflicted and uncertain in their decision-making, and recommended the tool highly. Those who saw the tool had more realistic perception about their long-term survival and the potential impact of treatment upon this. TAKE HOME MESSAGE: The use of an individualised risk communication tool, such as Predict Prostate, reduces patient decisional conflict and uncertainty when deciding about treatment for nonmetastatic prostate cancer. The tool leads to more realistic perceptions about survival outcomes and prognosis.

Correlates of intended COVID-19 vaccine acceptance across time and countries: results from a series of cross-sectional surveys.

OBJECTIVE: Describe demographical, social and psychological correlates of willingness to receive a COVID-19 vaccine. SETTING: Series of online surveys undertaken between March and October 2020. PARTICIPANTS: A total of 25 separate national samples (matched to country population by age and sex) in 12 different countries were recruited through online panel providers (n=25 334). PRIMARY OUTCOME MEASURES: Reported willingness to receive a COVID-19 vaccination. RESULTS: Reported willingness to receive a vaccine varied widely across samples, ranging from 63% to 88%. Multivariate logistic regression analyses reveal sex (female OR=0.59, 95% CI 0.55 to 0.64), trust in medical and scientific experts (OR=1.28, 95% CI 1.22 to 1.34) and worry about the COVID-19 virus (OR=1.47, 95% CI 1.41 to 1.53) as the strongest correlates of stated vaccine acceptance considering pooled data and the most consistent correlates across countries. In a subset of UK samples, we show that these effects are robust after controlling for attitudes towards vaccination in general. CONCLUSIONS: Our results indicate that the burden of trust largely rests on the shoulders of the scientific and medical community, with implications for how future COVID-19 vaccination information should be communicated to maximise uptake.

How well did experts and laypeople forecast the size of the COVID-19 pandemic?

Throughout the COVID-19 pandemic, social and traditional media have disseminated predictions from experts and nonexperts about its expected magnitude. How accurate were the predictions of 'experts'-individuals holding occupations or roles in subject-relevant fields, such as epidemiologists and statisticians-compared with those of the public? We conducted a survey in April 2020 of 140 UK experts and 2,086 UK laypersons; all were asked to make four quantitative predictions about the impact of COVID-19 by 31 Dec 2020. In addition to soliciting point estimates, we asked participants for lower and higher bounds of a range that they felt had a 75% chance of containing the true answer. Experts exhibited greater accuracy and calibration than laypersons, even when restricting the comparison to a subset of laypersons who scored in the top quartile on a numeracy test. Even so, experts substantially underestimated the ultimate extent of the pandemic, and the mean number of predictions for which the expert intervals contained the actual outcome was only 1.8 (out of 4), suggesting that experts should consider broadening the range of scenarios they consider plausible. Predictions of the public were even more inaccurate and poorly calibrated, suggesting that an important role remains for expert predictions as long as experts acknowledge their uncertainty.

How do the UK public interpret COVID-19 test results? Comparing the impact of official information about results and reliability used in the UK, USA and New Zealand: a randomised controlled trial.

OBJECTIVES: To assess the effects of different official information on public interpretation of a personal COVID-19 PCR test result. DESIGN: A 5×2 factorial, randomised, between-subjects experiment, comparing four wordings of information about the test result and a control arm of no additional information; for both positive and negative test results. SETTING: Online experiment using recruitment platform Respondi. PARTICIPANTS: UK participants (n=1744, after a pilot of n=1657) quota-sampled to be proportional to the UK national population on age and sex. INTERVENTIONS: Participants were given a hypothetical COVID-19 PCR test result for 'John' who was presented as having a 50% chance of having COVID-19 based on symptoms alone. Participants were randomised to receive either a positive or negative result for 'John', then randomised again to receive either no more information, or text information on the interpretation of COVID-19 test results copied in September 2020 from the public websites of the UK's National Health Service, the USA's Centers for Disease Control, New Zealand's Ministry of Health or a modified version of the UK's wording. Information identifying the source of the wording was removed. MAIN OUTCOME MEASURES: Participants were asked 'What is your best guess as to the percent chance that John actually had COVID-19 at the time of his test, given his result?'; questions about their feelings of trustworthiness in the result, their perceptions of the quality of the underlying evidence and what action they felt 'John' should take in the light of his result. RESULTS: Of those presented with a positive COVID-19 test result for 'John', the mean estimate of the probability that he had the virus was 73% (71.5%-74.5%); for those presented with a negative result, 38% (36.7%-40.0%). There was no main effect of information (wording) on these means. However, those participants given the official information from the UK website, which did not mention the possibility of false negatives or false positives, were more likely to give a categorical (100% or 0%) answer (UK: 68/343, 19.8% (15.9%-24.4%); control group: 42/356, 11.8% (8.8%-15.6%)); the reverse was true for those viewing the New Zealand (NZ) wording, which highlighted the uncertainties most explicitly (20/345: 5.8% (3.7%-8.8%)). Aggregated across test result (positive/negative), there was a main effect of wording (p<0.001) on beliefs about how 'John' should behave, with those seeing the NZ wording marginally more likely to agree that 'John' should continue to self-isolate than those viewing the control or the UK wording. The proportion of participants who felt that a symptomatic individual who tests negative definitely should not self-isolate was highest among those viewing the UK wording (31/178, 17.4% (12.5%-23.7%)), and lowest among those viewing the NZ wording (6/159, 3.8% (1.6%-8.2%)). Although the NZ wording was rated harder to understand, participants reacted to the uncertainties given in the text in the expected direction: there was a small main effect of wording on trust in the result (p=0.048), with people perceiving the test result as marginally less trustworthy after having read the NZ wording compared with the UK wording. Positive results were generally viewed as more trustworthy and as having higher quality of evidence than negative results (both p<0.001). CONCLUSIONS: The public's default assessment of the face value of both the positive and negative test results (control group) indicate an awareness that test results are not perfectly accurate. Compared with other messaging tested, participants shown the UK's 2020 wording about the interpretation of the test results appeared to interpret the results as more definitive than is warranted. Wording that acknowledges uncertainty can help people to have a more nuanced and realistic understanding of what a COVID-19 test result means, which supports decision making and behavioural response. PREREGISTRATION AND DATA REPOSITORY: Preregistration of pilot at osf.io/8n62f, preregistration of main experiment at osf.io/7rcj4, data and code available online (osf.io/pvhba).

Communicating personalized risks from COVID-19: guidelines from an empirical study.

As increasing amounts of data accumulate on the effects of the novel coronavirus SARS-CoV-2 and the risk factors that lead to poor outcomes, it is possible to produce personalized estimates of the risks faced by groups of people with different characteristics. The challenge of how to communicate these then becomes apparent. Based on empirical work (total n = 5520, UK) supported by in-person interviews with the public and physicians, we make recommendations on the presentation of such information. These include: using predominantly percentages when communicating the absolute risk, but also providing, for balance, a format which conveys a contrasting (higher) perception of risk (expected frequency out of 10 000); using a visual linear scale cut at an appropriate point to illustrate the maximum risk, explained through an illustrative 'persona' who might face that highest level of risk; and providing context to the absolute risk through presenting a range of other 'personas' illustrating people who would face risks of a wide range of different levels. These 'personas' should have their major risk factors (age, existing health conditions) described. By contrast, giving people absolute likelihoods of other risks they face in an attempt to add context was considered less helpful. We note that observed effect sizes generally were small. However, even small effects are meaningful and relevant when scaled up to population levels.

Susceptibility to misinformation about COVID-19 around the world.

Misinformation about COVID-19 is a major threat to public health. Using five national samples from the UK (n = 1050 and n = 1150), Ireland (n = 700), the USA (n = 700), Spain (n = 700) and Mexico (n = 700), we examine predictors of belief in the most common statements about the virus that contain misinformation. We also investigate the prevalence of belief in COVID-19 misinformation across different countries and the role of belief in such misinformation in predicting relevant health behaviours. We find that while public belief in misinformation about COVID-19 is not particularly common, a substantial proportion views this type of misinformation as highly reliable in each country surveyed. In addition, a small group of participants find common factual information about the virus highly unreliable. We also find that increased susceptibility to misinformation negatively affects people's self-reported compliance with public health guidance about COVID-19, as well as people's willingness to get vaccinated against the virus and to recommend the vaccine to vulnerable friends and family. Across all countries surveyed, we find that higher trust in scientists and having higher numeracy skills were associated with lower susceptibility to coronavirus-related misinformation. Taken together, these results demonstrate a clear link between susceptibility to misinformation and both vaccine hesitancy and a reduced likelihood to comply with health guidance measures, and suggest that interventions which aim to improve critical thinking and trust in science may be a promising avenue for future research.

The Idea of Liberty, 1600-1800: A Distributional Concept Analysis.

This article uses computational and statistical methods for analyzing the concept of liberty 1600-1800. Based on a bespoke set of tools for parsing conceptual structures it contributes to the literature on the concept of liberty and engages with the thesis concerning negative liberty first put forward by Isaiah Berlin and subsequently modified by Quentin Skinner.

Creating genetic reports that are understood by nonspecialists: a case study.

PURPOSE: Guidelines recommend that genetic reports should be clear to nonspecialists, including patients. We investigated the feasibility of creating reports for cystic fibrosis carrier testing through a rapid user-centered design process that built on a previously developed generic template. We evaluated the new reports' communication efficacy and effects on comprehension against comparable reports used in current clinical practice. METHODS: Thirty participants took part in three rounds of interviews. Usability problems were identified and rectified in each round. One hundred ninety-three participants took part in an evaluation of the resulting reports measuring subjective comprehension, risk probability comprehension, perceived communication efficacy, and other factors, as compared with standard reports. RESULTS: Participants viewing the user-centered reports rated them as clearer, easier to understand, and more effective at communicating key information than standard reports. Both groups ended up with equivalent knowledge of risk probabilities, although we observed differences in how those probabilities were perceived. CONCLUSION: Our findings demonstrate that by starting with a patient-friendly generic report template and modifying it for specific scenarios with a rapid user-centered design process, reports can be produced that are more effective at communicating key information. The resulting reports are now being implemented into clinical care.

Correction: Creating genetic reports that are understood by nonspecialists: a case study.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Projected Impact of Dengue Vaccination in Yucatán, Mexico.

Dengue vaccines will soon provide a new tool for reducing dengue disease, but the effectiveness of widespread vaccination campaigns has not yet been determined. We developed an agent-based dengue model representing movement of and transmission dynamics among people and mosquitoes in Yucatán, Mexico, and simulated various vaccine scenarios to evaluate effectiveness under those conditions. This model includes detailed spatial representation of the Yucatán population, including the location and movement of 1.8 million people between 375,000 households and 100,000 workplaces and schools. Where possible, we designed the model to use data sources with international coverage, to simplify re-parameterization for other regions. The simulation and analysis integrate 35 years of mild and severe case data (including dengue serotype when available), results of a seroprevalence survey, satellite imagery, and climatological, census, and economic data. To fit model parameters that are not directly informed by available data, such as disease reporting rates and dengue transmission parameters, we developed a parameter estimation toolkit called AbcSmc, which we have made publicly available. After fitting the simulation model to dengue case data, we forecasted transmission and assessed the relative effectiveness of several vaccination strategies over a 20 year period. Vaccine efficacy is based on phase III trial results for the Sanofi-Pasteur vaccine, Dengvaxia. We consider routine vaccination of 2, 9, or 16 year-olds, with and without a one-time catch-up campaign to age 30. Because the durability of Dengvaxia is not yet established, we consider hypothetical vaccines that confer either durable or waning immunity, and we evaluate the use of booster doses to counter waning. We find that plausible vaccination scenarios with a durable vaccine reduce annual dengue incidence by as much as 80% within five years. However, if vaccine efficacy wanes after administration, we find that there can be years with larger epidemics than would occur without any vaccination, and that vaccine booster doses are necessary to prevent this outcome.

Graph-Theoretic Properties of Networks Based on Word Association Norms: Implications for Models of Lexical Semantic Memory.

We compared the ability of three different contextual models of lexical semantic memory (BEAGLE, Latent Semantic Analysis, and the Topic model) and of a simple associative model (POC) to predict the properties of semantic networks derived from word association norms. None of the semantic models were able to accurately predict all of the network properties. All three contextual models over-predicted clustering in the norms, whereas the associative model under-predicted clustering. Only a hybrid model that assumed that some of the responses were based on a contextual model and others on an associative network (POC) successfully predicted all of the network properties and predicted a word's top five associates as well as or better than the better of the two constituent models. The results suggest that participants switch between a contextual representation and an associative network when generating free associations. We discuss the role that each of these representations may play in lexical semantic memory. Concordant with recent multicomponent theories of semantic memory, the associative network may encode coordinate relations between concepts (e.g., the relation between pea and bean, or between sparrow and robin), and contextual representations may be used to process information about more abstract concepts.

Archaeology Through Computational Linguistics: Inscription Statistics Predict Excavation Sites of Indus Valley Artifacts.

Computational techniques comparing co-occurrences of city names in texts allow the relative longitudes and latitudes of cities to be estimated algorithmically. However, these techniques have not been applied to estimate the provenance of artifacts with unknown origins. Here, we estimate the geographic origin of artifacts from the Indus Valley Civilization, applying methods commonly used in cognitive science to the Indus script. We show that these methods can accurately predict the relative locations of archeological sites on the basis of artifacts of known provenance, and we further apply these techniques to determine the most probable excavation sites of four sealings of unknown provenance. These findings suggest that inscription statistics reflect historical interactions among locations in the Indus Valley region, and they illustrate how computational methods can help localize inscribed archeological artifacts of unknown origin. The success of this method offers opportunities for the cognitive sciences in general and for computational anthropology specifically.

Encoding sequential information in semantic space models: comparing holographic reduced representation and random permutation.

Circular convolution and random permutation have each been proposed as neurally plausible binding operators capable of encoding sequential information in semantic memory. We perform several controlled comparisons of circular convolution and random permutation as means of encoding paired associates as well as encoding sequential information. Random permutations outperformed convolution with respect to the number of paired associates that can be reliably stored in a single memory trace. Performance was equal on semantic tasks when using a small corpus, but random permutations were ultimately capable of achieving superior performance due to their higher scalability to large corpora. Finally, "noisy" permutations in which units are mapped to other units arbitrarily (no one-to-one mapping) perform nearly as well as true permutations. These findings increase the neurological plausibility of random permutations and highlight their utility in vector space models of semantics.

Reproducing affective norms with lexical co-occurrence statistics: Predicting valence, arousal, and dominance.

Human ratings of valence, arousal, and dominance are frequently used to study the cognitive mechanisms of emotional attention, word recognition, and numerous other phenomena in which emotions are hypothesized to play an important role. Collecting such norms from human raters is expensive and time consuming. As a result, affective norms are available for only a small number of English words, are not available for proper nouns in English, and are sparse in other languages. This paper investigated whether affective ratings can be predicted from length, contextual diversity, co-occurrences with words of known valence, and orthographic similarity to words of known valence, providing an algorithm for estimating affective ratings for larger and different datasets. Our bootstrapped ratings achieved correlations with human ratings on valence, arousal, and dominance that are on par with previously reported correlations across gender, age, education and language boundaries. We release these bootstrapped norms for 23,495 English words.