ABSTRACT
A growing body of research indicates that transparent communication of statistical uncertainty around facts and figures does not undermine credibility. However, the extent to which these findings apply in the context of the COVID-19 pandemic--rife with uncertainties--is unclear. In a large international survey experiment, (Study 1; N = 10,519) we report that communicating uncertainty around COVID-19 statistics in the form of a numeric range (vs. no uncertainty) may lead to slightly lower trust in the number presented but has no impact on trust in the source of the information. We also report the minimal impact of numeric uncertainty on trust is consistent across estimates of current or future COVID-19 statistics (Study 2) and figures relating to environmental or economic research, rather than the pandemic (Study 3). Conversely, we find imprecise statements about the mere existence of uncertainty without quantification can undermine both trust in the numbers and their source - though effects vary across countries and contexts. Communicators can be transparent about statistical uncertainty without concerns about undermining perceptions of their trustworthiness, but ideally should aim to use numerical ranges rather than verbal statements.
ABSTRACT
ObjectivesTo assess the effects of different official information on public interpretation of a personal COVID-19 PCR ( swab) test result. DesignA 5xx2 factorial, randomised, between-subjects experiment, comparing four wordings of information about the test result and a control arm of no additional information; for both positive and negative test results. SettingOnline experiment using recruitment platform Respondi. ParticipantsUK participants (n=1,744, after a pilot of n=1,657) collected by quota sampling to be proportional to the UK national population on age and sex. InterventionsParticipants were given a hypothetical COVID-19 swab test result for John who was presented as having a 50% chance of having COVID-19 based on symptoms alone. Participants were randomised to receive either a positive or negative result for John, then randomised again to receive either no more information, or text information on the interpretation of COVID-19 test results copied from the public websites of the UKs National Health Service, the USs Centers for Disease Control, New Zealands Ministry of Health, or a modified version of the UKs wording incorporating uncertainty. Information identifying the source of the wording was removed. Main outcome measuresParticipants were asked "What is your best guess as to the percent chance that John actually had COVID-19 at the time of his test, given his result?"; questions about their feelings of trustworthiness in the result, their perceptions of the quality of the underlying evidence, and what action they felt John should take in the light of his result. ResultsOf those presented with a positive COVID-19 test result for John, the mean estimate of the probability that he had the virus was 73%; for those presented with a negative result, 38%. There was no main effect of information (wording) on these means. However, those participants given the official information on the UK website, which did not mention any uncertainty around the test result, were significantly more likely to give a categorical (100% or 0%) answer (for positive result, p <.001; negative, p =.006). When asked how much they agreed that John should self-isolate, those who were told his test was positive agreed to a greater extent (mean 86 on a 0-100 scale), but many of those who were told he had a negative result still agreed (mean 51). There was also an interaction between wording and test result (p < 0.001), with those seeing the New Zealand wording about the uncertainties of the test result significantly more likely to agree that he should continue to self-isolate after a negative test than those who saw the UK wording (p =.01), the experimental wording (p =.02) or no wording at all (p =.003). Participants rated positive test results more trustworthy and higher quality of evidence than negative results. ConclusionsThe UK public perceive positive test results for COVID-19 as more reliable and trustworthy than negative results without being given any information about the reliability of the tests. When additionally given the UKs current official wording about the interpretation of the test results, people became more likely to interpret the results as definitive. The publics assessment of the face value of both the positive and negative test results was generally conservative. The proportion of participants who felt that a symptomatic individual who tests negative definitely should not self-isolate was highest among those reading the UK wording (17.4%) and lowest among those reading the New Zealand wording (3.8%) and US wording (5.1%). Pre-registration and data repositorypre-registration of pilot at osf.io/8n62f, pre-registration of main experiment at osf.io/7rcj4, data and code in https://osf.io/pvhba/. What is already known on this topicO_LIDifferent countries have had different approaches to conveying the meaning of a COVID-19 swab test result, particularly regarding the uncertainties inherent in the result due to limitations of specificity and sensitivity. C_LIO_LIPrevious research has suggested that peoples trust and understanding is not affected by conveying quantified uncertainties numerically, but that perceptions of the quality of the underlying evidence can affect trust. C_LIO_LIIt is not known whether the different wordings around COVID-19 test uncertainties are likely to affect peoples trust in, or behavioural response to, the results they receive. C_LI What this study addsO_LIThis study provides the first empirical evidence to our knowledge of the responses the public have to COVID-19 swab test results. C_LIO_LIIt suggests that the public have a higher degree of trust and confidence in positive swab test results than negative when they are not given any other information accompanying the result. The experimental wording that we created for this study appeared to boost their trust in and assessment of quality of positive test results, but did not change their lower ratings of negative results. C_LIO_LIThe wording used by the UKs National Health Service, which does not include any cues of uncertainty in the result, was more likely to lead people to definitive (100% or 0%) answers to questions about the meaning of the result. C_LIO_LIThe wording used by New Zealands Ministry of Health, which is more explicit about the reliability of the tests, appears to lead people to be more cautious about recommending that a test participant with a negative test (but still symptomatic) no longer needs to self-isolate. C_LI
ABSTRACT
As increasing amounts of data accumulate on the effects of the novel coronavirus Sars-CoV-2 and the risk factors that lead to poor outcomes, it is possible to produce personalised estimates of the risks faced by groups of people with different characteristics. The challenge of how to communicate these then becomes apparent. Based on empirical work (total n=5,520, UK) supported by in-person interviews with the public and physicians, we make recommendations on the presentation of such information. These include: using predominantly percentages when communicating the absolute risk, but also providing, for balance, a format which conveys a contrasting (higher) perception of risk (expected frequency out of 10,000); using a visual linear scale cut at an appropriate point to illustrate the maximum risk, explained through an illustrative persona who might face that highest level of risk; and providing context to the absolute risk through presenting a range of other personas illustrating people who would face risks of a wide range of different levels. These personas should have their major risk factors (age, existing health conditions) described. By contrast, giving people absolute likelihoods of other risks they face in an attempt to add context was considered less helpful.
