Focusing on what matters: Restructuring the presentation of Bayesian reasoning problems.

We propose that a mismatch in problem presentation and question structures may promote errors on Bayesian reasoning problems. In this task, people determine the likelihood that a positive test actually indicates the presence of a condition. Research has shown that people routinely fail to correctly identify this positive predictive value (PPV). We point out that the typical problem structure is likely to confuse reasoners by focusing on the incorrect reference class for answering this diagnostic question; instead, providing the anchor needed to address the diagnostic question about sensitivity (SEN). Results of two experiments are described in which participants answer diagnostic questions using problems presented with congruent or incongruent reference classes. Aligning reference classes eases both representational and computational difficulties, increasing the proportion who were consistently accurate to an unprecedented 93% on PPV questions, and 69% on SEN questions. Analysis of response components from incongruent problems indicated that many errors reflect difficulties in identifying and applying appropriate values from the problem, which are prerequisite processes that contribute to computational errors. We conclude with a discussion of the need, especially in applied settings and on initial exposure, to adopt problem presentations to guide, rather than confuse, the organization and use of diagnostic information. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Improving Understanding of Diagnostic Test Outcomes.

BACKGROUND: Understanding diagnostic test outcomes requires determining the positive predictive value (PPV) of the test, which most laypeople and medical professionals struggle to do. Despite advances found with frequency formats and visual aids, less than 40% of people can typically identify this value. This study tests the impact of using congruent reference classes in problem-question pairings, evaluates the role of numeracy, and assesses how diagnostic value estimates affect the reported likelihood to use the test. METHOD: A 3 × 2, Pairing (congruent test-focus, congruent condition-focus, incongruent) × Response Format (frequency, percentage) mixed design experiment was conducted, in which participants answered diagnostic questions about 7 medical problems presented in a format focusing either on the reference class of those who test positive or those who have the condition. Answer accuracy, numeracy, and ratings of likelihood to use estimates were assessed. RESULTS: Focusing on the congruent test reference class allowed 87% of participants to consistently identify the PPV, and focusing on the congruent condition reference class led 63% of participants to consistently identify the sensitivity (SEN). Aligning reference classes was especially beneficial for those with lower numeracy, increasing accuracy on problems from 21% for incongruent pairings to 66% for congruent pairings. Ratings of likelihood to use the test were closely tied to participants' estimates of diagnostic values, regardless of the accuracy of those estimates. CONCLUSIONS: Although often overlooked, a straightforward mapping of reference classes from the relevant diagnostic information to the question of interest reduces confusion and substantially increases accuracy in estimates of diagnostic values. These findings can be used to strengthen training in the assessment of uncertainties associated with medical test results.