Analysis of Physicians' Probability Estimates of a Medical Outcome Based on a Sequence of Events.

Importance: The probability of a conjunction of 2 independent events is the product of the probabilities of the 2 components and therefore cannot exceed the probability of either component; violation of this basic law is called the conjunction fallacy. A common medical decision-making scenario involves estimating the probability of a final outcome resulting from a sequence of independent events; however, little is known about physicians' ability to accurately estimate the overall probability of success in these situations. Objective: To ascertain whether physicians are able to correctly estimate the overall probability of a medical outcome resulting from 2 independent events. Design, Setting, and Participants: This survey study consisted of 3 separate substudies, in which 215 physicians were asked via internet-based survey to estimate the probability of success of each of 2 components of a diagnostic or prognostic sequence as well as the overall probability of success of the 2-step sequence. Substudy 1 was performed from April 2 to 4, 2021, substudy 2 from November 2 to 11, 2021, and substudy 3 from May 13 to 19, 2021. All physicians were board certified or board eligible in the primary specialty germane to the substudy (ie, obstetrics and gynecology for substudies 1 and 3 and pulmonology for substudy 2), were recruited from a commercial survey service, and volunteered to participate in the study. Exposures: Case scenarios presented in an online survey. Main Outcomes and Measures: Respondents were asked to provide their demographic information in addition to 3 probability estimates. The first substudy included a scenario describing a brow presentation discovered during labor; the 2 conjuncts were the probabilities that the brow presentation would resolve and that the delivery would be vaginal. The second substudy involved a diagnostic evaluation of an incidentally discovered pulmonary nodule; the 2 conjuncts were the probabilities that the patient had a malignant condition and that a technically successful transthoracic needle biopsy would reveal a malignant condition. The third substudy included a modification of the first substudy in an attempt to debias the conjunction fallacy prevalent in the first substudy. Respondents' own probability estimates of the individual events were used to calculate the mathematically correct conjunctive probability. Results: Among 215 respondents, the mean (SD) age was 54.0 (9.5) years; 142 respondents (66.0%) were male. Data on race and ethnicity were not collected. A total of 168 physicians (78.1%) estimated the probability of the 2-step sequence to be greater than the probability of at least 1 of the 2 component events. Compared with the product of their 2 estimated components, respondents overestimated the combined probability by 12.8% (95% CI, 9.6%-16.1%; P < .001) in substudy 1, 19.8% (95% CI, 16.6%-23.0%; P < .001) in substudy 2, and 18.0% (95% CI, 13.4%-22.5%; P < .001) in substudy 3, results that were mathematically incoherent (ie, formally illogical and mathematically incorrect). Conclusions and Relevance: In this survey study of 215 physicians, respondents consistently overestimated the combined probability of 2 events compared with the probability calculated from their own estimates of the individual events. This biased estimation, consistent with the conjunction fallacy, may have substantial implications for diagnostic and prognostic decision-making.

Practical consequences of flawed social psychological research on bias.

The flaws in social psychological research pointed out by Cesario have societal costs. These include ignoring crucial base rates thereby degrading the effectiveness of policy decisions, generalizing the conclusions derived from experiments on non-professionals thereby distorting the public's view of professional law enforcement personnel, questionable accusations of racism, and mis-attributions of the causes of racial differences in behavior.

Conflicting Goals Influence Physicians' Expressed Beliefs to Patients and Colleagues.

BACKGROUND: Physicians who communicate their prognostic beliefs to patients must balance candor against other competing goals, such as preserving hope, acknowledging the uncertainty of medicine, or motivating patients to follow their treatment regimes. OBJECTIVE: To explore possible differences between the beliefs physicians report as their own and those they express to patients and colleagues. DESIGN: An online panel of 398 specialists in internal medicine who completed their medical degrees and practiced in the United States provided their estimated diagnostic accuracy and prognostic assessments for a randomly assigned case. In addition, they reported the diagnostic and prognostic assessments they would report to patients and colleagues more generally. Physicians answered questions about how and why their own beliefs differed from their expressed beliefs to patients and colleagues in the specific case and more generally in their practice. RESULTS: When discussing beliefs about prognoses to patients and colleagues, most physicians expressed beliefs that differed from their own beliefs. Physicians were more likely to express greater optimism when talking to patients about poor prognoses than good prognoses. Physicians were also more likely to express greater uncertainty to patients when prognoses were poor than when they were good. The most common reasons for the differences between physicians' own beliefs and their expressed beliefs were preserving hope and acknowledging the inherent uncertainty of medicine. CONCLUSION: To balance candor against other communicative goals, physicians tended to express beliefs that were more optimistic and contained greater uncertainty than the beliefs they said were their own, especially in discussions with patients whose prognoses were poor.

Author Correction: A consensus-based transparency checklist.

An amendment to this paper has been published and can be accessed via a link at the top of the paper.

Anesthesiologists' Overconfidence in Their Perceived Knowledge of Neuromuscular Monitoring and Its Relevance to All Aspects of Medical Practice: An International Survey.

BACKGROUND: In patients who receive a nondepolarizing neuromuscular blocking drug (NMBD) during anesthesia, undetected postoperative residual neuromuscular block is a common occurrence that carries a risk of potentially serious adverse events, particularly postoperative pulmonary complications. There is abundant evidence that residual block can be prevented when real-time (quantitative) neuromuscular monitoring with measurement of the train-of-four ratio is used to guide NMBD administration and reversal. Nevertheless, a significant percentage of anesthesiologists fail to use quantitative devices or even conventional peripheral nerve stimulators routinely. Our hypothesis was that a contributing factor to the nonutilization of neuromuscular monitoring was anesthesiologists' overconfidence in their knowledge and ability to manage the use of NMBDs without such guidance. METHODS: We conducted an Internet-based multilingual survey among anesthesiologists worldwide. We asked respondents to answer 9 true/false questions related to the use of neuromuscular blocking drugs. Participants were also asked to rate their confidence in the accuracy of each of their answers on a scale of 50% (pure guess) to 100% (certain of answer). RESULTS: Two thousand five hundred sixty persons accessed the website; of these, 1629 anesthesiologists from 80 countries completed the 9-question survey. The respondents correctly answered only 57% of the questions. In contrast, the mean confidence exhibited by the respondents was 84%, which was significantly greater than their accuracy. Of the 1629 respondents, 1496 (92%) were overconfident. CONCLUSIONS: The anesthesiologists surveyed expressed overconfidence in their knowledge and ability to manage the use of NMBDs. This overconfidence may be partially responsible for the failure to adopt routine perioperative neuromuscular monitoring. When clinicians are highly confident in their knowledge about a procedure, they are less likely to modify their clinical practice or seek further guidance on its use.

Forecasting tournaments, epistemic humility and attitude depolarization.

People often express political opinions in starkly dichotomous terms, such as "Trump will either trigger a ruinous trade war or save U.S. factory workers from disaster." This mode of communication promotes polarization into ideological in-groups and out-groups. We explore the power of an emerging methodology, forecasting tournaments, to encourage clashing factions to do something odd: to translate their beliefs into nuanced probability judgments and track accuracy over time and questions. In theory, tournaments advance the goals of "deliberative democracy" by incentivizing people to be flexible belief updaters whose views converge in response to facts, thus depolarizing unnecessarily polarized debates. We examine the hypothesis that, in the process of thinking critically about their beliefs, tournament participants become more moderate in their own political attitudes and those they attribute to the other side. We view tournaments as belonging to a broader class of psychological inductions that increase epistemic humility and that include asking people to explore alternative perspectives, probing the depth of their cause-effect understanding and holding them accountable to audiences with difficult-to-guess views.

Consensus Statement on Perioperative Use of Neuromuscular Monitoring.

A panel of clinician scientists with expertise in neuromuscular blockade (NMB) monitoring was convened with a charge to prepare a consensus statement on indications for and proper use of such monitors. The aims of this article are to: (a) provide the rationale and scientific basis for the use of quantitative NMB monitoring; (b) offer a set of recommendations for quantitative NMB monitoring standards; (c) specify educational goals; and (d) propose training recommendations to ensure proper neuromuscular monitoring and management. The panel believes that whenever a neuromuscular blocker is administered, neuromuscular function must be monitored by observing the evoked muscular response to peripheral nerve stimulation. Ideally, this should be done at the hand muscles (not the facial muscles) with a quantitative (objective) monitor. Objective monitoring (documentation of train-of-four ratio ≥0.90) is the only method of assuring that satisfactory recovery of neuromuscular function has taken place. The panel also recommends that subjective evaluation of the responses to train-of-four stimulation (when using a peripheral nerve stimulator) or clinical tests of recovery from NMB (such as the 5-second head lift) should be abandoned in favor of objective monitoring. During an interim period for establishing these recommendations, if only a peripheral nerve stimulator is available, its use should be mandatory in any patient receiving a neuromuscular blocking drug. The panel acknowledges that publishing this statement per se will not result in its spontaneous acceptance, adherence to its recommendations, or change in routine practice. Implementation of objective monitoring will likely require professional societies and anesthesia department leadership to champion its use to change anesthesia practitioner behavior.

Why do patients derogate physicians who use a computer-based diagnostic support system?

OBJECTIVE: To better understand 1) why patients have a negative perception of the use of computerized clinical decision support systems (CDSSs) and 2) what contributes to the documented heterogeneity in the evaluations of physicians who use a CDSS. METHODS: Three vignette-based studies examined whether negative perceptions stemmed directly from the use of a computerized decision aid or the need to seek external advice more broadly (experiment 1) and investigated the contributing role of 2 individual difference measures, attitudes toward statistics (ATS; experiment 2) and the Multidimensional Health Locus of Control Scale (MHLC; experiment 3), to these findings. RESULTS: A physician described as making an unaided diagnosis was rated significantly more positively on a number of attributes than a physician using a computerized decision aid but not a physician who sought the advice of an expert colleague (experiment 1). ATS were unrelated to perceptions of decision aid use (experiment 2); however, greater internal locus of control was associated with more positive feelings about unaided care and more negative feelings about care when a decision aid was used (experiment 3). CONCLUSION: Negative perceptions of computerized decision aid use may not be a product of the need to seek external advice more generally but may instead be specific to the use of a nonhuman tool and may be associated with individual differences in locus of control. Together, these 3 studies may be used to guide education efforts for patients.

Psychological research and the prostate-cancer screening controversy.

In October of 2011, the U.S. Preventive Services Task Force released a draft report in which they recommended against using the prostate-specific antigen (PSA) test to screen for prostate cancer. We attempt to show that four factors documented by psychological research can help explain the furor that followed the release of the task force's report. These factors are the persuasive power of anecdotal (as opposed to statistical) evidence, the influence of personal experience, the improper evaluation of data, and the influence of low base rates on the efficacy of screening tests. We suggest that augmenting statistics with facts boxes or pictographs might help such committees communicate more effectively with the public and with the U.S. Congress.

Are residents' decisions influenced more by a decision aid or a specialist's opinion? A randomized controlled trial.

BACKGROUND: Physicians are reluctant to use decision aids despite their ability to improve care. A potential reason may be that physicians do not believe decision aid advice. OBJECTIVE: To determine whether internal medicine residents lend more credence to contradictory decision aid or human advice. DESIGN: Randomized controlled trial. Residents read a scenario of a patient with community-acquired pneumonia and were asked whether they would admit the patient to the intensive care unit or the floor. Residents were randomized to receive contrary advice from either a referenced decision aid or an anonymous pulmonologist. They were then asked, in light of this new information, where they would admit the patient. PARTICIPANTS: One hundred eight internal medicine residents. MEASUREMENTS: The percentage of residents who changed their admission location and the change in confidence in the decision. MAIN RESULTS: Residents were more likely to change their original admission location (OR 2.3, 95% CI 1.04 to 5.1, P = 0.04) and to reduce their confidence in the decision (adjusted difference between means -12.9%, 95% CI -3.0% to -22.8%, P = 0.011) in response to the referenced decision aid than to the anonymous pulmonologist. Confidence in their decision was more likely to change if they initially chose to admit the patient to the floor. CONCLUSIONS: In a hypothetical case of community-acquired pneumonia, physicians were influenced more by contrary advice from a referenced decision aid than an anonymous specialist. Whether this holds for advice from a respected specialist or in actual practice remains to be studied.

The influence of treatment effect size on willingness to adopt a therapy.

BACKGROUND: Physicians are slow to adopt novel therapies, and the reasons for this are poorly understood. The authors sought to determine if the size of the treatment effect of a novel therapy influences willingness to adopt it. METHODS: We developed 2 experimental vignette pairs describing a trial of a therapy for a hypothetical disease that showed a statistically significant mortality benefit. The size of the mortality effect was varied in vignettes of a pair (3% v. 10%). The 2 experimental vignette pairs differed in whether study enrollment was reported. Vignettes were mailed to a random sample of physicians using an intersubject design. The main study outcome was respondents' willingness to adopt the hypothetical therapy, based on the results of the hypothetical trial. RESULTS: There were 124 and 89 respondents to vignette pairs 1 and 2, respectively. In vignette pair 1, 91% versus 71% of respondents adopted the therapy when it reduced mortality by 10% and 3%, respectively (P = 0.0058). For vignette pair 2, 88% versus 51% of respondents adopted the therapy when it reduced mortality by 10% and 3%, respectively (P = 0.0002). In both vignette pairs, nonadopters were more likely than adopters to report side effects of the therapy as a principal reason for their decision. CONCLUSIONS: In this study, respondents were less likely to adopt a lifesaving therapy if its associated mortality reduction was 3% compared to 10%. Because most therapies for major medical conditions reduce mortality within or below this range, and because there were no opportunity costs associated with the adoption of the therapy, we believe that this effect represents a bias. Further investigation will be required to determine its prevalence and mechanism.

My loss is your loss ... Sometimes: loss aversion and the effect of motivational biases.

Findings from previous studies of individual decision-making behavior predict that losses will loom larger than gains. It is less clear, however, if this loss aversion applies to the way in which individuals attribute value to the gains and losses of others, or if it is robust across a broad spectrum of policy and management decision contexts. Consistent with previous work, the results from a series of experiments reported here revealed that subjects exhibited loss aversion when evaluating their own financial gains and losses. The presence of loss aversion was also confirmed for the way in which individuals attribute value to the financial gains and losses of others. However, similar evaluations within social and environmental contexts did not exhibit loss aversion. In addition, research subjects expected that individuals who were unknown to them would significantly undervalue the subjects' own losses across all contexts. The implications of these findings for risk-based policy and management are many. Specifically, they warrant caution when relying upon loss aversion to explain or predict the reaction of affected individuals to risk-based decisions that involve moral or protected values. The findings also suggest that motivational biases may lead decisionmakers to assume that their attitudes and beliefs are common among those affected by a decision, while those affected may expect unfamiliar others to be unable to identify and act in accordance with shared values.

Patient and surrogate disagreement in end-of-life decisions: can surrogates accurately predict patients' preferences?

BACKGROUND: When a patient is too incapacitated to make important end-of-life decisions, doctors may ask a preappointed surrogate to predict the patient's preferences and make decisions on the patient's behalf. The current study investigates whether surrogates project their own views onto what they predict the patients' preferences are. METHODS: Using data from seriously ill patients and their surrogates, the authors created a "projection'' variable that addresses the following question: When surrogates are asked to predict a patient's end-of-life preferences, do they mistakenly replace this prediction with what they would want the patient to do? The authors examined the 144 patient-surrogate pairs in which surrogates inaccurately predicted patients' CPR (cardiopulmonary resuscitation) v. DNR (do not resuscitate) decisions and the 294 pairs in which surrogates inaccurately predicted patients' extend life v. relieve pain preferences. Among these patient-surrogate pairs, the authors determined the extent to which surrogates' wishes for the patient matched their incorrect predictions of what the patient wanted. RESULTS: Of the patient-surrogate pairs who disagreed on CPR v. DNR and extend life v. relieve pain preferences, 62.5% and 88.4% of surrogates demonstrated projection for CPR v. DNR decisions and extend life v. relieve pain preferences, respectively. Age-related and demographic variables did not predict cases in which projection did and did not occur. CONCLUSION: When surrogates inaccurately predict the CPR v. DNR and extend life v. relieve pain preferences of seriously ill, hospitalized loved ones, surrogates' prediction errors often represent surrogates' own wishes for the patient.

The influence of a physician's use of a diagnostic decision aid on the malpractice verdicts of mock jurors.

BACKGROUND: Mock jurors were more likely to side with the physician-defendant if he recommended an operation when there were many symptoms and refrained when there were few symptoms compared with a physician who did the converse. The use of a decision aid had no influence on this binary standard-of-care decision. Among those physicians deemed liable by the jurors, defying the aid resulted in heightened punishment compared with heeding it. CONCLUSION: . Contrary to many physicians' fears, use of a diagnostic decision aid did not influence the likelihood of an adverse malpractice verdict. Complying with the aid's recommendation provided a measure of protection against jurors' punitiveness for those physicians deemed liable for malpractice.

Patients derogate physicians who use a computer-assisted diagnostic aid.

OBJECTIVE: To ascertain whether a physician who uses a computer-assisted diagnostic support system (DSS) would be rated less capable than a physician who does not. METHOD: Students assumed the role of a patient with a possible ankle fracture (experiment 1) or a possible deep vein thrombosis (experiment 2). They read a scenario that described an interaction with a physician who used no DSS, one who used an unspecified DSS, or one who used a DSS developed at a prestigious medical center. Participants were then asked to rate the interaction on 5 criteria, the most important of which was the diagnostic ability of the physician. In experiment 3, 74 patients in the waiting room of a clinic were randomly assigned to the same 3 types of groups as used in experiment 1. In experiment 4, 131 3rd- and 4th-year medical students read a scenario of a physician-patient interaction and were randomly assigned to 1 of 4 groups: the physician used no DSS, heeded the recommendation of a DSS, defied a recommendation of a DSS by treating in a less aggressive manner, or defied a recommendation of a DSS by treating in a more aggressive manner . RESULTS: The participants always deemed the physician who used no decision aid to have the highest diagnostic ability. CONCLUSION: Patients may surmise that a physician who uses a DSS is not as capable as a physician who makes the diagnosis with no assistance from a DSS.