Teaching medical students to apply deliberate reflection.

PURPOSE: Deliberate reflection on initial diagnosis has been found to repair diagnostic errors. We investigated the effectiveness of teaching students to use deliberate reflection on future cases and whether their usage would depend on their perception of case difficulty. METHOD: One-hundred-nineteen medical students solved cases either with deliberate-reflection or without instructions to reflect. One week later, all participants solved six cases, each with two equally likely diagnoses, but some symptoms in the case were associated with only one of the diagnoses (discriminating features). Participants provided one diagnosis and subsequently wrote down everything they remembered from it. After the first three cases, they were told that the next three would be difficult cases. Reflection was measured by the proportion of discriminating features recalled (overall; related to their provided diagnosis; related to alternative diagnosis). RESULTS: The deliberate-reflection condition recalled more features for the alternative diagnosis than the control condition (p = .013) regardless of described difficulty. They also recalled more features related to their provided diagnosis on the first three cases (p = .004), but on the last three cases (described as difficult), there was no difference. CONCLUSION: Learning deliberate reflection helped students engage in more reflective reasoning when solving future cases.

Improving diagnostic decision support through deliberate reflection: a proposal.

Digital decision support (DDS) is expected to play an important role in improving a physician's diagnostic performance and reducing the burden of diagnostic error. Studies with currently available DDS systems indicate that they lead to modest gains in diagnostic accuracy, and these systems are expected to evolve to become more effective and user-friendly in the future. In this position paper, we propose that a way towards this future is to rethink DDS systems based on deliberate reflection, a strategy by which physicians systematically review the clinical findings observed in a patient in the light of an initial diagnosis. Deliberate reflection has been demonstrated to improve diagnostic accuracy in several contexts. In this paper, we first describe the deliberate reflection strategy, including the crucial element that would make it useful in the interaction with a DDS system. We examine the nature of conventional DDS systems and their shortcomings. Finally, we propose what DDS based on deliberate reflection might look like, and consider why it would overcome downsides of conventional DDS.

The use of deliberate reflection to reduce confirmation bias among orthopedic surgery residents / O uso da reflexão deliberada para reduzir o viés de confirmação em residentes de ortopedia

Introduction: cognitive biases might affect decision-making processes such as clinical reasoning and confirmation bias is among the most important ones. The use of strategies that stimulate deliberate reflection during the diagnostic process seems to reduce availability bias, but its effect in reducing confirmation bias needs to be evaluated. Aims: to examine whether deliberate reflection reduces confirmation bias and increases the diagnostic accuracy of orthopedic residents solving written clinical cases. Methods: experimental study comparing the diagnostic accuracy of orthopedic residents in the resolution of eight written clinical cases containing a referral diagnosis. Half of the written cases had a wrong referral diagnosis. One group of residents used deliberate reflection (RG), which stimulates comparison and contrast of clinical hypotheses in a systematic manner, and a control group (CG), was asked to provide differential diagnoses with no further instruction. The study included 55 third-year orthopedic residents, 27 allocated to the RG and 28 to the CG. Results: residents on the RG had higher diagnostic scores than the CG for clinical cases with a correct referral diagnosis (62.0±20.1 vs. 49.1±21.0 respectively; p = 0.021). For clinical cases with incorrect referral diagnosis, diagnostic accuracy was similar between residents on the RG and those on the CG (39.8±24.3 vs. 44.6±26.7 respectively; p = 0.662). We observed an overall confirmation bias in 26.3% of initial diagnoses (non-analytic phase) and 19.5% of final diagnoses (analytic phase) when solving clinical cases with incorrect referral diagnosis. Residents from RG showed a reduction in confirmation of incorrect referral diagnosis when comparing the initial diagnosis given in the non-analytic phase with the one provided as the final diagnosis (25.9±17.7 vs. 17.6±18.1, respectively; Cohen d: 0.46; p = 0.003). In the CG, the reduction in the confirmation of incorrect diagnosis was not statistically significant. Conclusions:confirmation bias was present when residents solved written clinical cases with incorrect referral diagnoses, and deliberate reflection reduced such bias. Despite the reduction in confirmation bias, diagnostic accuracy of residents from the RG was similar to those from the CG when solving the set of clinical cases with a wrong referral diagnosis.

Introdução: os vieses cognitivos podem afetar tanto os processos de tomada de decisão como o raciocínio clínico e o viés de confirmação está entre os mais importantes. O uso de estratégias que estimulem a reflexão deliberada durante o processo diagnóstico parece reduzir o viés de disponibilidade, mas seu efeito na redução do viés de confirmação precisa ser avaliado. Objetivos: examinar se a reflexão deliberada reduz o viés de confirmação e aumenta a acurácia do diagnóstico de residentes de ortopedia ao resolverem casos clínicos escritos. Métodos: estudo experimental comparando a acurácia diagnóstica de residentes de ortopedia na resolução de oito casos clínicos escritos contendo um diagnóstico de encaminhamento. Metade dos casos escritos tinha um diagnóstico de encaminhamento errado. Um grupo de residentes utilizou a reflexão deliberada (GR), que estimula a comparação e o contraste de hipóteses clínicas de maneira sistemática, e um grupo controle (GC) foi solicitado a fornecer diagnósticos diferenciais sem maiores instruções. O estudo incluiu 55 residentes de ortopedia do terceiro ano, 27 alocados no GR e 28 no GC. Resultados: residentes no GR tiveram escores diagnósticos mais altos do que o GC para casos clínicos com um diagnóstico de encaminhamento correto (62,0±20,1 vs. 49,1±21,0 respectivamente; p = 0,021). Para os casos clínicos com diagnóstico de encaminhamento incorreto, a acurácia diagnóstica foi semelhante entre os residentes do GR e os do GC (39,8±24,3 vs. 44,6±26,7 respectivamente; p = 0,662). Observamos viés geral de confirmação em 26,3% dos diagnósticos iniciais (fase não analítica) e 19,5% dos diagnósticos finais (fase analítica) na resolução de casos clínicos com diagnóstico de encaminhamento incorreto. Os residentes do GR mostraram uma redução na confirmação do diagnóstico de encaminhamento incorreto ao comparar o diagnóstico inicial dado na fase não analítica com aquele fornecido como diagnóstico final (25,9±17,7 vs. 17,6±18,1, respectivamente; Cohen d: 0,46; p = 0,003). No GC, a redução na confirmação do diagnóstico incorreto não foi estatisticamente significativa. Conclusões: o viés de confirmação esteve presente quando os residentes resolveram casos clínicos escritos com diagnósticos de encaminhamento incorretos e a reflexão deliberada reduziu esse viés. Apesar da redução do viés de confirmação, a acurácia diagnóstica dos residentes do GR foi semelhante à do GC na solução do conjunto de casos clínicos com diagnóstico de encaminhamento incorreto.

Effects of deliberate reflection on diagnostic accuracy, confidence and diagnostic calibration in dermatology.

BACKGROUND: Deliberate reflection on initial diagnoses improved diagnostic accuracy in internal medicine and general practice, but it is unknown if the same occurs in specialties that rely mostly on visual perception, such as dermatology. Moreover, whether reflection influences diagnostic calibration has not been studied yet. Diagnostic calibration, the relationship between diagnostic accuracy and confidence in that accuracy, affects diagnostic performance because overconfidence tends to induce premature closure. This study evaluated the effects of deliberate reflection on diagnostic accuracy and diagnostic calibration in dermatology. METHODS: Sixty-one sixth-year students from a Brazilian medical school were allocated to either a reflection group (RG) or a control group (CG). In both groups, students worked with the same 12 dermatological images, presented sequentially, providing an initial diagnosis and confidence in that diagnosis. Subsequently, RG students reflected on the case using a structured procedure, while CG students performed a time-filler activity. All students then provided a final diagnosis and confidence in that diagnosis. Outcome measurements were diagnostic accuracy, confidence, and calibration. RESULTS: Reflection increased diagnostic accuracy relative to control (49.7 ± 12.1 vs 38.4 ± 14.6; p = 0.002) but did not affect confidence (64.3 ± 13.2 vs 58.9 ± 20.1; p = 0.228) nor calibration (0.15 ± 0.16 vs 0.20 ± 0.19, p = 0.197). Overall, case difficulty influenced calibration, with students showing more overconfidence on more difficult cases (p <0.001). CONCLUSIONS: Deliberate reflection increased diagnostic accuracy in dermatology but did not affect confidence and calibration. Calibration was worse on more difficult cases, suggesting that calibration is a knowledge-related phenomenon.