The influence of viewing time on visual diagnostic accuracy: Less is more.

BACKGROUND: Understanding the factors that contribute to diagnostic errors is critical if we are to correct or prevent them. Some scholars influenced by the default interventionist dual-process theory of cognition (dual-process theory) emphasise a narrow focus on individual clinician's faulty reasoning as a significant contributor. In this paper, we examine the validity of claims that dual process theory is a key to error reduction. METHODS: We examined the relationship between a clinical experience (staff and resident physicians) and viewing time on accuracy for categorising chest X-rays (CXRs) and electrocardiograms (ECGs). In two studies, participants categorised images as normal or abnormal, presented at viewing times of 175, 250, 500 and 1000 ms, to encourage System 1 processing. Study 2 extended viewing times to 1, 5, 10 and 20 s to allow time for System 2 processing and a diagnosis. Descriptives and repeated measures analysis of variance were used to analyse the proportion of true and false positive rates (TP and FP) as well as correct diagnoses. RESULTS: In Study 1, physicians were able to detect abnormal CXRs (0.78) and ECGs (0.67) with relatively high accuracy. The effect of experience was found for ECGs only, as staff physicians (0.71, 95% CI = 0.66-0.75) had higher ECG TP than resident physicians (0.63, 95% CI = 0.58-0.68) in Study 1, and staff had lower ECG FP (0.10, 95% CI = 0.03-0.18) than resident physicians (0.27, 95% CI = 0.20-0.33) in Study 2. In other comparisons, experience was equivocal for ECG FPs and CXR TPs and FPs. In Study 2, overall diagnostic accuracy was similar for both ECGs and CXRs, (0.74). There were small interactions between experience and time for TP in ECGs and FP in CXRs, which are discussed further in the discussion and offer insights into the relationship between processing and experience. CONCLUSION: Overall, our findings raise concerns about the practical application of models that link processing type to diagnostic error, or to specific diagnostic error reduction strategies.

Crowdsourcing a diagnosis? Exploring the accuracy of the size and type of group diagnosis: an experimental study.

BACKGROUND: The consultation process, where a clinician seeks an opinion from another clinician, is foundational in medicine. However, the effectiveness of group diagnosis has not been studied. OBJECTIVE: To compare individual diagnosis to group diagnosis on two dimensions: group size (n=3 or 6) and group process (interactive or artificial groups). METHODOLOGY: Thirty-six internal or emergency medicine residents participated in the study. Initially, each resident worked through four written cases on their own, providing a primary diagnosis and a differential diagnosis. Next, participants formed into groups of three. Using a videoconferencing platform, they worked through four additional cases, collectively providing a single primary diagnosis and differential diagnosis. The process was repeated using a group of six with four new cases. Cases were all counterbalanced. Retrospectively, nominal (ie, artificial) groups were formed by aggregating individual participant data into subgroups of three and six and analytically computing scores. Presence of the correct diagnosis as primary diagnosis or included in the differential diagnosis, as well as the number of diagnoses mentioned, was calculated for all conditions. Means were compared using analysis of variance. RESULTS: For both authentic and nominal groups, the diagnostic accuracy of group diagnosis was superior to individual for both the primary diagnosis and differential diagnosis. However, there was no improvement in diagnostic accuracy when comparing a group of three to a group of six. Interactive and nominal groups were equivalent; however, this may be an artefact of the method used to combine data. CONCLUSIONS: Group diagnosis improves diagnostic accuracy. However, a larger group is not necessarily superior to a smaller group. In this study, interactive group discussion does not result in improved diagnostic accuracy.

Does a deep learning inventory predict knowledge transfer? Linking student perceptions to transfer outcomes.

Students are often encouraged to learn 'deeply' by abstracting generalizable principles from course content rather than memorizing details. So widespread is this perspective that Likert-style inventories are now routinely administered to students to quantify how much a given course or curriculum evokes deep learning. The predictive validity of these inventories, however, has been criticized based on sparse empirical support and ambiguity in what specific outcome measures indicate whether deep learning has occurred. Here we further tested the predictive validity of a prevalent deep learning inventory, the Revised Two-Factor Study Process Questionnaire, by selectively analyzing outcome measures that reflect a major goal of medical education-i.e., knowledge transfer. Students from two undergraduate health sciences courses completed the deep learning inventory before their course's final exam. Shortly after, a random subset of students rated how much each final exam item aligned with three task demands associated with transfer: (1) application of general principles, (2) integration of multiple ideas or examples, and (3) contextual novelty. We then used these ratings from students to examine performance on a subset of exam items that were collectively perceived to demand transfer. Despite good reliability, the resulting transfer outcomes were not substantively predicted by the deep learning inventory. These findings challenge the validity of this tool and others like it.

Intrinsic or Invisible? An Audit of CanMEDS Roles in Entrustable Professional Activities.

PURPOSE: Postgraduate medical education in Canada has quickly transformed to a competency-based model featuring new entrustable professional activities (EPAs) and associated milestones. It remains unclear, however, how these milestones are distributed between the central medical expert role and 6 intrinsic roles of the larger CanMEDS competency framework. A document review was thus conducted to measure how many EPA milestones are classified under each CanMEDS role, focusing on the overall balance between representation of intrinsic roles and that of medical expert. METHOD: Data were extracted from the EPA guides of 40 Canadian specialties in 2021 to measure the percentage of milestones formally linked to each role. Subsequent analyses explored for differences when milestones were separated by stage of postgraduate training, weighted by an EPA's minimum number of observations, or sorted by surgical and medical specialties. RESULTS: Approximately half of all EPA milestones (mean = 48.6%; 95% confidence interval [CI] = 45.9, 51.3) were classified under intrinsic roles overall. However, representation of the health advocate role was consistently low (mean = 2.95%; 95% CI = 2.49, 3.41), and some intrinsic roles-mainly leader, scholar, and professional-were more heavily concentrated in the final stage of postgraduate training. These findings held true under all conditions examined. CONCLUSIONS: The observed distribution of roles in EPA milestones fits with high-level descriptions of CanMEDS in that intrinsic roles are viewed as inextricably linked to medical expertise, implying both are equally important to cultivate through curricula. Yet a fine-grained analysis suggests that a low prevalence or late emphasis of some intrinsic roles may hinder how they are taught or assessed. Future work must explore whether the quantity or timing of milestones shapes the perceived value of each role, and other factors determining the optimal distribution of roles throughout training.

Should electronic differential diagnosis support be used early or late in the diagnostic process? A multicentre experimental study of Isabel.

BACKGROUND: Diagnostic errors unfortunately remain common. Electronic differential diagnostic support (EDS) systems may help, but it is unclear when and how they ought to be integrated into the diagnostic process. OBJECTIVE: To explore how much EDS improves diagnostic accuracy, and whether EDS should be used early or late in the diagnostic process. SETTING: 6 Canadian medical schools. A volunteer sample of 67 medical students, 62 residents in internal medicine or emergency medicine, and 61 practising internists or emergency medicine physicians were recruited in May through June 2020. INTERVENTION: Participants were randomised to make use of EDS either early (after the chief complaint) or late (after the complete history and physical is available) in the diagnostic process while solving each of 16 written cases. For each case, we measured the number of diagnoses proposed in the differential diagnosis and how often the correct diagnosis was present within the differential. RESULTS: EDS increased the number of diagnostic hypotheses by 2.32 (95% CI 2.10 to 2.49) when used early in the process and 0.89 (95% CI 0.69 to 1.10) when used late in the process (both p<0.001). Both early and late use of EDS increased the likelihood of the correct diagnosis being present in the differential (7% and 8%, respectively, both p<0.001). Whereas early use increased the number of diagnostic hypotheses (most notably for students and residents), late use increased the likelihood of the correct diagnosis being present in the differential regardless of one's experience level. CONCLUSIONS AND RELEVANCE: EDS increased the number of diagnostic hypotheses and the likelihood of the correct diagnosis appearing in the differential, and these effects persisted irrespective of whether EDS was used early or late in the diagnostic process.

Intuitive and deliberative approaches for diagnosing 'well' versus 'unwell': evidence from eye tracking, and potential implications for training.

Rapidly assessing how ill a patient is based on their immediate presentation-colloquially termed 'eyeballing' in practice-serves a vital role in acute care settings. Yet surprisingly little is known about how this diagnostic skill is learned or how it should be taught. Some authors have pointed to a dual-process model, suggesting that assessments of illness severity are driven by two distinct types of processing: an intuitive, fast, pattern recognition-like process (Type 1) that depends on many prior patient encounters and outcomes being stored in memory; and a deliberate, slow, analytic process (Type 2) characterized by additional data gathering, data scrutiny, or recollection of rules. But prior studies have supported a dual-process model for the assessment of illness severity only insofar as experienced clinicians chiefly displayed what was presumed to be Type 1 processing. Here we further explored a dual-process model by examining whether less experienced clinicians displayed both types of processing when assessing illness severity across a series of cases. Consistent with the model, a dissociation between Type 1 and Type 2 processing was observed through resident reports of deliberation, response times, and three eye tracking metrics associated with diagnostic expertise. We conclude by discussing potential implications for the training of this enigmatic diagnostic skill.