Crowdsourcing a diagnosis? Exploring the accuracy of the size and type of group diagnosis: an experimental study.

BACKGROUND: The consultation process, where a clinician seeks an opinion from another clinician, is foundational in medicine. However, the effectiveness of group diagnosis has not been studied. OBJECTIVE: To compare individual diagnosis to group diagnosis on two dimensions: group size (n=3 or 6) and group process (interactive or artificial groups). METHODOLOGY: Thirty-six internal or emergency medicine residents participated in the study. Initially, each resident worked through four written cases on their own, providing a primary diagnosis and a differential diagnosis. Next, participants formed into groups of three. Using a videoconferencing platform, they worked through four additional cases, collectively providing a single primary diagnosis and differential diagnosis. The process was repeated using a group of six with four new cases. Cases were all counterbalanced. Retrospectively, nominal (ie, artificial) groups were formed by aggregating individual participant data into subgroups of three and six and analytically computing scores. Presence of the correct diagnosis as primary diagnosis or included in the differential diagnosis, as well as the number of diagnoses mentioned, was calculated for all conditions. Means were compared using analysis of variance. RESULTS: For both authentic and nominal groups, the diagnostic accuracy of group diagnosis was superior to individual for both the primary diagnosis and differential diagnosis. However, there was no improvement in diagnostic accuracy when comparing a group of three to a group of six. Interactive and nominal groups were equivalent; however, this may be an artefact of the method used to combine data. CONCLUSIONS: Group diagnosis improves diagnostic accuracy. However, a larger group is not necessarily superior to a smaller group. In this study, interactive group discussion does not result in improved diagnostic accuracy.

Trainee selection of tasks in postgraduate medical education: Is there a role for 'cherry-picking' to optimise learning?

PURPOSE: Learning is optimised when postgraduate trainees engage in clinical tasks in their zone of proximal development (ZPD). However, workplace learning environments impose additional non-learning goals and additional tasks that may lead to trainees engaging in tasks that do not fall within their ZPD. We do not fully understand how trainees select clinical tasks in the workplace learning environment. If we knew the goals and factors they consider when selecting a task, we could better equip trainees with strategies to select tasks that maximise learning. We explored how postgraduate trainees select clinical tasks using echocardiography interpretation as a model. METHODS: Canadian General Cardiology residents and Echocardiography fellows were invited to participate in semi-structured interviews. Aligning with a theory-informed study, two independent researchers used a deductive, directed content analysis approach to identify codes and themes. RESULTS: Eleven trainees from seven Canadian universities participated (PGY4 = 4, PGY5 = 3, PGY6 = 1 and echocardiography fellows = 3). Goals included learning content, fulfilling assessment criteria and contributing to clinical demands. Trainees switched between goals throughout the day, as it was too effortful for them to engage in tasks within their ZPD at all times. When trainees had sufficient mental effort available, they selected higher complexity tasks that could advance learning content. When available mental effort was low, trainees selected less complex tasks that fulfilled numerically based assessment goals or contributed to clinical demands. Trainees predominantly used perceived complexity of the echocardiogram as a factor to select tasks to achieve their desired goals. CONCLUSION: Postgraduate trainees select tasks within their ZPD that enable them to maximise learning when they perceive to have sufficient mental effort available and workplace affordances are adequate. These findings can inform individual and systemic strategies to maximise learning when selecting tasks.

Challenges facing standardised patients representing equity-deserving groups: Insights from health care educators.

INTRODUCTION: Health professions training programmes increasingly rely on standardised patient (SP) programmes to integrate equity-deserving groups into learning and assessment opportunities. However, little is known about the optimal approach, and many SP programmes struggle to meet these growing needs. This study explored insights from health care educators working with SP programmes to deliver curricular content around equity-deserving groups. METHODS: We interviewed 14 key informants in 2021 who were involved in creating or managing SP-based education. Verbatim transcripts were analysed in an iterative coding process, anchored by qualitative content analysis methodology and informed by two theoretical frameworks: sociologic translation and simulation design. Repeated cycles of data collection and analyses continued until themes could be constructed, aligned with existing theories and grounded in empirical data, with sufficient relevance and robustness to inform educators and curricular leads. RESULTS: Three themes were constructed: (i) creating safety for SPs paid to be vulnerable, (ii) fidelity as an issue broader than who plays the role and (iii) engaging equity-deserving groups. SP work involving traditionally marginalised groups risk re-traumatization, highlighting the importance of (i) informed consent in recruiting SPs, (ii) separating role portrayal from lived experiences, (iii) adequately preparing learners and facilitators, (iv) creating time-outs and escapes for SPs and (v) building opportunity for de-roling with community support. CONCLUSIONS: SP programmes are well positioned to be allies and advocates to equity-deserving groups and to collaborate and share governance of the educational development process from its outset. SP programmes can support the delivery of curricular content around equity-deserving groups by advocating with curricular leadership, building relationships with community partners, facilitating co-creation and co-delivery of educational content and building safety into simulation.

Did You Get That? Predicting Learners' Comprehension of a Video Lecture from Visualizations of Their Gaze Data.

In online lectures, unlike in face-to-face lectures, teachers lack access to (nonverbal) cues to check if their students are still "with them" and comprehend the lecture. The increasing availability of low-cost eye-trackers provides a promising solution. These devices measure unobtrusively where students look and can visualize these data to teachers. These visualizations might inform teachers about students' level of "with-me-ness" (i.e., do students look at the information that the teacher is currently talking about) and comprehension of the lecture, provided that (1) gaze measures of "with-me-ness" are related to comprehension, (2) people not trained in eye-tracking can predict students' comprehension from gaze visualizations, (3) we understand how different visualization techniques impact this prediction. We addressed these issues in two studies. In Study 1, 36 students watched a video lecture while being eye-tracked. The extent to which students looked at relevant information and the extent to which they looked at the same location as the teacher both correlated with students' comprehension (score on an open question) of the lecture. In Study 2, 50 participants watched visualizations of students' gaze (from Study 1), using six visualization techniques (dynamic and static versions of scanpaths, heatmaps, and focus maps) and were asked to predict students' posttest performance and to rate their ease of prediction. We found that people can use gaze visualizations to predict learners' comprehension above chance level, with minor differences between visualization techniques. Further research should investigate if teachers can act on the information provided by gaze visualizations and thereby improve students' learning.

Building a theoretical model for virtual interprofessional education.

BACKGROUND: Virtual interprofessional education (IPE) has emerged as a promising alternative to traditional in-person IPE. However, theoretical frameworks to support virtual interprofessional learning are not well established. Two theoretical frameworks emerged as relevant to virtual IPE: (1) the Canadian Interprofessional Health Collaborative (CIHC) interprofessional learning framework and (2) Dornan's Experience-Based Learning Model (ExBL) of workplace learning. In this study, we sought to explore virtual IPE using both frameworks to develop new theoretical understandings and identify assumptions, gaps and barriers. METHODS: This was a qualitative study. Semi-structured interviews were conducted with medical and nursing student participants (n = 14) and facilitators (n = 3) from virtual IPE workshops. Transcripts were analysed using directed content analysis methodology, informed by the CIHC and ExBL frameworks. Themes were explored using mind-mapping transitional coding. Data collection and analysis were continued iteratively until themes with adequate conceptual depth, relevance and plausibility were identified. RESULTS: Three themes were identified: (1) a shift in the balance of personal and professional, (2) blunted sociologic fidelity and (3) uncertainty and threats to interpersonal connections. Professional distinctions and hierarchies are blurred virtually. This contributed to an increased sense of psychological safety among most learners and lowered the threshold for participation. Separation from workplace sociologic complexity facilitated communication and role clarification objectives. However, loss of immersion may limit deeper engagement. Interprofessional objectives that rely on deeper sociological fidelity, such as conflict resolution, may be threatened. Informal interactions between learners are hindered, which may threaten organic development of interprofessional relationships. CONCLUSIONS: Role clarification and communication objectives are preserved in virtual IPE. Educators should pay close attention to psychological safety and sociologic fidelity-both to leverage advantages and guard against threats to connection and transferability. Virtual IPE may be well suited as a primer to in-person activities or as scaffolding towards interprofessional workplace practice.

Incorporating artificial intelligence in medical diagnosis: A case for an invisible and (un)disruptive approach.

As big data becomes more publicly accessible, artificial intelligence (AI) is increasingly available and applicable to problems around clinical decision-making. Yet the adoption of AI technology in healthcare lags well behind other industries. The gap between what technology could do, and what technology is actually being used for is rapidly widening. While many solutions are proposed to address this gap, clinician resistance to the adoption of AI remains high. To aid with change, we propose facilitating clinician decisions through technology by seamlessly weaving what we call 'invisible AI' into existing clinician workflows, rather than sequencing new steps into clinical processes. We explore evidence from the change management and human factors literature to conceptualize a new approach to AI implementation in health organizations. We discuss challenges and provide recommendations for organizations to employ this strategy.

A Computed Tomography Protocol to Evaluate Coronary Artery Disease Before Transcatheter Aortic Valve Replacement.

BACKGROUND: Transcatheter aortic valve replacement (TAVR) computed tomographic angiography (CTA) images can be used to evaluate coronary artery disease (CAD). METHODS: We conducted a prospective cohort study of consecutive TAVR patients from November 2019 to February 2021 to evaluate TAVR CTA assessment of CAD on the rate of pre-TAVR invasive angiography. Patients had CTA first or invasive angiography first at the discretion of their treating physicians. TAVR CTA scans were categorised as normal/mild CAD, single-vessel disease, high risk (multivessel or left main disease), or nondiagnostic in patients without previous coronary artery bypass grafting (CABG) and as low risk or high risk in patients with previous CABG. Invasive angiography was recommended before TAVR for high-risk or nondiagnostic CTA findings. RESULTS: TAVR was performed on 354 patients; CTA first was performed in 273 and invasive angiography first in 81. Among 231 patients without previous CABG who had CTA first, 22.1% (51/231) had pre-TAVR invasive angiography and 1.3% (3/231) had pre-TAVR revascularisation. Normal/mild CAD or single-vessel disease was found on CTA in 174 patients, of whom 0.5% (1/174) had high-risk disease on invasive angiography. Among 42 patients with previous CABG who had CTA first, 14.3% (6/42) had pre-TAVR invasive angiography and 2.4% (1/42) had pre-TAVR revascularisation. CONCLUSION: TAVR CTA CAD evaluation can avoid pre-TAVR invasive angiography in more than 70% of patients while rarely missing high risk findings. A CTA-first strategy to assess CAD should be considered, especially among patients where conservative management of CAD is preferred.

Should electronic differential diagnosis support be used early or late in the diagnostic process? A multicentre experimental study of Isabel.

BACKGROUND: Diagnostic errors unfortunately remain common. Electronic differential diagnostic support (EDS) systems may help, but it is unclear when and how they ought to be integrated into the diagnostic process. OBJECTIVE: To explore how much EDS improves diagnostic accuracy, and whether EDS should be used early or late in the diagnostic process. SETTING: 6 Canadian medical schools. A volunteer sample of 67 medical students, 62 residents in internal medicine or emergency medicine, and 61 practising internists or emergency medicine physicians were recruited in May through June 2020. INTERVENTION: Participants were randomised to make use of EDS either early (after the chief complaint) or late (after the complete history and physical is available) in the diagnostic process while solving each of 16 written cases. For each case, we measured the number of diagnoses proposed in the differential diagnosis and how often the correct diagnosis was present within the differential. RESULTS: EDS increased the number of diagnostic hypotheses by 2.32 (95% CI 2.10 to 2.49) when used early in the process and 0.89 (95% CI 0.69 to 1.10) when used late in the process (both p<0.001). Both early and late use of EDS increased the likelihood of the correct diagnosis being present in the differential (7% and 8%, respectively, both p<0.001). Whereas early use increased the number of diagnostic hypotheses (most notably for students and residents), late use increased the likelihood of the correct diagnosis being present in the differential regardless of one's experience level. CONCLUSIONS AND RELEVANCE: EDS increased the number of diagnostic hypotheses and the likelihood of the correct diagnosis appearing in the differential, and these effects persisted irrespective of whether EDS was used early or late in the diagnostic process.

Managing cognitive load in simulations: exploring the role of simulation technologists.

BACKGROUND: Facilitating simulation is a complex task with high cognitive load. Often simulation technologists are recruited to help run scenarios and lower some of the extraneous load. We used cognitive load theory to explore the impact of technologists on instructors, identifying sources of instructor cognitive load with and without technologists present. METHODS: Data were collected from 56 simulation sessions for postgraduate emergency medicine residents. Instructors delivered 14 of the sessions without a technologist. After each session, the instructor and simulation technologist (if present) provided quantitative and qualitative data on the cognitive load of the simulation. RESULTS: Instructors rated their cognitive load similarly, regardless of whether simulation technologists were present. However, the composition of their cognitive load differed. Instructors experienced reduced cognitive load related to the simulator and technical resources when technologists were present. Qualitative feedback from instructors suggested real consequences to these differences in cognitive load in (1) perceived complexities in running the scenario, and (2) observations of learners. CONCLUSION: We provide evidence that simulation technologists can remove some of the extraneous load related to the simulator and technical resources for the instructor, allowing the instructor to focus more on observing the learner(s) and tailoring the scenario to their actions.

CONTEXTE: Faciliter la simulation est une tâche complexe qui comporte une charge cognitive élevée. Des technologues en simulation sont souvent recrutés pour aider à exécuter des scénarios et à alléger la charge extrinsèque. Nous avons utilisé la théorie de la charge cognitive pour explorer l'impact des technologues sur les instructeurs, en identifiant les sources de la charge cognitive de l'instructeur avec et sans la présence du technologue. MÉTHODES: Les données ont été recueillies à partir de 56 ateliers de simulation auprès des résidents en médecine d'urgence. Les instructeurs ont animé 14 de ces ateliers sans technologue. Après chaque session, l'instructeur et le technologue en simulation (s'il était présent) ont fourni des données quantitatives et qualitatives sur la charge cognitive associée à la simulation. RÉSULTATS: Les instructeurs ont évalué leur charge cognitive de façon similaire indépendamment de la présence du technologue en simulation. Cependant, la composition de leur charge cognitive était différente. Les instructeurs ont subi une moindre charge cognitive liée au simulateur et aux ressources techniques en présence des technologues. La rétroaction qualitative des instructeurs a suggéré des conséquences réelles liées aux différences de charges cognitives concernant (1) les complexités perçues en exécutant le scénario, et (2) les observations des apprenants. CONCLUSION: Nous fournissons des données probantes suggérant que les technologues en simulation puissent éliminer une partie de la charge extrinsèque liée au simulateur et aux ressources techniques, ce qui permet à l'instructeur de se concentrer davantage sur l'observation de l'apprenant et d'adapter le scénario à leurs actions.

Correction to: Debiasing versus knowledge retrieval checklists to reduce diagnostic error in ECG interpretation.

Due to an unfortunate turn of events, Fig. 3 was omitted from the original publication.

Debiasing versus knowledge retrieval checklists to reduce diagnostic error in ECG interpretation.

There is an ongoing debate regarding the cause of diagnostic errors. One view is that errors result from unconscious application of cognitive heuristics; the alternative is that errors are a consequence of knowledge deficits. The objective of this study was to compare the effectiveness of checklists that (a) identify and address cognitive biases or (b) promote knowledge retrieval, as a means to reduce errors in ECG interpretation. Novice postgraduate year (PGY) 1 emergency medicine and internal medicine residents (n = 40) and experienced cardiology fellows (PGY 4-6) (n = 21) were randomly allocated to three conditions: a debiasing checklist, a content (knowledge) checklist, or control (no checklist) to be used while interpreting 20 ECGs. Half of the ECGs were deliberately engineered to predispose to bias. Diagnostic performance under either checklist intervention was not significantly better than the control. As expected, more errors occurred when cases were designed to induce bias (F = 96.9, p < 0.0001). There was no significant interaction between the instructional condition and level of learner. Checklists attempting to help learners identify cognitive bias or mobilize domain-specific knowledge did not have an overall effect in reducing diagnostic errors in ECG interpretation, although they may help novices. Even when cognitive biases are deliberately inserted in cases, cognitive debiasing checklists did not improve participants' performance.

Do research findings on schema-based instruction translate to the classroom?

INTRODUCTION: Schema-based instruction has been shown to improve diagnostic performance and reduce cognitive load. However, to date, this has only been studied in controlled research settings. More distractions in classrooms may limit generalizability to real-world settings. We evaluated whether schema-based instruction would maintain its effects on cognitive load optimization and performance in a classroom. METHODS: Focused on the approach of interpreting cardiac auscultation findings, 101 first-year medical students at Western University were randomized to receive a traditional (n = 48) or a schema-based lecture (n = 53). Students completed four written questions to test diagnostic performance and a cognitive load assessment at the end of the lecture. Diagnostic performance and cognitive load were compared with independent t-tests. RESULTS: Schema-based instruction was associated with increased diagnostic performance on written questions (64 ± 22 % vs 44 ± 25 % p < 0.001) and reduced intrinsic cognitive load (mean difference = 15 %, standard error 3 %, p < 0.001). There was no significant difference in reported extraneous (p = 0.36) or germane (p = 0.42) cognitive load. CONCLUSIONS: Our results demonstrate that schema-based instruction can be used to reduce intrinsic load and improve diagnostic performance in a real-world classroom setting. The results would be strengthened by replication across other locations and topics.

Cardiac examination and the effect of dual-processing instruction in a cardiopulmonary simulator.

Use of dual-processing has been widely touted as a strategy to reduce diagnostic error in clinical medicine. However, this strategy has not been tested among medical trainees with complex diagnostic problems. We sought to determine whether dual-processing instruction could reduce diagnostic error across a spectrum of experience with trainees undertaking cardiac physical exam. Three experiments were conducted using a similar design to teach cardiac physical exam using a cardiopulmonary simulator. One experiment was conducted in each of three groups: experienced, intermediate and novice trainees. In all three experiments, participants were randomized to receive undirected or dual-processing verbal instruction during teaching, practice and testing phases. When tested, dual-processing instruction did not change the probability assigned to the correct diagnosis in any of the three experiments. Among intermediates, there was an apparent interaction between the diagnosis tested and the effect of dual-processing instruction. Among relative novices, dual processing instruction may have dampened the harmful effect of a bias away from the correct diagnosis. Further work is needed to define the role of dual-processing instruction to reduce cognitive error. This study suggests that it cannot be blindly applied to complex diagnostic problems such as cardiac physical exam.