Picking Apart a Program Evaluation Committee: A Multiple Case Study Characterizing Primary Care Residency Program Evaluation Committee Structure, Program Improvement, and Outcomes.

BACKGROUND: As of 2014, the Accreditation Council for Graduate Medical Education (ACGME) mandates initiating a Program Evaluation Committee (PEC) to guide ongoing program improvement. However, little guidance nor published reports exist about how individual PECs have undertaken this mandate. OBJECTIVE: To explore how four primary care residency PECs configure their committees, review program goals and undertake program evaluation and improvement. METHODS: We conducted a multiple case study between December 2022 and April 2023 of four purposively selected primary care residencies (e.g., family medicine, pediatrics, internal medicine). Data sources included semi-structured interviews with four PEC members per program and diverse program artifacts. Using a constructivist approach, we utilized qualitative coding to analyze participant interviews and content analysis for program artifacts. We then used coded transcripts and artifacts to construct logic models for each program guided by a systems theory lens.  Results: Programs adapt their PEC structure, execution, and outcomes to meet short- and long-term needs based on organizational and program-unique factors such as size and local practices. They relied on multiple data sources and sought diverse stakeholder participation to complete program evaluation and improvement. Identified deficiencies were often categorized as internal versus external to delineate PEC responsibility, boundaries, and feasibility of interventions. CONCLUSION: The broad guidance provided by the ACGME for PEC configuration allows programs to adapt the committee based on individual needs. However, further instruction on program evaluation and organizational change principles would augment existing PEC efforts.

Occupying liminal spaces: The figured worlds of student affairs senior leaders in the United States.

INTRODUCTION: Student Affairs Senior Leaders (SASLs) in the United States lead offices responsible for academic advising, administrative documentation, scheduling, student health, financial aid, and transition to residency, yet they infrequently draw attention in the field's literature. We explore the role of SASLs and how they describe the social space of medical education. METHODS: Using a constructivist approach informed by Figured Worlds theory, we conducted a sequential narrative and thematic analysis of the stories SASLs tell about their roles and experiences in the world of medical education. RESULTS: SASLs inhabit complex roles centred on advocating for medical students' academic, personal and social well-being. Their unique position within the medical school allows them to see the harm to vulnerable students made possible by misalignments inherent within medical education. Yet even with the challenges inherent in the environment, SASLs find reasons for hope. CONCLUSION: SASLs' identities are full of potential contradictions, but they have a unique view into the often-chaotic world of medical education.

The case for metacognitive reflection: a theory integrative review with implications for medical education.

The concepts of metacognitive reflection, reflection, and metacognition are distinct but have undergone shifts in meaning as they migrated into medical education. Conceptual clarity is essential to the construction of the knowledge base of medical education and its educational interventions. We conducted a theoretical integrative review across diverse bodies of literature with the goal of understanding what metacognitive reflection is. We searched PubMed, Embase, CINAHL, PsychInfo, and Web of Science databases, including all peer-reviewed research articles and theoretical papers as well as book chapters that addressed the topic, with no limitations for date, language, or location. A total of 733 articles were identified and 87 were chosen after careful review and application of exclusion criteria. The work of conceptually and empirically delineating metacognitive reflection has begun. Contributions have been made to root metacognitive reflection in the concept of metacognition and moving beyond it to engage in cycles of reflection. Other work has underscored its affective component, transformational nature, and contextual factors. Despite this merging of threads to develop a richer conceptualization, a theory of how metacognitive reflection works is elusive. Debates address whether metacognition drives reflection or vice versa. It has also been suggested that learners evolve along on a continuum from thinking, to task-related reflection, to self-reflection, and finally to metacognitive reflection. Based on prior theory and research, as well as the findings of this review, we propose the following conceptualization: Metacognitive reflection involves heightened internal observation, awareness, monitoring, and regulation of our own knowledge, experiences, and emotions by questioning and examining cognition and emotional processes to continually refine and enhance our perspectives and decisions while thoughtfully accounting for context. We argue that metacognitive reflection brings a shift in perspective and can support valuable reconceptualization for lifelong learning.

Physiologic measurements of cognitive load in clinical reasoning.

OBJECTIVES: Cognitive load is postulated to be a significant factor in clinical reasoning performance. Monitoring physiologic measures, such as heart rate variability (HRV) may serve as a way to monitor changes in cognitive load. The pathophysiology of why HRV has a relationship to cognitive load is unclear, but it may be related to blood pressure changes that occur in a response to mental stress. METHODS: Fourteen residents and ten attendings from Internal Medicine wore Holter monitors and watched a video depicting a medical encounter before completing a post encounter form used to evaluate their clinical reasoning and standard psychometric measures of cognitive load. Blood pressure was obtained before and after the encounter. Correlation analysis was used to investigate the relationship between HRV, blood pressure, self-reported cognitive load measures, clinical reasoning performance scores, and experience level. RESULTS: Strong positive correlations were found between increasing HRV and increasing mean arterial pressure (MAP) (p=0.01, Cohen's d=1.41). There was a strong positive correlation with increasing MAP and increasing cognitive load (Pearson correlation 0.763; 95 % CI [; 95 % CI [-0.364, 0.983]). Clinical reasoning performance was negatively correlated with increasing MAP (Pearson correlation -0.446; 95 % CI [-0.720, -0.052]). Subjects with increased HRV, MAP and cognitive load were more likely to be a resident (Pearson correlation -0.845; 95 % CI [-0.990, 0.147]). CONCLUSIONS: Evaluating HRV and MAP can help us to understand cognitive load and its implications on trainee and physician clinical reasoning performance, with the intent to utilize this information to improve patient care.

The SOAP Feedback Training Program.

BACKGROUND: Accessible and efficient opportunities for health professional faculty to hone feedback skills are limited. In addition, feedback models to apply to the objective structured clinical examination (OSCE) setting are lacking. APPROACH: Annually, paediatric interns from Children's National Hospital and Walter Reed National Military Medical Center participate in an OSCE, which includes faculty observation and immediate feedback to trainees. In 2018, we incorporated the subjective, objective, assessment, plan (SOAP) Feedback Training Program during 20 min of the pre-OSCE faculty orientation. The SOAP Feedback Training Program introduced the SOAP feedback model (subjective, objective, assessment, plan), facilitated practice in pairs and distributed a cognitive aid referencing the model. We evaluated the quality of faculty feedback exchanges during the 2018 OSCE via retrospective video review using the Direct Observation of Clinical Skills Feedback Scale (DOCS-FBS). We compared the results to the 2015 initial evaluation and used focus groups to understand how and why faculty feedback changed. EVALUATION: Comparison of the initial evaluation to the post-SOAP Feedback Training Program intervention data using a Wilcoxon signed rank test showed statistically significant improvement in six of eight feedback items on the DOCS-FBS. Causal coding of focus group transcripts revealed that the SOAP Feedback Training Program evoked affective responses, reinforced prior practice in feedback delivery, improved feedback organisation and increased feedback delivery preparation. IMPLICATIONS: The SOAP Feedback Training Program is an effective intervention to teach the SOAP feedback model and improve faculty feedback quality in an OSCE setting. It is efficient and low resource, facilitating its potential use in settings beyond the OSCE.

Cognitive biases in internal medicine: a scoping review.

BACKGROUND: Medical errors account for up to 440,000 deaths annually, and cognitive errors outpace knowledge deficits as causes of error. Cognitive biases are predispositions to respond in predictable ways, and they don't always result in error. We conducted a scoping review exploring which biases are most prevalent in Internal Medicine (IM), if and how they influence patient outcomes, and what, if any, debiasing strategies are effective. CONTENT: We searched PubMed, OVID, ERIC, SCOPUS, PsychINFO, and CINAHL. Search terms included variations of "bias", "clinical reasoning", and IM subspecialties. Inclusion criteria were: discussing bias, clinical reasoning, and physician participants. SUMMARY: Fifteen of 334 identified papers were included. Two papers looked beyond general IM: one each in Infectious Diseases and Critical Care. Nine papers distinguished bias from error, whereas four referenced error in their definition of bias. The most commonly studied outcomes were diagnosis, treatment, and physician impact in 47 % (7), 33 % (5), and 27 % (4) of studies, respectively. Three studies directly assessed patient outcomes. The most commonly cited biases were availability bias (60 %, 9), confirmation bias (40 %, 6), anchoring (40 %, 6), and premature closure (33 %, 5). Proposed contributing features were years of practice, stressors, and practice setting. One study found that years of practice negatively correlated with susceptibility to bias. Ten studies discussed debiasing; all reported weak or equivocal efficacy. OUTLOOK: We found 41 biases in IM and 22 features that may predispose physicians to bias. We found little evidence directly linking biases to error, which could account for the weak evidence of bias countermeasure efficacy. Future study clearly delineating bias from error and directly assessing clinical outcomes would be insightful.

Clinical Reasoning Curricula in Preclinical Undergraduate Medical Education: A Scoping Review.

PURPOSE: Clinical reasoning is the process of observing, collecting, analyzing, and interpreting patient information to arrive at a diagnosis and management plan. Although clinical reasoning is foundational in undergraduate medical education (UME), the current literature lacks a clear picture of the clinical reasoning curriculum in preclinical phase of UME. This scoping review explores the mechanisms of clinical reasoning education in preclinical UME. METHOD: A scoping review was performed in accordance with the Arksey and O'Malley framework methodology for scoping reviews and is reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis for Scoping Reviews. RESULTS: The initial database search identified 3,062 articles. Of these, 241 articles were selected for a full-text review. Twenty-one articles, each reporting a single clinical reasoning curriculum, were selected for inclusion. Six of the reports included a definition of clinical reasoning, and 7 explicitly reported the theory underlying the curriculum. Reports varied in the identification of clinical reasoning content domains and teaching strategies. Only 4 curricula reported assessment validity evidence. CONCLUSIONS: Based on this scoping review, we recommend 5 key principles for educators to consider when reporting clinical reasoning curricula in preclinical UME: (1) explicitly define clinical reasoning within the report, (2) report clinical reasoning theory(ies) used in the development of the curriculum, (3) clearly identify which clinical reasoning domains are addressed in the curriculum, (4) report validity evidence for assessments when available, and (5) describe how the reported curriculum fits into the larger clinical reasoning education at the institution.

State-of-the-art literature review methodology: A six-step approach for knowledge synthesis.

INTRODUCTION: Researchers and practitioners rely on literature reviews to synthesize large bodies of knowledge. Many types of literature reviews have been developed, each targeting a specific purpose. However, these syntheses are hampered if the review type's paradigmatic roots, methods, and markers of rigor are only vaguely understood. One literature review type whose methodology has yet to be elucidated is the state-of-the-art (SotA) review. If medical educators are to harness SotA reviews to generate knowledge syntheses, we must understand and articulate the paradigmatic roots of, and methods for, conducting SotA reviews. METHODS: We reviewed 940 articles published between 2014-2021 labeled as SotA reviews. We (a) identified all SotA methods-related resources, (b) examined the foundational principles and techniques underpinning the reviews, and (c) combined our findings to inductively analyze and articulate the philosophical foundations, process steps, and markers of rigor. RESULTS: In the 940 articles reviewed, nearly all manuscripts (98%) lacked citations for how to conduct a SotA review. The term "state of the art" was used in 4 different ways. Analysis revealed that SotA articles are grounded in relativism and subjectivism. DISCUSSION: This article provides a 6-step approach for conducting SotA reviews. SotA reviews offer an interpretive synthesis that describes: This is where we are now. This is how we got here. This is where we could be going. This chronologically rooted narrative synthesis provides a methodology for reviewing large bodies of literature to explore why and how our current knowledge has developed and to offer new research directions.

Situativity: a family of social cognitive theories for understanding clinical reasoning and diagnostic error.

The diagnostic error crisis suggests a shift in how we view clinical reasoning and may be vital for transforming how we view clinical encounters. Building upon the literature, we propose clinical reasoning and error are context-specific and proceed to advance a family of theories that represent a model outlining the complex interplay of physician, patient, and environmental factors driving clinical reasoning and error. These contemporary social cognitive theories (i.e. embedded cognition, ecological psychology, situated cognition, and distributed cognition) can emphasize the dynamic interactions occurring amongst participants in particular settings. The situational determinants that contribute to diagnostic error are also explored.

Examining the patterns of uncertainty across clinical reasoning tasks: effects of contextual factors on the clinical reasoning process.

Objectives Uncertainty is common in clinical reasoning given the dynamic processes required to come to a diagnosis. Though some uncertainty is expected during clinical encounters, it can have detrimental effects on clinical reasoning. Likewise, evidence has established the potentially detrimental effects of the presence of distracting contextual factors (i.e., factors other than case content needed to establish a diagnosis) in a clinical encounter on clinical reasoning. The purpose of this study was to examine how linguistic markers of uncertainty overlap with different clinical reasoning tasks and how distracting contextual factors might affect physicians' clinical reasoning process. Methods In this descriptive exploratory study, physicians participated in a live or video recorded simulated clinical encounter depicting a patient with unstable angina with and without contextual factors. Transcribed think-aloud reflections were coded using Goldszmidt's clinical reasoning task typology (26 tasks encompassing the domains of framing, diagnosis, management, and reflection) and then those coded categories were examined using linguistic markers of uncertainty (e.g., probably, possibly, etc.). Results Thirty physicians with varying levels of experience participated. Consistent with expectations, descriptive analysis revealed that physicians expressed more uncertainty in cases with distracting contextual factors compared to those without. Across the four domains of reasoning tasks, physicians expressed the most uncertainty in diagnosis and least in reflection. Conclusions These results highlight how linguistic markers of uncertainty can shed light on the role contextual factors might play in uncertainty which can lead to error and why it is essential to find ways of managing it.

Sequence matters: patterns in task-based clinical reasoning.

Background The cognitive pathways that lead to an accurate diagnosis and efficient management plan can touch on various clinical reasoning tasks (1). These tasks can be employed at any point during the clinical reasoning process and though the four distinct categories of framing, diagnosis, management, and reflection provide some insight into how these tasks map onto clinical reasoning, much is still unknown about the task-based clinical reasoning process. For example, when and how are these tasks typically used? And more importantly, do these clinical reasoning task processes evolve when patient encounters become complex and/or challenging (i.e. with contextual factors)? Methods We examine these questions through the lens of situated cognition, context specificity, and cognitive load theory. Sixty think-aloud transcripts from 30 physicians who participated in two separate cases - one with a contextual factor and one without - were coded for 26 clinical reasoning tasks (1). These tasks were organized temporally, i.e. when they emerged in their think-aloud process. Frequencies of each of the 26 tasks were aggregated, categorized, and visualized in order to analyze task category sequences. Results We found that (a) as expected, clinical tasks follow a general sequence, (b) contextual factors can distort this emerging sequence, and (c) the presence of contextual factors prompts more experienced physicians to clinically reason similar to that of less experienced physicians. Conclusions These findings add to the existing literature on context specificity in clinical reasoning and can be used to strengthen teaching and assessment of clinical reasoning.