Physician Leadership and Burnout: The Need for Agency; a Qualitative Study of an Academic Institution.

Purpose: Organizational approaches to physician burnout are limited. Training physician leaders to influence the organizational environment is a growing area of study. This study explored perceived physician leadership behaviors in response to burnout from the viewpoint of faculty physicians not in formal leadership positions. Understanding physician leadership behaviors from the viewpoint of those faculty being led can inform organizational strategy and leadership training to address physician burnout. Subjects and Methods: Interview requests were sent to 70 randomly identified faculty physicians from a roster containing all 1145 physician faculty that excluded the Pediatric Department, at an academic health care institution in Southern California. The first ten respondents were asked to participate in a 30-to-40-minute semi-structured virtual interview via Zoom. The interviewees were asked two questions pertaining to burnout and their perception of how leadership responded. The two questions were "What has leadership done to address burnout?" and "If you had five minutes to advise your leaders on burnout, what would you say?" The recorded interviews were transcribed, redacted, and then sent to two reviewers. Thematic analysis through iterative coding was completed, and categories were constructed that aligned with the two interview questions. Results: Overall, five themes were identified. These themes were organized according to the interview questions and broadly categorized as physician leadership behaviors observed that corresponded to the interview question of what leadership had done to address burnout and physician leadership behaviors desired corresponding to the second interview question of what advice should be given. Leadership behaviors observed in the context of burnout included three themes; referral to individual wellness programs, increased number of meetings and events, and a lack of agency in addressing wellness issues. The two themes of leadership behaviors desired were the obtainment of more resources and the granting of greater appreciation and recognition for work done through enhanced communication. Conclusion: This small study of faculty physician perceptions of leadership behaviors identified several themes that had been identified in previous studies of leadership and burnout; need for relationship building through communication, need for resources to address work issues, and referral to wellness programs. However, the identification of a lack of agency in addressing factors in the wellness environment has not been identified in the previous burnout and physician leadership literature. Further study into the causes of this perceived lack of agency should be explored. Understanding the root causes of physician leaders' lack of agency can further inform physician leadership education as an organizational approach to burnout.

Physician Burnout-Evidence That Leadership Behaviors Make a Difference: A Cross-Sectional Survey of an Academic Medical Center.

INTRODUCTION: High rates of physician burnout are well documented in the USA. Identifying beneficial leadership behaviors as an organizational approach to mitigating burnout can lead to improved wellness in the physicians that they lead; however, few studies have examined which leadership behaviors are beneficial and which may be detrimental. MATERIALS AND METHODS: This survey study of academic medical center physicians and their physician leaders assessed the correlation between burnout and leadership behaviors. Data were analyzed for the strength of correlation between scores for leadership behaviors and self-reported physician burnout with analysis of variance by sex, time from training, specialty, and age. RESULTS: Of 1,145 physicians surveyed, 305 returned surveys. Among the respondents, 45% were female, 25% were 56 years or older, and 57% self-identified as practitioners of medicine or medicine subspecialties. Two transformational leadership categories of behaviors (idealized influence behaviors and individualized consideration) and one transactional leadership behavior category (contingent reward) correlated favorably with all domains of burnout (P < .0001). Conversely, two transactional leadership categories of burnout (management by exception passive and laissez-faire) correlated unfavorably with all burnout domains. CONCLUSIONS: Organizational interventions are needed to improve burnout in physicians. Adopting favorable leadership behaviors while avoiding unfavorable leadership behaviors can improve burnout in those physicians being led. These findings could inform the conceptual basis of future physician leadership training programs as transactional leadership behaviors also have an impact on physician wellness.

Expectations of and for Clerkship Directors 2.0: A Collaborative Statement from the Alliance for Clinical Education.

This article presents an update of the collaborative statement on clerkship directors (CDs), first published in 2003, from the national undergraduate medical education organizations that comprise the Alliance for Clinical Education (ACE). The clerkship director remains an essential leader in the education of medical students on core clinical rotations, and the role of the CD has and continues to evolve. The selection of a CD should be an explicit contract between the CD, their department, and the medical school, with each party fulfilling their obligations to ensure the success of the students, the clerkship and of the CD. Educational innovations and accreditation requirements have evolved in the last two decades and therefore this article updates the 2003 standards for what is expected of a CD and provides guidelines for the resources and support to be provided.In their roles as CDs, medical student educators engage in several critical activities: administration, education/teaching, coaching, advising, and mentoring, faculty development, compliance with accreditation standards, and scholarly activity. This article describes (a) the work products that are the primary responsibility of the CD; (b) the qualifications for the CD; (c) the support structure, resources, and personnel that are necessary for the CD to accomplish their responsibilities; (d) incentives and career development for the CD; and (e) the dedicated time that should be provided for the clerkship and the CD to succeed. Given all that should rightfully be expected of a CD, a minimum of 50% of a full-time equivalent is recognized as appropriate. The complexity and needs of the clerkship now require that at least one full-time clerkship administrator (CA) be a part of the CD's team.To better reflect the current circumstances, ACE has updated its recommendations for institutions and departments to have clear standards for what is expected of the director of a clinical clerkship and have correspondingly clear guidelines as to what should be expected for CDs in the support they are provided. This work has been endorsed by each of the eight ACE member organizations.

Gathering Validity Evidence on an Internal Medicine Clerkship Multistep Exam to Assess Medical Student Analytic Ability.

Construct: The definition of clinical reasoning may vary among health profession educators. However, for the purpose of this paper, clinical reasoning is defined as the cognitive processes that are involved in the steps of information gathering, problem representation, generating a differential diagnosis, providing a diagnostic justification to arrive at a leading diagnosis, and formulating diagnostic and management plans. Background: Expert performance in clinical reasoning is essential for success as a physician, and has been difficult for clerkship directors to observe and quantify in a way that fosters the instruction and assessment of clinical reasoning. The purpose of this study was to gather validity evidence for the Multistep exam (MSX) format used by our medicine clerkship to assess analytical clinical reasoning abilities; we did this by examining the relationship between scores on the MSX and other external measures of clinical reasoning abilities. This analysis used dual process theory as the main theoretical framework of clinical reasoning, as well as aspects of Kane's validity framework to guide the selection of validity evidence for the investigation. We hypothesized that there would be an association between the MSX (a three-step clinical reasoning tool developed locally), and the USMLE Step 2 CS, as they share similar concepts in assessing the clinical reasoning of students. We examined the relationship between overall scores on the MSX and the Step 2 CS Integrated Clinical Encounter (ICE) score, in which the student articulates their reasoning for simulated patient cases, while controlling for examinee's internal medicine clerkship performance measures such as the NBME subject exam score and the Medicine clerkship OSCE score. Approach: A total 477 of 487 (97.9%) medical students, representing the graduating classes of 2015, 2016, 2017, who took the MSX at the end of each medicine clerkship (2012-2016), and Step 2 CS (2013-2017) were included in this study. Correlation analysis and multiple linear regression analysis were used to examine the impact of the primary explanatory variables of interest (MSX) onto the outcome variable (ICE score) when controlling for baseline variables (Medicine OSCE and NBME Medicine subject exam). Findings: The overall MSX score had a significant, positive correlation with the Step 2 CS ICE score (r = .26, P < .01). The overall MSX score was a significant predictor of Step 2 CS ICE score (ß = .19, P < .001), explaining an additional 4% of the variance of ICE beyond the NBME Medicine subject score and the Medicine OSCE score (Adjusted R2 = 13%). Conclusion: The stepwise format of the MSX provides a tool to observe clinical reasoning performance, which can be used in an assessment system to provide feedback to students on their analytical clinical reasoning. Future studies should focus on gaining additional validity evidence across different learners and multiple medical schools.

Post-Carnegie II curricular reform: a north American survey of emerging trends & challenges.

BACKGROUND: In 2010, coincident with the 100th anniversary of Flexner's sentinel report, the Carnegie Foundation published an updated review of North American medical education and challenged medical schools to initiate further educational reforms. Specific recommendations pertained to a) ensuring standardized outcomes while allowing for individualized processes, b) integrating foundational knowledge with clinical experience, c) cultivating habits of inquiry and innovation and d) professional identity formation. As we approach the 10-year anniversary of this latest report, we sought to determine what type of curricular revisions have been emerging within the past decade and what types of challenges have been encountered along the way? METHODS: In 2018, an electronic survey was sent to all 166 Liaison Committee on Medical Education (LCME) accredited North American Medical Schools, using the points of contact (educational deans) that were listed in a publicly available, Association of American Medical Colleges database. Free text comments were grouped into themes using the constant-comparative technique. RESULTS: Sixty unique responses yielding a 36.14% response rate. The distribution of responses was proportionally representative of the distribution of public vs. private, old vs. new vs. established North American medical schools. Self-reported curricular changes aggregated into five main themes: Changes in curricular structure/organization, changes in curricular content, changes in curricular delivery, changes in assessment, and changes involving increased use of technology/informatics. Challenges were predominantly focused on overcoming faculty resistance, faculty development, securing adequate resourcing, change management, and competition for limited amounts of curricular time. CONCLUSIONS: Changes in curricular organization, content, delivery, assessment and the use of technology reflect reforms that are broad and deep. Empowering faculty to "let go" of familiar constructs/processes requires strong leadership, particularly when initiating particularly disruptive curricular changes, such as relocating the Step 1 examination or shifting to a competency-based curriculum. While North American medical schools are responding to the calls for action described in the second (2010) Carnegie Foundation report, the full vision has yet to be achieved.

Specialty Choices, Practice Characteristics, and Long-term Outcomes of Two Cohorts of USUHS Medical School Graduates Compared with National Data.

Introduction: The education of health professionals and the assessment of their long-term goals are essential. Additionally, in the military health system (MHS), such programs also have the goals of improving readiness, providing high-quality care and meeting the needs and challenges of the MHS. The Uniformed Services University F. Edward Hebert School of Medicine (USUHS) has existed for over 40 years and the assessment of the longer term outcomes of its medical school graduates is an important endeavor. The purpose of this study is to describe the relationship of USUHS on the care in the MHS by reporting specialty choices, practice characteristics and locations of two consecutive cohorts of USUHS graduates and to compare these cohort findings with national data. Two cohorts were chosen as the first cohort has reached retirement and we sought to describe the impact of our graduates following their military service commitment. Materials and Methods: We performed a retrospective analysis of our graduates (1980-2009) using data obtained from the American Medical Association Physician Masterfile in two 15-year cohorts. USU graduate data are described and compared with national allopathic medical school data in order to explore USUHS graduates' practice characteristics and how these contrast with national data. Results: Family medicine was the most commonly identified primary care specialty across both USUHS cohorts. Other primary care specialties such as Internal Medicine and Pediatrics became more popular among USUHS graduates over time. There were a lower percentage of inactive USUHS graduates from the 1980-1994 cohorts than the U.S. national sample (3.23% vs. 3.98%). The proportion of USUHS graduates working in an office-based practice increased from 34.8% (1980-1994) to 43.6% (1995-2009) yet still was lower than U.S. national data (1980-1994: 73.5%; 1995-2009: 73.8%). The type of practice in which the majority of USUHS and national graduates were involved was direct patient care (over 70% of the population). Practice locations for medical school graduates included all 50 states. Furthermore, though several states such as Mississippi, Nevada, South Carolina, and Utah were among the top 20 practice locations among USUHS graduates, they did not appear among the top 20 practicing states for U.S. graduates. Conclusions: USUHS can play a major role by training health care professionals needed to meet the large health care strategic goals of a transforming military health system. A high percentage of USUHS graduates had significant changes in practice characteristics over time, are staying in practice for longer time compared with U.S. graduates, and practice in all 50 states including those that are underserved.

Staying Power: Does the Uniformed Services University Continue to Meet Its Obligation to the Nation's Health Care Needs?

INTRODUCTION: The Uniformed Services University F. Edward Hebert School of Medicine just passed its 45th anniversary, opening in 1972. A goal of the medical school, like those nationally, is the production of high-quality physicians. The purpose of this study is to describe the practice characteristics of our USU graduates and to compare data with the national cohort of U.S. MD graduates. MATERIALS AND METHODS: To accomplish this, we performed a retrospective analysis of U.S. graduates (1980-2009). We used the American Medical Association Physician Masterfile to describe our graduates' current practice profile and compare them with the national cohort. In order to ascertain if USU is meeting our goal to provide high-quality physicians, we also compare our findings with national allopathic school data to norm-reference our results. RESULTS: Our findings indicate that USU graduates contribute to both primary care and specialty care and they practice in all 50 states. USU graduates continue to serve their nation after their obligation is complete, with 64% continuing to practice in federal hospitals and agencies. USU graduates also have a higher board certification rate (90%) than the national cohort (88%). CONCLUSION: Following our 45th anniversary, we provide continuing evidence that USU is keeping its contract with society. We provide evidence that USU continues to meet its obligation to the nation's health care needs by producing high-quality physicians who serve the country in multiple ways after their military obligation is complete, thus extending the definition of staying power. Our study is not without limitations. First, we could not precisely define the cohort to exclude graduates who still had service obligations. Second, the AMA Physician Masterfile had some missing data fields, so nonresponse or misclassification bias is possible in our results. Study strengths include the long period of time and large number of graduates in each cohort.

Regular Formal Evaluation Sessions are Effective as Frame-of-Reference Training for Faculty Evaluators of Clerkship Medical Students.

BACKGROUND: Face-to-face formal evaluation sessions between clerkship directors and faculty can facilitate the collection of trainee performance data and provide frame-of-reference training for faculty. OBJECTIVE: We hypothesized that ambulatory faculty who attended evaluation sessions at least once in an academic year (attendees) would use the Reporter-Interpreter-Manager/Educator (RIME) terminology more appropriately than faculty who did not attend evaluation sessions (non-attendees). DESIGN: Investigators conducted a retrospective cohort study using the narrative assessments of ambulatory internal medicine clerkship students during the 2008-2009 academic year. PARTICIPANTS: The study included assessments of 49 clerkship medical students, which comprised 293 individual teacher narratives. MAIN MEASURES: Single-teacher written and transcribed verbal comments about student performance were masked and reviewed by a panel of experts who, by consensus, (1) determined whether RIME was used, (2) counted the number of RIME utterances, and (3) assigned a grade based on the comments. Analysis included descriptive statistics and Pearson correlation coefficients. KEY RESULTS: The authors reviewed 293 individual teacher narratives regarding the performance of 49 students. Attendees explicitly used RIME more frequently than non-attendees (69.8 vs. 40.4 %; p < 0.0001). Grades recommended by attendees correlated more strongly with grades assigned by experts than grades recommended by non-attendees (r = 0.72; 95 % CI (0.65, 0.78) vs. 0.47; 95 % CI (0.26, 0.64); p = 0.005). Grade recommendations from individual attendees and non-attendees each correlated significantly with overall student clerkship clinical performance [r = 0.63; 95 % CI (0.54, 0.71) vs. 0.52 (0.36, 0.66), respectively], although the difference between the groups was not statistically significant (p = 0.21). CONCLUSIONS: On an ambulatory clerkship, teachers who attended evaluation sessions used RIME terminology more frequently and provided more accurate grade recommendations than teachers who did not attend. Formal evaluation sessions may provide frame-of-reference training for the RIME framework, a method that improves the validity and reliability of workplace assessment.

Are commonly used premedical school or medical school measures associated with board certification?

PURPOSE: To determine if there is an association between several commonly obtained premedical school and medical school measures and board certification performance. We specifically included measures from our institution for which we have predictive validity evidence into the internship year. We hypothesized that board certification would be most likely to be associated with clinical measures of performance during medical school, and with scores on standardized tests, whether before or during medical school. METHODS: Achieving board certification in an American Board of Medical Specialties specialty was used as our outcome measure for a 7-year cohort of graduates (1995-2002). Age at matriculation, Medical College Admissions Test (MCAT) score, undergraduate college grade point average (GPA), undergraduate college science GPA, Uniformed Services University (USU) cumulative GPA, USU preclerkship GPA, USU clerkship year GPA, departmental competency committee evaluation, Internal Medicine (IM) clerkship clinical performance rating (points), IM total clerkship points, history of Student Promotion Committee review, and United States Medical Licensing Examination (USMLE) Step 1 score and USMLE Step 2 clinical knowledge score were associated with this outcome. RESULTS: Ninety-three of 1,155 graduates were not certified, resulting in an average rate of board certification of 91.9% for the study cohort. Significant small correlations were found between board certification and IM clerkship points (r = 0.117), IM clerkship grade (r = 0.108), clerkship year GPA (r = 0.078), undergraduate college science GPA (r = 0.072), preclerkship GPA and medical school GPA (r = 0.068 for both), USMLE Step 1 (r = 0.066), undergraduate college total GPA (r = 0.062), and age at matriculation (r = -0.061). In comparing the two groups (board certified and not board certified cohorts), significant differences were seen for all included variables with the exception of MCAT and USMLE Step 2 clinical knowledge scores. All the variables put together could explain 4.1% of the variance of board certification by logistic regression. CONCLUSIONS: This investigation provides some additional validity evidence that measures collected for purposes of student evaluation before and during medical school are warranted.

The clinical integrative puzzle for teaching and assessing clinical reasoning: preliminary feasibility, reliability, and validity evidence.

BACKGROUND: Expertise in clinical reasoning is essential for high-quality patient care. The Clinical Integrative Puzzle (CIP) is a novel assessment method for clinical reasoning. The purpose of our study was to further describe the CIP, providing feasibility, reliability, and validity evidence to support this tool for teaching and evaluating clinical reasoning. METHODS: We conducted a prospective, randomized crossover trial assessing the CIP in second-year medical students from a single institution. Feasibility was estimated through the time taken to complete a CIP during a CIP session and through comments from faculty developers. Reliability was addressed through calculating odd-even item reliability (split-half procedure) for grid questions within each CIP. Evidence for content, concurrent, and predictive validity was also measured. RESULTS: 36 students participated in the study. Data suggested successful randomization of participants and nonparticipants. The CIP was found to have high feasibility, acceptable reliability (0.43-0.73 with a mean of 0.60) with a short time for CIP completion. Spearman-Brown correction estimated a reliability of 0.75 with completing two grids (estimated time of 50 minutes) and 0.82 for three grids (estimated time of 75 minutes). Validity evidence was modest; the CIP is consistent with clinical reasoning literature and the CIP modestly correlated with small group performance (r = 0.3, p < 0.05). CONCLUSIONS: Assessing clinical reasoning in medical students is challenging. Our data provide good feasibility and reliability evidence for the use of CIPs; validity data was less robust.

The association of students requiring remediation in the internal medicine clerkship with poor performance during internship.

PURPOSE: To determine whether the Uniformed Services University (USU) system of workplace performance assessment for students in the internal medicine clerkship at the USU continues to be a sensitive predictor of subsequent poor performance during internship, when compared with assessments in other USU third year clerkships. METHOD: Utilizing Program Director survey results from 2007 through 2011 and U.S. Medical Licensing Examination (USMLE) Step 3 examination results as the outcomes of interest, we compared performance during internship for students who had less than passing performance in the internal medicine clerkship and required remediation, against students whose performance in the internal medicine clerkship was successful. We further analyzed internship ratings for students who received less than passing grades during the same time period on other third year clerkships such as general surgery, pediatrics, obstetrics and gynecology, family medicine, and psychiatry to evaluate whether poor performance on other individual clerkships were associated with future poor performance at the internship level. Results for this recent cohort of graduates were compared with previously published findings. RESULTS: The overall survey response rate for this 5 year cohort was 81% (689/853). Students who received a less than passing grade in the internal medicine clerkship and required further remediation were 4.5 times more likely to be given poor ratings in the domain of medical expertise and 18.7 times more likely to demonstrate poor professionalism during internship. Further, students requiring internal medicine remediation were 8.5 times more likely to fail USMLE Step 3. No other individual clerkship showed any statistically significant associations with performance at the intern level. On the other hand, 40% of students who successfully remediated and did graduate were not identified during internship as having poor performance. CONCLUSIONS: Unsuccessful clinical performance which requires remediation in the third year internal medicine clerkship at Uniformed Services University of the Health Sciences continues to be strongly associated with poor performance at the internship level. No significant associations existed between any of the other clerkships and poor performance during internship and Step 3 failure. The strength of this association with the internal medicine clerkship is most likely because of an increased level of sensitivity in detecting poor performance.

Referral for competency committee review for poor performance on the internal medicine clerkship is associated with poor performance in internship.

PURPOSE: To determine how students who are referred to a competency committee for concern over performance, and ultimately judged not to require remediation, perform during internship. METHODS: Uniformed Services University of the Health Sciences' students who graduated between 2007 and 2011 were included in this study. We compared the performance during internship of three groups: students who were referred to the internal medicine competency committee for review who met passing criterion, students who were reviewed by the internal medicine competency committee who were determined not to have passed the clerkship and were prescribed remediation, and students who were never reviewed by this competency committee. Program Director survey results and United States Medical Licensing Examination (USMLE) Step 3 examination results were used as the outcomes of interest. RESULTS: The overall survey response rate for this 5-year cohort was 81% (689/853). 102 students were referred to this competency committee for review. 63/102 students were reviewed by this competency committee, given passing grades in the internal medicine clerkship, and were not required to do additional remediation. 39/102 students were given less than passing grades by this competency committee and required to perform additional clinical work in the department of medicine to remediate their performance. 751 students were never presented to this competency committee. Compared to students who were never presented for review, the group of reviewed students who did not require remediation was 5.6 times more likely to receive low internship survey ratings in the realm of professionalism, 8.6 times more likely to receive low ratings in the domain of medical expertise, and had a higher rate of USMLE Step 3 failure (9.4% vs. 2.8%). When comparing the reviewed group to students who were reviewed and also required remediation, the only significant difference between groups regarding professionalism ratings with 50% of the group requiring remediation garnering low ratings compared to 18% of the reviewed group. CONCLUSIONS: Students who are referred to a committee for review following completion of their internal medicine clerkship are more likely to receive poor ratings in internship and fail USMLE Step 3 compared to students whose performance in the medicine clerkship does not trigger a committee review. These findings provide validity evidence for our competency committee review in that the students identified as requiring further clinical work had significantly higher rates of poor ratings in professionalism than students who were reviewed by the competency committee but not required to remediate. Additionally, students reviewed but not required to remediate were nonetheless at risk of low internship ratings, suggesting that these students might need some intervention prior to graduation.

Novel examination for evaluating medical student clinical reasoning: reliability and association with patients seen.

BACKGROUND: Medical students learn clinical reasoning, in part, through patient care. Although the numbers of patients seen is associated with knowledge examination scores, studies have not demonstrated an association between patient problems and an assessment of clinical reasoning. AIM: To examine the reliability of a clinical reasoning examination and investigate whether there was association between internal medicine core clerkship students' performance on this examination and the number of patients they saw with matching problems during their internal medicine clerkship. METHODS: Students on the core internal medicine clerkship at the Uniformed Services University students log 11 core patient problems based on the Clerkship Directors in Internal Medicine curriculum. On a final clerkship examination (Multistep), students watch a scripted video encounter between physician and patient actors that assesses three sequential steps in clinical reasoning: Step One focuses on history and physical examination; Step Two, students write a problem list after viewing additional clinical findings; Step Three, students complete a prioritized differential diagnosis and treatment plan. Each Multistep examination has three different cases. For graduating classes 2010-2012 (n = 497), we matched the number of patients seen with the problem most represented by the Multistep cases (epigastric pain, generalized edema, monoarticular arthritis, angina, syncope, pleuritic chest pain). We report two-way Pearson correlations between the number of patients students reported with similar problems and the student's percent score on: Step One, Step Two, Step Three, and Overall Test. RESULTS: Multistep reliability: Step 1, 0.6 to 0.8; Step 2, 0.41 to 0.65; Step 3, 0.53 to 0.78; Overall examination (3 cases): 0.74 to 0.83. For three problems, the number of patients seen had small to modest correlations with the Multistep Examination of Analytic Ability total score (r = 0.27 for pleuritic pain, p < 0.05, n = 81 patients; r = 0.14 for epigastric pain, p < 0.05, n = 324 patients; r = 0.19 for generalized edema, p < 0.05, n = 118 patients). DISCUSSION or CONCLUSION: Although a reliable assessment, student performance on a clinical reasoning examination was weakly associated with the numbers of patients seen with similar problems. This may be as a result of transfer of knowledge between clinical and examination settings, the complexity of clinical reasoning, or the limits of reliability with patient logs and the Multistep.