Excess Mortality Among US Physicians During the COVID-19 Pandemic.

This cross-sectional study examines the death rates among active and nonactive physicians aged 45 to 84 years.

Associations between initial American Board of Internal Medicine certification and maintenance of certification status of attending physicians and in-hospital mortality of patients with acute myocardial infarction or congestive heart failure: a retrospective cohort study of hospitalisations in Pennsylvania, USA.

OBJECTIVE: To determine whether internists' initial specialty certification and the maintenance of that certification (MOC) is associated with lower in-hospital mortality for their patients with acute myocardial infarction (AMI) or congestive heart failure (CHF). DESIGN: Retrospective cohort study of hospitalisations in Pennsylvania, USA, from 2012 to 2017. SETTING: All hospitals in Pennsylvania. PARTICIPANTS: All 184 115 hospitalisations for primary diagnoses of AMI or CHF where the attending physician was a self-designated internist. PRIMARY OUTCOME MEASURE: In-hospital mortality. RESULTS: Of the 2575 physicians, 2238 had initial certification and 820 were eligible for MOC. After controlling for patient demographics and clinical characteristics, hospital-level factors and physicians' demographic and medical school characteristics, both initial certification and MOC were associated with lower mortality. The adjusted OR for initial certification was 0.835 (95% CI 0.756 to 0.922; p<0.001). Patients cared for by physicians with initial certification had a 15.87% decrease in mortality compared with those cared for by non-certified physicians (mortality rate difference of 5.09 per 1000 patients; 95% CI 2.12 to 8.05; p<0.001). The adjusted OR for MOC was 0.804 (95% CI 0.697 to 0.926; p=0.003). Patients cared for by physicians who completed MOC had an 18.91% decrease in mortality compared with those cared for by MOC lapsed physicians (mortality rate difference of 6.22 per 1000 patients; 95% CI 2.0 to 10.4; p=0.004). CONCLUSIONS: Initial certification was associated with lower mortality for AMI or CHF. Moreover, for patients whose physicians had initial certification, an additional advantage was associated with its maintenance.

Specialty Board Certification Rate as an Outcome Metric for GME Training Institutions: A Relationship With Quality of Care.

Educational outcome measures, known to be associated with the quality of care, are needed to support improvements in graduate medical education (GME). This retrospective observational study sought to determine whether there was a relationship between the specialty board certification rates of GME training institutions and the quality of care delivered by their graduates. It is based on 7 years of hospitalizations in Pennsylvania (N = 354,767) with diagnoses of acute myocardial infarction, congestive heart failure, gastrointestinal hemorrhage, or pneumonia. The 2,265 attending physicians were self-identified internists, and they completed their training in 59 institutions. The percentage of board-certified physicians from each training institution, excluding the physician herself or himself, was calculated and an indicator of whether it exceeded 80% was created. This was analyzed against inhospital mortality and length of stay, adjusted for patient/physician/hospital characteristics. There were significantly lower odds of mortality (adjusted Odd's ratio [OR] = .92, 95% CI [0.86, 0.98]) and log length of stay (adjusted OR = .98, 95% CI [.94, .99]) when the attending physician trained in a residency program with an 80% or greater certification rate. The results suggest that specialty certification rates may be a useful educational outcome for residency training programs.

We Must Graduate Physicians, Not Doctors.

Today, medical schools graduate doctors, not physicians. Thousands of doctors who are U.S. citizens and graduates of U.S. and international medical schools will never become physicians because they do not obtain a residency position. Doctors need at least one year of residency to become a licensed physician. However, 4,099 applicants in 2018 and 4,170 in 2019 failed to get a position through the National Resident Matching Program Main Match; about 1,000 students get positions after the Main Match each year. The personal and societal cost is enormous: each year, approximately 3,000 nonphysician doctors cannot use 12,000 education years and three-quarters of a billion dollars they invested in medical education and cannot mitigate the shortfall of 112,000 physicians expected in 2030.To ameliorate this problem, medical schools could guarantee one year of residency. This is affordable: despite federally funded slots being capped, residency positions have increased for 17 consecutive years (20,602 in 2002 to 32,194 in 2019) because residents are cost-effective additions to the workforce. Alternatively, a 3-year curriculum plus required fourth-year primary care residency is another option. The salary during the residency year could equal other first-year residents', or there could be a token amount for this "internship." Both models decrease the cost of medical education; the second financially unburdens the hospital.Since the Flexner Report (when there was no formal postgraduate training), the end point of medical education has moved from readiness for independent medical practice (physician) to readiness for postgraduate training (doctor). To benefit individuals and society, medical education must take steps to ensure that all graduates are physicians, not just doctors.

Interprofessional communication in a socio-hierarchical culture: development of the TRI-O guide.

OBJECTIVES: Interprofessional education (IPE) and collaborative practice are essential for patient safety. Effective teamwork starting with partnership-based communications should be introduced early in the educational process. Many societies in the world hold socio-hierarchical culture with a wide power distance, which makes collaboration among health professionals challenging. Since an appropriate communication framework for this context is not yet available, this study filled that gap by developing a guide for interprofessional communication, which is best suited to the socio-hierarchical and socio-cultural contexts. MATERIALS AND METHODS: The draft of the guide was constructed based on previous studies of communication in health care in a socio-hierarchical context, referred to international IPE literature, and refined by focus group discussions among various health professionals. Nominal group technique, also comments from national and international experts of communication skills in health care, was used to validate the guide. A pilot study with a pre-posttest design was conducted with 53 first- and 107 fourth-year undergraduate medical, nursing, and health nutrition students. RESULTS: We developed the "TRI-O" guide of interprofessional communication skills, emphasizing "open for collaboration, open for information, open for discussion", and found that the application of the guide during training was feasible and positively influenced students' perceptions. CONCLUSION: The findings suggest that the TRI-O guide is beneficial to help students initiate partnership-based communication and mutual collaboration among health professionals in the socio-hierarchical and socio-cultural context.

Pathways to Independent Primary Care Clinical Practice: How Tall Is the Shortest Giant?

Patients can be treated by a physician, a nurse practitioner (NP), or a physician assistant (PA) despite marked differences in the education and training for these three professions. This natural experiment allows examination of a critical question: What is the minimum education and training required to practice primary care? In other words, how tall is the shortest giant? State licensing requirements, not educational bodies, legislate minimum training. The current minimum is 6 years, which includes 27.5 weeks of supervised clinical experience (SCE), for NPs. In comparison, PAs train for 6 years with 45 weeks of SCE, and physicians for at least 8 years with 110 weeks of SCE. Initial, flawed studies show equivalent patient outcomes among the professions. If rigorous follow-up studies confirm equivalence, the content and length of medical education for primary care physicians should be reconsidered. Unmatched medical school graduates, with 7 years of training and 65 weeks of SCE, more than the required minimum for NPs, deserve to practice independently. So do PAs. If equivalence is not confirmed, the minimum requirements for NPs and/or PAs should be raised, including considering a required residency (currently optional). Alternatively, the scope of practice for the three professions could be defined to reflect differences in training. There is an urgent need to set aside preconceived notions and turf battles, conduct rigorous independent studies, and generate meaningful data on practice patterns and patient outcomes. This should inform optimal training, scope of practice, and workforce development for each invaluable primary care clinical practitioner.

Assessing clinical reasoning skills using Script Concordance Test (SCT) and extended matching questions (EMQs): A pilot for urology trainees.

INTRODUCTION: Clinical reasoning skill is the core of medical competence. Commonly used assessment methods for medical competence have limited ability to evaluate critical thinking and reasoning skills. Script Concordance Test (SCT) and Extended Matching Questions (EMQs) are the evolving tests which are considered to be valid and reliable tools for assessing clinical reasoning and judgment. We performed this pilot study to determine whether SCT and EMQs can differentiate clinical reasoning ability among urology residents, interns and medical students. METHODS: This was a cross-sectional study in which an examination with 48 SCT-based items on eleven clinical scenarios and four themed EMQs with 21 items were administered to a total of 27 learners at three differing levels of experience i.e. 9 urology residents, 6 interns and 12 fifth year medical students. A non-probability convenience sampling was done. The SCTs and EMQs were developed from clinical situations representative of urological practice by 5 content experts (urologists) and assessed by a medical education expert. Learners' responses were scored using the standard and the graduated key. A one way analysis of variance (ANOVA) was conducted to compare the mean scores across the level of experience. A p-value of < 0.05 was considered statistically significant. Test reliability was estimated by Cronbach α. A focused group discussion with candidates was done to assess their perception of test. RESULTS: Both SCT and EMQs successfully differentiated residents from interns and students. Statistically significant difference in mean score was found for both SCT and EMQs among the 3 groups using both the standard and the graduated key. The mean scores were higher for all groups as measured by the graduated key compared to the standard key. The internal consistency (Cronbach's α) was 0.53 and 0.6 for EMQs and SCT, respectively. Majority of the participants were satisfied with regard to time, environment, instructions provided and the content covered and nearly all felt that the test helped them in thinking process particularly clinical reasoning. CONCLUSIONS: Our data suggest that both SCT and EMQs are capable of discriminating between learners according to their clinical experience in urology. As there is a wide acceptability by all candidates, these tests could be used to assess and enhance clinical reasoning skills. More research is needed to prove validity of these tests.

Considerations that will determine if competency-based assessment is a sustainable innovation.

Educational assessment for the health professions has seen a major attempt to introduce competency based frameworks. As high level policy developments, the changes were intended to improve outcomes by supporting learning and skills development. However, we argue that previous experiences with major innovations in assessment offer an important road map for developing and refining assessment innovations, including careful piloting and analyses of their measurement qualities and impacts. Based on the literature, numerous assessment workshops, personal interactions with potential users, and our 40 years of experience in implementing assessment change, we lament the lack of a coordinated approach to clarify and improve measurement qualities and functionality of competency based assessment (CBA). To address this worrisome situation, we offer two roadmaps to guide CBA's further development. Initially, reframe and address CBA as a measurement development opportunity. Secondly, using a roadmap adapted from the management literature on sustainable innovation, the medical assessment community needs to initiate an integrated plan to implement CBA as a sustainable innovation within existing educational programs and self-regulatory enterprises. Further examples of down-stream opportunities to refocus CBA at the implementation level within faculties and within the regulatory framework of the profession are offered. In closing, we challenge the broader assessment community in medicine to step forward and own the challenge and opportunities to reframe CBA as an innovation to improve the quality of the clinical educational experience. The goal is to optimize assessment in health education and ultimately improve the public's health.

UK nationals who received their medical degrees abroad: selection into, and subsequent performance in postgraduate training: a national data linkage study.

OBJECTIVES: To compare the likelihood of success at selection into specialty training for doctors who were UK nationals but obtained their primary medical qualification (PMQ) from outside the UK ('UK overseas graduates') with other graduate groups based on their nationality and where they gained their PMQ. We also compared subsequent educational performance during postgraduate training between the graduate groups. DESIGN: Observational study linking UK medical specialty recruitment data with postgraduate educational performance (Annual Review of Competence Progression (ARCP) ratings). SETTING: Doctors recruited into national programmes of postgraduate specialist training in the UK from 2012 to 2016. PARTICIPANTS: 34 755 UK-based trainee doctors recruited into national specialty training programmes with at least one subsequent ARCP outcome reported during the study period, including 1108 UK overseas graduates. MAIN OUTCOME MEASURES: Odds of being deemed appointable at specialty selection and subsequent odds of obtaining a less versus more satisfactory category of ARCP outcome. RESULTS: UK overseas graduates were more likely to be deemed appointable compared with non-EU medical graduates who were not UK citizens (OR 1.29, 95% CI 1.16 to 1.42), although less so than UK (OR 0.25, 95% CI 0.23 to 0.27) or European graduates (OR 0.66, 95% CI 0.58 to 0.75). However, UK overseas graduates were subsequently more likely to receive a less satisfactory outcome at ARCP than other graduate groups. Adjusting for age, sex, experience and the economic disparity between country of nationality and place of qualification reduced intergroup differences. CONCLUSIONS: The failure of recruitment patterns to mirror the ARCP data raises issues regarding consistency in selection and the deaneries' subsequent annual reviews. Excessive weight is possibly given to interview performance at specialty recruitment. Regulators and selectors should continue to develop robust processes for selection and assessment of doctors in training. Further support could be considered for UK overseas graduates returning to practice in the UK.

Patients of doctors further from medical school graduation have poorer outcomes.

CONTEXT: There is an apparent contradiction between the findings of studies indicating that patient outcomes are better when physicians have a greater volume of practice and those that find outcomes to be worse with increased time since training, which implies greater volume. OBJECTIVES: This study was designed to estimate the adjusted relationships between physicians' characteristics, including recent practice volume and time since medical school graduation, and patient outcomes. METHODS: This is a retrospective observational study based on all Pennsylvania hospitalisations over 7 years for acute myocardial infarction, congestive heart failure, gastrointestinal haemorrhage, hip fracture and pneumonia. It refers to 694 020 hospitalisations in 184 hospitals attended by 5280 internists and family physicians. Patient severity of illness at admission and in-hospital mortality, hospital location and volume, and the physician's recent practice volume, time since medical school graduation, board certification, and citizenship or medical school location were analysed. RESULTS: After adjustment, recent practice volume did not have a statistically significant association with in-hospital mortality for all of the conditions combined. By contrast, each decade since graduation from medical school was associated with a 4.5% increase in relative risk for patient mortality. CONCLUSIONS: Recent practice volume does not mitigate the increase in patient mortality associated with physicians' time since medical school graduation. These findings underscore the need to finds ways to support and encourage learning.

The ABMS MOC Part III Examination: Value, Concerns, and Alternative Formats.

This article describes the presentations and discussions at a conference co-convened by the Council on Medical Education of the American Medical Association (AMA) and by the American Board of Medical Specialties (ABMS). The conference focused on the ABMS Maintenance of Certification (MOC) Part III Examination. This article, reflecting the conference agenda, covers the value of and evidence supporting the examination, as well as concerns about the cost of the examination, and-given the current format-its relevance. In addition, the article outlines alternative formats for the examination that four ABMS member boards are currently developing or implementing. Lastly, the article presents contrasting views on the approach to professional self-regulation. One view operationalizes MOC as a high-stakes, pass-fail process while the other perspective holds MOC as an organized approach to support continuing professional development and improvement. The authors hope to begin a conversation among the AMA, the ABMS, and other professional stakeholders about how knowledge assessment in MOC might align with the MOC program's educational and quality improvement elements and best meet the future needs of both the public and the physician community.

Introducing a model for optimal design of sequential objective structured clinical examinations.

In a sequential OSCE which has been suggested to reduce testing costs, candidates take a short screening test and who fail the test, are asked to take the full OSCE. In order to introduce an effective and accurate sequential design, we developed a model for designing and evaluating screening OSCEs. Based on two datasets from a 10-station pre-internship OSCE and considering three factors, namely, the number of stations, the criteria for selecting the stations, and the cut-off score, several hypothetical tests were proposed. To investigate their accuracy, the positive predictive value (PPV), the pass rate, and the negative predictive value (NPV) were calculated. Also, a "desirable" composite outcome was defined as PPV = 100 %, pass rate ≥50 %, and NPV ≥25 %. Univariate and multiple logistic regression analyses were conducted to estimate the effects of independent factors on the occurrence of the desirable outcome. In half of the screening tests no false positive result was detected. Most of the screening OSCEs had acceptable levels of pass rate and NPV. Considering the desirable composite outcome 20 screening OSCEs could have successfully predicted the results of the corresponding full OSCE. The multiple regression analysis indicated significant contributions for the selection criteria (p values = 0.019) and the cut-off score (p values = 0.017). In order to have efficient screening OSCEs with the lowest probability of the error rate, careful selection of stations with high values of discrimination or item total correlation, and use of a relatively stringent cut-off score should be considered.

Competency Evaluations in the Next Accreditation System: Contributing to Guidelines and Implications.

UNLABELLED: CONSTRUCT: This study examines validity evidence of end-of-rotation evaluation scores used to measure competencies and milestones as part of the Next Accreditation System (NAS) of the Accreditation Council for Graduate Medical Education (ACGME). BACKGROUND: Since the implementation of the milestones, end-of-rotation evaluations have surfaced as a potentially useful assessment method. However, validity evidence on the use of rotation evaluation scores as part of the NAS has not been studied. This article examines validity evidence for end-of-rotation evaluations that can contribute to developing guidelines that support the NAS. APPROACH: Data from 2,701 end-of-rotation evaluations measuring 21 out of 22 Internal Medicine milestones for 142 residents were analyzed (July 2013-June 2014). Descriptive statistics were used to measure the distribution of ratings by evaluators (faculty, n = 116; fellows, n = 59; peer-residents, n = 131), by postgraduate years. Generalizability analysis and higher order confirmatory factor analysis were used to examine the internal structure of ratings. Psychometric implications for combining evaluation scores using composite score reliability were examined. RESULTS: Milestone ratings were significantly higher for each subsequent year of training (15/21 milestones). Faculty evaluators had greater variability in ratings across milestones, compared to fellows and residents; faculty ratings were generally correlated with milestone ratings from fellows (r = .45) and residents (r = .25), but lower correlations were found for Professionalism and Interpersonal and Communication Skills. The Φ-coefficient was .71, indicating good reliability. Internal structure supported a 6-factor solution, corresponding to the hierarchical relationship between the milestones and the 6 core competencies. Evaluation scores corresponding to Patient Care, Medical Knowledge, and Practice-Based Learning and Improvement had higher correlations to milestones reported to the ACGME. Mean evaluation ratings predicted problem residents (odds ratio = 5.82, p < .001). CONCLUSIONS: Guidelines for rotation evaluations proposed in this study provide useful solutions that can help program directors make decisions on resident progress and contribute to assessment systems in graduate medical education.

Implementation of competency-based medical education: are we addressing the concerns and challenges?

CONTEXT: Competency-based medical education (CBME) has emerged as a core strategy to educate and assess the next generation of physicians. Advantages of CBME include: a focus on outcomes and learner achievement; requirements for multifaceted assessment that embraces formative and summative approaches; support of a flexible, time-independent trajectory through the curriculum; and increased accountability to stakeholders with a shared set of expectations and a common language for education, assessment and regulation. OBJECTIVES: Despite the advantages of CBME, numerous concerns and challenges to the implementation of CBME frameworks have been described, including: increased administrative requirements; the need for faculty development; the lack of models for flexible curricula, and inconsistencies in terms and definitions. Additionally, there are concerns about reductionist approaches to assessment in CBME, lack of good assessments for some competencies, and whether CBME frameworks include domains of current importance. This study will outline these issues and discuss the responses of the medical education community. METHODS: The concerns and challenges expressed are primarily categorised as: (i) those related to practical, administrative and logistical challenges in implementing CBME frameworks, and (ii) those with more conceptual or theoretical bases. The responses of the education community to these issues are then summarised. CONCLUSIONS: The education community has begun to address the challenges involved in implementing CBME. Models and guidance exist to inform implementation strategies across the continuum of education, and focus on the more efficient use of resources and technology, and the use of milestones and entrustable professional activities-based frameworks. Inconsistencies in CBME definitions and frameworks remain a significant obstacle. Evolution in assessment approaches from in vitro task-based methods to in vivo integrated approaches is responsive to many of the theoretical and conceptual concerns about CBME, but much work remains to be done to bring rigour and quality to work-based assessment.

The relationship between licensing examination performance and the outcomes of care by international medical school graduates.

PURPOSE: The Step 2 Clinical Knowledge (CK) examination of the United States Medical Licensing Examination sequence is a requirement for the certification of international medical graduates (IMGs) by the Educational Commission for Foreign Medical Graduates. An association between scores on the test and the quality of care later provided by those who take it is central to its use in certification and licensure. The purpose of this study was to determine whether there is a relationship between scores on Step 2 CK and patient outcomes for IMGs. METHOD: This is a retrospective observational study of the 60,958 hospitalizations from 2003 to 2009 in Pennsylvania where the principal diagnosis was acute myocardial infarction or congestive heart failure and the attending physician (N = 2,525) was an IMG who had taken the Step 2 CK. The main measures were the three-digit scores on Step 2 CK and in-hospital mortality. RESULTS: After adjustment for severity of illness, physician characteristics, and hospital characteristics, performance on Step 2 CK had a statistically significant inverse relationship with mortality. Each additional point on the examination was associated with a 0.2% (95% CI: 0.1%-0.4%) decrease in mortality. The size of the effect is noteworthy, with each standard deviation (roughly 20 points) equivalent to a 4% change in mortality risk. CONCLUSIONS: These findings provide evidence for the validity of Step 2 CK scores. Given the magnitude of its relationship with patient outcomes, the results support the use of the examination as an effective screening strategy for licensure.

How to set standards on performance-based examinations: AMEE Guide No. 85.

This AMEE Guide offers an overview of methods used in determining passing scores for performance-based assessments. A consideration of various assessment purposes will provide context for discussion of standard setting methods, followed by a description of different types of standards that are typically set in health professions education. A step-by-step guide to the standard setting process will be presented. The Guide includes detailed explanations and examples of standard setting methods, and each section presents examples of research done using the method with performance-based assessments in health professions education. It is intended for use by those who are responsible for determining passing scores on tests and need a resource explaining methods for setting passing scores. The Guide contains a discussion of reasons for assessment, defines standards, and presents standard setting methods that have been researched with performance-based tests. The first section of the Guide addresses types of standards that are set. The next section provides guidance on preparing for a standard setting study. The following sections include conducting the meeting, selecting a method, implementing the passing score, and maintaining the standard. The Guide will support efforts to determine passing scores that are based on research, matched to the assessment purpose, and reproducible.