Entrustable professional activities as a training and assessment framework in undergraduate medical education: A case study of a multi-institutional pilot.

In 2014, the Association of American Medical Colleges (AAMC) published 13 Core Entrustable Professional Activities (EPAs) that graduating students should be able to perform with indirect supervision when entering residency. A ten-school multi-year pilot was commissioned to test feasibility of implementing training and assessment of the AAMC's 13 Core EPAs. In 2020-21, a case study was employed to describe pilot schools' implementation experiences. Teams from nine of ten schools were interviewed to identify means and contexts of implementing EPAs and lessons learned. Audiotapes were transcribed then coded by investigators using conventional content analysis and a constant comparative method. Coded passages were organized in a database and analyzed for themes. Consensus among school teams regarding facilitators of EPA implementation included team commitment to piloting EPAs; agreement that: proximal EPA adoption with curriculum reform facilitates EPA implementation; EPAs 'naturally fit' in clerkships and provided opportunity for schools to reflect on and adjust curricula and assessments; and inter-school collaboration bolstered individual school progress. Schools did not make high-stakes decisions about student progress (e.g., promotion, graduation), yet EPA assessment results complemented other forms of assessment in providing students with robust formative feedback about their progress. Teams had varied perceptions of school capability to implement an EPA framework, influenced by various levels of dean involvement, willingness, and capability of schools to invest in data systems and provide other resources, strategic deployment of EPAs and assessments, and faculty buy-in. These factors affected varied pace of implementation. Teams agreed on the worthiness of piloting the Core EPAs, but substantial work is still needed to fully employ an EPA framework at the scale of entire classes of students with enough assessments per EPA and with required data validity/reliability. Recommendations stemming from findings may help inform further implementation efforts across other schools adopting or considering an EPA framework.

Comparing Entrustment Decision-Making Outcomes of the Core Entrustable Professional Activities Pilot, 2019-2020.

Importance: Gaps in readiness for indirect supervision have been identified for essential responsibilities encountered early in residency, presenting risks to patient safety. Core Entrustable Professional Activities (EPAs) for entering residency have been proposed as a framework to address these gaps and strengthen the transition from medical school to residency. Objective: To assess progress in developing an entrustment process in the Core EPAs framework. Design, Setting, and Participants: In this quality improvement study in the Core EPAs for Entering Residency Pilot, trained faculty made theoretical entrustment determinations and recorded the number of workplace-based assessments (WBAs) available for each determination in 2019 and 2020. Four participating schools attempted entrustment decision-making for all graduating students or a randomly selected subset of students. Deidentified, individual-level data were merged into a multischool database. Interventions: Schools implemented EPA-related curriculum, WBAs, and faculty development; developed systems to compile and display data; and convened groups to make theoretical summative entrustment determinations. Main Outcomes and Measures: On an EPA-specific basis, the percentage of students for whom an entrustment determination could be made, the percentage of students ready for indirect supervision, and the volume of WBAs available were recorded. Results: Four participating schools made 4525 EPA-specific readiness determinations (2296 determinations in 2019 and 2229 determinations in 2020) for 732 graduating students (349 students in 2019 and 383 students in 2020). Across all EPAs, the proportion of determinations of "ready for indirect supervision" increased from 2019 to 2020 (997 determinations [43.4%] vs 1340 determinations [60.1%]; 16.7 percentage point increase; 95% CI, 13.8-19.6 percentage points; P < .001), as did the proportion of determinations for which there were 4 or more WBAs (456 of 2295 determinations with WBA data [19.9%] vs 938 [42.1%]; 22.2 percentage point increase; 95% CI, 19.6-24.8 percentage points; P < .001). The proportion of EPA-specific data sets considered for which an entrustment determination could be made increased from 1731 determinations (75.4%) in 2019 to 2010 determinations (90.2%) in 2020 (14.8 percentage point increase; 95% CI, 12.6-16.9 percentage points; P < .001). On an EPA-specific basis, there were 5 EPAs (EPA 4 [orders], EPA 8 [handovers], EPA 10 [urgent care], EPA 11 [informed consent], and EPA 13 [patient safety]) for which few students were deemed ready for indirect supervision and for which there were few WBAs available per student in either year. For example, for EPA 13, 0 of 125 students were deemed ready in 2019 and 0 of 127 students were deemed ready in 2020, while 0 determinations in either year included 4 or more WBAs. Conclusions and Relevance: These findings suggest that there was progress in WBA data collected, the extent to which entrustment determinations could be made, and proportions of entrustment determinations reported as ready for indirect supervision. However, important gaps remained, particularly for a subset of Core EPAs.

Providing Care with Cultural, Racial, and Ethnic Humility: the Framework for an Additional Entrustable Professional Activity.

Healthcare inequities are rampant in the USA. There is a lack of standardization for training medical students in providing care with cultural, racial, and ethnic humility. This innovation is the framework of an entrustable professional activity that can be utilized to address this vital issue.

Entrustment Decision Making in the Core Entrustable Professional Activities: Results of a Multi-Institutional Study.

PURPOSE: In 2014, the Association of American Medical Colleges defined 13 Core Entrustable Professional Activities (EPAs) that all graduating students should be ready to do with indirect supervision upon entering residency and commissioned a 10-school, 5-year pilot to test implementing the Core EPAs framework. In 2019, pilot schools convened trained entrustment groups (TEGs) to review assessment data and render theoretical summative entrustment decisions for class of 2019 graduates. Results were examined to determine the extent to which entrustment decisions could be made and the nature of these decisions. METHOD: For each EPA considered (4-13 per student), TEGs recorded an entrustment determination (ready, progressing but not yet ready, evidence against student progressing, could not make a decision); confidence in that determination (none, low, moderate, high); and the number of workplace-based assessments (WBAs) considered (0->15) per determination. These individual student-level data were de-identified and merged into a multischool database; chi-square analysis tested the significance of associations between variables. RESULTS: The 2,415 EPA-specific determinations (for 349 students by 4 participating schools) resulted in a decision of ready (n = 997/2,415; 41.3%), progressing but not yet ready (n = 558/2,415; 23.1%), or evidence against student progression (n = 175/2,415; 7.2%). No decision could be made for the remaining 28.4% (685/2,415), generally for lack of data. Entrustment determinations' distribution varied across EPAs (chi-square P < .001) and, for 10/13 EPAs, WBA availability was associated with making (vs not making) entrustment decisions (each chi-square P < .05). CONCLUSIONS: TEGs were able to make many decisions about readiness for indirect supervision; yet less than half of determinations resulted in a decision of readiness to perform this EPA with indirect supervision. More work is needed at the 10 schools to enable authentic summative entrustment in the Core EPAs framework.

Taking CBL to the Lecture Hall: a Comparison of Outcomes Between Traditional Small Group CBL and a Novel Large Group Team-Based CBL Teaching Method.

PURPOSE: Case-based learning (CBL), an important component of medical school curricula, is an effective inquiry-based teaching method associated with high levels of student and teacher satisfaction. However, because traditional CBL requires small groups, its feasibility is limited by faculty and resources. We developed and tested a novel team-based CBL (TB-CBL) method to be implemented in the lecture hall. METHODS: All second-year students at our institution (n = 121) were randomized to either traditional small group CBL or TB-CBL during the Endocrine block and to the other modality during the Renal block. All students were exposed to both methods. Case content was identical, and sessions were run concurrently. This cross-over, non-inferiority study tested the hypothesis that no difference in knowledge acquisition, clinical reasoning, or student satisfaction would be detected between groups. RESULTS: Based on student performance on case-relevant exam questions, no difference in knowledge acquisition was seen between groups for either block (p = 0.62 Endocrine, p = 0.38 Renal). There was also no difference in overall final exam performance between groups (p = 0.56 Endocrine, p = 0.26 Renal). Case-relevant script concordance testing revealed no difference in clinical reasoning skills between groups (p = 0.87 Endocrine, p = 0.17 Renal). Satisfaction was higher for the TB-CBL format (p = 0.005). Cost analysis revealed that each small group CBL session costs $2654, while each TB-CBL session costs approximately $221. CONCLUSIONS: TB-CBL, a novel case-based teaching method, appears to produce similar learner outcomes and higher student satisfaction when compared with small group CBL. TB-CBL may be used to supplement case-based curricula while optimizing resource allocation.

Developing an Entrustment Process: Insights from the AAMC CoreEPA Pilot.

One of the main goals of the CoreEPA pilot has been to determine the feasibility of developing a process to make summative entrustment decisions regarding entrustable professional activities (EPAs). Five years into the pilot, we report results of a research study we conducted to explore approaches to the entrustment process undertaken by our ten participating schools. We sought to identify the choices that participating schools made regarding the entrustment process and why these decisions were made. We are sharing these results, highlighting ongoing challenges that were identified with the intent of helping other medical schools that are moving toward EPA-based assessment. We conducted semi-structured interviews with representatives of all 10 medical schools in the CoreEPA pilot to understand their choices in designing the entrustment process. Additional information was obtained through follow-up communication to ensure completeness and accuracy of the findings. Several common themes are described. Our results indicate that, while approaches to the entrustment process vary considerably, all schools demonstrated consistent adherence to the guiding principles of the pilot. Several common barriers to the entrustment process emerged, and there was a consensus that more experience is needed with the process before consequential entrustment decisions can be made. The CoreEPA pilot schools continue to address challenges identified in implementing entrustment processes and making entrustment decisions for our students graduating in the Class of 2020.

From the Classroom to Entrustment - The Development of Motivational Interviewing Skills as an Entrustable Professional Activity.

This article was migrated. The article was marked as recommended. Introduction The move towards value-based care and population health has highlighted the prominent role of social and behavioral factors in determining health outcomes. Patient-centered behavioral guidance to improve patient self-management is recognized as an evidence-based intervention for a variety of chronic conditions but has yet to be adopted as a core competency or core entrustable professional activity (EPA). Motivational Interviewing (MI) is an evidence-based behavioral intervention involving an integrated set of competencies, featuring reflective listening, affirmation, evocation, and collaborative planning. An MI encounter is an observable, discrete task that can be framed as an EPA. Successful implementation of EPAs in the workplace requires institutional engagement, a thoughtful curricular approach, faculty development, and feasible, valid workplace-based assessment (WBA). Methods We implemented competency-based MI training and assessed competency outcomes for students and faculty. After joining the Association of American Medical Colleges Core EPA Pilot, we applied an iterative group process to develop an EPA and workplace-based assessment based on established MI competencies. Results Drawing upon nine years of developing MI curriculum, we present competency data for a student training study and a faculty coaching study, describe how we transitioned training from the classroom to the clinical setting employing an EPA framework, and present a one-page schematic and related WBA for an EPA based on MI. Conclusion We propose that MI is a core EPA for future physicians practicing value-based care, and offer a roadmap for curriculum implementation.

Simulation technology for skills training and competency assessment in medical education.

Medical education during the past decade has witnessed a significant increase in the use of simulation technology for teaching and assessment. Contributing factors include: changes in health care delivery and academic environments that limit patient availability as educational opportunities; worldwide attention focused on the problem of medical errors and the need to improve patient safety; and the paradigm shift to outcomes-based education with its requirements for assessment and demonstration of competence. The use of simulators addresses many of these issues: they can be readily available at any time and can reproduce a wide variety of clinical conditions on demand. In lieu of the customary (and arguably unethical) system, whereby novices carry out the practice required to master various techniques--including invasive procedures--on real patients, simulation-based education allows trainees to hone their skills in a risk-free environment. Evaluators can also use simulators for reliable assessments of competence in multiple domains. For those readers less familiar with medical simulators, this article aims to provide a brief overview of these educational innovations and their uses; for decision makers in medical education, we hope to broaden awareness of the significant potential of these new technologies for improving physician training and assessment, with a resultant positive impact on patient safety and health care outcomes.

A multicenter study to provide evidence of construct validity in a computer-based outcome measure of neurology clinical skills.

BACKGROUND: Using computer-based simulation to assess clinical skill-a key competence for medical trainees-enables standardization and exposure to a broad sample of physical findings. The purpose of this study is to provide evidence of construct validity for a computer-based outcome measure of neurology clinical skills. METHOD: A total of 128 medical students and neurology residents at four institutions volunteered to take a 34-question computer-based test designed to measure neurology clinical skills. Subjects were classified into three groups based on level of training: novice, intermediate, and experienced. RESULTS: Overall performance increased with level of training. Question difficulty discriminated between groups as predicted. Twenty-six of 34 individual items discriminated between novices and more advanced learners. This test separated learners at different levels of training with a consistency of .92. CONCLUSION: This study provided evidence of construct validity for a computer-based outcome measure of neurology clinical skills.