Defining a set of teaching EPAs tailored to an undergraduate medical curriculum using a modified Delphi approach.

BACKGROUND: The concept of entrustable professional activities (EPAs) has recently been extended to operationalize professional tasks in teacher training and faculty development in health professions education. The aim of this study is to report on the process and results of defining a set of teaching EPAs (t-EPAs) tailored to the local characteristics of a particular undergraduate medical program. METHODS: The undergraduate medical program at the Charité - Universitätsmedizin Berlin is competency-based, integrates thematic modules and spans 6 years. A writing team identified teaching EPAs based on the program's study regulations and drafted content descriptions with titles, specifications and knowledge, skills and attitudes. Content validation involved a modified Delphi procedure with a systematic, iterative interaction between a panel of content experts consisting of purposively selected educators and physicians from our faculty (n = 11) and the writing team. The threshold for a consensus was an agreement of 80% of the participants. RESULTS: After two Delphi rounds, a consensus was reached regarding the teaching activities to be included and their content descriptions. The response rate was 100% in both Delphi rounds. The Delphi results include the content descriptions of a total of 13 teaching EPAs, organized into the two overarching EPA domains of classroom-based (n = 10) and workplace-based (n = 3) activities. Tailoring the classroom EPAs to small group teaching and the workplace EPAs to supervising medical students led to several distinct EPAs. Another feature was the development of 2 teaching EPAs for interdisciplinary teaching. CONCLUSIONS: In systematic, Delphi-based process, we defined a set of 13 distinct teaching EPAs tailored to a specific undergraduate medical program that cover the core teaching tasks for faculty in this program. Our report on the principles of the process and the results may guide other medical schools and educators in defining and tailoring teaching EPAs according to their contexts.

Big five personality traits of medical students and workplace performance in the final clerkship year using an EPA framework.

BACKGROUND: The qualities of trainees play a key role in entrustment decisions by clinical supervisors for the assignments of professional tasks and levels of supervision. A recent body of qualitative research has shown that in addition to knowledge and skills, a number of personality traits are relevant in the workplace; however, the relevance of these traits has not been investigated empirically. The aim of this study was to analyse the workplace performance of final-year medical students using an Entrustable Professional Activity (EPA) framework in relation to their personality traits. METHODS: Medical students at the end of their final clerkship year were invited to participate in an online survey-based, cross-sectional field study. In the survey, the workplace performance was captured using a framework consisting of levels of experienced supervision and a defined set of 12 end-of-undergraduate medical training EPAs. The Big Five personality traits (extraversion, agreeableness, conscientiousness, neuroticism, and openness) of the participating medical students were measured using the Big Five Inventory-SOEP (BFI-S), which consists of 15 items that are rated on a seven-point Likert scale. The data were analysed using descriptive and inferential statistics. RESULTS: The study included 880 final-year medical students (mean age: 27.2 years, SD = 3.0; 65% female). The levels of supervision under which the final-year clerkship students carried out the EPAs varied considerably. Significant correlations were found between the levels of experienced supervision and all Big Five dimensions The correlations with the dimensions of extraversion, agreeableness, conscientiousness and openness were positive, and that for the neuroticism dimension was negative (range r = 0.17 to r = - 0.23). Multiple regression analyses showed that the combination of the Big Five personality traits accounted for 0.8-7.5% of the variance in supervision levels on individual EPAs. CONCLUSIONS: Using the BFI-S, we found that the levels of supervision on a set of end-of-undergraduate medical training EPAs were related to the personality traits of final-year medical students. The results of this study confirm the existing body of research on the role of conscientiousness and extraversion in entrustment decision-making and, in particular, add the personality trait of neuroticism as a new and relevant trainee quality to be considered.

Personal protective equipment and medical students in times of COVID-19: experiences and perspectives from the final clerkship year.

BACKGROUND: The availability and correct use of personal protective equipment (PPE) to prevent and control infections plays a critical role in the safety of medical students in clinical placements. This study explored their experiences and perspectives in their final clerkship year with PPE during the COVID-19 pandemic. METHODS: This qualitative study was based on social constructivism and was conducted in 2021 at the Charité - Universitätsmedizin Berlin. In three online focus group discussions, 15 medical students in their final clerkship year reported their experiences with PPE training and use during the COVID-19 pandemic. Data were recorded, transcribed and analysed based on Kuckartz's approach to content analysis. We drew upon the a priori dimensions of the capability, opportunity, motivation - behaviour (COM-B) model as main categories as well as emergent issues raised by the study participants (subcategories). RESULTS: In addition to the three main categories of the COM-B model, eleven subcategories were identified through inductive analysis. The study participants reported several factors that hindered the correct use of PPE. In the area of capabilities, these factors were related to learning experience with PPE in terms of both theoretical and practical learning together with later supervision in practice. In the area of opportunities, these factors included the limited availability of some PPE components, a lack of time for PPE instruction and supervision and inappropriate role modelling due to the inconsistent use of PPE by physicians and nursing staff. The area of motivation to use PPE was characterized by an ambivalent fear of infection by the SARS-CoV-2 virus and the prioritization of patient safety, i.e., the need to prevent the transmission of the virus to patients. CONCLUSIONS: Our study revealed several limitations pertaining to the enabling factors associated with the trainable behaviour "correct use of PPE". The concept of shared responsibility for student safety was used to derive recommendations for future improvement specifically for the medical school as an organization, the teachers and supervisors, and students themselves. This study may guide and stimulate other medical schools and faculties to explore and analyse components of student safety in clinical settings in times of infectious pandemics.

Entrustable professional activities for Junior Brazilian Medical Students in community medicine.

BACKGROUND: Entrustable professional activities (EPAs) have been defined to promote the workplace participation of undergraduate medical students, generally in the context of high-income countries with a focus on the secondary and tertiary health care sectors. These EPAs have limited applicability to training and health care contexts in low- to middle-income countries that have a focus on primary health care, for instance, the context of community medicine. The purpose of this article is to report the process and results of defining EPAs for undergraduate medical training in a community health care setting. METHODS: A modified Delphi study was performed to develop EPAs for the training of medical students in community medicine during their first and second years of education at the Marília Medical School (FAMEMA), Brazil. The supervision level was operationalized in terms of a student's ability to perform the EPA autonomously in an effective and safe manner with supervision readily available on request. Panellists (9 physicians and 6 nurses) rated the completeness of the proposed list of EPAs and EPA categories on four-point Likert scales. The threshold for consensus among panellists was a mean content validity index of at least 80%. RESULTS: Consensus was reached after two Delphi rounds, resulting in 11 EPAs for undergraduate medical education and training in community medicine. These EPAs were organized into three overarching EPA domains: integrality of care for individual health needs in all phases of the life cycle (5 EPAs), integrality of care for family health needs (3 EPAs), and integrality of care for community health needs (3 EPAs). For each EPA, descriptions of the following categories were created: title; specifications and limitations; conditions and implications of the entrustment decision; knowledge, skills, and attitudes; links to competencies; and assessment sources. CONCLUSION: The resulting 11 EPAs for training medical students in community medicine expand the application of the EPA framework to both early undergraduate medical education and the context of primary health care. This report can support and guide other medical schools in their attempts to train students in primary health care contexts and to incorporate EPAs into their curricula.

Design and usability testing of an in-house developed performance feedback tool for medical students.

BACKGROUND: Feedback is essential in a self-regulated learning environment such as medical education. When feedback channels are widely spread, the need arises for a system of integrating this information in a single platform. This article reports on the design and initial testing of a feedback tool for medical students at Charité-Universitätsmedizin, Berlin, a large teaching hospital. Following a needs analysis, we designed and programmed a feedback tool in a user-centered approach. The resulting interface was evaluated prior to release with usability testing and again post release using quantitative/qualitative questionnaires. RESULTS: The tool we created is a browser application for use on desktop or mobile devices. Students log in to see a dashboard of "cards" featuring summaries of assessment results, a portal for the documentation of acquired practical skills, and an overview of their progress along their course. Users see their cohort's average for each format. Learning analytics rank students' strengths by subject. The interface is characterized by colourful and simple graphics. In its initial form, the tool has been rated positively overall by students. During testing, the high task completion rate (78%) and low overall number of non-critical errors indicated good usability, while the quantitative data (system usability scoring) also indicates high ease of use. The source code for the tool is open-source and can be adapted by other medical faculties. CONCLUSIONS: The results suggest that the implemented tool LevelUp is well-accepted by students. It therefore holds promise for improved, digitalized integrated feedback about students' learning progress. Our aim is that LevelUp will help medical students to keep track of their study progress and reflect on their skills. Further development will integrate users' recommendations for additional features as well as optimizing data flow.

What can we expect from medical graduates? Empirical survey on the performance of Core EPAs in the first days of residency.

BACKGROUND: Core Entrustable Professional Activities (EPAs) have been defined to specify the performance expectations for entering residents worldwide. The content of these EPAs was elaborated and validated primarily via medical expert consent approaches. The present study aims to collect empirical information on the actual task performance and supervision level of entering residents as a complementary methodological approach to enhance the content validity of a set of institutional EPAs. METHODS: In the summers of 2017 and 2018, Charité medical graduates (n = 720) received a post-graduation survey by mail. The questionnaire covered the performance of Core EPAs, Core procedures and more advanced EPAs. Graduates were asked how frequently they had performed the respective EPAs since the start of residency and under what level of supervision. We expected the large majority of graduates (> 75%) to have performed the Core EPAs and procedures under at least indirect supervision. RESULTS: In total, 215 graduates (30%) returned the questionnaire, and 131 (18%) surveys could be included in the data analysis. The majority of participants were female (63%) and worked in hospitals (50%) or in university medical centres (30%) across various medical disciplines. Among the Core EPAs, 10 out of 11 tasks had been performed by more than 75% of graduates since the start of residency, 9 under indirect supervision. Regarding the Core procedures, only 3 out of 13 procedures had been performed by the large majority of graduates under indirect supervision, and 10 procedures had not been carried out by at least one-third of participants. Among the 5 advanced EPAs, none of 5 had been performed by more than 75% of the participants since the start of residency, and 4 had been carried out by 50% under indirect supervision. CONCLUSIONS: The results of this study largely and complementarily confirm the validity of the defined Core EPAs representing realistic expectations for entry into residence at our institution. The low actual performance rate of Core procedures serves to stimulate an institutional discussion on their adjustment to better match the workplace reality.

Clinical reasoning for acute dyspnoea: comparison between final-year medical students from discipline- and competency-based undergraduate programmes.

Clinical reasoning for acute dyspnoea: Comparison of final-year medical students from discipline- and competency-based undergraduate programmes. BACKGROUND: The global shift to competency-based medical education aims to improve the performance of its trainees, including in the key competency domain of clinical reasoning. However, research on whether such education actually improves clinical reasoning is sparse. The purpose of this study is to compare assessed clinical reasoning performance in digitally presented cases of acute dyspnoea between final-year medical students from a traditional, discipline-based and those from an integrated, competency-based undergraduate programme. METHODS: A total of 60 medical students in their final-year clerkships participated in the study; 30 were from a discipline-based programme, and 30 were from a competency-based programme of the same faculty. The students completed a knowledge test consisting of 22 single choice items and a computer-based test of clinical reasoning with six video-based case scenarios with different underlying diseases leading to dyspnoea. The operationalized measures of clinical reasoning were the number and relevance of the diagnostic tests chosen, time to diagnosis and diagnostic accuracy. RESULTS: The two groups did not differ in their knowledge of the acute dyspnoea content domain. With regard to clinical reasoning, the selection of relevant tests, time required to make a diagnosis and accuracy of the diagnosis varied across the six case scenarios in both groups. However, the results from the measures of the clinical reasoning process did not differ between the students from the two types of undergraduate medical programmes. No significant differences were found with regard to the selection of relevant diagnostic tests (M = 63.8% vs. M = 62.8%), the time to a diagnosis (M = 128.7 s vs. M = 136.4 s) or the accuracy of diagnosis (M = 82.2% vs. M = 77.0%). CONCLUSIONS: Key indicators of the clinical reasoning process, when assessed with objectively measured parameters, did not differ between final-year medical students from a traditional, discipline-based and those from an integrated, competency-based undergraduate programme in the domain of acute dyspnoea. The results substantiate and expand those of previous studies based on subjective assessor ratings that showed limited change in the clinical reasoning performance of medical students with competency-based undergraduate education.

Tempo do Brasil traduzir para a prática o currículo Médico Baseado em Competência por meio de Atividades Profissionais Confiáveis (APCs) / Tiempo de Brasil traducir a la práctica el Currículo Médico Basado en Competencia por medio de Actividades Profesionales Confiables (APCs) / The time has come for Brazil: translating Competence Based Medical Education into practice by Entrustable Professional Activities (EPAs)

As competências médicas apresentadas pelas Diretrizes Curriculares Nacionais (DCN) e as políticas de incentivo à formação de mais médicos popularizaram temas como aprendizado ativo e competência. Distorções na implementação curricular resultaram em abordagens fragmentadas e reprodutivas das práticas com distanciamento do processo de trabalho. Vários países ocidentais, ao buscarem o aprimoramento da Educação Médica Baseada em Competência, adotaram as Atividades Profissionais Confiáveis (APCs) na tradução bem-sucedida desse referencial para a prática clínica. Este estudo perspectivo apresenta as APCs no âmbito da educação médica brasileira como possível solução para a melhor efetivação do Currículo Médico Baseado em Competência (CMBC). O estudo descreve as APCs, de seu conceito à realização; provê subsídios para seu entendimento e análise de sua capacidade em mediar uma formação médica mais qualificada e responsiva às necessidades de saúde locais; e contribui com a literatura brasileira na área.(AU)

Las competencias médicas presentadas por las Directrices Curriculares Nacionales y las políticas de incentivo a la formación de más médicos popularizaron temas tales como aprendizaje activo y competencia. Distorsiones en la implementación curricular resultaron en abordajes fragmentados y reproductivos de las prácticas con distanciamiento del proceso de trabajo. Diversos países occidentales, al buscar el perfeccionamiento de la Educación Médica Basada en Competencia, adoptaron las Actividades Profesionales Confiables (APCs) en la traducción exitosa de esas referencias a la práctica clínica. Este estudio perspectivo presenta las APCs en el ámbito de la educación médica brasileña, como posible solución para la mejor efectuación del Currículo Médico Basado en Competencia (CMBC). El estudio describe las APCs, desde su concepto hasta su realización, proporciona subsidios para su entendimiento y análisis de su capacidad para mediar una formación médica más calificada y responsiva a las necesidades de salud local y contribuye con la literatura brasileña en el área..(AU)

The medical competencies presented by the National Curriculum Guidelines and the policies to encourage the education of more physicians have popularized themes such as competence and active learning. Distortions in the curricular implementation resulted in a fragmented and reproductive approach of the practices distancing them of the work process. Several Western countries seeking improvement of the Competence-Based Medical Education (CBME), have recently adopted the concept of Entrustable Professional Activities (EPAs) to better translate this benchmark into clinical practice. This article presents the EPAs for Brazilian medical education as a possible solution for CBME accomplishment. The study explores the EPAs, from their concept to their realization. It aims to provide insights for their understanding and analysis of their capacity as a mediator for more qualified training and more responsivity to local health needs, contributing to the literature in the area..(AU)

Exploring the introduction of entrustment rating scales in an existing objective structured clinical examination.

BACKGROUND: The concept of EPAs is increasingly applied to assess trainees' workplace performance by means of entrustment ratings. OSCEs assess performance in a simulated setting, and it is unclear whether entrustment ratings can be integrated into these exams. This study explores the introduction of an entrustment rating scale into an existing OSCE. METHODS: A 6-point entrustment scale was added to the standard ratings in an OSCE administered prior to students' final clerkship year in an undergraduate medical programme. Standard OSCE ratings assess clinical and communication skills. Assessors (n = 54) rated students' performance (n = 227) on a diverse set of clinical tasks and evaluated the addition of entrustment scales to OSCEs. Descriptive and inferential statistics were calculated for analyses. RESULTS: Student performance varied across the stations, as reflected in both the standard OSCE ratings and the added entrustment ratings. Students received generally high standard OSCE ratings, whereas entrustment ratings were more widely distributed. All students passed the OSCE, and only a small proportion of students did not reach the expected pass threshold of 60% on the standard ratings in the single stations. The proportion of students who did not reach the expected entrustment level in the respective stations was noticeably higher. Both the clinical and communication skill ratings were related to the entrustment rating in most OSCE stations. A majority of the assessors positively evaluated the addition of entrustment ratings into the OSCE. DISCUSSION: The findings provide an empirical basis to broaden our understanding of the potential use of entrustment ratings in existing OSCEs. They provide directions for future, more specific studies. The ratings might be used for formative feedback on students' readiness for workplace practice.

Introducing an assessment tool based on a full set of end-of-training EPAs to capture the workplace performance of final-year medical students.

BACKGROUND: While literature on the theoretical value of entrustable professional activities (EPAs) for assessment is rapidly expanding, little experience exists on its application. The aims of this study are to develop and explore the utility of an EPA-based assessment tool for capturing the workplace performance of final-year medical students based on a full set of end-of-training EPAs. METHODS: The tool was developed in a systematic iterative process. Twelve 12 end-of-undergraduate medical training EPAs were nested into 72 smaller EPAs and cross-mapped onto a 6-point supervision level scale, both adjusted to the context of final-year clerkships. One version was created for students' self-assessment of their ability to carry out tasks and their history of carrying out tasks, and another version was created for supervisors' assessment of students' ability to carry out tasks. The tool was administered to final-year clerkship students and their clinical supervisors to explore its utility as an assessment approach. The results were analysed using descriptive and interferential statistics. RESULTS: We enrolled a total of 60 final-year medical students. For 33 students, ratings were provided from one supervisor and for 27 students from two supervisors. With regard to the reliability and validity of the tool, students' and supervisors' ratings showed an overall good internal consistency as well as variability between and within the EPAs. Over the full EPA range, students rated their ability to perform a task slightly higher than their task performance history and slightly lower than the supervisors' ratings. Students' self-ratings of their ability to perform a task correlated with their history in performing the task. Supervisors' ratings correlated among supervisors and not with students' ratings. Concerning educational outcomes, supervisors' average rating of students' ability to perform the EPAs without direct supervision was 64%, and key findings being double-checked. CONCLUSIONS: This study introduces a tool that is adjusted to the final-year clerkship context and can assess the workplace performance of trainees based on a full set of end-of-training EPAs. Its utility characteristics suggest that the tool may be employed as a formative and outcome-aligned approach to the assessment of final-year students before entering into residency.

Patient safety during final-year clerkships: A qualitative study of possible error sources and of the potential of Entrustable Professional Activities.

Aim: In final-year clerkships, such as the Practical Year in Germany, students' workplace learning has to be balanced with the ensuring of patient safety. In this qualitative study, we investigated problems concerning patient safety as perceived from the perspective of supervising physicians, and whether and to what extent Entrustable Professional Activities (EPAs) can lead to an improvement in patient safety. Method: Data was collected through focus groups. Participants were specialist physicians with experience of final-year clerkship training (n=11). The analysis of problems influencing patient safety was carried out deductively with an existing system of categories (error factors in the clinic). To identify potential improvements through EPAs, an inductively developed category system on the influence of EPAs in final-year clerkships was used. Results: Supervising physicians perceive a variety of problems which affect patient safety. These can be found in the categories organization and management, individual factors, task factors and work environment. The physicians feel that EPAs may lead to an improvement in training and subsequently in patient safety. Their comments can be collated to the categories improvement in training, performance levels and supporting learning processes, transparency and minimizing uncertainty. Conclusions: Statements by supervising physicians indicate a variety of problems in patient safety during the training of final-year clerkship students, for instance the lack of structure to the training. In their view, the implementation of EPAs can substantially reduce such risks, as they provide better content and organizational structure to the final-year clerkship.

Development of Entrustable Professional Activities for entry into residency at the Charité Berlin.

Background: Entrustable Professional Activities (EPAs) have emerged as a new approach to operationalise the workplace performance expectations for the transition from under- to postgraduate medical training. However, the transferability of such EPAs from one context to another appears limited. In this article, we report on the results of our approach to define a full set of core EPAs for entry into residency with the expectation to be performed under distant supervision. Methods: The EPA development involved a modified, three round Delphi study, conducted at the Charité - Universitätsmedizin Berlin. The supervision level was operationalised as supervisor being distantly available and findings being reviewed. The threshold for consent was reaching a content validity index of a least 80%. The Delphi study involved experienced physicians (n=45) and resulted in a set of core EPAs with the descriptions of the categories: title, specification/limitations, conditions and implications of entrustment decision, knowledge, skills, and attitude, link to competencies and assessment sources. Results: The response rates were 76-80% in the Delphi rounds. Key to the content validation process for the performance expectation was deciding on "to act under distant supervision". The results are full descriptions of 12 core EPAs, organised into 5 overarching EPA domains. Conclusions: Our systematic approach yielded the definition of 12 core EPAs for entry into residency at the level of "act with distant supervision" according to the practice in our context. This report may support other medical schools who plan to implement EPAs into their curricula.

Medical students' preparedness for professional activities in early clerkships.

BACKGROUND: Sufficient preparedness is important for transitions to workplace participation and learning in clinical settings. This study aims to analyse medical students' preparedness for early clerkships using a three-dimensional, socio-cognitive, theory-based model of preparedness anchored in specific professional activities and their supervision level. METHODS: Medical students from a competency-based undergraduate curriculum were surveyed about preparedness for 21 professional activities and level of perceived supervision during their early clerkships via an online questionnaire. Preparedness was operationalized by the three dimensions of confidence to carry out clerkship activities, being prepared through university teaching and coping with failure by seeking support. Factors influencing preparedness and perceived stress as outcomes were analysed through step-wise regression. RESULTS: Professional activities carried out by the students (n = 147; 19.0%) and their supervision levels varied. While most students reported high confidence to perform the tasks, the activity-specific analysis revealed important gaps in preparation through university teaching. Students regularly searched for support in case of difficulty. One quarter of the variance of each preparedness dimension was explained by self-efficacy, supervision quality, amount of prior clerkship experience and nature of professional activities. Preparedness contributed to predicting perceived stress. CONCLUSIONS: The applied three-dimensional concept of preparedness and the task-specific approach provided a detailed and meaningful view on medical students' workplace participation and experiences in early clerkships.

Twelve tips for the implementation of EPAs for assessment and entrustment decisions.

The concept of entrustable professional activities (EPAs) reframes the approach to assessment in competency-based medical education. Key to this concept is the linking of assessment to decision making about entrusting learners with clinical responsibilities. Based on recent literature and the authors' experiences with implementing EPAs, this article provides practical recommendations for how to implement EPAs for assessment and entrustment decisions in the workplace. Tips for supervising clinicians include talking to learners about trust, using EPA descriptions to guide learning and teaching, providing learners with greater ad hoc responsibilities, using EPAs to identify/create opportunities for assessment and feedback, including case-based discussions and acknowledging gut feelings about learner readiness for more autonomy. Tips for curriculum leaders entail enabling the trust development, applying trust decisions at all levels of the supervision scale, employing all available information sources for entrustment, empowering learner ownership of the assessment process and using technology for learner tracking and program evaluation.

Applying occupational and organizational psychology theory to entrustment decision-making about trainees in health care: a conceptual model.

In medical contexts around the world, supervising physicians continuously decide what degree of supervision to apply as trainees carry out professional activities. Although the implications for patients can be far-reaching, little is known about how these entrustment decisions are formed. The concept of 'Entrustable Professional Activities' has initiated interest and valuable research on factors that may influence the entrustment decision process.The aim of the current article is to link models of entrustment developed in the fields of occupational and organizational psychology and military psychology to medical education studies that have explored the factors influencing physicians' entrustment decisions. We provide a conceptual framework of the entrustment decision-making process, which we suggest will contribute to the understanding of how supervising physicians arrive at the decision to entrust a medical trainee with a professional activity.