Exploring Medical Students' Learning Around Uncertainty Management Using a Digital Educational Escape Room: A Design-based Research Approach.

Introduction: Medical professionals meet many transitions during their careers, and must learn to adjust rapidly to unfamiliar workplaces and teams. This study investigated the use of a digital educational escape room (DEER) in facilitating medical students' learning around managing uncertainty in transitioning from classroom to clinical placement. Methods: We used design-based research to explore the design, build, and test of a DEER, as well as gain insight into how these novel learning environments work, using Community of Inquiry (CoI) as a guiding conceptual framework. This study represented a mixed methods pilot test of a prototype DEER. Twenty-two medical students agreed to participate, and data were collected through qualitative (i.e., focus groups, game-play observations) and quantitative (i.e., questionnaires) methods. Results: Eighty-two per cent of participants agreed or strongly agreed that the DEER supported their learning around uncertainty. Participants offered diverse examples of how the game had facilitated new insights on, and approaches to, uncertainty. With respect to the learning environment, multiple indicators and examples of the three domains of CoI - cognitive, teaching and social presence - were observed. Discussion: Our findings suggested that DEERs offer a valuable online learning environment for students to engage with complex and emotion-provoking challenges, such as those experienced at transitions. The study also suggested that CoI can be applied to the design, implementation, and evaluation of DEER learning environments, and we have proposed a set of design principles that may offer guidance here.

Building digital escape rooms for learning: From theory to practice.

Digital games are increasingly used to support learning across a diverse range of cognitive, affective and psychomotor domains in health professions education. Game-based learning will likely become an important competency for educators. However, educators can perceive game building as out of their reach due to a lack of expertise in digital technology. This toolbox offers advice to health professions educators who would like to build a simple game for learning known as a digital educational escape room.

Shaping the right conditions in programmatic assessment: how quality of narrative information affects the quality of high-stakes decision-making.

BACKGROUND: Programmatic assessment is increasingly being implemented within competency-based health professions education. In this approach a multitude of low-stakes assessment activities are aggregated into a holistic high-stakes decision on the student's performance. High-stakes decisions need to be of high quality. Part of this quality is whether an examiner perceives saturation of information when making a holistic decision. The purpose of this study was to explore the influence of narrative information in perceiving saturation of information during the interpretative process of high-stakes decision-making. METHODS: In this mixed-method intervention study the quality of the recorded narrative information was manipulated within multiple portfolios (i.e., feedback and reflection) to investigate its influence on 1) the perception of saturation of information and 2) the examiner's interpretative approach in making a high-stakes decision. Data were collected through surveys, screen recordings of the portfolio assessments, and semi-structured interviews. Descriptive statistics and template analysis were applied to analyze the data. RESULTS: The examiners perceived less frequently saturation of information in the portfolios with low quality of narrative feedback. Additionally, they mentioned consistency of information as a factor that influenced their perception of saturation of information. Even though in general they had their idiosyncratic approach to assessing a portfolio, variations were present caused by certain triggers, such as noticeable deviations in the student's performance and quality of narrative feedback. CONCLUSION: The perception of saturation of information seemed to be influenced by the quality of the narrative feedback and, to a lesser extent, by the quality of reflection. These results emphasize the importance of high-quality narrative feedback in making robust decisions within portfolios that are expected to be more difficult to assess. Furthermore, within these "difficult" portfolios, examiners adapted their interpretative process reacting on the intervention and other triggers by means of an iterative and responsive approach.

Implementation of a Blended Learning Module to Teach Handling, Restraint, and Physical Examination of Cats in Undergraduate Veterinary Training.

Cats can be easily stressed in a clinical (training) setting and may show unpredictable reactions and patterns of defensive aggression. This can be a complicating factor in undergraduate veterinary training. Inexperienced veterinary students can evoke defensive feline behavior that negatively affects learning outcomes and animal welfare. As a result, restraint techniques and physical examination of cats was hardly practiced in pre-clinical training at Utrecht University. To overcome this, a new blended learning module was developed using a lecture on feline behavior; e-learning modules about feline behavior, handling, restraint, and physical examination skills; and redesigned practical sessions in which live animals and manikins were used. The aim of this study was to investigate how students' perceptions of competence and confidence changed regarding feline behavior, handling, restraint, and physical examination skills after the new module was implemented. Questionnaires were used for quantitative analysis, and focus groups were used for qualitative analysis. The results show that compared with students who followed the standard module, students who participated in the blended learning module scored higher in feeling confident with handling animals, feeling competent to perform physical examination on cats, and ability to assess whether a cat is stressed. Students with less experience with cats were more likely to show improvement in assessing a cat's stress level than students who had much experience with cats. The results demonstrate that the blended learning module improves students' learning outcomes regarding feline skills training and adds to reduction, refinement, and replacement of the use of live cats.

Ottawa 2020 consensus statements for programmatic assessment - 2. Implementation and practice.

INTRODUCTION: Programmatic assessment is a longitudinal, developmental approach that fosters and harnesses the learning function of assessment. Yet the implementation, a critical step to translate theory into practice, can be challenging. As part of the Ottawa 2020 consensus statement on programmatic assessment, we sought to provide descriptions of the implementation of the 12 principles of programmatic assessment and to gain insight into enablers and barriers across different institutions and contexts. METHODS: After the 2020 Ottawa conference, we surveyed 15 Health Profession Education programmes from six different countries about the implementation of the 12 principles of programmatic assessment. Survey responses were analysed using a deductive thematic analysis. RESULTS AND DISCUSSION: A wide range of implementations were reported although the principles remained, for the most part, faithful to the original enunciation and rationale. Enablers included strong leadership support, ongoing faculty development, providing students with clear expectations about assessment, simultaneous curriculum renewal and organisational commitment to change. Most barriers were related to the need for a paradigm shift in the culture of assessment. Descriptions of implementations in relation to the theoretical principles, across multiple educational contexts, coupled with explanations of enablers and barriers, provided new insights and a clearer understanding of the strategic and operational considerations in the implementation of programmatic assessment. Future research is needed to further explore how contextual and cultural factors affect implementation.

Electromyographic Analysis of Muscle Activation Patterns During Bovine Transrectal Palpation and the Development of the Bovine Pregnancy Diagnosis Improvement Exercise Program.

The unusual and tiring physical activity of bovine transrectal palpation (TRP) requires a novel approach to improve students' TRP and pregnancy diagnosis (PD) skills. It has been shown that students who participated in an exercise program and students who had a grip strength (GS) of more than 30 kilograms performed better in bovine PDs. Participation in the exercise program increased students' sensitivity (ability to identify pregnant cows) but did not increase total arm muscle strength. To identify which muscles are used during TRPs and to improve the exercise program, an electromyographic (EMG) analysis was used to identify muscle activation patterns and muscle activity levels during bovine TRPs. Eight subject matter experts (SMEs) each palpated two live cows and one Breed'n Betsy® rectal examination simulator while an EMG Triggered Stimulator recorded muscle activity. Muscle activation was higher for forearm muscles compared with all other examined muscle groups (p < .001); was higher during retraction of the uterus and palpation of left and right uterine horn, compared with palpation of cervix, uterine body, left ovary, and right ovary (p < .001); and showed an endurance pattern. Findings have been used to modify the previously developed exercise program in effort to improve students' TRP and PD skills. The Bovine PD Improvement Exercise Program is available to students through an online application (http://icarus.up.ac.za/vetmlp/) and aims to not only improve GS and TRP accuracy but also stamina and well-being while adding fun to busy study schedules.

Influence of an Exercise Program, Muscle Strength, Proprioception, and Arm Length on Veterinary Students' Bovine Pregnancy Diagnosis Accuracy.

Bovine pregnancy diagnosis (PD) by transrectal palpation (TRP) is an important skill for veterinary graduates. Factors influencing students' PD accuracy were investigated to optimize bovine PD by TRP training without increasing live animal exposure. The objective was to determine whether arm length and strength, proprioception, and exposure to a 6-week exercise training program were significantly associated with students' PD accuracy. Veterinary students (n = 128) who had previously received formal theoretical and practical training in bovine TRP and PD (live cows and TRP simulators) were assessed for PD accuracy on live cows. Prior to assessment, arm muscle strength measurement, an exercise program, and additional TRP sessions on Breed'n Betsy® simulators and live cows were offered to the students. Seventy-eight students volunteered to participate in the arm length measurement, muscle strength, and proprioception testing. Of these, 35 randomly allocated students completed a 6-week exercise program, after which muscle strength was reassessed. Each student performed PDs on six cows of which the pregnancy status, ranging from 6 weeks to 9 months pregnant or not pregnant, was predetermined by an experienced veterinarian. PD accuracy was measured as sensitivity and specificity, being defined as the proportion of pregnant or nonpregnant cows, respectively, correctly identified by the student. It is concluded that hand grip strength and participation in an exercise program are significant predictors of veterinary students' PD accuracy. Implementation of an exercise program aimed at improving grip strength in the veterinary curriculum is a novel approach to improve bovine TRP and PD training.

Bridging the Gap between Undergraduate Veterinary Training and Veterinary Practice with Entrustable Professional Activities.

The transition from being a veterinary student to becoming a member of the veterinary profession is known to be challenging. Despite being licensed directly after graduation, many veterinarians do not feel fully equipped to practice unsupervised when they graduate. The increasing rate of attrition from veterinary practice, and a relatively high percentage of burnout during the first years in practice, has been suggested to be related to a lack of early career support. Over the past decade, medical education has adopted the concept of entrustable professional activities (EPAs). Recently, EPAs have been proposed to restructure veterinary education to help support the transition from veterinary student to practicing veterinarian. Implementing an EPA-based approach could help to bridge the gap between school and clinical practice, potentially preventing veterinary graduates from dropping out early on from what could have been a promising and exciting professional career.

Effect of a high-intensity one-week training programme and student-level variables on the bovine transrectal palpation and pregnancy diagnosis skills of final-year veterinary students.

BACKGROUND: To improve bovine transrectal palpation (TRP) and pregnancy diagnosis (PD) training, the effect of a high-intensity one-week training programme for veterinary elective students (N=59) with an interest in production animal practice was evaluated. METHODS: Training consisted of exposure to rectal examination simulators, abattoir organs, theory materials and live cow PDs supervised by experienced large animal practitioners. Palpation skills were assessed before and after training using a validated TRP Objective Structured Clinical Examination (OSCE) in non-pregnant cows. Each student then performed PDs (n=12) on cows of known pregnancy status. Students' PD accuracy was measured as sensitivity and specificity, being respectively defined as the proportion of pregnant and non-pregnant cows correctly identified. RESULTS: Students' scores improved from the first to the second OSCE (P=0.03), mostly as a result of improved ability to identify uterine symmetry/asymmetry and the presence/absence of a corpus luteum on the right ovary (P<0.01 and P=0.03, respectively). Overall student sensitivity and specificity of PD were 89.1 per cent (95 per cent CI 78.1-92.2 per cent) and 67.7 per cent (95 per cent CI 60.1-74.5 per cent), respectively. CONCLUSION: This prospective cohort study describes a strategy to improve students' TRP skills with the potential to reduce training time and animal use at teaching institutions by outsourcing student training to private practitioners.

Collaborative Development of a Shared Framework for Competency-Based Veterinary Education.

Competency-based medical education is an educational innovation implemented in health professions worldwide as a means to ensure graduates meet patient and societal needs. The focus on student-centered education and programmatic outcomes offers a series of benefits to learners, institutions and society. However, efforts to establish a shared, comprehensive competency-based framework in veterinary education have lagged. This article reports on the development and outcome of a competency-based veterinary education (CBVE) framework created through multi-institutional collaboration with international input from veterinary educators and veterinary educational leaders. The CBVE Framework is designed to reflect the competencies expected of new graduates from member institutions of the Association of American Veterinary Medical Colleges (AAVMC). The CBVE Framework consists of nine domains of competence and 32 competencies, each supplemented with illustrative sub-competencies to guide veterinary schools in implementing competency-based education in their local context. The nine domains of competence are: clinical reasoning and decision-making; individual animal care and management; animal population care and management; public health; communication; collaboration; professionalism and professional identity; financial and practice management; and scholarship. Developed through diverse input to facilitate broad adoption, the CBVE Framework provides the foundation for competency-based curricula and outcomes assessment in veterinary education internationally. We believe that other groups seeking to design a collective product for broad adoption might find useful the methods used to develop the CBVE Framework, including establishing expertise diversity within a small-to-medium size working group, soliciting progressive input and feedback from stakeholders, and engaging in consensus building and critical reflection throughout the development process.

Collaborative Development of Core Entrustable Professional Activities for Veterinary Education.

Entrustable professional activities (EPAs) have been proposed as a practical framework for the implementation of competency-based education. As veterinary education moves toward a competency-based approach, core EPAs provide a context for assessment of workplace activities. This article reports on the development of eight core clinical EPAs for veterinary education created through multi-institutional collaboration, with international input from veterinary educators and veterinary educational leaders. These core EPAs are intended as minimal expectations for clinical activities that should be assessed for every graduate of Association of American Veterinary Medical Colleges member institutions. Adoption of the core EPAs and the associated Competency-Based Veterinary Education (CBVE) framework by veterinary schools is intended to promote Day One graduate competence and thereby enhance patient care and client service.

Identifying entrustable professional activities for surgical skills training in companion animal health.

BACKGROUND: Veterinary medical education is increasingly moving towards outcome-based training based on competency frameworks. A source of concern is the translation of competencies into the practice of clinical teaching, for example, surgical skills training. It is suggested that the use of entrustable professional activities (EPAs) might bridge this gap. The purpose of this study, therefore, was to identify EPAs related to surgical skills for companion animal health to enhance competency-based education. METHODS: Draft versions of EPAs related to surgical skills were established by an iterative consensus-based approach through 45-min interview sessions. These draft versions were used to explore the opinion of companion animal veterinarians, both veterinarians (specialists, residents and interns) involved in undergraduate teaching and veterinarians working in private practice involved in extramural clinical teaching, on the relevance and level of entrustment of the EPAs through a modified Delphi procedure. Mean (relevance) and median (level of entrustment) scores were calculated and textual comments were analysed to create a final framework of EPAs related to surgical skills. RESULTS AND CONCLUSION: The Delphi panel reached consensus in three rounds. Thirty-four per cent of those invited to participate in the study completed the final survey. Finally, a list of 13 EPAs related to companion animal surgical skills a student should be entrusted to perform at time of graduation was established.

Qualified but not yet fully competent: perceptions of recent veterinary graduates on their day-one skills.

BACKGROUND: The goal of veterinary education is to prepare learners to successfully enter the profession. However, the transition from learner to professional can be an intense and stressful phase. In this study, recently graduated veterinarians' perceptions of readiness to work independently and to successfully cope with early career challenges are addressed. METHODS: A survey based on five commonly occurring entrustable professional activities (EPAs) in primary care was sent to newly qualified veterinarians (graduated between six months and three-and-a-half years ago and working in primary veterinary clinics). The survey was a combination of open and Likert scale-type questions and contained items on the self-reported need for supervision for these EPAs. One hundred and fifty-six participants (response rate 41.2 per cent) answered the survey. Descriptive statistics were used to analyse and present the quantitative data. RESULTS: The day-one after graduation levels varied per EPA between 'with direct, proactive supervision' and 'supervision at a distance'. On average after 6.8 months participants felt ready to execute all five tasks with distant supervision. After almost 10 months, participants had the feeling of being fully competent to execute the EPAs unsupervised. CONCLUSION: This study provides insight into early career challenges faced by recently graduated veterinarians. The results emphasise the importance of adequate preparation of veterinarians during education and the importance of guidance during early career to foster a successful transition from veterinary school to clinical practice.

Assessment Tools for Feedback and Entrustment Decisions in the Clinical Workplace: A Systematic Review.

BACKGROUND: Entrustable Professional Activities (EPAs) combine feedback and evaluation with a permission to act under a specified level of supervision and the possibility to schedule learners for clinical service. This literature review aims to identify workplace-based assessment tools that indicate progression toward unsupervised practice, suitable for entrustment decisions and feedback to learners. METHODS: A systematic search was performed in the PubMed, Embase, ERIC, and PsycINFO databases. Based on title/abstract and full text, articles were selected using predetermined inclusion and exclusion criteria. Information on workplace-based assessment tools was extracted using data coding sheets. The methodological quality of studies was assessed using the medical education research study quality instrument (MERSQI). RESULTS: The search yielded 6,371 articles (180 were evaluated in full text). In total, 80 articles were included, identifying 67 assessment tools. Only a few studies explicitly mentioned assessment tools used as a resource for entrustment decisions. Validity evidence was frequently reported, and the MERSQI score was 10.0 on average. CONCLUSIONS: Many workplace-based assessment tools were identified that potentially support learners with feedback on their development and support supervisors with providing feedback. As expected, only few articles referred to entrustment decisions. Nevertheless, the existing tools or the principals could be used for entrustment decisions, supervision level, or autonomy.

Development of core entrustable professional activities linked to a competency-based veterinary education framework.

Purpose: Despite the adoption of competency-based education in some veterinary schools over the past 15 years, only recently has a concerted effort been directed toward this in veterinary education internationally.Methods: In 2015, educational leaders from the Association of American Veterinary Medical Colleges (AAVMC) member schools came together with a strong call to action to create shared tools for clinical competency assessment.Results: This resulted in the formation of the AAVMC Competency-Based Veterinary Education (CBVE) Working Group, which then embarked on the creation of a shared competency framework and the development of eight core entrustable professional activities (EPAs) linked to this framework.Conclusions: This paper will report on the development of these EPAs and their integration with the concurrently-developed CBVE Framework.

Ability of a bovine transrectal palpation objective structured clinical examination to predict veterinary students' pregnancy diagnosis accuracy.

Bovine pregnancy diagnosis (PD) by transrectal palpation (TRP) is one of the most frequently performed procedures in bovine practice, and an important competency for veterinary graduates. It is currently not known if pre-existing TRP skills on non-pregnant cows can be used to predict students' future PD accuracy. The study objective was to evaluate if TRP objective structured clinical examination (OSCE) scores can predict students' future PD accuracy.Fourth year (of a six-year programme) veterinary students (n=128) received TRP and PD training on Breed'n Betsy (BB) simulators and live cows. Students' TRP skills were assessed using a live cow TRP OSCE after completion of the fourth year training. The same students received additional TRP (BB and live cows) and PD (BB) training sessions in the first semester of their fifth year. PD accuracy was assessed after the additional TRP and PD training, five months after the TRP OSCE assessment and measured as sensitivity and specificity (the ability to correctly identify the presence and absence of pregnancy, respectively). Each student palpated six cows transrectally to diagnose pregnancy status and stage for the PD assessment. The TRP OSCE results were analysed as predictors for students' PD accuracy.Students with 'competent palpation skills' on the TRP OSCE had higher PD specificity. The individual OSCE components that were predictive of higher PD accuracy were students' ability to estimate ovarian size, identify uterine position and exclude intrauterine fluid. It was concluded that a TRP OSCE has the ability to predict students' future PD accuracy.

Inter-Rater Reliability of Grading Undergraduate Portfolios in Veterinary Medical Education.

The reliability of high-stakes assessment of portfolios containing an aggregation of quantitative and qualitative data based on programmatic assessment is under debate, especially when multiple assessors are involved. In this study carried out at the Faculty of Veterinary Medicine, Utrecht University, the Netherlands, two independent assessors graded the portfolios of students in their second year of the 3-year clinical phase. The similarity of grades (i.e., equal grades) and the level of the grades were studied to estimate inter-rater reliability, taking into account the potential effects of the assessor's background (i.e., originating from a clinical or non-clinical department) and student's cohort group, gender, and chosen master track (Companion Animal Health, Equine Health, or Farm Animal/Public Health). Whereas the similarity between the two grades increased from 58% in the first year the grading system was introduced to around 80% afterwards, the grade level was lower over the next 3 years. The assessor's background had a minor effect on the proportion of similar grades, as well as on grading level. The assessor intraclass correlation was low (i.e., all assessors scored with a similar grading pattern [same range of grades]). The grades awarded to female students were higher but more often dissimilar. We conclude that the grading system was well implemented and has a high inter-rater reliability.