Making assessment a team sport: a qualitative study of facilitated group feedback in internal medicine residency.

Purpose: Competency-based medical education relies on feedback from workplace-based assessment (WBA) to direct learning. Unfortunately, WBAs often lack rich narrative feedback and show bias towards Medical Expert aspects of care. Building on research examining interactive assessment approaches, the Queen's University Internal Medicine residency program introduced a facilitated, team-based assessment initiative ("Feedback Fridays") in July 2017, aimed at improving holistic assessment of resident performance on the inpatient medicine teaching units. In this study, we aim to explore how Feedback Fridays contributed to formative assessment of Internal Medicine residents within our current model of competency-based training. Method: A total of 53 residents participated in facilitated, biweekly group assessment sessions during the 2017 and 2018 academic year. Each session was a 30-minute facilitated assessment discussion done with one inpatient team, which included medical students, residents, and their supervising attending. Feedback from the discussion was collected, summarized, and documented in narrative form in electronic WBA forms by the program's assessment officer for the residents. For research purposes, verbatim transcripts of feedback sessions were analyzed thematically. Results: The researchers identified four major themes for feedback: communication, intra- and inter-personal awareness, leadership and teamwork, and learning opportunities. Although feedback related to a broad range of activities, it showed strong emphasis on competencies within the intrinsic CanMEDS roles. Additionally, a clear formative focus in the feedback was another important finding. Conclusions: The introduction of facilitated team-based assessment in the Queen's Internal Medicine program filled an important gap in WBA by providing learners with detailed feedback across all CanMEDS roles and by providing constructive recommendations for identified areas for improvement.

Objectif: La formation médicale fondée sur les compétences s'appuie sur la rétroaction faite lors de l'évaluation des apprentissages par observation directe dans le milieu de travail. Malheureusement, les évaluations dans le milieu de travail omettent souvent de fournir une rétroaction narrative exhaustive et privilégient les aspects des soins relevant de l'expertise médicale. En se basant sur la recherche ayant étudié les approches d'évaluation interactive, le programme de résidence en médecine interne de l'Université Queen's a introduit en juillet 2017 une initiative d'évaluation facilitée et en équipe (« Les vendredis rétroaction ¼), visant à améliorer l'évaluation holistique du rendement des résidents dans les unités d'enseignement clinique en médecine interne. Dans cette étude, nous visons à explorer comment ces « vendredis rétroaction ¼ ont contribué à l'évaluation formative des résidents en médecine interne dans le cadre de notre modèle actuel de formation axée sur les compétences. Méthode: Au total, 53 résidents ont participé à des séances d'évaluation de groupe facilitées et bi-hebdomadaires au cours de l'année universitaire 2017-2018. Chaque séance consistait en une discussion d'évaluation facilitée de 30 minutes menée avec une équipe de l'unité de soins, qui comprenait des étudiants en médecine, des résidents et le médecin superviseur. Les commentaires issus de la discussion ont été recueillis, résumés et documentés sous forme narrative dans des formulaires électroniques d'observation directe dans le milieu de travail par le responsable de l'évaluation du programme de résidence. À des fins de recherche, les transcriptions verbatim des séances de rétroaction ont été analysées de façon thématique. Résultats: Les chercheurs ont identifié quatre thèmes principaux pour les commentaires : la communication, la conscience intra- et interpersonnelle, le leadership et le travail d'équipe, et les occasions d'apprentissage. Bien que la rétroaction concerne un large éventail d'activités, elle met fortement l'accent sur les compétences liées aux rôles intrinsèques de CanMEDS. De plus, le fait que la rétroaction avait un rôle clairement formatif est une autre constatation importante. Conclusions: L'introduction de l'évaluation en équipe facilitée dans le programme de médecine interne à Queen's a comblé une lacune importante dans l'apprentissage par observation directe dans le milieu de travail en fournissant aux apprenants une rétroaction détaillée sur tous les rôles CanMEDS et en formulant des recommandations constructives sur les domaines à améliorer.

Closing the loop in minimally supervised human-robot interaction: formative and summative feedback.

Human instructors fluidly communicate with hand gestures, head and body movements, and facial expressions, but robots rarely leverage these complementary cues. A minimally supervised social robot with such skills could help people exercise and learn new activities. Thus, we investigated how nonverbal feedback from a humanoid robot affects human behavior. Inspired by the education literature, we evaluated formative feedback (real-time corrections) and summative feedback (post-task scores) for three distinct tasks: positioning in the room, mimicking the robot's arm pose, and contacting the robot's hands. Twenty-eight adults completed seventy-five 30-s-long trials with no explicit instructions or experimenter help. Motion-capture data analysis shows that both formative and summative feedback from the robot significantly aided user performance. Additionally, formative feedback improved task understanding. These results show the power of nonverbal cues based on human movement and the utility of viewing feedback through formative and summative lenses.

Feedback practices in undergraduate clinical teaching in Sri Lanka - a qualitative study.

BACKGROUND: Feedback is integral to medical education, enabling students to improve their knowledge, skills, and attitudes. Feedback practices may vary according to prevalent cultural and contextual factors. This study aimed to explore how feedback is conceptualized and practised in the clinical education of medical students in Sri Lanka. METHODS: The study was conducted in three medical schools and affiliated hospitals that represent the cultural diversity of Sri Lanka. Purposive sampling was utilized to recruit clinical teachers and students who would provide rich information for the study. The study had three components: an observation study, interviews with clinical teachers and focus group discussions with clinical students. During the observation study, video recording was used as a data collection tool to observe feedback in real-life clinical teaching/learning settings. A constructivist grounded theory approach was adapted for analysis to explore current practices and perceptions inductively. RESULTS: Feedback was conceptualised as spontaneous unidirectional provision of information for the improvement of students. It was often provided in public settings and in student groups. Error correction was the primary focus of feedback, but both teachers and students desired a balanced approach with reinforcement and reflection. Although the direct approach to corrective feedback was found beneficial for student learning, participants agreed that harsh feedback was to be avoided. The hierarchical culture and lack of programmed feedback in the curricula influenced feedback practices, suggesting the need for modification. CONCLUSIONS: This study highlighted feedback practices in the local context, emphasizing the need to address the hierarchical gap in clinical settings, balance reinforcement and correction, and promote dialogue and reflection in the feedback processes. The findings will help clinical teachers from both the global south as well as the global north to recognize cultural and contextual differences in providing feedback.

Radiologist preferences for faculty development initiatives to improve resident feedback in the era of competency-based medical education.

INTRODUCTION: Since 2022, all Canadian post-graduate medical programs have transitioned to a Competence by Design (CBD) model within a Competency-Based Medical Education (CBME) framework. The CBME model emphasized more frequent, formative assessment of residents to evaluate their progress towards predefined competencies in comparison to traditional medical education models. Faculty members therefore have increased responsibility for providing assessments to residents on a more regular basis, which has associated challenges. Our study explores faculty assessment behaviours within the CBD framework and assesses their openness to opportunities aimed at improving the quality of written feedback. Specifically, we explore faculty's receptiveness to routine metric performance reports that offer comprehensive feedback on their assessment patterns. METHODS: Online surveys were distributed to all 28 radiology faculty at Queen's University. Data were collected on demographics, feedback practices, motivations for improving the teacher-learner feedback exchange, and openness to metric performance reports and quality improvement measures. Following descriptive statistics, unpaired t-tests and one-way analysis of variance were conducted to compare groups based on experience and subspecialty. RESULTS: The response rate was 89% (25/28 faculty). 56% of faculty were likely to complete evaluations after working with a resident. Regarding the degree to which faculty felt written feedback is important, 62% found it at least moderately important. A majority (67%) believed that performance reports could influence their evaluation approach, with volume of written feedback being the most likely to change. Faculty expressed interest in feedback-focused development opportunities (67%), favouring Grand Rounds and workshops. CONCLUSION: Assessment of preceptor perceptions reveals that faculty recognize the importance of offering high-quality written feedback to learners. Faculty openness to quality improvement interventions for curricular reform relies on having sufficient time, knowledge, and skills for effective assessments. This suggests that integrating routine performance metrics into faculty assessments could serve as a catalyst for enhancing future feedback quality.

Exploring the content and delivery of feedback facilitation co-interventions: a systematic review.

BACKGROUND: Policymakers and researchers recommend supporting the capabilities of feedback recipients to increase the quality of care. There are different ways to support capabilities. We aimed to describe the content and delivery of feedback facilitation interventions delivered alongside audit and feedback within randomised controlled trials. METHODS: We included papers describing feedback facilitation identified by the latest Cochrane review of audit and feedback. The piloted extraction proforma was based upon a framework to describe intervention content, with additional prompts relating to the identification of influences, selection of improvement actions and consideration of priorities and implications. We describe the content and delivery graphically, statistically and narratively. RESULTS: We reviewed 146 papers describing 104 feedback facilitation interventions. Across included studies, feedback facilitation contained 26 different implementation strategies. There was a median of three implementation strategies per intervention and evidence that the number of strategies per intervention is increasing. Theory was used in 35 trials, although the precise role of theory was poorly described. Ten studies provided a logic model and six of these described their mechanisms of action. Both the exploration of influences and the selection of improvement actions were described in 46 of the feedback facilitation interventions; we describe who undertook this tailoring work. Exploring dose, there was large variation in duration (15-1800 min), frequency (1 to 42 times) and number of recipients per site (1 to 135). There were important gaps in reporting, but some evidence that reporting is improving over time. CONCLUSIONS: Heterogeneity in the design of feedback facilitation needs to be considered when assessing the intervention's effectiveness. We describe explicit feedback facilitation choices for future intervention developers based upon choices made to date. We found the Expert Recommendations for Implementing Change to be valuable when describing intervention components, with the potential for some minor clarifications in terms and for greater specificity by intervention providers. Reporting demonstrated extensive gaps which hinder both replication and learning. Feedback facilitation providers are recommended to close reporting gaps that hinder replication. Future work should seek to address the 'opportunity' for improvement activity, defined as factors that lie outside the individual that make care or improvement behaviour possible. REVIEW REGISTRATION: The study protocol was published at: https://www.protocols.io/private/4DA5DE33B68E11ED9EF70A58A9FEAC02 .

Development and validation of immediate self-feedback very short answer questions for medical students: practical implementation of generalizability theory to estimate reliability in formative examination designs.

BACKGROUND: Very Short Answer Questions (VSAQs) reduce cueing and simulate better real-clinical practice compared with multiple-choice questions (MCQs). While integrating them into formative exams has potential, addressing marking time and ideal occasions and items is crucial. This study gathers validity evidence of novel immediate self-feedback VSAQ (ISF-VSAQ) format and determines the optimal number of items and occasions for reliable assessment. METHODS: Ninety-four third-year pre-clinical students took two ten-item ISF-VSAQ exams on cardiovascular drugs. Each question comprised two sections: (1) Questions with space for student responses and (2) a list of possible correct answers offering partial-credit scores ranging from 0.00 to 1.00, along with self-marking and self-feedback options to indicate whether they fully, partially, or did not understand the possible answers. Messick's validity framework guided the collection of validity evidence. RESULTS: Validity evidence included five sources: (1) Content: The expert reviewed the ISF-VSAQ format, and the question was aligned with a standard examination blueprint. (2) Response process: Before starting, students received an example and guide to the ISF-VSAQ, and the teacher detailed the steps in the initial session to aid self-assessment. Unexpected answers were comprehensively reviewed by experts. (3) Internal structure: The Cronbach alphas are good for both occasions (≥ 0.70). A generalizability study revealed Phi-coefficients of 0.60, 0.71, 0.76, and 0.79 for one to four occasions with ten items, respectively. One occasion requires twenty-five items for acceptable reliability (Phi-coefficient = 0.72). (4) Relations to other variables: Inter-rater reliability between self-marking and teacher is excellent for each item (rs(186) = 0.87-0.98,p = 0.001). (5) Consequences: Path analysis revealed that the self-reflected understanding score in the second attempt directly affected the final MCQ score (ß = 0.25,p = 0.033). However, the VSAQ score did not. Regarding perceptions, over 80% of students strongly agreed/agreed that the ISF-VSAQ format enhances problem analysis, presents realistic scenarios, develops knowledge, offers feedback, and supports electronic usability. CONCLUSION: Electronic ISF-VSAQs enhanced understanding elevates learning outcomes, rendering them suitable for formative assessments with clinical scenarios. Increasing the number of occasions effectively enhances reliability. While self-marking is reliable and may reduce grading efforts, instructors should review answers to identify common student errors.

Comparison of the effects of apprenticeship training by sandwich feedback and traditional methods on final-semester operating room technology students' perioperative competence and performance: a randomized, controlled trial.

BACKGROUND: Effective feedback is fundamental in clinical education, as it allows trainers to constantly diagnose the trainees' condition, determine their weaknesses, and intervene at proper times. Recently, different feedback-based approaches have been introduced in clinical training; however, the effectiveness of such interventions still needs to be studied extensively, especially in the perioperative field. Therefore, this study sought to compare the effects of apprenticeship training using sandwich feedback and traditional methods on the perioperative competence and performance of Operating Room (OR) technology students. METHODS: Thirty final-semester undergraduate OR technology students taking the apprenticeship courses were randomly allocated into experimental (n = 15) and control (n = 15) groups through the stratified randomization approach. The students in the experimental group experienced Feedback-Based Learning (FBL) using a sandwich model, and the students in the control group participated in Traditional-Based Training (TBT) in six five-hour sessions weekly for three consecutive weeks. All students completed the Persian version of the Perceived Perioperative Competence Scale-Revised (PPCS-R) on the first and last days of interventions. Also, a blinded rater completed a checklist to evaluate all students' performance via Direct Observation of Procedural Skills (DOPS) on the last intervention day. Besides, the students in the FBL filled out a questionnaire regarding their attitude toward the implemented program. RESULTS: The mean total score of the PPCS-R was significantly higher in the FBL than in the TBT on the last intervention day (P < 0.001). Additionally, the increase in mean change of PPCS-R total score from the first to last days was significantly more in the FBL (P < 0.001). Likewise, the FBL students had higher DOPS scores than the TBT ones (P < 0.001). Most FBL students also had a good attitude toward the implemented program (n = 8; 53.3%). CONCLUSION: Apprenticeship training using a sandwich feedback-based approach was superior to the traditional method for enhancing perioperative competence and performance of final-semester OR technology students. Additional studies are required to identify the sustainability of the findings.

Critical reflections on formal teaching observations.

Formal teaching observations not only have an essential role in the visibility of teaching and learning but also allow for constructive, critical feedback for improvement. In both nursing and teaching, there is an emphasis on the need for efficient, regular reflective practice; this not to identify negative aspects but to transform professional practice. This article provides a personal insight into formal teaching observations, and themes arising from them of motivation, the teacher's voice, the use of abbreviations and acronyms and the use of exemplars, using a critical reflective approach.

Creation and Implementation of an Online Tool for Feedback on Resident Teaching: A Pilot Study.

OBJECTIVE: There are few assessments of the competence and growth of surgical residents as educators. We developed and piloted an observation-based feedback tool (FT) to provide residents direct feedback during a specific teaching session, as perceived by medical students (MS). We hypothesized that residents' performance would improve with frequent, low stakes, observation-based feedback. SETTING: This prospective study took place at an academic general surgery program. PARTICIPANTS: Focus groups of MS, surgical residents, and faculty informed FT development. MS completed the FT regarding resident teaching. DESIGN: The FT utilized 5 slider-bar ratings (0 to 100) about the teaching encounter and a checklist of 16 desirable teaching behaviors. QR codes and weekly email links were distributed for 12 months (6 clerkship blocks) to promote use. Residents were sent their results after each block. A survey after each block assessed motivation for use and gathered feedback on the FT. Descriptive statistics were used for analysis (medians, IQRs). Primary measures of performance were median of the slider-bar scores and the number of teaching behaviors. RESULTS: The FT was used 111 times; 37 of 46 residents were rated by up to 65 MS. The median rating on the slider-bars was 100 and the median number of desirable teaching behaviors was 12; there were no differences based on gender or PGY level. 10 residents had 5 or more FT observations during the year. Four residents had evaluations completed in 4 or more blocks and 19 residents had evaluations completed in at least 2 blocks. Over time, 13 residents had consistent slider-bar scores, 1 resident had higher scores, and 5 residents had lower scores (defined as a more than 5-point change from initial rating). Frequency of use of the FT decreased over time (38, 32, 9, 21, 7, 5 uses per block). The post-use survey was completed by 24 MS and 19 residents. Most common reasons for usage were interest in improving surgical learning environment, giving positive feedback (MS), and improving teaching skills (residents). Most common reasons for lack of usage from residents were "I did not think I taught enough to ask for feedback," "I forgot it existed," and "I did not know it existed." CONCLUSIONS: The FT did not lead to any meaningful improvement in resident scores over the course of the year. This may be due to overall high scores, suggesting that the components of the FT may require reevaluation. Additionally, decreased utilization of the instrument over time made it challenging to assess change in performance of specific residents, likely due to lack of awareness of the FT despite frequent reminders. Successful implementation of observation-based teaching assessments may require better integration with residency or clerkship objectives.

Impact of a coaching program on resident perceptions of communication confidence and feedback quality.

BACKGROUND: While communication is an essential skill for providing effective medical care, it is infrequently taught or directly assessed, limiting targeted feedback and behavior change. We sought to evaluate the impact of a multi-departmental longitudinal residency communication coaching program. We hypothesized that program implementation would result in improved confidence in residents' communication skills and higher-quality faculty feedback. METHODS: The program was implemented over a 3-year period (2019-2022) for surgery and neurology residents at a single institution. Trained faculty coaches met with assigned residents for coaching sessions. Each session included an observed clinical encounter, self-reflection, feedback, and goal setting. Eligible residents completed baseline and follow-up surveys regarding their perceptions of feedback and communication. Quantitative responses were analyzed using paired t-tests; qualitative responses were analyzed using content analysis. RESULTS: The baseline and follow-up survey response rates were 90.0% (126/140) and 50.5% (46/91), respectively. In a paired analysis of 40 respondents, residents reported greater confidence in their ability to communicate with patients (inpatient: 3.7 vs. 4.3, p < 0.001; outpatient: 3.5 vs. 4.2, p < 0.001), self-reflect (3.3 vs. 4.3, p < 0.001), and set goals (3.6 vs. 4.3, p < 0.001), as measured on a 5-point scale. Residents also reported greater usefulness of faculty feedback (3.3 vs. 4.2, p = 0.001). The content analysis revealed helpful elements of the program, challenges, and opportunities for improvement. Receiving mentorship, among others, was indicated as a core program strength, whereas solving session coordination and scheduling issues, as well as lowering the coach-resident ratio, were suggested as some of the improvement areas. CONCLUSIONS: These findings suggest that direct observation of communication in clinical encounters by trained faculty coaches can facilitate long-term trainee growth across multiple core competencies. Future studies should evaluate the impact on patient outcomes and workplace-based assessments.

Designing feedback processes in the workplace-based learning of undergraduate health professions education: a scoping review.

BACKGROUND: Feedback processes are crucial for learning, guiding improvement, and enhancing performance. In workplace-based learning settings, diverse teaching and assessment activities are advocated to be designed and implemented, generating feedback that students use, with proper guidance, to close the gap between current and desired performance levels. Since productive feedback processes rely on observed information regarding a student's performance, it is imperative to establish structured feedback activities within undergraduate workplace-based learning settings. However, these settings are characterized by their unpredictable nature, which can either promote learning or present challenges in offering structured learning opportunities for students. This scoping review maps literature on how feedback processes are organised in undergraduate clinical workplace-based learning settings, providing insight into the design and use of feedback. METHODS: A scoping review was conducted. Studies were identified from seven databases and ten relevant journals in medical education. The screening process was performed independently in duplicate with the support of the StArt program. Data were organized in a data chart and analyzed using thematic analysis. The feedback loop with a sociocultural perspective was used as a theoretical framework. RESULTS: The search yielded 4,877 papers, and 61 were included in the review. Two themes were identified in the qualitative analysis: (1) The organization of the feedback processes in workplace-based learning settings, and (2) Sociocultural factors influencing the organization of feedback processes. The literature describes multiple teaching and assessment activities that generate feedback information. Most papers described experiences and perceptions of diverse teaching and assessment feedback activities. Few studies described how feedback processes improve performance. Sociocultural factors such as establishing a feedback culture, enabling stable and trustworthy relationships, and enhancing student feedback agency are crucial for productive feedback processes. CONCLUSIONS: This review identified concrete ideas regarding how feedback could be organized within the clinical workplace to promote feedback processes. The feedback encounter should be organized to allow follow-up of the feedback, i.e., working on required learning and performance goals at the next occasion. The educational programs should design feedback processes by appropriately planning subsequent tasks and activities. More insight is needed in designing a full-loop feedback process, in which specific attention is needed in effective feedforward practices.

Structured feedback and operative video debriefing with critical view of safety annotation in training of laparoscopic cholecystectomy: a randomized controlled study.

BACKGROUND: The learning curve in minimally invasive surgery (MIS) is lengthened compared to open surgery. It has been reported that structured feedback and training in teams of two trainees improves MIS training and MIS performance. Annotation of surgical images and videos may prove beneficial for surgical training. This study investigated whether structured feedback and video debriefing, including annotation of critical view of safety (CVS), have beneficial learning effects in a predefined, multi-modal MIS training curriculum in teams of two trainees. METHODS: This randomized-controlled single-center study included medical students without MIS experience (n = 80). The participants first completed a standardized and structured multi-modal MIS training curriculum. They were then randomly divided into two groups (n = 40 each), and four laparoscopic cholecystectomies (LCs) were performed on ex-vivo porcine livers each. Students in the intervention group received structured feedback after each LC, consisting of LC performance evaluations through tutor-trainee joint video debriefing and CVS video annotation. Performance was evaluated using global and LC-specific Objective Structured Assessments of Technical Skills (OSATS) and Global Operative Assessment of Laparoscopic Skills (GOALS) scores. RESULTS: The participants in the intervention group had higher global and LC-specific OSATS as well as global and LC-specific GOALS scores than the participants in the control group (25.5 ± 7.3 vs. 23.4 ± 5.1, p = 0.003; 47.6 ± 12.9 vs. 36 ± 12.8, p < 0.001; 17.5 ± 4.4 vs. 16 ± 3.8, p < 0.001; 6.6 ± 2.3 vs. 5.9 ± 2.1, p = 0.005). The intervention group achieved CVS more often than the control group (1. LC: 20 vs. 10 participants, p = 0.037, 2. LC: 24 vs. 8, p = 0.001, 3. LC: 31 vs. 8, p < 0.001, 4. LC: 31 vs. 10, p < 0.001). CONCLUSIONS: Structured feedback and video debriefing with CVS annotation improves CVS achievement and ex-vivo porcine LC training performance based on OSATS and GOALS scores.

Risky business: medical students' feedback-seeking behaviours: a mixed methods study.

There are differing views on how learners' feedback-seeking behaviours (FSB) develop during training. With globalisation has come medical student migration and programme internationalisation. Western-derived educational practices may prove challenging for diverse learner populations. Exploring undergraduate activity using a model of FSB may give insight into how FSB evolves and the influence of situational factors, such as nationality and site of study. Our findings seek to inform medical school processes that support feedback literacy. Using a mixed methods approach, we collected questionnaire and interview data from final-year medical students in Ireland, Bahrain, and Malaysia. A validated questionnaire investigated relationships with FSB and goal orientation, leadership style preference, and perceived costs and benefits. Interviews with the same student population explored their FSB experiences in clinical practice, qualitatively, enriching this data. The data were integrated using the 'following the thread' technique. Three hundred and twenty-five of a total of 514 completed questionnaires and 57 interviews were analysed. Learning goal orientation (LGO), instrumental leadership and supportive leadership related positively to perceived feedback benefits (0.23, 0.2, and 0.31, respectively, p < 0.05). Perceived feedback benefits are related positively to feedback monitoring and inquiry (0.13 and 0.38, respectively, p < 0.05). The personal cost of feedback is unsupported in quantitative data, but was a strong theme in interviews, as was feedback avoidance, peer feedback, and unsupportive learning environment. No differences were observed across sub-groups based on gender, study site, or student nationality. Integrated analysis describes FSB: avoiding 'unsafe' feedback (first, do no harm) and overcoming barriers (beat the system) and goal-centred curation (shop around) to optimise benefits. Diverse medical students across three continents undertake FSB with careful navigation, as a valued but risky business, that is highly contextualised. Promoting a constructive FSB is complex. Overcoming outdated theory and practices on the wards remains a challenge to psychologically safe, learner-centred feedback.

Implementation and Impact of an Adapted Digital Perioperative Competency-Building Tool to Enhance Teaching, Learning And Feedback in Postgraduate Competency-Based Medical Education.

OBJECTIVES: The purpose of this educational intervention was to introduce, iteratively adapt, and implement a digital formative assessment tool in a surgical speciality. The study also evaluated the intervention's impact on perioperative teaching, learning, feedback, and surgical competency. DESIGN: A participatory action research model with a mixed methods approach. SETTING: This study was performed over 10 months in an institutional hospital in South Africa with a general surgery department. PARTICIPANTS: Twelve supervising surgical trainers/faculty and 12 surgical trainees/residents consented to participate in the intervention. RESULTS: The first 4 months of the intervention focused on relationship building, a multi-stakeholder contextual needs assessment and training sessions to support a shared mindset and shift in the teaching and learning culture. The final adapted perioperative competency-building tool comprised a 23-item assessment with four open-text answers (Table 1). Over the following 6-month period, 48 workplace-based competency-building perioperative evaluations were completed. Most trainees took less than 5 minutes to self-assess (67%) before most trainers (67%) took less than 5 minutes to give oral feedback to the trainee after the perioperative supervised learning encounter. On average, the digital tool took 6 minutes to complete during the bidirectional perioperative teaching and learning encounter with no negative impact on the operational flow. All trainers and trainees reported the training and implementation of the digital tool to be beneficial to teaching, learning, feedback, and the development of surgical competency. Analysis of the completed tools revealed several trainees showing evidence of progression in surgical competency for index procedures within the speciality. The focus groups and interviews also showed a change in the teaching and learning culture: more positively framed, frequent, structured, and specific feedback, improved accountability, and trainee-trainer perioperative readiness for teaching. Highlighted changes included the usefulness of trainee self-assessment before perioperative trainer feedback and the tool's value in improving competency to Kirkpatrick Level 4. CONCLUSION: Implementing an adapted digital Workplace-Based Assessment (WBA) tool using a participatory action research model has proven successful in enhancing the effectiveness of supervised perioperative teaching and learning encounters. This approach has improved teaching and feedback practices, facilitated the development of surgical competency, and ultimately impacted the overall culture to Kirkpatrick level 4. Importantly, it has positively influenced the trainee-trainer relationship dynamic. Based on these positive outcomes, we recommend using this effective method and our relationship-centred framework for implementing formative competency-building tools in future studies. By doing so, larger-scale and successful implementation of Competency-Based Medical Education (CBME) could be achieved in various contexts. This approach can potentially enhance teaching and learning encounters, promote competency development, and improve the overall educational experience for surgical trainees and trainers.

Can peer feedback substitute for faculty feedback in predoctoral dental education?

PURPOSE/OBJECTIVES: The ability to give and receive feedback is a key skill to develop during predoctoral dental education, and the use of peer feedback specifically offers distinct benefits including a different understanding of material due to peers' proximity of knowledge development and assisting with overburdened instructors. However, it is unclear if peer feedback offers similar quality to instructor feedback. METHODS: Dental students in two different graduation years provided quantitative and qualitative peer feedback on a case-based oral and maxillofacial pathology simulation. The data from these exercises were aggregated and analyzed to compare the quality of qualitative feedback to course examination scores. Student perceptions of peer feedback were also recorded. RESULTS: The mean quality of feedback was not correlated with course examination scores, though the number of times students gave high-quality feedback and received high-quality feedback was correlated with course examination scores. Student feedback overall had a lower quality than instructor feedback, though there was no significant difference between instructor feedback quality and the maximum student feedback quality received. Student perceptions of the utility of feedback were positive. CONCLUSION: While instructor feedback is more reliable and consistent, our findings suggest that in most instances, at least one peer in moderate-sized groups is able to approximate the quality of instructor feedback on case-based assignments.

Improving feedback literacy in a primary care rotation.

Primary care education is a unique clinical experience for medical students. It is community-based and provides an opportunity for students to learn consultation skills with multiple sources of workplace-based feedback. Meaningful and demonstrable utilisation of this feedback by students remains an educational challenge. We showcase achievable changes to educational tasks in an established curriculum, which aim to improve student feedback literacy and create a feedback loop which improves on previous provision of unidirectional, terminal feedback. The changes have been well-received, with student and educator engagement being positive. Students have demonstrated critical reflection on feedback, and development in consultation and clinical reasoning skills.

How Do Learners Receive Feedback on Note Writing? A Scoping Review.

PURPOSE: The literature assessing the process of note-writing based on gathered information is scant. This scoping review investigates methods of providing feedback on learners' note-writing abilities. METHOD: Scopus and Web of Science were searched for studies that investigated feedback on student notes or reviewed notes written on an information or data-gathering activity in health care and other fields in August 2022. Of 426 articles screened, 23 met the inclusion criteria. Data were extracted on the article title, publication year, study location, study aim, study design, number of participants, participant demographics, level of education, type of note written, field of study, form of feedback given, source of the feedback, and student or participant rating of feedback method from the included articles. Then possible themes were identified and a final consensus-based thematic analysis was performed. RESULTS: Themes identified in the 23 included articles were as follows: (1) learners found faculty and peer feedback beneficial; (2) direct written comments and evaluation tools, such as rubrics or checklists, were the most common feedback methods; (3) reports on notes in real clinical settings were limited (simulated clinical scenarios in preclinical curriculum were the most studied); (4) feedback providers and recipients benefit from having prior training on providing and receiving feedback; (5) sequential or iterative feedback was beneficial for learners but can be time intensive for faculty and confounded by maturation effects; and (6) use of technology and validated assessment tools facilitate the feedback process through ease of communication and improved organization. CONCLUSIONS: The various factors influencing impact and perception of feedback include the source, structure, setting, use of technology, and amount of feedback provided. As the utility of note-writing in health care expands, studies are needed to clarify the value of note feedback in learning and the role of innovative technologies in facilitating note feedback.

Una revisión sistemática del uso de la inteligencia artificial en la educación / A systematic review of the use of artificial intelligence in education

Introducción. El uso de la inteligencia artificial (IA) en la educación ha sido objeto de una creciente atención en los últimos años. La IA se ha utilizado para mejorar la personalización del aprendizaje, la retroalimentación y la evaluación de los estudiantes. Sin embargo, también hay desafíos y limitaciones asociados. El objetivo de este trabajo fue identificar las principales tendencias y áreas de aplicación de la inteligencia artificial en la educación, así como analizar los beneficios y limitaciones de su uso en este ámbito. Métodos. Se llevó a cabo una revisión sistemática que exploró el empleo de la inteligencia artificial en el ámbito educativo. Esta revisión siguió una metodología de investigación basada en la búsqueda de literatura, compuesta por cinco etapas. La investigación se realizó utilizando Scopus como fuente de consulta primaria y se empleó la herramienta VOSviewer para analizar los resultados obtenidos. Resultados. Se encontraron numerosos estudios que investigan el uso de la IA en la educación. Los resultados sugieren que la IA puede mejorar significativamente la personalización del aprendizaje, proporcionando recomendaciones de actividades y retroalimentación adaptadas a las necesidades individuales de cada estudiante. Conclusiones. A pesar de las ventajas del uso de la IA en la educación, también hay desafíos y limitaciones que deben abordarse, como la calidad de los datos utilizados por la IA, la necesidad de capacitación para educadores y estudiantes, y las preocupaciones sobre la privacidad y la seguridad de los datos de los estudiantes. Es importante seguir evaluando los efectos del uso de la IA en la educación para garantizar su uso efectivo y responsable.

Introduction. The use of artificial intelligence (AI) in education has been the subject of increasing attention in recent years. AI has been used to improve personalized learning, feedback, and student assessment. However, there are also challenges and limitations. The aim of this study was to identify the main trends and areas of application of artificial intelligence in education, as well as to analyze the benefits and limitations of its use in this field. Methods. A systematic review was carried out on the use of artificial intelligence in education, using a literature search research methodology with five stages, based on the Scopus query and the tool for analyzing results with VOSviewer. Results. Numerous studies investigating the use of AI in education were found. The results suggest that AI can significantly improve personalized learning by providing activity recommendations and feedback tailored to the individual needs of each student. Conclusions. Despite the advantages of using AI in education, there are also challenges and limitations that need to be addressed, such as the quality of data used by AI, the need for training for educators and students, and concerns about the privacy and security of student data. It is important to continue evaluating the effects of AI use in education to ensure its effective and responsible use.

Effect of an online educational module incorporating real-time feedback on accuracy of polyp sizing in trainees: a randomized controlled trial.

BACKGROUND: Although polyp size dictates surveillance intervals, endoscopists often estimate polyp size inaccurately. We hypothesized that an intervention providing didactic instruction and real-time feedback could significantly improve polyp size classification. METHODS: We conducted a multicenter randomized controlled trial to evaluate the impact of different components of an online educational module on polyp sizing. Participants were randomized to control (no video, no feedback), video only, feedback only, or video + feedback. The primary outcome was accuracy of polyp size classification into clinically relevant categories (diminutive [1-5mm], small [6-9mm], large [≥10mm]). Secondary outcomes included accuracy of exact polyp size (inmm), learning curves, and directionality of inaccuracy (over- vs. underestimation). RESULTS: 36 trainees from five training programs provided 1360 polyp size assessments. The feedback only (80.1%, P=0.01) and video + feedback (78.9%, P=0.02) groups had higher accuracy of polyp size classification compared with controls (71.6%). There was no significant difference in accuracy between the video only group (74.4%) and controls (P=0.42). Groups receiving feedback had higher accuracy of exact polyp size (inmm) and higher peak learning curves. Polyps were more likely to be overestimated than underestimated, and 29.3% of size inaccuracies impacted recommended surveillance intervals. CONCLUSIONS: Our online educational module significantly improved polyp size classification. Real-time feedback appeared to be a critical component in improving accuracy. This scalable and no-cost educational module could significantly decrease under- and overutilization of colonoscopy, improving patient outcomes while increasing colonoscopy access.