ABSTRACT
Evaluating a resident's development as a bedside educator in the emergency department (ED) is challenging. Teaching consults, where trainees are observed and assessed in their teaching skills, have been used to improve bedside teaching. Within emergency medicine, there are a few assessment tools to evaluate a clinician's bedside teaching, with the majority focusing on faculty. A user-friendly assessment tool adapted to the ED that emphasizes behaviorally anchored, milestone-based evaluations for residents has yet to be developed. We sought to develop such an assessment tool for evaluating residents' bedside teaching in the ED. Using a nominal-group consensus-building technique, we derived the bedside teaching assessment tool. The consensus-building panel was composed of clinician-educators with extensive experience in resident education. The teaching consult process consisted of the consultant, a faculty member with a focus in medical education, directly observing a resident's bedside teaching throughout their shift while filling out the evaluation form based on observed behaviors. A total of 35 consults were provided to 30 individual residents. The mean (±SD) scores for the 35 consults for the learning climate, content teaching, supervision, feedback and evaluation, and self-assessment were 3.84 (±0.75), 3.56 (±0.58), 3.70 (±0.60), 3.64 (±0.77), and 3.92 (±0.45), respectively. The median scores for the above domains were 4, 3.5, 4, 3.5, and 4, respectively. The tool has acceptable internal consistency with a Cronbach's alpha of 0.723 (95% CI 0.469-0.839). Eleven of 13 (85%) residents who provided feedback agreed or strongly agreed that the quantitative feedback provided by the assessment tool was useful. Twelve of 13 (92%) residents found the consultation process to be unobtrusive to their clinical performance. In conclusion, this novel behaviorally anchored assessment tool for bedside teaching can serve as a useful adjunct to a teaching consult and provide useful feedback for the development of residents' bedside teaching skills.
ABSTRACT
BACKGROUND: Delivering quality lectures is a critical skill for residents seeking careers in academia yet no validated tools for assessing resident lecture skills exist. OBJECTIVES: The authors sought to develop and validate a lecture assessment tool. METHODS: Using a nominal group technique, the authors derived a behaviorally anchored assessment tool. Baseline characteristics of resident lecturers including prior lecturing experience and perceived comfort with lecturing were collected. Faculty and senior residents used the tool to assess lecturer performance at weekly conference. A postintervention survey assessed the usability of the form and the quantity and quality of the feedback. Analysis of variance was used to identify relationships in performance within individual domains to baseline data. Generalizability coefficients and scatterplots with jitter were used to assess inter-rater reliability. RESULTS: Of 64 residents assessed, most (68.8%) had previous lecturing experience and 6.3% had experience as a regional/national speaker. There was a significant difference in performance within the domains of Content Expertise (p < 0.001), Presentation Design/Structure (p = 0.014), and Lecture Presence (p = 0.001) for first-year versus fourth-year residents. Residents who had higher perceived comfort with lecturing performed better in the domains of Content Expertise (p = 0.035), Presentation Design/Structure (p = 0.037), and Lecture Presence (p < 0.001). We found fair agreement between raters in all domains except Goals and Objectives. Both lecturers and evaluators perceived the feedback delivered as specific and of adequate quantity and quality. Evaluators described the form as highly useable. CONCLUSIONS: The derived behaviorally anchored assessment tool is a sufficiently valid instrument for the assessment of resident-delivered lectures.
