Can general surgery interns accurately measure their own technical skills? Analysis of cognitive bias in surgical residents' self-assessments.

BACKGROUND: Accurate self-assessment of knowledge and technical skills is key to self-directed education required in surgical training. We aimed to investigate the presence and magnitude of cognitive bias in self-assessment among a cohort of surgical interns. METHODS: First-year general surgery residents self-assessed performance on a battery of technical skill tasks (knot tying, suturing, vascular anastomosis, Fundamentals of Laparoscopic Skills peg transfer and intracorporeal suturing) at the beginning of residency. Each self-assessment was compared to actual performance. Bias and deviation were defined as arithmetic and absolute difference between actual and estimated scores. Spearman correlation assessed covariation between actual and estimated scores. Improvement in participant performance was analyzed after an end-of-year assessment. RESULTS: Participants (N = 34) completed assessments from 2017 to 2019. Actual and self-assessment scores were positively correlated (0.55, P < .001). Residents generally underestimated performance (bias -4.7 + 8.1). Participants who performed above cohort average tended to assess themselves more negatively (bias -7.3 vs -2.3) and had a larger discrepancy between self and actual scores than below average performers (deviation index 9.7 + 8.2 vs 3.8 + 3.1, P < .05). End-of-year total scores improved in 31 (91.2%) participants by an average of 11 points (90 possible). Least accurate residents in initial self-assessments (deviation indices >75th percentile) improved less than more accurate residents (median 5 vs 16 points, P < .05). All residents with a deviation index >75 percentile underestimated their performance. CONCLUSION: Cognitive bias in technical surgical skills is apparent in first-year surgical residents, particularly in those who are higher performers. Inaccuracy in self-assessment may influence improvement and should be addressed in surgical training.

Does variability among surgical skills diminish throughout surgical internship? Analysis of a 5-task surgical simulation assessment program starting Day 1.

BACKGROUND: Simulation assessments are not yet standardized among surgical programs. We instituted a 5-task simulation program to assess surgical technical skills longitudinally during internship. METHODS: First-year residents completed 5 simulation tasks: suturing, knot-tying, vascular anastomosis, and the peg-transfer and the intracorporeal suturing of the Fundamentals of Laparoscopic Skills. Assessments occurred just before residency, mid-year, and at the completion of the intern year. RESULTS: This study involved 19 residents: 8 categorical, 4 urology, 3 interventional radiology, 2 plastics, and 2 non-designated preliminary interns. Mean completion times improved in both the Fundamentals of Laparoscopic Skills peg-transfer (145 ± 50, 111 ± 47, and 95 ± 28 seconds) and suturing (526 ± 92, 392 ± 131, and 351 ± 158 seconds; each P < .001) tasks, and decreased variability was noted in the former. Total scores trended to improve (P = .013). Interns underwent similar training; 95% completed at least 1 core rotation by mid-year. Surgical specialty was associated with total scores during the first knot-tying session, with plastics residents scoring highest; however, all scores progressed toward the group median over time. CONCLUSION: Technical skills of beginning surgery residents were assessed longitudinally with the institution of a 5-task curriculum. Periodic assessments showed improvement in each task. Furthermore, as residents were exposed to equal surgical training, the variability in resident scores showed the greatest decrease in simpler motor tasks.

Using GoPro to Give Video-Assisted Operative Feedback for Surgery Residents: A Feasibility and Utility Assessment.

OBJECTIVE: As an adjunct to simulation-based teaching, laparoscopic video-based surgical coaching has been an effective tool to augment surgical education. However, the wide use of video review in open surgery has been limited primarily due to technological and logistical challenges. The aims of our study were to (1) evaluate perceptions of general surgery (GS) residents on video-assisted operative instruction and (2) conduct a pilot study using a head-mounted GoPro in conjunction with the operative performance rating system to assess feasibility of providing video review to enhance operative feedback during open procedures. DESIGN: GS residents were anonymously surveyed to evaluate their perceptions of oral and written operative feedback and use of video-based operative resources. We then conducted a pilot study of 10 GS residents to assess the utility and feasibility of using a GoPro to record resident performance of an arteriovenous fistula creation with an attending surgeon. Categorical variables were analyzed using the chi-square test. SETTING: Academic, tertiary medical center. PARTICIPANTS: GS residents and faculty. RESULTS: A total of 59 GS residents were anonymously surveyed (response rate = 65.5%). A total of 40% (n = 24) of residents reported that structured evaluations rarely or never provided meaningful feedback. When feedback was received, 55% (n = 32) residents reported that it was only rarely or sometimes in regard to their operative skills. There was no significant difference in surveyed responses among junior postgraduate year (PGY 1-2), senior (PGY 3-4), or chief residents (PGY-5). A total of 80% (n = 8) of residents found the use of GoPro video review very or extremely useful for education; they also deemed video review more useful for operative feedback than written or communicative feedback. An overwhelming majority (90%, n = 9) felt that video review would lead to improved technical skills, wanted to review the video with the attending surgeon for further feedback, and desired expansion of this tool to include additional procedures. CONCLUSIONS: Although there has been progress toward improving operative feedback, room for further improvement remains. The use of a head-mounted GoPro is a dynamic tool that provides high-quality video for operative review and has the potential to augment the training experience of GS residents. Future studies exploring a wide array of open procedures involving a greater number of trainees will be needed to further define the use of this resource.