Neurosurgical Skills Assessment: Measuring Technical Proficiency in Neurosurgery Residents Through Intraoperative Video Evaluations.

OBJECTIVES: Although technical skills are fundamental in neurosurgery, there is little agreement on how to describe, measure, or compare skills among surgeons. The primary goal of this study was to develop a quantitative grading scale for technical surgical performance that distinguishes operator skill when graded by domain experts (residents, attendings, and nonsurgeons). Scores provided by raters should be highly reliable with respect to scores from other observers. METHODS: Neurosurgery residents were fitted with a head-mounted video camera while performing craniotomies under attending supervision. Seven videos, 1 from each postgraduate year (PGY) level (1-7), were anonymized and scored by 16 attendings, 8 residents, and 7 nonsurgeons using a grading scale. Seven skills were graded: incision, efficiency of instrument use, cauterization, tissue handling, drilling/craniotomy, confidence, and training level. RESULTS: A strong correlation was found between skills score and PGY year (P < 0.001, analysis of variance). Junior residents (PGY 1-3) had significantly lower scores than did senior residents (PGY 4-7, P < 0.001, t test). Significant variation among junior residents was observed, and senior residents' scores were not significantly different from one another. Interrater reliability, measured against other observers, was high (r = 0.581 ± 0.245, Spearman), as was assessment of resident training level (r = 0.583 ± 0.278, Spearman). Both variables were strongly correlated (r = 0.90, Pearson). Attendings, residents, and nonsurgeons did not score differently (P = 0.46, analysis of variance). CONCLUSIONS: Technical skills of neurosurgery residents recorded during craniotomy can be measured with high interrater reliability. Surgeons and nonsurgeons alike readily distinguish different skill levels. This type of assessment could be used to coach residents, to track performance over time, and potentially to compare skill levels. Developing an objective tool to evaluate surgical performance would be useful in several areas of neurosurgery education.

Operator experience determines performance in a simulated computer-based brain tumor resection task.

PURPOSE: Develop measures to differentiate between experienced and inexperienced neurosurgeons in a virtual reality brain surgery simulator environment. METHODS: Medical students (n = 71) and neurosurgery residents (n = 12) completed four simulated Glioblastoma multiforme resections. Simulated surgeries took place over four days with intermittent spacing in between (average time between surgeries of 4.77 ± 0.73 days). The volume of tumor removed (cc), volume of healthy brain removed (cc), and instrument path length (mm) were recorded. Additionally, surgical effectiveness (% tumor removed divided by % healthy brain removed) and efficiency (% tumor removed divided by instrument movement in mm) were calculated. Performance was compared (1) between groups, and (2) for each participant over time to assess the learning curve. In addition, the effect of real-time instruction ("coaching") was assessed with a randomly selected group of medical students. RESULTS: Neurosurgery residents removed less healthy brain, were more effective in removing tumor and sparing healthy brain tissue, required less instrument movement, and were more efficient in removing tumor tissue than medical students. Medical students approached the resident level of performance over serial sessions. Coached medical students showed more conservative surgical behavior, removing both less tumor and less healthy brain. In sum, neurosurgery residents removed more tumor, removed less healthy brain, and required less instrument movement than medical students. Coaching modified medical student performance. CONCLUSIONS: Virtual Reality brain surgery can differentiate operators based on both recent and long-term experience and may be useful in the acquisition and assessment of neurosurgical skills. Coaching alters the learning curve of naïve inexperienced individuals.