Effect of Feedback Modality on Simulated Surgical Skills Learning Using Automated Educational Systems- A Four-Arm Randomized Control Trial.

OBJECTIVE: To explore optimal feedback methodologies to enhance trainee skill acquisition in simulated surgical bimanual skills learning during brain tumor resections. HYPOTHESES: (1) Providing feedback results in better learning outcomes in teaching surgical technical skill when compared to practice alone with no tailored performance feedback. (2) Providing more visual and visuospatial feedback results in better learning outcomes when compared to providing numerical feedback. DESIGN: A prospective 4-parallel-arm randomized controlled trial. SETTING: Neurosurgical Simulation and Artificial Intelligence Learning Centre, McGill University, Canada. PARTICIPANTS: Medical students (n = 120) from 4 Quebec medical schools. RESULTS: Participants completed a virtually simulated tumor resection task 5 times while receiving 1 of 4 feedback based on their group allocation: (1) practice-alone without feedback, (2) numerical feedback, (3) visual feedback, and (4) visuospatial feedback. Outcome measures were participants' scores on 14-performance metrics and the number of expert benchmarks achieved during each task. There were no significant differences in the first task which determined baseline performance. A statistically significant interaction between feedback allocation and task repetition was found on the number of benchmarks achieved, F (10.558, 408.257)=3.220, p < 0.001. Participants in all feedback groups significantly improved their performance compared to baseline. The visual feedback group achieved significantly higher number of benchmarks than the practice-alone group by the third repetition of the task, p = 0.005, 95%CI [0.42 3.25]. Visual feedback and visuospatial feedback improved performance significantly by the second repetition of the task, p = 0.016, 95%CI [0.19 2.71] and p = 0.003, 95%CI [0.4 2.57], respectively. CONCLUSION: Simulations with autonomous visual computer assistance may be effective pedagogical tools in teaching bimanual operative skills via visual and visuospatial feedback information delivery.

Artificial Neural Network Approach to Competency-Based Training Using a Virtual Reality Neurosurgical Simulation.

BACKGROUND: The methodology of assessment and training of surgical skills is evolving to deal with the emergence of competency-based training. Artificial neural networks (ANNs), a branch of artificial intelligence, can use newly generated metrics not only for assessment performance but also to quantitate individual metric importance and provide new insights into surgical expertise. OBJECTIVE: To outline the educational utility of using an ANN in the assessment and quantitation of surgical expertise. A virtual reality vertebral osteophyte removal during a simulated surgical spine procedure is used as a model to outline this methodology. METHODS: Twenty-one participants performed a simulated anterior cervical diskectomy and fusion on the Sim-Ortho virtual reality simulator. Participants were divided into 3 groups, including 9 postresidents, 5 senior residents, and 7 junior residents. Data were retrieved from the osteophyte removal component of the scenario, which involved using a simulated burr. The data were manipulated to initially generate 83 performance metrics spanning 3 categories (safety, efficiency, and motion) of which only the most relevant metrics were used to train and test the ANN. RESULTS: The ANN model was trained on 6 safety metrics to a testing accuracy of 83.3%. The contributions of these performance metrics to expertise were revealed through connection weight products and outlined 2 identifiable learning patterns of technical skills. CONCLUSION: This study outlines the potential utility of ANNs which allows a deeper understanding of the composites of surgical expertise and may contribute to the paradigm shift toward competency-based surgical training.

Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation.

In procedural-based medicine, the technical ability can be a critical determinant of patient outcomes. Psychomotor performance occurs in real-time, hence a continuous assessment is necessary to provide action-oriented feedback and error avoidance guidance. We outline a deep learning application, the Intelligent Continuous Expertise Monitoring System (ICEMS), to assess surgical bimanual performance at 0.2-s intervals. A long-short term memory network was built using neurosurgeon and student performance in 156 virtually simulated tumor resection tasks. Algorithm predictive ability was tested separately on 144 procedures by scoring the performance of neurosurgical trainees who are at different training stages. The ICEMS successfully differentiated between neurosurgeons, senior trainees, junior trainees, and students. Trainee average performance score correlated with the year of training in neurosurgery. Furthermore, coaching and risk assessment for critical metrics were demonstrated. This work presents a comprehensive technical skill monitoring system with predictive validation throughout surgical residency training, with the ability to detect errors.

Quantitation of Tissue Resection Using a Brain Tumor Model and 7-T Magnetic Resonance Imaging Technology.

BACKGROUND: Animal brain tumor models can be useful educational tools for the training of neurosurgical residents in risk-free environments. Magnetic resonance imaging (MRI) technologies have not used these models to quantitate tumor, normal gray and white matter, and total tissue removal during complex neurosurgical procedures. This pilot study was carried out as a proof of concept to show the feasibility of using brain tumor models combined with 7-T MRI technology to quantitatively assess tissue removal during subpial tumor resection. METHODS: Seven ex vivo calf brain hemispheres were used to develop the 7-T MRI segmentation methodology. Three brains were used to quantitate brain tissue removal using 7-T MRI segmentation methodology. Alginate artificial brain tumor was created in 4 calf brains to assess the ability of 7-T MRI segmentation methodology to quantitate tumor and gray and white matter along with total tissue volumes removal during a subpial tumor resection procedure. RESULTS: Quantitative studies showed a correlation between removed brain tissue weights and volumes determined from segmented 7-T MRIs. Analysis of baseline and postresection alginate brain tumor segmented 7-T MRIs allowed quantification of tumor and gray and white matter along with total tissue volumes removed and detection of alterations in surrounding gray and white matter. CONCLUSIONS: This pilot study showed that the use of animal tumor models in combination with 7-T MRI technology provides an opportunity to increase the granularity of data obtained from operative procedures and to improve the assessment and training of learners.

Creating a Comprehensive Research Platform for Surgical Technique and Operative Outcome in Primary Brain Tumor Neurosurgery.

BACKGROUND: The operative environment poses many challenges to studying the relationship between surgical acts and patient outcomes in intracranial oncological neurosurgery. We sought to develop a framework in which neurosurgical performance and extent of resection could be precisely quantified in a controlled setting. METHODS: The stiffness of an alginate hydrogel-based tumor was modified with differing concentrations of the cross-linking agent calcium sulfate until biomechanical properties similar to those of human primary brain tumors measured at resection were achieved. The artificial tumor was subsequently incorporated into an ex-vivo animal brain as a final model. Magnetic resonance imaging enhancement and ultraviolet fluorescence was achieved by incorporating gadolinium and fluorescein solution, respectively. Video recordings from the operative microscope, ceiling cameras, and instrument-mounted fiducial markers within a surgical suite environment captured operative performance. RESULTS: A total of 24 rheometer measurements were conducted on alginate hydrogels containing 10-, 11-, and 12-mM concentrations of calcium sulfate. Sixty-eight stiffness measurements were conducted on eight patient tumor samples. No differences were found between the alginate and brain tumor stiffness values [Kruskal-Wallis χ2(4) = 9.187; P = 0.057]. Tumor was identified using ultraviolet fluorescence and ultrasonography. The volume and location of the resected white and gray matter and residual tumor could be quantified in 0.003-mm3 increments using a 7T magnetic resonance imaging coil. Ultrasonic aspirator and bipolar electrocautery movement data were successfully transformed into performance metrics. CONCLUSION: The developed framework can offer clinicians, learners, and researchers the ability to perform operative rehearsal, teaching, and studies involving brain tumor surgery in a controlled laboratory environment and represents a crucial step in the understanding and training of expertise in neurosurgery.