Utilizing artificial intelligence and electroencephalography to assess expertise on a simulated neurosurgical task.

Virtual reality surgical simulators have facilitated surgical education by providing a safe training environment. Electroencephalography (EEG) has been employed to assess neuroelectric activity during surgical performance. Machine learning (ML) has been applied to analyze EEG data split into frequency bands. Although EEG is widely used in fields requiring expert performance, it has yet been used to classify surgical expertise. Thus, the goals of this study were to (a) develop an ML model to accurately differentiate skilled and less-skilled performance using EEG data recorded during a simulated surgery, (b) explore the relative importance of each EEG bandwidth to expertise, and (c) analyze differences in EEG band powers between skilled and less-skilled individuals. We hypothesized that EEG recordings during a virtual reality surgery task would accurately predict the expertise level of the participant. Twenty-one participants performed three simulated brain tumor resection procedures on the NeuroVR™ platform (CAE Healthcare, Montreal, Canada) while EEG data was recorded. Participants were divided into 2 groups. The skilled group was composed of five neurosurgeons and five senior neurosurgical residents (PGY4-6), and the less-skilled group was composed of six junior residents (PGY1-3) and five medical students. A total of 13 metrics from EEG frequency bands and ratios (e.g., alpha, theta/beta ratio) were generated. Seven ML model types were trained using EEG activity to differentiate between skilled and less-skilled groups. The artificial neural network achieved the highest testing accuracy of 100% (AUROC = 1.0). Model interpretation via Shapley analysis identified low alpha (8-10 Hz) as the most important metric for classifying expertise. Skilled surgeons displayed higher (p = 0.044) low-alpha than the less-skilled group. Furthermore, skilled surgeons displayed significantly lower TBR (p = 0.048) and significantly higher beta (13-30 Hz, p = 0.049), beta 1 (15-18 Hz, p = 0.014), and beta 2 (19-22 Hz, p = 0.015), thus establishing these metrics as important markers of expertise. ACGME CORE COMPETENCIES: Practice-Based Learning and Improvement.

Continuous monitoring of surgical bimanual expertise using deep neural networks in virtual reality simulation.

In procedural-based medicine, the technical ability can be a critical determinant of patient outcomes. Psychomotor performance occurs in real-time, hence a continuous assessment is necessary to provide action-oriented feedback and error avoidance guidance. We outline a deep learning application, the Intelligent Continuous Expertise Monitoring System (ICEMS), to assess surgical bimanual performance at 0.2-s intervals. A long-short term memory network was built using neurosurgeon and student performance in 156 virtually simulated tumor resection tasks. Algorithm predictive ability was tested separately on 144 procedures by scoring the performance of neurosurgical trainees who are at different training stages. The ICEMS successfully differentiated between neurosurgeons, senior trainees, junior trainees, and students. Trainee average performance score correlated with the year of training in neurosurgery. Furthermore, coaching and risk assessment for critical metrics were demonstrated. This work presents a comprehensive technical skill monitoring system with predictive validation throughout surgical residency training, with the ability to detect errors.

The Effect of an Integrated Neurofeedback and Biofeedback Training Intervention on Ice Hockey Shooting Performance.

The purpose of this study was to investigate the effectiveness of a sensorimotor rhythm (SMR) neurofeedback training (NFT) and biofeedback training (BFT) intervention on ice hockey shooting performance. Specifically, the purpose was to examine (a) whether an NFT/BFT program could improve ice hockey shooting performance, (b) whether the implementation of an SMR-NFT intervention leads to neurological adaptations during performance, and (c) whether such neurological changes account for improvement in shooting performance. Using a longitudinal stratified random control design, results demonstrated that while both SMR-NFT/BFT and control groups improved performance, the rate of improvement for the SMR-NFT/BFT group was significantly higher than the control. Participants in the SMR-NFT/BFT group demonstrated the ability to significantly increase SMR power from pre- to postintervention in the lab. However, no significant changes in SMR power were found during shooting performance. This result may be suggestive of differing cortical activity present during motor-skill preparation.

Individual Alpha Peak Frequency in Ice Hockey Shooting Performance.

There are several important inter- and intra-individual variations in individual alpha peak frequency (IAPF) in the cognitive domain. The rationale for the present study was to extend the research on IAPF in the cognitive domain to IAPF in the sport domain. Specifically, the purpose of the present study was twofold: (a) to explore whether baseline IAPF is related to performance in an ice hockey shooting task and (b) to explore whether a shooting task has an effect on IAPF variability. The present investigation is one of the first studies to examine links between IAPF and sport performance. Study results did not show significant changes in IAPF when comparing baseline IAPF and pre- to post-task IAPF across three performance levels. The findings support previous literature in the cognitive domain suggesting that IAPF is a stable neurophysiological marker. Future research should consider the following methodological suggestions: (a) measuring IAPF during sport performance instead of at a resting state, (b) changing the pre-performance resting baseline instructions to take into account sport-specific mental preparation,

Impact of acute stress on psychomotor bimanual performance during a simulated tumor resection task.

OBJECTIVE Severe bleeding during neurosurgical operations can result in acute stress affecting the bimanual psychomotor performance of the operator, leading to surgical error and an adverse patient outcome. Objective methods to assess the influence of acute stress on neurosurgical bimanual psychomotor performance have not been developed. Virtual reality simulators, such as NeuroTouch, allow the testing of acute stress on psychomotor performance in risk-free environments. Thus, the purpose of this study was to explore the impact of a simulated stressful virtual reality tumor resection scenario by utilizing NeuroTouch to answer 2 questions: 1) What is the impact of acute stress on bimanual psychomotor performance during the resection of simulated tumors? 2) Does acute stress influence bimanual psychomotor performance immediately following the stressful episode? METHODS Study participants included 6 neurosurgeons, 6 senior and 6 junior neurosurgical residents, and 6 medical students. Participants resected a total of 6 simulated tumors, 1 of which (Tumor 4) involved uncontrollable "intraoperative" bleeding resulting in simulated cardiac arrest and thus providing the acute stress scenario. Tier 1 metrics included extent of blood loss, percentage of tumor resected, and "normal" brain tissue volume removed. Tier 2 metrics included simulated suction device (sucker) and ultrasonic aspirator total tip path length, as well as the sum and maximum forces applied in using these instruments. Advanced Tier 2 metrics included efficiency index, coordination index, ultrasonic aspirator path length index, and ultrasonic aspirator bimanual forces ratio. All metrics were assessed before, during, and after the stressful scenario. RESULTS The stress scenario caused expected significant increases in blood loss in all participant groups. Extent of tumor resected and brain volume removed decreased in the junior resident and medical student groups. Sucker total tip path length increased in the neurosurgeon group, whereas sucker forces increased in the senior resident group. Psychomotor performance on advanced Tier 2 metrics was altered during the stress scenario in all participant groups. Performance on all advanced Tier 2 metrics returned to pre-stress levels in the post-stress scenario tumor resections. CONCLUSIONS Results demonstrated that acute stress initiated by simulated severe intraoperative bleeding significantly decreases bimanual psychomotor performance during the acute stressful episode. The simulated intraoperative bleeding event had no significant influence on the advanced Tier 2 metrics monitored during the immediate post-stress operative performance.

Bimanual Psychomotor Performance in Neurosurgical Resident Applicants Assessed Using NeuroTouch, a Virtual Reality Simulator.

OBJECTIVE: Current selection methods for neurosurgical residents fail to include objective measurements of bimanual psychomotor performance. Advancements in computer-based simulation provide opportunities to assess cognitive and psychomotor skills in surgically naive populations during complex simulated neurosurgical tasks in risk-free environments. This pilot study was designed to answer 3 questions: (1) What are the differences in bimanual psychomotor performance among neurosurgical residency applicants using NeuroTouch? (2) Are there exceptionally skilled medical students in the applicant cohort? and (3) Is there an influence of previous surgical exposure on surgical performance? DESIGN: Participants were instructed to remove 3 simulated brain tumors with identical visual appearance, stiffness, and random bleeding points. Validated tier 1, tier 2, and advanced tier 2 metrics were used to assess bimanual psychomotor performance. Demographic data included weeks of neurosurgical elective and prior operative exposure. SETTING: This pilot study was carried out at the McGill Neurosurgical Simulation Research and Training Center immediately following neurosurgical residency interviews at McGill University, Montreal, Canada. PARTICIPANTS: All 17 medical students interviewed were asked to participate, of which 16 agreed. RESULTS: Performances were clustered in definable top, middle, and bottom groups with significant differences for all metrics. Increased time spent playing music, increased applicant self-evaluated technical skills, high self-ratings of confidence, and increased skin closures statistically influenced performance on univariate analysis. A trend for both self-rated increased operating room confidence and increased weeks of neurosurgical exposure to increased blood loss was seen in multivariate analysis. CONCLUSIONS: Simulation technology identifies neurosurgical residency applicants with differing levels of technical ability. These results provide information for studies being developed for longitudinal studies on the acquisition, development, and maintenance of psychomotor skills. Technical abilities customized training programs that maximize individual resident bimanual psychomotor training dependant on continuously updated and validated metrics from virtual reality simulation studies should be explored.