Deep neural networks are effective tools for assessing performance during surgical training.

Surgical education courses and certification tests require human evaluators to assess performance. Deep neural network (DNN) methods include techniques for classifying the content of videos which may enable automated scoring of video performance. Researchers collected 254 videos of two simulation-based exercises performed by attending surgeons. The performance in each video was scored by experienced instructors and converted into three class labels-expert, intermediate, and novice. The videos were cut into 2227 10 s clips for training DNNs in the Google Video Intelligence AutoML service. The DNN models matched the classifications applied by human evaluators with 83.1% accuracy for the Ring & Rail exercise and 80.8% for the Suture Sponge exercise. DNN models trained on individual exercises delivered very good results (80 + % accuracy) in matching the classifications assigned by human instructors and may eventually be able to supplement or replace human evaluators.

Design methodology for a simulator of a robotic surgical system.

Traditional spinal surgery procedures are completed with limited direct visualization. This imposes limitations on the surgeon's ability to place screws into the spine. The Mazor Renaissance robotic system was developed to improve the accuracy of pedicle screw insertion. Current training for this device comes with significant constraints. This suggests that a simulation-based solution may be valuable to the current training. This paper describes efforts to apply the theories of human-system integration (HSI) and instructional system design to define the requirements for a design of a simulator for specific robotic surgery system. From this, an instructional plan was conducted, to which an HSI-driven design document for a simulation system was developed. This paper describes the efforts to create a design method for a simulator of a specific robotic surgery system and provides a blended design process, which can be used during the early life cycle of any surgical simulation design.

A model for predicting the GEARS score from virtual reality surgical simulator metrics.

BACKGROUND: Surgical education relies heavily upon simulation. Assessment tools include robotic simulator assessments and Global Evaluative Assessment of Robotic Skills (GEARS) metrics, which have been validated. Training programs use GEARS for proficiency testing; however, it requires a trained human evaluator. Due to limited time, learners are reliant on surgical simulator feedback to improve their skills. GEARS and simulator scores have been shown to be correlated but in what capacity is unknown. Our goal is to develop a model for predicting GEARS score using simulator metrics. METHODS: Linear and multivariate logistic regressions were used on previously reported data by this group. Subjects performed simple (Ring and Rail 1) and complex (Suture Sponge 1) tasks on simulators, the dV-Trainer (dVT) and the da Vinci Skills Simulator (dVSS). They were scored via simulator metrics and GEARS. RESULTS: A linear model for each simulator and exercise showed a positive linear correlation. Equations were developed for predicting GEARS Total Score from simulator Overall Score. Next, the effects of each individual simulator metric on the GEARS Total Score for each simulator and exercise were examined. On the dVSS, Excessive Instrument Force was significant for Ring and Rail 1 and Instrument Collision was significant for Suture Sponge 1. On the dVT, Time to Complete was significant for both exercises. Once the significant variables were identified, multivariate models were generated. Comparing the predicted GEARS Total Score from the linear model (using only simulator Overall Score) to that using the multivariate model (using the significant variables for each simulator and exercise), the results were similar. CONCLUSIONS: Our results suggest that trainees can use simulator Overall Score to predict GEARS Total Score using our linear regression equations. This can improve the training process for those preparing for high-stakes assessments.

A Comparison of Robotic Simulation Performance on Basic Virtual Reality Skills: Simulator Subjective Versus Objective Assessment Tools.

STUDY OBJECTIVE: To answer the question of whether there is a difference between robotic virtual reality simulator performance assessment and validated human reviewers. Current surgical education relies heavily on simulation. Several assessment tools are available to the trainee, including the actual robotic simulator assessment metrics and the Global Evaluative Assessment of Robotic Skills (GEARS) metrics, both of which have been independently validated. GEARS is a rating scale through which human evaluators can score trainees' performances on 6 domains: depth perception, bimanual dexterity, efficiency, force sensitivity, autonomy, and robotic control. Each domain is scored on a 5-point Likert scale with anchors. We used 2 common robotic simulators, the dV-Trainer (dVT; Mimic Technologies Inc., Seattle, WA) and the da Vinci Skills Simulator (dVSS; Intuitive Surgical, Sunnyvale, CA), to compare the performance metrics of robotic surgical simulators with the GEARS for a basic robotic task on each simulator. DESIGN: A prospective single-blinded randomized study. SETTING: A surgical education and training center. PARTICIPANTS: Surgeons and surgeons in training. INTERVENTIONS: Demographic information was collected including sex, age, level of training, specialty, and previous surgical and simulator experience. Subjects performed 2 trials of ring and rail 1 (RR1) on each of the 2 simulators (dVSS and dVT) after undergoing randomization and warm-up exercises. The second RR1 trial simulator performance was recorded, and the deidentified videos were sent to human reviewers using GEARS. Eight different simulator assessment metrics were identified and paired with a similar performance metric in the GEARS tool. The GEARS evaluation scores and simulator assessment scores were paired and a Spearman rho calculated for their level of correlation. MEASUREMENTS AND MAIN RESULTS: Seventy-four subjects were enrolled in this randomized study with 9 subjects excluded for missing or incomplete data. There was a strong correlation between the GEARS score and the simulator metric score for time to complete versus efficiency, time to complete versus total score, economy of motion versus depth perception, and overall score versus total score with rho coefficients greater than or equal to 0.70; these were significant (p < .0001). Those with weak correlation (rho ≥0.30) were bimanual dexterity versus economy of motion, efficiency versus master workspace range, bimanual dexterity versus master workspace range, and robotic control versus instrument collisions. CONCLUSION: On basic VR tasks, several simulator metrics are well matched with GEARS scores assigned by human reviewers, but others are not. Identifying these matches/mismatches can improve the training and assessment process when using robotic surgical simulators.

Minilaparotomy Versus Laparoscopic Myomectomy After Cessation of Power Morcellation: Rate of Wound Complications.

STUDY OBJECTIVE: After the US Food and Drug Administration statement warning against electronic morcellation devices, gynecologic surgeons are performing laparoscopic and robotic myomectomies with minilaparotomy incisions for tissue morcellation and removal. No data exist that focus on the superficial wound complications as a result of these larger incisions. The objective of this study is to compare the rate of wound complications for myomectomy via minilaparotomy versus laparoscopic or robotic myomectomy. DESIGN: Retrospective cohort study (Canadian Task Force classification II-2). SETTING: Kaiser Permanente Northern California, a large integrated healthcare delivery system. PATIENTS: Women > 18 years of age who underwent a myomectomy from either complete laparoscopic or robotic approach (LR) were compared with minilaparotomy myomectomy (MM), comprising complete minilaparotomy (ML) and laparoscopic or robotic assisted by a minilaparotomy for morcellation purposes only (LRM) from January 2011 through December 2014. INTERVENTION: Myomectomy via LR, complete ML, and LRM. MEASUREMENTS AND MAIN RESULTS: Medical records were reviewed for outcomes of interest, including superficial wound complications and surgical and demographic data. After exclusion criteria were met, 405 cases were included in the study; 270 cases were classified as MM, which included ML (n = 224), or LRM (n = 46). One hundred thirty-five cases were classified as LR. Parametric and nonparametric analyses were used to compare the 2 groups. There was no significant difference between the groups insofar as patient morbidity, including the primary outcome of wound complications and other postoperative complications; emergency visits; or readmissions. There were 2 (1.5%) wound complications in the LR group and 7 (2.6%) in the MM group (p = .72). Similarly, there were no significant differences in the subcategories of wound complications, including cellulitis, seroma, hematoma, skin separation, wound infection, or postprocedure wound complication. The distribution of estimated blood loss was significantly different between LR and MM groups with an interquartile range of 50 to 150 mL in the LR group versus 50 to 300 mL in the MM group (p < .01). The MM group experienced a shorter procedure time with a median procedure time of 125 minutes compared with 169.5 minutes in LR surgeries (p < .01). The LR group demonstrated a significantly shorter median length of hospital stay (LR 5.0 hours vs MM 23 hours; p < .01). CONCLUSION: Compared with MM, LR is associated with a shorter length of hospital stay and longer operating time but no reduction in wound complication or other patient morbidity.