Cognitive skills assessment during robot-assisted surgery: separating the wheat from the chaff.

OBJECTIVE: To investigate the utility of cognitive assessment during robot-assisted surgery (RAS) to define skills in terms of cognitive engagement, mental workload, and mental state; while objectively differentiating between novice and expert surgeons. SUBJECTS AND METHODS: In all, 10 surgeons with varying operative experience were assigned to beginner (BG), combined competent and proficient (CPG), and expert (EG) groups based on the Dreyfus model. The participants performed tasks for basic, intermediate and advanced skills on the da Vinci Surgical System. Participant performance was assessed using both tool-based and cognitive metrics. RESULTS: Tool-based metrics showed significant differences between the BG vs CPG and the BG vs EG, in basic skills. While performing intermediate skills, there were significant differences only on the instrument-to-instrument collisions between the BG vs CPG (2.0 vs 0.2, P = 0.028), and the BG vs EG (2.0 vs 0.1, P = 0.018). There were no significant differences between the CPG and EG for both basic and intermediate skills. However, using cognitive metrics, there were significant differences between all groups for the basic and intermediate skills. In advanced skills, there were no significant differences between the CPG and the EG except time (1116 vs 599.6 s), using tool-based metrics. However, cognitive metrics revealed significant differences between both groups. CONCLUSION: Cognitive assessment of surgeons may aid in defining levels of expertise performing complex surgical tasks once competence is achieved. Cognitive assessment may be used as an adjunct to the traditional methods for skill assessment during RAS.

Simulation-based robot-assisted surgical training: a health economic evaluation.

OBJECTIVE: To determine the overall cost effectiveness of surgical skills training on Robotic Surgical Simulator (RoSS). METHODS: This study evaluates the cost analysis of utilizing RoSS for robot-assisted surgical training, at Roswell Park Center for Robotic Surgery. Trainees were queried for time spent on the RoSS console over a period of 1 year, starting from June 2010 to June 2011. Time spent was converted to training time consumed on robotic console, resulting in loss of OR time and revenue. The mechanical durability of the RoSS was also determined. RESULTS: 105 trainees spent 361 h on the RoSS. This duration converted to 73 robot-assisted radical prostatectomy cases, and 72 animal lab sessions. RoSS prevented a potential loss of $600,000, while 72 animal labs would have cost more than $72,000 without including initial robot installation, annual maintenance and personnel expenses. The mechanical durability testing determined breakdown at 180 and 360 h for master control and pinch device, which were repaired under warranty. CONCLUSION: RoSS is a cost effective surgical simulator for implementation of a simulation-based robot-assisted surgical training program.

Development and validation of a composite scoring system for robot-assisted surgical training--the Robotic Skills Assessment Score.

BACKGROUND: A standardized scoring system does not exist in virtual reality-based assessment metrics to describe safe and crucial surgical skills in robot-assisted surgery. This study aims to develop an assessment score along with its construct validation. MATERIALS AND METHODS: All subjects performed key tasks on previously validated Fundamental Skills of Robotic Surgery curriculum, which were recorded, and metrics were stored. After an expert consensus for the purpose of content validation (Delphi), critical safety determining procedural steps were identified from the Fundamental Skills of Robotic Surgery curriculum and a hierarchical task decomposition of multiple parameters using a variety of metrics was used to develop Robotic Skills Assessment Score (RSA-Score). Robotic Skills Assessment mainly focuses on safety in operative field, critical error, economy, bimanual dexterity, and time. Following, the RSA-Score was further evaluated for construct validation and feasibility. Spearman correlation tests performed between tasks using the RSA-Scores indicate no cross correlation. Wilcoxon rank sum tests were performed between the two groups. RESULTS: The proposed RSA-Score was evaluated on non-robotic surgeons (n = 15) and on expert-robotic surgeons (n = 12). The expert group demonstrated significantly better performance on all four tasks in comparison to the novice group. Validation of the RSA-Score in this study was carried out on the Robotic Surgical Simulator. CONCLUSION: The RSA-Score is a valid scoring system that could be incorporated in any virtual reality-based surgical simulator to achieve standardized assessment of fundamental surgical tents during robot-assisted surgery.