Evaluation of different approaches to define expert benchmark scores for new robotic training simulators based on the Medtronic HUGO™ RAS surgical robot experience.

New robot-assisted surgery platforms being developed will be required to have proficiency-based simulation training available. Scoring methodologies and performance feedback for trainees are currently not consistent across all robotic simulator platforms. Also, there are virtually no prior publications on how VR simulation passing benchmarks have been established. This paper compares methods evaluated to determine the proficiency-based scoring thresholds (a.k.a. benchmarks) for the new Medtronic Hugo™ RAS robotic simulator. Nine experienced robotic surgeons from multiple disciplines performed the 49 skills exercises 5 times each. The data were analyzed in 3 different ways: (1) include all data collected, (2) exclude first sessions, (3) exclude outliers. Eliminating the first session discounts becoming familiar with the exercise. Discounting outliers allows removal of potentially erroneous data that may be due to technical issues, unexpected distractions, etc. Outliers were identified using a common statistical technique involving the interquartile range of the data. Using each method above, mean and standard deviations were calculated, and the benchmark was set at a value of 1 standard deviation above the mean. In comparison to including all the data, when outliers are excluded, fewer data points are removed than just excluding first sessions, and the metric benchmarks are made more difficult by an average of 11%. When first sessions are excluded, the metric benchmarks are made easier by an average of about 2%. In comparison with benchmarks calculated using all data points, excluding outliers resulted in the biggest change making the benchmarks more challenging. We determined that this method provided the best representation of the data. These benchmarks should be validated with future clinical training studies.

Proficiency Levels and Validity Evidence for Scoring Metrics for a Virtual Reality and Inanimate Robotic Surgery Simulation Curriculum.

OBJECTIVE: Our institution recently implemented a virtual reality (VR) skills curriculum for general surgery residents using the SimNow simulator. Based on a content alignment study, we revised the curriculum to include only 20 of 33 VR tasks and we added 3 previously validated inanimate tasks. The purpose of this study was to establish expert-derived proficiency levels for all tasks and to evaluate the validity of the scoring for the VR tasks. DESIGN: Two expert robotic surgeons performed 5 repetitions of each VR and inanimate task. The trimmed mean (lowest scoring attempt and outliers [>2 standard deviations] were eliminated) was defined as the expert level for each task. For the VR tasks, expert levels were compared to resident performance to evaluate validity. SETTING: This study was conducted at the University of Texas Southwestern Medical Center (Dallas, TX), a tertiary care academic teaching hospital. PARTICIPANTS: Two expert robotic surgeons participated in this study. The data from 42 residents (PGY2-4) who completed the original curriculum was used to represent novice performance. RESULTS: Comparison of expert levels and resident performance was statistically significant for 15 VR tasks (supporting validity) and approached significance (p = 0.06, 0.09) for 2 VR tasks; expert levels were designated as proficiency levels for these 17 tasks. Group comparisons were clearly not significant (p = 0.2-0.8) for 3 VR tasks; 2 of these 3 tasks were retained as introductory exercises (with 3 repetitions required) and 1 was excluded. For the 3 inanimate tasks, expert levels minus 2 standard deviations were designated as proficiency levels. CONCLUSIONS: This analysis generated validity evidence for 15 VR tasks and established expert-derived proficiency levels for 17 VR tasks and 3 inanimate tasks. Our proposed curriculum now consists of 19 VR and 3 inanimate tasks using the selected proficiency levels. We anticipate that this design will maximize curriculum efficiency and effectiveness.

Developing procedural metrics for training in class II posterior composite resin restorations: Outcomes from an international Delphi panel.

OBJECTIVE: The objective of this study was to develop and operationally define procedural performance metrics for the Class II posterior composite resin restoration procedure and to obtain face and content validity through a consensus meeting. METHODS: A Metrics Group consisting of 4 experienced Consultants in Restorative Dentistry, an experienced member of staff from the Restorative Dentistry department in CUDSH and a senior behavioural scientist and education/training expert deconstructed the performance of the Class II posterior composite resin restoration and defined performance metrics. At a modified Delphi meeting, 20 experts in the field of Restorative Dentistry from 11 different Dental Institutions critiqued these metrics and their operational definitions before reaching consensus. RESULTS: Initially performance metrics consisting of 15 Phases, 45 steps, 42 errors and 34 critical errors were identified that characterize the performance of the Class II posterior resin composite procedure. During the Delphi panel these were modified and consensus was reached on 15 Phases (with a change to the initial sequence) with 46 Steps (1 added, 13 modified), 37 Errors (2 added, 1 deleted, 6 reclassified as Critical Error), and 43 Critical Errors (9 added). Consensus on the resulting metrics was obtained and face and content validity verified. CONCLUSIONS: It is possible to develop and objectively define performance metrics that comprehensively characterize a Class II posterior composite resin restoration. It is also possible to achieve consensus on the metrics from a Delphi panel of experts and to confirm the face and content validity of those procedure metrics.

Development of a proficiency-based training curriculum for beginners in endoscopic endonasal surgery.

OBJECTIVE: Precise endoscopic skills are critical for endoscopic endonasal surgery (EES) to ensure patient safety, as well as to succeed as a surgeon. Among these skills, basic skills such as hand-eye coordination and bimanual dexterity are the most important skills for novice surgeons. However, there is no proficiency-based training for the basic skills in EES. In this study, we aimed to develop a proficiency-based training curriculum for beginners in EES using tractable tasks and to assess the curriculum's validity and educational benefits. METHODS: Training tasks for basic surgical skills of EES were created by experts, using a low-cost and simple simulator. Subsequently, a proficiency-based training curriculum was developed through a preliminary study. The face and content validity of the curriculum was evaluated by expert surgeons. The construct validity was confirmed by comparing the scores of the medical students with those of the experts. In addition, the educational benefits of the curriculum were assessed by comparing the scores of the medical students before and after the implementation of the curriculum. RESULTS: A proficiency-based training curriculum using six tractable tasks was developed in the preliminary study. Replication of real surgical situation, camera navigation, and instrument operability under endoscopy were all highly appreciated (4.6, 4.6, and 4.8 out of 5 points, respectively), which indicates face validity. The content validity of the curriculum was demonstrated by the results of the questionnaire (4.4 out of 5 points). The curriculum was implemented by 12 medical students. The initial student scores were significantly lower than the expert scores (mean score: 61.9 vs. 100.0, p < 0.001), and each coefficient of variation of the student scores was higher than that of the expert scores (p < 0.01), supporting the construct validity of the curriculum. The mean student composite scores were statistically improved after the training (61.9 vs. 93.3, p < 0.0001). In addition, improvement of five other medical students' EES skills through this training curriculum was confirmed in cadaver dissection (1.7 vs. 3.7, p < 0.0001). CONCLUSION: The validity of the proficiency-based training curriculum developed through the preliminary study was established based on the questionnaire of experts and the difference between student and expert scores. In addition, the educational benefits were demonstrated by the students' learning curves. The low-cost and simple simulator seems appropriate as an entry model for beginners in EES, and this curriculum can provide rapid intervention and objective assessment of basic skills in EES.

Training on Congenital 3D Cardiac Models - Will Models Improve Surgical Performance?

Technical skill development in congenital heart surgery (CHS) is challenging due to numerous factors which potentially limit the hands-on operative exposure in surgical training. These challenges have stimulated the growth of simulation-based training through the development of 3D-printed models, providing hands-on surgical training (HOST). From its inception in 2015, the models used in the HOST program have constantly improved, and now include valvar/subvalvar apparatus and better materials that mimic real tissue. Evidence shows that deliberate, regular simulation practice can improve a surgeon's technical skills across the spectrum of CHS. Furthermore, surgical trainees who undergo simulation training are able to translate this improved performance into the operative environment with improved patient outcomes. Despite evidence to support the incorporation of simulation methods into congenital training, its widespread adoption into training curricula remains low. This is due to numerous factors including funding, lack of dedicated time or proctorship and access to models-all of which can be overcome with the newer generation of models and committed trainers. Training programs should consider incorporating simulation-methods as a routine component of congenital training programs.

Objective assessment of intraoperative skills for robot-assisted partial nephrectomy (RAPN).

RAPN training usually takes place in-vivo and methods vary across countries/institutions. No common system exists to objectively assess trainee capacity to perform RAPN at predetermined performance levels prior to in-vivo practice. The identification of objective performance metrics for RAPN training is a crucial starting point to improve training and surgical outcomes. The authors sought to examine the reliability, construct and discriminative validity of objective intraoperative performance metrics which best characterize the optimal and suboptimal performance of a reference approach for training novice RAPN surgeons. Seven Novice and 9 Experienced RAPN surgeons video recorded one or two independently performed RAPN procedures in the human. The videos were anonymized and two experienced urology surgeons were trained to reliably score RAPN performance, using previously developed metrics. The assessors were blinded to the performing surgeon, hospital and surgeon group. They independently scored surgeon RAPN performance. Novice and Experienced group performance scores were compared for procedure steps completed and errors made. Each group was divided at the median for Total Errors score, and subgroup scores (i.e., Novice HiErrs and LoErrs, Experienced HiErrs and LoErrs) were compared. The mean inter-rater reliability (IRR) for scoring was 0.95 (range 0.84-1). Compared with Novices, Experienced RAPN surgeons made 69% fewer procedural Total Errors. This difference was accentuated when the LoErr Expert RAPN surgeon's performance was compared with the HiErrs Novice RAPN surgeon's performance with an observed 170% fewer Total Errors. GEARS showed poor reliability (Mean IRR = 0.44; range 0.0-0.8), for scoring RAPN surgical performance. The RAPN procedure metrics reliably distinguish Novice and Experienced surgeon performances. They further differentiated performance levels within a group with similar experiences. Reliable and valid metrics will underpin quality-assured novice RAPN surgical training.

What are clinically relevant performance metrics in robotic surgery? A systematic review of the literature.

A crucial element of any surgical training program is the ability to provide procedure-specific, objective, and reliable measures of performance. During robotic surgery, objective clinically relevant performance metrics (CRPMs) can provide tailored contextual feedback and correlate with clinical outcomes. This review aims to define CRPMs, assess their validity in robotic surgical training and compare CRPMs to existing measures of robotic performance. A systematic search of Medline and Embase databases was conducted in May 2022 following the PRISMA guidelines. The search terms included Clinically Relevant Performance Metrics (CRPMs) OR Clinically Relevant Outcome Measures (CROMs) AND robotic surgery. The study settings, speciality, operative context, study design, metric details, and validation status were extracted and analysed. The initial search yielded 116 citations, of which 6 were included. Citation searching identified 3 additional studies, resulting in 9 studies included in this review. Metrics were defined as CRPMs, CROMs, proficiency-based performance metrics and reference-procedure metrics which were developed using a modified Delphi methodology. All metrics underwent both contents and construct validation. Two studies found a strong correlation with GEARS but none correlated their metrics with patient outcome data. CRPMs are a validated and objective approach for assessing trainee proficiency. Evaluating CRPMs with other robotic-assessment tools will facilitate a multimodal metric evaluation approach to robotic surgery training. Further studies should assess the correlation with clinical outcomes. This review highlights there is significant scope for the development and validation of CRPMs to establish proficiency-based progression curricula that can be translated from a simulation setting into clinical practice.

International Expert Consensus on Metric-based Characterization of Robot-assisted Partial Nephrectomy.

BACKGROUND: Robot-assisted partial nephrectomy (RAPN) training usually takes place in vivo, and methods vary across countries/institutions. No common system exists to objectively assess trainee ability to perform RAPN at predetermined performance levels prior to in vivo practice. The identification of objective performance metrics for RAPN training is a crucial starting point to improve training and surgical outcomes. OBJECTIVE: We sought to identify objective performance metrics that best characterize a reference approach to RAPN, and obtain face and content validity from procedure experts through a modified Delphi meeting. DESIGN, SETTING, AND PARTICIPANTS: During a series of online meetings, a core metrics team of three RAPN experts and a senior behavioral scientist performed a detailed task deconstruction of a transperitoneal left-sided RAPN procedure. OUTCOME MEASUREMENTS AND STATISTICAL ANALYSIS: Based on published guidelines, manufacturers' instructions, and unedited videos of RAPN, the team identified performance metrics that constitute an optimal approach for training purposes. The metrics were then subjected to an in-person modified international Delphi panel meeting with 19 expert surgeons. RESULTS AND LIMITATIONS: Eleven procedure phases, with 64 procedure steps, 43 errors, and 39 critical errors, were identified. After the modified Delphi process, the international expert panel added 13 metrics (two steps), six were deleted, and three were modified; 100% panel consensus on the resulting metrics was obtained. Limitations are that the metrics are applicable only to left-sided RAPN cases and some might have been excluded. CONCLUSIONS: Performance metrics that accurately characterize RAPN procedure were developed by a core group of experts. The metrics were then presented to and endorsed by an international panel of very experienced peers. Reliable and valid metrics underpin effective, quality-assured, structured surgical training for RAPN. PATIENT SUMMARY: We organize a meeting among robot-assisted partial nephrectomy (RAPN) experts to identify and reach consensus on objective performance metrics for RAPN training. The metrics are a crucial starting point to improve and quality assure surgical training and patients' clinical outcomes.

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients.

Background: Transurethral resection of bladder tumours (TURBT) is the initial diagnostic treatment for patients with bladder cancer. TURBT is not an easy procedure to master and simulator training may play a role in improving the learning curve. Objective: To implement a national training programme for simulation-based mastery learning in TURBT and explore operating theatre performance after training. Design setting and participants: From June 2019 to March 2021, 31 doctors at urology departments in Denmark performed two pretraining TURBT procedures on patients, followed by proficiency-based mastery learning on a virtual reality simulator and then two post-training TURBTs on patients. Outcome measurements and statistical analyses: Operating theatre performances were video-recorded and assessed by two independent, blinded raters using the Objective Structured Assessment for Transurethral Resection of Bladder Tumours Skills (OSATURBS) assessment tool. Paired-sample t tests were used to compare pretraining and post-training analyses and independent t tests for between-group comparisons. This trial is registered at ClinicalTrials.gov as NCT03864302. Results and limitations: Before training, novices had significantly lower performance scores in comparison to those with intermediate experience (p = 0.017) and experienced doctors (p < 0.001). After training, novices significantly improved their clinical performance score (from 11.4 to 17.1; p = 0.049, n = 10). Those with intermediate experience and experienced doctors did not benefit significantly from simulator training (p = 0.9 and p = 0.8, respectively). Conclusions: Novices improved their TURBT performance in the operating theatre after completing a proficiency-based training programme on a virtual reality simulator. Patient summary: We trained surgeons in an operation to remove bladder tumours using a virtual reality simulator. Novice doctors improved their performance significantly after the training, but the training effects for more experienced doctors were minimal. Therefore, we suggest the introduction of mandatory simulator training in the residency programme for urologists.

Transfer of virtual reality endoscopy training to live animal colonoscopy: a randomized control trial of proficiency vs. repetition-based training.

BACKGROUND: Low first-time pass rates of the Fundamentals of Endoscopic Surgery (FES) exam stimulated development of virtual reality (VR) simulation curricula for test preparation. This study evaluates the transfer of VR endoscopy training to live porcine endoscopy performance and compares the relative effectiveness of a proficiency-based vs repetition-based VR training curriculum. METHODS: Novice endoscopists completed pretesting including the FES manual skills examination and Global Assessment of GI Endoscopic Skills (GAGES) assessment of porcine upper and lower endoscopy. Participants were randomly assigned one of two curricula: proficiency-based or repetition-based. Following curriculum completion, participants post-tested via repeat FES examination and GAGES porcine endoscopy assessments. The two cohorts pre-to-post-test differences were compared using ANCOVA. RESULTS: Twenty-two residents completed the curricula. There were no differences in demographics or clinical endoscopy experience between the groups. The repetition group spent significantly more time on the simulator (repetition: 242.2 min, SD 48.6) compared to the proficiency group (proficiency: 170.0 min, SD 66.3; p = 0.013). There was a significant improvement in porcine endoscopy (pre: 10.6, SD 2.8, post: 16.6, SD 3.4; p < 0.001) and colonoscopy (pre: 10.4, SD 2.7, post: 16.4, SD 4.2; p < 0.001) GAGES scores as well as FES manual skills performance (pre: 270.9, SD 105.5, post: 477.4, SD 68.9; p < 0.001) for the total cohort. There was no difference in post-test GAGES performance or FES manual skills exam performance between the two groups. Both the proficiency and repetition group had a 100% pass rate on the FES skills exam following VR curriculum completion. CONCLUSION: A VR endoscopy curriculum translates to improved performance in upper and lower endoscopy in a live animal model. VR curricula type did not affect FES manual skills examination or live colonoscopy outcomes; however, a proficiency curriculum is less time-consuming and can provide a structured approach to prepare for both the FES exam and clinical endoscopy.

Remote training in laparoscopy: a randomized trial comparing home-based self-regulated training to centralized instructor-regulated training.

BACKGROUND: Simulation-based surgical training (SBST) is key to securing future surgical expertise. Proficiency-based training (PBT) in laparoscopy has shown promising results on skills transfer. However, time constraints and limited possibilities for distributed training constitute barriers to effective PBT. Home-based training may provide a solution to these barriers and may be a feasible alternative to centralized training in times of assembly constraints. METHODS: We randomly assigned first-year trainees in abdominal surgery, gynecology, and urology to either centralized instructor-regulated training (CIRT) or home-based self-regulated training (HSRT) in laparoscopy. All participants trained on portable box trainers providing feedback on metrics and possibility for video reviewing. Training in both groups was structured as PBT with graded proficiency exercises adopted from the Fundamentals of Laparoscopic Surgery (FLS). The HSRT group trained at home guided by online learning materials, while the CIRT group attended two training sessions in the simulation center with feedback from experienced instructors. Performance tests consisted of hand-eye and bimanual coordination, suture and knot-tying, and FLS exercises. We analyzed passing rates, training time and distribution, and test performances. RESULTS: Passing rates were 87% and 96% in the CIRT and HSRT group, respectively. HSRT facilitated distributed training and resulted in greater variation in training times. Task times for hand-eye and bimanual coordination were significantly reduced between pretest and posttest in both groups. Trainees maintained their posttest performances at the 6-month retention test. Our analyses revealed no significant inter-group differences in performances at pretest, posttest, or retention test. Performance improvements in the two groups followed similar patterns. CONCLUSION: CIRT and HSRT in laparoscopy result in comparable performance improvements. HSRT in laparoscopy is a feasible and effective alternative to CIRT when offered inside a supportive instructional design. Further research is needed to clarify trainees' preferences and explore facilitators and barriers to HSRT.

Demonstrating the effectiveness of the fundamentals of robotic surgery (FRS) curriculum on the RobotiX Mentor Virtual Reality Simulation Platform.

Fundamentals of robotic surgery (FRS) is a proficiency-based progression curriculum developed by robotic surgery experts from multiple specialty areas to address gaps in existing robotic surgery training curricula. The RobotiX Mentor is a virtual reality training platform for robotic surgery. Our aims were to determine if robotic surgery novices would demonstrate improved technical skills after completing FRS training on the RobotiX Mentor, and to compare the effectiveness of FRS across training platforms. An observational, pre-post design, multi-institutional rater-blinded trial was conducted at two American College of Surgeons Accredited Education Institutes-certified simulation centers. Robotic surgery novices (n = 20) were enrolled and trained to expert-derived benchmarks using FRS on the RobotiX Mentor. Participants' baseline skill was assessed before (pre-test) and after (post-test) training on an avian tissue model. Tests were video recorded and graded by blinded raters using the Global Evaluative Assessment of Robotic Skills (GEARS) and a 32-criteria psychomotor checklist. Post hoc comparisons were conducted against previously published comparator groups. On paired-samples T tests, participants demonstrated improved performance across all GEARS domains (p < 0.001 to p = 0.01) and for time (p < 0.001) and errors (p = 0.003) as measured by psychometric checklist. By ANOVA, improvement in novices' skill after FRS training on the RobotiX Mentor was not inferior to improvement reported after FRS training on previously published platforms. Completion of FRS on the RobotiX Mentor resulted in improved robotic surgery skills among novices, proving effectiveness of training. These data provide additional validity evidence for FRS and support use of the RobotiX Mentor for robotic surgery skill acquisition.

Objective assessment of intraoperative skills for robot-assisted radical prostatectomy (RARP): results from the ERUS Scientific and Educational Working Groups Metrics Initiative.

OBJECTIVE: To develop and seek consensus from procedure experts on the metrics that best characterise a reference robot-assisted radical prostatectomy (RARP) and determine if the metrics distinguished between the objectively assessed RARP performance of experienced and novice urologists, as identifying objective performance metrics for surgical training in robotic surgery is imperative for patient safety. MATERIALS AND METHODS: In Study 1, the metrics, i.e. 12 phases of the procedure, 81 steps, 245 errors and 110 critical errors for a reference RARP were developed and then presented to an international Delphi panel of 19 experienced urologists. In Study 2, 12 very experienced surgeons (VES) who had performed >500 RARPs and 12 novice urology surgeons performed a RARP, which was video recorded and assessed by two experienced urologists blinded as to subject and group. Percentage agreement between experienced urologists for the Delphi meeting and Mann-Whitney U- and Kruskal-Wallis tests were used for construct validation of the newly identified RARP metrics. RESULTS: At the Delphi panel, consensus was reached on the appropriateness of the metrics for a reference RARP. In Study 2, the results showed that the VES performed ~4% more procedure steps and made 72% fewer procedure errors than the novices (P = 0.027). Phases VIIa and VIIb (i.e. neurovascular bundle dissection) best discriminated between the VES and novices. LIMITATIONS: VES whose performance was in the bottom half of their group demonstrated considerable error variability and made five-times as many errors as the other half of the group (P = 0.006). CONCLUSIONS: The international Delphi panel reached high-level consensus on the RARP metrics that reliably distinguished between the objectively scored procedure performance of VES and novices. Reliable and valid performance metrics of RARP are imperative for effective and quality assured surgical training.

Assessment of validity evidence for the RobotiX robot assisted surgery simulator on advanced suturing tasks.

BACKGROUND: Robot assisted surgery has expanded considerably in the past years. Compared to conventional open or laparoscopic surgery, virtual reality (VR) training is an essential component in learning robot assisted surgery. However, for tasks to be implemented in a curriculum, the levels of validity should be studied for proficiency-based training. Therefore, this study was aimed to assess the validity evidence of advanced suturing tasks on a robot assisted VR simulator. METHOD: Participants were voluntary recruited and divided in the robotic experienced, laparoscopic experienced or novice group, based on self-reported surgical experience. Subsequently, a questionnaire on a five-point Likert scale was completed to assess the content validity. Three component tasks of complex suturing were performed on the RobotiX simulator (Task1: tilted plane needle transfer, Task: 2 intracorporal suturing, Task 3: anastomosis needle transfer). Accordingly, the outcome of the parameters was used to assess construct validity between robotic experienced and novice participants. Composite scores (0-100) were calculated from the construct parameters and corresponding pass/fail scores with false positive (FP) and false negative (FN) percentages. RESULTS: Fifteen robotic experienced, 26 laparoscopic experienced and 29 novices were recruited. Overall content validity outcomes were scored positively on the realism (mean 3.7), didactic value (mean 4.0) and usability (mean 4.2). Robotic experienced participants significantly outperformed novices and laparoscopic experienced participants on multiple parameters on all three tasks of complex suturing. Parameters showing construct validity mainly consisted of movement parameters, needle precision and task completion time. Calculated composite pass/fail scores between robotic experienced and novice participants resulted for Task 1 in 73/100 (FP 21%, FN 5%), Task 2 in 85/100 (FP 28%, FN 4%) and Task 3 in 64/100 (FP 49%, FN 22%). CONCLUSION: This study assessed the validity evidence on multiple levels of the three studied tasks. The participants score the RobotiX good on the content validity level. The composite pass/fail scores of Tasks 1 and 2 allow for proficiency-based training and could be implemented in a robot assisted surgery training curriculum.

Development and validation of a curriculum for laparoscopic supracervical hysterectomy.

STUDY OBJECTIVE: To develop and validate a three-step curriculum for laparoscopic supracervical hysterectomy (LSH) designed for a busy clinical setting. NETHODS: Single-centre, prospective, cohort study. Twelve eligible gynaecological trainees were included (group 1). The theoretical part (step 1) was a validated multiple-choice test. The practical part (step 2) consisted of five tasks on a virtual reality simulator. The participants had to reach a pre-defined proficiency level before advancing to performing a LSH (step 3). The validation of the curriculum was based on the surgical performance. The surgical procedure was recorded and assessed by two experts using Global Operative Assessment of Laparoscopic Skills (GOALS) and Competence Assessment Tool - Laparoscopic Supracervical Hysterectomy (CAT-LSH). The scores were compared with scores from gynaecological trainees who performed their first LSH without virtual reality simulator training (group 2). RESULTS: Ten trainees completed the curriculum and performed a LSH that was recorded and evaluated. Mean duration of the training period (step 1 and 2) was 57 days (SD 26.0), and mean training time spent on the simulator to reach the pre-set proficiency level was 173 min (SD 49). The mean GOALS score was 18.5 (SD 5.8) in group 1 and 13.6 (SD 3.3) in group 2, p=0.027. The mean CAT-LSH score of the performance of the hysterectomy was 42.1 (SD 6.9) in group 1 and 34.8 (SD 4.3) in group 2, p= 0.009. CONCLUSIONS: Trainees who completed the curriculum appeared to have a higher performance score compared with trainees who did not perform structured training.

Optimizing Assessment of Surgical Knot Tying Skill.

OBJECTIVES: Knot tying is a fundamental surgical skill. Existing knot tying models assess tying efficiency and errors but do not address respect for tissue. Development of a model that assesses tissue displacement during knot tying may provide a good surrogate for respect for tissue, allow detection of expertise, and offer an improved training platform for skill acquisition. This study aimed to develop a novel, low-cost, knot tying board (KTB) that collects objective, automated metrics of knot displacement, and assesses knot displacement by level of surgical expertise. DESIGN: The novel KTB was developed in collaboration between engineering students and surgical educators. Joystick potentiometers were incorporated on 2 parallel rubber tubes to measure vertical and horizontal displacement while tying. Participants used a standardized technique to tie 1- and 2-handed knots. Differences in time and tubing displacement were compared among junior residents (postgraduate year 1-2), senior residents (postgraduate year 3-5), and attending surgeons; p < 0.05 was considered statistically significant. SETTING: This study was conducted at the Indiana University Surgical Skills Center in Indianapolis, IN. PARTICIPANTS: Forty-seven residents and faculty participated in the study (26 juniors, 14 seniors, 7 attendings). RESULTS: KTB development required 100 hours and $70.00. The attending surgeons tied 2-handed knots faster and with significantly more vertical tubing displacement than residents. Senior residents tied knots significantly faster but with similar tubing displacement as juniors. Similar trends were found for 1-handed knots. CONCLUSIONS: A novel, low-cost KTB was developed to measure knot displacement as a surrogate for tissue handling. The new performance metric of vertical knot displacement proved more sensitive in detecting performance differences among groups compared with horizontal knot displacement. This board and its novel metrics may promote the development of robust knot tying skill by residents.

Orsi Consensus Meeting on European Robotic Training (OCERT): Results from the First Multispecialty Consensus Meeting on Training in Robot-assisted Surgery.

To improve patient outcomes in robotic surgery, robotic training and education need to be modernised and augmented. The skills and performance levels of trainees need to be objectively assessed before they operate on real patients. The main goal of the first Orsi Consensus Meeting on European Robotic Training (OCERT) was to establish the opinions of experts from different scientific societies on standardised robotic training pathways and training methodology. After a 2-d consensus conference, 36 experts identified 23 key statements allotted to three themes: training standardisation pathways, validation metrics, and implementation prerequisites and certification. After two rounds of Delphi voting, consensus was obtained for 22 of 23 questions among these three categories. Participants agreed that societies should drive and support the implementation of benchmarked training using validated proficiency-based pathways. All courses should deliver an internationally agreed curriculum with performance standards, be accredited by universities/professional societies, and, trainees should receive a certificate approved by professional societies and/or universities after successful completion of the robotic training courses. This OCERT meeting established a basis for bringing surgical robotic training out of the operating room by seeking input and consensus across surgical specialties for an objective, validated, and standardised training programme with transparent, metric-based training outcomes. PATIENT SUMMARY: The Orsi Consensus Meeting on European Robotic Training (OCERT) is an international, multidisciplinary, Delphi-panel study of scientific societies and experts focused on training in robotic surgery. The panel achieved consensus that standardised international training pathways should be the basis for a structured, validated, replicable, and certified approach to implementation of robotic technology.

Development and Evaluation of a Proficiency-based and Simulation-based Surgical Skills Training for Technical Medicine Students.

This article was migrated. The article was marked as recommended. Objective Surgical graduate training to achieve practice-ready students is needed, yet is often lacking. This study developed and evaluated a proficiency-based, simulation-based course for basic surgical skills at graduate level. Learning outcomes were measured at the level of knowledge and skills and evaluated with a post-course questionnaire after students' clinical rotations. Methods The surgical skills course was anchored to surgical patient flow and covered topics and skills related to pre-, intra-, and post-operative care, including case-based medical reasoning, patient safety, infection management, operating theatre etiquette, scrubbing and donning, instrument handling, local anaesthesia, excision of tissue, and suturing. Students were assessed on knowledge and procedural skills. Results 155 graduate Technical Medicine students from academic years 2015-2016 and 2016-2017 entered this 10-week, 3 ECTS credits graduate Surgical Skills course. Pass rates of the knowledge test were 78%, and 87% for the procedural skill assessment. Graduate students reached proficiency level in a simulation-based basic surgical skills course. Students stated to go with confidence to the operating room and felt competent in performing four basic surgical skills. Conclusion Based on this study, we recommend that proficiency-based training using simulation should be standard in surgical curricula before students are allowed to practice on patients.

Development of proficiency-based knot-tying and suturing curriculum for otolaryngology residents: A pilot study.

OBJECTIVE: Basic surgical skills such as knot-tying and suturing are important for all otolaryngologists, regardless of subspecialty. The present study was undertaken in order to assess basic surgical techniques such as knot-tying and suturing required for novice otolaryngology residents with taking the variety of subspecialties into consideration, and evaluate the impact of a proficiency-based training curriculum based on these techniques. METHODS: A prospective study was performed for developing of proficiency-based knot-tying and suturing curriculum for otolaryngology residents in the third post-graduate year (PGY-3). The proficiency-based training curriculum was developed based on the tasks selected by RAND/UCLA method with expert panel, which is an iterative and anonymous survey used to establish consensus among participants. Expert panelists were selected from various divisions to reflect variety of their subspecialties. PGY-3 residents trained with the developed curriculum that included proctored pre-test, self-training to proficiency, and proctored post-test. Visual analogue scale (VAS) of trainees' overall competence in the operating room was self-assessed by each resident, before and after completing the training curriculum. RESULTS: Nine PGY-3 residents were enrolled as trainees. Eleven experts chosen as panelists had various subspecialty, including 2 from otology, 2 from rhinology, 2 from laryngology, 2 from head and neck surgery, and 3 from general otolaryngology. Seven tasks were selected from RAND/UCLA method and used to develop the curriculum. Trainee scores at pre-test were significantly lower than expert scores for all 7 tasks (p < 0.01) and each coefficient of variation of trainee score was larger than that of expert score (p < 0.05), supporting construct validity. The mean of composite scores between pre-test and post-test had statistical significance (68.6 ± 11.6 vs 95.9 ± 3.6, p < 0.01), documenting substantial improvement after training. Self-assessment VAS was also improved pre- to post-training (1.2 ± 0.9 vs 4.5 ± 1.4, p < 0.01). A follow-up questionnaire showed that trainees felt the educational curriculum to be beneficial. CONCLUSION: In the present study, seven basic technical skills were selected using the RAND/UCLA method and used to create a proficiency-based training curriculum. Our results indicate that this curriculum significantly improves proficiency of basic surgical skills of junior otolaryngology residents.

Implementation and evaluation of a comprehensive proficiency-based curriculum in an advanced, minimally invasive procedure: a multi-institutional Canadian experience.

INTRODUCTION: A comprehensive proficiency-based curriculum for an advanced, minimally invasive procedure was previously developed and shown to be more educationally effective than conventional surgery training. OBJECTIVE: To implement and evaluate this proficiency-based curriculum in 2 academic general surgery residency programs in Canada. SETTING: Two academic, general surgery residency programs at university hospitals in Ontario, Canada. METHODS: An 8-week, proficiency-based curriculum consisted of a didactic component (lectures, small group sessions, assigned readings) and a simulation-based component (proficiency-based training in laparoscopic enteroenterostomy and a simulated operating room crisis scenario). It was offered to postgraduate years 2-5 general surgery residents in 2 academic programs in Canada. Pre- and postcurriculum procedure-specific knowledge and psychomotor skills were assessed using a 25-item knowledge test and a procedure-specific assessment scale. Postcurriculum nontechnical skills were assessed using the Nontechnical Skills for Surgeons scale. Participants' perceptions about the curriculum were assessed using a questionnaire. Direct costs for curriculum implementation were recorded. RESULTS: Twenty-five residents participated in the curriculum across 2 programs. Completion of the curriculum resulted in significant improvement in technical skills (45 [37.5-65] versus 88 [85-93]; P < .01) and demonstration of "acceptable" situational awareness (3 [3-4]), decision-making (3 [3-4]), teamwork and communication (3 [2-4]), and leadership (3 [3-4]) skills. There was no improvement in procedure-specific knowledge (48 [40-64] versus 58 [48-60]; P = .39). Participants perceived all components of the curriculum as educationally valuable, and 96% agreed and/or strongly agreed that this curriculum should continue to be a part of academic curriculum. The average cost of curriculum implementation was $613.05 Canadian dollars per participant. Lack of faculty supervision was the main barrier to implementation with only 65% of participants agreeing and/or strongly agreeing that quantity of faculty supervision was optimal. CONCLUSIONS: A comprehensive proficiency-based curriculum for an advanced, minimally invasive procedure was successfully implemented and evaluated at 2 academic general surgery residency programs in Canada. Adequate faculty preceptor resources are essential for widespread implementation.