Discrimination, Reliability, Sensitivity, and Specificity of Robotic Surgical Proficiency Assessment With Global Evaluative Assessment of Robotic Skills and Binary Scoring Metrics: Results From a Randomized Controlled Trial.

Objective: To compare binary metrics and Global Evaluative Assessment of Robotic Skills (GEARS) evaluations of training outcome assessments for reliability, sensitivity, and specificity. Background: GEARS-Likert-scale skills assessment are a widely accepted tool for robotic surgical training outcome evaluations. Proficiency-based progression (PBP) training is another methodology but uses binary performance metrics for evaluations. Methods: In a prospective, randomized, and blinded study, we compared conventional with PBP training for a robotic suturing, knot-tying anastomosis task. Thirty-six surgical residents from 16 Belgium residency programs were randomized. In the skills laboratory, the PBP group trained until they demonstrated a quantitatively defined proficiency benchmark. The conventional group were yoked to the same training time but without the proficiency requirement. The final trial was video recorded and assessed with binary metrics and GEARS by robotic surgeons blinded to individual, group, and residency program. Sensitivity and specificity of the two assessment methods were evaluated with area under the curve (AUC) and receiver operating characteristics (ROC) curves. Results: The PBP group made 42% fewer objectively assessed performance errors than the conventional group (P < 0.001) and scored 15% better on the GEARS assessment (P = 0.033). The mean interrater reliability for binary metrics and GEARS was 0.87 and 0.38, respectively. Binary total error metrics AUC was 97% and for GEARS 85%. With a sensitivity threshold of 0.8, false positives rates were 3% and 25% for, respectively, the binary and GEARS assessments. Conclusions: Binary metrics for scoring a robotic VUA task demonstrated better psychometric properties than the GEARS assessment.

The Learning Curve for Radical Nephrectomy for Kidney Cancer: Implications for Surgical Training.

Although radical nephrectomy (RN) is the most common treatment for kidney cancer, no data on the learning curve for RN are available. In this study we investigated the effect of surgical experience (EXP) on RN outcomes using data for 1184 patients treated with RN for a cT1-3a cN0 cM0 renal mass. EXP was defined as the total number of RNs performed by each surgeon before the patient's operation. The primary study outcomes were all-cause mortality, clinical progression, Clavien-Dindo grade ≥2 postoperative complications (CD ≥2), and the estimated glomerular filtration rate (eGFR). Secondary outcomes were operative time, estimated blood loss, and length of stay. Multivariable analyses adjusted for case mix revealed no evidence of association between EXP and all-cause mortality (p = 0.7), clinical progression (p = 0.2), CD ≥2 (p = 0.6), or 12-mo eGFR (p = 0.9). Conversely, EXP was associated with shorter operative time (estimate -0.9; p < 0.01). Mortality, cancer control, morbidity, and renal function might not be affected by EXP. The very large cohort examined and the extensive follow-up support the validity of these negative findings. Patient summary: For patients with kidney cancer undergoing surgical removal of a kidney, those treated by novice surgeons have similar clinical outcomes to those treated by experienced surgeons. Thus, this procedure represents a convenient scenario for surgical training if longer operating theatre time can be planned.