Evaluation Tools for Assessing Autonomy of Surgical Residents in the Operating Room and Factors Influencing Access to Autonomy: A Systematic Literature Review.

BACKGROUND: Surgical residents in France lack a clear pedagogical framework for achieving autonomy in the operating room. The progressive acquisition of surgical autonomy is a determining factor in the confidence of operators for their future independent practice. Currently, there is no autonomy scale commonly used in Europe. The objective of this study is to identify existing tools for quantifying the autonomy of residents and the factors that influence it. MATERIALS AND METHODS: We conducted a qualitative systematic review following the recommendations of the Systematic Review Without Meta-Analysis (SWiM) guidelines. Publications were extracted from the MEDLINE (PubMed), EMBASE, and PSYCINFO databases. All publications without date restrictions up to July 2022 were identified. RESULTS: Among the 231 identified publications, 21 met the inclusion criteria. Seventeen publications used a graded autonomy assessment tool by the student and/or the teacher, while 4 used evaluations by an observing third party. We found 8 different autonomy scales, with the Zwisch Scale representing 57.1% of the cases. Factors influencing autonomy were diverse, including the work context, experience, and gender of the resident and their teacher. DISCUSSION: We found heterogeneity in the tools used to "measure" the autonomy of a resident in the operating room. The SIMPL tool or the Zwisch Scale appear to be the most frequently used tools. The relationship between autonomy, performance, confidence, and knowledge may require multidimensional tools that encompass various areas of competence, but this could make their daily application more challenging. The factors influencing autonomy are numerous; and understanding them would improve teaching in the operating room. There is a significant lack of data on surgical autonomy in France, as well as a lack of evaluation in the field of gynecology-obstetrics worldwide.

Residency graduates should not be expected to perform hypospadias repair without fellowship training: Results from a survey of SPU membership.

BACKGROUND: Trainee autonomy has eroded over time as surgery has become more subspecialized and as attending oversight has increased, causing many trainees to seek additional fellowship training beyond residency. Less clear is whether there are cases that attendings view as "fellowship-level" or "privileged" cases in which resident-level trainees should not have high levels of autonomy due to complexity or high-stakes outcomes. OBJECTIVE: We sought to better understand current attitudes and practices with regards to trainee autonomy in hypospadias repair as it represents a high complexity procedure within pediatric urology. STUDY DESIGN: We administered a RedCap survey to the SPU membership, asking respondents to describe the level of autonomy afforded to trainees in various types of hypospadias repair (distal, midshaft, proximal, perineal) as measured by the Zwisch scale. The Zwisch scale describes the role of the attending in the attending-trainee relationship in a low-to-high trainee autonomy fashion: show and tell; active help; passive help; supervision only. RESULTS: 177 of 761 (23%) unique recipients completed our survey and 174 of 177 (98%) of respondents felt that trainees should not perform hypospadias repair independently in practice without additional fellowship training. Among pediatric urologists who train residents, trainee autonomy as measured by the Zwisch scale decreased as the type of hypospadias repair moved from distal to proximal. DISCUSSION: There was near unanimous agreement among respondents that urology trainees should not perform hypospadias repair in practice without additional pediatric urology fellowship training, and that current practice affords little trainee autonomy in hypospadias repair at the resident level. These findings introduce a new wrinkle into the issue of trainee autonomy: cases in which trainees perhaps should not have autonomy. Concurrently, the concern with such findings is that this intentional lack of autonomy may extend to other urologic procedures that one would expect trainees to be able to perform independently. CONCLUSION: Urology trainees are not expected to be able to perform hypospadias in practice without additional training. This raises the question that there may be other such procedures in urology, and if so, should we as instructors, be forthcoming about the limitations of urology residency training to set appropriate trainee expectations?

Assessment of surgical autonomy: in support of aspirational goals.

Drawing on key principles of adult learning theory, a number of surgical autonomy assessment scales have been developed for use in resident evaluation. These assessment scales allow graded autonomy in resident surgical education, balancing patient safety with the need for achieving resident competency during training. The main scales used, the Zwisch scale and the Dreyfus scale, differ only in the inclusion of an "expert" level, and there is controversy in surgical education on whether inclusion of these types of aspirational goals is appropriate. This clinical opinion article reviews key aspects of adult learning theory that pertain to surgical skill acquisition and use of aspirational goals in education, and situates existing surgical autonomy assessment scales within this context. Existing evidence argues for the continued inclusion of aspirational goals in surgical education, but with a concomitant update to the surgical autonomy assessment scales to more closely align with the typical progression of surgical skills during residency. The current process for milestone evaluation put forth by the Accreditation Council for Graduate Medical Education provides an example of a potential framework that could be adapted for use in surgical skill assessment.

Preoperative Briefings and Postoperative Debriefings to Increase Resident Operative Autonomy and Performance.

OBJECTIVE: To implement the use of standardized preoperative briefings and postoperative debriefings for surgical cases involving residents in an effort to improve resident autonomy and skill acquisition. DESIGN: Prospective longitudinal study. SETTING: Johns Hopkins Department of Otolaryngology-Head and Neck Surgery. PARTICIPANTS: Resident and attending physicians. RESULTS: Joint Huddles for Improving Resident Education (JHFIRE) tool was created and successfully implemented by 19 residents and 17 faculty members. Over the course of three data collection periods spanning an academic year, overall scores improved though not statistically significantly in the metrics of Zwisch autonomy, Resident Performance, and Objective Structured Assessment of Technical Skills (OSATS) scores. Female residents were scored significantly higher by attendings than their male counterparts in the assessment of baseline Resident Performance. CONCLUSIONS: (1) JHFIRE tool implemented a standardized preoperative briefing and postoperative debriefing to improve communication and resident skill acquisition; (2) The tool was accepted and utilized throughout an academic year; (3) Zwisch, Resident Performance, and OSATS scores improved though not significantly.

Effect modification of resident autonomy and seniority on perioperative outcomes in laparoscopic cholecystectomy.

BACKGROUND: Resident operative involvement is an integral aspect of general surgery residency training. However, current data examining the effect of resident autonomy on perioperative outcomes remain limited. METHODS: Patient and operator-specific data were collected from 344 adult laparoscopic cholecystectomies at a tertiary academic institution and its regional affiliates between 2018 and 2019. Multivariate modeling compared postoperative outcomes between cases completed with or without resident involvement and its effect modification by resident seniority and autonomy per Zwisch scale. Outcomes include 30-day postoperative complications, hospital readmission rate, and operative time. RESULTS: Multivariate analysis revealed resident involvement in laparoscopic cholecystectomy did not significantly change odds of 30-day postoperative complications (OR 2.52, p = 0.185, 95% CI 0.64-9.92) or hospital readmission (OR 1.61, p = 0.538, 95% CI 0.36-7.23). Operative time is significantly increased compared to faculty-only cases (IRR 1.37, p < 0.001, 95% CI 1.26-1.48). While accounting for case difficulty and resident performance evaluated by SIMPL criteria, stratification by resident autonomy measured by Zwisch scale or seniority reveal no effect modification on 30-day postoperative complications, readmissions, or operative time. The effect of resident involvement on longer relative rates of operative time loses its significance in supervision-only cases (IRR 1.18, p = 0.069, 95% CI 0.99-1.41). CONCLUSION: While resident involvement and autonomy are associated with significantly longer operative times in laparoscopic cholecystectomy, their lack of significant effect on postoperative outcomes argues strongly for continued resident involvement and supervised operative independence.

Gender perception bias of operative autonomy evaluations among residents and faculty in general surgery training.

BACKGROUND: Resident operative autonomy (ROA) is critical and a shared responsibility of both faculty and residents during training. We hypothesize that there is a perception of gender bias in residents' performance as evaluated by faculty and residents. METHOD: Over a period of five academic years, between July 2014 and June 2019, ROA was evaluated using the Zwisch score. Reciprocal evaluations were completed by faculty and residents. RESULTS: 39 surgeons (30 males, 9 females) and 42 residents (25 males, 15 females) completed 2360 evaluations (1180 by faculty, and a matched number by residents). PGY level was significantly associated with granting a higher level of autonomy. Gender of residents didn't affect the level of granted autonomy as evaluated by faculty. However, on self-evaluations, female residents rated their degree of autonomy lower than that of their male counterparts. CONCLUSION: Gender did not influence the perception of autonomy granted as evaluated by faculty. However, on self-evaluations, female residents reported a lower degree of autonomy received.

Mind the Gap: The Autonomy Perception Gap in the Operating Room by Surgical Residents and Faculty.

OBJECTIVE: Examine the concordance of perceived operative autonomy between attendings and resident trainees. DESIGN: Faculty and trainees rated trainee operative autonomy using the 4-level Zwisch scale over a variety of cases and training years. The respective ratings were then compared to explore the effects of experience, gender, case complexity, trainee, trainer, and other covariates to perceived autonomy. SETTING: This study was conducted over 14 general surgery programs in the United States, members of the Procedural Learning and Safety Collaborative. PARTICIPANTS: Participants included faculty and categorical trainees from 14 general surgery programs. RESULTS: A total of 8681 observations was obtained. The sample included 619 unique residents and 457 different attendings. A total of 598 distinct procedures was performed. In 60% of the cases, the autonomy ratings between trainees and attendings were concordant, with only 3.5% of cases discrepant by more than 1 level. An autonomy perception gap was modeled based on the discrepancy between the trainee and attending Zwisch ratings for the same case. The mean Zwisch score expected for a trainee was lower than the attending across all scenarios. Trainees were more likely to perceive relatively more autonomy in the second half of the year. The autonomy perception gap decreased with increasing case complexity. As trainees gained experience, the perception gap increased with trainees underestimating autonomy. CONCLUSIONS: Trainees and attendings generally demonstrated concordance on autonomy perception scores. However, in 40% of cases, a perception gap exists between trainee and attending with the trainee generally underestimating autonomy. The gap worsens as the trainee progresses through residency. This perception gap suggests that attendings and trainees could be better aligned on teaching goals and expectations.

Transferencia de autonomía quirúrgica en apendicectomía abierta y laparoscópica: Un estudio de métodos mixtos desde la perspectiva de residentes y supervisores / Surgical autonomy transfer in open and laparoscopic appendectomy: A study of mixed methods from the perspective of residents and supervisors

Introducción. Existe información limitada sobre el desarrollo apropiado de curvas quirúrgicas de aprendizaje, con altos niveles de autonomía, en residentes de cirugía general en Colombia. El objetivo de este estudio fue caracterizar los niveles de autonomía, para la realización de apendicectomía laparoscópica o abierta en un programa de especialización, desde la perspectiva de médicos residentes y supervisores. Métodos. Estudio de métodos mixtos que se realizó en dos fases. La primera fase incluyó la recolección prospectiva de la información de cada procedimiento (apendicectomía abierta o laparoscópica), realizado entre agosto de 2015 y diciembre de 2018, en la que participaron 29 médicos residentes. Cada residente evaluó su función (cirujano, ayudante), el nivel de supervisión y el nivel de autonomía intraoperatoria mediante la Escala de Zwisch (EZ). En la segunda fase (cualitativa), se realizaron entrevistas a un total de 15 cirujanos generales que supervisaron la práctica de los residentes con preguntas que buscaban explicar los hallazgos cuantitativos. Resultados. Se analizaron 1732 intervenciones: 629 (36 %) se realizaron por vía abierta y 1103 (63 %) por vía laparoscópica. El 81,4 % (n=1411) de los procedimientos fueron realizados en hospitales privados. La percepción global de autonomía reportada por los residentes de acuerdo con la Escala de Zwisch tuvo nivel A 28,9 % (n=500), nivel B 18,1 % (n=313), nivel C 30,4 % (n=526) y nivel D 22,7 % (n=393). El 35,2 % (n=388) de apendicectomías laparoscópicas y el 17,8 % (n=112) por vía abierta fueron realizadas con un nivel A, mientras el 19,5 % (n=215) de apendicectomías laparoscópicas y el 28,2 % (n=178) por vía abierta fueron realizadas con un nivel D. La explicación dada de los hallazgos cuantitativos fue la frecuencia de apendicectomías abiertas en hospitales públicos, aspectos relacionados con la transferencia de autonomía hacia el residente y el aumento progresivo en el nivel de autonomía avanzada entre 2015-2018. Discusión. Se encontró un mayor nivel de autonomía en la realización de apendicectomía por vía abierta compa-rada con la vía laparoscópica, y los niveles de autonomía fueron mayores en hospitales públicos. La explicación a estos hallazgos estuvo relacionada con el contexto clínico y profesional de los residentes

Introduction. There is limited information on the appropriate development of surgical learning curves, with high levels of autonomy, in general surgery residents in Colombia. The objective of this study was to characterize the levels of autonomy for performing laparoscopic or open appendectomy in a specialization program, from the perspective of resident physicians and supervisors. Methods. Study carried out in two phases. The first phase included the prospective collection of information on each procedure (open or laparoscopic appendectomy), performed between August 2015 and December 2018, in which 29 resident physicians participated. Each resident evaluated his/her function (surgeon, assistant), the level of supervision and the level of intraoperative autonomy using the Zwisch Scale (EZ). In the second phase (qualitative), a total of 15 general surgeons were interviewed who supervised the residents' practice with questions that sought to explain the quantitative findings.Results. 1732 interventions were analyzed: 629 (36%) were performed open and 1103 (63%) were performed la-paroscopically. 81.4% (n = 1411) of the procedures were performed in private hospitals. The global perception of autonomy reported by residents according to the Zwisch Scale had level A 28.9% (n = 500), level B 18.1% (n = 313), level C 30.4% (n = 526) and level D 22.7% (n = 393). 35.2% (n = 388) of laparoscopic appendectomies and 17.8% (n = 112) by open approach were performed with a level A, while 19.5% (n = 215) of laparoscopic appendectomies and 28.2% (n = 178) by open approach were performed with a level D. The explanation of the quantitative findings was the frequency of open appendectomies in public hospitals, aspects related to the transfer of autonomy to the resident and the patient. progressive increase in the level of advanced autonomy between 2015-2018.Discusion. A higher level of autonomy was found in performing open appendectomy compared with the lapa-roscopic approach, and levels of autonomy were higher in public hospitals. The explanation for these findings was related to the clinical and professional context of the residents

The Role of Manual Dexterity and Cognitive Functioning in Enhancing Resident Operative Autonomy.

INTRODUCTION: Autonomy, both operative and nonoperative, is one of the most critical aspects of successful surgical training. Both surgeon and resident share the responsibility of achieving this goal. We hypothesize that operative autonomy is distinct and depends, for the most part, on the resident's manual dexterity, knowledge of, and preparation for the procedure. METHODS: Over a period of 4 academic years, between July 2014 and June 2018, a total of 958 Global Rating Scale of Operative Performance evaluations were completed by 32 general and subspecialty faculty surgeons for 35 residents. Elective procedures were evaluated, including 165 (17.2%) by postgraduate year (PGY)1 residents, 253 (26.4%) by PGY2, 199 (20.8%) by PGY3, 147 (15.3%) by PGY4, and 194 (20.3%) by PGY5. The procedures evaluated were: 261 (27.2%) hernia repairs; 178 (18.6%) cholecystectomies; 102 (10.6%) colorectal and anal procedures; 73 (7.6%) vascular procedures; 56 (5.8%) thyroid and parathyroidectomies; 39 (4.1%) foregut (esophagus and stomach) procedures; 38 (4%) skin, soft tissue, and breast; 92 (10%) hepatopancreatic; 20 (2.1%) pediatric procedures; and 99 (10.3%) other procedures including amputations, cardiothoracic, and solid organs procedures. Each resident was scored from 1 to 5 (1 lowest, 5 highest) in each of the following categories of Global Rating Scale of Operative Performance: respect for tissue (RT), time and motion (T&M), instrument handling (IH), knowledge of the instrument (KI), flow of operation (FO) and resident's preparation for the procedure (RP). Resident operative autonomy (ROA) was assessed using the Zwisch scale, a 4-point scale describing faculty supervision behaviors associated with different degrees of resident autonomy (1: Show and Tell, 2: Active Help, 3: Passive Help, and 4: Supervision Only). RESULTS: Correlation and ordinal regression analyses were conducted to examine the relationship between ROA and manual dexterity (RT, T&M, IH, and FO), and cognitive functioning (knowledge of instruments and resident preparation). Results indicated a positive correlation between ROA and RT (r = 0.528, p < 0.001), T&M (r = 0.630, p < 0.001), IH (r = 0.597, p < 0.001), KI (r = 0.490, p < 0.001), FO (r = 0.637, p < 0.001), and RP (r = 0.525, p < 0.001). Additionally, there was a weak inverse correlation between ROA and the number of years the surgeon had been in practice (r = -0.127, p = 0.001). The significant predictors of resident autonomy found by the ordinal logistic regression include time and motion (p < 0.001), flow of operation (p < 0.001), and resident's preparation for the procedure (p < 0.001). CONCLUSIONS: Resident operative autonomy is a product of shared responsibility between the faculty and resident. However, residents' inherent and/or acquired skills and preparation for the operative procedures play a critical role. Residents should be advised to use available resources such as simulation to augment their skills preoperatively and to enhance their autonomy in the operating room.

The Feasibility of Real-Time Intraoperative Performance Assessment With SIMPL (System for Improving and Measuring Procedural Learning): Early Experience From a Multi-institutional Trial.

PURPOSE: Intraoperative performance assessment of residents is of growing interest to trainees, faculty, and accreditors. Current approaches to collect such assessments are limited by low participation rates and long delays between procedure and evaluation. We deployed an innovative, smartphone-based tool, SIMPL (System for Improving and Measuring Procedural Learning), to make real-time intraoperative performance assessment feasible for every case in which surgical trainees participate, and hypothesized that SIMPL could be feasibly integrated into surgical training programs. METHODS: Between September 1, 2015 and February 29, 2016, 15 U.S. general surgery residency programs were enrolled in an institutional review board-approved trial. SIMPL was made available after 70% of faculty and residents completed a 1-hour training session. Descriptive and univariate statistics analyzed multiple dimensions of feasibility, including training rates, volume of assessments, response rates/times, and dictation rates. The 20 most active residents and attendings were evaluated in greater detail. RESULTS: A total of 90% of eligible users (1267/1412) completed training. Further, 13/15 programs began using SIMPL. Totally, 6024 assessments were completed by 254 categorical general surgery residents (n = 3555 assessments) and 259 attendings (n = 2469 assessments), and 3762 unique operations were assessed. There was significant heterogeneity in participation within and between programs. Mean percentage (range) of users who completed ≥1, 5, and 20 assessments were 62% (21%-96%), 34% (5%-75%), and 10% (0%-32%) across all programs, and 96%, 75%, and 32% in the most active program. Overall, response rate was 70%, dictation rate was 24%, and mean response time was 12 hours. Assessments increased from 357 (September 2015) to 1146 (February 2016). The 20 most active residents each received mean 46 assessments by 10 attendings for 20 different procedures. CONCLUSIONS: SIMPL can be feasibly integrated into surgical training programs to enhance the frequency and timeliness of intraoperative performance assessment. We believe SIMPL could help facilitate a national competency-based surgical training system, although local and systemic challenges still need to be addressed.