Early Discharge after Uncomplicated Elective Colectomy and Risk of Postdischarge Complication.

BACKGROUND: This was a retrospective cohort study of adult patients undergoing uncomplicated elective colectomy using the NSQIP database from January 2012 to December 2019. A colectomy is deemed uncomplicated if there are no complications reported during the hospitalization. The objective of this study was to examine the association between discharge timing and postdischarge complications in patients who undergo uncomplicated elective colectomy. STUDY DESIGN: Patients were stratified into an early discharge group if their length of postoperative hospitalization was ≤3 days for laparoscopic or robotic approaches, or ≤5 days for the open approach, and otherwise into delayed discharge groups. The association between early discharge and any postdischarge complication was examined using unadjusted logistic regression after propensity score matching between early and delayed discharge groups. RESULTS: Of the 113,940 patients included, 77,979, 15,877, and 20,084 patients underwent uncomplicated laparoscopic, robotic, and open colectomy, respectively. After propensity score matching, the odds of a postdischarge complication were lower for the early discharge group in laparoscopic (odds ratio 0.73, 95% CI 0.68 to 0.79) and robotic (odds ratio 0.63, 95% CI 0.52 to 0.76) approaches, and not different in the open approach (odds ratio 1.02, 95% CI 0.91 to 1.15). There were no clinically meaningful differences in the risk of return to the operating room for all surgical approaches. CONCLUSIONS: Early discharge after uncomplicated colectomy appears to be safe and is associated with lower odds of postdischarge complications in minimally invasive approaches. Our findings suggest that surgical teams practice sound clinical judgments on selecting patients who benefit from early discharge.

A Proposed Blueprint for Operative Performance Training, Assessment, and Certification.

OBJECTIVE: The aim of this study was to propose an evidence-based blueprint for training, assessment, and certification of operative performance for surgical trainees. SUMMARY BACKGROUND DATA: Operative skill is a critical aspect of surgical performance. High-quality assessment of operative skill therefore has profound implications for training, accreditation, certification, and the public trust of the profession. Current methods of operative skill assessment for surgeons rely heavily on global assessment strategies across a very broad domain of procedures. There is no mechanism to assure technical competence for individual procedures. The science and scalability of operative skill assessment has progressed significantly in recent decades, and can inform a much more meaningful strategy for competency-based assessment of operative skill than has been previously achieved. METHODS: The present article reviews the current status and science of operative skill assessment and proposes a template for competency-based assessment which could be used to update training, accreditation, and certification processes. The proposal is made in reference to general surgery but is more generally applicable to other procedural specialties. RESULTS: Streamlined, routine assessment of every procedure performed by surgical trainees is feasible and would enable a more competency-based educational paradigm. In light of the constraints imposed by both clinical volume and assessment bias, trainees should be expected to become proficient and be measured against a mastery learning standard only for the most important and highest-frequency procedures. For less frequently observed procedures, performance can be compared to a norm-referenced standard and, to provide an overall trajectory of performance, analyzed in aggregate. Key factors in implementing this approach are the number of evaluations, the number of raters, the timeliness of evaluation, and evaluation items. CONCLUSIONS: A competency-based operative skill assessment can be incorporated into surgical training, assessment, and certification. The time has come to develop a systematic approach to this issue as a means of demonstrating professional standards worthy of the public trust.

Operating Room Times For Teaching and Nonteaching Cases are Converging: Less Time for Learning?

OBJECTIVE: To compare differences in operating room (OR) times between teaching and nonteaching cases across calendar years. We hypothesize that time devoted to intraoperative resident education is decreasing, therefore, OR times for teaching and nonteaching cases will be converging. BACKGROUND: Teaching cases take longer than similar nonteaching cases, in part due to intraoperative resident education. Pressures to improve OR efficiency and patient safety may threaten resident education and leave less time for intraoperative learning; however, the magnitude of impact is unknown. SETTING/PARTICIPANTS: National Surgical Quality Improvement Program (NSQIP) deidentified national databases from 2006 to 2012, queried for 30 most common General surgery procedures and case teaching status (i.e., teaching vs. nonteaching cases). DESIGN: The NSQIP database was retrospectively reviewed to identify the 30 most common General Surgery procedures. Teaching cases included all operations in which a resident participated. Multivariable regression analyses were constructed to determine the impact of resident involvement on OR times, controlling for year, resident participation, procedure, and patient demographics and comorbidities. Difference-in-difference analysis was performed to assess OR time differences between teaching and nonteaching cases across calendar years and within subpopulations. RESULTS: A total of 693,223 cases met inclusion criteria. Average overall OR times were 98.89 minutes (teaching) vs. 74.22 minutes (nonteaching), with a difference of 24.67 minutes (95% confidence interval [CI] 24.34-24.99 minutes, p < 0.001). In multivariable analyses, the difference between teaching and nonteaching cases was 21.94 minutes (95% CI = 21.11-22.76) in 2006 and 13.95 minutes (95% CI = 10.62-17.28) in 2012, with a difference-in-difference of 7.99 minutes per case. A similar trend was observed across individual PGYs and several individual procedures. CONCLUSIONS: OR times for teaching and nonteaching cases converged by approximately 8 minutes per general surgery procedure during the 7-year study period, representing a 36% reduction in the difference between groups. We must seek to better understand the source of this convergence, and in doing so ensure to preserve and enhance the intraoperative learning experience of surgical trainees.

The reliability of resident self-evaluation of operative performance.

BACKGROUND: Self-assessment is critical to professional self-regulation yet many trainees may not reliably self-evaluate. We examine the gap between resident and faculty perceptions of trainee operative performance and contributing factors. METHODS: Surgery resident and faculty evaluations of trainee performance were collected from 14 academic institutions using smartphone-based performance assessments. Differences in resident/faculty ratings evaluating the same procedure were analyzed using descriptive statistics and Bayesian mixed models. RESULTS: Of 7382 evaluations, 46% trainees and faculty performance ratings were discrepant (r = 0.47), with 80% residents rating themselves lower than faculty in those cases. This gap existed regardless of case complexity and widened as trainees gained experience. Trainees who overrated themselves had the lowest mean performance scores from faculty. CONCLUSION: Half of residents perceived their performance differently from faculty, and this difference widened for senior residents. Future focus should be to provide opportunity for trainees to improve skills to reliably assess themselves before graduation.

Trainee Autonomy in Minimally Invasive General Surgery in the United States: Establishing a National Benchmark.

OBJECTIVE: Minimally invasive surgery (MIS) is an integral component of General Surgery training and practice. Yet, little is known about how much autonomy General Surgery residents achieve in MIS procedures, and whether that amount is sufficient. This study aims to establish a contemporary benchmark for trainee autonomy in MIS procedures. We hypothesize that trainees achieve progressive autonomy, but fail to achieve meaningful autonomy in a substantial percentage of MIS procedures prior to graduation. SETTING/PARTICIPANTS: Fifty General Surgery residency programs in the United States, from September 1, 2015 to March 19, 2020. All Categorical General Surgery Residents and Attending Surgeons within these programs were eligible. DESIGN: Data were collected prospectively from attending surgeons and categorical General Surgery residents. Trainee autonomy was assessed using the 4-level Zwisch scale (Show and Tell, Active Help, Passive Help, and Supervision Only) on a smartphone application (SIMPL). MIS procedures included all laparoscopic, thoracoscopic, endoscopic, and endovascular/percutaneous procedures performed by residents during the study. Primary outcomes of interest were "meaningful autonomy" rates (i.e., scores in the top 2 categories of the Zwisch scale) by postgraduate year (PGY), and "progressive autonomy" (i.e., differences in autonomy between PGYs) in MIS procedures, as rated by attending surgeons. Primary outcomes were determined with descriptive statistics, one-way analysis of variance (ANOVA) and Z-tests. Secondary analyses compared (i) progressive autonomy between common MIS procedures, and (ii) progressive autonomy in MIS vs. non-MIS procedures. RESULTS: A total of 106,054 evaluations were performed across 50 General Surgery residency programs, of which 38,985 (37%) were for MIS procedures. Attendings performed 44,842 (42%) of all evaluations, including 16,840 (43%) of MIS evaluations, while residents performed the rest. Overall, meaningful autonomy in MIS procedures increased from 14.1% (PGY1s) to 75.9% (PGY5s), with significant (p < 0.001) increases between each PGY level. Meaningful autonomy rates were higher in the MIS vs. non-MIS group [57.2% vs. 48.0%, p < 0.001], and progressed more rapidly in MIS vs. non-MIS, (p < 0.05). The 7 most common MIS procedures accounted for 83.5% (n = 14,058) of all MIS evaluations. Among PGY5s performing these procedures, meaningful autonomy rates (%) were: laparoscopic appendectomy (95%); laparoscopic cholecystectomy (93%); diagnostic laparoscopy (87%); upper/lower endoscopy (85%); laparoscopic hernia repair (72%); laparoscopic partial colectomy (58%); and laparoscopic sleeve gastrectomy (45%). CONCLUSIONS: US General Surgery residents receive progressive autonomy in MIS procedures, and appear to progress more rapidly in MIS versus non-MIS procedures. However, residents fail to achieve meaningful autonomy in nearly 25% of MIS cases in their final year of residency, with higher rates of meaningful autonomy only achieved in a small subset of basic MIS procedures.

Mind the Gap: The Autonomy Perception Gap in the Operating Room by Surgical Residents and Faculty.

OBJECTIVE: Examine the concordance of perceived operative autonomy between attendings and resident trainees. DESIGN: Faculty and trainees rated trainee operative autonomy using the 4-level Zwisch scale over a variety of cases and training years. The respective ratings were then compared to explore the effects of experience, gender, case complexity, trainee, trainer, and other covariates to perceived autonomy. SETTING: This study was conducted over 14 general surgery programs in the United States, members of the Procedural Learning and Safety Collaborative. PARTICIPANTS: Participants included faculty and categorical trainees from 14 general surgery programs. RESULTS: A total of 8681 observations was obtained. The sample included 619 unique residents and 457 different attendings. A total of 598 distinct procedures was performed. In 60% of the cases, the autonomy ratings between trainees and attendings were concordant, with only 3.5% of cases discrepant by more than 1 level. An autonomy perception gap was modeled based on the discrepancy between the trainee and attending Zwisch ratings for the same case. The mean Zwisch score expected for a trainee was lower than the attending across all scenarios. Trainees were more likely to perceive relatively more autonomy in the second half of the year. The autonomy perception gap decreased with increasing case complexity. As trainees gained experience, the perception gap increased with trainees underestimating autonomy. CONCLUSIONS: Trainees and attendings generally demonstrated concordance on autonomy perception scores. However, in 40% of cases, a perception gap exists between trainee and attending with the trainee generally underestimating autonomy. The gap worsens as the trainee progresses through residency. This perception gap suggests that attendings and trainees could be better aligned on teaching goals and expectations.

Concordance Between Expert and Nonexpert Ratings of Condensed Video-Based Trainee Operative Performance Assessment.

OBJECTIVE: We examined the impact of video editing and rater expertise in surgical resident evaluation on operative performance ratings of surgical trainees. DESIGN: Randomized independent review of intraoperative video. SETTING: Operative video was captured at a single, tertiary hospital in Boston, MA. PARTICIPANTS: Six common general surgery procedures were video recorded of 6 attending-trainee dyads. Full-length and condensed versions (n = 12 videos) were then reviewed by 13 independent surgeon raters (5 evaluation experts, 8 nonexperts) using a crossed design. Trainee performance was rated using the Operative Performance Rating Scale, System for Improving and Measuring Procedural Learning (SIMPL) Performance scale, the Zwisch scale, and ten Cate scale. These ratings were then standardized before being compared using Bayesian mixed models with raters and videos treated as random effects. RESULTS: Editing had no effect on the Operative Performance Rating Scale Overall Performance (-0.10, p = 0.30), SIMPL Performance (0.13, p = 0.71), Zwisch (-0.12, p = 0.27), and ten Cate scale (-0.13, p = 0.29). Additionally, rater expertise (evaluation expert vs. nonexpert) had no effect on the same scales (-0.16 (p = 0.32), 0.18 (p = 0.74), 0.25 (p = 0.81), and 0.25 (p = 0.17). CONCLUSIONS: There is little difference in operative performance assessment scores when raters use condensed videos or when raters who are not experts in surgical resident evaluation are used. Future validation studies of operative performance assessment scales may be facilitated by using nonexpert surgeon raters viewing videos condensed using a standardized protocol.

Using smartphones for trainee performance assessment: A SIMPL case study.

Workplace-based assessments are used by raters to evaluate observed performance of trainees in actual clinical practice. These types of assessments are of growing interest, especially because observed performance is prioritized within the larger competency-based medical educational movement. Implementation of workplace-based assessments has, however, been challenging. This article describes the motivations and implications for workplace-based assessments that leverage smartphone technology. It does so in reference to an app called SIMPL (System for Improving and Measuring Procedural Learning) in order to highlight some of the challenges and benefits one might encounter during implementation of similar systems.

Design and Impact of a Novel Surgery-Specific Second Victim Peer Support Program.

BACKGROUND: Surgeons are prone to feelings of sadness, guilt, and anxiety when involved in major adverse events. We aimed to create and evaluate a second victim peer support program for surgeons and surgical trainees. STUDY DESIGN: The second victim peer support program was an intervention performed in the Department of Surgery at a tertiary care academic medical center. Surgical attendings and trainees participated as peer supporters or affected peers. In this article, we describe the design of the program and its 1-year impact, which was evaluated through the number of interventions attempted and realized and feedback received from all participants using an anonymous qualitative and quantitative survey. RESULTS: The program was established using the following 5 steps: creation of a conceptual framework, choice of peer supporters, training of peer supporters, multifaceted identification of major adverse events, and design of a systematic intervention plan. In 1 year, the program had 47 interventions distributed evenly between attendings and trainees; 19% of affected peers opted out of receiving support. Most participants expressed satisfaction with the program's confidentiality, the safe/trusting environment it provided, and the timeliness of the intervention (89%, 73%, and 83%, respectively); 81% suggested that the program had a positive impact on the department's "safety and support" culture and would recommend the program to a colleague. Several areas for improvement were identified, including the need to improve identification of events requiring outreach, and the desire for increased awareness of the program throughout the department. CONCLUSIONS: We successfully designed, implemented, and assessed the impact of the first surgery-specific peer support program in the US. Our 1-year experience suggests that the program is highly used and well received, albeit with opportunities for improvement.

Assessments of Otolaryngology Resident Operative Experiences Using Mobile Technology: A Pilot Study.

OBJECTIVES: Surgical education has shifted from the Halstedian model of "see one, do one, teach one" to a competency-based model of training. Otolaryngology residency programs can benefit from a fast and simple system to assess residents' surgical skills. In this quality initiative, we hypothesize that a novel smartphone application called System for Improving and Measuring Procedural Learning (SIMPL) could be applied in an otolaryngology residency to facilitate the assessment of resident operative experiences. METHODS: The Plan Do Study Act method of quality improvement was used. After researching tools of surgical assessment and trialing SIMPL in a resident-attending pair, we piloted SIMPL across an otolaryngology residency program. Faculty and residents were trained to use SIMPL to rate resident operative performance and autonomy with a previously validated Zwisch Scale. RESULTS: Residents (n = 23) and faculty (n = 17) were trained to use SIMPL using a standardized curriculum. A total of 833 assessments were completed from December 1, 2017, to June 30, 2018. Attendings completed a median 20 assessments, and residents completed a median 14 self-assessments. All evaluations were resident initiated, and attendings had a 78% median response rate. Evaluations took residents a median 22 seconds to complete; 126 unique procedures were logged, representing all 14 key indicator cases for otolaryngology. DISCUSSION: This is the first residency-wide application of a mobile platform to track the operative experiences of otolaryngology residents. IMPLICATIONS FOR PRACTICE: We adapted and implemented a novel assessment tool in a large otolaryngology program. Future multicenter studies will benchmark resident operative experiences nationwide.

The effect of gender on operative autonomy in general surgery residents.

BACKGROUND: Despite an increasing number of women in the field of surgery, bias regarding cognitive or technical ability may continue to affect the experience of female trainees differently than their male counterparts. This study examines the differences in the degree of operative autonomy given to female compared with male general surgery trainees. METHODS: A smartphone app was used to collect evaluations of operative autonomy measured using the 4-point Zwisch scale, which describes defined steps in the progression from novice ("show and tell") to autonomous surgeon ("supervision only"). Differences in autonomy between male and female residents were compared using hierarchical logistic regression analysis. RESULTS: A total of 412 residents and 524 faculty from 14 general surgery training programs evaluated 8,900 cases over a 9-month period. Female residents received less autonomy from faculty than did male residents overall (P < .001). Resident level of training and case complexity were the strongest predictors of autonomy. Even after controlling for potential confounding factors, including level of training, intrinsic procedural difficulty, patient-related case complexity, faculty sex, and training program environment, female residents still received less operative autonomy than their male counterparts. The greatest discrepancy was in the fourth year of training. CONCLUSION: There is a sex-based difference in the autonomy granted to general surgery trainees. This gender gap may affect female residents' experience in training and possibly their preparation for practice. Strategies need to be developed to help faculty and residents work together to overcome this gender gap.

Value and Barriers to Use of the SIMPL Tool for Resident Feedback.

OBJECTIVE: The System for Improving and Measuring Procedural Learning (SIMPL) is a smart-phone application used to provide residents with an evaluation of operative autonomy and feedback. This study investigated the perceived benefits and barriers to app use. DESIGN: A database of previously performed SIMPL evaluations was analyzed to identify high, low, and never users. Potential predisposing factors to use were explored. A survey investigating key areas of value and barriers to use for the SIMPL application was sent to resident and faculty users. Respondents were asked to self-identify how often they used the app. The perceived benefits and barriers were correlated with the level of usage. Qualitative analysis of free text responses was used to determine strategies to increase usage. SETTING: General surgery training programs who are members of the Procedural Learning and Safety Collaborative. PARTICIPANTS: Surgical residents and faculty. RESULTS: At least 1 SIMPL evaluation was created for 411 residents and 524 faculty. Thirty percent of both faculty and residents were high-frequency users. Thirty percent of faculty were never users. One hundred eighty-eight residents and 207 faculty (response rate 46%) completed the survey. High-frequency resident users were more likely to perceive a benefit for both numerical evaluations (76% vs 30%) and dictated feedback (92% vs 30%). Faculty and residents commonly blamed each other for not creating and completing evaluations regularly (87% of residents, 81% of faculty). Suggested strategies to increase usage included reminders and integration with existing data systems. CONTRIBUTIONS: Frequent users perceive value from the application, particularly from dictated feedback and see a positive impact on feedback in their programs. Faculty engagement represents a major barrier to adoption. Mechanisms which automatically remind residents to initiate an evaluation will help improve utilization but programs must work to enhance faculty willingness to respond and dictate feedback.

Tracking operative autonomy and performance in otolaryngology training using smartphone technology: A single institution pilot study.

BACKGROUND: In the era of duty hour restrictions, otolaryngology residents may not gain the operative experience necessary to function autonomously by the end of training. This study quantifies residents' autonomy during key indicator cases, defined by the Accreditation Council for Graduate Medical Education. STUDY DESIGN: Prospective cohort study. METHODS: Faculty and residents at a large academic institution were surveyed on the surgical autonomy trainees should achieve for otolaryngology key indicator surgeries at each training level. Residents and faculty used the mobile application "System for Improving and Measuring Procedural Learning" (SIMPL) between December 2017 and July 2018 to log trainees' operative autonomy during cases on a validated four-level Zwisch scale, from "show and tell" to "supervision only." RESULTS: The study included 40 participants (23 residents and 17 attendings). The survey response rate was 83%. In surveys, residents overestimated the autonomy PGY5 residents should achieve for parotidectomy, rhinoplasty, thyroid/parathyroidectomy, and airway procedures compared with faculty (P < .05). Using SIMPL, 833 evaluations were logged of which 253 were paired evaluations for key indicator cases. Comparing survey predictions with actual cases logged in SIMPL, residents and faculty overestimated the autonomy achieved by senior trainees performing mastoidectomy (PGY5, P < .05) and ethmoidectomy (PGY4/5, P < .05); both felt that senior residents should operate with between "passive help" and "supervision only" whereas residents actually had "passive help." Residents overestimated their autonomy during rhinoplasty (PGY5, P = .017) and parotidectomy (PGY5, P = .007) while attendings accurately expected chief residents to have "passive help." CONCLUSIONS: Resident surgical autonomy varies across otolaryngology procedures. Multicenter studies are needed to elucidate national trends. LEVEL OF EVIDENCE: 2.

How Many Observations are Needed to Assess a Surgical Trainee's State of Operative Competency?

OBJECTIVE: To establish the number of operative performance observations needed for reproducible assessments of operative competency. BACKGROUND: Surgical training is transitioning from a time-based to a competency-based approach, but the number of assessments needed to reliably establish operative competency remains unknown. METHODS: Using a smart phone based operative evaluation application (SIMPL), residents from 13 general surgery training programs were evaluated performing common surgical procedures. Two competency metrics were investigated separately: autonomy and overall performance. Analyses were performed for laparoscopic cholecystectomy performances alone and for all operative procedures combined. Variance component analyses determined operative performance score variance attributable to resident operative competency and measurement error. Generalizability and decision studies determined number of assessments needed to achieve desired reliability (0.80 or greater) and determine standard errors of measurement. RESULTS: For laparoscopic cholecystectomy, 23 ratings are needed to achieve reproducible autonomy ratings and 17 ratings are needed to achieve reproducible overall operative performance ratings. For the undifferentiated mix of procedures, 60 ratings are needed to achieve reproducible autonomy ratings and 40 are needed for reproducible overall operative performance ratings. CONCLUSION: The number of observations needed to achieve reproducible assessments of operative competency far exceeds current certification requirements, yet remains an important and achievable goal. Attention should also be paid to the mix of cases and raters in order to assure fair judgments about operative competency and fair comparisons of trainees.

A SIMPL application to assess oral medicine residents' performance and level of autonomy.

OBJECTIVES: The aim of this study was to evaluate a smartphone application, SIMPL, to optimize faculty guidance and oral medicine residents' performance, and to measure resident-faculty agreement for performance and supervision levels. STUDY DESIGN: Raters used the 5-level "Performance" scale to assess resident's readiness for independent practice and the 4-level Zwisch scale to assess faculty guidance, between June 2016 and June 2017. Junior residents were trainees with less than 18 months of training, and senior residents had more than 18 months of training. Descriptive statistics were used to analyze supervision and autonomy. Performance and supervision agreement was estimated as percentage-agreement and measured using κ and 95% confidence intervals. RESULTS: A total of 660 evaluations were performed by 6 residents and 6 faculty members. Senior residents received higher performance scores compared with junior residents (P for trend < .01). In terms of complexity of cases and level of supervision, there were no significant differences between juniors and seniors (P = .69 and P = .39, respectively). Residents were "practice ready" or greater than for 80% cases. Residents received meaningful autonomy for 64.8% cases. Faculty-resident concordance was 86.1% for performance (with κ of 0.77 [95% CI 66.1%-87.6%]) and 92.4% for supervision (with κ of 0.84 [95% CI 80%-88%]). CONCLUSIONS: SIMPL can feasibly be used for real-time assessment of residents' performance and autonomy.

Is the operative autonomy granted to a resident consistent with operative performance quality.

BACKGROUND: We investigated attending surgeon decisions regarding resident operative autonomy, including situations where operative autonomy was discordant with performance quality. METHODS: Attending surgeons assessed operative performance and documented operative autonomy granted to residents from 14 general surgery residency programs. Concordance between performance and autonomy was defined as "practice ready performance/meaningfully autonomous" or "not practice ready/not meaningfully autonomous." Discordant circumstances were practice ready/not meaningfully autonomous or not practice ready/meaningfully autonomous. Resident training level, patient-related case complexity, procedure complexity, and procedure commonality were investigated to determine impact on autonomy. RESULTS: A total of 8,798 assessments were collected from 429 unique surgeons assessing 496 unique residents. Practice-ready and exceptional performances were 20 times more likely to be performed under meaningfully autonomous conditions than were other performances. Meaningful autonomy occurred most often with high-volume, easy and common cases, and less complex procedures. Eighty percent of assessments were concordant (38% practice ready/meaningfully autonomous and 42% not practice ready/not meaningfully autonomous). Most discordant assessments (13.8%) were not practice ready/meaningfully autonomous. For fifth-year residents, practice ready/not meaningfully autonomous ratings (9.7%) were more frequent than not practice ready/meaningfully autonomous ratings (7.5%). Ten surgeons (2.3%) failed to afford residents meaningful autonomy on any occasion. CONCLUSION: Resident operative performance quality is the most important determinant in attending surgeon decisions regarding resident autonomy.