Impact of Rater Training on Residents Technical Skill Assessments: A Randomized Trial.

OBJECTIVE: The ACS/APDS Resident Skills Curriculum's Objective Structured Assessment of Technical Skills (OSATS) consists of task-specific checklists and a global rating scale (GRS) completed by raters. Prior work demonstrated a need for rater training. This study evaluates the impact of a rater-training curriculum on scoring discrimination, consistency, and validity for handsewn bowel anastomosis (HBA) and vascular anastomosis (VA). DESIGN/ METHODS: A rater training video model was developed, which included a GRS orientation and anchoring performances representing the range of potential scores. Faculty raters were randomized to rater training or no rater training and were asked to score videos of resident HBA/VA. Consensus scores were assigned to each video using a modified Delphi process (Gold Score). Trained and untrained scores were analyzed for discrimination and score spread and compared to the Gold Score for relative agreement. RESULTS: Eight general and eight vascular surgery faculty were randomized to score 24 HBA/VA videos. Rater training increased rater discrimination and decreased rating scale shrinkage for both VA (mean trained score: 2.83, variance 1.88; mean untrained score: 3.1, variance 1.14, p = 0.007) and HBA (mean trained score: 3.52, variance 1.44; mean untrained score: 3.42, variance 0.96, p = 0.033). On validity analyses, a comparison between each rater group vs Gold Score revealed a moderate training impact for VA, trained κ=0.65 vs untrained κ=0.57 and no impact for HBA, R1 κ = 0.71 vs R2 κ = 0.73. CONCLUSION: A rater-training curriculum improved raters' ability to differentiate performance levels and use a wider range of the scoring scale. However, despite rater training, there was persistent disagreement between faculty GRS scores with no groups reaching the agreement threshold for formative assessment. If technical skill exams are incorporated into high stakes assessments, consensus ratings via a standard setting process are likely a more valid option than individual faculty ratings.

Interprofessional Trauma Team Training: Leveraging Each Specialties' Expertise to Teach Procedural-Based Skills.

OBJECTIVE: The goal of this study was to utilize interprofessional trauma team training to teach procedural-based skills, teamwork, and assess the impact on the procedural comfort and interprofessional collaboration. DESIGN: Interdisciplinary skills sessions were created to focus on chest tube placement and advanced ultrasound techniques. Chest tube sessions were taught by senior general surgery (GS) residents and faculty. Ultrasound sessions were taught by emergency medicine (EM) fellows and faculty. Mock trauma simulations for EM and GS residents and EM nurses, were developed to also focus on improving interprofessional trauma-bay collaboration. Sessions were held throughout the year for 2 consecutive academic years. After completing skills sessions and trauma scenarios, participants were surveyed on skill comfort, session utility, and willingness to collaborate with the other specialty. Likert scale responses were analyzed by specialty cohort and in aggregate. Free-text feedback responses were analyzed for common themes. SETTING: Large, tertiary, urban academic medical center PARTICIPANTS: Forty seven EM residents and 32 GS residents completed instructional chest tube and ultrasound simulations, respectively. Twenty two EM residents, 24 GS residents, and 29 EM nurses participated in interprofessional trauma simulations. RESULTS: For chest tube placement: 71% of EM residents reported feeling uncomfortable with the procedure prior to the session, with 100% reporting improved confidence afterwards. Seventy percent stated the model was realistic. One hundred percent thought it improved their procedural skills. All participants thought it was worthwhile, should be offered again in future years, and planned to incorporate what they learned in their future practice. For the ultrasound sessions: 61% of GS residents felt uncomfortable with the Focused Assessment with Sonography in Trauma prior to the simulation. Ninety four percent reported the improved skill and confidence, and felt the model was realistic. All participants felt sessions were worthwhile, should be offered again, and planned to incorporate what they learned in their future practice. For trauma simulations: 97% of participants felt scenarios were realistic and clinically relevant and planned to incorporate lessons learned in their future clinical practice. All participants thought participation was worthwhile. Ninety seven percent thought it improved their confidence with trauma clinical management and 56% reported it improved their skills. Many participants reported they appreciated learning from the other specialty's perspective, with greater than 95% of all participants reporting improved comfort and willingness to collaborate across disciplines when caring for future trauma patients. All participants requested the simulation sessions continue in future academic years. CONCLUSION: Interprofessional trauma simulation sessions can harness the unique skill sets of different disciplines to teach procedural-based skills and improve interprofessional collaboration within the trauma bay.

Expression of IGF-1 receptor and GH receptor in hepatic tissue of patients with nonalcoholic fatty liver disease and nonalcoholic steatohepatitis.

OBJECTIVE: The GH and IGF-1 axis is a candidate disease-modifying target in nonalcoholic fatty liver disease (NAFLD) given its lipolytic, anti-inflammatory and anti-fibrotic properties. IGF-1 receptor (IGF-1R) and GH receptor (GHR) expression in adult, human hepatic tissue is not well understood across the spectrum of NAFLD severity. Therefore, we sought to investigate hepatic IGF-1R and GHR expression in subjects with NAFLD utilizing gene expression analysis (GEA) and immunohistochemistry (IHC). DESIGN: GEA (n = 318) and IHC (n = 30) cohorts were identified from the Massachusetts General Hospital NAFLD Tissue Repository. GEA subjects were categorized based on histopathology as normal liver histology (NLH), steatosis only (Steatosis), nonalcoholic steatohepatitis (NASH) without fibrosis (NASH F0), and NASH with fibrosis (NASH F1-4) with GEA by the Nanostring nCounter assay. IHC subjects were matched for age, body mass index (BMI), sex, and diabetic status across three groups (n = 10 each): NLH, Steatosis, and NASH with fibrosis (NASH F1-3). IHC for IGF-1R, IGF-1 and GHR was performed on formalin-fixed, paraffin-embedded hepatic tissue samples. RESULTS: IGF-1R gene expression did not differ across NAFLD severity while IGF-1 gene expression decreased with increasing NAFLD severity, including when controlled for BMI and age. GHR expression did not differ by severity of NAFLD based on GEA or IHC. CONCLUSIONS: IGF-1R and GHR expression levels were not significantly different across NAFLD disease severity. However, expression of IGF-1 was lower with increasing severity of NAFLD. Additional research is needed regarding the contribution of the GH/IGF-1 axis to the pathophysiology of NAFLD and NASH.

Transfer of virtual reality endoscopy training to live animal colonoscopy: a randomized control trial of proficiency vs. repetition-based training.

BACKGROUND: Low first-time pass rates of the Fundamentals of Endoscopic Surgery (FES) exam stimulated development of virtual reality (VR) simulation curricula for test preparation. This study evaluates the transfer of VR endoscopy training to live porcine endoscopy performance and compares the relative effectiveness of a proficiency-based vs repetition-based VR training curriculum. METHODS: Novice endoscopists completed pretesting including the FES manual skills examination and Global Assessment of GI Endoscopic Skills (GAGES) assessment of porcine upper and lower endoscopy. Participants were randomly assigned one of two curricula: proficiency-based or repetition-based. Following curriculum completion, participants post-tested via repeat FES examination and GAGES porcine endoscopy assessments. The two cohorts pre-to-post-test differences were compared using ANCOVA. RESULTS: Twenty-two residents completed the curricula. There were no differences in demographics or clinical endoscopy experience between the groups. The repetition group spent significantly more time on the simulator (repetition: 242.2 min, SD 48.6) compared to the proficiency group (proficiency: 170.0 min, SD 66.3; p = 0.013). There was a significant improvement in porcine endoscopy (pre: 10.6, SD 2.8, post: 16.6, SD 3.4; p < 0.001) and colonoscopy (pre: 10.4, SD 2.7, post: 16.4, SD 4.2; p < 0.001) GAGES scores as well as FES manual skills performance (pre: 270.9, SD 105.5, post: 477.4, SD 68.9; p < 0.001) for the total cohort. There was no difference in post-test GAGES performance or FES manual skills exam performance between the two groups. Both the proficiency and repetition group had a 100% pass rate on the FES skills exam following VR curriculum completion. CONCLUSION: A VR endoscopy curriculum translates to improved performance in upper and lower endoscopy in a live animal model. VR curricula type did not affect FES manual skills examination or live colonoscopy outcomes; however, a proficiency curriculum is less time-consuming and can provide a structured approach to prepare for both the FES exam and clinical endoscopy.

Simulation versus reality: what can interprofessional simulation teach us about team dynamics in the trauma bay?

Purpose: Surgical consultation and the joint management of trauma patients is a common scenario in the emergency department. The goal of this study was to utilize interprofessional trauma team training to understand the role of simulation and its impact on the overall culture of trauma-related care. Methods: Interdisciplinary trauma simulation scenarios were completed by 12 groups of emergency medicine residents, general surgery residents, and emergency medicine nurses across two academic years. Following each simulation, a debriefing session was held to reflect on the scenario, focusing on team interactions. Debriefing sessions were audio-recorded, transcribed, deidentified, and independently, inductively coded by two members of the research team. Using the constant comparative method, a codebook was developed and refined until interrater reliability was confirmed with a kappa of > 0.9. Codes were organized into higher level themes. Results: There were 72 participants, including 23 general surgery residents, 19 emergency medicine residents, and 30 emergency medicine nurses. 214 primary codes were collapsed into 29 coding categories, with 6 emerging themes. Pre-trauma bay impact describes how interactions prior to the trauma scenario can impact how team members communicate, trust one another, and ultimately care for the patient. Role and team identity explores the importance of one knowing their individual role in the trauma bay and how it impacts overall team identity. Resource allocation describes the balance of having appropriate resources to efficiently care for patients while not negatively impacting crowd control or role identity. Impact of the simulation experience highlights the impact of the lower stakes simulation scenario on learning and reflection as well as concerns with simulation fidelity. Trauma leader traits and actions outlines inherent traits and learned actions of trauma leaders that impact how the trauma scenario unfolds. Interprofessional team performance describes the overall performance of the trauma team, including but not limited to the type of communication used, teamwork behaviors, and transition of care of the patient. Conclusions: Interdisciplinary trauma simulations and structured debriefing sessions provide insights into team dynamics and interprofessional relationships. Simulations and debriefing sessions can promote understanding, respect, and familiarity of team members' roles; recognition of key characteristics of high functioning leaders and teams; and discovery of conflict mitigating strategies for future interdisciplinary team improvement. Simulation sessions allow implementation of quality improvement measures and communication and leadership strategy practice in a safe, collaborative learning environment. The lessons learned from these sessions can encourage participants to reexamine how they interact and function as a team within the real-life trauma bay.

Surgeons' Coaching Techniques in the Surgical Coaching for Operative Performance Enhancement (SCOPE) Program.

OBJECTIVE: To evaluate coaching techniques used by practicing surgeons who underwent dedicated coach training in a peer surgical coaching program. BACKGROUND: Surgical coaching is a developing strategy for improving surgeons' intraoperative performance. How to cultivate effective coaching skills among practicing surgeons is uncertain. METHODS: Through the Surgical Coaching for Operative Performance Enhancement (SCOPE) program, 46 surgeons within 4 US academic medical centers were assigned 1:1 into coach/coachee pairs. All attended a 3-hour Surgical Coaching Workshop-developed using evidence from the fields of surgery and education-then received weekly reminders. We analyzed workshop evaluations and audio transcripts of postoperative debriefs between coach/coachee pairs, co-coding themes based on established principles of effective coaching: (i) self-identified goals, (ii) collaborative analysis, (iii) constructive feedback, and (iv) action planning. Coaching principles were cross-referenced with intraoperative performance topics: technical, nontechnical, and teaching skills. RESULTS: For the 8 postoperative debriefs analyzed, mean duration was 24.4 min (range 7-47 minutes). Overall, 326 coaching examples were identified, demonstrating application of all 4 core principles of coaching. Constructive feedback (17.6 examples per debrief) and collaborative analysis (16.3) were utilized more frequently than goal-setting (3.9) and action planning (3.0). Debriefs focused more often on nontechnical skills (60%) than technical skills (32%) or teaching-specific skills (8%). Among surgeons who completed the workshop evaluation (82% completion rate), 90% rated the Surgical Coaching Workshop "good" or "excellent." CONCLUSIONS: Short-course coach trainings can help practicing surgeons use effective coaching techniques to guide their peers' performance improvement in a way that aligns with surgical culture.

ADAMTSL2 protein and a soluble biomarker signature identify at-risk non-alcoholic steatohepatitis and fibrosis in adults with NAFLD.

BACKGROUND & AIMS: Identifying fibrosis in non-alcoholic fatty liver disease (NAFLD) is essential to predict liver-related outcomes and guide treatment decisions. A protein-based signature of fibrosis could serve as a valuable, non-invasive diagnostic tool. This study sought to identify circulating proteins associated with fibrosis in NAFLD. METHODS: We used aptamer-based proteomics to measure 4,783 proteins in 2 cohorts (Cohort A and B). Targeted, quantitative assays coupling aptamer-based protein pull down and mass spectrometry (SPMS) validated the profiling results in a bariatric and NAFLD cohort (Cohort C and D, respectively). Generalized linear modeling-logistic regression assessed the ability of candidate proteins to classify fibrosis. RESULTS: From the multiplex profiling, 16 proteins differed significantly by fibrosis in cohorts A (n = 62) and B (n = 98). Quantitative and robust SPMS assays were developed for 8 proteins and validated in Cohorts C (n = 71) and D (n = 84). The A disintegrin and metalloproteinase with thrombospondin motifs like 2 (ADAMTSL2) protein accurately distinguished non-alcoholic fatty liver (NAFL)/non-alcoholic steatohepatitis (NASH) with fibrosis stage 0-1 (F0-1) from at-risk NASH with fibrosis stage 2-4, with AUROCs of 0.83 and 0.86 in Cohorts C and D, respectively, and from NASH with significant fibrosis (F2-3), with AUROCs of 0.80 and 0.83 in Cohorts C and D, respectively. An 8-protein panel distinguished NAFL/NASH F0-1 from at-risk NASH (AUROCs 0.90 and 0.87 in Cohort C and D, respectively) and NASH F2-3 (AUROCs 0.89 and 0.83 in Cohorts C and D, respectively). The 8-protein panel and ADAMTSL2 protein had superior performance to the NAFLD fibrosis score and fibrosis-4 score. CONCLUSION: The ADAMTSL2 protein and an 8-protein soluble biomarker panel are highly associated with at-risk NASH and significant fibrosis; they exhibited superior diagnostic performance compared to standard of care fibrosis scores. LAY SUMMARY: Non-alcoholic fatty liver disease (NAFLD) is one of the most common causes of liver disease worldwide. Diagnosing NAFLD and identifying fibrosis (scarring of the liver) currently requires a liver biopsy. Our study identified novel proteins found in the blood which may identify fibrosis without the need for a liver biopsy.

Distinct Hepatic Gene-Expression Patterns of NAFLD in Patients With Obesity.

Approaches to manage nonalcoholic fatty liver disease (NAFLD) are limited by an incomplete understanding of disease pathogenesis. The aim of this study was to identify hepatic gene-expression patterns associated with different patterns of liver injury in a high-risk cohort of adults with obesity. Using the NanoString Technologies (Seattle, WA) nCounter assay, we quantified expression of 795 genes, hypothesized to be involved in hepatic fibrosis, inflammation, and steatosis, in liver tissue from 318 adults with obesity. Liver specimens were categorized into four distinct NAFLD phenotypes: normal liver histology (NLH), steatosis only (steatosis), nonalcoholic steatohepatitis without fibrosis (NASH F0), and NASH with fibrosis stage 1-4 (NASH F1-F4). One hundred twenty-five genes were significantly increasing or decreasing as NAFLD pathology progressed. Compared with NLH, NASH F0 was characterized by increased inflammatory gene expression, such as gamma-interferon-inducible lysosomal thiol reductase (IFI30) and chemokine (C-X-C motif) ligand 9 (CXCL9), while complement and coagulation related genes, such as C9 and complement component 4 binding protein beta (C4BPB), were reduced. In the presence of NASH F1-F4, extracellular matrix degrading proteinases and profibrotic/scar deposition genes, such as collagens and transforming growth factor beta 1 (TGFB1), were simultaneously increased, suggesting a dynamic state of tissue remodeling. Conclusion: In adults with obesity, distinct states of NAFLD are associated with intrahepatic perturbations in genes related to inflammation, complement and coagulation pathways, and tissue remodeling. These data provide insights into the dynamic pathogenesis of NAFLD in high-risk individuals.

Challenges of virtual interviewing for surgical fellowships: a qualitative analysis of applicant experiences.

BACKGROUND: The COVID-19 pandemic forced surgical fellowship programs to transition from in-person to remote applicant interviews; the virtual interviewing format presented new and unique challenges. We sought to understand applicants' perceived challenges to virtual interviewing for a surgical fellowship program. METHOD: A grounded theory-based qualitative study was performed utilizing semi-structured interviews with fellowship applicants from the 2020 fellowship match. All Fellowship Council-registered applicants were eligible. We purposefully sampled participants to balance across gender, specialty-choice, and academic versus community-program affiliation. Interviews were inductively analyzed by two researchers for prominent themes. RESULTS: Fifteen interviews were conducted. Participants were 60% male (n = 9), with 33% (n = 5) from non-academic institutions. They applied for the following fellowships: Advanced Gastrointestinal/Minimal Invasive (55%), Bariatric (30%), Hepatopancreatobiliary (10%) and Surgical Oncology (5%). Four main themes emerged to describe virtual interview process challenges: (1) perceived data deficiency, (2) superficial personal connections, (3) magnification of non-professionalism, and (4) logistical frustrations. Applicants recommend program directors provide more information about the fellowship prior to interview day and offer informal independent interactions with current and previous fellows. CONCLUSIONS: According to fellowship applicants, virtual interviews resulted in a lack of information for rank-list decision making ultimately requiring them to rely on other information avenues to base their decisions. These applicants have offered advice to fellowship program directors and future applicants to better optimize this process.

A Needs Assessment of Video-based Education Resources Among General Surgery Residents.

BACKGROUND: Video-based education (VBE) is an effective tool for knowledge and skill acquisition for medical students, but its utility is less clear for resident physicians. We sought to determine how to incorporate VBE into a general surgery resident operative curriculum. METHODS: We conducted a single-institution, survey-based needs assessment of general surgery residents to determine desired content and format of an operative VBE module. RESULTS: The response rate was 84% (53/63), with 66% senior (postgraduate year ≥3) resident respondents. VBE was the most commonly cited resource that residents used to prepare for an operation (93%) compared with surgical textbooks (89%) and text-based website content (57%). Junior residents were more likely to utilize text-based website content than senior residents (P < 0.01). The three most important operative video components were accuracy, length, and cost. Senior residents significantly preferred videos that were peer-reviewed (P < 0.05) and featured attending surgeons whom they knew (P = 0.03). A majority of residents (59%) believed 5-10 min is the ideal length of an operative video. Across all postgraduate year levels, residents indicated that detailed instruction of each operative step was the most important content of a VBE module. Senior residents believed that the overall indications and details of each step of the operation were the most important contents of VBE for a junior resident. CONCLUSIONS: At this institution, general surgery residents preferentially use VBE resources for operative preparation. A centralized, standardized operative resource would likely improve resident studying efficiency, but would require personalized learning options to work for both junior and senior surgery residents.

Implementation of the ACS/ APDS Resident Skills Curriculum reveals a need for rater training: An analysis using generalizability theory.

BACKGROUND: The American College of Surgeons (ACS)/Association of Program Directors in Surgery (APDS) Resident Skills Curriculum includes validated task-specific checklists and global rating scales (GRS) for Objective Structured Assessment of Technical Skills (OSATS). However, it does not include instructions on use of these assessment tools. Since consistency of ratings is a key feature of assessment, we explored rater reliability for two skills. METHODS: Surgical faculty assessed hand-sewn bowel and vascular anastomoses in real-time using the OSATS GRS. OSATS were video-taped and independently evaluated by a research resident and surgical attending. Rating consistency was estimated using intraclass correlation coefficients (ICC) and generalizability analysis. RESULTS: Three-rater ICC coefficients across 24 videos ranged from 0.12 to 0.75. Generalizability reliability coefficients ranged from 0.55 to 0.8. Percent variance attributable to raters ranged from 2.7% to 32.1%. Pairwise agreement showed considerable inconsistency for both tasks. CONCLUSIONS: Variability of ratings for these two skills indicate the need for rater training to increase scoring agreement and decrease rater variability for technical skill assessments.

Surgical Coaching for Operative Performance Enhancement (SCOPE): skill ratings and impact on surgeons' practice.

BACKGROUND: Evidence for surgical coaching has yet to demonstrate an impact on surgeons' practice. We evaluated a surgical coaching program by analyzing quantitative and qualitative data on surgeons' intraoperative performance. METHODS: In the 2018-2019 Surgical Coaching for Operative Performance Enhancement (SCOPE) program, 46 practicing surgeons in multiple specialties at four academic medical centers were recruited to complete three peer coaching sessions, each comprising preoperative goal-setting, intraoperative observation, and postoperative debriefing. Coach and coachee rated the coachee's performance using modified Objective Structured Assessment of Technical Skills (OSATS, range 1-5) and Non-Technical Skills for Surgeons (NOTSS, range 4-16). We used generalized estimating equations to evaluate trends in skill ratings over time, adjusting for case difficulty, clinical experience, and coaching role. Upon program completion, we analyzed semi-structured interviews with individual participants regarding the perceived impact of coaching on their practice. RESULTS: Eleven of 23 coachees (48%) completed three coaching sessions, three (13%) completed two sessions, and six (26%) completed one session. Adjusted mean OSATS ratings did not vary over three coaching sessions (4.39 vs 4.52 vs 4.44, respectively; P = 0.655). Adjusted mean total NOTSS ratings also did not vary over three coaching sessions (15.05 vs 15.50 vs 15.08, respectively; P = 0.529). Regarding patient care, participants self-reported improved teamwork skills, communication skills, and awareness in and outside the operating room. Participants acknowledged the potential for coaching to improve burnout due to reduced intraoperative stress and enhanced peer support but also the potential to worsen burnout by adding to chronic work overload. CONCLUSIONS: Surgeons reported high perceived impact of peer coaching on patient care and surgeon well-being, although changes in coachees' technical and non-technical skills were not detected over three coaching sessions. While quantitative skill measurement warrants further study, longitudinal peer surgical coaching should be considered a meaningful strategy for surgeons' professional development.

Adaptation to Open-Book Online Examination During the COVID-19 Pandemic.

INTRODUCTION: COVID-19 altered medical education systems worldwide as many medical schools quickly changed to online assessment systems. However, the feasibility of online assessment and how it compares to traditional examinations is unclear. METHODS: We compared 4th year medical students' online surgery clerkship assessment scores to the traditional written examinations. The percent of correct scores using online open-book examination was compared to the results of the traditional closed-book examination in the previous three rotations. Additional correlation between grade point average(GPA) and examination performance were reviewed. RESULTS: Compared with the traditional groups, medical students who took the online, open-book examination had a significantly higher mean score in both MCQ(85.21 vs. 77.36, 72.43, 83.00, p<0.001) and essay examinations (187.36 vs. 158.77, 152.17, 152.29, p<0.001), but a significantly lower mean score in short answer examination (60.09 vs. 66.79, 67.73, 64.82, p<0.001). The online open-book examination group had a significantly lower correlation between the essay score and their GPA than the previous traditional groups (z=2.81 p=0.005, z=2.23 p=0.026, z=2.19 p=0.029). CONCLUSION: Although an online, open-book examination was feasible during the COVID-19 pandemic, this study indicates that mean scores are significantly different which has important implications regarding grading and standard setting. More research is required to assess other effects of this new assessment on long-term knowledge retention and application.

Optimizing the Implementation of Surgical Coaching Through Feedback From Practicing Surgeons.

Importance: Surgical coaching is maturing as a tangible strategy for surgeons' continuing professional development. Resources to spread this innovation are not yet widely available. Objective: To identify surgeon-derived implementation recommendations for surgical coaching programs from participants' exit interviews and ratings of their coaching interactions. Design, Setting, and Participants: This qualitative analysis of the Surgical Coaching for Operative Performance Enhancement (SCOPE) program, a quality improvement intervention, was conducted at 4 US academic medical centers. Participants included 46 practicing surgeons. The SCOPE program ran from December 7, 2018, to October 31, 2019. Data were analyzed from November 1, 2019, to January 31, 2020. Interventions: Surgeons were assigned as either a coach or a coachee, and each coach was paired with 1 coachee by a local champion who knew the surgeons professionally. Coaching pairs underwent training and were instructed to complete 3 coaching sessions-consisting of preoperative goal setting, intraoperative observation, and postoperative debriefing-focused on intraoperative performance. Main Outcomes and Measures: Themes from the participants' exit interviews covering 3 major domains: (1) describing the experience, (2) coach-coachee relationship, and (3) facilitators and barriers to implementing surgical coaching. Surgeons' responses were stratified by the net promoter score (NPS), a scale ranging from 0 to 10 points, indicating how likely they were to recommend their coaching session to others, with 9 to 10 indicating promoters; 7 to 8, passives; and 0 to 6, detractors. Results: Among the 46 participants (36 men [78.3%]), 23 were interviewed (50.0%); thematic saturation was reached with 5 coach-coachee pairs (10 interviews). Overall, coaches and coachees agreed on key implementation recommendations for surgical coaching, including how to optimize coach-coachee relationships and facilitate productive coaching sessions. The NPS categories were associated with how participants experienced their own coaching sessions. Specifically, participants who reported excellent first sessions, had a coaching partner in the same clinical specialty, and were transparent about each other's intentions in the program tended to be promoters. Participants who described suboptimal first sessions, less clinical overlap, and unclear goals with their partner were more likely detractors. Conclusions and Relevance: These exit interviews with practicing surgeons offer critical insights for addressing cultural barriers and practical challenges for successful implementation of peer coaching programs focused on surgical performance improvement. With empirical evidence on optimizing coach-coachee relationships and facilitating participants' experience, organizations can establish effective coaching programs that enable meaningful continuous professional development for surgeons and ultimately enhance patient care.

Trainee Autonomy in Minimally Invasive General Surgery in the United States: Establishing a National Benchmark.

OBJECTIVE: Minimally invasive surgery (MIS) is an integral component of General Surgery training and practice. Yet, little is known about how much autonomy General Surgery residents achieve in MIS procedures, and whether that amount is sufficient. This study aims to establish a contemporary benchmark for trainee autonomy in MIS procedures. We hypothesize that trainees achieve progressive autonomy, but fail to achieve meaningful autonomy in a substantial percentage of MIS procedures prior to graduation. SETTING/PARTICIPANTS: Fifty General Surgery residency programs in the United States, from September 1, 2015 to March 19, 2020. All Categorical General Surgery Residents and Attending Surgeons within these programs were eligible. DESIGN: Data were collected prospectively from attending surgeons and categorical General Surgery residents. Trainee autonomy was assessed using the 4-level Zwisch scale (Show and Tell, Active Help, Passive Help, and Supervision Only) on a smartphone application (SIMPL). MIS procedures included all laparoscopic, thoracoscopic, endoscopic, and endovascular/percutaneous procedures performed by residents during the study. Primary outcomes of interest were "meaningful autonomy" rates (i.e., scores in the top 2 categories of the Zwisch scale) by postgraduate year (PGY), and "progressive autonomy" (i.e., differences in autonomy between PGYs) in MIS procedures, as rated by attending surgeons. Primary outcomes were determined with descriptive statistics, one-way analysis of variance (ANOVA) and Z-tests. Secondary analyses compared (i) progressive autonomy between common MIS procedures, and (ii) progressive autonomy in MIS vs. non-MIS procedures. RESULTS: A total of 106,054 evaluations were performed across 50 General Surgery residency programs, of which 38,985 (37%) were for MIS procedures. Attendings performed 44,842 (42%) of all evaluations, including 16,840 (43%) of MIS evaluations, while residents performed the rest. Overall, meaningful autonomy in MIS procedures increased from 14.1% (PGY1s) to 75.9% (PGY5s), with significant (p < 0.001) increases between each PGY level. Meaningful autonomy rates were higher in the MIS vs. non-MIS group [57.2% vs. 48.0%, p < 0.001], and progressed more rapidly in MIS vs. non-MIS, (p < 0.05). The 7 most common MIS procedures accounted for 83.5% (n = 14,058) of all MIS evaluations. Among PGY5s performing these procedures, meaningful autonomy rates (%) were: laparoscopic appendectomy (95%); laparoscopic cholecystectomy (93%); diagnostic laparoscopy (87%); upper/lower endoscopy (85%); laparoscopic hernia repair (72%); laparoscopic partial colectomy (58%); and laparoscopic sleeve gastrectomy (45%). CONCLUSIONS: US General Surgery residents receive progressive autonomy in MIS procedures, and appear to progress more rapidly in MIS versus non-MIS procedures. However, residents fail to achieve meaningful autonomy in nearly 25% of MIS cases in their final year of residency, with higher rates of meaningful autonomy only achieved in a small subset of basic MIS procedures.

Development and Evaluation of a Novel Instrument to Measure Severity of Intraoperative Events Using Video Data.

OBJECTIVE: To develop and evaluate a novel instrument to measure SEVERE processes using video data. BACKGROUND: Surgical video data can serve an important role in understanding the relationship between intraoperative events and postoperative outcomes. However, a standard tool to measure severity of intraoperative events is not yet available. METHODS: Items to be included in the instrument were identified through literature and video reviews. A committee of experts guided item reduction, including pilot tests and revisions, and determined weighted scores. Content validity was evaluated using a validated sensibility questionnaire. Inter-rater reliability was assessed by calculating intraclass correlation coefficient. Construct validity was evaluated on a sample of 120 patients who underwent laparoscopic Roux-en-Y gastric bypass procedure, in which comprehensive video data was obtained. RESULTS: SEVERE index measures severity of 5 event types using ordinal scales. Each intraoperative event is given a weighted score out of 10. Inter-rater reliability was excellent [0.87 (95%-confidence interval, 0.77-0.92)]. In a sample of consecutive 120 patients undergoing gastric bypass procedures, a median of 12 events [interquartile range (IQR) 9-18] occurred per patient and bleeding was the most frequent type (median 10, IQR 7-14). The median SEVERE score per case was 11.3 (IQR 8.3-16.9). In risk-adjusted multivariable regression models, history of previous abdominal surgery (P = 0.02) and body mass index (P = 0.005) were associated with SEVERE scores, demonstrating construct validity evidence. CONCLUSION: The SEVERE index may prove to be a useful instrument in identifying patients with high risk of developing postoperative complications.

#SurgEdVidz: Using Social Media to Create a Supplemental Video-Based Surgery Didactic Curriculum.

BACKGROUND: A general surgery resident may have competing clinical responsibilities that limit the ability to participate in educational didactics. Social media is an alternative delivery method for educational content that can be viewed at a resident's convenience. We sought to assess the feasibility of using social media to provide educational content for surgical residents. MATERIALS AND METHODS: We created a novel, supplemental surgical curriculum consisting of two-minute videos based on the This Week in SCORE modules. Videos were posted to Twitter weekly, indexed with the #SurgEdVidz hashtag and uploaded to YouTube to create a catalog of videos. Twitter and YouTube analytics were used to calculate public viewership and impact. RESULTS: A total of 43 videos were disseminated between January 30, 2019 and January 31, 2020. An average of 6.9 h (SD: 1.4; range: 5.0-10.0) was required to generate each video, including 1.5 h (SD: 0.6; range: 0-2.0) for content review and 5.4 h (SD: 1.1; range: 4.0-8.0) for video production. Between the two platforms, videos generated a total of 51,313 views with an average of 1193 views (SD: 715; range: 412-4096) per video. Account followers are geographically distributed across 28 states, 28 countries, and six continents. CONCLUSIONS: Social media can serve as an effective tool for the distribution of surgical educational content. Twitter, in conjunction with YouTube, allows for rapid dissemination of didactic content packaged into brief videos that is flexible in viewership with low time commitment. Social media offers visibility and engagement beyond the classroom without geographical or temporal borders.