Surgeons' Coaching Techniques in the Surgical Coaching for Operative Performance Enhancement (SCOPE) Program.

OBJECTIVE: To evaluate coaching techniques used by practicing surgeons who underwent dedicated coach training in a peer surgical coaching program. BACKGROUND: Surgical coaching is a developing strategy for improving surgeons' intraoperative performance. How to cultivate effective coaching skills among practicing surgeons is uncertain. METHODS: Through the Surgical Coaching for Operative Performance Enhancement (SCOPE) program, 46 surgeons within 4 US academic medical centers were assigned 1:1 into coach/coachee pairs. All attended a 3-hour Surgical Coaching Workshop-developed using evidence from the fields of surgery and education-then received weekly reminders. We analyzed workshop evaluations and audio transcripts of postoperative debriefs between coach/coachee pairs, co-coding themes based on established principles of effective coaching: (i) self-identified goals, (ii) collaborative analysis, (iii) constructive feedback, and (iv) action planning. Coaching principles were cross-referenced with intraoperative performance topics: technical, nontechnical, and teaching skills. RESULTS: For the 8 postoperative debriefs analyzed, mean duration was 24.4 min (range 7-47 minutes). Overall, 326 coaching examples were identified, demonstrating application of all 4 core principles of coaching. Constructive feedback (17.6 examples per debrief) and collaborative analysis (16.3) were utilized more frequently than goal-setting (3.9) and action planning (3.0). Debriefs focused more often on nontechnical skills (60%) than technical skills (32%) or teaching-specific skills (8%). Among surgeons who completed the workshop evaluation (82% completion rate), 90% rated the Surgical Coaching Workshop "good" or "excellent." CONCLUSIONS: Short-course coach trainings can help practicing surgeons use effective coaching techniques to guide their peers' performance improvement in a way that aligns with surgical culture.

Non-technical skill assessments across levels of US surgical training.

BACKGROUND: To ensure safe patient care, regulatory bodies worldwide have incorporated non-technical skills proficiency in core competencies for graduation from surgical residency. We describe normative data on non-technical skill ratings of surgical residents across training levels using the US-adapted Non-Technical Skills for Surgeons (NOTSS-US) assessment tool. METHODS: We undertook an exploratory, prospective cohort study of 32 residents-interns (postgraduate year 1), junior residents (postgraduate years 2-3), and senior residents (postgraduate years 4-5)-across 3 US academic surgery residency programs. Faculty went through online training to rate residents, directly observed residents while operating together, then submitted NOTSS-US ratings on specific resident's intraoperative performance. Mean NOTSS-US ratings (total range 4-20, sum of category scores; situation awareness, decision-making, communication/teamwork, leadership each ranged 1-5, with 1=poor, 3=average, 5=excellent) were stratified by residents' training level and adjusted for resident-, rater-, and case-level variables, using mixed-effects linear regression. RESULTS: For 80 operations, the overall mean total NOTSS-US rating was 12.9 (standard deviation, 3.5). The adjusted mean total NOTSS-US rating was 16.0 for senior residents, 11.6 for junior residents, and 9.5 for interns. Adjusted differences for total NOTSS-US ratings were statistically significant across the following training levels: senior residents to interns (6.5; 95% confidence interval, 4.3-8.7; P < .001), senior to junior residents (4.4; 95% confidence interval, 2.5-6.2; P < .001), and junior residents to interns (2.1; 95% confidence interval, 0.3-3.9; P = .017). Differences in adjusted NOTSS-US ratings across residents' training levels persisted for individual NOTSS-US behavior categories. CONCLUSION: These data and online training materials can support US residency programs in determining competency-based performance milestones to develop surgical trainees' non-technical skills.

Surgical Coaching for Operative Performance Enhancement (SCOPE): skill ratings and impact on surgeons' practice.

BACKGROUND: Evidence for surgical coaching has yet to demonstrate an impact on surgeons' practice. We evaluated a surgical coaching program by analyzing quantitative and qualitative data on surgeons' intraoperative performance. METHODS: In the 2018-2019 Surgical Coaching for Operative Performance Enhancement (SCOPE) program, 46 practicing surgeons in multiple specialties at four academic medical centers were recruited to complete three peer coaching sessions, each comprising preoperative goal-setting, intraoperative observation, and postoperative debriefing. Coach and coachee rated the coachee's performance using modified Objective Structured Assessment of Technical Skills (OSATS, range 1-5) and Non-Technical Skills for Surgeons (NOTSS, range 4-16). We used generalized estimating equations to evaluate trends in skill ratings over time, adjusting for case difficulty, clinical experience, and coaching role. Upon program completion, we analyzed semi-structured interviews with individual participants regarding the perceived impact of coaching on their practice. RESULTS: Eleven of 23 coachees (48%) completed three coaching sessions, three (13%) completed two sessions, and six (26%) completed one session. Adjusted mean OSATS ratings did not vary over three coaching sessions (4.39 vs 4.52 vs 4.44, respectively; P = 0.655). Adjusted mean total NOTSS ratings also did not vary over three coaching sessions (15.05 vs 15.50 vs 15.08, respectively; P = 0.529). Regarding patient care, participants self-reported improved teamwork skills, communication skills, and awareness in and outside the operating room. Participants acknowledged the potential for coaching to improve burnout due to reduced intraoperative stress and enhanced peer support but also the potential to worsen burnout by adding to chronic work overload. CONCLUSIONS: Surgeons reported high perceived impact of peer coaching on patient care and surgeon well-being, although changes in coachees' technical and non-technical skills were not detected over three coaching sessions. While quantitative skill measurement warrants further study, longitudinal peer surgical coaching should be considered a meaningful strategy for surgeons' professional development.

Identifying Naturalistic Coaching Behavior Among Practicing Surgeons in the Operating Room.

OBJECTIVE: The aim of this study was to identify examples of naturalistic coaching behavior among practicing surgeons operating together by analyzing their intraoperative discussion. BACKGROUND: Opportunities to improve surgical performance are limited for practicing surgeons; surgical coaching is one strategy to address this need. To develop peer coaching programs that integrate with surgical culture, a better understanding is needed of how surgeons routinely discuss operative performance. METHODS: As part of a "co-surgery" quality improvement program, 20 faculty surgeons were randomized into 10 dyads who performed an operation together. Discourse analysis was conducted on transcribed intraoperative discussions. Themes were coded using an existing framework of surgical coaching principles (self-identified goals, collaborative analysis, constructive feedback, peer learning support) and surgical coaching content (technical skills, nontechnical skills). Coaching principles were cross-referenced with coaching content; c-coefficient measured the strength of association between pairs of themes. RESULTS: Overall, 44 unique coaching examples were identified in 10 operations. Of the 4 principles of surgical coaching, only self-identified goals and collaborative analysis were identified consistently. Self-identified goals were most associated with discussions regarding technical skills of "tissue exposure," "flow of operation," and "instrument handling" and the nontechnical skill "situation awareness." Collaborative analysis was most associated with discussions regarding technical skills of "respect for tissue" and "flow of operation" and nontechnical skills of "communication and teamwork." CONCLUSIONS: In naturalistic discussions between practicing surgeons in the operating room, numerous examples of unprompted coaching behavior were identified that target intraoperative performance. Prominent coaching gaps-constructive feedback and peer learning support-were also observed. Surgical coach trainings should address these gaps.

Optimizing the Implementation of Surgical Coaching Through Feedback From Practicing Surgeons.

Importance: Surgical coaching is maturing as a tangible strategy for surgeons' continuing professional development. Resources to spread this innovation are not yet widely available. Objective: To identify surgeon-derived implementation recommendations for surgical coaching programs from participants' exit interviews and ratings of their coaching interactions. Design, Setting, and Participants: This qualitative analysis of the Surgical Coaching for Operative Performance Enhancement (SCOPE) program, a quality improvement intervention, was conducted at 4 US academic medical centers. Participants included 46 practicing surgeons. The SCOPE program ran from December 7, 2018, to October 31, 2019. Data were analyzed from November 1, 2019, to January 31, 2020. Interventions: Surgeons were assigned as either a coach or a coachee, and each coach was paired with 1 coachee by a local champion who knew the surgeons professionally. Coaching pairs underwent training and were instructed to complete 3 coaching sessions-consisting of preoperative goal setting, intraoperative observation, and postoperative debriefing-focused on intraoperative performance. Main Outcomes and Measures: Themes from the participants' exit interviews covering 3 major domains: (1) describing the experience, (2) coach-coachee relationship, and (3) facilitators and barriers to implementing surgical coaching. Surgeons' responses were stratified by the net promoter score (NPS), a scale ranging from 0 to 10 points, indicating how likely they were to recommend their coaching session to others, with 9 to 10 indicating promoters; 7 to 8, passives; and 0 to 6, detractors. Results: Among the 46 participants (36 men [78.3%]), 23 were interviewed (50.0%); thematic saturation was reached with 5 coach-coachee pairs (10 interviews). Overall, coaches and coachees agreed on key implementation recommendations for surgical coaching, including how to optimize coach-coachee relationships and facilitate productive coaching sessions. The NPS categories were associated with how participants experienced their own coaching sessions. Specifically, participants who reported excellent first sessions, had a coaching partner in the same clinical specialty, and were transparent about each other's intentions in the program tended to be promoters. Participants who described suboptimal first sessions, less clinical overlap, and unclear goals with their partner were more likely detractors. Conclusions and Relevance: These exit interviews with practicing surgeons offer critical insights for addressing cultural barriers and practical challenges for successful implementation of peer coaching programs focused on surgical performance improvement. With empirical evidence on optimizing coach-coachee relationships and facilitating participants' experience, organizations can establish effective coaching programs that enable meaningful continuous professional development for surgeons and ultimately enhance patient care.

Surgical coaching to achieve the ABMS vision for the future of continuing board certification.

In February 2019, the American Board of Medical Specialties (ABMS) released the final report of the Continuing Board Certification: Vision for the Future initiative, issuing strong recommendations to replace ineffective, traditional mechanisms for physicians' maintenance of certification with meaningful strategies that strengthen professional self-regulation and simultaneously engender public trust. The Vision report charges ABMS Member Boards, including the American Board of Surgery (ABS), to develop and implement a more formative, less summative approach to continuing certification. To realize the ABMS's Vision in surgery, new programs must support the assessment of surgeons' performance in practice, identification of individualized performance gaps, tailored goals to address those gaps, and execution of personalized action plans with accountability and longitudinal support. Peer surgical coaching, especially when paired with video-based assessment, provides a structured approach that can meet this need. Surgical coaching was one of the approaches to continuing professional development that was discussed at an ABS-sponsored retreat in January 2020; this commentary review provides an overview of that discussion. The professional surgical societies, in partnership with the ABS, are uniquely positioned to implement surgical coaching programs to support the continuing certification of their membership. In this article, we provide historical context for board certification in surgery, interpret how the ABMS's Vision applies to surgical performance, and highlight recent developments in video-based assessment and peer surgical coaching. We propose surgical coaching as a foundational strategy for accomplishing the ABMS's Vision for continuing board certification in surgery.

Training Novice Raters to Assess Nontechnical Skills of Operating Room Teams.

OBJECTIVE: To our knowledge, no curricula have been described for training novice, nonclinician raters of nontechnical skills in the operating room (OR). We aimed to report the reliability of Oxford Non-Technical Skills (NOTECHS) ratings provided by novice raters who underwent a scalable curriculum for learning to assess nontechnical skills of OR teams. DESIGN: In-person training course to apply the NOTECHS framework to assessing OR teams' nontechnical skill performance, led by 2 facilitators and involving 5 partial-day sessions of didactic presentations, video simulation, and live OR observation with postassessment debriefing. NOTECHS ratings were submitted after each of 11 video scenarios and 8 live operations for the total NOTECHS team rating (including surgical/anesthesiology/nursing subteams) and for each NOTECHS skill category-situation awareness, problem solving and decision making, teamwork and cooperation, leadership and management. Inter-rater reliability was determined by calculating the intraclass correlation coefficient (ICC, range 0-1). SETTING: Training for outcome measurement during a quality improvement initiative focused on surgical safety in 3 public hospitals in Singapore. Two trainings were conducted in May 2019 and January 2020. PARTICIPANTS: Ten novice raters who were existing hospital staff and had overall minimal OR experience and no prior experience with nontechnical skill assessment. RESULTS: ICC for the total NOTECHS team rating was 0.89 (95% confidence interval [CI], 0.87-0.91). ICCs for each NOTECHS category were as follows: situation awareness, 0.83 (95% CI, 0.78-0.88); problem solving and decision-making, 0.76 (95% CI, 0.70-0.83); teamwork and cooperation, 0.84 (95% CI, 0.79-0.88); leadership and management, 0.81 (95% CI, 0.75-0.86). CONCLUSIONS: This training curriculum for nontechnical skill assessments of OR teams was associated with high inter-rater reliability from novice raters with minimal collective OR experience. Using scalable training materials to produce reliable measurements of OR team performance, this nontechnical skills assessment curriculum may contribute to future QI projects aimed at improving surgical safety.

Performance Coaching for Practicing Surgeons Enhancing Clinical Performance, Well-Being, and Trainee Experience.

Performance coaching can help surgeons, hospitals, and healthcare systems to continually improve patient care delivery by enhancing surgeons' professional development. Equally important, coaching also has great potential to combat burnout, promote physician well-being, and subsequently improve trainee education and experience. In this article, we discuss the rationale for, early evidence for, and ways to address implementation barriers for performance coaching among practicing surgeons and surgical trainees.

Using Smartphones to Capture Novel Recovery Metrics After Cancer Surgery.

Importance: Patient-generated health data captured from smartphone sensors have the potential to better quantify the physical outcomes of surgery. The ability of these data to discriminate between postoperative trends in physical activity remains unknown. Objective: To assess whether physical activity captured from smartphone accelerometer data can be used to describe postoperative recovery among patients undergoing cancer operations. Design, Setting, and Participants: This prospective observational cohort study was conducted from July 2017 to April 2019 in a single academic tertiary care hospital in the United States. Preoperatively, adults (age ≥18 years) who spoke English and were undergoing elective operations for skin, soft tissue, head, neck, and abdominal cancers were approached. Patients were excluded if they did not own a smartphone. Exposures: Study participants downloaded an application that collected smartphone accelerometer data continuously for 1 week preoperatively and 6 months postoperatively. Main Outcomes and Measures: The primary end points were trends in daily exertional activity and the ability to achieve at least 60 minutes of daily exertional activity after surgery among patients with vs without a clinically significant postoperative event. Postoperative events were defined as complications, emergency department presentations, readmissions, reoperations, and mortality. Results: A total of 139 individuals were approached. In the 62 enrolled patients, who were followed up for a median (interquartile range [IQR]) of 147 (77-179) days, there were no preprocedural differences between patients with vs without a postoperative event. Seventeen patients (27%) experienced a postoperative event. These patients had longer operations than those without a postoperative event (median [IQR], 225 [152-402] minutes vs 107 [68-174] minutes; P < .001), as well as greater blood loss (median [IQR], 200 [35-515] mL vs 25 [5-100] mL; P = .006) and more follow-up visits (median [IQR], 2 [2-4] visits vs 1 [1-2] visits; P = .002). Compared with mean baseline daily exertional activity, patients with a postoperative event had lower activity at week 1 (difference, -41.6 [95% CI, -75.1 to -8.0] minutes; P = .02), week 3 (difference, -40.0 [95% CI, -72.3 to -3.6] minutes; P = .03), week 5 (difference, -39.6 [95% CI, -69.1 to -10.1] minutes; P = .01), and week 6 (difference, -36.2 [95% CI, -64.5 to -7.8] minutes; P = .01) postoperatively. Fewer of these patients were able to achieve 60 minutes of daily exertional activity in the 6 weeks postoperatively (proportions: week 1, 0.40 [95% CI, 0.31-0.49]; P < .001; week 2, 0.49 [95% CI, 0.40-0.58]; P = .003; week 3, 0.39 [95% CI, 0.30-0.48]; P < .001; week 4, 0.47 [95% CI, 0.38-0.57]; P < .001; week 5, 0.51 [95% CI, 0.42-0.60]; P < .001; week 6, 0.73 [95% CI, 0.68-0.79] vs 0.43 [95% CI, 0.33-0.52]; P < .001). Conclusions and Relevance: Smartphone accelerometer data can describe differences in postoperative physical activity among patients with vs without a postoperative event. These data help objectively quantify patient-centered surgical recovery, which have the potential to improve and promote shared decision-making, recovery monitoring, and patient engagement.

Comparing the Learning Effectiveness of Healthcare Simulation in the Observer Versus Active Role: Systematic Review and Meta-Analysis.

STATEMENT: The benefits of observation in simulation-based education in healthcare are increasingly recognized. However, how it compares with active participation remains unclear. We aimed to compare effectiveness of observation versus active participation through a systematic review and meta-analysis. Effectiveness was defined using Kirkpatrick's 4-level model, namely, participants' reactions, learning outcomes, behavior changes, and patient outcomes. The peer-reviewed search strategy included 8 major databases and gray literature. Only randomized controlled trials were included. A total of 13 trials were included (426 active participants and 374 observers). There was no significant difference in reactions (Kirkpatrick level 1) to training between groups, but active participants learned (Kirkpatrick level 2) significantly better than observers (standardized mean difference = -0.2, 95% confidence interval = -0.37 to -0.02, P = 0.03). Only one study reported behavior change (Kirkpatrick level 3) and found no significant difference. No studies reported effects on patient outcomes (Kirkpatrick level 4). Further research is needed to understand how to effectively integrate and leverage the benefits of observation in simulation-based education in healthcare.

"Development of an Academic Surgical Student Program for Enhancing Student-Faculty Engagement".

OBJECTIVE: We describe an innovative medical student surgery interest group and its influence on mentorship and career exploration. DESIGN: SCRUBS, created to promote interest in academic surgery, is student-led, with continual surgical faculty and resident involvement. Its 3-component programming focuses on clinical skills, research, and mentorship opportunities for medical students to get involved in academic surgery early in medical education. SETTING: The University of Michigan Medical School, Ann Arbor, MI. PARTICIPANTS: First through fourth year medical students, surgery residents, and attending surgeons. RESULTS: SCRUBS is a multifaceted student organization providing longitudinal exposure to various aspects of surgery and academic medicine. The group grew annually from 2010 to 2014, with students and faculty expressing positive feedback. Over the time period reviewed, we had a greater percentage of students applying into surgical specialties compared with the national average (16.8 vs 12% in 2014). The group supported and facilitated mentorship, clinical skills development, and research opportunities for interested students. CONCLUSIONS: This innovative surgery interest group has been well received by students and surgeons, and our institution has seen above-average interest in surgical careers. Early, preclinical mentorship and exposure provided by SCRUBS may contribute to this higher surgical interest.

Hospital Teaching Status and Medicare Expenditures for Complex Surgery.

OBJECTIVE: To evaluate the relationship between hospital teaching intensity, Medicare payments, and perioperative outcomes. BACKGROUND: Several emerging payment policies penalize hospitals for low-value healthcare. Teaching hospitals may be at a disadvantage given the perception that they deliver care less efficiently. METHODS: Using Medicare Provider and Analysis Review files, we studied patients from age 65 to 100 years who underwent abdominal aortic aneurysm (AAA) repair (n = 71,422), pulmonary resection (n = 93,056), or colectomy (n = 277,619) from 2009 to 2012. Patients' hospitals were categorized into quintiles of teaching intensity (very major, major, minor, very minor, and nonteaching hospitals) based on the resident-to-bed ratio. Risk-adjusted 30-day Medicare payments were price-standardized to account for graduate medical education payments, disproportionate share costs, and regional wage-index adjustments. Risk-adjusted perioperative outcomes were also assessed. RESULTS: Comparing risk-adjusted Medicare payments per episode of surgery, very major teaching hospitals were $14,145 more expensive than nonteaching hospitals for AAA repair ($45,570 vs $31,426; P < 0.001), $9,812 more expensive for pulmonary resection ($39,550 vs $29,738; P < 0.001), and $19,147 more expensive for colectomy ($51,893 vs $32,746; P < 0.001). However, after accounting for social subsidies and regional variation in Medicare spending, risk-adjusted Medicare payments were not statistically different between very major teaching hospitals and nonteaching hospitals for AAA repair ($29,946 vs $27,993; P = 0.18) and pulmonary resection ($25,407 vs $26,813; P = 1.00); a statistically significant but attenuated difference persisted for colectomy ($34,949 vs $30,352; P < 0.001). Very major teaching hospitals generally had higher risk-adjusted rates of serious complications and readmissions, but lower risk-adjusted rates of failure to rescue and 30-day mortality than did nonteaching hospitals. CONCLUSIONS: After price-standardization to account for intended differences in Medicare spending, risk-adjusted Medicare payments for an episode of surgical care were similar at teaching hospitals and nonteaching hospitals for three complex inpatient operations.