Decision-Making Confidence of Clinical Competency Committees for Entrustable Professional Activities.

Importance: A competency-based assessment framework using entrustable professional activities (EPAs) was endorsed by the American Board of Surgery following a 2-year feasibility pilot study. Pilot study programs' clinical competency committees (CCCs) rated residents on EPA entrustment semiannually using this newly developed assessment tool, but factors associated with their decision-making are not yet known. Objective: To identify factors associated with variation in decision-making confidence of CCCs in EPA summative entrustment decisions. Design, Setting, and Participants: This cohort study used deidentified data from the EPA Pilot Study, with participating sites at 28 general surgery residency programs, prospectively collected from July 1, 2018, to June 30, 2020. Data were analyzed from September 27, 2022, to February 15, 2023. Exposure: Microassessments of resident entrustment for pilot EPAs (gallbladder disease, inguinal hernia, right lower quadrant pain, trauma, and consultation) collected within the course of routine clinical care across four 6-month study cycles. Summative entrustment ratings were then determined by program CCCs for each study cycle. Main Outcomes and Measures: The primary outcome was CCC decision-making confidence rating (high, moderate, slight, or no confidence) for summative entrustment decisions, with a secondary outcome of number of EPA microassessments received per summative entrustment decision. Bivariate tests and mixed-effects regression modeling were used to evaluate factors associated with CCC confidence. Results: Among 565 residents receiving at least 1 EPA microassessment, 1765 summative entrustment decisions were reported. Overall, 72.5% (1279 of 1765) of summative entrustment decisions were made with moderate or high confidence. Confidence ratings increased with increasing mean number of EPA microassessments, with 1.7 (95% CI, 1.4-2.0) at no confidence, 1.9 (95% CI, 1.7-2.1) at slight confidence, 2.9 (95% CI, 2.6-3.2) at moderate confidence, and 4.1 (95% CI, 3.8-4.4) at high confidence. Increasing number of EPA microassessments was associated with increased likelihood of higher CCC confidence for all except 1 EPA phase after controlling for program effects (odds ratio range: 1.21 [95% CI, 1.07-1.37] for intraoperative EPA-4 to 2.93 [95% CI, 1.64-5.85] for postoperative EPA-2); for preoperative EPA-3, there was no association. Conclusions and Relevance: In this cohort study, the CCC confidence in EPA summative entrustment decisions increased as the number of EPA microassessments increased, and CCCs endorsed moderate to high confidence in most entrustment decisions. These findings provide early validity evidence for this novel assessment framework and may inform program practices as EPAs are implemented nationally.

Faculty and Resident Perspectives on the Implementation of Entrustable Professional Activities in General Surgery Residency.

OBJECTIVE: This study aimed to identify what best practices facilitate implementation of Entrustable Professional Activities (EPAs) into surgical training programs. DESIGN: This is a mixed methods study utilizing both survey data as well as semi-structured interviews of faculty and residents involved in the American Board of Surgery (ABS) EPA pilot study. SETTING: From 2018 to 2020, the ABS conducted a pilot that introduced five EPAs across 28 general surgery training programs. PARTICIPANTS: All faculty members and residents at the 28 pilot programs were invited to participate in the study. RESULTS: About 117 faculty members and 79 residents responded to the survey. The majority of faculty (81%) and residents (66%) felt that EPAs were useful and were a valuable addition to training. While neither group felt that EPAs were overly time consuming to complete, residents did report difficulty incorporating them into their daily workflow (44%). Semi-structured interviews found that programs that focused on faculty and resident -development and utilized frequent reminders about the importance and necessity of EPAs tended to perform better. CONCLUSIONS: EPA implementation is feasible in general surgery training programs but requires significant effort and engagement from all levels of program personnel. As EPAs are implemented by the ABS nationally a focus on resident and faculty development will be critical to success.

Entrustable Professional Activities in Surgery: A Review.

Importance: Entrustable professional activities (EPAs) compose a competency-based education (CBE) assessment framework that has been increasingly adopted across medical specialties as a workplace-based assessment tool. EPAs focus on directly observed behaviors to determine the level of entrustment a trainee has for a given activity of that specialty. In this narrative review, we highlight the rationale for EPAs in general surgery, describe current evidence supporting their use, and outline some of the practical considerations for EPAs among residency programs, faculty, and trainees. Observations: An expanding evidence base for EPAs in general surgery has provided moderate validity evidence for their use as well as practical recommendations for implementation across residency programs. Challenges to EPA use include garnering buy-in from individual faculty and residents to complete EPA microassessments and engage in timely, specific feedback after a case or clinical encounter. When successfully integrated into a program's workflow, EPAs can provide a more accurate picture of residents' competence for a fundamental surgical task or activity compared with other assessment methods. Conclusions and Relevance: EPAs represent the next significant shift in the evaluation of general surgery residents as part of the overarching progression toward CBE among all US residency programs. While pragmatic challenges to the implementation of EPAs remain, the best practices from EPA and other CBE assessment literature summarized in this review may assist individuals and programs in implementing EPAs. As EPAs become more widely used in general surgery resident training, further analysis of barriers and facilitators to successful and sustainable EPA implementation will be needed to continue to optimize and advance this new assessment framework.

Building on Strengths: An Affirmational and Systems-Level Approach to Revisiting Character in Medicine.

ABSTRACT: Although U.S. medical education has continued to place increased emphasis on defining competency standards and ensuring accountability to the public, health care inequities have persisted, several basic health outcomes have worsened, public trust in the health care system has eroded, and moral distress, burnout, and attrition among practicing physicians have escalated. These opposing trends beg the question of how the "good doctor" concept may be strengthened. In this perspective, the authors argue that revisiting the construct of physician character from an affirmational perspective could meaningfully improve medical education's impact on overall health by more holistically conceptualizing what-and who-a good doctor is. The authors introduce positive psychology's framework of character strengths, probe the distinction between character strengths and medical professionalism, and summarize the role of character strengths in promoting physician engagement and well-being in health care work. They contend that a systems-level approach to cultivating character strengths will foster physician moral agency and well-being and, by extension, transformational change in health care. Consistent with best practice in modern character education, the authors propose that institutions mindfully cultivate moral community among all stakeholders (students, faculty, staff, postgraduate trainees, and patients) and that moral community interaction centers on each member's personal aspirations with respect to living a good life, guided by the character strengths framework and informed by patient perspectives.

Validity of Entrustable Professional Activities in a National Sample of General Surgery Residency Programs.

BACKGROUND: The American Board of Surgery has endorsed competency-based education as vital to the assessment of surgical training. From 2018 to 2020, a national pilot study was conducted at 28 general surgery programs to evaluate feasibility of implementing entrustable professional activities (EPAs) for 5 common general surgical conditions. ACGME core competency Milestones were also rated for each resident by program clinical competency committees. This study aimed to evaluate the validity of general surgery EPAs compared with Milestones. STUDY DESIGN: Prospectively collected, de-identified EPA Pilot Study data were analyzed. EPAs studied were EPA-1 (gallbladder), EPA-2 (inguinal hernia), EPA-3 (right lower quadrant pain), EPA-4 (trauma), and EPA-5 (consult). Variables abstracted included levels of EPA entrustment (1 to 5) and corresponding ACGME Milestone subcompetency ratings (1 to 5) for the same study cycle. Spearman's correlations were used to evaluate the relationship between summative EPA scores and corresponding Milestone ratings. RESULTS: A total of 493 unique residents received a summative entrustment decision. EPA summative entrustment scores had moderate-to-strong positive correlation with mapped Milestone subcompetencies, with median rho value of 0.703. Among operation-focused EPAs, median rho values were similar between EPA-1 (0.688) and EPA-2 (0.661), but higher for EPA-3 (0.833). EPA-4 showed a strong positive correlation with diagnosis and communication competencies (0.724), whereas EPA-5, mapped to the most Milestone subcompetencies, had the lowest median rho value (0.455). CONCLUSIONS: Moderate-to-strong positive correlation was noted between EPAs and patient care, medical knowledge, and communication Milestones. These findings support the validity of EPAs in general surgery and suggest that EPA assessments can be used to inform Milestone ratings by clinical competency committees.

Surgical Education.

This paper summarizes the proceedings of the joint European Surgical Association ESA/American Surgical Association symposium on Surgical Education that took place in Bordeaux, France, as part of the celebrations for 30 years of ESA scientific meetings. Three presentations on the use of quantitative metrics to understand technical decisions, coaching during training and beyond, and entrustable professional activities were presented by American Surgical Association members and discussed by ESA members in a symposium attended by members of both associations.

Implementation of Entrustable Professional Activities in General Surgery: Results of a National Pilot Study.

OBJECTIVE: The ongoing complexity of general surgery training has led to an increased focus on ensuring the competence of graduating residents. Entrustable professional activities (EPAs) are units of professional practice that provide an assessment framework to drive competency-based education. The American Board of Surgery convened a group from the American College of Surgeons, Accreditation Council for Graduate Medical Education (ACGME) Surgery Review Committee, and Association of Program Directors in Surgery to develop and implement EPAs in a pilot group of residency programs across the country. The objective of this pilot study was to determine the feasibility and utility of EPAs in general surgery resident training. METHODS: 5 EPAs were chosen based on the most common procedures reported in ACGME case logs and by practicing general surgeons (right lower quadrant pain, biliary disease, inguinal hernia), along with common activities covering additional ACGME milestones (performing a consult, care of a trauma patient). Levels of entrustment assigned (1 to 5) were observation only, direct supervision, indirect supervision, unsupervised, and teaching others. Participating in site recruitment and faculty development occurred from 2017 to 2018. EPA implementation at individual residency programs began on July 1, 2018, and was completed on June 30, 2020. Each site was assigned 2 EPAs to implement and collected EPA microassessments on residents for those EPAs. The site clinical competency committees (CCC) used these microassessments to make summative entrustment decisions. Data submitted to the independent deidentified data repository every 6 months included the number of microassessments collected per resident per EPA and CCC summative entrustment decisions. RESULTS: Twenty-eight sites were selected to participate in the program and represented geographic and size variability, community, and university-based programs. Over the course of the 2-year pilot programs reported on 14 to 180 residents. Overall, 6,272 formative microassessments were collected (range, 0 to 1144 per site). Each resident had between 0 and 184 microassessments. The mean number of microassessments per resident was 5.6 (SD = 13.4) with a median of 1 [interquartile range (IQR) = 6]. There were 1,763 summative entrustment ratings assigned to 497 unique residents. The average number of observations for entrustment was 3.24 (SD 3.61) with a median of 2 (IQR 3). In general, PGY1 residents were entrusted at the level of direct supervision and PGY5 residents were entrusted at unsupervised practice or teaching others. For each EPA other than the consult EPA, the degree of entrustment reported by the CCC increased by resident level. CONCLUSIONS: These data provide evidence that widespread implementation of EPAs across general surgery programs is possible, but variable. They provide meaningful data that graduating chief residents are entrusted by their faculty to perform without supervision for several common general surgical procedures and highlight areas to target for the successful widespread implementation of EPAs.

Impact of Endocrinologist and Surgeon Density on Well-Differentiated Thyroid Cancer Survival.

BACKGROUND: The influence of provider density and access on well-differentiated thyroid cancer (DTC) survival is uncertain. METHODS: We used the SEER18 database to study DTC patients undergoing surgery from 2000-2012. County-level endocrinologist and surgeon density were calculated. We evaluated the relationship between provider density and cause-specific survival controlling for demographic, socioeconomic, and treatment characteristics. RESULTS: Median endocrinologist density was 1.4/100 000 residents, with 15.5% of patients living in a county with no endocrinologist. Survival increased by 11% for each endocrinologist/100,000 people (P = .007). Median surgeon density was 14.8/100 000 residents, with only 1.7% of patients living in a county with no general surgeon or otolaryngologist. No significant association between surgeon density and survival was identified (P = .06). Rural residence was independently associated with lower survival (P = .009). CONCLUSIONS: County-level endocrinologist density is associated with improved DTC survival. Results may reflect endocrinologist expertise, earlier diagnosis, or represent surrogacy for higher county-level access to specialized care.

The Quality of Operative Performance Narrative Feedback: A Retrospective Data Comparison Between End of Rotation Evaluations and Workplace-based Assessments.

OBJECTIVE: To describe the quality of operative performance feedback using evaluation tools commonly used by general surgery residency training programs. SUMMARY OF BACKGROUND DATA: The majority of surgical training programs administer an evaluation through which faculty members may rate and comment on trainee operative performance at the end of the rotation (EOR). Many programs have also implemented the system for improving and measuring procedural learning (SIMPL), a workplace-based assessment tool with which faculty can rate and comment on a trainee's operative performance immediately after a case. It is unknown how the quality of narrative operative performance feedback delivered with these tools compares. METHODS: The authors collected EOR evaluations and SIMPL narrative comments on trainees' operative performance from 3 university-based surgery training programs during the 2016-2017 academic year. Two surgeon raters categorized comments relating to operative skills as being specific or general and as encouraging and/or corrective. Comments were then classified as effective, mediocre, ineffective, or irrelevant. The frequencies with which comments were rated as effective were compared using Chi-square analysis. RESULTS: The authors analyzed a total of 600 comments. 10.7% of EOR and 58.3% of SIMPL operative performance evaluation comments were deemed effective (P < 0.0001). CONCLUSIONS: Evaluators give significantly higher quality operative performance feedback when using workplace-based assessment tools rather than EOR evaluations.

Career Advancement for Surgeon-Educators:Findings from a Modified Delphi Process.

OBJECTIVES: Reward and recognition of surgical education as an academic activity remains a highly variable process between institutions. The goal of this study is to provide expert consensus definition of an academic surgical educator, with focus on criteria for academic promotion. STUDY DESIGN AND SETTING: Following IRB approval, a Web-based modified Delphi process was used to generate prioritized academic promotion criteria for surgical educators. PARTICIPANTS AND SETTING: Participants were recruited nationally from a pool of senior academic surgeons who are members of the Society of University Surgeons and the Society of Surgical Chairs. RESULTS: Following a three-round modified Delphi process, the top domains of educational activity for promotion to associate professor and professor were scholarship, teaching, and administration; mentorship was also a priority category for promotion to professor. The top three activities described for promotion to Associate Professor were active participation in conferences/ departmental educational activities for medical students and residents; educational portfolio demonstrating commitment to activities as an educator; and clinical teaching excellence at their home institution. The three activities most highly scored items for promotion to Professor were mentorship of junior surgical educators; active participation in conferences/ departmental educational activities for medical students and residents; and a record of teaching excellence at the medical student and resident levels. CONCLUSIONS: These findings demonstrate a progression from teacher to scholar to leader across a surgical educator's career, with each level incorporating and building upon the prior activities. Identification of categories and criteria may meaningfully inform best practices to be incorporated into the career development and promotion processes for surgeons on an educator academic pathway.

Association of socioeconomic status with 30- and 90-day readmission following open and laparoscopic hernia repair: a nationwide readmissions database analysis.

BACKGROUND: Socioeconomic disparities have been associated with outcomes in many medical conditions. The association of socioeconomic status (SES) with readmissions after ventral and inguinal hernia repair has not been well studied on a national level. This study aims to evaluate the association of SES with readmission as a significant outcome in patients undergoing ventral and inguinal hernia repair. METHODS: A retrospective cohort study was performed evaluating patients undergoing ventral hernia and inguinal hernia repair with 1:1 propensity score matching using the Nationwide Readmissions Database (2016-2017). Both 30- and 90-day readmissions were examined. After matching, a multivariate logistic regression analysis was performed using confounding variables including hospital setting, comorbidities, urgency of repair, sociodemographic status, and payer. Likelihood of readmission was reported in odds ratio form. RESULTS: Readmission rates were 11.56% (24,323 out of 210,381) and 17.94% (30,893 out of 172,210) for 30- and 90-day readmissions, respectively. Patients with Medicaid and in the lower income quartile were more likely to present in an emergent fashion for hernia repair. After matching, a multivariate logistic regression analysis showed socioeconomic status (OR 1.250 and 1.229) was a statistically significant independent predictor of readmission at 30 and 90 days, respectively. Inversely, factors associated with the least likely chance of readmission were a laparoscopic approach (OR 0.646 and 0.641), elective admission (OR 0.824 and 0.779), and care in a teaching hospital (OR 0.784 and 0.798). CONCLUSION: SES is an independent predictor of readmission at 30 and 90 days following open and laparoscopic ventral and inguinal hernia repair. Patients with a lower socioeconomic status were more likely to undergo hernia repair in the emergent setting. Efforts toward mitigating SES disparities by potentially promoting MIS techniques, enhancing access to elective cases, and systematic approaches to perioperative care for this disadvantaged population can potentially enhance overall hernia outcomes.

On pandemics and pivots: a COVID-19 reflection on envisioning the future of medical education.

The required adjustments precipitated by the coronavirus disease 2019 crisis have been challenging, but also represent a critical opportunity for the evolution and potential disruptive and constructive change of medical education. Given that the format of medical education is not fixed, but malleable and in fact must be adaptable to societal needs through ongoing reflexivity, we find ourselves in a potentially transformative learning phase for the field. An Association for Medical Education in Europe ASPIRE Academy group of 18 medical educators from seven countries was formed to consider this opportunity, and identified critical questions for collective reflection on current medical education practices and assumptions, with the attendant challenge to envision the future of medical education. This was achieved through online discussion as well as asynchronous collective reflections by group members. Four major themes and related conclusions arose from this conversation: Why we teach: the humanitarian mission of medicine should be reinforced; what we teach: disaster management, social accountability and embracing an environment of complexity and uncertainty should be the core; how we teach: open pathways to lean medical education and learning by developing learners embedded in a community context; and whom we teach: those willing to take professional responsibility. These collective reflections provide neither fully matured digests of the challenges of our field, nor comprehensive solutions; rather they are offered as a starting point for medical schools to consider as we seek to harness the learning opportunities stimulated by the pandemic.

Leadership skills curriculum development for residents and fellows: A needs-assessment.

BACKGROUND: Non-technical skills (NTS) curricula have demonstrated success in surgical residencies. The purpose of this study is to examine the need for a structured leadership curriculum at our institution. METHODS: A needs-assessment survey analyzing the importance of leadership domains, previously validated by Kazley et al. was delivered to 240 general surgery staff. Respondent groups were broken down into: Attendings, Residents, and Multi-Disciplinary. Statistical analyses were conducted using Cronbach's Alpha (α = 0.9259) and Fisher's exact test (pre-set p-value = 0.05). The importance of each competency was compared among groups and importance was defined as >75% important and very important responses. RESULTS: Nineteen of 33 competencies were important for all groups, including 3 with 100% importance: interpersonal communication, team-work, and problem-solving. Several competencies showed statistically significant differences among groups. CONCLUSION: A diverse range of surgery staff agreed that 19 leadership domains are important to teach residents, with some variance among respondent groups.

International Delphi Expert Consensus on Safe Return to Surgical and Endoscopic Practice: From the Coronavirus Global Surgical Collaborative.

OBJECTIVE: The aim of this work is to formulate recommendations based on global expert consensus to guide the surgical community on the safe resumption of surgical and endoscopic activities. BACKGROUND: The COVID-19 pandemic has caused marked disruptions in the delivery of surgical care worldwide. A thoughtful, structured approach to resuming surgical services is necessary as the impact of COVID-19 becomes better controlled. The Coronavirus Global Surgical Collaborative sought to formulate, through rigorous scientific methodology, consensus-based recommendations in collaboration with a multidisciplinary group of international experts and policymakers. METHODS: Recommendations were developed following a Delphi process. Domain topics were formulated and subsequently subdivided into questions pertinent to different aspects of surgical care in the COVID-19 crisis. Forty-four experts from 15 countries across 4 continents drafted statements based on the specific questions. Anonymous Delphi voting on the statements was performed in 2 rounds, as well as in a telepresence meeting. RESULTS: One hundred statements were formulated across 10 domains. The statements addressed terminology, impact on procedural services, patient/staff safety, managing a backlog of surgeries, methods to restart and sustain surgical services, education, and research. Eighty-three of the statements were approved during the first round of Delphi voting, and 11 during the second round. A final telepresence meeting and discussion yielded acceptance of 5 other statements. CONCLUSIONS: The Delphi process resulted in 99 recommendations. These consensus statements provide expert guidance, based on scientific methodology, for the safe resumption of surgical activities during the COVID-19 pandemic.

Attendings' Perceptions of Authentic Evaluation Criteria for Effective Surgical Consults.

OBJECTIVE: The authors aimed to investigate faculty evaluation criteria for an effective oral surgical presentation in actual patient care contexts. DESIGN: We conducted a 2-step observation-based qualitative study. Residents audiotaped oral presentations of a surgical consult to an attending. Evaluation panels listened to the recordings and discussed to develop joint feedback for the resident. The panel discussions were recorded and served as the data source for this study. We analyzed the data following the grounded theory approach using open coding and axial coding. SETTING: The study setting was at Southern Illinois University School of Medicine, a 5-year general surgery residency program in Springfield, Illinois. PARTICIPANTS: Thirteen residents out of 19 in the program participated by virtue of having submitted recordings of a patient care consult presentation via phone. Evaluation panels consisted of general surgery academic and community faculty, as well as senior residents. RESULTS: Several criteria for effective oral presentations emerged that have rarely been discussed in prior literature. Themes included: (1) The strategic opening is critical as it "sets the stage" and frames how the attending will listen. Situational factors, such as consideration of time of the day and urgency, should be accounted for in the opening. (2) A deductive structure defines the relevance of the presented information. Clinical judgement should precede supporting evidence. Attending physicians perceive important information as unnecessary if provided outside of this framework. (3) Established trust between a resident and a surgeon determines the level of detail expected of the presenting resident. With increasing trust, surgeons expect residents to present fewer details; if too much detail is included, the presentation may be assessed as ineffective. (4) Surgical descriptions are appreciated for their value in promoting the attending's visualization or mental picture of the patient condition. (5) Oral emphasis using voice tone and pace can be helpful for capturing attending attention. CONCLUSIONS: These findings can be utilized to improve the current training program and assessment rubrics toward contextualized work-based assessment practices in surgery. Oral patient presentation skills are neither static nor universal, but fluid and reflexive, based on trust, and situational factors.

Establishment of an endoscopic retrograde cholangiopancreatography (ERCP) program in rural Kenya: a review of patient and trainee outcomes.

BACKGROUND: Endoscopic retrograde cholangiopancreatography (ERCP) is a leading modality for treatment of biliary and pancreatic disease but is not widely available in sub-Saharan Africa. We aimed to assess the development and outcomes of an ERCP service in southwestern Kenya, including case volumes, success rates, infrastructure, and training. METHODS: We conducted a retrospective review of all ERCPs performed at Tenwek Hospital in Bomet, Kenya between January 1, 2011 and March 31, 2020. RESULTS: In total 277 ERCP procedures were attempted during the study period. The commonest indication was obstructive jaundice: 91 patients (32.9%) had malignancy and 85 (30.7%) had choledocholithiasis. Overall clinical success rate was 76.1% and was the highest in patients with biliary stones (81.2%) and lowest in those with tumors (73.5%) (p = 0.094). Procedure-related adverse events occurred in 11.9%, including post-ERCP pancreatitis in 3.6%, with a procedure-related mortality rate of 1.4%. Annual case volumes increased, and mean procedure duration decreased from 162 to 115 min (p = 0.0007) over time. A previously- rained endoscopist initially performed all cases; two staff endoscopists were trained in ERCP during the study period, performing 130 and 89 ERCPs during training, with clinical success rates of 84% and 74% during their subsequent independent practice. CONCLUSION: An ERCP service can be successfully developed at a rural African hospital, with acceptable success and adverse event rates. Biliary obstruction due to stones or tumors are the most common findings. While a previously trained endoscopist should initiate and champion the service, staff endoscopists can be successfully trained despite limited case volumes.