Decision-Making Confidence of Clinical Competency Committees for Entrustable Professional Activities.

Importance: A competency-based assessment framework using entrustable professional activities (EPAs) was endorsed by the American Board of Surgery following a 2-year feasibility pilot study. Pilot study programs' clinical competency committees (CCCs) rated residents on EPA entrustment semiannually using this newly developed assessment tool, but factors associated with their decision-making are not yet known. Objective: To identify factors associated with variation in decision-making confidence of CCCs in EPA summative entrustment decisions. Design, Setting, and Participants: This cohort study used deidentified data from the EPA Pilot Study, with participating sites at 28 general surgery residency programs, prospectively collected from July 1, 2018, to June 30, 2020. Data were analyzed from September 27, 2022, to February 15, 2023. Exposure: Microassessments of resident entrustment for pilot EPAs (gallbladder disease, inguinal hernia, right lower quadrant pain, trauma, and consultation) collected within the course of routine clinical care across four 6-month study cycles. Summative entrustment ratings were then determined by program CCCs for each study cycle. Main Outcomes and Measures: The primary outcome was CCC decision-making confidence rating (high, moderate, slight, or no confidence) for summative entrustment decisions, with a secondary outcome of number of EPA microassessments received per summative entrustment decision. Bivariate tests and mixed-effects regression modeling were used to evaluate factors associated with CCC confidence. Results: Among 565 residents receiving at least 1 EPA microassessment, 1765 summative entrustment decisions were reported. Overall, 72.5% (1279 of 1765) of summative entrustment decisions were made with moderate or high confidence. Confidence ratings increased with increasing mean number of EPA microassessments, with 1.7 (95% CI, 1.4-2.0) at no confidence, 1.9 (95% CI, 1.7-2.1) at slight confidence, 2.9 (95% CI, 2.6-3.2) at moderate confidence, and 4.1 (95% CI, 3.8-4.4) at high confidence. Increasing number of EPA microassessments was associated with increased likelihood of higher CCC confidence for all except 1 EPA phase after controlling for program effects (odds ratio range: 1.21 [95% CI, 1.07-1.37] for intraoperative EPA-4 to 2.93 [95% CI, 1.64-5.85] for postoperative EPA-2); for preoperative EPA-3, there was no association. Conclusions and Relevance: In this cohort study, the CCC confidence in EPA summative entrustment decisions increased as the number of EPA microassessments increased, and CCCs endorsed moderate to high confidence in most entrustment decisions. These findings provide early validity evidence for this novel assessment framework and may inform program practices as EPAs are implemented nationally.

Goals, Structure and Financing of Surgical Residency Training: A Subcommittee Report of the Blue Ribbon Committee II.

OBJECTIVE: As part of the Blue Ribbon Committee II, review current goals, structure and financing of surgical training in Graduate Medical Education (GME) and recommend needed changes. SUMMARY BACKGROUND DATA: Surgical training has continually undergone major transitions with the 80-hour work week, earlier specialization (vascular, plastics and cardiovascular) and now entrustable professional activities (EPAs) as part of competency based medical education (CBME). Changes are needed to ensure the efficiencies of CBME are utilized, that stable graduate medical education funding is secured, and that support for surgeons who teach is made available. METHODS: Convened subcommittee discussions to determine needed focus for recommendations. RESULTS: Five recommendations are offered for changes to GME financing, incorporation of CBME, and support for educators, students and residents in training. CONCLUSIONS: Changes in surgical training related to CBME offer opportunity for change and innovation. Our subcommittee has laid out a potential path forward for improvements in GME funding, training structure, compensation of surgical educators, and support of students and residents in training.

Validity of Entrustable Professional Activities in a National Sample of General Surgery Residency Programs.

BACKGROUND: The American Board of Surgery has endorsed competency-based education as vital to the assessment of surgical training. From 2018 to 2020, a national pilot study was conducted at 28 general surgery programs to evaluate feasibility of implementing entrustable professional activities (EPAs) for 5 common general surgical conditions. ACGME core competency Milestones were also rated for each resident by program clinical competency committees. This study aimed to evaluate the validity of general surgery EPAs compared with Milestones. STUDY DESIGN: Prospectively collected, de-identified EPA Pilot Study data were analyzed. EPAs studied were EPA-1 (gallbladder), EPA-2 (inguinal hernia), EPA-3 (right lower quadrant pain), EPA-4 (trauma), and EPA-5 (consult). Variables abstracted included levels of EPA entrustment (1 to 5) and corresponding ACGME Milestone subcompetency ratings (1 to 5) for the same study cycle. Spearman's correlations were used to evaluate the relationship between summative EPA scores and corresponding Milestone ratings. RESULTS: A total of 493 unique residents received a summative entrustment decision. EPA summative entrustment scores had moderate-to-strong positive correlation with mapped Milestone subcompetencies, with median rho value of 0.703. Among operation-focused EPAs, median rho values were similar between EPA-1 (0.688) and EPA-2 (0.661), but higher for EPA-3 (0.833). EPA-4 showed a strong positive correlation with diagnosis and communication competencies (0.724), whereas EPA-5, mapped to the most Milestone subcompetencies, had the lowest median rho value (0.455). CONCLUSIONS: Moderate-to-strong positive correlation was noted between EPAs and patient care, medical knowledge, and communication Milestones. These findings support the validity of EPAs in general surgery and suggest that EPA assessments can be used to inform Milestone ratings by clinical competency committees.

Faculty Entrustment and Resident Entrustability.

Importance: As the surgical education paradigm transitions to entrustable professional activities, a better understanding of the factors associated with resident entrustability are needed. Previous work has demonstrated intraoperative faculty entrustment to be associated with resident entrustability. However, larger studies are needed to understand if this association is present across various surgical training programs. Objective: To assess intraoperative faculty-resident behaviors and determine if faculty entrustment is associated with resident entrustability across 4 university-based surgical training programs. Design, Setting, and Participants: This cross-sectional study was conducted at 4 university-based surgical training programs from October 2018 to May 2022. OpTrust, a validated tool designed to assess both intraoperative faculty entrustment and resident entrustability behaviors independently, was used to assess faculty-resident interactions. A total of 94 faculty and 129 residents were observed. Purposeful sampling was used to create variation in type of operation performed, case difficulty, faculty-resident pairings, faculty experience, and resident training level. Main Outcomes and Measures: Observed resident entrustability scores (scale 1-4, with 4 indicating full entrustability) were compared with reported measures (faculty level, case difficulty, resident postgraduate year [PGY], resident gender, observation month) and observed faculty entrustment scores (scale 1-4, with 4 indicating full entrustment). Path analysis was used to explore direct and indirect effects of the predictors. Associations between resident entrustability and faculty entrustment scores were assessed by pairwise Pearson correlation coefficients. Results: A total of 338 cases were observed. Cases observed were evenly distributed by faculty experience (1-5 years' experience: 67 [20.9%]; 6-14 years' experience: 186 [58%]; ≥15 years' experience: 67 [20.9%]), resident PGY (PGY 1: 28 [8%]; PGY 2: 74 [22%]; PGY 3: 64 [19%]; PGY 4: 40 [12%]; PGY 5: 97 [29%]; ≥PGY 6: 36 [11%]), and resident gender (female: 183 [54%]; male: 154 [46%]). At the univariate level, PGY (mean [SD] resident entrustability score range, 1.44 [0.46] for PGY 1 to 3.24 [0.65] for PGY 6; F = 38.92; P < .001) and faculty entrustment (2.55 [0.86]; R2 = 0.94; P < .001) were significantly associated with resident entrustablity. Path analysis demonstrated that faculty entrustment was associated with resident entrustability and that the association of PGY with resident entrustability was mediated by faculty entrustment at all 4 institutions. Conclusions and Relevance: Faculty entrustment remained associated with resident entrustability across various surgical training programs. These findings suggest that efforts to develop faculty entrustment behaviors may enhance intraoperative teaching and resident progression by promoting resident entrustability.

Enhancing the Value of Surgical Entrustable Professional Activities through Integrative Learning Analytics.

OBJECTIVE: To demonstrate the value of integrating surgical resident Entrustable Professional Activity (EPA) data into a learning analytics platform that provides meaningful feedback for formative and summative decision-making. DESIGN: Description of the Surgical Entrustable Professional Activities (SEPA) analytics dashboard, and examples of summary analytics and intuitive display features. SETTING: Department of Surgery, University of Wisconsin Hospital and Clinics. PARTICIPANTS: Surgery residents, faculty, and residency program administrators. RESULTS: We outline the major functionalities of the SEPA dashboard and offer concrete examples of how these features are utilized by various stakeholders to support progressive entrustment decisions for surgical residents. CONCLUSIONS: Our intuitive analytics platform allows for seamless integration of SEPA microassessment data to support Clinical Competency Committee (CCC) decisions for resident evaluation and provides point of training feedback to faculty and trainees in support of progressive autonomy.

Implementation of entrustable professional activities into fellowship council accredited programs: a pilot project.

OBJECTIVE: The Fellowship Council (FC) is transitioning to a competency-based medical education (CBME) model, including the introduction of Entrustable Professional Activities (EPAs) for training and assessment of Fellows. This study describes the implementation process employed by the FC during a ten-month pilot project and presents data regarding feasibility and perceived value. METHODS: The FC coordinated the development of EPAs in collaboration with the sponsoring societies for Advanced GI/MIS, Bariatrics, Foregut, Endoscopy and Hepatopancreaticobiliary (HPB) fellowships encompassing the preoperative, intraoperative, and postoperative phases of care for key competencies. Fifteen accredited fellowship programs participated in this project. The assessments were collected through a unique platform on the FC website. Programs were asked to convene a Clinical Competency Committee (CCC) on a quarterly basis. The pilot group met monthly to support and improve the process. An exit survey evaluated the perceived value of EPAs. RESULTS: The 15 participating programs included 18 fellows and 106 faculty. A total of 655 assessments were initiated with 429 (65%) completed. The average (SD) number of EPAs completed for each fellow was 24(18); range 0-72. Intraoperative EPAs were preferentially completed (71%). The average(SD) time for both the fellow and faculty to complete an EPA was 27(78) hours. Engagement increased from 39% of fellows completing at least one EPA in September to 72% in December and declining to 50% in May. Entrustment level increased from 6% of EPAs evaluated as "Practice Ready" in September to 75% in June. The exit survey was returned by 63% of faculty and 72% of fellows. Overall, 46% of fellows and 74% of program directors recommended full-scale implementation of the EPA framework. CONCLUSION: A competency-based assessment framework was developed by the FC and piloted in several programs. Participation was variable and required ongoing strategies to address barriers. The pilot project has prepared the FC to introduce CBME across all FC training programs.

TrustEd: A Tool for Developing Intraoperative Entrustment Skills.

OBJECTIVE: Toolkits to assess progressive resident autonomy are integral to the movement toward competency-based surgical education. OpTrust is one such tool validated for intraoperative assessment of both faculty and resident entrustment behaviors. We developed a supplementary tool to OpTrust that would aid faculty and residents in making meaningful improvements in entrustment behavior by providing talking points and reflection items tailored to different motivational styles as defined by Regulatory Focus Theory (RFT). DESIGN: Existing literature about surgical entrustment was used to build a list of sample dialogue and self-reflection items to use in the operating room. This list was distributed as a survey to individuals familiar with OpTrust and RFT, asking them to categorize each item as Promotion-oriented, Prevention-oriented, or Either. The respondents then met to discuss survey items that did not reach a consensus until the group agreed on their categorization. SETTING: University of Wisconsin, School of Medicine and Public Health, Madison, Wisconsin Michigan Medicine, Ann Arbor, Michigan PARTICIPANTS: Clinician and education researchers familiar with intraoperative entrustment and RFT RESULTS: Eight respondents completed the survey categorizing the talking points and reflection items by RFT (100% response rate). Six of these respondents attended the additional meeting to discuss discordant items. The input from this panel was used to develop "TrustEd," the supplementary tool that faculty and residents can quickly reference before beginning a case. CONCLUSION: Although tools such as OpTrust allow intraoperative entrustment behaviors to be quantified, TrustEd offers concrete strategies for faculty and residents who are interested in improving those behaviors over time. Further study is needed to assess whether the use of TrustEd does in fact lead to durable behavior change and improvement in OpTrust scores.

Imaging Assessment of Pancreatic Cancer Resectability After Neoadjuvant Therapy: AJR Expert Panel Narrative Review.

Despite important innovations in the treatment of pancreatic ductal adenocarcinoma (PDAC), PDAC remains a disease with poor prognosis and high mortality. A key area for potential improvement in the management of PDAC, aside from earlier detection in patients with treatable disease, is the improved ability of imaging techniques to differentiate treatment response after neoadjuvant therapy (NAT) from worsening disease. It is well established that current imaging techniques cannot reliably make this distinction. This narrative review provides an update on the imaging assessment of pancreatic cancer resectability after NAT. Current definitions of borderline resectable PDAC, as well as implications for determining likely patient benefit from NAT, are described. Challenges associated with PDAC pathologic evaluation and surgical decision making that are of relevance to radiologists are discussed. Also explored are the specific limitations of imaging in differentiating the response after NAT from stable or worsening disease, including issues relating to protocol optimization, tumor size assessment, vascular assessment, and liver metastasis detection. The roles of MRI as well as PET and/or hybrid imaging are considered. Finally, a short PDAC reporting template is provided for use after NAT. The highlighted methods seek to improve radiologists' assessment of PDAC treatment response after NAT.

Increased Intraoperative Faculty Entrustment and Resident Entrustability Does Not Compromise Patient Outcomes After General Surgery Procedures.

BACKGROUND: Intraoperative resident autonomy has been compromised secondary to expectations for increased supervision without defined parameters for safe progressive independence, diffusion of training experience, and more to learn with less time. Surgical residents who are insufficiently entrusted during training attain less autonomy, confidence, and even clinical competency, potentially affecting future patient outcomes. OBJECTIVE: To determine if OpTrust, an educational intervention for increasing intraoperative faculty entrustment and resident entrustability, negatively impacts patient outcomes after general surgery procedures. METHODS: Surgical faculty and residents received OpTrust training and instruction to promote intraoperative faculty entrustment and resident entrustability. A post-intervention OpTrust cohort was compared to historical and pre-intervention OpTrust cohorts. Multivariable logistic and negative binomial regression was used to evaluate the impact of the OpTrust intervention and time on patient outcomes. SETTING: Single tertiary academic center. PARTICIPANTS: General surgery faculty and residents. MAIN OUTCOMES AND MEASURES: Thirty-day postoperative outcomes, including mortality, any complication, reoperation, readmission, and length of stay. RESULTS: A total of 8890 surgical procedures were included. After risk adjustment, overall patient outcomes were similar. Multivariable regression estimating the effect of the OpTrust intervention and time revealed similar patient outcomes with no increased risk (P > 0.05) of mortality {odds ratio (OR), 2.23 [95% confidence interval (CI), 0.87-5.6]}, any complication [OR, 0.98 (95% CI, 0.76-1.3)], reoperation [OR, 0.65 (95% CI, 0.42-1.0)], readmission [OR, 0.82 (95% CI, 0.57-1.2)], and length of stay [OR, 0.99 (95% CI, 0.86-1.1)] compared to the historic and pre-intervention OpTrust cohorts. CONCLUSIONS: OpTrust, an educational intervention to increase faculty entrustment and resident entrustability, does not compromise postoperative patient outcomes. Integrating faculty and resident development to further enhance entrustment and entrustability through OpTrust may help facilitate increased resident autonomy within the safety net of surgical training without negatively impacting clinical outcomes.

Gender Differences in Entrustable Professional Activity Evaluations of General Surgery Residents.

OBJECTIVE: To determine differences in entrustable professional activity (EPA) assessments between male and female general surgery residents. SUMMARY BACKGROUND DATA: Evaluations play a critical role in career advancement for physicians. However, female physicians in training receive lower evaluations and underrate their own performance. Competency-based assessment frameworks, such as EPAs, may help address gender bias in surgery by linking evaluations to specific, observable behaviors. METHODS: In this cohort study, EPA assessments were collected from July 2018 to May 2020. The effect of resident sex on EPA entrustment levels was analyzed using multiple linear and ordered logistic regressions. Narrative comments were analyzed using latent dirichlet allocation to identify topics correlated with resident sex. RESULTS: Of the 2480 EPAs, 1230 EPAs were submitted by faculty and 1250 were submitted by residents. After controlling for confounding factors, faculty evaluations of residents were not impacted by resident sex (estimate = 0.09, P = 0.08). However, female residents rated themselves lower by 0.29 (on a 0-4 scale) compared to their male counterparts (P < 0.001). Within narrative assessments, topics associated with resident sex demonstrated that female residents focus on the "guidance" and "supervision" they received while performing an EPA, while male residents were more likely to report "independent" action. CONCLUSIONS: Faculty assessments showed no difference in EPA levels between male and female residents. Female residents rate themselves lower by nearly an entire post graduate year (PGY) level compared to male residents. Latent dirichlet allocation -identified topics suggest this difference in self-assessment is related to differences in perception of autonomy.

Peripheral nerve blocks with liposomal bupivacaine are associated with increased opioid use compared to thoracic epidural in patients with an epigastric incision.

BACKGROUND: Thoracic epidurals are commonly recommended in enhanced recovery protocols, though they may cause hypotension and urinary retention. Peripheral nerve blocks using liposomal bupivacaine are a potential alternative, though they have not been extensively studied in major cancer operations with an epigastric incision. METHODS: We conducted a retrospective review of prospectively collected data following the transition from thoracic epidural to liposomal peripheral nerve blocks in patients undergoing major oncologic surgery. Patients receiving peripheral nerve blocks were compared to those receiving thoracic epidural. Outcome variables included postoperative opioid use (milligram morphine equivalents [MME]), severe pain, and postoperative complications. RESULTS: Forty-seven of 102 patients studied (46%) received peripheral nerve blocks. Opioid use was higher in the peripheral nerve block group during the 0-24 h (116 vs. 94 MME, p = 0.04) and 24-48 h postoperative period (94 vs. 23 MME, p < 0.01). There was no significant difference in severe pain, hypotension, urinary retention, or ileus. Peripheral nerve blocks were associated with earlier ambulation (1 vs. 2 days, p = 0.04), though other milestones were similar. CONCLUSIONS: Liposomal peripheral nerve blocks were associated with increased opioid use compared to thoracic epidural. On the basis of our results, thoracic epidural might be preferred in surgical oncology patients with an epigastric incision.

Trainee Reliance on Public Service Loan Forgiveness.

OBJECTIVE: The Public Service Loan Forgiveness (PSLF) program is an option to trainees to help alleviate federal education debt. The prevalence of PSLF utilization and how this may impact career decisions of trainees is unknown. The purpose of this study was to understand the prevalence, impact, and understanding of PSLF participation on trainees. DESIGN: IRB-approved anonymous survey asking study subjects to report demographics, financial status, and reliance on PSLF. In addition, study subjects were asked to report their participation in PSLF, the possible impact of PSLF participation on career decisions, and to identify the qualifications needed to complete PSLF. SETTING: Online anonymous survey. PARTICIPANTS: The survey was offered to all physician trainees in all specialties at the University of Texas, Southwestern, University of Wisconsin, Madison, and University of Michigan, Ann Arbor. RESULTS: There were 934 respondents, yielding a 37.6% response rate. A total of 416/934 (44.5%) respondents were actively or planning on participating in the PSLF program with 175/934 (18.7%) belonging to a surgical specialty. Those belonging to a surgical specialty were more likely to be PSLF participants compared to medical specialties (53.1% versus 42.6%, p = 0.01). For those participating in PSLF, 82/416 (19.7%) stated this participation impacted career decisions. A total of 275/934 (29.4%) respondents obtained and 437/934 (46.8%) wanted to receive formal training/lectures in regards to the PSLF program. Of those actively or planning on participating in the PSLF program, only 58/416 (13.9%) were able to correctly identify all of the qualifications/criteria to complete the program. CONCLUSIONS: A large proportion of trainees rely on the PSLF program for education loan forgiveness with approximately 20% reporting participation impacted career decisions. Additionally, the majority may not fully understand PSLF criteria. Programs should strongly consider providing a formal education regarding PSLF to their trainees.

Facilitating Surveillance of Incidental Findings Using a Novel Reporting Template: Proof of Concept in Patients With Pancreatic Abnormalities.

OBJECTIVE: To determine the surveillance impact of utilizing a discrete field in structured radiology reports in patients with incidental pancreatic findings. METHODS: We implemented a dictation template containing a discrete structured field element to auto-trigger listing of patients with incidental pancreatic findings on a pancreas clinic registry in the electronic health record. We isolated CT and MRI reports with incidental pancreatic findings over a 24-month period. We stratified patients by presence or absence of the discrete field element in reports (flagged versus unflagged) and evaluated the impact of report flagging on likelihood of clinic follow-up, follow-up imaging, endoscopic ultrasound, surgical intervention, genetics referral, obtaining pathologic diagnosis, and time interval between index imaging to various outcomes. RESULTS: Patients with flagged reports were more likely to be seen or discussed in a pancreas clinic compared with those with unflagged reports (189 of 376, 50.3% versus 79 of 474, 16.7%; P <. 001). Patients with flagged reports were more likely to get follow-up imaging than patients with unflagged reports (188 of 376, 50.0% versus 121 of 474, 25.5%; P < .001) and were more likely to undergo appropriate management of actionable findings compared with patients in the unflagged group (23 of 62, 37.1% versus 28 of 129, 21.7%; P = .036). DISCUSSION: Implementation of a structured discrete field element for reporting of patients with incidental pancreatic findings had positive impact on surveillance measures and can be applied in other organ systems with established surveillance guidelines to standardize patient care.

A Phased Approach: The General Surgery Experience Adopting Entrustable Professional Activities in the United States.

Entrustable professional activities (EPAs) have been increasingly used as an assessment framework to formally capture the myriad ad hoc entrustment decisions that occur on a daily basis in clinical settings with learners present. Following the definition of Core EPAs for Entering Residency by the Association of American Medical Colleges in 2014, the American Board of Surgery (ABS) began to explore the utility of EPAs as a framework to support competency-based resident education within general surgery in 2016. As the complement of EPAs drafted for a specialty serve to define the core tasks of a professional within that discipline, initial efforts to define the entire scope of general surgery were fraught with difficulty as no commonly accepted definition of a general surgeon currently exists. Opting to prioritize a pilot of the EPA conceptual framework within surgical training rather than defining the entirety of the specialty, ABS leaders identified 5 EPAs that represent a core of general surgery with which to begin. This article details the process of choosing the initial set of EPAs and provides a roadmap for other disciplines interested in testing the feasibility of this assessment framework while garnering buy-in among the community of educators. Future steps, including implementation of the existing 5 EPAs beyond the initial pilot sites and drafting and implementation of the additional complement of EPAs, are also described.

OpTrust: An Effective Educational Bundle for Enhancing Faculty-resident Intraoperative Entrustment.

OBJECTIVE: The purpose of this study was to measure the efficacy of a novel faculty and resident educational bundle focused on development of faculty-resident behaviors and entrustment in the operating room. SUMMARY BACKGROUND DATA: As surgical training environments are orienting to entrustable professional activities (EPAs), successful transitions to this model will require significant faculty and resident development. Identifying an effective educational initiative which prepares faculty and residents for optimizing assessment, teaching, learning, and interacting in this model is critical. METHODS: From September 2015 to June 2017, an experimental study was conducted in the Department of Surgery at the University of Michigan Health System (UMHS). Case observations took place across general, plastic, thoracic, and vascular surgical specialties. A total of 117 operating room observations were conducted during Phase I of the study and 108 operating room observations were conducted during Phase II following the educational intervention. Entrustment behaviors were rated for 56 faculty and 73 resident participants using OpTrust, a validated intraoperative entrustment instrument. RESULTS: Multiple regression analysis showed a significant increase in faculty entrustment (Phase I = 2.32 vs Phase II = 2.56, P < 0.027) and resident entrustability (Phase I = 2.16 vs Phase II = 2.40, P < 0.029) scores following exposure to the educational intervention. CONCLUSIONS: Our study shows improved intraoperative entrustment following implementation of faculty and resident development, indicating the efficacy of this innovative educational bundle. This represents a crucial component in the implementation of a competency-based assessment framework like EPAs.

Clinical and Cost Profile of Controlled Grade B Postoperative Pancreatic Fistula: Rationale for Their Consideration as Low Risk.

BACKGROUND: Despite standardization, the 2016 ISGPF criteria are limited by their wider applicability and oversimplification of grade B POPF. This work applied the 2016 ISGPF grading criteria within a US academic cancer center to verify clinical and fiscal distinctions and sought to improve grading criteria for grade B POPF. METHODS: The 2008-2018 cost and NSQIP data from pancreaticoduodenectomy to postoperative day 90 were merged. All POPFs were coded by 2016 ISGPF criteria. The Clavien-Dindo Classification (CD) defined complication severity. On sub-analyses, grade B POPFs were divided into those with adequate drainage and those requiring additional drainage. Chi-square, ANOVA, and Fisher's least significant difference test were employed. RESULTS: Two hundred thirty-two patients were in the final analyses, 72 (31%) of whom had POPFs: 16 (7%) biochemical leaks, 54 (23%) grade B (28% required additional drainage), and 2 (1%) grade C. There was no significant difference in length of stay, CD, readmission, or cost in patients without a POPF, with biochemical leak or grade B POPF. On sub-analyses, 92% of adequately drained grade B POPFs had CD 1-2 and readmission equivalent to patients without POPF (p > 0.05). One hundred percent of grade B POPF requiring drainage had CD 3-4a, and 67% were readmitted. Cost was significantly increased in grade B POPF requiring additional drainage (p = 0.02) and grade C POPF (p < 0.01). CONCLUSIONS: This analysis did not confirm an incremental increase in morbidity and cost with POPF grade. Sub-analyses enabled accurate clinical and cost distinctions in grade B POPF; adequately drained grade B POPF are low risk and clinically insignificant.

Natural language processing and entrustable professional activity text feedback in surgery: A machine learning model of resident autonomy.

BACKGROUND: Entrustable Professional Activities (EPAs) contain narrative 'entrustment roadmaps' designed to describe specific behaviors associated with different entrustment levels. However, these roadmaps were created using expert committee consensus, with little data available for guidance. Analysis of actual EPA assessment narrative comments using natural language processing may enhance our understanding of resident entrustment in actual practice. METHODS: All text comments associated with EPA microassessments at a single institution were combined. EPA-entrustment level pairs (e.g. Gallbladder Disease-Level 1) were identified as documents. Latent Dirichlet Allocation (LDA), a common machine learning algorithm, was used to identify latent topics in the documents associated with a single EPA. These topics were then reviewed for interpretability by human raters. RESULTS: Over 18 months, 1015 faculty EPA microassessments were collected from 64 faculty for 80 residents. LDA analysis identified topics that mapped 1:1 to EPA entrustment levels (Gammas >0.99). These LDA topics appeared to trend coherently with entrustment levels (words demonstrating high entrustment were consistently found in high entrustment topics, word demonstrating low entrustment were found in low entrustment topics). CONCLUSIONS: LDA is capable of identifying topics relevant to progressive surgical entrustment and autonomy in EPA comments. These topics provide insight into key behaviors that drive different level of resident autonomy and may allow for data-driven revision of EPA entrustment maps.

Comparison of the Perspectives of Medical Students and Residents on the Surgery Learning Environment.

BACKGROUND: The learning environment plays a critical role in learners' satisfaction and outcomes. However, we often lack insight into learners' perceptions and assessments of these environments. It can be difficult to discern learners' expectations, making their input critical. When medical students and surgery residents are asked to evaluate their teachers, what do they focus on? MATERIALS AND METHODS: Open-ended comments from medical students' evaluations of residents and attending surgeons and from residents' evaluations of attendings during the 2016-2017 academic year were analyzed. Content analysis was used, and codes derived from the data. A matrix of theme by learner role was created to distinguish differences between medical student and resident learners. Subthemes were grouped based on similarity into high-order themes. RESULTS: Two overarching themes were Creating a positive environment for learning by modeling professional behaviors and Intentionally engaging learners in training and educational opportunities. Medical students and residents made similar comments for the subthemes of appropriate demeanor, tone and dialog, respect, effective direct instruction, feedback, debriefing, giving appropriate levels of autonomy, and their expectations as team members on a service. Differences existed in the subthemes of punctuality, using evidence, clinical knowledge, efficiency, direct interactions with patients, learning outcomes, and career decisions. CONCLUSIONS: Faculty development efforts should target professional communication, execution of teaching skills, and relationships among surgeons, other providers, and patients. Attendings should make efforts to discuss their approach to clinical decision making and patient interactions and help residents and medical students voice their opinions and questions through trusting adult learner-teacher relationships.

Early vs Late Readmissions in Pancreaticoduodenectomy Patients: Recognizing Comprehensive Episodic Cost to Help Guide Bundled Payment Plans and Hospital Resource Allocation.

INTRODUCTION: Previous studies on readmission cost in pancreaticoduodenectomy patients use estimated cost data and do not delineate etiology or cost differences between early and late readmissions. We sought to identify relationships between postoperative complication type and readmission timing and cost in pancreaticoduodenectomy patients. METHODS: Hospital cost data from date of discharge to postoperative day 90 were merged with 2008-2018 NSQIP data. Early readmission was within 30 days of surgery, and late readmission was 30 to 90 days from surgery. Regression analyses for readmission controlled for patient comorbidities, complications, and surgeon. RESULTS: Of 230 patients included, 58 (25%) were readmitted. The mean early and late readmission costs were $18,365 ± $20,262 and $24,965 ± $34,435, respectively. Early readmission was associated with index stay deep vein thrombosis (p < 0.01), delayed gastric emptying (p < 0.01), and grade B pancreatic fistula (p < 0.01). High-cost early readmission had long hospital stays or invasive procedures. Common late readmission diagnoses were grade B pancreatic fistula requiring drainage (n = 5, 14%), failure to thrive (n = 4, 14%), and bowel obstruction requiring operation (n = 3, 11%). High-cost late readmissions were associated with chronic complications requiring reoperation. CONCLUSION: Early and late readmissions following pancreaticoduodenectomy differ in both etiology and cost. Early readmission and cost are driven by common complications requiring percutaneous intervention while late readmission and cost are driven by chronic complications and reoperation. Late readmissions are frequent and a significant source of resource utilization. Negotiations of bundled care payment plans should account for significant late readmission resource utilization.