Response Process Validity Evidence for Video Commentary Assessment in Surgery: A Qualitative Study.

OBJECTIVES: Well-developed mental representations of a task are fundamental to proficient performance. 'Video Commentary' (VC) is a novel assessment intended to measure mental representations of surgical tasks that would reflect an important aspect of task proficiency. Whether examinees' actual response processes align with this intent remains unknown. As part of ongoing validation of the assessment, we sought to understand examinees' response processes in VC. DESIGN: Grounded theory qualitative study. In 2019, residents were interviewed about their understanding of and approach to VC. Using grounded theory, we created a theoretical model explaining relationships among factors that influence residents' response processes and performance. Residents' perceived purpose of VC was also explored using Likert-type questions. SETTING: Academic surgical residency program. PARTICIPANTS: Forty-eight surgical residents (PGY-1 to PGY-5). RESULTS: Analysis of narrative comments indicated that residents' perceived purposes of VC generally align with the educator's intent. Resident response processes are influenced by test characteristics, residents' perception and understanding of VC, and residents' personal characteristics. Four strategies seem to guide how residents respond, namely a focus on speed, points, logic, and relevance. Quantitative results indicated residents believe VC scores reflect their ability to speak quickly, ability to think quickly, and knowledge of anatomy (mean = 5.0, 4.5, and 4.4 respectively [1 = strongly disagree, 6 = strongly agree]). PGY-1 and PGY-2 residents tend to focus on naming facts whereas PGY-4 and PGY-5 residents focus on providing comprehensive descriptions. CONCLUSIONS: Residents generally have an accurate understanding of the purpose of VC. However, their use of different approaches could represent a threat to validity. The response strategies of speed, points, logic, and relevance may inform other clinical skills assessments.

Video Commentary & Machine Learning: Tell Me What You See, I Tell You Who You Are.

BACKGROUND & OBJECTIVE: Teaching and assessment of complex problem solving are a challenge for medical education. Integrating Machine Learning (ML) into medical education has the potential to revolutionize teaching and assessment of these problem-solving processes. In order to demonstrate possible applications of ML to education, we sought to apply ML in the context of a structured Video Commentary (VC) assessment, using ML to predict residents' training level. SETTING: A secondary analysis of multi-institutional, IRB approved study. Participants had completed the VC assessment consisting of 13 short (20-40 seconds) operative video clips. They were scored in real-time using an extensive checklist by an experienced proctor in the assessment. A ML model was developed using TensorFlow and Keras. The individual scores of the 13 video clips from the VC assessment were used as the inputs for the ML model as well as for regression analysis. PARTICIPANTS: A total of 81 surgical residents of all postgraduate years (PGY) 1-5 from 7 institutions constituted the study sample. RESULTS: Scores from individual VC clips were strongly positively correlated with PGY level (p = 0.001). Some video clips were identified to be strongly correlated with a higher total score on the assessment; others had significant influence when used to predict trainees' PGY levels. Using a supervised machine learning model to predict trainees' PGY resulted in a 40% improvement over traditional statistical analysis. CONCLUSIONS: Performing better in a few select video clips was key to obtaining a higher total score but not necessarily foretelling of a higher PGY level. The use of the total score as a sole measure may fail to detect deeper relationships. Our ML model is a promising tool in gauging learners' levels on an assessment as extensive as VC. The model managed to approximate residents' PGY levels with a lower MAE than using traditional statistics. Further investigations with larger datasets are needed.

International Medical Graduates are Comparable to American Medical Graduates as General Surgery Interns.

BACKGROUND: International medical graduates (IMGs) are often relegated to preliminary positions in general surgery (GS) owing to uncertainties about the candidate's performance in the American healthcare setting. We aimed to determine the comparative performance of IMGs and American medical graduates (AMGs) at baseline and assess these trends over the course of their GS internship. METHODS: Evaluations of all IMG preliminary and AMG categorical interns from 2013 to 2017 at our GS residency program were obtained from three faculty members to score overall performance, technical skills, interpersonal communication, and medical knowledge on a 10-point Likert scale. Scores on the American Board of Surgery In-Training Exam, an in-house preparation test, United States Medical Licensing Exam, and performance during the biannual multistation objective assessments were compared between the two resident groups. RESULTS: Seventy-two interns (28 [39%] AMG categorical and 44 [61%] IMG preliminary) met inclusion criteria. The AMG group had significantly higher median Step 1 and Step 2 scores compared with our IMG group (243 versus 231, P = 0.002, and 250 versus 246, P = 0.03, respectively).Although in-house preparation test scores were higher among IMGs (median [interquartile range] of 36 [33-40] among AMGs and 38 [34-45] among IMGs; P = 0.002), there were no statistically significant differences between the American Board of Surgery In-Training Exam scores of the two groups. The median scores for the four faculty evaluation components were similar between the AMG (7, 8, 7, 7) and IMG resident groups (7, 7, 7, 7; P = nonsignificant). IMGs scored significantly higher in both biannual multistation objective assessments than AMGs (median [interquartile range] July: 59 [47-91] versus 55 [37-62], P = 0.005; January: 103 [86-116] versus 91 [87-104], P = 0.03). CONCLUSIONS: It is reassuring to confirm that no matter where they are from, great candidates can perform well as surgical interns in a GS training program.

Differences in outcomes of bilateral adrenalectomy in patients with ectopic ACTH producing tumor of known and unknown origin.

BACKGROUND: Endogenous Cushing syndrome (CS) can be caused by ectopic corticotropin-producing tumors of known (EK) and unknown origin (EU). Bilateral adrenalectomy (BA) can be used as definite treatment of hypercortisolism in such cases. This study compared patients undergoing BA for CS secondary to EK vs EU. METHODS: Retrospective review (1995-2017) of patients undergoing BA due to EK or EU. We analyzed demographic characteristics, laboratory values, intraoperative variables, surgical outcomes, and survival. RESULTS: 48 patients (26 EU, 22 EK) were identified. Serum cortisol and ACTH concentrations were similar. 92% of BA for EU were performed minimally invasively vs 77% for EK, P = 0.22. Complications occurred in 19% of EU and 4.5% EK, P = 0.2. Mean survival was 4.3 years for EU and 4.0 years for EK without difference in all-cause mortality P = 0.63. CONCLUSION: BA cure rate was 100% for CS in EU and EK. Morbidity, long term and all-cause mortality differences were not statistically significant between EK and EU.

Is same-day discharge associated with increased 30-day postoperative complications and readmissions in patients undergoing laparoscopic adrenalectomy?

BACKGROUND: Advances in minimally invasive surgery and perioperative care have decreased substantially the duration of time that patients spend recovering in hospital, with many laparoscopic procedures now being performed on an ambulatory basis. There are limited studies, however, on same-day discharge after laparoscopic adrenalectomy. The objectives of this study were to investigate the outcomes and trends of ambulatory laparoscopic adrenalectomy in a multicenter cohort of patients. METHODS: Adult patients who underwent elective laparoscopic adrenalectomy between 2005 and 2016 were identified in the database of the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). Baseline demographics and 30-day outcomes were compared between patients who underwent ambulatory laparoscopic adrenalectomy and those who were discharged after an inpatient stay. Multivariable logistic regression and Cox proportional hazards modelling were used to investigate the association between same-day discharge and 30-day complications and unplanned readmissions. RESULTS: Of the 4,807 patients included in the study, 88 (1.8%) underwent ambulatory laparoscopic adrenalectomy and 4,719 (98.2%) were admitted after the adrenalectomy. The same-day discharge group contained fewer obese patients (37.2% vs 50%; P = .04), a lesser proportion of American Society of Anesthesiologists class III patients (45.5% vs 61%; P = .003), and more patients with primary aldosteronism (14.8% vs 6%; P = .002) compared with the inpatient group. After adjustment for confounders, same-day discharge was not associated with 30-day overall complications (OR 1.17, 95% CI 0.35-3.85; P = .80) or unplanned readmissions (HR 2.77, 95% CI 0.86-8.96; P = .09). The percentage of laparoscopic adrenalectomies performed on an ambulatory basis at hospitals participating in the ACS NSQIP remained low throughout the study period (0-3.1% per year) with no evidence of an increasing trend over time (P = .21). CONCLUSION: Ambulatory laparoscopic adrenalectomy is a safe and feasible alternative to inpatient hospitalization in selected patients. Further study is needed to determine the cost savings, barriers to uptake, and optimal selection criteria for this approach.

Video Commentary: Rapid-Fire Assessment of Surgical Cognitive Skills.

OBJECTIVE: We describe the use of "Video Commentary", an assessment using a set of operative videos, to assess trainees' surgical cognitive skills (operative knowledge, spatial awareness, and surgical insight). DESIGN & SETTING: The Video Commentary assessment has been routinely administered to Postgraduate Year (PGY) 1-5 general surgery residents since 2014 as part of a biannual multistation, OSCE-type exam at Mayo Clinic, Rochester, MN. Video Commentary is a rapid-fire, 6-minute assessment, where trainees watch a series of 20 to 30 second operative video clips and comment on them as they play. Each clip varies in procedure, approach, difficulty, and complexity. The combination of video clips differs according to trainees' PGY level except for a few videos that overlap among PGY groups. The name of the procedure is provided at the beginning of each clip with a countdown timer showing in the corner of the screen. A comprehensive checklist is used to score trainees' performance in real-time. DISCUSSION: Assessment of trainees of different levels and staff surgeons show a positive correlation with the experience level (p = 0.0001). The assessment provides a safe alternative to assess trainees in the operating room and encourages them to become more effective communicators. With the use of technology, large video databases can be created to provide just-in-time tailored feedback to the trainees. CONCLUSIONS: Video Commentary can serve as a time and resource-efficient assessment of trainees' surgical cognitive skills and insight. The use and demand of real-time commentary on operative videos may provide a viable approach to help surgeon educators determine trainees' baseline, progression, and readiness to advance.

APDS Consensus Statement: Ideal Senior Medical Student Experiences for Preparedness for General Surgery Internship.

OBJECTIVE: Guide optimal standards on ideal senior medical student experiences for preparedness for general surgery internship DESIGN: Work product of task force, approved by the Association of Program Directors in Surgery CONCLUSION: General surgery rotations should mirror the learning and working environment of a surgical intern. Opportunities should mimic the next phase of learning to help guide informed decisions regarding entrustability for entry into residency training. These opportunities will also help identify students who may have an aptitude for pursuing a general surgery internship. Students should achieve entrustability in Association of American Medical Colleges Core Entrustable Professional Activities (EPAs); curricula should align Core EPAs and modified American Board of Surgery EPAs to guide essential general surgery components. Experiences should include required night, holiday, and/or weekend shifts, a dedicated critical care experience, and a resident preparatory curriculum focusing on nontechnical and essential technical skills. We encourage the opportunity for additional surgical mentorship and subspecialty experience through Surgical Interest Groups or Surgical Honors or Specialty Tracks.

Occurrence and Natural History of Thyroid Cancer in Patients with Cowden Syndrome.

INTRODUCTION: Cowden syndrome patients are at an increased risk of developing differentiated thyroid carcinoma (DTC). There are limited studies of thyroid cancer biology within this population. We aimed to describe the frequency and progression of DTC in this population. METHOD: We reviewed all patients with clinically or genetically confirmed Cowden syndrome treated at our institution between 1979 and 2017. Presentation, operative details, pathology, and adjuvant treatment for all thyroid diseases were analyzed. RESULTS: Forty-seven patients were identified. Thirty-seven received a clinical diagnosis while 10 tested positive on genetic testing for the PTEN gene. Average follow-up time was 10 years. Twenty-three patients underwent a total thyroidectomy. Nine patients were diagnosed with thyroid cancer, with a mean age of 28.5 years, 4 cases of the follicular variant of papillary thyroid cancer (PTC), 4 of PTC, and 1 of cystic PTC. One patient required an interval lateral neck dissection but no patients developed distant metastasis. Fourteen patients underwent thyroidectomy, 12 due to follicular adenoma, Hürthle cell adenoma, or multinodular goiter, 1 due to Graves' disease, and 1 who elected to undergo a prophylactic thyroidectomy. No mortality was associated with thyroid cancer in these patients. CONCLUSION: Patients with Cowden syndrome are at a significantly increased risk of developing DTC and have a younger age of onset. In this study, 20% of the patients with Cowden syndrome were diagnosed with DTC, but distant metastasis and overall mortality were absent. Clinical features were similar to those of sporadic DTC, proving that, apart from screening, Cowden syndrome-related DTC can be treated in similar fashion.

Value of Research Years for International Medical Graduates Applying to General Surgery Residency.

OBJECTIVE: While dedicated research time (DRT) offers international medical graduates (IMGs) exposure to the US healthcare system and helps boost their residency application, it can lengthen time away from clinical activity. We aimed to determine the value of DRT/protected research time for an IMG applying to general surgeryresidency (GSR). DESIGN: Cross-sectional survey. SETTING: Academic, tertiary care center with a large GSR program (Mayo Clinic, Rochester, MN). PARTICIPANTS: IMG applicants to our GSR program (2015-2018) and general surgery program directors (PDs) in the United States. METHODOLOGY: Separate surveys were emailed to all IMG applicants to our GSR program (2015-2018) and general surgery PDs. Surveys to the applicants focused on the details of DRT spent in the US prior to residency and match outcomes. Surveys to the PDs focused on the importance of different aspects of an IMG applicant's research experience. RESULTS: Applicants: 320 applicants responded of which 148 (46%) applicants spent DRT in the US prior to residency. Among the research group, the median (interquartile range [IQR]) number of first author podium and poster presentations from the DRT were 1 (0-4) and 2 (1-5), respectively. At the time of match application, the median (IQR) number of published manuscripts among the research group was higher than that among the applicant group which did not pursue DRT (5[2-12] vs 1[0-2], p < 0.0001). Pursuing DRT and/or the duration of the research did not affect the match outcomes (p > 0.05). PROGRAM DIRECTORS: Seventy-six PDs responded to our survey. A median (IQR) of 2.5 (2-3) publications, and 2 (2-3) presentations per year of DRT were considered "productive." Sixty-three percent of PDs did not recommend IMGs take time off to engage in research; only 54% attributed some importance to an IMG's research experience. CONCLUSIONS: DRT does not seem to be associated with better outcomes for IMGs targeting GSR. Most PDs advocate against taking time off after graduation to engage in research activity. Decisions to pursue DRT should be individualized.

Bilateral Adrenalectomy: Differences between Cushing Disease and Ectopic ACTH-Producing Tumors.

BACKGROUND: Bilateral adrenalectomy (BA) is a curative treatment for hypercortisolism in patients with corticotropin (ACTH)-dependent Cushing syndrome. Indications include refractory Cushing's disease (CD) following failed pituitary surgery, and occult or unresectable ectopic ACTH-producing neoplasms (EA). METHODS: This was a single-center, retrospective review (1995-2017) of patients undergoing BA for CD or EA. Demographics, laboratory findings, and intraoperative and postoperative variables were analyzed. RESULTS: Of 137 patients, 83 (61%) had CD and 54 (39%) had EA; 87% of CD patients were female versus 46% of EA patients (p < 0.0001). Mean age at diagnosis was 40 ± 15 years for CD and 49 ± 18 years for EA (p = 0.004). Preoperative serum cortisol concentrations were higher in the EA cohort (63 ± 40 µg/dL) versus the CD cohort (33 ± 19 µg/dL) [p < 0.001], with no significant differences in serum ACTH. Time from diagnosis until adrenalectomy was 54 ± 69 months for CD versus 4 ± 13 for EA (p = 0.002). Most patients underwent minimally invasive surgery (MIS; 86% CD vs. 75% EA; p = 0.19). There was no difference between the rates of conversion to an open approach, intraoperative blood loss, or operative time between groups, and no difference between complications in CD versus EA (p = 1.0). Five-year survival was significantly shorter among the EA cohort (30% for EA vs. 80% for CD; p < 0.001). CONCLUSION: Patients with EA presented with higher serum cortisol levels compared with patients with CD. EA patients were more likely to require intraoperative transfusion and postoperative intensive care. BA in patients with CD and EA can be performed safely in an MIS fashion, with similar morbidity; however, survival at 5 years was significantly less in the EA cohort.

Not all adrenal incidentalomas require biochemical testing to exclude pheochromocytoma: Mayo clinic experience and a meta-analysis.

BACKGROUND: Excluding a pheochromocytoma is important when a patient presents with an incidentally discovered adrenal mass. However, biochemical testing for pheochromocytoma can be cumbersome, time consuming, or falsely positive. Our objective was to determine if unenhanced computed tomography (CT) imaging alone can be used to rule out pheochromocytoma. METHODS: We performed a retrospective study of all patients with a pathologically confirmed pheochromocytoma and unenhanced CT imaging who were treated at the Mayo Clinic between 1998 and 2016. Additionally, we performed a systematic review and meta-analysis of original studies published after 2005 with patients who had adrenal masses, more than 10 patients with pheochromocytomas, and reported attenuation on unenhanced CT imaging in Hounsfield units (HU). RESULTS: In the Mayo cohort, we identified 186 patients and 199 pheochromocytomas with unenhanced CT imaging. The mean unenhanced CT attenuation was 35±9 HU (range, 15-62), and only 15 tumors had attenuation ≤20 HU. The systematic review identified 26 studies (1,217 tumors), and 23 studies provided a mean unenhanced CT attenuation. The overall mean unenhanced CT attenuation across the studies was 35.6 HU (95% CI, 22.0-49.1 HU). A cutoff of >10 HU had a 100% sensitivity (95% CI, 1.00-1.00) for pheochromocytoma with low heterogeneity between the 21 qualified studies (I2=0%). Sensitivity for pheochromocytoma was 100% and 99% for an unenhanced CT attenuation cutoff of >15 and >20 HU. CONCLUSIONS: Biochemical testing may not be required to exclude pheochromocytoma if an incidental adrenal mass has low attenuation (<10 HU) on unenhanced CT images.

Inspiring Young Learners Toward a Career in Healthcare Using Simulation.

STATEMENT: To inspire young learners toward a career in healthcare and provide them opportunities to learn lifesaving skills, a hospital-based simulation center collaborated with a local middle school to develop a 88-minute simulation-based educational activity. The activity consisted of eight 10-minute stations on lifesaving and basic medical skills. One hundred fifty students participated. Evaluation surveys showed students favored stations with extensive hands-on activities and valued the opportunity to interact with health professionals. Students also reported more interest in science careers after the visit. This collaborative effort is time-efficient and low-budget simulation-based learning experience that had an immediate impact on middle school students.

Group Video Feedback is an Effective and Efficient Tool for Enhancing Skills of Surgical Interns.

BACKGROUND: Our group has shown that personalized video feedback (PVF) is better than a task demonstration video at increasing wound closure skills among incoming surgical interns. However, offering PVF can be time-consuming. We sought to compare the educational effects and time required for group video feedback (GVF) and PVF. METHODS: We have mailed our matched incoming "surgical" interns a "welcome package" in mid-March for the past 3 y. The package includes similar resources each year. Incoming interns were asked to video record themselves performing six tasks three separate times between April 1 and June 15. After each submission in 2016 and 2017, incoming interns received 2 min of personalized feedback on their three separate wound closure videos (PVF). In 2018, incoming interns received 5 min of group-based feedback three separate times covering all six tasks (GVF). We compared performance (July Surgical Olympics) of these six skills against the interns from the previous 2 y who received PVF on only one skill (suturing). RESULTS: Twenty-three incoming interns received the pre-residency package and participated in the 2018 Surgical Olympics. This 2018 GVF class had a higher overall mean score for six stations (31.5 [standard deviation = ±7.7]) than the 2016 and 2017 PVF classes (25.6 [standard deviation = ±8]; P < 0.0001). Knot tying ability and suturing skill were similar between the classes. The GVF group performed better on the remaining four skills . The total time of surgical staff and educators spent per class in 2018 (GVF class) was 30 min and includes six tasks compared with 276 min of effort in each of the 2016 and 2017 classes (PVF included one task). CONCLUSIONS: GVF and PVF as a component of preemptive training had the same effect on improving suturing skills among the interns. GVF required less educator editing and voice over time. GVF is effective and efficient in enhancing incoming interns' performance in multiple skills.

Workload Differentiates Breast Surgical Procedures: NSM Associated with Higher Workload Demand than SSM.

BACKGROUND: Breast surgery has evolved with more focus on improving cosmetic outcomes, which requires increased operative time and technical complexity. Implications of these technical advances in surgery for the surgeon are unclear, but they may increase intraoperative demands, both mentally and physically. We prospectively evaluated mental and physical demand across breast surgery procedures, and compared surgeon ergonomic risk between nipple-sparing (NSM) and skin-sparing mastectomy (SSM) using subjective and objective measures. METHODS: From May 2017 to July 2017, breast surgeons completed modified NASA-Task Load Index (TLX) workload surveys after cases. From January 2018 to July 2018, surgeons completed workload surveys and wore inertial measurement units to evaluate their postures during NSM and SSM cases. Mean angles of surgical postures, ergonomic risk, survey items, and patient factors were analyzed. RESULTS: Procedural duration was moderately related to surgeon frustration, mental and physical demand, and fatigue (p < 0.001). NSMs were rated 23% more physically demanding (M = 13.3, SD = 4.3) and demanded 28% more effort (M = 14.4, SD = 4.6) than SSMs (M = 10.8, SD = 4.7; M = 11.8, SD = 5.0). Incision type was a contributing factor in workload and procedural difficulty. Left arm mean angle was significantly greater for NSM (M = 30.1 degrees, SD = 6.6) than SSMs (M = 18.2 degrees, SD = 4.3). A higher musculoskeletal disorder risk score for the trunk was significantly associated with higher surgeon physical workload (p = 0.02). CONCLUSION: Nipple-sparing mastectomy required the highest surgeon-reported workload of all breast procedures, including physical demand and effort. Objective measures identified the surgeons' left upper arm as being at the greatest risk for a work-related musculoskeletal disorder, specifically from performing NSMs.

Comparison between functional and non-functional adrenocortical carcinoma.

BACKGROUND: While roughly half of adrenocortical carcinomas are functional, whether functional status impacts outcomes remains controversial. We compared presentation and survival for functional and nonfunctional neoplasms. METHODS: Adult patients presented with adrenocortical carcinomas at the Mayo Clinic were included. Tumor characteristics and outcomes were analyzed. RESULTS: The 266 identified patients presented with stage I (6%), II (33%), III (26%), and IV disease (32%); stage was unknown in 3%. Fifty-three percent of tumors were functional; patients with functional adrenocortical carcinomas were younger, more likely to be female, and more likely to present with metastatic disease. Surgical resection was undertaken in 84% of patients with 69% having R0 resection. While 30-day morbidity was similar between functional and nonfunctional adrenocortical carcinomas, median overall survival was better for nonfunctional adrenocortical carcinomas (median 66 vs 22 months, P = .01). Functional adrenocortical carcinomas was independently associated with shorter survival after adjusting for age, sex, grade, stage, and resection attempt: hazard ratio = 1.5 (95% confidence interval, 1.04-2.14, P = .03). CONCLUSION: In our cohort, long term survival was worse for all patients with functional tumors. However, when analyzing patients with R0 resection, there was no survival difference between functional and nonfunctional adrenocortical carcinomas, signaling need for better understanding of adrenocortical carcinomas behavior to individualize and optimize treatment strategies.

Making EPAs a 59 minute objective measure for surgical trainees - A pilot study.

INTRODUCTION: Accurately confirming surgical trainees have met the requirements of entrusted professional activities (EPAs) will require rigorous staff input. We pondered whether such simulation-driven evaluations might prove useful to the current ABS/APDS pilot effort on EPA analysis. METHODS: Our surgical trainees participate biannually in a 59-min simulation-based assessment (X-Games) which covers surgical technique, knowledge and critical thinking in a host of domains. The content and difficulty of the stations differed between the PGY levels. Only 4 EPA topics (inguinal hernia repair, appendectomy, cholecystectomy, and trauma resuscitation) were tabulated for this study. RESULTS: 16-PGY-2s, 8-PGY-3s, 10-PGY-4s, and 10-PGY-5s surgical residents completed the X-Games. Performance within and between PGY levels was variable. The mean(SD) EPA scores for all PGY levels are listed in Table- 1. CONCLUSION: Analysis gleaned from a 59-min OSCE in a simulated-setting offers objective data that appears to have construct-validity. Refining our Surgical X-Games to cater to the specific EPA scoring system may better allow objective analysis of when trainees cross the threshold from "can do with some help" to "can do autonomously".