Association Between Flexible Duty Hour Policies and General Surgery Resident Examination Performance: A Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial Analysis.

BACKGROUND: Concerns persist about the effect of current duty hour reforms on resident educational outcomes. We investigated whether a flexible, less-restrictive duty hour policy (Flexible Policy) was associated with differential general surgery examination performance compared with current ACGME duty hour policy (Standard Policy). STUDY DESIGN: We obtained examination scores on the American Board of Surgery In-Training Examination, Qualifying Examination (written boards), and Certifying Examination (oral boards) for residents in 117 general surgery residency programs that participated in the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial. Using bivariate analyses and regression models, we compared resident examination performance across study arms (Flexible Policy vs Standard Policy) for 2015 and 2016, and 1 year of the Qualifying Examination and Certifying Examination. Adjusted analyses accounted for program-level factors, including the stratification variable for randomization. RESULTS: In 2016, FIRST trial participants were 4,363 general surgery residents. Mean American Board of Surgery In-Training Examination scores for residents were not significantly different between study groups (Flexible Policy vs Standard Policy) overall (Flexible Policy: mean [SD] 502.6 [100.9] vs Standard Policy: 502.7 [98.6]; p = 0.98) or for any individual postgraduate year level. There was no difference in pass rates between study arms for either the Qualifying Examination (Flexible Policy: 90.4% vs Standard Policy: 90.5%; p = 0.99) or Certifying Examination (Flexible Policy: 86.3% vs Standard Policy: 88.6%; p = 0.24). Results from adjusted analyses were consistent with these findings. CONCLUSIONS: Flexible, less-restrictive duty hour policies were not associated with differences in general surgery resident performance on examinations during the FIRST Trial. However, more years under flexible duty hour policies might be needed to observe an effect.

Use and Underlying Reasons for Duty Hour Flexibility in the Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial.

BACKGROUND: The Flexibility in Duty Hour Requirements for Surgical Trainees (FIRST) Trial randomly assigned surgical residency programs to either standard duty hour policies or flexible policies that eliminated caps on shift lengths and time off between shifts. Our objectives were to assess adherence to duty hour requirements in the Standard Policy arm and examine how often and why duty hour flexibility was used in the Flexible Policy arm. STUDY DESIGN: A total of 3,795 residents in the FIRST trial completed a survey in January 2016 (response rate >95%) that asked how often and why they exceeded current standard duty hour limits in both study arms. RESULTS: Flexible Policy interns worked more than 16 hours continuously at least once in a month more frequently than Standard Policy residents (86% vs 37.8%). Flexible Policy residents worked more than 28 hours once in a month more frequently than Standard Policy residents (PGY1: 64% vs 2.9%; PGY2 to 3: 62.4% vs 41.9%; PGY4 to 5: 52.2% vs 36.6%), but this occurred most frequently only 1 to 2 times per month. Although residents reported working more than 80 hours in a week 3 or more times in the most recent month more frequently under Flexible Policy vs Standard Policy (19.9% vs 16.2%), the difference was driven by interns (30.9% vs 19.6%), and there were no significant differences in exceeding 80 hours among PGY2 to 5 residents. The most common reasons reported for extending duty hours were facilitating care transitions (76.6%), stabilizing critically ill patients (70.7%), performing routine responsibilities (67.9%), and operating on patients known to the trainee (62.0%). CONCLUSIONS: There were differences in duty hours worked by residents in the Flexible vs Standard Policy arms of the FIRST trial, but it appeared that residents generally used the flexibility for patient care and educational opportunities selectively.

General surgery resident rotations in surgical critical care, trauma, and burns: what is optimal for residency training?

BACKGROUND: There are no specific Accreditation Council for Graduate Medical Education General Surgery Residency Program Requirements for rotations in surgical critical care (SCC), trauma, and burn. We sought to determine the experience of general surgery residents in SCC, trauma, and burn rotations. METHODS: Data analysis of surgical rotations of American Board of Surgery general surgery resident applicants (n = 7,299) for the last 8 years (2006 to 2013, inclusive) was performed through electronic applications to the American Board of Surgery Qualifying Examination. Duration (months) spent in SCC, trauma, and burn rotations, and postgraduate year (PGY) level were examined. RESULTS: The total months in SCC, trauma and burn rotations was mean 10.2 and median 10.0 (SD 3.9 months), representing approximately 16.7% (10 of 60 months) of a general surgery resident's training. However, there was great variability (range 0 to 29 months). SCC rotation duration was mean 3.1 and median 3.0 months (SD 2, min to max: 0 to 15), trauma rotation duration was mean 6.3 and median 6.0 months (SD 3.5, min to max: 0 to 24), and burn rotation duration was mean 0.8 and median 1.0 months (SD 1.0, min to max: 0 to 6). Of the total mean 10.2 months duration, the longest exposure was 2 months as PGY-1, 3.4 months as PGY-2, 1.9 months as PGY-3, 2.2 months as PGY-4 and 1.1 months as PGY-5. PGY-5 residents spent a mean of 1 month in SCC, trauma, and burn rotations. PGY-4/5 residents spent the majority of this total time in trauma rotations, whereas junior residents (PGY-1 to 3) in SCC and trauma rotations. CONCLUSIONS: There is significant variability in total duration of SCC, trauma, and burn rotations and PGY level in US general surgery residency programs, which may result in significant variability in the fund of knowledge and clinical experience of the trainee completing general surgery residency training. As acute care surgery programs have begun to integrate emergency general surgery with SCC, trauma, and burn rotations, it is an ideal time to determine the optimal curriculum and duration of these important rotations for general surgery residency training.

Understanding the Operative Experience of the Practicing Pediatric Surgeon: Implications for Training and Maintaining Competency.

IMPORTANCE: The number of practicing pediatric surgeons has increased rapidly in the past 4 decades, without a significant increase in the incidence of rare diseases specific to the field. Maintenance of competency in the index procedures for these rare diseases is essential to the future of the profession. OBJECTIVE: To describe the demographic characteristics and operative experiences of practicing pediatric surgeons using Pediatric Surgery Board recertification case log data. DESIGN, SETTING, AND PARTICIPANTS: We performed a retrospective review of 5 years of pediatric surgery certification renewal applications submitted to the Pediatric Surgery Board between 2009 and 2013. A surgeon's location was defined by population as urban, large rural, small rural, or isolated. Case log data were examined to determine case volume by category and type of procedures. Surgeons were categorized according to recertification at 10, 20, or 30 years. MAIN OUTCOME AND MEASURE: Number of index cases during the preceding year. RESULTS: Of 308 recertifying pediatric surgeons, 249 (80.8%) were men, and 143 (46.4%) were 46 to 55 years of age. Most of the pediatric surgeons (304 of 308 [98.7%]) practiced in urban areas (ie, with a population >50 000 people). All recertifying applicants were clinically active. An appendectomy was the most commonly performed procedure (with a mean [SD] number of 49.3 [35.0] procedures per year), nonoperative trauma management came in second (with 20.0 [33.0] procedures per year), and inguinal hernia repair for children younger than 6 months of age came in third (with 14.7 [13.8] procedures per year). In 6 of 10 "rare" pediatric surgery cases, the mean number of procedures was less than 2. Of 308 surgeons, 193 (62.7%) had performed a neuroblastoma resection, 170 (55.2%) a kidney tumor resection, and 123 (39.9%) an operation to treat biliary atresia or choledochal cyst in the preceding year. Laparoscopy was more frequently performed in the 10-year recertification group for Nissen fundoplication, appendectomy, splenectomy, gastrostomy/jejunostomy, orchidopexy, and cholecystectomy (P < .05) but not lung resection (P = .70). It was more frequently used by surgeons recertifying in the 10-year group (used in 11 375 of 14 456 procedures [78.7%]) than by surgeons recertifying in the 20-year (used in 6214 of 8712 procedures [71.3%]) or 30-year group (used in 2022 of 3805 procedures [53.1%]). CONCLUSIONS AND RELEVANCE: Practicing pediatric surgeons receive limited exposure to index cases after training. With regard to maintaining competency in an era in which health care outcomes have become increasingly important, these results are concerning.

National Cluster-Randomized Trial of Duty-Hour Flexibility in Surgical Training.

BACKGROUND: Concerns persist regarding the effect of current surgical resident duty-hour policies on patient outcomes, resident education, and resident well-being. METHODS: We conducted a national, cluster-randomized, pragmatic, noninferiority trial involving 117 general surgery residency programs in the United States (2014-2015 academic year). Programs were randomly assigned to current Accreditation Council for Graduate Medical Education (ACGME) duty-hour policies (standard-policy group) or more flexible policies that waived rules on maximum shift lengths and time off between shifts (flexible-policy group). Outcomes included the 30-day rate of postoperative death or serious complications (primary outcome), other postoperative complications, and resident perceptions and satisfaction regarding their well-being, education, and patient care. RESULTS: In an analysis of data from 138,691 patients, flexible, less-restrictive duty-hour policies were not associated with an increased rate of death or serious complications (9.1% in the flexible-policy group and 9.0% in the standard-policy group, P=0.92; unadjusted odds ratio for the flexible-policy group, 0.96; 92% confidence interval, 0.87 to 1.06; P=0.44; noninferiority criteria satisfied) or of any secondary postoperative outcomes studied. Among 4330 residents, those in programs assigned to flexible policies did not report significantly greater dissatisfaction with overall education quality (11.0% in the flexible-policy group and 10.7% in the standard-policy group, P=0.86) or well-being (14.9% and 12.0%, respectively; P=0.10). Residents under flexible policies were less likely than those under standard policies to perceive negative effects of duty-hour policies on multiple aspects of patient safety, continuity of care, professionalism, and resident education but were more likely to perceive negative effects on personal activities. There were no significant differences between study groups in resident-reported perception of the effect of fatigue on personal or patient safety. Residents in the flexible-policy group were less likely than those in the standard-policy group to report leaving during an operation (7.0% vs. 13.2%, P<0.001) or handing off active patient issues (32.0% vs. 46.3%, P<0.001). CONCLUSIONS: As compared with standard duty-hour policies, flexible, less-restrictive duty-hour policies for surgical residents were associated with noninferior patient outcomes and no significant difference in residents' satisfaction with overall well-being and education quality. (FIRST ClinicalTrials.gov number, NCT02050789.).

Factors influencing the decision of surgery residency graduates to pursue general surgery practice versus fellowship.

OBJECTIVES: Surgery residency serves 2 purposes-prepare graduates for general surgery (GS) practice or postresidency surgical fellowship, leading to specialty surgical practice (SS). This study was undertaken to elucidate factors influencing career choice for these 2 groups. METHODS: All US allopathic surgery residency graduates from 2009 to 2013 (n = 5512) were surveyed by the American Board of Surgery regarding confidence, autonomy, and reasons for career selection between GS and SS. Surveys were distributed by mail in November 2013, with follow-up mailings to initial nonrespondents. RESULTS: Sixty-one percent (3354) of graduates completed the survey; 26% pursued GS, and 74% SS. GS expressed greater levels of confidence than SS across the common surgical procedures queried. Confidence increased with each year after completion of residency for GS but not SS. The decision to pursue GS or SS was made during residency by 77% and 74%, respectively. Fifty-seven percent of those who chose GS indicated that a GS mentor significantly influenced their decision. GS rated procedural variety, opportunity for practice autonomy, choice of practice location, and influence of a mentor as reasons to pursue GS practice. SS listed control over scope of practice, prestige, salary, and specialty interest as reasons to pursue SF. Both groups expressed a high degree of satisfaction with their career choice (GS, 94%; SS, 90%). CONCLUSIONS: Most graduates who pursue GS practice are confident and content. The decision to pursue GS is strongly influenced by a GS mentor. Lack of confidence may be a more significant factor for choosing SS. These findings suggest opportunities for improvements in confidence and mentorship during residency.

Review courses for the American Board of Surgery certifying examination do not provide an advantage.

PURPOSE: The purpose of this study was to evaluate whether participation in a commercially available board review course affected the likelihood of passing the general surgery certifying exam (CE) of the American Board of Surgery (ABS). METHODS: Candidates who took the ABS CE during the 2012-2013 academic year were surveyed electronically about their participation in commercial board review courses on the ABS website immediately before receiving their exam results. The primary outcome variable was passing the CE. Results were adjusted for background variables shown to correlate with CE pass rates using multilevel logistic regression. RESULTS: Of the 1,386 candidates who took the CE and had sufficient data for analysis, 974 of 1,064 first-time examinees (92%) and 272 of 322 repeat examinees (84%) completed the survey. Nearly 78% of survey respondents took a review course. Repeat examinees (85%) were more likely to attend a review course than first-time examinees (76%, P = .002). There were no significant differences in CE pass rates for first-time or repeat examinees who took a review course compared with those who did not (83.7% vs 80.7% for first-time examinees and 77.8% vs 69.0% for repeat examinees, P = .32 and P = .24, respectively). First-time examinee nonrespondents did not differ in their CE pass rates from those who responded to the survey (P = .113); however, repeat examinee nonrespondents had lesser CE pass rates than survey responders (P = .009). None of the review courses included had CE pass rates that differed significantly from the others after we controlled for program characteristics, ABS qualifying exam scores, medical school, and sex of the examinees. CONCLUSION: This study used a large, prospectively collected national sample with a high response rate to study the effect of board review courses on CE performance on the ABS examination. On the basis of this survey, there was no evidence that participating in a board review course provided a benefit to passing the CE of the ABS. These results should be considered when preparation for the CE is undertaken.

Association of the 2011 ACGME resident duty hour reform with general surgery patient outcomes and with resident examination performance.

IMPORTANCE: In 2011, the Accreditation Council for Graduate Medical Education (ACGME) restricted resident duty hour requirements beyond those established in 2003, leading to concerns about the effects on patient care and resident training. OBJECTIVE: To determine if the 2011 ACGME duty hour reform was associated with a change in general surgery patient outcomes or in resident examination performance. DESIGN, SETTING, AND PARTICIPANTS: Quasi-experimental study of general surgery patient outcomes 2 years before (academic years 2009-2010) and after (academic years 2012-2013) the 2011 duty hour reform. Teaching and nonteaching hospitals were compared using a difference-in-differences approach adjusted for procedural mix, patient comorbidities, and time trends. Teaching hospitals were defined based on the proportion of cases at which residents were present intraoperatively. Patients were those undergoing surgery at hospitals participating in the American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP). General surgery resident performance on the annual in-training, written board, and oral board examinations was assessed for this same period. EXPOSURES: National implementation of revised resident duty hour requirements on July 1, 2011, in all ACGME accredited residency programs. MAIN OUTCOMES AND MEASURES: Primary outcome was a composite of death or serious morbidity; secondary outcomes were other postoperative complications and resident examination performance. RESULTS: In the main analysis, 204,641 patients were identified from 23 teaching (n = 102,525) and 31 nonteaching (n = 102,116) hospitals. The unadjusted rate of death or serious morbidity improved during the study period in both teaching (11.6% [95% CI, 11.3%-12.0%] to 9.4% [95% CI, 9.1%-9.8%], P < .001) and nonteaching hospitals (8.7% [95% CI, 8.3%-9.0%] to 7.1% [95% CI, 6.8%-7.5%], P < .001). In adjusted analyses, the 2011 ACGME duty hour reform was not associated with a significant change in death or serious morbidity in either postreform year 1 (OR, 1.12; 95% CI, 0.98-1.28) or postreform year 2 (OR, 1.00; 95% CI, 0.86-1.17) or when both postreform years were combined (OR, 1.06; 95% CI, 0.93-1.20). There was no association between duty hour reform and any other postoperative adverse outcome. Mean (SD) in-training examination scores did not significantly change from 2010 to 2013 for first-year residents (499.7 [ 85.2] to 500.5 [84.2], P = .99), for residents from other postgraduate years, or for first-time examinees taking the written or oral board examinations during this period. CONCLUSIONS AND RELEVANCE: Implementation of the 2011 ACGME duty hour reform was not associated with a change in general surgery patient outcomes or differences in resident examination performance. The implications of these findings should be considered when evaluating the merit of the 2011 ACGME duty hour reform and revising related policies in the future.

Using the American Board of Surgery In-Training Examination to predict board certification: a cautionary study.

OBJECTIVE: Although designed as a low-stakes formative examination, the American Board of Surgery In-Training Examination (ABSITE) is often used in high-stakes decisions such as promotion, remediation, and retention owing to its perceived ability to predict the outcome of board certification. Because of the discrepancy between intent and use, the ability of ABSITE scores to predict passing the American Board of Surgery certification examinations was analyzed. METHODS: All first-time American Board of Surgery qualifying examination (QE) examinees between 2006 and 2012 were reviewed. Examinees' postgraduate year (PGY) 1 and PGY5 ABSITE standard scores were linked to QE scores and pass/fail outcomes (n = 6912 and 6846, respectively) as well as first-time certifying examination (CE) pass/fail results (n = 1329). Linear and logistic regression analyses were performed to evaluate the utility of ABSITE scores to predict board certification scores and pass/fail outcomes. RESULTS: PGY1 ABSITE scores accounted for 22% of the variance in QE scores (p < 0.001). PGY5 scores were a slightly better predictor, accounting for 30% of QE score variance (p < 0.001). Analyses showed that selecting a PGY5 ABSITE score that maximized overall decision accuracy for predicting QE pass/fail outcomes (86% accuracy) resulted in 98% sensitivity, 13% specificity, a positive predictive value of 87%, and a negative predictive value of 57%. ABSITE scores were not predictive of success on the CE. CONCLUSIONS: ABSITE scores are a useful predictor of QE scores and outcomes but do not predict passing the CE. Although scoring well on the ABSITE is highly predictive of QE success, using low ABSITE scores to predict QE failure results in frequent decision errors. Program directors and other evaluators should use additional sources of information when making high-stakes decisions about resident performance.

General surgery residency training issues.

The practice of general surgery has undergone a marked evolution in the past 20 years, which has been inadequately recognized and minimally addressed. The changes that have occurred have been disruptive to residency training, and to date there has been inadequate compensation for these. Evidence is now emerging of significant issues in the overall performance of recent graduates from at least 3 sources: the evaluation of external agents who incorporate these graduates into their practice or group, the opinions of the residents themselves, and the performance of graduates on the oral examination of the American Board of Surgery during the past 8 years. The environmental and technological causes of the present situation represent improvements in care for patients, and are clearly irreversible. Hence, solutions to the problems must be sought in other areas. To address the issues effectively, greater recognition and engagement are needed by the surgical community so that effective solutions can be crafted. These will need to include improvements in the efficiency of teaching, with the assumption of greater individual resident responsibility for their knowledge, the establishment of more defined standards for knowledge and skills acquisition by level of residency training, with flexible self-assessment available online, greater focus of the curriculum on current rather than historical practice, increased use of structured assessments (including those in a simulated environment), and modifications to the overall structure of the traditional 5-year residency.

Operative experience of surgery residents: trends and challenges.

OBJECTIVE: To evaluate trends in operative experience and to determine the effect of establishing the Surgical Council on Resident Education (SCORE) operative classification system on changes in operative volume among graduating surgery residents. DESIGN: The general surgery operative logs of graduating surgery residents from 2005 were retrospectively compared with residents who completed training in 2010 and 2011. Nonparametric statistical analyses were used (Mann-Whitney and median test) with significance set at p<0.01. PARTICIPANTS: A total of 1022 residents completing residency in 2005 were compared with 1923 residents completing training in 2010-2011. RESULTS: Total operations reported increased from a median of 1023 to 1238 (21%) between 2005 and 2010-2011 (p<0.001). Cases increased in most SCORE categories. The median numbers of total, basic, and complex laparoscopic operations increased by 49%, 37%, and 82%, respectively, over the 5-year interval (p<0.001). Open cavitary (thoracic + abdominal) operations decreased by 5%, whereas other major operations increased by 35% (both p<0.001). The frequency of discrete operations done at least 10 times during residency did not change. The median number of SCORE essential-common operations performed ranged from 1 to 107, whereas essential-uncommon operations ranged from 0 to 4. Twenty-three of 67 SCORE essential-common operations (34%) had a median of less than 5 and 4 had a median of 0. CONCLUSIONS: The operative volume of graduating surgical residents has increased by 21% since 2005; however, the number of operations done 10 times or greater has not changed. Although open cavitary procedures continue to decline, there has been a large increase in endoscopy, complex laparoscopic, and other major operations. Some essential-common operations continue to be performed infrequently. These results suggest that education in the operating room must improve and alternate methods for teaching infrequently performed procedures are needed.

Delay in taking the American Board of Surgery qualifying examination affects examination performance.

BACKGROUND: The American Board of Surgery (ABS) Qualifying Examination (QE) represents an important step along the pathway to board certification. We investigated whether candidates who delayed taking the QE had worse performance on the examination. METHODS: QE pass rates and equated scaled scores for all first-time examinees from 2006 to 2010 (n = 5,193) were reviewed. After eliminating examinees who could not be matched to final ABS In-Training Examination (ABSITE) scores, the remaining cohort (n = 4,909) was analyzed by comparing those who took the exam immediately after residency (Immediate, n = 4,488) to those who delayed for 1 or more years (Delay, n = 421). RESULTS: The Immediate group had a mean first-time QE pass rate of 87% compared to 57% for those who delayed 1 year and 48% for those who delayed 2 or more years (P < .001). Regression analysis demonstrated that delay in taking the QE remained a significant determinant of exam failure after controlling for ABSITE scores (odds ratio = 0.35; 95% CI, 0.29-0.43; P = .001). Undergraduate medical education and postresidency training did not affect the results. The Delay group had lower equated scaled scores, a greater ultimate failure rate on the QE, and was more likely to fail the ABS Certifying Examination on the first attempt. CONCLUSION: These results demonstrate that candidates who delayed taking the QE immediately are at extremely high risk for exam failure and failure to achieve board certification. These findings presumably are due to deterioration of knowledge over time, but they also may represent characteristics of the Delay group that are currently undefined.

A national study of attrition in general surgery training: which residents leave and where do they go?

OBJECTIVE(S): Implementation of the 80-hour mandate was expected to reduce attrition from general surgery (GS) residency. This is the first quantitative report from a national prospective study of resident/program characteristics associated with attrition. METHODS: Analysis included all categorical GS residents entered on American Board of Surgery residency rosters in 2007 to 2008. Cases of attrition were identified by program report, individually confirmed, and linked to demographic data from the National Study of Expectations and Attitudes of Residents in Surgery administered January 2008. RESULTS: All surgical categorical GS residents active on the 2007-2008 resident rosters (N = 6,303) were analyzed for attrition. Complete National Study of Expectations and Attitudes of Residents in Surgery demographic information was available for 3959; the total and survey groups were similar with regard to important characteristics. About 3% of US categorical residents resigned in 2007 to 2008, and 0.4% had contracts terminated. Across all years (including research), there was a 19.5% cumulative risk of resignation. Attrition was highest in PGY-1 (5.9%), PGY-2 (4.3%), and research year(s) (3.9%). Women were no more likely to leave programs than men (2.1% vs. 1.9%). Of several program/resident variables examined, postgraduate year-level was the only independent predictor of attrition in multivariate analysis. Residents who left GS whose plans were known most often pursued nonsurgical residencies (62%), particularly anesthesiology (21%) and radiology (11%). Only 13% left for surgical specialties. CONCLUSIONS: Attrition rates are high despite mandated work hour reductions; 1 in 5 GS categorical residents resigns, and most pursue nonsurgical careers. Demographic factors, aside from postgraduate year do not appear predictive. Residents are at risk for attrition early in training and during research, and this could afford educators a target for intervention.