ABSTRACT
OBJECTIVES: To determine whether faculty could successfully evaluate residents using a competency-based modified Milestones global evaluation tool. DESIGN: A program's leadership team modified a draft Surgery Milestones Working Group summative global assessment instrument into a modified Milestones tool (MMT) for local use during faculty meetings devoted to semiannual resident review. Residents were scored on 15 items spanning all competencies using an 8-point graphic response scale; unstructured comments also were solicited. Arithmetic means were computed at the resident and postgraduate year cohort levels for items and competency item sets. Score ranges (highest minus lowest score) were calculated; variability was termed "low" (range <2.0 points), "moderate" (range = 2.0), or "high" (range >2.0). A subset of "low" was designated "small" (1.0-1.9). Trends were sought among item, competency, and total Milestones scores. MMT correlations with examination scores and multisource (360°) assessments were explored. The success of implementing MMT was judged using published criteria for educational assessment methods. SETTING: Fully accredited, independently sponsored residency. PARTICIPANTS: Program leaders and 22 faculty members (71% voluntary, mean 12y of experience). RESULTS: Twenty-six residents were assessed, yielding 7 to 13 evaluations for MMT per categorical resident and 3 to 6 per preliminary trainee. Scores spanned the entire response scale. All MMT evaluations included narrative comments. Individual resident score variability was low (96% within competencies and 92% across competencies). Subset analysis showed that small variations were common (35% within competencies and 54% across competencies). Postgraduate year cohort variability was higher (61% moderate or high within competencies and 50% across competencies). Cohort scores at the item, competency, and total score levels exhibited rising trajectories, suggesting MMT construct validity. MMT scores did not demonstrate concurrent validity, correlating poorly with other metrics. The MMT met multiple criteria for good assessment. CONCLUSIONS: A modified Milestones global evaluation tool can be successfully adopted for semiannual assessments of resident performance by volunteer faculty members.
Subject(s)
Clinical Competence , Educational Measurement/methods , Evaluation Studies as Topic , Faculty, Medical/standards , Internship and Residency/organization & administration , Adult , Education, Medical, Graduate/organization & administration , Female , Humans , Male , Problem-Based Learning , Sensitivity and Specificity , Time Factors , United StatesABSTRACT
BACKGROUND: This study assesses the outcomes of nondesignated preliminary (NDP) residents in general surgery (GS) at an independent, nonuniversity training program. METHODS: Records of all NDP residents from 1984-1985 through 2008-2009 were reviewed, and residents' careers were followed. Designated preliminary and categorical residents were excluded. RESULTS: Sixty-two residents completed the NDP year. Three of these residents also completed a second postgraduate NDP year. A total of 60 NDPs (97%) continued in accredited postgraduate programs. Forty-eight graduates (77%) pursued surgery-associated careers: 26 (42%) in GS and 22 (35%) in other surgery-related specialties. Eleven of the 26 NDPs who entered GS (42%) became categorical residents in our program. All NDP GS graduates are board certified, board eligible, or are residents in training. CONCLUSIONS: After a preliminary year in GS, NDPs continued in postgraduate medical education followed by board certification, usually in GS or surgery-related specialties. NDPs often obtain categorical positions in the parent GS program.
