Milestones for neurosurgery sub-interns: a novel evaluation tool to quantitatively differentiate residency applicants.

OBJECTIVE: The study objective was to create a novel milestones evaluation form for neurosurgery sub-interns and assess its potential as a quantitative and standardized performance assessment to compare potential residency applicants. In this pilot study, the authors aimed to determine the form's interrater reliability, relationship to percentile assignments in the neurosurgery standardized letter of recommendation (SLOR), ability to quantitatively differentiate tiers of students, and ease of use. METHODS: Medical student milestones were either adapted from the resident Neurological Surgery Milestones or created de novo to evaluate a student's medical knowledge, procedural aptitude, professionalism, interpersonal and communication skills, and evidence-based practice and improvement. Four milestone levels were defined, corresponding to estimated 3rd-year medical student through 2nd-year resident levels. Faculty and resident evaluations as well as student self-evaluations were completed for 35 sub-interns across 8 programs. A cumulative milestone score (CMS) was computed for each student. Student CMSs were compared both within and between programs. Interrater reliability was determined with Kendall's coefficient of concordance (Kendall's W). Student CMSs were compared against their percentile assignments in the SLOR using analysis of variance with post hoc testing. CMS-derived percentile rankings were assigned to quantitatively distinguish tiers of students. Students and faculty were surveyed on the form's usefulness. RESULTS: The average faculty rating overall was 3.20, similar to the estimated competency level of an intern. Student and faculty ratings were similar, whereas resident ratings were lower (p < 0.001). Students were rated most highly in coachability and feedback (3.49 and 3.67, respectively) and lowest in bedside procedural aptitude (2.90 and 2.85, respectively) in both faculty and self-evaluations. The median CMS was 26.5 (IQR 21.75-29.75, range 14-32) with only 2 students (5.7%) achieving the highest rating of 32. Programs that evaluated the most students differentiated the highest-performing students from the lowest by at least 13 points. A program with 3 faculty raters demonstrated scoring agreement across 5 students (p = 0.024). The CMS differed significantly between SLOR percentile assignments, despite 25% of students being assigned to the top fifth percentile. CMS-driven percentile assignment significantly differentiated the bottom, middle, and top third of students (p < 0.001). Faculty and students strongly endorsed the milestones form. CONCLUSIONS: The medical student milestones form was well received and differentiated neurosurgery sub-interns both within and across programs. This form has potential as a replacement for numerical Step 1 scoring as a standardized, quantitative performance assessment for neurosurgery residency applicants.

Developing a model for effective leadership in healthcare: a concept mapping approach.

PURPOSE: Despite increasing awareness of the importance of leadership in healthcare, our understanding of the competencies of effective leadership remains limited. We used a concept mapping approach (a blend of qualitative and quantitative analysis of group processes to produce a visual composite of the group's ideas) to identify stakeholders' mental model of effective healthcare leadership, clarifying the underlying structure and importance of leadership competencies. METHODS: Literature review, focus groups, and consensus meetings were used to derive a representative set of healthcare leadership competency statements. Study participants subsequently sorted and rank-ordered these statements based on their perceived importance in contributing to effective healthcare leadership in real-world settings. Hierarchical cluster analysis of individual sortings was used to develop a coherent model of effective leadership in healthcare. RESULTS: A diverse group of 92 faculty and trainees individually rank-sorted 33 leadership competency statements. The highest rated statements were "Acting with Personal Integrity", "Communicating Effectively", "Acting with Professional Ethical Values", "Pursuing Excellence", "Building and Maintaining Relationships", and "Thinking Critically". Combining the results from hierarchical cluster analysis with our qualitative data led to a healthcare leadership model based on the core principle of Patient Centeredness and the core competencies of Integrity, Teamwork, Critical Thinking, Emotional Intelligence, and Selfless Service. CONCLUSION: Using a mixed qualitative-quantitative approach, we developed a graphical representation of a shared leadership model derived in the healthcare setting. This model may enhance learning, teaching, and patient care in this important area, as well as guide future research.

Repeat abdominal computed tomography scans after pediatric blunt abdominal trauma: missed injuries, extra costs, and unnecessary radiation exposure.

PURPOSE: We hypothesized that pediatric blunt trauma patients, initially evaluated at nontrauma centers with abdominal computed tomography (CT) scans, often undergo repeat scans after transfer. This study was designed to quantify this phenomenon, assess consequences, and elucidate possible causes. METHODS: This article is an institutional review board-approved, retrospective chart review of pediatric blunt abdominal trauma patients transferred to a level I trauma center from 2002 to 2007 and evaluated with abdominal CT at the trauma center or at a referring facility. RESULTS: A total of 388 patients met the study criteria, with 6 patients being excluded because of inability to verify outside records resulting in study group of 382 patients. Of those 382 patients, 199 (52%) underwent abdominal CT before transfer. Thirty-six (18%) of those 199 patients underwent repeat CT scanning at our level I trauma center. Of these 36 patients, 19 (53%) were transferred without their outside CT scans, with 10 (53%) of these 19 having significant abdominal injuries. Of the remaining 17, 6 (17%) had repeat scans to assess changes in vital signs, or patient condition, or because of inadequate outside imaging. The remaining 11 (30%) were repeated despite an acceptable outside CT and no change in patient condition. Only 2 of 11 resulted in changed management. Additional radiation delivered from these repeat scans totaled 180 mSv, and additional patient charges totaled more than $110,000. There was an apparent trend toward increased repeat scanning (from 6.7% in 2002 to 16.7% in 2007). CONCLUSIONS: Abdominal CT scans, for evaluation of pediatric blunt trauma, are frequently repeated after transfer from outside hospitals. In many cases, repeat scans provide useful diagnostic information. However, more than 80% of repeat scanning is potentially preventable with better education of transport personnel (paramedics, emergency medical technicians, and nurses) and emergency department physicians.