Variation in faculty evaluations of clerkship students attributable to surgical service.

OBJECTIVE: The purpose of this study was to determine whether students' performance evaluations by faculty were influenced by the clinical service on which the student was evaluated. METHODS: Third-year medical students spent 8 weeks rotating on 3 (or 2) surgical services. Typically, students rotate on one 4-week general surgery service and two 2-week subspecialty services. Faculty members rated student performance on 5 characteristics and provided a numeric grade. Data were analyzed to determine whether any significant variations in evaluation patterns emerged. RESULTS: A total of 1033 evaluations were included in the analyses. Based on an analysis of variance, the numeric grade varied significantly by service (p < 0.001). The partial eta squared statistic was large (0.21). Ratings of students' performance on specific performance characteristics also varied significantly by service (p < 0.001). CONCLUSIONS: The assessment of a surgical student's clinical performance is influenced by the specific services on which he/she has rotated and may be related to the length of the rotation. Research is needed to determine whether the differences among services should be considered as a source of error in grading or considered to reflect the particular challenge of the service.

Factors in faculty evaluation of medical students' performance.

CONTEXT: Faculty members often use global rating scales as a method of assessing various characteristics of medical students' clinical performance. The purpose of this study was to determine if some performance characteristics are more highly associated with the overall faculty grade than others. METHODS: The clinical performance of 211 surgery clerkship students was evaluated by 2 or 3 faculty preceptors. Faculty rated students on 10 specific performance characteristics, using a 5-point scale. Faculty then assigned a numerical grade summarising the faculty's view of the student's performance. Reliability of the ratings was estimated by the intraclass correlation, and 1-way (analysis of variance) anova was used to test for differences among the students' mean ratings. Logistic regression was employed to determine the accuracy of each performance measure in predicting students' grades (A or B). Stepwise logistic regression was used to determine if there was a combination of performance characteristics that best predicted students' grades. RESULTS: The inter-rater reliabilities were low (

What do faculty observe of medical students' clinical performance?

BACKGROUND: An earlier study of our faculty's evaluation of junior medical students indicated that performance ratings were unreliable and reflected 1 underlying dimension. Other researchers have obtained similar results. PURPOSE: The purpose of this study was to identify which aspects of students' clinical performance faculty actually observe. METHODS: We analyzed the responses of 9 faculty members to an open-ended questionnaire concerning which aspects of clinical performance attending faculty observe. We also reviewed and summarized the written comments of 331 faculty evaluations of third-year medical students. RESULTS: Analysis of the questionnaires and evaluations indicated that faculty members gauge medical knowledge, professionalism, and clinical reasoning skills from direct interaction with students. History-taking and physical examination skills are inferred from the quality of verbal presentations. Faculty have little basis for evaluating other important aspects of clinical performance. CONCLUSIONS: Faculty primarily observe medical students' cognitive skills and professionalism. Faculty have little basis for evaluating most other features of clinical performance.