ABSTRACT
PURPOSE: Students' professional development is an essential aim of medical school. The purpose of this paper is to report how an assessment of first-year medical students' professional behavior was designed and to investigate its measurement characteristics. METHODS: The assessment was implemented as a peer assessment of professional skills, which were delineated according to a formal professional code developed by our curriculum committee. During the last week of the Fall 2005 semester, the professionalism assessment was administered online to students in a problem-based learning course. RESULTS: The internal consistency of the assessment is adequate. The generalizability study found that raters nested within persons accounted for the majority of variance. While the inter-rater reliability is relatively low, using multiple raters may yield an acceptable estimate of the relative reliability. CONCLUSIONS: The results suggest that this peer assessment is a practical assessment, evidenced by the 91% compliance rate. However, future research and modifications will be needed to address the variance of responses, helping to discriminate between "poor" and "good" observations of professionalism. In addition, multiple raters are required to supply reliable estimates of students' professional behavior. Coupling this evaluation with other professionalism evaluations may help reveal a more complete picture of students' professional behavior.
