Sharpening the eye of the OSCE with critical action analysis.

PURPOSE: When interpreting performance scores on an objective structured clinical examination (OSCE), are all checklist items created equal? Although assigning priority through checklist item weighting is often used to emphasize the clinical importance of selected checklist items, the authors propose the use of critical action analysis as an additional method for analyzing and discriminating clinical performance in clinical skill assessment exercises. A critical action is defined as an OSCE checklist item whose performance is critical to ensure an optimal patient outcome and avoid medical error. In this study, the authors analyzed a set of clerkship OSCE performance outcome data and compared the results of critical action analysis versus traditional checklist item performance scores. METHOD: OSCE performance scores of 398 third-year clerkship students from 2003 to 2006 at the University of Virginia School of Medicine were analyzed using descriptive statistics and a logistic regression model. Through a consensus process, 10 of 25 OSCE cases were identified as containing critical actions. RESULTS: Students who scored above the median correctly performed the critical actions more often than those scoring lower. However, for 9 of 10 cases, 6% to 46% of higher-scoring students failed to perform the critical action correctly. CONCLUSIONS: Failure to address this skill assessment outcome is a missed opportunity to more fully understand and apply the results of such examinations to the clinical performance development of medical students. Including critical action analysis in OSCE data interpretation sharpens the eye of the OSCE and enhances its value in clinical skill assessment.

Enhancing clinical skills education: University of Virginia School of Medicine's Clerkship Clinical Skills Workshop Program.

In 1993, the University of Virginia School of Medicine began a clinical skills workshop program in an effort to improve the preparation of all clerkship students to participate in clinical care. This program involved the teaching of selected basic clinical skills by interested faculty to small groups of third-year medical students. Over the past 14 years, the number of workshops has increased from 11 to 31, and they now involve clerkship faculty from family medicine, internal medicine, and pediatrics. Workshops include a variety of common skills from the communication, physical examination, and clinical test and procedure domains such as pediatric phone triage, shoulder examination, ECG interpretation, and suturing. Workshop sessions allow students to practice skills on each other, with standardized patients, or with models, with the goal of improving competence and confidence in the performance of basic clinical skills. Students receive direct feedback from faculty on their skill performance. The style and content of these workshops are guided by an explicit set of educational criteria.A formal evaluation process ensures that faculty receive regular feedback from student evaluation comments so that adherence to workshop criteria is continuously reinforced. Student evaluations confirm that these workshops meet their skill-learning needs. Preliminary outcome measures suggest that workshop teaching can be linked to student assessment data and may improve students' skill performance. This program represents a work-in-progress toward the goal of providing a more comprehensive and developmental clinical skills curriculum in the school of medicine.