Rating errors in the assessment of clinical competence of newly graduated doctors

ABSTRACT

Clinicians and academicians attached to health care institutions, who share the responsibility for training medical graduates, need to guard against common errors of measurement when assessing trainees' competencies. This study aims to identify the rating errors that may occur during the assessment of the clinical competence of interns. Trainers assigned responsibility for the internship program assessed the competencies of 45 internship trainees in the March 2002 cohort in six domains [case presentation, clinical skills, diagnosis, therapy, handling of emergencies and professional behavior] using a structured format provided. Competencies related to different clinical problems were rated under 1 = Unsatisfactory, 2 = Below expected standard, 3 = At expected standard, 4 = Above expected standard or 5 = Excellent. 4868 assessments were made of the performances of 45 interns who completed rotations in Surgery, Medicine, Obstetrics and Gynecology, and Pediatrics. 66% of all ratings fell in Excellent category [mean = 4, mode = 5]. The rotations in Surgery and Pediatrics elicited more Excellent ratings [37.6% and 31 .8% respectively] than other specialties. Of all trainers, Registrars allocated the highest proportion [46%] of Excellent ratings. When considering the number of trainers in each category, Excellent ratings had been assigned to a high extent by Assistant Registrars [72%], and Registrars [70%]. A high proportion of the ratings given by the trainers fell at the upper end of the Rating Scale. This could be a result of the assessments being affected by errors of leniency, compromizing their usefulness. It is desirable that the Heads of the respective Clinical Departments take steps to organize appropriate training programs so that the trainers improve their knowledge in the concepts underlying the assessment of clinical competencies and skills and methods of the use of the instruments