A preliminary study on the reliability and validity of objective structured skills assessment for the orthopeadic postgraduates / 中华医学教育探索杂志

ABSTRACT

Objectives To design an objective and structured evaluation system for the clinical competence of orthopedic postgraduates in the diagnosis and treatment of distal radius fractures, and to ana-lyze its reliability and validity. Methods 28 orthopaedic postgraduates representing six levels of surgical training were tested for competence in performing surgical approach for distal radius fracture on cadaver specimens during which four measures were used to assess competency examination of basic theory based on network item bank, objective structured operation assessment,overall assessment and operation examina-tion results. In addition, the time for completion of the surgery was also recorded. Each assessment tool was correlated with the others as well as with the resident’s level of training. Results There was a significant correlation between the seniority of candidates and the score of theoretical examination (F=6.193, P=0.000), the score of structured operation examination (F=6.374, P=0.002), the score of overall assessment (F=2.321, P=0.030), and the passing rate of final operation examination (F=36.300, P=0.000). No significant differ- ences were found between seniority and time to completion of the surgical approach exposure (F=2.282, P<0.073). Conclusions The results of the present study suggested that both theoretical examination and cadaver testing discriminate between novice and accomplished postgraduates. However, although the theo-retical test scores could predict the operational test results, but the theoretical results can not guarantee excellent operational skills.