The Effect of Using Two Duplicated Examination Sites to Simulate the Same Cases on the OSCE Reliability / 한국의학교육

If large-scale testing programs are being used, OSCE stations may be duplicated into two or more sites. There are a few studies on the reliability of OSCE with duplicated stations in Korea. The purpose of this study was to investigate the effect of duplication on the reliability of OSCE. At Hanyang university college of medicine, an OSCE is given to all senior medical students(91 per class) upon completion of all clinical clerkship rotations. The examination consisted of twenty one stations and eighteen cases that represented commonly encountered problems in primary care. Each station required seven minutes for its administration, with 6 to 6.5 minutes for the student-SP or model encounter, during which the students performs a complete focused history and/or physical examination and/or procedure and/or management, and another 0.5 to 1 minutes for the evaluator to feedback case-related comments. We analysed the reliability of duplication by comparing total OSCE scores and case scores between two exam sites. We also evaluated the reliability of duplicated stations from student's and professor's subjective response to the OSCE. All 91 fourth-year students attended the OSCE. Standardized Cronbach coefficient of the OSCE was 0.67. The station scores and OSCE total scores were different between two duplication sites. The total OSCE score of one site was slight higher than that of the other site(p=0.03). Of total 19 stations in which students were evaluated by staff evaluator, six stations are more advantageous to one part compared with counterpart stations, other six stations are vice CONCLUSIONS: OSCE reliability can be affected by duplication of examination sites and inter-rater reliability is the most important determining factor. The results demonstrate a need for caution in the interpretation of scores obtained from OSCE with duplicated stations.

An OSCE Assessment of Fourth-year Medical Students as A Comprehensive Evaluation of Medical Skills Before Graduation / 한국의학교육

BACKGROUND: Experiences on Objective Structured Clinical Examination(OSCE) for student assessment are limited in medical colleges in Korea. The purposes of this study were to develop an OSCE to fourth-year medical students after completion of all clerkships at Hanyang University Medical College. METHODS: The OSCE was a 8-station examination, with each station taking five minutes. We conducted two parallel OSCEs simultaneously by duplication. We checked each stage of the process in the development of OSCEs with recording of diary. We analysed validity and costs of the OSCE. We got feedback from all participants by questionnaires. RESULTS: Of the total 102 fourth-year students, only 74 persons(72.5%) were evaluated. The rate of pass was 63.5%. Cronbach alpha of the OSCE was 0.14. The scores were different between sites according to stations, especially in items related to attitude and physical. The total OSCE score was not significantly different either between duplication sites or among groups. OSCE scores didn't relate to those of both multiple choice tests and entrance exam for internship. Our use of only a few SP's contributed to relatively low cost of $85 per examinee. Both students and faculty were satisfied with the examination, and felt that the material tested was relevant and appropriate for primary care. The OSCE process served to identify weakness in the curriculum and/or teaching methods, and thus could serve as a mechanism to improve educational effectiveness. CONCLUSIONS: Problems of validity and reliability were detected in the developing process of the OSCE. It appeared financially feasible. Setting appropriate goal, optimum number of station, training of evaluators, development of good checklist, and enthusiastic support of the school's administartion were all needed more to success of such a program.