ABSTRACT
STATEMENT: A phenomenon is occurring in international settings where the language of program delivery and assessment does not match the primary language of practice. It is unknown whether determining competence in English disadvantages students for practice in non-English settings. As such, we conducted a pilot study to determine student performance and perceptions after completion of two Objective Structured Clinical Examinations (OSCEs) examinations, one conducted in English and one conducted in Arabic within an Arabic-speaking Middle Eastern setting. Twenty-two students completed both OSCEs. Overall scores were similar but student rankings differed. Students were more confident performing in Arabic, felt that the Arabic examination was more reflective of practice, and believed that use of Arabic OSCEs can promote better patient care. Findings support the notion that student success may be influenced by language of assessment and that we may need to rethink how we determine assessment validity in these emerging international education settings.
Subject(s)
Educational Measurement/methods , Language , Clinical Competence/standards , Educational Measurement/standards , Humans , Pilot Projects , QatarABSTRACT
BACKGROUND AND PURPOSE: Objective structured clinical examinations (OSCEs) are considered gold standard performance-based assessments yet comprehensive evaluation data is currently lacking. The objective of this study was to critically evaluate the psychometric properties of a cumulative OSCE for graduating pharmacy students in Qatar for which policies and procedures were adapted from a Canadian context. EDUCATIONAL ACTIVITY AND SETTING: A 10-station OSCE was conducted for graduating students in Qatar. Evaluation included assessment of pass rates, predictive validity, concurrent validity, internal validity, content validity, interrater reliability, and internal consistency. FINDINGS: Twenty-six students completed the OSCE. Three stations achieved pass rates < 80%. Scores from professional skills and case-based learning courses, formative OSCEs, and cumulative grade point averages correlated with OSCE scores (p < 0.05). Average correlation between assessors' analytical and global scoring was moderate (r = 0.52). Average interrater reliability was excellent for analytical scoring (ICC = 0.88) and moderate for global scoring (ICC = 0.61). Excellent internal consistency was demonstrated for overall performance (α = 0.927). Students generally agreed stations represented real practice scenarios (range per station, 30-100%). DISCUSSION AND SUMMARY: The evaluation model identified strengths and weaknesses in assessment and curricular considerations. The OSCE demonstrated acceptable validity and reliability as an adapted assessment.
