Reliability of physiology MCQs' examinations at the Faculty of Medicine, University of Khartoum

ABSTRACT

Background: One of the major challenges that face exam constructors is generating highly reliable exams. An assessment cannot be viewed as valid unless it is reliable. Test reliability, which is the best single measure of test accuracy, is the extent to which test results are consistent, stable, reproducible and free of error variance. Reliability isinfluenced by internal factors related to exam construction, as well as external factors which depend on the situation of test administration. Objective: To estimate the reliability of multiple choice questions (MCQs) of physiology exams as part of an overall quality assessment at the Faculty of Medicine, University of Khartoum.Methods:Reliability influential factors related to exam construction and administration were controlled and catered for by departmental and administrative staff according to the exam regulations of the faculty. Remark software was used for post-examination analysis of scores of ten consecutive summative physiology MCQ exams at the Faculty of Medicine, University of Khartoum. The number of the examinees who sat for each of the ten exams ranged from 332­359. In addition to reliability coefficients, item difficulty index (DIF I) and point-biserial correlation coefficient (rpbis)as a measure of item discrimination ability, were calculated as part of item analysis results.Results:The study revealed high exams' reliability (0.84-0.95) as measured by different formulas [Kuder-Richard-son Formulas (KR-20, KR-21) and Cronbach's-Alpha], and low standard error of measurement/SEM (3.073.80).Factorswhichcontributed to the high reliability of our ten exams were their high discrimination power (0.32-0.47), their recommended mean difficulty (48.62-65.67%), and the relatively large numbers of items (60­80) per each exam.Conclusion:The high exams reliability of this study was an indicator of the precise control of external and internal factors influencing reliability. The most important contributing factor was the proper construction of exams with high quality items; in addition to careful exam administration and meticulous scoring system