effect on student performance of scrambling questions and their stems in medical colleges admission tests

ABSTRACT

Assessment is an indispensable part of an educational program. Multiple Choice Questions [MCQs] is an objective tool of assessment provided cheating is controlled. A method employed to reduce the chance of cheating is to scramble the sequence of the MCQs and responses in multiple papers having the same content. It is assumed that the performance of students is mainly dependent on the difficulty of the items and not the order in which they are placed within the instrument. The marks obtained by 1,02,211 candidates sitting in Medical Colleges Admission Test [MCAT] from 2008 to 2011 and given similar-content but scrambled-sequence question paper codes were analyzed using parametric tests. A significant difference amongst the mean marks of candidates in the different codes of MCAT 2008 [F = 22.15, p < 0.001] and MCAT 2011 [F = 3.85, p = 0.009] was identified. No significant difference was found in the mean marks of the candidates' each year for different codes in each centre.