Exploring the validity of assessment in anatomy: do images influence cognitive processes used in answering extended matching questions?

Assessment is an important aspect of medical education because it tests students' competence and motivates them to study. Various assessment methods, with and without images, are used in the study of anatomy. In this study, we investigated the use of extended matching questions (EMQs). To gain insight into the influence of images on the validity of test items, we focused on students' cognitive processes while they answered questions with and without images. Seventeen first-year medical students answered EMQs about gross anatomy, combined with either labeled images or answer lists, while thinking aloud. The participants' verbal reports were transcribed verbatim and then coded. Initial codes were based on a task analysis and were adapted into final codes during the coding process. Results showed that students used more cues from EMQs with images and visualized more often in EMQs with answer lists. Ready knowledge and verbal reasoning were used equally often in both conditions. In conclusion, EMQs with and without images elicit different results in this think aloud experiment, indicating different cognitive processes. They seem to measure different skills, making them valid for different testing purposes. The take-home message for anatomy teachers is that questions without images seem to test the quality of students' mental images while questions with images test their ability to interpret visual information. It makes sense to use both response formats in tests. Using images from clinical practice instead of anatomical drawings will help to improve test validity.

Construction and validation of the Nijmegen Evaluation of the Simulated Patient (NESP): assessing simulated patients' ability to role-play and provide feedback to students.

PURPOSE: To develop an instrument that could be used at different medical schools to assess the performance of individual simulated patients (SPs) on their ability to role-play and provide feedback, including feedback on medical students' communication skills and medical knowledge. METHOD: In 2011, the authors sought to develop and validate a new instrument to achieve these goals. During Phase 1, one of the authors conducted semistructured interviews with medical students, medical psychologists, physicians, and experts to explore what was required of SPs. During Phase 2, the authors assessed the reliability and feasibility of the instrument that they had developed during Phase 1, using a principal components analysis with varimax rotation. During Phase 3, they performed a confirmatory factor analysis on the items in the final instrument. RESULTS: During Phase 1, the authors constructed the Nijmegen Evaluation of the Simulated Patient (NESP), which included three components-role-play, process of feedback, and application of feedback rules. They then determined that approximately 8 to 20 completed instruments per SP led to satisfactory to excellent reliability estimates. In Phase 3, their confirmatory analysis confirmed the three components that they had determined in Phase 2. Reliability estimates for each component (role-play, process of feedback, application of feedback rules) and the final NESP as a whole were 0.86, 0.83, 0.79, and 0.92, respectively. CONCLUSIONS: The authors concluded that the NESP is a feasible, valid, and reliable instrument that could be used at different medical schools to assess the performance of individual SPs.