Item format statistics and readability of extended matching questions as an effective tool to assess medical students.

Testing based on multiple choice questions (MCQ) is one of the most established forms of assessment, not only in the medical field. Extended matching questions (EMQ) represent a specific type of MCQ designed to require higher levels of cognition, such as problem-solving. The purpose of this evaluation was to assess the suitability and efficiency of EMQ as an assessment method. EMQ were incorporated into the end-of-semester examination in internal medicine, in which 154 students participated, and compared with three established MCQ types. Item and examination quality were investigated, as well as readability and processing time. EMQ were slightly more difficult to score; however, both item discrimination and discrimination index were higher when compared to other item types. EMQ were found to be significantly longer and required more processing time, but readability was improved. Students judged EMQ as clearly challenging, but attributed significantly higher clinical relevance when compared to established MCQ formats. Using the Spearman-Brown prediction, only ten EMQ items would be needed to reproduce the Cronbach's alpha value of 0.75 attained for the overall examination. EMQ proved to be both efficient and suitable when assessing medical students, demonstrating powerful characteristics of reliability. Their expanded use in favor of common MCQ could save examination time without losing out on statistical quality.

[Examinations while studying medicine - more than simply grades]. / Prüfungen im Medizinstudium ­ mehr als nur Noten.

Assessment drives learning. Examinations need to be aligned primarily with learning objectives, as well as teaching and assessment methods of the courses on offer. In doing so, various examination instruments are required to measure on levels of competency that build on one another. An appropriate mix is essential to reflect the variety of learning outcomes of a chosen curriculum. Furthermore, examinations also possess the characteristics of evaluation: They reflect the knowledge and abilities of students and assess the teaching at a defined location. Digital examinations in the form of multiple-choice-question (MCQ) testing enable a higher degree of automation and accelerate the processes of creation, implementation, and evaluation of the examination results. Thus, they enjoy increasing popularity, provided that the technical requirements for large semester cohorts are met. Shifting examination processes to computers or tablets entails not only a wealth of new challenges but also opportunities.

Cost analysis for computer supported multiple-choice paper examinations.

INTRODUCTION: Multiple-choice-examinations are still fundamental for assessment in medical degree programs. In addition to content related research, the optimization of the technical procedure is an important question. Medical examiners face three options: paper-based examinations with or without computer support or completely electronic examinations. Critical aspects are the effort for formatting, the logistic effort during the actual examination, quality, promptness and effort of the correction, the time for making the documents available for inspection by the students, and the statistical analysis of the examination results. METHODS: Since three semesters a computer program for input and formatting of MC-questions in medical and other paper-based examinations is used and continuously improved at Wuerzburg University. In the winter semester (WS) 2009/10 eleven, in the summer semester (SS) 2010 twelve and in WS 2010/11 thirteen medical examinations were accomplished with the program and automatically evaluated. For the last two semesters the remaining manual workload was recorded. RESULTS: The cost of the formatting and the subsequent analysis including adjustments of the analysis of an average examination with about 140 participants and about 35 questions was 5-7 hours for exams without complications in the winter semester 2009/2010, about 2 hours in SS 2010 and about 1.5 hours in the winter semester 2010/11. Including exams with complications, the average time was about 3 hours per exam in SS 2010 and 2.67 hours for the WS 10/11. DISCUSSION: For conventional multiple-choice exams the computer-based formatting and evaluation of paper-based exams offers a significant time reduction for lecturers in comparison with the manual correction of paper-based exams and compared to purely electronically conducted exams it needs a much simpler technological infrastructure and fewer staff during the exam.

Acceptance of medical training cases as supplement to lectures.

INTRODUCTION: Medical training cases (virtual patients) are in widespread use for student education. Most publications report about development and experiences in one course with training cases. In this paper we compare the acceptance of different training case courses with different usages deployed as supplement to lectures of the medical faculty of Wuerzburg university during a period of three semesters. METHODS: The training cases were developed with the authoring tool CaseTrain and are available for students via the Moodle-based eLearning platform WueCampus at Wuerzburg university. Various data about usage and acceptance is automatically collected. RESULTS: From WS (winter semester) 08/09 till WS 09/10 19 courses with about 200 cases were available. In each semester, about 550 different medical students from Würzburg and 50 students from other universities processed about 12000 training cases and filled in about 2000 evaluation forms. In different courses, the usage varied between less than 50 and more than 5000 processed cases. DISCUSSION: Although students demand training cases as supplement to all lectures, the data show that the usage does not primarily depend on the quality of the available training cases. Instead, the training cases of nearly all case collections were processed extremely often shortly before the examination. It shows that the degree of usage depends primarily on the perceived relevance of the training cases for the examination.

[Virtual patients in medical education: a comparison of various strategies for curricular integration]. / Virtuelle Patienten in der medizinischen Ausbildung: Vergleich verschiedener Strategien zur curricularen Integration.

INTRODUCTION: E-Learning applications are an integral part of modern medical curricula. Nevertheless, there is only a narrow empirical basis for an optimal integration strategy of computerized teaching methods in medical education. This article focuses on the integration experiences with three different online systems for interactive virtual patients (VPs) in clinical undergraduate medical education at the universities of Heidelberg, Munich (LMU), and Wuerzburg. METHODS: Five different integration strategies for VPs are presented and compared with respect to motivational aspects, logistics and acceptance of both students and teachers. RESULTS: A voluntary use of VPs without curricular integration leads to low acceptance. Exam relevance of VPs and integration with face-to-face teaching are key factors for a high level of acceptance. The assets and drawbacks of all described strategies are discussed in the conclusions of this article. CONCLUSIONS: Case-based e-learning can be successfully integrated into clinical undergraduate medical education when key factors are considered.

Assessment of a case-based training system (d3web.Train) in rheumatology.

To improve the learning experiences for third year medical students, a case-based training program based on real patient records has been introduced as a supplement to lectures. It was built with the novel training environment d3web.Train. Optimization of the learning environment d3web.Train for both teachers and learners has been considered as equally important and offers several trade-offs. The training program has been evaluated during three subsequent rheumatology courses, the first one voluntary and the latter two mandatory. For each course, we tracked how often and intensive each student used the system. In addition, the students answered questionnaires about their expectations and experiences. In the first voluntary course, 39/92 students finished at least one case and solved 187 cases in total (average: 4.8 cases from 12 cases available). In the second and third mandatory courses, all students (60 resp. 56) finished at least the required 20 cases with an average usage of 22.5 resp. 29.3 out of 31 available cases. Most of the cases were processed twice by the students in preparation for the final examinations. A positive correlation between processed cases per student and the exam score could be shown (30, 73 and 95% in the first, second and third courses, respectively). The findings clearly exceeded the expectations of the students concerning its usefulness as a supplement to lectures and as good preparation for their job as physician, as well as the learning period to use the program. For working through one case, students needed about 9 min on average. The case-based training system d3web.Train offers a new tool for medical education in rheumatology. The main advantage of the system is the relatively low effort needed to create a case-based program starting from the available dismissal records.

Evaluation of a novel case-based training program (d3web.Train) in hematology.

The new media such as the internet and digital imaging offer new opportunities in medical education. In addition to conventional lectures, we developed a case-based simulation training program of 17 hematology cases using the novel training system d3web.Train. We evaluated the assessment of this internet course by medical students, as well as their results in the hematology exam. From a group of 150 students, 47 worked through at least one case and solved 435 cases in total; in average, these students solved 9.5 cases. Eighteen different students filled in a questionnaire about the training system and 68 questionnaires about individual cases. The main results were the students found the cases very helpful (1.5+/-0.6 on a scale from 1=very helpful to 5=not at all), the training system very good (1.4+/-0.5 on a scale from 1 to 6), and want to work with it further (1.2+/-0.4 on a scale from 1 to 5). During the final examination, those 16 students who answered that they had solved more than 5 from the 17 cases scored significantly better (two-sided t test, p<0.01) in the hematological part of the exam than those 34 students solving 0 to 5 cases. To our knowledge, this is the first student evaluation of a case-based training program in general hematology. The d3web.Train system offers a new and great tool for creating a training program in a reasonable amount of time, because it is able to process available patient records.