Effects and sustainability of trainings for the oral and practical part of the German final exam in medicine.

STUDY GOALS: It is known that the manifold limitations of oral and practical examinations can be improved by specific training. With the help of an online survey, our present study analyzes the effects that can be achieved by the training conducted at the University of Ulm for examiners in the final medical examination, the long-lasting impact of the training, and differences among participant subgroups. METHOD: All 367 participants in the training at Ulm (2007- 2012) were contacted via email. Sixty-three persons responded to the survey that included 28 items concerning demographic data, effectiveness, and sustainability. RESULTS: Six main effects of the training were identified (meaning effects rated with a grade of 1 or 2 on a 6-point scale by two thirds of the participants, with 1="applicable" and 6="not applicable"; cumulated percentage of answers of 1 or 2 in parentheses): 1. Conscious handling of strengths and weaknesses of oral examinations (71%), 2. Knowledge of factors contributing to the reliability of oral/practical examinations (76%), 3. Knowledge of factors contributing to the validity of oral/practical examinations (75%), 4. Improvement of competence in task construction (68%), 5. Improvement of competence in respect to examination formalities (75%), 6. Implementation of the concept of "structured oral examinations" (a priori planning of examination subjects, tasks, levels of expectation and grading criteria) (86%). The responses of participants trained more than two years ago were not significantly different from the answers given by recently trained persons. This is an argument for the sustainability of the training effects. Furthermore, participants without relevant prior experience in oral/practical examinations profited significantly more from the trainings, especially in the areas of stress reduction, confidence in grading, and competence in critical discrimination of grading. CONCLUSION: The positive and sustained effects of the examiner training argue for continuing the training program, especially for inexperienced examiners. Expansion of the successful training program to include the first medical exam should be considered.

The impact of a dedicated training program for oral examiners at a medical school in Germany: a survey among participants from operative and non-operative disciplines.

BACKGROUND: Oral examinations have been a crucial format in ancient and modern assessment to evaluate and guarantee quality of medical education and thereby to secure patient safety. To achieve a high level of quality in the oral part of the final examination of medical students, a training program for oral examiners at the Medical Faculty of Ulm (Germany) has been established since 2007.However, little is known about the attitude of the examiners in regard to the impact of this training program and of oral examinations as instruments to ensure patient safety. METHODS: All 367 academic clinicians from operative and non-operative disciplines, attending the one-day examiner training program at the University of Ulm between 2007 and 2012 have been asked to answer an online survey (EvaSys 5.0). Focus of the survey was to find out in which respect the examiners profited from the trainings, if the training effects were discipline-dependent, and to which degree the oral examinations could contribute to patient safety. Statistical analysis was performed using the t-test for independent samples. Results were considered statistically significant when p < 0.05. RESULTS: A total of 63 participants answered the survey, but in 4 cases the questionnaire was not fully completed (with single items missing). More than half of the study participants (n = 34/59; 58%) have experienced (at least sometimes or rarely) candidates that they deemed incompetent and perhaps even dangerous to the patients' health who nevertheless passed the oral exam successfully. The majority of participants were convinced that oral examinations using concrete clinical cases could significantly contribute to patient safety, if grading is based on clear criteria and if examinations as well as grading are performed more critically. The impact of the training program was rated significantly stronger by surgeons than by non-surgeons in several categories. These categories included "strengths and weaknesses of oral examinations", "reliability", "validity", "competence in grading", "critical grading", and "departmental improvements" concerning oral examinations. CONCLUSIONS: In respect to patient safety, it seems crucial to prevent incompetent candidates from passing the oral examination. The present study indicates the importance to continue and to develop our examiner trainings, with main emphasis on concrete clinical problems and a criteria-based critical grading system for oral examinations. Since the impact of the training was particularly high for colleagues from the operative disciplines, the training program should be offered especially in surgical departments.