Introducing Summative Progress Testing in Radiology Residency: Little Change in Residents' Test Results After Transitioning from Formative Progress Testing.

INTRODUCTION: Educational effects of transitioning from formative to summative progress testing are unclear. Our purpose was to investigate whether such transitioning in radiology residency is associated with a change in progress test results. METHODS: We investigated a national cohort of radiology residents (N > 300) who were semi-annually assessed through a mandatory progress test. Until 2014, this test was purely formative for all residents, but in 2014/2015, it was transitioned (as part of a national radiology residency program revision) to include a summative pass requirement for new residents. In 7 posttransitioning tests in 2015-2019, including summatively and formatively tested residents who followed the revised and pre-transitioning residency program, respectively, we assessed residents' relative test scores and percentage of residents that reached pass standards. RESULTS: Due to our educational setting, most posttransitioning tests had no residents in the summative condition in postgraduate year 4-5, nor residents in the formative condition in year 0.5-2. Across the 7 tests, relative test scores in postgraduate year 1-3 of the summative resident group and year 3.5-4.5 of the formative group differed significantly (p < 0.01 and p < 0.05, respectively, Kruskal-Wallis test). However, scores fluctuated without consistent time trends and without consistent differences between both resident groups. Percentage of residents reaching the pass standard did not differ significantly across tests or between groups. DISCUSSION: Transitioning from formative to summative progress testing was associated with overall steady test results of the whole resident group in 4 post-transitioning years. We do not exclude that transitioning may have positive educational effects for resident subgroups.

A review of interprofessional training wards: Enhancing student learning and patient outcomes.

Background: In an interprofessional training ward (ITW), students from different health professions collaboratively perform patient care with the goal of improving patient care. In the past two decades, ITWs have been established world-wide and studies have investigated their benefits. We aimed to compare ITWs with respect to their logistics, interprofessional learning outcomes and patient outcomes. Methods: We explored PubMed, CINAHL, Web of Science and EMBASE (1990-June 2017) and included articles focusing on interprofessional, in-patient training wards with student teams of medical and other health professions students. Two independent reviewers screened studies for eligibility and extracted data. Results: Thirty-seven articles from twelve different institutions with ITWs were included. ITWs world-wide are organized similarly with groups of 2-12 students (i.e. medical, nursing, physiotherapy, occupational therapy, and pharmacy) being involved in patient care, usually for a period of two weeks. However, the type of clinical ward and the way supervisors are trained differ. Conclusions: ITWs show promising results in short-term student learning outcomes and patient satisfaction rates. Future ITW studies should measure students' long-term interprofessional competencies using standardized tools. Furthermore, a research focus on the impact of ITWs on patient satisfaction and relevant patient care outcomes is important.

Successful e-learning programme on the detection of child abuse in emergency departments: a randomised controlled trial.

OBJECTIVE: To evaluate the effectiveness of an electronic learning (e-learning) programme on the performance of nurses in the recognition of child abuse in a simulated case in the Emergency Department (ED). DESIGN: Blinded, randomised controlled trial using pre- and postintervention design. SETTING: The ED of a University Medical Center in the Netherlands. PARTICIPANTS: 38 ED nurses were included, 25 nurses were analysed. INTERVENTION: Half of the participants followed a 2-hour e-learning programme focused on the recognition of child abuse, the others acted as a control group. MAIN OUTCOME MEASUREMENTS: Individual performance during a case-simulated parent interview to detect child abuse and self-reported self-efficacy for the detection of child abuse. Performance on the simulation was scored by an expert panel using a standardised assessment form which was designed to score quantity and quality of the questions posed by the nurse (minimum score 0; maximum score 114). RESULTS: During post-test, nurses in the intervention group performed significantly better during the simulation than the control group, (89 vs 71, 95% CI 2.9 to 33.3), and reported higher self-efficacy (502 vs 447, 95% CI -25.4 to 134.7). Performance in detecting child abuse correlated positively with the self-efficacy score (Spearman correlation 0.387, p value 0.056). Comparing post- and pretest results separately for the intervention and the control group showed a significant increase in performance in the intervention group. CONCLUSION: E-learning improved the performance in case simulations and the self-efficacy of the nurses in the ED in the detection of child abuse. Wider implementation of the e-learning programme to improve the first step in the detection of child abuse is recommended.

Transatlantic comparison of the competence of surgeons at the start of their professional career.

BACKGROUND: Although the objective in European Union and North American surgical residency programmes is similar-to train competent surgeons-residents' working hours are different. It was hypothesized that practice-ready surgeons with more working hours would perform significantly better than those being educated within shorter working week curricula. METHODS: At each test site, 21 practice-ready candidate surgeons were recruited. Twenty qualified Canadian and 19 qualified Dutch surgeons served as examiners. At both sites, three validated outcome instruments assessing multiple aspects of surgical competency were used. RESULTS: No significant differences were found in performance on the integrative and cognitive examination (Comprehensive Integrative Puzzle) or the technical skills test (Objective Structured Assessment of Technical Skill; OSATS). A significant difference in outcome was observed only on the Patient Assessment and Management Examination, which focuses on skills needed to manage patients with complex problems (P < 0.001). A significant interaction was observed between examiner and candidate origins for both task-specific OSATS checklist (P = 0.001) and OSATS global rating scale (P < 0.001) scores. CONCLUSION: Canadian residents, serving many more working hours, perform equivalently to Dutch residents when assessed on technical skills and cognitive knowledge, but outperformed Dutch residents in skills for patient management. Secondary analyses suggested that cultural differences influence the assessment process significantly.

Assessment of medical students' communicative behaviour and attitudes: estimating the reliability of the use of the Amsterdam attitudes and communication scale through generalisability coefficients.

It is widely accepted that adequate attitudes and communicative skills are among the essential objectives in medical education. The Amsterdam attitude and communication scale (AACS) was developed to assess communicative skills and professional attitudes of medical students. More specifically, it was designed to evaluate the clinical behaviour of clerks to establish their suitability for the medical profession. The AACS covers nine dimensions. Moreover, an overall judgement of the student's performance is included. The present paper reports first results on the reliability of the use of the AACS. Data were collected in the course of an AACS training programme for future judges: senior medical and nursing staff members (N=98). Participants judged three videotapes of clerks interviewing patients at the bedside. For the assessment of videotapes, the first four dimensions of the AACS and the overall judgement are relevant. By applying Generalisability Theory to the training data we can forecast the reliability of the AACS in practice and gain insight in the number of raters that is needed to achieve sufficient reliability in clinical practice. If clerk behaviour is rated by six judges, summative assessment is sufficiently precise, i.e. <0.25. When using the full AACS, covering 10 items, the same number of judges is needed. Scores on individual AACS items are not sufficiently reliable. In conclusion, the results indicate that students' behaviour can be evaluated in a reliable manner using the AACS as long as enough judges and items are involved.

What happens to the student? The neglected variable in educational outcome research.

Disputes about the superiority of teaching methods often remain unresolved. The essential question we continuously want to answer is: Which teaching methods yield the best knowledge and skills in students? Abundant literature, in medical education and in education in general, on research with educational methods as independent variables and measures of outcome (e.g., test scores) as the dependent variable often point at "no significant difference" or only small differences between methods. Many factors do influence the educational outcome in students and large statistical power (such as meta analysis) should be helpful to eliminate many sources of error. However, one source we cannot tackle this way. That is, students will usually adapt quantity and quality of studying to meet testing requirements. In doing so, they may compensate for teaching quality. Some teaching may generate more effort in students than other teaching. Since test scores reflect primarily student activities, it is their efforts that may bring differences in teaching methods close to equality in test scores. Therefore, knowledge and skills should not be considered the primary outcome of teaching but the outcome of learning activities. If we want to discriminate between teaching methods, we must at least consider what happens to students.

Strategic planning in medical education: enhancing the learning environment for students in clinical settings.

BACKGROUND: The 1999 Cambridge Conference was held in Northern Queensland, Australia, on the theme of clinical teaching and learning. It provided an opportunity for groups of academic medical educators to consider some of the challenges posed by recent changes to health care delivery and medical education across a number of countries. PURPOSE: This paper describes the issues raised by the practical challenges posed by the current environment and how they might be addressed in ways that could promote more effective learning in clinical settings. METHOD: A SWOT analysis is a tool that can help in forward planning by identifying the strengths, weaknesses, opportunities and threats presented by any situation. Our SWOT analysis was used to generate a list of items, from which we chose those most feasible and most likely to promote positive change. RESULTS: Twenty different issues were identified, with four of them chosen by consensus for further elaboration. The discussion gave rise to four main recommended strategies: ensuring that clinical teachers thoroughly understand the purpose and process of learning in clinical settings; equipping learners with 'survival skills'; making the best use of learning resources within different clinical environments and making judicious use of information technology to enhance learning efficiency. CONCLUSIONS: The four strategies were selected not only because of their inherent importance, but also because of their feasibility. Modest changes can motivate students to feel part of a clinical team and a 'community of practice' and enhance their capacity for self-regulated practice.