Chatbot-based serious games: A useful tool for training medical students? A randomized controlled trial.

OBJECTIVES: Chatbots, conversational agents that walk medical students (MS) though a clinical case, are serious games that seem to be appreciated by MS. Their impact on MS's performance in exams however was not yet evaluated. Chatprogress is a chatbot-based game developed at Paris Descartes University. It contains 8 pulmonology cases with step-by-step answers delivered with pedagogical comments. The CHATPROGRESS study aimed to evaluate the impact of Chatprogress on students' success rate in their end-term exams. METHODS: We conducted a post-test randomized controlled trial held on all fourth-year MS at Paris Descartes University. All MS were asked to follow the University's regular lectures, and half of them were randomly given access to Chatprogress. At the end of the term, medical students were evaluated on pulmonology, cardiology and critical care medicine. MAIN OUTCOMES MEASURES: The primary aim was to evaluate an increase in scores in the pulmonology sub-test for students who had access to Chatprogress, compared to those who didn't. Secondary aims were to evaluate an increase in scores in the overall test (Pulmonology, Cardiology and Critical care medicine test (PCC)) and to evaluate the correlation between access to Chatprogress and overall test score. Finally, students' satisfaction was assessed using a survey. RESULTS: From 10/2018 to 06/2019, 171 students had access to Chatprogress (the Gamers) and among them, 104 ended up using it (the Users). Gamers and Users were compared to 255 Controls with no access to Chatprogress. Differences in scores on the pulmonology sub-test over the academic year were significantly higher among Gamers and Users vs Controls (mean score: 12.7/20 vs 12.0/20, p = 0.0104 and mean score: 12.7/20 vs 12.0/20, p = 0.0365 respectively). This significant difference was present as well in the overall PCC test scores: (mean score: 12.5/20 vs 12.1/20, p = 0.0285 and 12.6/20 vs 12.1/20, p = 0.0355 respectively). Although no significant correlation was found between the pulmonology sub-test's scores and MS's assiduity parameters (number of finished games among the 8 proposed to Users and number of times a User finished a game), there was a trend to a better correlation when users were evaluated on a subject covered by Chatprogress. MS were also found to be fans of this teaching tool, asking for more pedagogical comments even when they got the questions right. CONCLUSION: This randomised controlled trial is the first to demonstrate a significant improvement in students' results (in both the pulmonology subtest and the overall PCC exam) when they had access to Chatbots, and even more so when they actually used it.

Introducing an Open-Source Course Management System (Moodle) for Blended learning on infectious diseases and microbiology: A pre-post observational study.

INTRODUCTION: Attendance at face-to-face courses is low in the 2nd and 3rd years of medical school in France, possibly because of a lack of interactivity. We used Moodle (an open-source course management system) to introduce blended learning on Infectious Diseases and Microbiology through interactive quizzes and sessions of online-based continuous assessment. This pre-post observational study assessed changes in students' attendance and student as well as teacher satisfaction. METHODS: During the 2017-2018 academic session of Infectious Diseases and Microbiology, we used Moodle to include interactive quizzes during courses and to organize five continuous assessment sessions. Pre-post comparisons (2017-2018 vs. 2016-2017) were performed for the following outcomes: attendance rate, satisfaction questionnaire and exam performance. In addition, the students' and teachers' perception of Moodle-based interactive quizzes and continuous assessment sessions in 2017-2018 was assessed with Likert-like scales, closed and open-ended questions. A thematic analysis of the free comments was performed through inductive coding by two coders. RESULTS: In 2017-2018 vs. 2016-2017, mean (±SD) attendance rate was higher [12.5 (±7.2) % vs. 7.9 (±3.5) % of students, P<0.001] and clinical case-based courses, which encompassed 93% of Moodle-based courses in 2017-18, were more frequently considered to improve teaching and learning (81.9% vs. 73.8%, P=0.01). Students more frequently judged the teaching organization and structure to be adequate (85.5% vs. 80.2%, p=0.03) and more frequently recommended to next-year students that they attend courses (96.1% vs. 42.1%, P<0.001). CONCLUSION: Using Moodle for blended learning on Infectious Diseases and Microbiology improved student satisfaction and attendance at face-to-face courses.

Impact of integrating objective structured clinical examination into academic student assessment: Large-scale experience in a French medical school.

PURPOSE: Objective structured clinical examinations (OSCE) evaluate clinical reasoning, communication skills, and interpersonal behavior during medical education. In France, clinical training has long relied on bedside clinical practice in academic hospitals. The need for a simulated teaching environment has recently emerged, due to the increasing number of students admitted to medical schools, and the necessity of objectively evaluating practical skills. This study aimed at investigating the relationships between OSCE grades and current evaluation modalities. METHODS: Three-hundred seventy-nine 4th-year students of University-of-Paris Medical School participated to the first large-scale OSCE at this institution, consisting in three OSCE stations (OSCE#1-3). OSCE#1 and #2 focused on cardiovascular clinical skills and competence, whereas OSCE#3 focused on relational skills while providing explanations before planned cholecystectomy. We investigated correlations of OSCE grades with multiple choice (MCQ)-based written examinations and evaluations of clinical skills and behavior (during hospital traineeships); OSCE grade distribution; and the impact of integrating OSCE grades into the current evaluation in terms of student ranking. RESULTS: The competence-oriented OSCE#1 and OSCE#2 grades correlated only with MCQ grades (r = 0.19, P<0.001) or traineeship skill grades (r = 0.17, P = 0.001), respectively, and not with traineeship behavior grades (P>0.75). Conversely, the behavior-oriented OSCE#3 grades correlated with traineeship skill and behavior grades (r = 0.19, P<0.001, and r = 0.12, P = 0.032), but not with MCQ grades (P = 0.09). The dispersion of OSCE grades was wider than for MCQ examinations (P<0.001). When OSCE grades were integrated to the final fourth-year grade with an incremental 10%, 20% or 40% coefficient, an increasing proportion of the 379 students had a ranking variation by ±50 ranks (P<0.001). This ranking change mainly affected students among the mid-50% of ranking. CONCLUSION: This large-scale French experience showed that OSCE designed to assess a combination of clinical competence and behavioral skills, increases the discriminatory capacity of current evaluations modalities in French medical schools.

DIVA, a 3D virtual reality platform, improves undergraduate craniofacial trauma education.

Craniofacial fractures management is challenging to teach due to the complex anatomy of the head, even when using three-dimensional CT-scan images. DIVA is a software allowing the straightforward visualization of CT-scans in a user-friendly three-dimensional virtual reality environment. Here, we assess DIVA as an educational tool for craniofacial trauma for undergraduate medical students. Three craniofacial trauma cases (jaw fracture, naso-orbital-ethmoid complex fracture and Le Fort 3 fracture) were submitted to 50 undergraduate medical students, who had to provide diagnoses and treatment plans. Each student then filled an 8-item questionnaire assessing satisfaction, potential benefit, ease of use and tolerance. Additionally, 4 postgraduate students were requested to explore these cases and to place 6 anatomical landmarks on both virtual reality renderings and usual slice-based three-dimensional CT-scan visualizations. High degrees of satisfaction (98%) without specific tolerance issues (86%) were reported. The potential benefit in a better understanding of craniofacial trauma using virtual reality was reported by almost all students (98%). Virtual reality allowed a reliable localization of key anatomical landmarks when compared with standard three-dimensional CT-scan visualization. Virtual reality interfaces such DIVA are beneficial to medical students for a better understanding of craniofacial trauma and allow a reliable rendering of craniofacial anatomy.