Can ChatGPT generate practice question explanations for medical students, a new faculty teaching tool?

INTRODUCTION: Multiple-choice questions (MCQs) are frequently used for formative assessment in medical school but often lack sufficient answer explanations given time-restraints of faculty. Chat Generated Pre-trained Transformer (ChatGPT) has emerged as a potential student learning aid and faculty teaching tool. This study aims to evaluate ChatGPT's performance in answering and providing explanations for MCQs. METHOD: Ninety-four faculty-generated MCQs were collected from the pre-clerkship curriculum at a US medical school. ChatGPT's accuracy in answering MCQ's were tracked on first attempt without an answer prompt (Pass 1) and after being given a prompt for the correct answer (Pass 2). Explanations provided by ChatGPT were compared with faculty-generated explanations, and a 3-point evaluation scale was used to assess accuracy and thoroughness compared to faculty-generated answers. RESULTS: On first attempt, ChatGPT demonstrated a 75% accuracy in correctly answering faculty-generated MCQs. Among correctly answered questions, 66.4% of ChatGPT's explanations matched faculty explanations, and 89.1% captured some key aspects without providing inaccurate information. The amount of inaccurately generated explanations increases significantly if the questions was not answered correctly on the first pass (2.7% if correct on first pass vs. 34.6% if incorrect on first pass, p < 0.001). CONCLUSION: ChatGPT shows promise in assisting faculty and students with explanations for practice MCQ's but should be used with caution. Faculty should review explanations and supplement to ensure coverage of learning objectives. Students can benefit from ChatGPT for immediate feedback through explanations if ChatGPT answers the question correctly on the first try. If the question is answered incorrectly students should remain cautious of the explanation and seek clarification from instructors.

Where are we now? Evaluating the one year impact of an anti-racism curriculum review.

PURPOSE: Undergraduate medical education has had a call to action to acknowledge racist practices that are impacting learners throughout their training. In 2020, our school performed a detailed curricular review and provided recommendations to address racism in the curriculum. Many schools have now undergone a similar curricular review process, but little is known about whether suggested antiracist curricular changes impact faculty teaching behavior or the overall curriculum. MATERIAL AND METHODS: In 2021, as part of the medical school's annual educational quality improvement process, course directors were required to answer a question about the changes they made to address racism in their courses based on recommendations provided the year prior from an antiracism curricular review. The documented changes were analyzed for themes and then organized by course and curricular year. These changes were compared with the suggested recommendations to analyze the number and types of changes implemented after one year. To evaluate student perceptions of change the general comments from academic years 2019-2021 were reviewed. RESULTS: After 1 year, approximately, 74% of our school's 328 anti-racism curricular review recommendations were implemented in courses. Over 80% were implemented in curricular year 1. The greatest number of recommendations implemented were related to the theme of critiquing the strength of evidence in race-based medical practices. The least amount change was made around the theme of challenging the biologic notion of race. CONCLUSIONS: An antiracism curricular review followed by an embedded continuous quality improvement process can be an effective approach to address racism in medical school curricula. Addressing racism in medical education requires medical schools to regularly identify curricular gaps, faculty needs and monitor their progress.

Curricular Reform in Serious Illness Communication and Palliative Care: Using Medical Students' Voices to Guide Change.

PURPOSE: To gather and leverage the voices of students to drive creation of required, integrated palliative care curricula within undergraduate medical education in Massachusetts, which is lacking in a majority of U.S. medical schools. METHOD: The study was conducted by the Massachusetts Medical Schools' Collaborative, a working group committed to ensuring all medical students in Massachusetts receive foundational training in serious illness communication (SIC) and palliative care. Eight focus groups (2 per participating medical school) were conducted during January-May 2021 and included a total of 50 students from Boston University Chobanian & Avedisian School of Medicine, Harvard Medical School, Tufts University School of Medicine, and the UMass Chan Medical School. Data collected from focus groups were discussed and coded. Themes were identified using the immersion/crystallization qualitative data analysis approach. RESULTS: Six key themes emerged. Students viewed SIC as essential to high-quality medical practice regardless of specialty, and believed training in SIC skills and palliative care should be required in medical school curricula. Students preferred to learn and practice these skills using frameworks, particularly in real-world situations. Students recognized the expertise of palliative care specialists and described them as a scarce, often misunderstood resource in health care. Students reported it was mostly "luck" if they were included in family meetings and observed good role models. Finally, students desired practice in debriefing after difficult and emotional situations. CONCLUSIONS: This study confirms long-standing themes on students' experiences with SIC and palliative care topics, including feeling inadequately prepared to care for seriously ill patients as future physicians. Our study collected students' perspectives as actionable data to develop recommendations for curricular change. Collaborative faculty also created recommendations based on the focus group data for immediate and ongoing SIC and palliative care curricular change in Massachusetts, which can apply to medical schools nationwide.

A Call to Action: Using Curriculum Mapping at Four Medical Schools in Massachusetts to Advance Serious Illness Communication Training in Undergraduate Medical Education.

Background: Practicing physicians require serious illness communication (SIC) skills to ensure high-quality, humanistic care for patients and families as they face life-changing medical decisions. However, a majority of U.S. medical schools do not require formal training in SIC and fail to provide students deliberate practice before graduation. The Massachusetts Medical Schools' Collaborative was created to ensure that students receive foundational SIC training in undergraduate medical education. This Collaborative developed a curriculum-mapping tool to assess SIC at four medical schools. Objective: We aimed to understand existing educational activities across four medical schools and identify opportunities to build longitudinal, developmentally based curricular threads in SIC. Design: From July 2019 to April 2021, faculty, staff, and medical students assessed current educational activities related to five core competencies in SIC, adapted for students from national competencies for palliative medicine fellows, using a curriculum mapping tool. Measurements: The group selected 23 keywords and collected metrics to describe the timing, instruction and assessment for each school's educational activities. Results: On average, there were only 40 hours of required curricula in SIC over four years. Over 80% of relevant SIC hours occurred as elective experiences, mostly during the postclerkship phase, with limited capacity in these elective experiences. Only one school had SIC educational activities during the clerkship phase when students are developing clinical competencies. Assessment methods focused on student participation, and no school-assessed clinical performance in the clerkship or postclerkship phase. Conclusions: Medical schools are failing to consistently train and ensure basic competency in effective, compassionate SIC. Curriculum mapping allows schools to evaluate their current state on a particular topic such as SIC, ensure proper assessment, and evaluate curricular changes over time. Through the deliberate inclusion of SIC competencies in longitudinal curriculum design, we can fill this training gap and create best practices in undergraduate medical education.

Identifying Student Mistreatment Through Peer-Facilitated Learning Environment Sessions.

PROBLEM: Data from the Association of American Medical Colleges (AAMC) Medical School Graduation Questionnaire (GQ) show persistent high rates of medical student mistreatment, and multiple barriers to student reporting of mistreatment exist. The authors examined whether learning environment sessions (LESs) allow students opportunities to identify and describe patterns of mistreatment missed by other avenues of reporting. APPROACH: Peer-facilitated LESs were instituted in 2018-2019 at Boston University School of Medicine. The LESs were scheduled once during every third-year core clerkship block. Third- and fourth-year students trained as peer-facilitators led discussions of topics relevant to the student clinical experience using a standardized facilitator guide. Minutes, including details of reported events, were completed during the session and visible to all students participating. These minutes were sent to clerkship leadership and the medical education office for action once student grades were submitted. OUTCOMES: Summative content analysis was conducted on 44 LES minutes from sessions held in January-November 2019. Reported incidents were categorized into broad categories of negative treatment (NT), negative learning environment (NLE), and positive learning environment (PLE). Sixty-three instances of NT were identified. Of these, 37 fit within the scope of the AAMC GQ mistreatment categories. The remaining 26 instances of NT were classified into 7 novel categories of medical student mistreatment. Instances of NLE were most discussed by students and categorized into 5 subthemes. Examples of PLE were categorized into 4 subthemes, which encompassed 11 descriptors of core qualities of an ideal preceptor or educational environment. NEXT STEPS: LESs have aided in identifying and describing new patterns of mistreatment. They fulfill a unique role by allowing students to identify, analyze, and report mistreatment in a facilitated and protected space. Formal evaluations of institutional improvement in the learning environment, reduction in medical student mistreatment, and subsequent improvement in AAMC GQ data are needed.

Virtual Reality Simulated Learning Environments: A Strategy to Teach Interprofessional Students About Social Determinants of Health.

PURPOSE: Physician assistants (PAs) and medical degree students (MDs) often lack training in addressing the social determinants of health (SDOH). Social work students (SWs), meanwhile, have extensive SDOH training; however, few medical professionals have opportunities to engage in interprofessional training with SWs. This study examined the feasibility, acceptability, and students' perceptions of an interprofessional virtual reality (VR) simulated learning environment (SLE) for teaching health professions students about the SDOH. METHOD: In January 2020, 15 students at Boston University School of Medicine attended web-based video conferences focused on SDOH, health equity, and team-based care. Subsequently, student dyads participated in a case-based learning activity using an immersive VR SLE to develop teamwork skills. Evaluation included a postsurvey and a focus group examining their experiences in the course to gauge feasibility and acceptability. Thematic analysis of open-ended survey responses from the postsurvey and focus group data was conducted. RESULTS: A total of 8 VR simulations were run. Findings indicated both MD and PA students learned patient engagement strategies from SW students, who enhanced their health care leadership capacity. Participants found the means of instruction acceptable, valued the hands-on VR interprofessional training, and expressed interest in learning more about the scope of one another's roles and the community resources available to patients. CONCLUSIONS: VR SLE is a feasible and acceptable means of instruction. It allowed students to connect across programmatic and geographic boundaries in a collaborative working environment mimicking the team approach to care they will use in their professional life. This experience illustrated for students the strengths a multidisciplinary team has to offer.

Deconstructing Racism, Hierarchy, and Power in Medical Education: Guiding Principles on Inclusive Curriculum Design.

In the context of current U.S. racial justice movements, analysis of racism in medicine within medical education is a critical task for all institutions. To educate the next generation of physicians about racism in medicine and out of concern that the curriculum required critical assessment and change, a group of students and faculty at Boston University School of Medicine (BUSM) initiated a longitudinal curricular analysis through a vertical integration group, commissioned by the Medical Education Committee, from May 2019 to June 2020. The curriculum analysis and the major outcomes and guiding principles that emerged from it are described as a path forward, toward a more inclusive curriculum. The major elements of this analysis included a comprehensive internal curricular assessment and an external assessment of peer institutions that led to the development of key curricular recommendations and overarching equity and specific racially focused equity competencies. The curricular recommendations fall into the following domains: (1) challenging the persistence of biological/genetic notions of race, (2) embedding structural practices in medical education to dismantle racism in medicine, and (3) promoting institutional climate change. Initial steps to implement these recommendations are described. The authors believe that the historic and present reality of racism in America and in medicine has impacted medical education specifically, and more broadly, the practice of medicine, trainee experience, and patient outcomes. The key findings of the BUSM analysis are transferable to other medical education institutions, and the described review process can support peer institutions as they engage in the imperative work of institutional reflection and addressing the salient ideas and practices that uphold racism in medicine.

Training Medical Students as Peer-Facilitators to Identify Medical Student Mistreatment in the Clerkship Year.

Introduction: Data from the Association of American Medical Colleges' Medical School Graduation Questionnaire show persistent trends of medical student mistreatment nationwide. To reduce the barriers and increase actionable reporting of mistreatment, we integrated peer-facilitated learning environment sessions led by a group of trained third- and fourth-year medical students in all core clinical clerkships. Methods: During the 2018-2019 academic year, third-year medical students were recruited, oriented, and trained to act as facilitators of sessions on mistreatment. The sessions occurred once every clerkship block, using a standardized session introduction and guide. After a 6-month pilot, new medical students were recruited and worked as scribe/facilitator pairs, receiving an additional 1.5-hour training midyear, which was evaluated with a postworkshop survey. Results: Thirty-eight students implemented 43 peer-facilitated sessions and completed deidentified minutes of each session, which were shared with clerkship directors and the Medical Education Office for review. Survey data from midyear facilitator training indicated that facilitators highly agreed peer-led sessions were an important avenue for students to process experiences of mistreatment (3.9 out of 4), understood barriers to reporting (3.8 out of 4) and definitions of mistreatment (3.6 out of 4), and felt confident to facilitate these sessions (3.6 out of 4). Discussion: Peer-facilitated sessions offer a method to learn more about student experiences with mistreatment in real time and create a new avenue for communication between faculty and students. Assembling a stable core team of third- and fourth-year students trained in facilitation skills ensures the sustainability and relevance of the program.

Machine learning and medical education.

Artificial intelligence (AI) driven by machine learning (ML) algorithms is a branch in computer science that is rapidly gaining popularity within the healthcare sector. Recent regulatory approvals of AI-driven companion diagnostics and other products are glimmers of a future in which these tools could play a key role by defining the way medicine will be practiced. Educating the next generation of medical professionals with the right ML techniques will enable them to become part of this emerging data science revolution.