Adapted large language models can outperform medical experts in clinical text summarization.

Analyzing vast textual data and summarizing key information from electronic health records imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown promise in natural language processing (NLP) tasks, their effectiveness on a diverse range of clinical summarization tasks remains unproven. Here we applied adaptation methods to eight LLMs, spanning four distinct clinical summarization tasks: radiology reports, patient questions, progress notes and doctor-patient dialogue. Quantitative assessments with syntactic, semantic and conceptual NLP metrics reveal trade-offs between models and adaptation methods. A clinical reader study with 10 physicians evaluated summary completeness, correctness and conciseness; in most cases, summaries from our best-adapted LLMs were deemed either equivalent (45%) or superior (36%) compared with summaries from medical experts. The ensuing safety analysis highlights challenges faced by both LLMs and medical experts, as we connect errors to potential medical harm and categorize types of fabricated information. Our research provides evidence of LLMs outperforming medical experts in clinical text summarization across multiple tasks. This suggests that integrating LLMs into clinical workflows could alleviate documentation burden, allowing clinicians to focus more on patient care.

Interpreter and limited-English proficiency patient training helps develop medical and physician assistant students' cross-cultural communication skills.

BACKGROUND: The increasing linguistic and cultural diversity in the United States underscores the necessity of enhancing healthcare professionals' cross-cultural communication skills. This study focuses on incorporating interpreter and limited-English proficiency (LEP) patient training into the medical and physician assistant student curriculum. This aims to improve equitable care provision, addressing the vulnerability of LEP patients to healthcare disparities, including errors and reduced access. Though training is recognized as crucial, opportunities in medical curricula remain limited. METHODS: To bridge this gap, a novel initiative was introduced in a medical school, involving second-year students in clinical sessions with actual LEP patients and interpreters. These sessions featured interpreter input, patient interactions, and feedback from interpreters and clinical preceptors. A survey assessed the perspectives of students, preceptors, and interpreters. RESULTS: Outcomes revealed positive reception of interpreter and LEP patient integration. Students gained confidence in working with interpreters and valued interpreter feedback. Preceptors recognized the sessions' value in preparing students for future clinical interactions. CONCLUSIONS: This study underscores the importance of involving experienced interpreters in training students for real-world interactions with LEP patients. Early interpreter training enhances students' communication skills and ability to serve linguistically diverse populations. Further exploration could expand languages and interpretation modes and assess long-term effects on students' clinical performance. By effectively training future healthcare professionals to navigate language barriers and cultural diversity, this research contributes to equitable patient care in diverse communities.

Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts.

Sifting through vast textual data and summarizing key information from electronic health records (EHR) imposes a substantial burden on how clinicians allocate their time. Although large language models (LLMs) have shown immense promise in natural language processing (NLP) tasks, their efficacy on a diverse range of clinical summarization tasks has not yet been rigorously demonstrated. In this work, we apply domain adaptation methods to eight LLMs, spanning six datasets and four distinct clinical summarization tasks: radiology reports, patient questions, progress notes, and doctor-patient dialogue. Our thorough quantitative assessment reveals trade-offs between models and adaptation methods in addition to instances where recent advances in LLMs may not improve results. Further, in a clinical reader study with ten physicians, we show that summaries from our best-adapted LLMs are preferable to human summaries in terms of completeness and correctness. Our ensuing qualitative analysis highlights challenges faced by both LLMs and human experts. Lastly, we correlate traditional quantitative NLP metrics with reader study scores to enhance our understanding of how these metrics align with physician preferences. Our research marks the first evidence of LLMs outperforming human experts in clinical text summarization across multiple tasks. This implies that integrating LLMs into clinical workflows could alleviate documentation burden, empowering clinicians to focus more on personalized patient care and the inherently human aspects of medicine.

Chatbot vs Medical Student Performance on Free-Response Clinical Reasoning Examinations.

This study compares performance on free-response clinical reasoning examinations of first- and second-year medical students vs 2 models of a popular chatbot.

Performance of ChatGPT on free-response, clinical reasoning exams.

Importance: Studies show that ChatGPT, a general purpose large language model chatbot, could pass the multiple-choice US Medical Licensing Exams, but the model's performance on open-ended clinical reasoning is unknown. Objective: To determine if ChatGPT is capable of consistently meeting the passing threshold on free-response, case-based clinical reasoning assessments. Design: Fourteen multi-part cases were selected from clinical reasoning exams administered to pre-clerkship medical students between 2019 and 2022. For each case, the questions were run through ChatGPT twice and responses were recorded. Two clinician educators independently graded each run according to a standardized grading rubric. To further assess the degree of variation in ChatGPT's performance, we repeated the analysis on a single high-complexity case 20 times. Setting: A single US medical school. Participants: ChatGPT. Main Outcomes and Measures: Passing rate of ChatGPT's scored responses and the range in model performance across multiple run throughs of a single case. Results: 12 out of the 28 ChatGPT exam responses achieved a passing score (43%) with a mean score of 69% (95% CI: 65% to 73%) compared to the established passing threshold of 70%. When given the same case 20 separate times, ChatGPT's performance on that case varied with scores ranging from 56% to 81%. Conclusions and Relevance: ChatGPT's ability to achieve a passing performance in nearly half of the cases analyzed demonstrates the need to revise clinical reasoning assessments and incorporate artificial intelligence (AI)-related topics into medical curricula and practice.

Student engagement in the online classroom: comparing preclinical medical student question-asking behaviors in a videoconference versus in-person learning environment.

The COVID-19 pandemic forced medical schools to rapidly transform their curricula using online learning approaches. At our institution, the preclinical Practice of Medicine (POM) course was transitioned to large-group, synchronous, video-conference sessions. The aim of this study is to assess whether there were differences in learner engagement, as evidenced by student question-asking behaviors between in-person and videoconferenced sessions in one preclinical medical student course. In Spring, 2020, large-group didactic sessions in POM were converted to video-conference sessions. During these sessions, student microphones were muted, and video capabilities were turned off. Students submitted typed questions via a Q&A box, which was monitored by a senior student teaching assistant. We compared student question asking behavior in recorded video-conference course sessions from POM in Spring, 2020 to matched, recorded, in-person sessions from the same course in Spring, 2019. We found that, on average, the instructors answered a greater number of student questions and spent a greater percentage of time on Q&A in the online sessions compared with the in-person sessions. We also found that students asked a greater number of higher complexity questions in the online version of the course compared with the in-person course. The video-conference learning environment can promote higher student engagement when compared with the in-person learning environment, as measured by student question-asking behavior. Developing an understanding of the specific elements of the online learning environment that foster student engagement has important implications for instructional design in both the online and in-person setting.

Recognizing the Role of Language in the Hidden Curriculum of Undergraduate Medical Education: Implications for Equity in Medical Training.

Medical education involves a transition from "outsider" to "insider" status, which entails both rigorous formal training and an inculturation of values and norms via a hidden curriculum. Within this transition, the ability to "talk the talk" designates an individual as an insider, and learning to talk this talk is a key component of professional socialization. This Article uses the framework of "patterns of medical language" to explore the role of language in the hidden curriculum of medical education, exploring how students must learn to recognize and participate fluently within patterns of medical language to be acknowledged and evaluated as competent trainees. The authors illustrate this by reframing the Association of American Medical Colleges' Core Entrustable Professional Activities for Entering Residency as a series of overlapping patterns of medical language that students are expected to master before residency. The authors propose that many of these patterns of medical language are learned through trial and error, taught via a hidden curriculum rather than through explicit instruction. Medical students come from increasingly diverse backgrounds and therefore begin medical training further from or closer to insider status. Thus, evaluative practices based on patterns of medical language, which are not explicitly taught, may exacerbate and perpetuate existing inequities in medical education. This Article aims to bring awareness to the importance of medical language within the hidden curriculum of medical education, to the role of medical language as a marker of insider status, and to the centrality of medical language in evaluative practices. The authors conclude by offering possible approaches to ameliorate the inequities that may exist due to current evaluative practices.

Shared Decision-Making During Inpatient Rounds: Opportunities for Improvement in Patient Engagement and Communication.

BACKGROUND: Shared decision-making (SDM) improves patient engagement and may improve outpatient health outcomes. Little is known about inpatient SDM. OBJECTIVE: To assess overall quality, provider behaviors, and contextual predictors of SDM during inpatient rounds on medicine and pediatrics hospitalist services. DESIGN: A 12-week, cross-sectional, single-blinded observational study of team SDM behaviors during rounds, followed by semistructured patient interviews. SETTING: Two large quaternary care academic medical centers. PARTICIPANTS: Thirty-five inpatient teams (18 medicine, 17 pediatrics) and 254 unique patient encounters (117 medicine, 137 pediatrics). INTERVENTION: Observational study. MEASUREMENTS: We used a 9-item Rochester Participatory Decision-Making Scale (RPAD) measured team-level SDM behaviors. Same-day interviews using a modified RPAD assessed patient perceptions of SDM. RESULTS: Characteristics associated with increased SDM in the multivariate analysis included the following: service, patient gender, timing of rounds during patient's hospital stay, and amount of time rounding per patient (P < .05). The most frequently observed behaviors across all services included explaining the clinical issue and matching medical language to the patient's level of understanding. The least frequently observed behaviors included checking understanding of the patient's point of view, examining barriers to follow-through, and asking if the patient has any questions. Patients and guardians had substantially higher ratings for SDM quality compared to peer observers (7.2 vs 4.4 out of 9). CONCLUSIONS: Important opportunities exist to improve inpatient SDM. Team size, number of learners, patient census, and type of decision being made did not affect SDM, suggesting that even large, busy services can perform SDM if properly trained.

The Five-Minute Moment.

In today's hospital and clinic environment, the obstacles to bedside teaching for both faculty and trainees are considerable. As electronic health record systems become increasingly prevalent, trainees are spending more time performing patient care tasks from computer workstations, limiting opportunities to learn at the bedside. Physical examination skills rarely are emphasized, and low confidence levels, especially in junior faculty, pose additional barriers to teaching the bedside examination.