A Generative Pretrained Transformer (GPT)-Powered Chatbot as a Simulated Patient to Practice History Taking: Prospective, Mixed Methods Study.

BACKGROUND: Communication is a core competency of medical professionals and of utmost importance for patient safety. Although medical curricula emphasize communication training, traditional formats, such as real or simulated patient interactions, can present psychological stress and are limited in repetition. The recent emergence of large language models (LLMs), such as generative pretrained transformer (GPT), offers an opportunity to overcome these restrictions. OBJECTIVE: The aim of this study was to explore the feasibility of a GPT-driven chatbot to practice history taking, one of the core competencies of communication. METHODS: We developed an interactive chatbot interface using GPT-3.5 and a specific prompt including a chatbot-optimized illness script and a behavioral component. Following a mixed methods approach, we invited medical students to voluntarily practice history taking. To determine whether GPT provides suitable answers as a simulated patient, the conversations were recorded and analyzed using quantitative and qualitative approaches. We analyzed the extent to which the questions and answers aligned with the provided script, as well as the medical plausibility of the answers. Finally, the students filled out the Chatbot Usability Questionnaire (CUQ). RESULTS: A total of 28 students practiced with our chatbot (mean age 23.4, SD 2.9 years). We recorded a total of 826 question-answer pairs (QAPs), with a median of 27.5 QAPs per conversation and 94.7% (n=782) pertaining to history taking. When questions were explicitly covered by the script (n=502, 60.3%), the GPT-provided answers were mostly based on explicit script information (n=471, 94.4%). For questions not covered by the script (n=195, 23.4%), the GPT answers used 56.4% (n=110) fictitious information. Regarding plausibility, 842 (97.9%) of 860 QAPs were rated as plausible. Of the 14 (2.1%) implausible answers, GPT provided answers rated as socially desirable, leaving role identity, ignoring script information, illogical reasoning, and calculation error. Despite these results, the CUQ revealed an overall positive user experience (77/100 points). CONCLUSIONS: Our data showed that LLMs, such as GPT, can provide a simulated patient experience and yield a good user experience and a majority of plausible answers. Our analysis revealed that GPT-provided answers use either explicit script information or are based on available information, which can be understood as abductive reasoning. Although rare, the GPT-based chatbot provides implausible information in some instances, with the major tendency being socially desirable instead of medically plausible information.

Factors associated with HIV Testing within the National Health Interview Survey (2006-2018).

The Centers for Disease Control and Prevention recommends that individuals aged 13-64 test for HIV at least once during their lifetime. However, screening has been disproportionate among racial/ethnic populations. Using the National Health Interview Survey data (2006-2018), we examined HIV screening prevalence within racial/ethnic groups in the United States (US), and factors associated with testing among 301,191 individuals. This consisted of 195,696 White, 42,409 Black, 47,705 Hispanic and 15,381 Asian individuals. Logistic regression was used to calculate odds ratios (OR) and 95% confidence intervals (CI) to estimate the association between ever testing for HIV and demographic, socioeconomic and health-related factors. Approximately 36% of White, 61% of Black, 47% of Hispanic and 36% of Asian individuals reported ever testing for HIV. Hispanic (OR = 1.28, 95% CI [1.25-1.32]) and Black individuals (OR = 2.44, 95% CI [2.38-2.50]) had higher odds of HIV testing, whereas Asian individuals (OR = 0.74, 95% CI [0.71-0.77]) had lower odds of HIV testing compared to White individuals. Individuals who identified as males, married, between the ages of 18-26 years or greater than or equal to 50 years were less likely to ever test for HIV compared to their counterparts. Similarly, those with lower education, lower income, better self-reported health, no health professional visits or living in the midwestern US were less likely to ever test for HIV compared to their counterparts (OR range: 0.14-0.92). Understanding the factors associated with HIV testing opens opportunities to increase testing rates for all and reduce health disparities in HIV detection.

Performance of ChatGPT on free-response, clinical reasoning exams.

Importance: Studies show that ChatGPT, a general purpose large language model chatbot, could pass the multiple-choice US Medical Licensing Exams, but the model's performance on open-ended clinical reasoning is unknown. Objective: To determine if ChatGPT is capable of consistently meeting the passing threshold on free-response, case-based clinical reasoning assessments. Design: Fourteen multi-part cases were selected from clinical reasoning exams administered to pre-clerkship medical students between 2019 and 2022. For each case, the questions were run through ChatGPT twice and responses were recorded. Two clinician educators independently graded each run according to a standardized grading rubric. To further assess the degree of variation in ChatGPT's performance, we repeated the analysis on a single high-complexity case 20 times. Setting: A single US medical school. Participants: ChatGPT. Main Outcomes and Measures: Passing rate of ChatGPT's scored responses and the range in model performance across multiple run throughs of a single case. Results: 12 out of the 28 ChatGPT exam responses achieved a passing score (43%) with a mean score of 69% (95% CI: 65% to 73%) compared to the established passing threshold of 70%. When given the same case 20 separate times, ChatGPT's performance on that case varied with scores ranging from 56% to 81%. Conclusions and Relevance: ChatGPT's ability to achieve a passing performance in nearly half of the cases analyzed demonstrates the need to revise clinical reasoning assessments and incorporate artificial intelligence (AI)-related topics into medical curricula and practice.

Reactivated disseminated tuberculosis in pregnancy: Case report and review of the literature.

This case report describes the evaluation and management of a 32-year-old woman who presented shortly after a fetal demise at 23 weeks of gestation with multiple symptoms, including bloody vaginal discharge. Although the initial diagnostic concern was for metastatic malignancy, the patient was ultimately determined to have disseminated tuberculosis. Genital tuberculosis is common worldwide, yet guidelines for evaluation are limited. This report highlights the relationship between pregnancy-reactivated tuberculosis, and guides clinicians on diagnostic and management considerations in the peripartum period.

Intoxication and pitch control in tonal and non-tonal language speakers.

Alcohol intoxication is known to affect pitch variability in non-tonal languages. In this study, intoxication's effects on pitch were examined in tonal and non-tonal language speakers, in both their native language (L1; German, Korean, Mandarin) and nonnative language (L2; English). Intoxication significantly increased pitch variability in the German group (in L1 and L2), but not in the Korean or Mandarin groups (in L1 or L2), although there were individual differences. These results support the view that pitch control is related to the functional load of pitch and is an aspect of speech production that can be advantageously transferred across languages, overriding the expected effects of alcohol.

A Call to Action From the California Consortium for the Assessment of Clinical Competence: Making the Case for Regional Collaboration.

The discontinuation of the United States Medical Licensing Examination Step 2 Clinical Skills Examination emphasizes the need for other reliable standardized assessments of medical student clinical skills. For 30 years, the California Consortium for the Assessment of Clinical Competence (CCACC) has collaborated in the development of clinical skills assessments and has become a valuable resource for clinicians, standardized patient educators, psychometricians, and medical educators. There are many merits to strong multi-institutional partnerships, including the integration of data across multiple schools to provide feedback to both students and curricula, pooled test performance statistics for analysis and quality assurance, shared best practices and resources, individual professional development, and opportunities for research and scholarship. The infrastructure of the CCACC allows member schools to adapt to a changing medical landscape, from emerging trends in clinical medicine to the limitations imposed by a global pandemic. In the absence of a national examination, there is now a greater need for all medical schools to develop a comprehensive, dynamic, and psychometrically sound assessment that accurately evaluates clinical skills. Medical schools working together in regional consortia have the opportunity to create and implement innovative and robust assessments that evaluate a wide range of clinical skills, ensure that medical students have met an expected level of clinical competency before graduation, and provide a framework that contributes to ongoing efforts for the development of new national clinical skills standards.

Perspectives, benefits and challenges of a live OSCE during the COVID-19 pandemic in a cross-sectional study.

OBJECTIVES: Restrictions due to the COVID-19 pandemic mandated fundamental changes to student evaluations, including the administration of the observed structured clinical examination (OSCE). This study aims to conduct an in-person OSCE to verify students' practical skills under necessary infection control practices and the impact of face masks on student-patient interactions. DESIGN: Cross-sectional design. SETTING: The OSCE at Medical School of Tuebingen takes place in October 2020. PARTICIPANTS: A total of 149 students (third year of study) completed the survey (RR=80.1%). It was their first OSCE. PRIMARY AND SECONDARY OUTCOME MEASURES: Primary outcome measure was how this type of OSCE was evaluated by participating students in regard to preparation, content and difficulty as well as in real life. Secondary outcome measures were how the implemented hygiene actions influenced the OSCE, including the interaction and communication between students and standardised patients (SPs). Items were rated on a 6-point Likert scale (1=completely to 6=not at all). Means, SDs, frequencies and percentages were calculated. RESULTS: 149 students, 32 SPs and 59 examiners participated. The students rated the OSCE with 2.37 (±0.52) for preparation and 2.07 (±0.32) for content. They perceived the interaction to be significantly disrupted by the use of face masks (3.03±1.54) (p<0.001) compared with the SPs (3.84±1.44) and the examiners (4.14±1.55). In general, the three groups considered the use of face masking the OSCE to be helpful (1.60±1.15). CONCLUSIONS: An in-person OSCE, even in the midst of a global pandemic, is feasible and acceptable to both students and faculty. When compared the students' results to previous students' results who completed the OSCE before the pandemic, the results indicated that students felt less prepared than under non-pandemic circumstances; however, their performances on this OSCE were not lower.

A Multi-Institution Collaboration to Define Core Content and Design Flexible Curricular Components for a Foundational Medical School Course: Implications for National Curriculum Reform.

Medical educators have not reached widespread agreement on core content for a U.S. medical school curriculum. As a first step toward addressing this, five U.S. medical schools formed the Robert Wood Johnson Foundation Reimagining Medical Education collaborative to define, create, implement, and freely share core content for a foundational medical school course on microbiology and immunology. This proof-of-concept project involved delivery of core content to preclinical medical students through online videos and class-time interactions between students and facilitators. A flexible, modular design allowed four of the medical schools to successfully implement the content modules in diverse curricular settings. Compared with the prior year, student satisfaction ratings after implementation were comparable or showed a statistically significant improvement. Students who took this course at a time point in their training similar to when the USMLE Step 1 reference group took Step 1 earned equivalent scores on National Board of Medical Examiners-Customized Assessment Services microbiology exam items. Exam scores for three schools ranged from 0.82 to 0.84, compared with 0.81 for the national reference group; exam scores were 0.70 at the fourth school, where students took the exam in their first quarter, two years earlier than the reference group. This project demonstrates that core content for a foundational medical school course can be defined, created, and used by multiple medical schools without compromising student satisfaction or knowledge. This project offers one approach to collaboratively defining core content and designing curricular resources for preclinical medical school education that can be shared.

Promoting student case creation to enhance instruction of clinical reasoning skills: a pilot feasibility study.

BACKGROUND: It is a common educational practice for medical students to engage in case-based learning (CBL) exercises by working through clinical cases that have been developed by faculty. While such faculty-developed exercises have educational strengths, there are at least two major drawbacks to learning by this method: the number and diversity of cases is often limited; and students decrease their engagement with CBL cases as they grow accustomed to the teaching method. We sought to explore whether student case creation can address both of these limitations. We also compared student case creation to traditional clinical reasoning sessions in regard to tutorial group effectiveness, perceived gains in clinical reasoning, and quality of student-faculty interaction. METHODS: Ten first-year medical students participated in a feasibility study wherein they worked in small groups to develop their own patient case around a preassigned diagnosis. Faculty provided feedback on case quality afterwards. Students completed pre- and post-self-assessment surveys. Students and faculty also participated in separate focus groups to compare their case creation experience to traditional CBL sessions. RESULTS: Students reported high levels of team engagement and peer learning, as well as increased ownership over case content and understanding of clinical reasoning nuances. However, students also reported decreases in student-faculty interaction and the use of visual aids (P < 0.05). CONCLUSION: The results of our feasibility study suggest that student-generated cases can be a valuable adjunct to traditional clinical reasoning instruction by increasing content ownership, encouraging student-directed learning, and providing opportunities to explore clinical nuances. However, these gains may reduce student-faculty interaction. Future studies may be able to identify an improved model of faculty participation, the ideal timing for incorporation of this method in a medical curriculum, and a more rigorous assessment of the impact of student case creation on the development of clinical reasoning skills.

Effects of subject-case marking on agreement processing: ERP evidence from Basque.

Previous cross-linguistic research has found that comprehenders are immediately sensitive to various kinds of agreement violations across languages. We focused on Basque, a verb-final ergative language with both subject-verb (SV) and object-verb (OV) agreement. We compared the effects of SV agreement violations on comprehenders' event-related brain potentials (ERPs) in transitive sentences (where OV agreement is present, and the subject is ergative) and intransitive sentences (where OV agreement is absent, and the subject is absolutive). We observed a P600 effect in both cases, but only violations with intransitive subjects elicited an early posterior negativity. Such a qualitative difference suggests that distinct neurocognitive mechanisms are involved in processing agreement with transitive subjects (which are marked with ergative case) versus intransitive subjects (which bear absolutive case). Building on theoretical proposals that in languages such as Basque, true agreement occurs with absolutive subjects but not with ergative subjects, we submit that the early posterior negativity may be an electrophysiological signature for true agreement.

When linearity prevails over hierarchy in syntax.

Hierarchical structure has been cherished as a grammatical universal. We use experimental methods to show where linear order is also a relevant syntactic relation. An identical methodology and design were used across six research sites on South Slavic languages. Experimental results show that in certain configurations, grammatical production can in fact favor linear order over hierarchical structure. However, these findings are limited to coordinate structures and distinct from the kind of production errors found with comparable configurations such as "attraction" errors. The results demonstrate that agreement morphology may be computed in a series of steps, one of which is partly independent from syntactic hierarchy.

Validity Evidence and Scoring Guidelines for Standardized Patient Encounters and Patient Notes From a Multisite Study of Clinical Performance Examinations in Seven Medical Schools.

PURPOSE: To examine validity evidence of local graduation competency examination scores from seven medical schools using shared cases and to provide rater training protocols and guidelines for scoring patient notes (PNs). METHOD: Between May and August 2016, clinical cases were developed, shared, and administered across seven medical schools (990 students participated). Raters were calibrated using training protocols, and guidelines were developed collaboratively across sites to standardize scoring. Data included scores from standardized patient encounters for history taking, physical examination, and PNs. Descriptive statistics were used to examine scores from the different assessment components. Generalizability studies (G-studies) using variance components were conducted to estimate reliability for composite scores. RESULTS: Validity evidence was collected for response process (rater perception), internal structure (variance components, reliability), relations to other variables (interassessment correlations), and consequences (composite score). Student performance varied by case and task. In the PNs, justification of differential diagnosis was the most discriminating task. G-studies showed that schools accounted for less than 1% of total variance; however, for the PNs, there were differences in scores for varying cases and tasks across schools, indicating a school effect. Composite score reliability was maximized when the PN was weighted between 30% and 40%. Raters preferred using case-specific scoring guidelines with clear point-scoring systems. CONCLUSIONS: This multisite study presents validity evidence for PN scores based on scoring rubric and case-specific scoring guidelines that offer rigor and feedback for learners. Variability in PN scores across participating sites may signal different approaches to teaching clinical reasoning among medical schools.

Assessing 3rd year medical students' interprofessional collaborative practice behaviors during a standardized patient encounter: A multi-institutional, cross-sectional study.

BACKGROUND: To understand how third-year medical student interprofessional collaborative practice (IPCP) is affected by self-efficacy and interprofessional experiences (extracurricular experiences and formal curricula). METHODS: The authors measured learner IPCP using an objective structured clinical examination (OSCE) with a standardized nurse (SN) and standardized patient (SP) during a statewide clinical performance examination. At four California medical schools from April to August 2012, SPs and SNs rated learner IPCP (10 items, range 0-100) and patient-centered communication (10 items, range 0-100). Post-OSCE, students reported their interprofessional self-efficacy (16 items, 2 factors, range 1-10) and prior extracurricular interprofessional experiences (3 items). School representatives shared their interprofessional curricula during guided interviews. RESULTS: Four hundred sixty-four of 530 eligible medical students (88%) participated. Mean IPCP performance was 79.6 ± 14.1 and mean self-efficacy scores were 7.9 (interprofessional teamwork) and 7.1 (interprofessional feedback and evaluation). Seventy percent of students reported prior extracurricular interprofessional experiences; all schools offered formal interprofessional curricula. IPCP was associated with self-efficacy for interprofessional teamwork (ß = 1.6, 95% CI [0.1, 3.1], p = 0.04) and patient-centered communication (ß = 12.5, 95% CI [2.7, 22.3], p = 0.01). CONCLUSIONS: Medical student IPCP performance was associated with self-efficacy for interprofessional teamwork and patient-centered communication. Increasing interprofessional opportunities that influence medical students' self-efficacy may increase engagement in IPCP.

Impact of student ethnicity and patient-centredness on communication skills performance.

OBJECTIVES: The development of patient-centred attitudes by health care providers is critical to improving health care quality. A prior study showed that medical students with more patient-centred attitudes scored higher in communication skills as judged by standardised patients (SPs) than students with less patient-centred attitudes. We designed this multicentre study to examine the relationships among students' demographic characteristics, patient-centredness and communication scores on an SP examination. METHODS: Early Year 4 medical students at three US schools completed a 12-item survey during an SP examination. Survey items addressed demographics (gender, ethnicity, primary childhood language) and patient-centredness. Factor analysis on the patient-centredness items defined specific patient-centred attitudes. We used multiple regression analysis incorporating demographic characteristics, school and patient-centredness items and examined the effect of these variables on the outcome variable of communication score. RESULTS: A total of 351 students took the SP examination and 329 (94%) completed the patient-centredness questionnaire. Responses indicated generally high patient-centredness. Student ethnicity and medical school were significantly associated with communication scores; gender and primary childhood language were not. Two attitudinal factors were identified: patient perspective and impersonal attitude. Multiple regression analysis revealed that school and scores on the impersonal factor were associated with communication scores. The effect size was modest. CONCLUSIONS: In a medical student SP examination, modest differences in communication scores based on ethnicity were observed and can be partially explained by student attitudes regarding patient-centredness. Curricular interventions to enhance clinical experiences, teaching and feedback are needed to address key elements of a patient-centred approach to care.

Beyond knowledge, toward linguistic competency: an experiential curriculum.

INTRODUCTION/AIMS: Training is essential for future health care providers to effectively communicate with limited English proficient (LEP) patients during interpreted encounters. Our aim is to describe an innovative skill-based medical school linguistic competency curriculum and its impact on knowledge and skills. SETTING: At Stanford University School of Medicine, we incorporated a linguistic competency curriculum into a 2-year Practice of Medicine preclinical doctoring course and pediatrics clerkship over three cohorts. PROGRAM DESCRIPTION: First year students participated in extensive interpreter-related training including: a knowledge-based online module, interactive role-play exercises, and didactic skill-building sessions. Students in the pediatrics clerkship participated in interpreted training exercises with facilitated feedback. PROGRAM EVALUATION: Knowledge and skills were evaluated in the first and fourth years. First year students' knowledge scores increased (pre-test = 0.62, post-test = 0.89, P < 0.001), and they demonstrated good skill attainment during an end-year performance assessment. One cohort of students participated in the entire curriculum and maintained performance into the fourth year. DISCUSSION: Our curriculum increased knowledge and led to skill attainment, each of which showed good durability for a cohort of students evaluated 3 years later. With a growing LEP population, these skills are essential to foster in future health care providers to effectively communicate with LEP patients and reduce health disparities.

International cohort analysis of the antiviral activities of zidovudine and tenofovir in the presence of the K65R mutation in reverse transcriptase.

A K65R mutation in HIV-1 reverse transcriptase can occur with the failure of tenofovir-, didanosine-, abacavir-, and, in some cases, stavudine-containing regimens and leads to reduced phenotypic susceptibility to these drugs and hypersusceptibility to zidovudine, but its clinical impact is poorly described. We identified isolates with the K65R mutation within the Stanford Resistance Database and a French cohort for which subsequent treatment and virological response data were available. The partial genotypic susceptibility score (pGSS) was defined as the genotypic susceptibility score (GSS) excluding the salvage regimen's nucleoside reverse transcriptase inhibitor (NRTI) component. A three-part virologic response variable was defined (e.g., complete virologic response, partial virologic response, and no virologic response). Univariate, multivariate, and bootstrap analyses evaluated factors associated with the virologic response, focusing on the contributions of zidovudine and tenofovir. Seventy-one of 130 patients (55%) achieved a complete virologic response (defined as an HIV RNA level of <200 copies/ml). In univariate analyses, pGSS and zidovudine use in the salvage regimen were predictors of the virologic response. In a multivariate analysis, pGSS and zidovudine and tenofovir use were associated with the virologic response. Bootstrap analyses showed similar reductions in HIV RNA levels with zidovudine or tenofovir use (0.5 to 0.9 log(10)). In the presence of K65R, zidovudine and tenofovir are associated with similar reductions in HIV RNA levels. Given its tolerability, tenofovir may be the preferred agent over zidovudine even in the presence of the K65R mutation.

The role of feature-number and feature-type in processing Hindi verb agreement violations.

This article presents studies of Hindi that investigate whether responses to syntactic agreement violations vary as a function of the type and number of incorrect agreement features, using both electrophysiological (ERP) and behavioral measures. Hindi is well suited to investigation of this issue, since verbs in Hindi mark agreement with the person, number, and gender features of the nominative subject noun phrase. In an ERP study evoked responses were recorded for visually presented verbs appearing at the end of a sentence-initial adverbial clause, comparing responses in a grammatically correct condition with four grammatically incorrect conditions that mismatched the correct agreement on different dimensions (Gender, Number, Gender/Number, Person/Gender). A P600 response was elicited in all grammatically incorrect conditions. No amplitude differences were found among the Gender, Number, and combined Gender/Number violations. This suggests that the feature distance between observed and expected word forms at the morphosyntactic level does not impact ERP responses, contrasting with findings on semantic and auditory processing, and suggests that the P600 response to agreement violations is not additive based on the number of mismatching features and does not reflect top-down, predictive mechanisms. A significantly larger P600 response was elicited by the combined Person/Gender violation, and two different violations involving the Person feature were judged as more severe and recognized more quickly in the behavioral studies. This effect is attributed to the greater salience of the Person feature at multiple levels of representation.

Acute pancreatitis after gastrointestinal endoscopy.

Acute pancreatitis is a well-recognized complication of endoscopic retrograde cholangiopancreatography but is not considered to be a complication associated with other endoscopic procedures. We present a case of acute pancreatitis that occurred after uneventful upper and lower gastrointestinal endoscopy. The temporal relationship of the endoscopic procedures and development of acute pancreatitis suggests a causal relation. Furthermore, the patient had none of the usual etiologic factors associated with pancreatitis, i.e., alcoholism, cholelithiasis, hypertriglyceridemia, hypercalcemia, or use of a drug associated with pancreatitis. The causal mechanism of acute pancreatitis is uncertain but might potentially involve local trauma to the pancreas during a procedure or release of as yet undefined inflammatory mediators. In summary, three previous reports of clinical pancreatitis associated with endoscopy, in addition to the current case, suggests that acute pancreatitis should be considered as a rare complication of routine upper endoscopy or colonoscopy.