Performance Comparison of ChatGPT-4 and Japanese Medical Residents in the General Medicine In-Training Examination: Comparison Study.

BACKGROUND: The reliability of GPT-4, a state-of-the-art expansive language model specializing in clinical reasoning and medical knowledge, remains largely unverified across non-English languages. OBJECTIVE: This study aims to compare fundamental clinical competencies between Japanese residents and GPT-4 by using the General Medicine In-Training Examination (GM-ITE). METHODS: We used the GPT-4 model provided by OpenAI and the GM-ITE examination questions for the years 2020, 2021, and 2022 to conduct a comparative analysis. This analysis focused on evaluating the performance of individuals who were concluding their second year of residency in comparison to that of GPT-4. Given the current abilities of GPT-4, our study included only single-choice exam questions, excluding those involving audio, video, or image data. The assessment included 4 categories: general theory (professionalism and medical interviewing), symptomatology and clinical reasoning, physical examinations and clinical procedures, and specific diseases. Additionally, we categorized the questions into 7 specialty fields and 3 levels of difficulty, which were determined based on residents' correct response rates. RESULTS: Upon examination of 137 GM-ITE questions in Japanese, GPT-4 scores were significantly higher than the mean scores of residents (residents: 55.8%, GPT-4: 70.1%; P<.001). In terms of specific disciplines, GPT-4 scored 23.5 points higher in the "specific diseases," 30.9 points higher in "obstetrics and gynecology," and 26.1 points higher in "internal medicine." In contrast, GPT-4 scores in "medical interviewing and professionalism," "general practice," and "psychiatry" were lower than those of the residents, although this discrepancy was not statistically significant. Upon analyzing scores based on question difficulty, GPT-4 scores were 17.2 points lower for easy problems (P=.007) but were 25.4 and 24.4 points higher for normal and difficult problems, respectively (P<.001). In year-on-year comparisons, GPT-4 scores were 21.7 and 21.5 points higher in the 2020 (P=.01) and 2022 (P=.003) examinations, respectively, but only 3.5 points higher in the 2021 examinations (no significant difference). CONCLUSIONS: In the Japanese language, GPT-4 also outperformed the average medical residents in the GM-ITE test, originally designed for them. Specifically, GPT-4 demonstrated a tendency to score higher on difficult questions with low resident correct response rates and those demanding a more comprehensive understanding of diseases. However, GPT-4 scored comparatively lower on questions that residents could readily answer, such as those testing attitudes toward patients and professionalism, as well as those necessitating an understanding of context and communication. These findings highlight the strengths and limitations of artificial intelligence applications in medical education and practice.

Performance of GPT-3.5 and GPT-4 on the Japanese Medical Licensing Examination: Comparison Study.

BACKGROUND: The competence of ChatGPT (Chat Generative Pre-Trained Transformer) in non-English languages is not well studied. OBJECTIVE: This study compared the performances of GPT-3.5 (Generative Pre-trained Transformer) and GPT-4 on the Japanese Medical Licensing Examination (JMLE) to evaluate the reliability of these models for clinical reasoning and medical knowledge in non-English languages. METHODS: This study used the default mode of ChatGPT, which is based on GPT-3.5; the GPT-4 model of ChatGPT Plus; and the 117th JMLE in 2023. A total of 254 questions were included in the final analysis, which were categorized into 3 types, namely general, clinical, and clinical sentence questions. RESULTS: The results indicated that GPT-4 outperformed GPT-3.5 in terms of accuracy, particularly for general, clinical, and clinical sentence questions. GPT-4 also performed better on difficult questions and specific disease questions. Furthermore, GPT-4 achieved the passing criteria for the JMLE, indicating its reliability for clinical reasoning and medical knowledge in non-English languages. CONCLUSIONS: GPT-4 could become a valuable tool for medical education and clinical support in non-English-speaking regions, such as Japan.

Right-Sided Urinary Extravasation Caused by a Ureteral Stone and Associated With Peritonitis in an Older Woman.

Right upper quadrant pain can originate from the liver, cholecystic duct, gallbladder, pancreas, or surrounding organs. Peritonitis in the right upper quadrant of the abdomen can be caused by lesions in these organs as well as the adjacent organs, such as the kidney and colon. The kidneys are surrounded by Gerota's fascia and fat; therefore, mild local inflammation may not cause peritonitis. Herein, we report the case of a 72-year-old woman with right-sided abdominal pain who was diagnosed with urinary extravasation due to a ureteral stone. Urinary extravasations can present with peritonitis. For effective diagnosis, prompt physical examination and abdominal ultrasound are essential, with the extent of extravasation being key to effective management. Therefore, general physicians should consider urinary extravasation, which is typically caused by kidney and urinary stones, in patients with right upper quadrant pain.

Secondary Failure of Tocilizumab in Treating Elderly-Onset Rheumatoid Arthritis With Systemic Symptoms Complicated by Diverticulum Perforation.

The treatment of rheumatoid arthritis (RA) has advanced from the use of steroids to disease-modifying anti-rheumatic drugs (DMARDs) and biologics such as tumor necrosis factor (TNF) and interleukin-6 (IL-6) inhibitors. Historically, steroids have been the mainstream in the clinical treatment of RA; however, the development of DMARDs has changed the RA treatment structure. In addition, biologics can alleviate RA symptoms. This case report describes the secondary failure of tocilizumab in treating RA with fatigue symptoms. Treatment with tocilizumab decreases C-reactive protein (CRP) levels, which may make detecting RA exacerbation difficult; therefore, obtaining the patient's precise history and thorough physical examinations are necessary. This case demonstrates the complexity of treating elderly-onset RA and reports practical methods for effective treatment.

A review of food poisoning caused by local food in Japan.

Increasingly popular worldwide, Japanese cuisine includes several raw preparations such as sashimi and sushi; however, limited information on food poisoning from Japanese local food is available in English literature. Without appropriate knowledge, physicians may underdiagnose traveler's diarrhea among people returning from Japan. To provide accurate information to primary care physicians worldwide, we conducted a narrative review on food poisoning research published in Japanese and English over the past four years, considering the frequency and clinical importance of various presentations.