Evaluation of ChatGPT-Generated Differential Diagnosis for Common Diseases With Atypical Presentation: Descriptive Research.

Background: The persistence of diagnostic errors, despite advances in medical knowledge and diagnostics, highlights the importance of understanding atypical disease presentations and their contribution to mortality and morbidity. Artificial intelligence (AI), particularly generative pre-trained transformers like GPT-4, holds promise for improving diagnostic accuracy, but requires further exploration in handling atypical presentations. Objective: This study aimed to assess the diagnostic accuracy of ChatGPT in generating differential diagnoses for atypical presentations of common diseases, with a focus on the model's reliance on patient history during the diagnostic process. Methods: We used 25 clinical vignettes from the Journal of Generalist Medicine characterizing atypical manifestations of common diseases. Two general medicine physicians categorized the cases based on atypicality. ChatGPT was then used to generate differential diagnoses based on the clinical information provided. The concordance between AI-generated and final diagnoses was measured, with a focus on the top-ranked disease (top 1) and the top 5 differential diagnoses (top 5). Results: ChatGPT's diagnostic accuracy decreased with an increase in atypical presentation. For category 1 (C1) cases, the concordance rates were 17% (n=1) for the top 1 and 67% (n=4) for the top 5. Categories 3 (C3) and 4 (C4) showed a 0% concordance for top 1 and markedly lower rates for the top 5, indicating difficulties in handling highly atypical cases. The χ2 test revealed no significant difference in the top 1 differential diagnosis accuracy between less atypical (C1+C2) and more atypical (C3+C4) groups (χ²1=2.07; n=25; P=.13). However, a significant difference was found in the top 5 analyses, with less atypical cases showing higher accuracy (χ²1=4.01; n=25; P=.048). Conclusions: ChatGPT-4 demonstrates potential as an auxiliary tool for diagnosing typical and mildly atypical presentations of common diseases. However, its performance declines with greater atypicality. The study findings underscore the need for AI systems to encompass a broader range of linguistic capabilities, cultural understanding, and diverse clinical scenarios to improve diagnostic utility in real-world settings.

Performance evaluation of ChatGPT in detecting diagnostic errors and their contributing factors: an analysis of 545 case reports of diagnostic errors.

BACKGROUND: Manual chart review using validated assessment tools is a standardised methodology for detecting diagnostic errors. However, this requires considerable human resources and time. ChatGPT, a recently developed artificial intelligence chatbot based on a large language model, can effectively classify text based on suitable prompts. Therefore, ChatGPT can assist manual chart reviews in detecting diagnostic errors. OBJECTIVE: This study aimed to clarify whether ChatGPT could correctly detect diagnostic errors and possible factors contributing to them based on case presentations. METHODS: We analysed 545 published case reports that included diagnostic errors. We imputed the texts of case presentations and the final diagnoses with some original prompts into ChatGPT (GPT-4) to generate responses, including the judgement of diagnostic errors and contributing factors of diagnostic errors. Factors contributing to diagnostic errors were coded according to the following three taxonomies: Diagnosis Error Evaluation and Research (DEER), Reliable Diagnosis Challenges (RDC) and Generic Diagnostic Pitfalls (GDP). The responses on the contributing factors from ChatGPT were compared with those from physicians. RESULTS: ChatGPT correctly detected diagnostic errors in 519/545 cases (95%) and coded statistically larger numbers of factors contributing to diagnostic errors per case than physicians: DEER (median 5 vs 1, p<0.001), RDC (median 4 vs 2, p<0.001) and GDP (median 4 vs 1, p<0.001). The most important contributing factors of diagnostic errors coded by ChatGPT were 'failure/delay in considering the diagnosis' (315, 57.8%) in DEER, 'atypical presentation' (365, 67.0%) in RDC, and 'atypical presentation' (264, 48.4%) in GDP. CONCLUSION: ChatGPT accurately detects diagnostic errors from case presentations. ChatGPT may be more sensitive than manual reviewing in detecting factors contributing to diagnostic errors, especially for 'atypical presentation'.

The Big Three diagnostic errors through reflections of Japanese internists.

OBJECTIVES: To analyze the Big Three diagnostic errors (malignant neoplasms, cardiovascular diseases, and infectious diseases) through internists' self-reflection on their most memorable diagnostic errors. METHODS: This secondary analysis study, based on a web-based cross-sectional survey, recruited participants from January 21 to 31, 2019. The participants were asked to recall the most memorable diagnostic error cases in which they were primarily involved. We gathered data on internists' demographics, time to error recognition, and error location. Factors causing diagnostic errors included environmental conditions, information processing, and cognitive bias. Participants scored the significance of each contributing factor on a Likert scale (0, unimportant; 10, extremely important). RESULTS: The Big Three comprised 54.1 % (n=372) of the 687 cases reviewed. The median physician age was 51.5 years (interquartile range, 42-58 years); 65.6 % of physicians worked in hospital settings. Delayed diagnoses were the most common among malignancies (n=64, 46 %). Diagnostic errors related to malignancy were frequent in general outpatient settings on weekdays and in the mornings and were not identified for several months following the event. Environmental factors often contributed to cardiovascular disease-related errors, which were typically identified within days in emergency departments, during night shifts, and on holidays. Information gathering and interpretation significantly impacted infectious disease diagnoses. CONCLUSIONS: The Big Three accounted for the majority of cases recalled by Japanese internists. The most relevant contributing factors were different for each of the three categories. Addressing these errors may require a unique approach based on the disease associations.

Diagnostic errors in uncommon conditions: a systematic review of case reports of diagnostic errors.

OBJECTIVES: To assess the usefulness of case reports as sources for research on diagnostic errors in uncommon diseases and atypical presentations. CONTENT: We reviewed 563 case reports of diagnostic error. The commonality of the final diagnoses was classified based on the description in the articles, Orphanet, or epidemiological data on available references; the typicality of presentation was classified based on the description in the articles and the judgment of the physician researchers. Diagnosis Error Evaluation and Research (DEER), Reliable Diagnosis Challenges (RDC), and Generic Diagnostic Pitfalls (GDP) taxonomies were used to assess the factors contributing to diagnostic errors. SUMMARY AND OUTLOOK: Excluding three cases in that commonality could not be classified, 560 cases were classified into four categories: typical presentations of common diseases (60, 10.7 %), atypical presentations of common diseases (35, 6.2 %), typical presentations of uncommon diseases (276, 49.3 %), and atypical presentations of uncommon diseases (189, 33.8 %). The most important DEER taxonomy was "Failure/delay in considering the diagnosis" among the four categories, whereas the most important RDC and GDP taxonomies varied with the categories. Case reports can be a useful data source for research on the diagnostic errors of uncommon diseases with or without atypical presentations.

Sternoclavicular Septic Arthritis Due to Invasive Pneumococcal Infection After Type A Influenza Virus Infection.

A 24-year-old female patient who had a type A influenza virus infection prior to admission visited our hospital complaining of a fever and right sternoclavicular pain. Blood culture was positive for penicillin-sensitive Streptococcus pneumoniae (pneumococcus). Magnetic resonance imaging of the right sternoclavicular joint (SCJ) showed a high signal intensity area on the diffusion-weighted images. Consequently, the patient was diagnosed with septic arthritis due to invasive pneumococcus. When a patient complains of gradually increasing chest pain after an influenza virus infection, SCJ septic arthritis should be considered in the differential diagnosis.

A Common Diagnosis Delayed by Three "Wrong Footing" Anchoring Errors - it is Difficult to Remember What You Have Forgotten.

Very often in clinical practice, an inflamed pelvic appendix shows left lower quadrant abdominal pain as the primary painful area. The clinicians are anchored to the most prominent symptom, thereby taking an unnecessary detour in reaching an accurate diagnosis. A 40-year-old man presented to our emergency department with persistent lower left abdominal pain with a fever of 38 oC from a day earlier. He had a good appetite and repeatedly complained of severe constipation at the time of his visit. Physical examination revealed tenderness in the lower left abdomen without a peritoneal sign. Abdominal ultrasound and non-contrast-enhanced computed tomography revealed a left hydroureter. The next day, a radiologist pointed out the possibility of appendicitis. An urgent laparoscopic appendectomy was performed. The intriguing point of this case is the diagnostic delay because of three anchoring biases. First, the typical right lower abdominal pain of appendicitis was shielded by the intense left lower abdominal pain. Moreover, the presence of a left hydroureter distracted the physicians from the actual location of the pain. Furthermore, the presence of constipation anchored the physicians to constipation as the cause of abdominal pain. In overcoming these biases, specific diagnostic strategies to avoid biases should be implemented. LEARNING POINTS: If a patient has unexplained lower left abdominal pain, it is advisable to deploy a "searchlight" strategy.When a hydroureter was found to have no apparent source obstruction, a vertical tracing strategy should have been undertaken to detect its root cause.To avoid the wrong diagnosis through anchoring bias, pivot and cluster strategy - deploying differential diagnosis specific to the initial diagnosis (constipation in this case) - should be adopted at the start, considering the important differential diagnosis and thus preventing a missed diagnosis.

Retained rice cake in the stomach leading to potential intestinal obstruction.

A 66-year-old Japanese woman presented with upper abdominal pain and nausea after having eaten several rice cakes. An abdominal CT scan showed retained rice cakes in the stomach, which led to intestinal obstruction three days later.

Cognitive biases encountered by physicians in the emergency room.

BACKGROUND: Diagnostic errors constitute an important medical safety problem that needs improvement, and their frequency and severity are high in emergency room settings. Previous studies have suggested that diagnostic errors occur in 0.6-12% of first-time patients in the emergency room and that one or more cognitive factors are involved in 96% of these cases. This study aimed to identify the types of cognitive biases experienced by physicians in emergency rooms in Japan. METHODS: We conducted a questionnaire survey using Nikkei Medical Online (Internet) from January 21 to January 31, 2019. Of the 159,519 physicians registered with Nikkei Medical Online when the survey was administered, those who volunteered their most memorable diagnostic error cases in the emergency room participated in the study. EZR was used for the statistical analyses. RESULTS: A total of 387 physicians were included. The most common cognitive biases were overconfidence (22.5%), confirmation (21.2%), availability (12.4%), and anchoring (11.4%). Of the error cases, the top five most common initial diagnoses were upper gastrointestinal disease (22.7%), trauma (14.7%), cardiovascular disease (10.9%), respiratory disease (7.5%), and primary headache (6.5%). The corresponding final diagnoses for these errors were intestinal obstruction or peritonitis (27.3%), overlooked traumas (47.4%), other cardiovascular diseases (66.7%), cardiovascular disease (41.4%), and stroke (80%), respectively. CONCLUSIONS: A comparison of the initial and final diagnoses of cases with diagnostic errors shows that there were more cases with diagnostic errors caused by overlooking another disease in the same organ or a disease in a closely related organ.

Skin and soft tissue infections and bacteremia caused by Vibrio cincinnatiensis.

Vibrio cincinnatiensis is a halophilic species found in marine and estuarine environments worldwide. It is a rare pathogen whose impact on humans has remained unclear; only two cases of V. cincinnatiensis infection have been reported in humans, so far. A 63-year-old man with a history of myocardial infarction and type 2 diabetes mellitus presented to the emergency department with fever and dyspnea. Physical examination demonstrated notable abdominal distension and bilateral lower leg edema. marked abdominal distension and bilateral lower leg edema. The patient was diagnosed with bacteremia and exacerbated heart failure. Blood and skin cultures revealed the presence of the gram-negative pathogen V. cincinnatiensis. Combined antibiotic therapy using intravenous tazobactam /piperacillin resulted in a gradual recovery with no recurrence observed at the 9-month follow-up. To the best of our knowledge, this is the third case of V. cincinnatiensis infection reported in humans and the first one to be associated with skin and soft tissue infection. We suggest that although V. cincinnatiensis is a rare pathogen, it should be considered as a potential infective agent in the differential diagnosis of immunocompromised patients, regardless of any recent exposure to seawater or marine products.

Carbon Monoxide Poisoning in a Psychiatric Patient Mistaken for Indefinite Complaints.

Introduction: Carbon monoxide (CO) binds to haemoglobin with a much higher affinity than oxygen, forming carboxyhaemoglobin (COHb), which impairs oxygen transport and utilization. As CO concentrations can easily peak in closed environments, non-fire-related CO poisoning can also occur. However, because CO poisoning is often a nonspecific clinical finding, it can result in a diagnostic error. This report details the misdiagnosis of a 42-year-old male patient with psychiatric disorders. Case description: The patient presented to the hospital with dizziness, abdominal pain and nausea on multiple occasions. His symptoms were ascribed to his psychiatric conditions. On his fifth visit, we diagnosed the patient with CO poisoning. Discussion: It is apparent that this patient was misdiagnosed because of his medical history, and standard analysis was overlooked. When patients with psychiatric disorders have nonspecific symptoms, it is important to check for urgent underlying conditions during diagnosis. LEARNING POINTS: Patients with psychiatric disorders who present with nonspecific symptoms should be evaluated for underlying conditions, including carbon monoxide poisoning.Physicians must make every effort to obtain the accurate medical history of patients with psychiatric disorders.

Diagnostic error rates and associated factors for lower gastrointestinal perforation.

Lower gastrointestinal perforation is rare and challenging to diagnose in patients presenting with an acute abdomen. However, no study has examined the frequency and associated factors of diagnostic errors related to lower gastrointestinal perforation. This large-scale multicenter retrospective study investigated the frequency of diagnostic errors and identified the associated factors. Factors at the level of the patient, symptoms, situation, and physician were included in the analysis. Data were collected from nine institutions, between January 1, 2015 and December 31, 2019. Timely diagnosis was defined as diagnosis at the first visit in computed tomography (CT)-capable facilities or referral to an appropriate medical institution immediately following the first visit to a non-CT-capable facility. Cases not meeting this definition were defined as diagnostic errors that resulted in delayed diagnosis. Of the 439 cases of lower gastrointestinal perforation identified, delayed diagnosis occurred in 138 cases (31.4%). Multivariate logistic regression analysis revealed a significant association between examination by a non-generalist and delayed diagnosis. Other factors showing a tendency with delayed diagnosis included presence of fever, absence of abdominal tenderness, and unavailability of urgent radiology reports. Initial misdiagnoses were mainly gastroenteritis, constipation, and small bowel obstruction. In conclusion, diagnostic errors occurred in about one-third of patients with a lower gastrointestinal perforation.

Clinical Decision Support Systems for Diagnosis in Primary Care: A Scoping Review.

Diagnosis is one of the crucial tasks performed by primary care physicians; however, primary care is at high risk of diagnostic errors due to the characteristics and uncertainties associated with the field. Prevention of diagnostic errors in primary care requires urgent action, and one of the possible methods is the use of health information technology. Its modes such as clinical decision support systems (CDSS) have been demonstrated to improve the quality of care in a variety of medical settings, including hospitals and primary care centers, though its usefulness in the diagnostic domain is still unknown. We conducted a scoping review to confirm the usefulness of the CDSS in the diagnostic domain in primary care and to identify areas that need to be explored. Search terms were chosen to cover the three dimensions of interest: decision support systems, diagnosis, and primary care. A total of 26 studies were included in the review. As a result, we found that the CDSS and reminder tools have significant effects on screening for common chronic diseases; however, the CDSS has not yet been fully validated for the diagnosis of acute and uncommon chronic diseases. Moreover, there were few studies involving non-physicians.

Analysis of diagnostic error cases among Japanese residents using diagnosis error evaluation and research taxonomy.

The process of diagnostic errors among Japanese residents has not been previously studied. This descriptive study was conducted in June 2019 on junior residents at a single-center educational hospital in Japan. Diagnosis Error Evaluation and Research taxonomy was used to measure the process of diagnostic error in the most memorable error cases. High frequency of diagnostic errors resulted from inaccurate/misinterpretation of history, failure/delay in eliciting physical examination findings, inaccurate/misinterpretation of physical examination, failure in weighting of physical examination, and failure/delay in considering the diagnosis. Residents made diagnostic errors mainly during history taking, physical examination, and assessment.