Exploring the proficiency of ChatGPT-4: An evaluation of its performance in the Taiwan advanced medical licensing examination.

Background: Taiwan is well-known for its quality healthcare system. The country's medical licensing exams offer a way to evaluate ChatGPT's medical proficiency. Methods: We analyzed exam data from February 2022, July 2022, February 2023, and July 2033. Each exam included four papers with 80 single-choice questions, grouped as descriptive or picture-based. We used ChatGPT-4 for evaluation. Incorrect answers prompted a "chain of thought" approach. Accuracy rates were calculated as percentages. Results: ChatGPT-4's accuracy in medical exams ranged from 63.75% to 93.75% (February 2022-July 2023). The highest accuracy (93.75%) was in February 2022's Medicine Exam (3). Subjects with the highest misanswered rates were ophthalmology (28.95%), breast surgery (27.27%), plastic surgery (26.67%), orthopedics (25.00%), and general surgery (24.59%). While using "chain of thought," the "Accuracy of (CoT) prompting" ranged from 0.00% to 88.89%, and the final overall accuracy rate ranged from 90% to 98%. Conclusion: ChatGPT-4 succeeded in Taiwan's medical licensing exams. With the "chain of thought" prompt, it improved accuracy to over 90%.

Performance of ChatGPT incorporated chain-of-thought method in bilingual nuclear medicine physician board examinations.

Objective: This research explores the performance of ChatGPT, compared to human doctors, in bilingual, Mandarin Chinese and English, medical specialty exam in Nuclear Medicine in Taiwan. Methods: The study employed generative pre-trained transformer (GPT-4) and integrated chain-of-thoughts (COT) method to enhance performance by triggering and explaining the thinking process to answer the question in a coherent and logical manner. Questions from the Taiwanese Nuclear Medicine Specialty Exam served as the basis for testing. The research analyzed the correctness of AI responses in different sections of the exam and explored the influence of question length and language proportion on accuracy. Results: AI, especially ChatGPT with COT, exhibited exceptional capabilities in theoretical knowledge, clinical medicine, and handling integrated questions, often surpassing, or matching human doctor performance. However, AI struggled with questions related to medical regulations. The analysis of question length showed that questions within the 109-163 words range yielded the highest accuracy. Moreover, an increase in the proportion of English words in questions improved both AI and human accuracy. Conclusions: This research highlights the potential and challenges of AI in the medical field. ChatGPT demonstrates significant competence in various aspects of medical knowledge. However, areas like medical regulations require improvement. The study also suggests that AI may help in evaluating exam question difficulty and maintaining fairness in examinations. These findings shed light on AI role in the medical field, with potential applications in healthcare education, exam preparation, and multilingual environments. Ongoing AI advancements are expected to further enhance AI utility in the medical domain.

Artificial Intelligence of Object Detection in Skeletal Scintigraphy for Automatic Detection and Annotation of Bone Metastases.

BACKGROUND: When cancer has metastasized to bone, doctors must identify the site of the metastases for treatment. In radiation therapy, damage to healthy areas or missing areas requiring treatment should be avoided. Therefore, it is necessary to locate the precise bone metastasis area. The bone scan is a commonly applied diagnostic tool for this purpose. However, its accuracy is limited by the nonspecific character of radiopharmaceutical accumulation. The study evaluated object detection techniques to improve the efficacy of bone metastases detection on bone scans. METHODS: We retrospectively examined the data of 920 patients, aged 23 to 95 years, who underwent bone scans between May 2009 and December 2019. The bone scan images were examined using an object detection algorithm. RESULTS: After reviewing the image reports written by physicians, nursing staff members annotated the bone metastasis sites as ground truths for training. Each set of bone scans contained anterior and posterior images with resolutions of 1024 × 256 pixels. The optimal dice similarity coefficient (DSC) in our study was 0.6640, which differs by 0.04 relative to the optimal DSC of different physicians (0.7040). CONCLUSIONS: Object detection can help physicians to efficiently notice bone metastases, decrease physician workload, and improve patient care.

Real-World Testing of a Machine Learning-Derived Visual Scale for Tc99m TRODAT-1 for Diagnosing Lewy Body Disease: Comparison with a Traditional Approach Using Semiquantification.

Objectives: Abnormal dopamine transporter (DAT) uptake is an important biomarker for diagnosing Lewy body disease (LBD), including Parkinson's disease (PD) and dementia with Lewy bodies (DLB). We evaluated a machine learning-derived visual scale (ML-VS) for Tc99m TRODAT-1 from one center and compared it with the striatal/background ratio (SBR) using semiquantification for diagnosing LBD in two other centers. Patients and Methods: This was a retrospective analysis of data from a history-based computerized dementia diagnostic system. MT-VS and SBR among normal controls (NCs) and patients with PD, PD with dementia (PDD), DLB, or Alzheimer's disease (AD) were compared. Results: We included 715 individuals, including 122 NCs, 286 patients with PD, 40 with AD, 179 with DLB, and 88 with PDD. Compared with NCs, patients with PD exhibited a significantly higher prevalence of abnormal DAT uptake using all methods. Compared with the AD group, PDD and DLB groups exhibited a significantly higher prevalence of abnormal DAT uptake using all methods. The distribution of ML-VS was significantly different between PD and NC, DLB and AD, and PDD and AD groups (all p < 0.001). The correlation coefficient of ML-VS/SBR in all participants was 0.679. Conclusions: The ML-VS designed in one center is useful for differentiating PD from NC, DLB from AD, and PDD from AD in other centers. Its correlation with traditional approaches using different scanning machines is also acceptable. Future studies should develop models using data pools from multiple centers for increasing diagnostic accuracy.

Detection of Bone Metastases on Bone Scans through Image Classification with Contrastive Learning.

Patients with bone metastases have poor prognoses. A bone scan is a commonly applied diagnostic tool for this condition. However, its accuracy is limited by the nonspecific character of radiopharmaceutical accumulation, which indicates all-cause bone remodeling. The current study evaluated deep learning techniques to improve the efficacy of bone metastasis detection on bone scans, retrospectively examining 19,041 patients aged 22 to 92 years who underwent bone scans between May 2011 and December 2019. We developed several functional imaging binary classification deep learning algorithms suitable for bone scans. The presence or absence of bone metastases as a reference standard was determined through a review of image reports by nuclear medicine physicians. Classification was conducted with convolutional neural network-based (CNN-based), residual neural network (ResNet), and densely connected convolutional networks (DenseNet) models, with and without contrastive learning. Each set of bone scans contained anterior and posterior images with resolutions of 1024 × 256 pixels. A total of 37,427 image sets were analyzed. The overall performance of all models improved with contrastive learning. The accuracy, precision, recall, F1 score, area under the receiver operating characteristic curve, and negative predictive value (NPV) for the optimal model were 0.961, 0.878, 0.599, 0.712, 0.92 and 0.965, respectively. In particular, the high NPV may help physicians safely exclude bone metastases, decreasing physician workload, and improving patient care.

Low-Parameter Small Convolutional Neural Network Applied to Functional Medical Imaging of Tc-99m Trodat-1 Brain Single-Photon Emission Computed Tomography for Parkinson's Disease.

Parkinson's disease (PD), a progressive disease that affects movement, is related to dopaminergic neuron degeneration. Tc-99m Trodat-1 brain (TRODAT) single-photon emission computed tomography (SPECT) aids the functional imaging of dopamine transporters and is used for dopaminergic neuron enumeration. Herein, we employed a convolutional neural network to facilitate PD diagnosis through TRODAT SPECT, which is simpler than models such as VGG16 and ResNet50. We retrospectively collected the data of 3188 patients (age range 20-107 years) who underwent TRODAT SPECT between June 2011 and December 2019. We developed a set of functional imaging multiclassification deep learning algorithms suitable for TRODAT SPECT on the basis of the annotations of medical experts. We then applied our self-proposed model and compared its results with those of four other models, including deep and machine learning models. TRODAT SPECT included three images collected from each patient: one presenting the maximum absorption of the metabolic function of the striatum and two adjacent images. An expert physician determined that our model's accuracy, precision, recall, and F1-score were 0.98, 0.98, 0.98, and 0.98, respectively. Our TRODAT SPECT model provides an objective, more standardized classification correlating to the severity of PD-related diseases, thereby facilitating clinical diagnosis and preventing observer bias.