RESUMO
OBJECTIVES: To evaluate the efficacy of ChatGPT 4 (GPT-4) in delivering genetic information about BRCA1, HFE, and MLH1, building on previous findings with ChatGPT 3.5 (GPT-3.5). To focus on assessing the utility, limitations, and ethical implications of using ChatGPT in medical settings. MATERIALS AND METHODS: A structured survey was developed to assess GPT-4's clinical value. An expert panel of genetic counselors and clinical geneticists evaluated GPT-4's responses to these questions. We also performed comparative analysis with GPT-3.5, utilizing descriptive statistics and using Prism 9 for data analysis. RESULTS: The findings indicate improved accuracy in GPT-4 over GPT-3.5 (P < .0001). However, notable errors in accuracy remained. The relevance of responses varied in GPT-4, but was generally favorable, with a mean in the "somewhat agree" range. There was no difference in performance by disease category. The 7-question subset of the Bot Usability Scale (BUS-15) showed no statistically significant difference between the groups but trended lower in the GPT-4 version. DISCUSSION AND CONCLUSION: The study underscores GPT-4's potential role in genetic education, showing notable progress yet facing challenges like outdated information and the necessity of ongoing refinement. Our results, while showing promise, emphasizes the importance of balancing technological innovation with ethical responsibility in healthcare information delivery.
RESUMO
Genetic disorders are complex and can greatly impact an individual's health and well-being. In this study, we assess the ability of ChatGPT, a language model developed by OpenAI, to answer questions related to three specific genetic disorders: BRCA1, MLH1, and HFE. ChatGPT has shown it can supply articulate answers to a wide spectrum of questions. However, its ability to answer questions related to genetic disorders has yet to be evaluated. The aim of this study is to perform both quantitative and qualitative assessments of ChatGPT's performance in this area. The ability of ChatGPT to provide accurate and useful information to patients was assessed by genetic experts. Here we show that ChatGPT answered 64.7% of the 68 genetic questions asked and was able to respond coherently to complex questions related to the three genes/conditions. Our results reveal that ChatGPT can provide valuable information to individuals seeking information about genetic disorders, however, it still has some limitations and inaccuracies, particularly in understanding human inheritance patterns. The results of this study have implications for both genomics and medicine and can inform future developments in this area. AI platforms, like ChatGPT, have significant potential in the field of genomics. As these technologies become integrated into consumer-facing products, appropriate oversight is required to ensure accurate and safe delivery of medical information. With such oversight and training specifically for genetic information, these platforms could have the potential to augment some clinical interactions.
