A comparative evaluation of ChatGPT 3.5 and ChatGPT 4 in responses to selected genetics questions.

OBJECTIVES: To evaluate the efficacy of ChatGPT 4 (GPT-4) in delivering genetic information about BRCA1, HFE, and MLH1, building on previous findings with ChatGPT 3.5 (GPT-3.5). To focus on assessing the utility, limitations, and ethical implications of using ChatGPT in medical settings. MATERIALS AND METHODS: A structured survey was developed to assess GPT-4's clinical value. An expert panel of genetic counselors and clinical geneticists evaluated GPT-4's responses to these questions. We also performed comparative analysis with GPT-3.5, utilizing descriptive statistics and using Prism 9 for data analysis. RESULTS: The findings indicate improved accuracy in GPT-4 over GPT-3.5 (P < .0001). However, notable errors in accuracy remained. The relevance of responses varied in GPT-4, but was generally favorable, with a mean in the "somewhat agree" range. There was no difference in performance by disease category. The 7-question subset of the Bot Usability Scale (BUS-15) showed no statistically significant difference between the groups but trended lower in the GPT-4 version. DISCUSSION AND CONCLUSION: The study underscores GPT-4's potential role in genetic education, showing notable progress yet facing challenges like outdated information and the necessity of ongoing refinement. Our results, while showing promise, emphasizes the importance of balancing technological innovation with ethical responsibility in healthcare information delivery.

Evaluating ChatGPT as an Agent for Providing Genetic Education.

Genetic disorders are complex and can greatly impact an individual's health and well-being. In this study, we assess the ability of ChatGPT, a language model developed by OpenAI, to answer questions related to three specific genetic disorders: BRCA1, MLH1, and HFE. ChatGPT has shown it can supply articulate answers to a wide spectrum of questions. However, its ability to answer questions related to genetic disorders has yet to be evaluated. The aim of this study is to perform both quantitative and qualitative assessments of ChatGPT's performance in this area. The ability of ChatGPT to provide accurate and useful information to patients was assessed by genetic experts. Here we show that ChatGPT answered 64.7% of the 68 genetic questions asked and was able to respond coherently to complex questions related to the three genes/conditions. Our results reveal that ChatGPT can provide valuable information to individuals seeking information about genetic disorders, however, it still has some limitations and inaccuracies, particularly in understanding human inheritance patterns. The results of this study have implications for both genomics and medicine and can inform future developments in this area. AI platforms, like ChatGPT, have significant potential in the field of genomics. As these technologies become integrated into consumer-facing products, appropriate oversight is required to ensure accurate and safe delivery of medical information. With such oversight and training specifically for genetic information, these platforms could have the potential to augment some clinical interactions.

The Development of an Infrastructure to Facilitate the Use of Whole Genome Sequencing for Population Health.

The clinical use of genomic analysis has expanded rapidly resulting in an increased availability and utility of genomic information in clinical care. We have developed an infrastructure utilizing informatics tools and clinical processes to facilitate the use of whole genome sequencing data for population health management across the healthcare system. Our resulting framework scaled well to multiple clinical domains in both pediatric and adult care, although there were domain specific challenges that arose. Our infrastructure was complementary to existing clinical processes and well-received by care providers and patients. Informatics solutions were critical to the successful deployment and scaling of this program. Implementation of genomics at the scale of population health utilizes complicated technologies and processes that for many health systems are not supported by current information systems or in existing clinical workflows. To scale such a system requires a substantial clinical framework backed by informatics tools to facilitate the flow and management of data. Our work represents an early model that has been successful in scaling to 29 different genes with associated genetic conditions in four clinical domains. Work is ongoing to optimize informatics tools; and to identify best practices for translation to smaller healthcare systems.

Unanswered Questions: Hereditary breast and gynecological cancer risk assessment in transgender adolescents and young adults.

Transgender (TG) individuals have higher rates of mortality associated with cancer diagnoses, in part due to avoidance of gender-assigned cancer screenings resulting in later stages at diagnosis. Knowledge about the risks of breast or gynecological cancer in TG and nonbinary (NB) persons receiving gender-affirming hormone therapy is limited. Even less information exists regarding the subset of individuals with genetic predisposition for these malignancies. We performed a retrospective literature review of studies from the last 15 years on breast cancer rates and identified risks in TG persons. An accumulating body of data on breast cancer incidence in TG persons suggests higher than previously believed rates of breast cancer in TG women compared with cisgender men and risk correlating with duration of hormone use. Few studies have examined other cancer risks in TG populations. To date, only three publications address the association with BRCA1/2 mutation presence and breast cancer incidence in TG persons. Meanwhile, there is growing awareness and social acceptance of TG/NB identities coupled with recognition of gender dysphoria at increasingly earlier ages. No information directly addressing cancer risk counseling in TG/NB adolescents and young adults with a family history of cancer or hereditary cancer syndrome exists. Whether the presence of a known genetic predisposition or strong family cancer history may affect cancer risk in these populations is unknown, leading to significant gaps in clinicians' ability to accurately and appropriately estimate cancer risks and counsel those with genetic predisposition on the risks/benefits associated with surgical options and the initiation, duration, and dosing of gender-affirming hormone therapies. A series of three cases illustrates the utility of cancer risk assessment and genetic testing in TG/NB adolescents and young adults, and the unique challenges and unanswered questions that are encountered in the process.