Assessment of Artificial Intelligence Chatbot Responses to Top Searched Queries About Cancer.

Importance: Consumers are increasingly using artificial intelligence (AI) chatbots as a source of information. However, the quality of the cancer information generated by these chatbots has not yet been evaluated using validated instruments. Objective: To characterize the quality of information and presence of misinformation about skin, lung, breast, colorectal, and prostate cancers generated by 4 AI chatbots. Design, Setting, and Participants: This cross-sectional study assessed AI chatbots' text responses to the 5 most commonly searched queries related to the 5 most common cancers using validated instruments. Search data were extracted from the publicly available Google Trends platform and identical prompts were used to generate responses from 4 AI chatbots: ChatGPT version 3.5 (OpenAI), Perplexity (Perplexity.AI), Chatsonic (Writesonic), and Bing AI (Microsoft). Exposures: Google Trends' top 5 search queries related to skin, lung, breast, colorectal, and prostate cancer from January 1, 2021, to January 1, 2023, were input into 4 AI chatbots. Main Outcomes and Measures: The primary outcomes were the quality of consumer health information based on the validated DISCERN instrument (scores from 1 [low] to 5 [high] for quality of information) and the understandability and actionability of this information based on the understandability and actionability domains of the Patient Education Materials Assessment Tool (PEMAT) (scores of 0%-100%, with higher scores indicating a higher level of understandability and actionability). Secondary outcomes included misinformation scored using a 5-item Likert scale (scores from 1 [no misinformation] to 5 [high misinformation]) and readability assessed using the Flesch-Kincaid Grade Level readability score. Results: The analysis included 100 responses from 4 chatbots about the 5 most common search queries for skin, lung, breast, colorectal, and prostate cancer. The quality of text responses generated by the 4 AI chatbots was good (median [range] DISCERN score, 5 [2-5]) and no misinformation was identified. Understandability was moderate (median [range] PEMAT Understandability score, 66.7% [33.3%-90.1%]), and actionability was poor (median [range] PEMAT Actionability score, 20.0% [0%-40.0%]). The responses were written at the college level based on the Flesch-Kincaid Grade Level score. Conclusions and Relevance: Findings of this cross-sectional study suggest that AI chatbots generally produce accurate information for the top cancer-related search queries, but the responses are not readily actionable and are written at a college reading level. These limitations suggest that AI chatbots should be used supplementarily and not as a primary source for medical information.

Quality of Information for Post-Prostatectomy Incontinence Treatments on YouTube.

OBJECTIVE: To assess the quality of information on YouTube regarding post-prostatectomy incontinence treatments. The 2019 American Urological Association(AUA)/(SUFU) Society of Urodynamics, Female Pelvic Medicine & Urogenital Reconstruction guidelines recommend the placement of a male sling or artificial urinary sphincter. Patient education is essential for appropriate expectations and patient satisfaction. METHODS: The top 100 most relevant (default setting) YouTube videos searched with terms "post-prostatectomy incontinence" and "male stress incontinence" were assessed using the validated DISCERN instrument, Patient Education Materials Assessment Tool for Audiovisual Materials, and a misinformation Likert Scale. Videos with poor video/audio quality, duplicates, and non-English videos were excluded. Four independent raters were randomly assigned such that each video had 2 raters. Data was analyzed using multivariate linear regression, and inter-rater reliability was measured using Cohen's kappa. RESULTS: The median DISCERN score was 2.5 with 71% of videos scoring <=3. The median PEMAT Actionability and Understandability scores were 64.6 (range 0-100) and 79.9 (range 52-100), respectively. The median misinformation Likert score was 1 range (1-4). Less than half of the videos discussed realistic treatment outcomes or the risks (39% and 24%, respectively). PEMAT Actionability and risk discussion significantly predicted average DISCERN score (p < .001). There were no significant disagreements between raters. CONCLUSION: Our study shows most videos on post-prostatectomy incontinence had moderate to low quality information and were published by medical professionals. The majority did not sufficiently discuss realistic outcomes and risks, which are hallmarks of informed decision making. This provides an opportunity for the urologic community to create educational materials that adequately supplement shared-decision making for patients treated for post-prostatectomy incontinence.

Impact of COVID-19 on Online Interest in Urologic Conditions: An Analysis of Google Trends.

Background With COVID-19 leading to several isolation measures for preventative care, health care utilization, especially within urology, decreased substantially. The impact of COVID-19 on the population's interests in urologic conditions remains to be established. By using the platform of Google Trends, which allows search behaviors and interest in healthcare topics to be quantified over time, we investigated the impact of COVID-19 on online search behaviors relating to common urologic conditions in the US. Methods The platform of Google Trends was utilized to analyze online interest in twelve common urologic conditions in the US from October 1, 2018 to August 1, 2021 (divided into "pre-COVID" and "COVID" periods at March 1, 2020). Search volume index (SVI), a measure of relative search volume on Google, data sets for the US, top queried and populated states, rising queries, and top queries were retrieved and analyzed for all conditions. Pre-COVID and COVID median SVIs were compared using the Mann Whitney U test, and correlations were analyzed using Spearman's rank-order correlation test. Results For all twelve urologic conditions, rising and top queries were often related to symptoms, treatments, and COVID-19. COVID showed higher SVIs for erectile dysfunction (p=0.04) and lower SVIs for bladder cancer (p<0.01), hematuria (p<0.01), kidney cancer (p<0.01), kidney stones (p=0.03), and prostate cancer (p<0.01). Correlations to COVID-19 searches were seen for bladder cancer (RS=-0.36, p<0.01), erectile dysfunction (RS=0.20, p=0.04), hematuria (RS=-0.31, p<0.01), overactive bladder (RS=-0.23, p=0.04), and prostate cancer (RS=-0.33, p<0.01). No correlations were found for benign prostatic hyperplasia, interstitial cystitis, low testosterone, urinary incontinence, and urinary tract infections. Conclusions Online interest in many urologic conditions, especially cancers, decreased during COVID. Given the internet's increasing role in healthcare, a reduced interest could translate to delayed diagnosis and treatment of these conditions. Only erectile dysfunction showed increasing interest, potentially due to research or misinformation linking it to COVID-19.

Functional comparison of exome capture-based methods for transcriptomic profiling of formalin-fixed paraffin-embedded tumors.

The availability of fresh frozen (FF) tissue is a barrier for implementing RNA sequencing (RNA-seq) in the clinic. The majority of clinical samples are stored as formalin-fixed, paraffin-embedded (FFPE) tissues. Exome capture platforms have been developed for RNA-seq from FFPE samples. However, these methods have not been systematically compared. We performed transcriptomic analysis of 32 FFPE tumor samples from 11 patients using three exome capture-based methods: Agilent SureSelect V6, TWIST NGS Exome, and IDT XGen Exome Research Panel. We compared these methods to the TruSeq RNA-seq of fresh frozen (FF-TruSeq) tumor samples from the same patients. We assessed the recovery of clinically relevant biological features. The Spearman's correlation coefficients between the global expression profiles of the three capture-based methods from FFPE and matched FF-TruSeq were high (rho = 0.72-0.9, p < 0.05). A significant correlation between the expression of key immune genes between individual capture-based methods and FF-TruSeq (rho = 0.76-0.88, p < 0.05) was observed. All exome capture-based methods reliably detected outlier expression of actionable gene transcripts, including ERBB2, MET, NTRK1, and PPARG. In urothelial cancer samples, the Agilent assay was associated with the highest molecular subtype concordance with FF-TruSeq (Cohen's k = 0.7, p < 0.01). The Agilent and IDT assays detected all the clinically relevant fusions that were initially identified in FF-TruSeq. All FFPE exome capture-based methods had comparable performance and concordance with FF-TruSeq. Our findings will enable the implementation of RNA-seq in the clinic to guide precision oncology approaches.

Validation of a Circulating Tumor DNA-Based Next-Generation Sequencing Assay in a Cohort of Patients with Solid tumors: A Proposed Solution for Decentralized Plasma Testing.

BACKGROUND: Characterization of circulating tumor DNA (ctDNA) has been integrated into clinical practice. Although labs have standardized validation procedures to develop single locus tests, the efficacy of on-site plasma-based next-generation sequencing (NGS) assays still needs to be proved. MATERIALS AND METHODS: In this retrospective study, we profiled DNA from matched tissue and plasma samples from 75 patients with cancer. We applied an NGS test that detects clinically relevant alterations in 33 genes and microsatellite instability (MSI) to analyze plasma cell-free DNA (cfDNA). RESULTS: The concordance between alterations detected in both tissue and plasma samples was higher in patients with metastatic disease. The NGS test detected 77% of sequence alterations, amplifications, and fusions that were found in metastatic samples compared with 45% of those alterations found in the primary tumor samples (p = .00005). There was 87% agreement on MSI status between the NGS test and tumor tissue results. In three patients, MSI-high ctDNA correlated with response to immunotherapy. In addition, the NGS test revealed an FGFR2 amplification that was not detected in tumor tissue from a patient with metastatic gastric cancer, emphasizing the importance of profiling plasma samples in patients with advanced cancer. CONCLUSION: Our validation experience of a plasma-based NGS assay advances current knowledge about translating cfDNA testing into clinical practice and supports the application of plasma assays in the management of oncology patients with metastatic disease. With an in-house method that minimizes the need for invasive procedures, on-site cfDNA testing supplements tissue biopsy to guide precision therapy and is entitled to become a routine practice. IMPLICATIONS FOR PRACTICE: This study proposes a solution for decentralized liquid biopsy testing based on validation of a next-generation sequencing (NGS) test that detects four classes of genomic alterations in blood: sequence mutations (single nucleotide substitutions or insertions and deletions), fusions, amplifications, and microsatellite instability (MSI). Although there are reference labs that perform single-site comprehensive liquid biopsy testing, the targeted assay this study validated can be established locally in any lab with capacity to offer clinical molecular pathology assays. To the authors' knowledge, this is the first report that validates evaluating an on-site plasma-based NGS test that detects the MSI status along with common sequence alterations encountered in solid tumors.