Even with ChatGPT, race matters.

BACKGROUND: Applications of large language models such as ChatGPT are increasingly being studied. Before these technologies become entrenched, it is crucial to analyze whether they perpetuate racial inequities. METHODS: We asked Open AI's ChatGPT-3.5 and ChatGPT-4 to simplify 750 radiology reports with the prompt "I am a ___ patient. Simplify this radiology report:" while providing the context of the five major racial classifications on the U.S. census: White, Black or African American, American Indian or Alaska Native, Asian, and Native Hawaiian or other Pacific Islander. To ensure an unbiased analysis, the readability scores of the outputs were calculated and compared. RESULTS: Statistically significant differences were found in both models based on the racial context. For ChatGPT-3.5, output for White and Asian was at a significantly higher reading grade level than both Black or African American and American Indian or Alaska Native, among other differences. For ChatGPT-4, output for Asian was at a significantly higher reading grade level than American Indian or Alaska Native and Native Hawaiian or other Pacific Islander, among other differences. CONCLUSION: Here, we tested an application where we would expect no differences in output based on racial classification. Hence, the differences found are alarming and demonstrate that the medical community must remain vigilant to ensure large language models do not provide biased or otherwise harmful outputs.

Quantitative Evaluation of Large Language Models to Streamline Radiology Report Impressions: A Multimodal Retrospective Analysis.

Background The complex medical terminology of radiology reports may cause confusion or anxiety for patients, especially given increased access to electronic health records. Large language models (LLMs) can potentially simplify radiology report readability. Purpose To compare the performance of four publicly available LLMs (ChatGPT-3.5 and ChatGPT-4, Bard [now known as Gemini], and Bing) in producing simplified radiology report impressions. Materials and Methods In this retrospective comparative analysis of the four LLMs (accessed July 23 to July 26, 2023), the Medical Information Mart for Intensive Care (MIMIC)-IV database was used to gather 750 anonymized radiology report impressions covering a range of imaging modalities (MRI, CT, US, radiography, mammography) and anatomic regions. Three distinct prompts were employed to assess the LLMs' ability to simplify report impressions. The first prompt (prompt 1) was "Simplify this radiology report." The second prompt (prompt 2) was "I am a patient. Simplify this radiology report." The last prompt (prompt 3) was "Simplify this radiology report at the 7th grade level." Each prompt was followed by the radiology report impression and was queried once. The primary outcome was simplification as assessed by readability score. Readability was assessed using the average of four established readability indexes. The nonparametric Wilcoxon signed-rank test was applied to compare reading grade levels across LLM output. Results All four LLMs simplified radiology report impressions across all prompts tested (P < .001). Within prompts, differences were found between LLMs. Providing the context of being a patient or requesting simplification at the seventh-grade level reduced the reading grade level of output for all models and prompts (except prompt 1 to prompt 2 for ChatGPT-4) (P < .001). Conclusion Although the success of each LLM varied depending on the specific prompt wording, all four models simplified radiology report impressions across all modalities and prompts tested. © RSNA, 2024 Supplemental material is available for this article. See also the editorial by Rahsepar in this issue.

Immunotherapy Initiation at the End of Life in Patients With Metastatic Cancer in the US.

Importance: While immunotherapy is being used in an expanding range of clinical scenarios, the incidence of immunotherapy initiation at the end of life (EOL) is unknown. Objective: To describe patient characteristics, practice patterns, and risk factors concerning EOL-initiated (EOL-I) immunotherapy over time. Design, Setting, and Participants: Retrospective cohort study using a US national clinical database of patients with metastatic melanoma, non-small cell lung cancer (NSCLC), or kidney cell carcinoma (KCC) diagnosed after US Food and Drug Administration approval of immune checkpoint inhibitors for the treatment of each disease through December 2019. Mean follow-up was 13.7 months. Data analysis was performed from December 2022 to May 2023. Exposures: Age, sex, race and ethnicity, insurance, location, facility type, hospital volume, Charlson-Deyo Comorbidity Index, and location of metastases. Main Outcomes and Measures: Main outcomes were EOL-I immunotherapy, defined as immunotherapy initiated within 1 month of death, and characteristics of the cohort receiving EOL-I immunotherapy and factors associated with its use. Results: Overall, data for 242 371 patients were analyzed. The study included 20 415 patients with stage IV melanoma, 197 331 patients with stage IV NSCLC, and 24 625 patients with stage IV KCC. Mean (SD) age was 67.9 (11.4) years, 42.5% were older than 70 years, 56.0% were male, and 29.3% received immunotherapy. The percentage of patients who received EOL-I immunotherapy increased over time for all cancers. More than 1 in 14 immunotherapy treatments in 2019 were initiated within 1 month of death. Risk-adjusted patients with 3 or more organs involved in metastatic disease were 3.8-fold more likely (95% CI, 3.1-4.7; P < .001) to die within 1 month of immunotherapy initiation than those with lymph node involvement only. Treatment at an academic or high-volume center rather than a nonacademic or very low-volume center was associated with a 31% (odds ratio, 0.69; 95% CI, 0.65-0.74; P < .001) and 30% (odds ratio, 0.70; 95% CI, 0.65-0.76; P < .001) decrease in odds of death within a month of initiating immunotherapy, respectively. Conclusions and Relevance: Findings of this cohort study show that the initiation of immunotherapy at the EOL is increasing over time. Patients with higher metastatic burden and who were treated at nonacademic or low-volume facilities had higher odds of receiving EOL-I immunotherapy. Tracking EOL-I immunotherapy can offer insights into national prescribing patterns and serve as a harbinger for shifts in the clinical approach to patients with advanced cancer.

Cost-Effectiveness of CT, CTA, MRI, and Specialized MRI for Evaluation of Patients Presenting to the Emergency Department With Dizziness.

BACKGROUND. Underlying stroke is often misdiagnosed in patients presenting with dizziness. Although such patients are usually ineligible for acute stroke treatment, accurate diagnosis may still improve outcomes through selection of patients for secondary prevention measures. OBJECTIVE. The purpose of our study was to investigate the cost-effectiveness of differing neuroimaging approaches in the evaluation of patients presenting to the emergency department (ED) with dizziness who are not candidates for acute intervention. METHODS. A Markov decision-analytic model was constructed from a health care system perspective for the evaluation of a patient presenting to the ED with dizziness. Four diagnostic strategies were compared: noncontrast head CT, head and neck CTA, conventional brain MRI, and specialized brain MRI (including multiplanar high-resolution DWI). Differing long-term costs and outcomes related to stroke detection and secondary prevention measures were compared. Cost-effectiveness was calculated in terms of lifetime expenditures in 2022 U.S. dollars for each quality-adjusted life year (QALY); deterministic and probabilistic sensitivity analyses were performed. RESULTS. Specialized MRI resulted in the highest QALYs and was the most cost-effective strategy with US$13,477 greater cost and 0.48 greater QALYs compared with noncontrast head CT. Conventional MRI had the next-highest health benefit, although was dominated by extension with incremental cost of US$6757 and 0.25 QALY; CTA was also dominated by extension, with incremental cost of US$3952 for 0.13 QALY. Non-contrast CT alone had the lowest utility among the four imaging choices. In the deterministic sensitivity analyses, specialized MRI remained the most cost-effective strategy. Conventional MRI was more cost-effective than CTA across a wide range of model parameters, with incremental cost-effectiveness remaining less than US$30,000/QALY. Probabilistic sensitivity analysis yielded similar results as found in the base-case analysis, with specialized MRI being more cost-effective than conventional MRI, which in turn was more cost-effective than CTA. CONCLUSION. The use of MRI in patients presenting to the ED with dizziness improves stroke detection and selection for subsequent preventive measures. MRI-based evaluation leads to lower long-term costs and higher cumulative QALYs. CLINICAL IMPACT. MRI, incorporating specialized protocols when available, is the preferred approach for evaluation of patients presenting to the ED with dizziness, to establish a stroke diagnosis and to select patients for secondary prevention measures.

Artificial Intelligence to Improve Patient Understanding of Radiology Reports.

Diagnostic imaging reports are generally written with a target audience of other providers. As a result, the reports are written with medical jargon and technical detail to ensure accurate communication. With implementation of the 21st Century Cures Act, patients have greater and quicker access to their imaging reports, but these reports are still written above the comprehension level of the average patient. Consequently, many patients have requested reports to be conveyed in language accessible to them. Numerous studies have shown that improving patient understanding of their condition results in better outcomes, so driving comprehension of imaging reports is essential. Summary statements, second reports, and the inclusion of the radiologist's phone number have been proposed, but these solutions have implications for radiologist workflow. Artificial intelligence (AI) has the potential to simplify imaging reports without significant disruptions. Many AI technologies have been applied to radiology reports in the past for various clinical and research purposes, but patient focused solutions have largely been ignored. New natural language processing technologies and large language models (LLMs) have the potential to improve patient understanding of their imaging reports. However, LLMs are a nascent technology and significant research is required before LLM-driven report simplification is used in patient care.

American College of Radiology Appropriateness Criteria®: a bibliometric analysis of panel members.

OBJECTIVE: To assess the features of panel members involved in the writing of the ACR-AC and identify alignment with research output and topic-specific research publications. METHODS: A cross-sectional analysis was performed on the research output of panel members of 34 ACR-AC documents published in 2021. For each author, we searched Medline to record total number of papers (P), total number of ACR-AC papers (C) and total number of previously published papers that are relevant to the ACR-AC topic (R). RESULTS: Three hundred eighty-three different panel members constituted 602 panel positions for creating 34 ACR-AC in 2021 with a median panel size of 17 members. Sixty-eight (17.5%) of experts had been part of ≥10 previously published ACR-AC papers and 154 (40%) were members in ≥ 5 published ACR-AC papers. The median number of previously published papers relevant to the ACR-AC topic was 1 (IQR: 0-5). 44% of the panel members had no previously published paper relevant to the ACR-AC topic. The proportion of ACR-AC papers (C/P) was higher for authors with ≥ 5 ACR-AC papers (0.21) than authors with < 5 ACR-AC papers (0.11, p < 0.0001); however, proportion of relevant papers per topic (R/P) was higher for authors with < 5 ACR-AC papers (0.10) than authors with ≥ 5 ACR-AC papers (0.07). CONCLUSION: The composition of the ACR Appropriateness Criteria panels reflects many members with little or no previously published literature on the topic of consideration. Similar pool of experts exists on multiple expert panels formulating imaging appropriateness guidelines. KEY POINTS: There were 68 (17.5%) panel experts on ≥ 10 ACR-AC panels. Nearly 45% of the panel experts had zero median number of relevant papers. Fifteen panels (44%) had > 50% of members having zero relevant papers.

CT With CTA Versus MRI in Patients Presenting to the Emergency Department With Dizziness: Analysis Using Propensity Score Matching.

BACKGROUND. CT with CTA is widely used to exclude stroke in patients with dizziness, although MRI has higher sensitivity. OBJECTIVE. The purpose of this article was to compare patients presenting to the emergency department (ED) with dizziness who undergo CT with CTA alone versus those who undergo MRI in terms of stroke-related management and outcomes. METHODS. This retrospective study included 1917 patients (mean age, 59.5 years; 776 men, 1141 women) presenting to the ED with dizziness from January 1, 2018, to December 31, 2021. A first propensity score matching analysis incorporated demographic characteristics, medical history, findings from the review of systems, physical examination findings, and symptoms to construct matched groups of patients discharged from the ED after undergoing head CT with head and neck CTA alone and patients who underwent brain MRI (with or without CT and CTA). Outcomes were compared. A second analysis compared matched patients discharged after CT with CTA alone and patients who underwent specialized abbreviated MRI using multiplanar high-resolution DWI for increased sensitivity for posterior circulation stroke. Sensitivity analyses were performed involving MRI examinations performed as the first or only neuroimaging examination and involving alternative matching and imputation techniques. RESULTS. In the first analysis (406 patients per group), patients who underwent MRI, compared with patients who underwent CT with CTA alone, showed greater frequency of critical neuroimaging results (10.1% vs 4.7%, p = .005), change in secondary stroke prevention medication (9.6% vs 3.2%, p = .001), and subsequent echocardiography evaluation (6.4% vs 1.0%, p < .001). In the second analysis (100 patients per group), patients who underwent specialized abbreviated MRI, compared with patients who underwent CT with CTA alone, showed greater frequency of critical neuroimaging results (10.0% vs 2.0%, p = .04), change in secondary stroke prevention medication (14.0% vs 1.0%, p = .001), and subsequent echocardiography evaluation (12.0% vs 2.0%, p = .01) and lower frequency of 90-day ED readmissions (12.0% vs 28.0%, p = .008). Sensitivity analyses showed qualitatively similar findings. CONCLUSION. A proportion of patients discharged after CT with CTA alone may have benefitted from alternative or additional evaluation by MRI (including MRI using a specialized abbreviated protocol). CLINICAL IMPACT. Use of MRI may motivate clinically impactful management changes in patients presenting with dizziness.

Characterization of the Landscape of Joint MD/MBA Programs in the US, 2002 to 2022.

This cross-sectional study characterizes the landscape of joint MD/MBA programs in the US from 2002 to 2022.

Distribution and Disparities of Industry Payments to Radiologists (2016-2020).

BACKGROUND: The frequency, magnitude, and distribution of industry payments to radiologists are not well understood. RATIONALE AND OBJECTIVES: The aim of this study was to analyze the distribution of industry payments to physicians working in diagnostic radiology, interventional radiology, and radiation oncology, study the categories of payments and determine their correlation. MATERIALS AND METHODS: The Open Payments Database from the Centers for Medicare & Medicaid Services was accessed and analyzed for the period from January 1, 2016 to December 31, 2020. Payments were grouped into six categories: consulting fees, education, gifts, research, speaker fees, and royalties/ownership. The total amount and types of industry payments going to the top 5% group were determined overall and for each category of payment. RESULTS: From 2016 to 2020, a total of 513 020 payments, amounting to $370 782 608, were made to 28 739 radiologists suggesting that approximately 70% of the 41 000 radiologists in the US received at least one industry payment during the 5-year period. The median payment value was $27 (IQR: $15-$120) and the median number of payments per physician over the 5-year period was 4 (IQR: 1-13). Gifts were the most frequent payment type made (76.4%), but accounted for only 4.8% of payment value. The median total value of payments earned by members of the top 5% group over the 5-year period was $58 878 (IQR: $29 686-$162 425) ($11 776 per year) compared to $172 (IQR: $49-877) ($34 per year) in the bottom 95% group. Members of the top 5% group received a median of 67 (IQR: 26-147) individual payments (13 payments per year) while members of the bottom 95% group received a median of 3 (IQR: 1-11) (0.6 payments per year). CONCLUSION: Between 2016 and 2020, industry payments to radiologists were highly concentrated both in terms of number/frequency and value of payments.

Industry Payments to Radiologists During the Last 5 Years and Impact of COVID-19 Pandemic.

OBJECTIVE: The aim of this study is to assess the trends in industry payments to radiologists and the impact of the COVID-19 pandemic, including trends in different categories of payments. METHODS: The Open Payments Database from CMS was accessed and analyzed for the period from January 1, 2016, to December 31, 2021. Payments were grouped into six categories: consulting fees, education, gifts, research, speaker fees, and royalties or ownership. The total number, value, and types of industry payments to radiologists were subsequently determined and compared pre- and postpandemic from 2016 to 2021. RESULTS: The total number of industry payments and the number of radiologists receiving these payments dropped by 50% and 32%, respectively, between 2019 and 2020, with only partial recovery in 2021. However, the mean payment value and total payment value increased by 177% and 37%, respectively, between 2019 and 2020. Gifts and speaker fees experienced the largest decreases between 2019 and 2020 (54% and 63%, respectively). Research and education grants were also disrupted, with the number of payments decreasing by 37% and 36% and payment value decreasing by 37% and 25%, respectively. However, royalty or ownership increased during the first year of the pandemic (8% for number of payments and 345% for value of payments). CONCLUSIONS: There was significant decline in overall industry payments coinciding with the COVID-19 pandemic, with biggest declines in gifts and speaker fees. The impact on the different categories of payments and recovery in the last 2 years has been heterogeneous.

Clinical criteria to exclude acute vascular pathology on CT angiogram in patients with dizziness.

BACKGROUND: Patients presenting to the emergency department (ED) with dizziness may be imaged via CTA head and neck to detect acute vascular pathology including large vessel occlusion. We identify commonly documented clinical variables which could delineate dizzy patients with near zero risk of acute vascular abnormality on CTA. METHODS: We performed a cross-sectional analysis of adult ED encounters with chief complaint of dizziness and CTA head and neck imaging at three EDs between 1/1/2014-12/31/2017. A decision rule was derived to exclude acute vascular pathology tested on a separate validation cohort; sensitivity analysis was performed using dizzy "stroke code" presentations. RESULTS: Testing, validation, and sensitivity analysis cohorts were composed of 1072, 357, and 81 cases with 41, 6, and 12 instances of acute vascular pathology respectively. The decision rule had the following features: no past medical history of stroke, arterial dissection, or transient ischemic attack (including unexplained aphasia, incoordination, or ataxia); no history of coronary artery disease, diabetes, migraines, current/long-term smoker, and current/long-term anti-coagulation or anti-platelet medication use. In the derivation phase, the rule had a sensitivity of 100% (95% CI: 0.91-1.00), specificity of 59% (95% CI: 0.56-0.62), and negative predictive value of 100% (95% CI: 0.99-1.00). In the validation phase, the rule had a sensitivity of 100% (95% CI: 0.61-1.00), specificity of 53% (95% CI: 0.48-0.58), and negative predictive value of 100% (95% CI: 0.98-1.00). The rule performed similarly on dizzy stroke codes and was more sensitive/predictive than all NIHSS cut-offs. CTAs for dizziness might be avoidable in 52% (95% CI: 0.47-0.57) of cases. CONCLUSIONS: A collection of clinical factors may be able to "exclude" acute vascular pathology in up to half of patients imaged by CTA for dizziness. These findings require further development and prospective validation, though could improve the evaluation of dizzy patients in the ED.

No such thing as a free lunch: The direct marginal costs of breastfeeding.

Understanding costs associated with breastfeeding is critical to developing maximally effective policy to support breastfeeding by addressing financial barriers. Breastfeeding is not without cost; direct costs include those of equipment, modified nutritional intake, and time (opportunity cost). Breastfeeding need not require more equipment than formula feeding, though maternal equipment use varies by maternal preference. Meeting increased nutritional demands requires increased spending on food and potentially dietary supplementation, the marginal cost of which depends on a mother's baseline diet. The opportunity cost of the three to four hours per day breastfeeding demands may be prohibitively high, particularly to low-income workers. These costs are relatively highest for low-income individuals, a group disproportionately comprising racial and ethnic minorities, and who demonstrate lower rates of breastfeeding than their white and higher-income peers. Acknowledging and addressing these costs and their regressive nature represents a critical component of effective breastfeeding policy and promotion.

Clinical Implementation of a Combined Artificial Intelligence and Natural Language Processing Quality Assurance Program for Pulmonary Nodule Detection in the Emergency Department Setting.

OBJECTIVE: This quality assurance study assessed the implementation of a combined artificial intelligence (AI) and natural language processing (NLP) program for pulmonary nodule detection in the emergency department setting. The program was designed to function outside of normal reading workflows to minimize radiologist interruption. MATERIALS AND METHODS: In all, 19,246 CT examinations including at least some portion of the lung anatomy performed in the emergent setting from October 1, 2021, to June 1, 2022, were processed by the combined AI-NLP program. The program used an AI algorithm trained on 6-mm to 30-mm pulmonary nodules to analyze CT images and an NLP to analyze radiological reports. Cases flagged as negative for pulmonary nodules by the NLP but positive by the AI algorithm were classified as suspected discrepancies. Discrepancies result in secondary review of examinations for possible addenda. RESULTS: Out of 19,246 CT examinations, 50 examinations (0.26%) resulted in secondary review, and 34 of 50 (68%) reviews resulted in addenda. Of the 34 addenda, 20 patients received instruction for new follow-up imaging. Median time to addendum was 11 hours. The majority of reviews and addenda resulted from missed pulmonary nodules on CT examinations of the abdomen and pelvis. CONCLUSION: A background quality assurance process using AI and NLP helped improve the detection of pulmonary nodules and resulted in increased numbers of patients receiving appropriate follow-up imaging recommendations. This was achieved without disrupting in-shift radiologist workflow or causing significant delays in patient follow for the diagnosed pulmonary nodule.