Evaluation of GPT-4 ability to identify and generate patient instructions for actionable incidental radiology findings.

OBJECTIVES: To evaluate the proficiency of a HIPAA-compliant version of GPT-4 in identifying actionable, incidental findings from unstructured radiology reports of Emergency Department patients. To assess appropriateness of artificial intelligence (AI)-generated, patient-facing summaries of these findings. MATERIALS AND METHODS: Radiology reports extracted from the electronic health record of a large academic medical center were manually reviewed to identify non-emergent, incidental findings with high likelihood of requiring follow-up, further sub-stratified as "definitely actionable" (DA) or "possibly actionable-clinical correlation" (PA-CC). Instruction prompts to GPT-4 were developed and iteratively optimized using a validation set of 50 reports. The optimized prompt was then applied to a test set of 430 unseen reports. GPT-4 performance was primarily graded on accuracy identifying either DA or PA-CC findings, then secondarily for DA findings alone. Outputs were reviewed for hallucinations. AI-generated patient-facing summaries were assessed for appropriateness via Likert scale. RESULTS: For the primary outcome (DA or PA-CC), GPT-4 achieved 99.3% recall, 73.6% precision, and 84.5% F-1. For the secondary outcome (DA only), GPT-4 demonstrated 95.2% recall, 77.3% precision, and 85.3% F-1. No findings were "hallucinated" outright. However, 2.8% of cases included generated text about recommendations that were inferred without specific reference. The majority of True Positive AI-generated summaries required no or minor revision. CONCLUSION: GPT-4 demonstrates proficiency in detecting actionable, incidental findings after refined instruction prompting. AI-generated patient instructions were most often appropriate, but rarely included inferred recommendations. While this technology shows promise to augment diagnostics, active clinician oversight via "human-in-the-loop" workflows remains critical for clinical implementation.

Evaluation of a Structured Review Process for Emergency Department Return Visits with Admission.

BACKGROUND: Review of emergency department (ED) revisits with admission allows the identification of improvement opportunities. Applying a health equity lens to revisits may highlight potential disparities in care transitions. Universal definitions or practicable frameworks for these assessments are lacking. The authors aimed to develop a structured methodology for this quality assurance (QA) process, with a layered equity analysis. METHODS: The authors developed a classification instrument to identify potentially preventable 72-hour returns with admission (PPRA-72), accounting for directed, unrelated, unanticipated, or disease progression returns. A second review team assessed the instrument reliability. A self-reported race/ethnicity (R/E) and language algorithm was developed to minimize uncategorizable data. Disposition distribution, return rates, and PPRA-72 classifications were analyzed for disparities using Pearson chi-square and Fisher's exact tests. RESULTS: The PPRA-72 rate was 4.8% for 2022 ED return visits requiring admission. Review teams achieved 93% agreement (κ = 0.51) for the binary determination of PPRA-72 vs. nonpreventable returns. There were significant differences between R/E and language in ED dispositions (p < 0.001), with more frequent admissions for the R/E White at the index visit and Other at the 72-hour return visit. Rates of return visits within 72 hours differed significantly by R/E (p < 0.001) but not by language (p = 0.156), with the R/E Black most frequent to have a 72-hour return. There were no differences between R/E (p = 0.446) or language (p = 0.248) in PPRA-72 rates. The initiative led to system improvements through informatics optimizations, triage protocols, provider feedback, and education. CONCLUSION: The authors developed a review methodology for identifying improvement opportunities across ED 72-hour returns. This QA process enabled the identification of areas of disparity, with the continuous aim to develop next steps in ensuring health equity in care transitions.

Expanding Diabetes Screening to Identify Undiagnosed Cases Among Emergency Department Patients.

Introduction: Diabetes screening traditionally occurs in primary care settings, but many who are at high risk face barriers to accessing care and therefore delays in diagnosis and treatment. These same high-risk patients do frequently visit emergency departments (ED) and, therefore, might benefit from screening at that time. Our objective in this study was to analyze one year of results from a multisite, ED-based diabetes screening program. Methods: We assessed the demographics of patients screened, identified differences in rates of newly diagnosed diabetes by clinical site, and the geographic distribution of high and low hemoglobin A1c (HbA1c) results. Results: We performed diabetes screening (HbA1c) among 4,211 ED patients 40-70 years old, with a body mass index ≥25, and no prior history of diabetes. Of these patients screened for diabetes, 9% had a HbA1c result consistent with undiagnosed diabetes, and nearly half of these patients had a HbA1c ≥9.0%. Rates of newly diagnosed diabetes were notably higher at EDs located in neighborhoods of lower socioeconomic status. Conclusion: Emergency department-based diabetes screening may be a practical and scalable solution to screen high-risk patients and reduce health disparities experienced in specific neighborhoods and demographic groups.

The Effect of Pay-for-Performance Compensation Model Implementation on Vaccination Rate: A Systematic Review.

BACKGROUND AND OBJECTIVES: Pay-for-performance (P4P) is broadly defined as financial incentives to providers for attaining prespecified quality outcomes. Providers, payers, and public officials have worked over the years to develop innovative solutions to rapidly and consistently bring new diagnostic tests and therapies to our patients. P4P has been instituted in various forms over the last 30 years. Vaccines are one of society's greatest public health innovations and vaccination programs provide a unique opportunity for P4P programs. We attempted to investigate the effect of P4P compensation model implementation on the vaccination rate. METHODS: Utilizing a systematic review and meta-analysis approach, we searched PubMed, Embase, Scopus, and Web of Science from inception to December 2018. RESULTS: Nine articles were included with poor to moderate quality. Improvements in vaccination rates after implementation of P4P were statistically significant in 8 of 9 of studies. However, due to the heterogeneity of the methods used, we could not pool the data. CONCLUSION: The results of this systematic review indicate that the implementation of P4P programs can increase the vaccination rate. In recent times when it has become increasingly more popular not to vaccinate, implementing P4P becomes even more important if it is shown to be an effective tool in increasing vaccination rates.