The First Generative AI Prompt-A-Thon in Healthcare: A Novel Approach to Workforce Engagement with a Private Instance of ChatGPT.

BACKGROUND: Healthcare crowdsourcing events (e.g. hackathons) facilitate interdisciplinary collaboration and encourage innovation. Peer-reviewed research has not yet considered a healthcare crowdsourcing event focusing on generative artificial intelligence (GenAI), which generates text in response to detailed prompts and has vast potential for improving the efficiency of healthcare organizations. Our event, the New York University Langone Health (NYULH) Prompt-a-thon, primarily sought to inspire and build AI fluency within our diverse NYULH community, and foster collaboration and innovation. Secondarily, we sought to analyze how participants' experience was influenced by their prior GenAI exposure and whether they received sample prompts during the workshop. METHODS: Executing the event required the assembly of an expert planning committee, who recruited diverse participants, anticipated technological challenges, and prepared the event. The event was composed of didactics and workshop sessions, which educated and allowed participants to experiment with using GenAI on real healthcare data. Participants were given novel "project cards" associated with each dataset that illuminated the tasks GenAI could perform and, for a random set of teams, sample prompts to help them achieve each task (the public repository of project cards can be found at https://github.com/smallw03/NYULH-Generative-AI-Prompt-a-thon-Project-Cards). Afterwards, participants were asked to fill out a survey with 7-point Likert-style questions. RESULTS: Our event was successful in educating and inspiring hundreds of enthusiastic in-person and virtual participants across our organization on the responsible use of GenAI in a low-cost and technologically feasible manner. All participants responded positively, on average, to each of the survey questions (e.g., confidence in their ability to use and trust GenAI). Critically, participants reported a self-perceived increase in their likelihood of using and promoting colleagues' use of GenAI for their daily work. No significant differences were seen in the surveys of those who received sample prompts with their project task descriptions. CONCLUSION: The first healthcare Prompt-a-thon was an overwhelming success, with minimal technological failures, positive responses from diverse participants and staff, and evidence of post-event engagement. These findings will be integral to planning future events at our institution, and to others looking to engage their workforce in utilizing GenAI.

Evaluation of GPT-4 ability to identify and generate patient instructions for actionable incidental radiology findings.

OBJECTIVES: To evaluate the proficiency of a HIPAA-compliant version of GPT-4 in identifying actionable, incidental findings from unstructured radiology reports of Emergency Department patients. To assess appropriateness of artificial intelligence (AI)-generated, patient-facing summaries of these findings. MATERIALS AND METHODS: Radiology reports extracted from the electronic health record of a large academic medical center were manually reviewed to identify non-emergent, incidental findings with high likelihood of requiring follow-up, further sub-stratified as "definitely actionable" (DA) or "possibly actionable-clinical correlation" (PA-CC). Instruction prompts to GPT-4 were developed and iteratively optimized using a validation set of 50 reports. The optimized prompt was then applied to a test set of 430 unseen reports. GPT-4 performance was primarily graded on accuracy identifying either DA or PA-CC findings, then secondarily for DA findings alone. Outputs were reviewed for hallucinations. AI-generated patient-facing summaries were assessed for appropriateness via Likert scale. RESULTS: For the primary outcome (DA or PA-CC), GPT-4 achieved 99.3% recall, 73.6% precision, and 84.5% F-1. For the secondary outcome (DA only), GPT-4 demonstrated 95.2% recall, 77.3% precision, and 85.3% F-1. No findings were "hallucinated" outright. However, 2.8% of cases included generated text about recommendations that were inferred without specific reference. The majority of True Positive AI-generated summaries required no or minor revision. CONCLUSION: GPT-4 demonstrates proficiency in detecting actionable, incidental findings after refined instruction prompting. AI-generated patient instructions were most often appropriate, but rarely included inferred recommendations. While this technology shows promise to augment diagnostics, active clinician oversight via "human-in-the-loop" workflows remains critical for clinical implementation.

Treating COVID-19 With Hydroxychloroquine (TEACH): A Multicenter, Double-Blind Randomized Controlled Trial in Hospitalized Patients.

BACKGROUND: Effective therapies to combat coronavirus 2019 (COVID-19) are urgently needed. Hydroxychloroquine (HCQ) has in vitro antiviral activity against severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), but the clinical benefit of HCQ in treating COVID-19 is unclear. Randomized controlled trials are needed to determine the safety and efficacy of HCQ for the treatment of hospitalized patients with COVID-19. METHODS: We conducted a multicenter, double-blind randomized clinical trial of HCQ among patients hospitalized with laboratory-confirmed COVID-19. Subjects were randomized in a 1:1 ratio to HCQ or placebo for 5 days and followed for 30 days. The primary efficacy outcome was a severe disease progression composite end point (death, intensive care unit admission, mechanical ventilation, extracorporeal membrane oxygenation, and/or vasopressor use) at day 14. RESULTS: A total of 128 patients were included in the intention-to-treat analysis. Baseline demographic, clinical, and laboratory characteristics were similar between the HCQ (n = 67) and placebo (n = 61) arms. At day 14, 11 (16.4%) subjects assigned to HCQ and 6 (9.8%) subjects assigned to placebo met the severe disease progression end point, but this did not achieve statistical significance (P = .350). There were no significant differences in COVID-19 clinical scores, number of oxygen-free days, SARS-CoV-2 clearance, or adverse events between HCQ and placebo. HCQ was associated with a slight increase in mean corrected QT interval, an increased D-dimer, and a trend toward an increased length of stay. CONCLUSIONS: In hospitalized patients with COVID-19, our data suggest that HCQ does not prevent severe outcomes or improve clinical scores. However, our conclusions are limited by a relatively small sample size, and larger randomized controlled trials or pooled analyses are needed.

The Financial and Clinical Impact of an Electronic Health Record Integrated Pathway in Elective Colon Surgery.

BACKGROUND: Enhanced Recovery after Surgery (ERAS) pathways have been shown to reduce length of stay, but there have been limited evaluations of novel electronic health record (EHR)-based pathways. Compliance with ERAS in real-world settings has been problematic. OBJECTIVE: This article evaluates a novel ERAS electronic pathway (E-Pathway) activity integrated with the EHR for patients undergoing elective colorectal surgery. METHODS: We performed a retrospective cohort study of surgical patients age ≥ 18 years hospitalized from March 1, 2013 to August 31, 2016. The primary cohort consisted of patients admitted for elective colon surgery. We also studied a control group of patients undergoing other elective procedures. The E-Pathway was implemented on March 2, 2015. The primary outcome was variable costs per case. Secondary outcomes were observed to expected length of stay and 30-day readmissions. RESULTS: We included 823 (470 and 353 in the pre- and postintervention, respectively) colon surgery patients and 3,415 (1,819 and 1,596 in the pre- and postintervention) surgical control patients in the study. Among the colon surgery cohort, there was statistically significant (p = 0.040) decrease in costs of 1.28% (95% confidence interval [CI] 0.06-2.48%) per surgical encounter per month over the 18-month postintervention period, amounting to a total savings of $2,730 per patient at the 1-year postintervention period. The surgical control group had a nonsignificant (p = 0.231) decrease in monthly costs of 0.57% (95% CI 1.51 to - 0.37%) postintervention. For the 30-day readmission rates, there were no statistically significant changes in either cohort. CONCLUSION: Our study is the first to report on the reduced costs after implementation of a novel sophisticated E-Pathway for ERAS. E-Pathways can be a powerful vehicle to support ERAS adoption.

Design and implementation of a clinical decision support tool for primary palliative Care for Emergency Medicine (PRIM-ER).

BACKGROUND: The emergency department is a critical juncture in the trajectory of care of patients with serious, life-limiting illness. Implementation of a clinical decision support (CDS) tool automates identification of older adults who may benefit from palliative care instead of relying upon providers to identify such patients, thus improving quality of care by assisting providers with adhering to guidelines. The Primary Palliative Care for Emergency Medicine (PRIM-ER) study aims to optimize the use of the electronic health record by creating a CDS tool to identify high risk patients most likely to benefit from primary palliative care and provide point-of-care clinical recommendations. METHODS: A clinical decision support tool entitled Emergency Department Supportive Care Clinical Decision Support (Support-ED) was developed as part of an institutionally-sponsored value based medicine initiative at the Ronald O. Perelman Department of Emergency Medicine at NYU Langone Health. A multidisciplinary approach was used to develop Support-ED including: a scoping review of ED palliative care screening tools; launch of a workgroup to identify patient screening criteria and appropriate referral services; initial design and usability testing via the standard System Usability Scale questionnaire, education of the ED workforce on the Support-ED background, purpose and use, and; creation of a dashboard for monitoring and feedback. RESULTS: The scoping review identified the Palliative Care and Rapid Emergency Screening (P-CaRES) survey as a validated instrument in which to adapt and apply for the creation of the CDS tool. The multidisciplinary workshops identified two primary objectives of the CDS: to identify patients with indicators of serious life limiting illness, and to assist with referrals to services such as palliative care or social work. Additionally, the iterative design process yielded three specific patient scenarios that trigger a clinical alert to fire, including: 1) when an advance care planning document was present, 2) when a patient had a previous disposition to hospice, and 3) when historical and/or current clinical data points identify a serious life-limiting illness without an advance care planning document present. Monitoring and feedback indicated a need for several modifications to improve CDS functionality. CONCLUSIONS: CDS can be an effective tool in the implementation of primary palliative care quality improvement best practices. Health systems should thoughtfully consider tailoring their CDSs in order to adapt to their unique workflows and environments. The findings of this research can assist health systems in effectively integrating a primary palliative care CDS system seamlessly into their processes of care. TRIAL REGISTRATION: ClinicalTrials.gov Identifier: NCT03424109. Registered 6 February 2018, Grant Number: AT009844-01.

Interrupting providers with clinical decision support to improve care for heart failure.

BACKGROUND: Evidence-based therapy for heart failure remains underutilized at hospital discharge, particularly for patients with heart failure who are hospitalized for another cause. We developed clinical decision support (CDS) to recommend an angiotensin converting enzyme (ACE) inhibitor during hospitalization to promote its continuation at discharge. The CDS was designed to be implemented in both interruptive and non-interruptive versions. OBJECTIVES: To compare the effectiveness and implementation of interruptive and non-interruptive versions of a CDS to improve care for heart failure. METHODS: Hospitalizations of patients with reduced ejection fraction were pseudo-randomized to deliver interruptive or non-interruptive CDS alerts to providers based on even or odd medical record number. We compared discharge utilization of an ACE inhibitor or angiotensin receptor blocker (ARB) for these two implementation approaches. We also assessed adoption and implementation fidelity of the CDS. RESULTS: Of 958 hospitalizations, interruptive alert hospitalizations had higher rates of discharge utilization of ACE inhibitors or ARBs than non-interruptive alert hospitalizations (79.6% vs. 74.2%, p = 0.05). Utilization was higher for interruptive alert versus non-interruptive alert hospitalizations which were principally for causes other than heart failure (79.8% vs. 73.4%; p = 0.05) but no difference was observed among hospitalizations with a principal heart failure diagnosis (85.9% vs.81.7%; p = 0.49). As compared to non-interruptive hospitalizations, interruptive alert hospitalizations were more likely to have had: an alert with any response (40.6% vs. 13.1%, p < 0.001), contraindications reported (33.1% vs 11.3%, p < 0.001), and an ACE inhibitor ordered within twelve hours of the alert (17.6% vs 10.3%, p < 0.01). The response rate for the interruptive alert was 1.7%, and a median (25th, 75th percentile) of 14 (5,32) alerts were triggered per hospitalization. CONCLUSIONS: A CDS implemented as an interruptive alert was associated with improved quality of care for heart failure. Whether the potential benefits of CDS in improving cardiovascular care were worth the high burden of interruptive alerts deserves further consideration. CLINICALTRIALS. GOV IDENTIFIER: NCT02858674.

Interruptive Versus Noninterruptive Clinical Decision Support: Usability Study.

BACKGROUND: Clinical decision support (CDS) has been shown to improve compliance with evidence-based care, but its impact is often diminished because of issues such as poor usability, insufficient integration into workflow, and alert fatigue. Noninterruptive CDS may be less subject to alert fatigue, but there has been little assessment of its usability. OBJECTIVE: This study aimed to study the usability of interruptive and noninterruptive versions of a CDS. METHODS: We conducted a usability study of a CDS tool that recommended prescribing an angiotensin-converting enzyme inhibitor for inpatients with heart failure. We developed 2 versions of the CDS: an interruptive alert triggered at order entry and a noninterruptive alert listed in the sidebar of the electronic health record screen. Inpatient providers were recruited and randomly assigned to use the interruptive alert followed by the noninterruptive alert or vice versa in a laboratory setting. We asked providers to "think aloud" while using the CDS and then conducted a brief semistructured interview about usability. We used a constant comparative analysis informed by the CDS Five Rights framework to analyze usability testing. RESULTS: A total of 12 providers participated in usability testing. Providers noted that the interruptive alert was readily noticed but generally impeded workflow. The noninterruptive alert was felt to be less annoying but had lower visibility, which might reduce engagement. Provider role seemed to influence preferences; for instance, some providers who had more global responsibility for patients seemed to prefer the noninterruptive alert, whereas more task-oriented providers generally preferred the interruptive alert. CONCLUSIONS: Providers expressed trade-offs between impeding workflow and improving visibility with interruptive and noninterruptive versions of a CDS. In addition, 2 potential approaches to effective CDS may include targeting alerts by provider role or supplementing a noninterruptive alert with an occasional, well-timed interruptive alert.

Impact of an emergency department electronic sepsis surveillance system on patient mortality and length of stay.

Objective: The purpose of this study was to determine whether an electronic health record-based sepsis alert system could improve quality of care and clinical outcomes for patients with sepsis. Materials and Methods: We performed a patient-level interrupted time series study of emergency department patients with severe sepsis or septic shock between January 2013 and April 2015. The intervention, introduced in February 2014, was a system of interruptive sepsis alerts triggered by abnormal vital signs or laboratory results. Primary outcomes were length of stay (LOS) and in-hospital mortality; other outcomes included time to first lactate and blood cultures prior to antibiotics. We also assessed sensitivity, positive predictive value (PPV), and clinician response to the alerts. Results: Mean LOS for patients with sepsis decreased from 10.1 to 8.6 days (P < .001) following alert introduction. In adjusted time series analysis, the intervention was associated with a decreased LOS of 16% (95% CI, 5%-25%; P = .007, with significance of α = 0.006) and no change thereafter (0%; 95% CI, -2%, 2%). The sepsis alert system had no effect on mortality or other clinical or process measures. The intervention had a sensitivity of 80.4% and a PPV of 14.6%. Discussion: Alerting based on simple laboratory and vital sign criteria was insufficient to improve sepsis outcomes. Alert fatigue due to the low PPV is likely the primary contributor to these results. Conclusion: A more sophisticated algorithm for sepsis identification is needed to improve outcomes.

Impact of an Intervention to Improve Weekend Hospital Care at an Academic Medical Center: An Observational Study.

BACKGROUND: Hospital care on weekends has been associated with delays in care, reduced quality, and poor clinical outcomes. OBJECTIVE: The purpose of this study was to evaluate the impact of a weekend hospital intervention on processes of care and clinical outcomes. The multifaceted intervention included expanded weekend diagnostic services, improved weekend discharge processes, and increased physician and care management services on weekends. DESIGN AND PATIENTS: This was an interrupted time series observational study of adult non-obstetric patients hospitalized at a single academic medical center between January 2011 and January 2014. The study included 18 months prior to and 19 months following the implementation of the intervention. Data were analyzed using segmented regression analysis with adjustment for confounders. MAIN MEASURES: The primary outcome was average length of stay. Secondary outcomes included percent of patients discharged on weekends, 30-day readmission rate, and in-hospital mortality rate. KEY RESULTS: The study included 57,163 hospitalizations. Following implementation of the intervention, average length of stay decreased by 13 % (95 % CI 10-15 %) and continued to decrease by 1 % (95 % CI 1-2 %) per month as compared to the underlying time trend. The proportion of weekend discharges increased by 12 % (95 % CI 2-22 %) at the time of the intervention and continued to increase by 2 % (95 % CI 1-3 %) per month thereafter. The intervention had no impact on readmissions or mortality. During the post-implementation period, the hospital was evacuated and closed for 2 months due to damage from Hurricane Sandy, and a new hospital-wide electronic health record was introduced. The contributions of these events to our findings are not known. We observed a lower inpatient census and found differences in patient characteristics, including higher rates of Medicaid insurance and comorbidities, in the post-Hurricane Sandy period as compared to the pre-Sandy period. CONCLUSIONS: The intervention was associated with a reduction in length of stay and an increase in weekend discharges. Our longitudinal study also illuminated the challenges of evaluating the effectiveness of a large-scale intervention in a real-world hospital setting.

Electronic health record use, intensity of hospital care, and patient outcomes.

OBJECTIVE: Previous studies have suggested that weekend hospital care is inferior to weekday care and that this difference may be related to diminished care intensity. The purpose of this study was to determine whether a metric for measuring intensity of hospital care based on use of the electronic health record was associated with patient-level outcomes. METHODS: We performed a cohort study of hospitalizations at an academic medical center. Intensity of care was defined as the hourly number of provider accessions of the electronic health record, termed "electronic health record interactions." Hospitalizations were categorized on the basis of the mean difference in electronic health record interactions between the first Friday and the first Saturday of hospitalization. We used regression models to determine the association of these categories with patient outcomes after adjusting for covariates. RESULTS: Electronic health record interactions decreased from Friday to Saturday in 77% of the 9051 hospitalizations included in the study. Compared with hospitalizations with no change in Friday to Saturday electronic health record interactions, the relative lengths of stay for hospitalizations with a small, moderate, and large decrease in electronic health record interactions were 1.05 (95% confidence interval [CI], 1.00-1.10), 1.11 (95% CI, 1.05-1.17), and 1.25 (95% CI, 1.15-1.35), respectively. Although a large decrease in electronic health record interactions was associated with in-hospital mortality, these findings were not significant after risk adjustment (odds ratio 1.74, 95% CI, 0.93-3.25). CONCLUSIONS: Intensity of inpatient care, measured by electronic health record interactions, significantly diminished from Friday to Saturday, and this decrease was associated with length of stay. Hospitals should consider monitoring and correcting temporal fluctuations in care intensity.

Monitoring the pulse of hospital activity: electronic health record utilization as a measure of care intensity.

BACKGROUND: Hospital care on weekends has been associated with reduced quality and poor clinical outcomes, suggesting that decreases in overall intensity of care may have important clinical effects. We describe a new measure of hospital intensity of care based on utilization of the electronic health record (EHR). METHODS: We measured global intensity of care at our academic medical center by monitoring the use of the EHR in 2011. Our primary measure, termed EHR interactions, was the number of accessions of a patient's electronic record by a clinician, adjusted for hospital census, per unit of time. Our secondary measure was percent of total available central processing unit (CPU) power used to access EHR servers at a given time. RESULTS: EHR interactions were lower on weekend days as compared to weekdays at every hour (P < 0.0001), and the daytime peak in intensity noted each weekday was blunted on weekends. The relative rate and 95% confidence interval (CI) of census-adjusted record accessions per patient on weekdays compared with weekends were: 1.76 (95% CI: 1.74-1.77), 1.52 (95% CI: 1.50-1.55), and 1.14 (95% CI: 1.12-1.17) for day, morning/evening, and night hours, respectively. Percent CPU usage correlated closely with EHR interactions (r = 0.90). CONCLUSIONS: EHR usage is a valid and easily reproducible measure of intensity of care in the hospital. Using this measure we identified large, hour-specific differences between weekend and weekday intensity. EHR interactions may serve as a useful measure for tracking and improving temporal variations in care that are common, and potentially deleterious, in hospital systems.

The impact of the heparin-induced thrombocytopenia (HIT) computerized alert on provider behaviors and patient outcomes.

OBJECTIVE: The aim of this study was to measure the effect of an electronic heparin-induced thrombocytopenia (HIT) alert on provider ordering behaviors and on patient outcomes. MATERIALS AND METHODS: A pop-up alert was created for providers when an individual's platelet values had decreased by 50% or to <100,000/mm(3) in the setting of recent heparin exposure. The authors retrospectively compared inpatients admitted between January 24, 2008 and August 24, 2008 to a control group admitted 1 year prior to the HIT alert. The primary outcome was a change in HIT antibody testing. Secondary outcomes included an assessment of incidence of HIT antibody positivity, percentage of patients started on a direct thrombin inhibitor (DTI), length of stay and overall mortality. RESULTS: There were 1006 and 1081 patients in the control and intervention groups, respectively. There was a 33% relative increase in HIT antibody test orders (p=0.01), and 33% more of these tests were ordered the first day after the criteria were met when a pop-up alert was given (p=0.03). Heparin was discontinued in 25% more patients in the alerted group (p=0.01), and more direct thrombin inhibitors were ordered for them (p=0.03). The number who tested HIT antibody-positive did not differ, however, between the two groups (p=0.99). The length of stay and mortality were similar in both groups. CONCLUSIONS: The HIT alert significantly impacted provider behaviors. However, the alert did not result in more cases of HIT being detected or an improvement in overall mortality. Our findings do not support implementation of a computerized HIT alert.

Perceived barriers to trying self-management approaches for chronic pain in older persons.

OBJECTIVES: To examine older persons' willingness to participate in exercise and relaxation programs for managing chronic pain, to identify characteristics associated with willingness to participate, and to ascertain their barriers to participation. DESIGN: Cross-sectional survey. SETTING: A geriatric ambulatory care practice located in New York, New York, and the General Clinical Research Center of Weill Cornell Medical College. PARTICIPANTS: Sixty-eight English-speaking patients aged 70 and older with chronic pain. MEASUREMENTS: Participants' level of willingness to participate in the programs was assessed using 5-point Likert scales, and information regarding their demographic, clinical, psychological, and pain status was obtained. Qualitative methods were used to ascertain participants' perceived barriers to participation. RESULTS: Participants had a mean age+/-standard deviation of 81.9+/-7.0 and were mostly white (85%) and female (68%). Although only 16% of participants reported current use of exercise as a pain-management strategy, 73% reported a willingness to try the exercise program. Four percent reported current use of relaxation methods; 70% reported an interest in learning these techniques. Moderate correlations were found between increasing days of restricted activity due to pain and greater willingness to participate. Seventeen unique barriers were identified; the mean number of barriers reported per person was 2.9+/-1.8 for exercise and 2.2+/-1.4 for relaxation. Commonly reported barriers to participating in either program included time conflicts, transportation, treatment efficacy concerns, and fear of pain or injury. CONCLUSION: Older primary care patients with chronic pain are willing to try exercise and relaxation therapies to help manage pain but report a substantial number of barriers to participating in these programs. Addressing patient-level barriers could improve engagement in and adherence to exercise and relaxation therapies for managing pain in older persons.