Effect of an Electronic Health Record-Based Intervention on Documentation Practices.

Please see title page and main document for latest version of abstract.

Artificial Intelligence-Generated Draft Replies to Patient Inbox Messages.

Importance: The emergence and promise of generative artificial intelligence (AI) represent a turning point for health care. Rigorous evaluation of generative AI deployment in clinical practice is needed to inform strategic decision-making. Objective: To evaluate the implementation of a large language model used to draft responses to patient messages in the electronic inbox. Design, Setting, and Participants: A 5-week, prospective, single-group quality improvement study was conducted from July 10 through August 13, 2023, at a single academic medical center (Stanford Health Care). All attending physicians, advanced practice practitioners, clinic nurses, and clinical pharmacists from the Divisions of Primary Care and Gastroenterology and Hepatology were enrolled in the pilot. Intervention: Draft replies to patient portal messages generated by a Health Insurance Portability and Accountability Act-compliant electronic health record-integrated large language model. Main Outcomes and Measures: The primary outcome was AI-generated draft reply utilization as a percentage of total patient message replies. Secondary outcomes included changes in time measures and clinician experience as assessed by survey. Results: A total of 197 clinicians were enrolled in the pilot; 35 clinicians who were prepilot beta users, out of office, or not tied to a specific ambulatory clinic were excluded, leaving 162 clinicians included in the analysis. The survey analysis cohort consisted of 73 participants (45.1%) who completed both the presurvey and postsurvey. In gastroenterology and hepatology, there were 58 physicians and APPs and 10 nurses. In primary care, there were 83 physicians and APPs, 4 nurses, and 8 clinical pharmacists. The mean AI-generated draft response utilization rate across clinicians was 20%. There was no change in reply action time, write time, or read time between the prepilot and pilot periods. There were statistically significant reductions in the 4-item physician task load score derivative (mean [SD], 61.31 [17.23] presurvey vs 47.26 [17.11] postsurvey; paired difference, -13.87; 95% CI, -17.38 to -9.50; P < .001) and work exhaustion scores (mean [SD], 1.95 [0.79] presurvey vs 1.62 [0.68] postsurvey; paired difference, -0.33; 95% CI, -0.50 to -0.17; P < .001). Conclusions and Relevance: In this quality improvement study of an early implementation of generative AI, there was notable adoption, usability, and improvement in assessments of burden and burnout. There was no improvement in time. Further code-to-bedside testing is needed to guide future development and organizational strategy.

Building Pandemic-Resilient Primary Care Systems: Lessons Learned From COVID-19.

On January 30, 2023, the Biden Administration announced its intention to end the existing COVID-19 public health emergency declaration. The transition to a "postpandemic" landscape presents a unique opportunity to sustain and strengthen pandemic-era changes in care delivery. With this in mind, we present 3 critical lessons learned from a primary care perspective during the COVID-19 pandemic. First, clinical workflows must support both in-person and internet-based care delivery. Second, the integration of asynchronous care delivery is critical. Third, planning for the future means planning for everyone, including those with potentially limited access to health care due to barriers in technology and communication. While these lessons are neither unique to primary care settings nor all-encompassing, they establish a grounded foundation on which to construct higher-quality, more resilient, and more equitable health systems.

Voice Assistants' Responses to Questions About the COVID-19 Vaccine: National Cross-sectional Study.

BACKGROUND: Artificial intelligence-powered voice assistants (VAs), such as Apple Siri, Google Assistant, and Amazon Alexa, interact with users in natural language and are capable of responding to simple commands, searching the internet, and answering questions. Despite being an increasingly popular way for the public to access health information, VAs could be a source of ambiguous or potentially biased information. OBJECTIVE: In response to the ongoing prevalence of vaccine misinformation and disinformation, this study aims to evaluate how smartphone VAs respond to information- and recommendation-seeking inquiries regarding the COVID-19 vaccine. METHODS: A national cross-sectional survey of English-speaking adults who owned a smartphone with a VA installed was conducted online from April 22 to 28, 2021. The primary outcomes were the VAs' responses to 2 questions: "Should I get the COVID vaccine?" and "Is the COVID vaccine safe?" Directed content analysis was used to assign a negative, neutral, or positive connotation to each response and website title provided by the VAs. Statistical significance was assessed using the t test (parametric) or Mann-Whitney U (nonparametric) test for continuous variables and the chi-square or Fisher exact test for categorical variables. RESULTS: Of the 466 survey respondents included in the final analysis, 404 (86.7%) used Apple Siri, 53 (11.4%) used Google Assistant, and 9 (1.9%) used Amazon Alexa. In response to the question "Is the COVID vaccine safe?" 419 (89.9%) users received a direct response, of which 408 (97.3%) had a positive connotation encouraging users to get vaccinated. Of the websites presented, only 5.3% (11/207) had a positive connotation and 94.7% (196/207) had a neutral connotation. In response to the question "Should I get the COVID vaccine?" 93.1% (434/466) of users received a list of websites, of which 91.5% (1155/1262) had a neutral connotation. For both COVID-19 vaccine-related questions, there was no association between the connotation of a response and the age, gender, zip code, race or ethnicity, and education level of the respondent. CONCLUSIONS: Our study found that VAs were much more likely to respond directly with positive connotations to the question "Is the COVID vaccine safe?" but not respond directly and provide a list of websites with neutral connotations to the question "Should I get the COVID vaccine?" To our knowledge, this is the first study to evaluate how VAs respond to both information- and recommendation-seeking inquiries regarding the COVID-19 vaccine. These findings add to our growing understanding of both the opportunities and pitfalls of VAs in supporting public health information dissemination.