Effect of an Electronic Health Record-Based Intervention on Documentation Practices.

BACKGROUND: Documentation burden is one of the largest contributors to physician burnout. Evaluation and Management (E&M) coding changes were implemented in 2021 to alleviate documentation burden. OBJECTIVES: We used this opportunity to develop documentation best practices, implement new electronic health record (EHR) tools, and study the potential impact on provider experiences with documentation related to these 2021 E&M changes, documentation length, and time spent documenting at an academic medical center. METHODS: Five actionable best practices, developed through a consensus-driven, multidisciplinary approach in November 2020, led to the creation of two new ambulatory note templates, one for E&M visits (implemented in January 2021) and another for preventative visits (implemented in May 2021). As part of a quality-improvement initiative at nine faculty primary care clinics, surveys were developed utilizing a 5-point Likert scale to assess provider perceptions and deidentified EHR metadata (Signal, Epic Systems) were analyzed to measure changes in EHR use metrics between a pre-E&M changes timeframe (August 2020-December 2020) and a post-E&M change timeframe (August 2021-December 2021). A subgroup analysis was conducted comparing EHR use metrics among note template utilizers versus nonutilizers. Any provider who used one of the note templates at least once was categorized as a utilizer. RESULTS: Between January 2021 and December 2021, the adoption of the E&M visit template was 31,480 instances among 120 unique ambulatory providers, and adoption of the preventative visit template was 1,464 instances among 22 unique ambulatory providers. Survey response rate among faculty primary care providers was 82% (88/107): 55% (48/88) believed the 2021 E&M changes provided an opportunity to reduce documentation burden, and 28% reported favorable satisfaction with time spent documenting. Among providers who reported using one or both of the new note templates, 81% (35/43) of survey respondents reported favorable satisfaction with new note templates. EHR use metric analyses revealed a small, yet significant reduction in time in notes per appointment (p = 0.004) with no significant change in documentation length of notes (p = 0.45). Note template utilization was associated with a statistically significant reduction in documentation length (p = 0.034). CONCLUSION: This study shows modest progress in improving EHR use measures of documentation length and time spent documenting following the 2021 E&M changes, but without great improvement in perceived documentation burden. Additional tools are needed to reduce documentation burden and further research is needed to understand the impact of these interventions.

Artificial Intelligence-Generated Draft Replies to Patient Inbox Messages.

Importance: The emergence and promise of generative artificial intelligence (AI) represent a turning point for health care. Rigorous evaluation of generative AI deployment in clinical practice is needed to inform strategic decision-making. Objective: To evaluate the implementation of a large language model used to draft responses to patient messages in the electronic inbox. Design, Setting, and Participants: A 5-week, prospective, single-group quality improvement study was conducted from July 10 through August 13, 2023, at a single academic medical center (Stanford Health Care). All attending physicians, advanced practice practitioners, clinic nurses, and clinical pharmacists from the Divisions of Primary Care and Gastroenterology and Hepatology were enrolled in the pilot. Intervention: Draft replies to patient portal messages generated by a Health Insurance Portability and Accountability Act-compliant electronic health record-integrated large language model. Main Outcomes and Measures: The primary outcome was AI-generated draft reply utilization as a percentage of total patient message replies. Secondary outcomes included changes in time measures and clinician experience as assessed by survey. Results: A total of 197 clinicians were enrolled in the pilot; 35 clinicians who were prepilot beta users, out of office, or not tied to a specific ambulatory clinic were excluded, leaving 162 clinicians included in the analysis. The survey analysis cohort consisted of 73 participants (45.1%) who completed both the presurvey and postsurvey. In gastroenterology and hepatology, there were 58 physicians and APPs and 10 nurses. In primary care, there were 83 physicians and APPs, 4 nurses, and 8 clinical pharmacists. The mean AI-generated draft response utilization rate across clinicians was 20%. There was no change in reply action time, write time, or read time between the prepilot and pilot periods. There were statistically significant reductions in the 4-item physician task load score derivative (mean [SD], 61.31 [17.23] presurvey vs 47.26 [17.11] postsurvey; paired difference, -13.87; 95% CI, -17.38 to -9.50; P < .001) and work exhaustion scores (mean [SD], 1.95 [0.79] presurvey vs 1.62 [0.68] postsurvey; paired difference, -0.33; 95% CI, -0.50 to -0.17; P < .001). Conclusions and Relevance: In this quality improvement study of an early implementation of generative AI, there was notable adoption, usability, and improvement in assessments of burden and burnout. There was no improvement in time. Further code-to-bedside testing is needed to guide future development and organizational strategy.

Building Pandemic-Resilient Primary Care Systems: Lessons Learned From COVID-19.

On January 30, 2023, the Biden Administration announced its intention to end the existing COVID-19 public health emergency declaration. The transition to a "postpandemic" landscape presents a unique opportunity to sustain and strengthen pandemic-era changes in care delivery. With this in mind, we present 3 critical lessons learned from a primary care perspective during the COVID-19 pandemic. First, clinical workflows must support both in-person and internet-based care delivery. Second, the integration of asynchronous care delivery is critical. Third, planning for the future means planning for everyone, including those with potentially limited access to health care due to barriers in technology and communication. While these lessons are neither unique to primary care settings nor all-encompassing, they establish a grounded foundation on which to construct higher-quality, more resilient, and more equitable health systems.

Voice Assistants' Responses to Questions About the COVID-19 Vaccine: National Cross-sectional Study.

BACKGROUND: Artificial intelligence-powered voice assistants (VAs), such as Apple Siri, Google Assistant, and Amazon Alexa, interact with users in natural language and are capable of responding to simple commands, searching the internet, and answering questions. Despite being an increasingly popular way for the public to access health information, VAs could be a source of ambiguous or potentially biased information. OBJECTIVE: In response to the ongoing prevalence of vaccine misinformation and disinformation, this study aims to evaluate how smartphone VAs respond to information- and recommendation-seeking inquiries regarding the COVID-19 vaccine. METHODS: A national cross-sectional survey of English-speaking adults who owned a smartphone with a VA installed was conducted online from April 22 to 28, 2021. The primary outcomes were the VAs' responses to 2 questions: "Should I get the COVID vaccine?" and "Is the COVID vaccine safe?" Directed content analysis was used to assign a negative, neutral, or positive connotation to each response and website title provided by the VAs. Statistical significance was assessed using the t test (parametric) or Mann-Whitney U (nonparametric) test for continuous variables and the chi-square or Fisher exact test for categorical variables. RESULTS: Of the 466 survey respondents included in the final analysis, 404 (86.7%) used Apple Siri, 53 (11.4%) used Google Assistant, and 9 (1.9%) used Amazon Alexa. In response to the question "Is the COVID vaccine safe?" 419 (89.9%) users received a direct response, of which 408 (97.3%) had a positive connotation encouraging users to get vaccinated. Of the websites presented, only 5.3% (11/207) had a positive connotation and 94.7% (196/207) had a neutral connotation. In response to the question "Should I get the COVID vaccine?" 93.1% (434/466) of users received a list of websites, of which 91.5% (1155/1262) had a neutral connotation. For both COVID-19 vaccine-related questions, there was no association between the connotation of a response and the age, gender, zip code, race or ethnicity, and education level of the respondent. CONCLUSIONS: Our study found that VAs were much more likely to respond directly with positive connotations to the question "Is the COVID vaccine safe?" but not respond directly and provide a list of websites with neutral connotations to the question "Should I get the COVID vaccine?" To our knowledge, this is the first study to evaluate how VAs respond to both information- and recommendation-seeking inquiries regarding the COVID-19 vaccine. These findings add to our growing understanding of both the opportunities and pitfalls of VAs in supporting public health information dissemination.