A qualitative study of stakeholders' experiences with and acceptability of a technology-supported health coaching intervention (SHARE-S) delivered in coordination with cancer survivorship care.

PURPOSE: Healthy cancer survivorship involves patients' active engagement with preventative health behaviors and follow-up care. While clinicians and patients have typically held dual responsibility for activating these behaviors, transitioning some clinician effort to technology and health coaches may enhance guideline implementation. This paper reports on the acceptability of the Shared Healthcare Actions & Reflections Electronic systems in survivorship (SHARE-S) program, an entirely virtual multicomponent intervention incorporating e-referrals, remotely-delivered health coaching, and automated text messages to enhance patient self-management and promote healthy survivorship. METHODS: SHARE-S was evaluated in single group hybrid implementation-effectiveness pilot study. Patients were e-referred from the clinical team to health coaches for three health self-management coaching calls and received text messages to enhance coaching. Semi-structured qualitative interviews were conducted with 21 patient participants, 2 referring clinicians, and 2 health coaches to determine intervention acceptability (attitudes, appropriateness, suitability, convenience, and perceived effectiveness) and to identify important elements of the program and potential mechanisms of action to guide future implementation. RESULTS: SHARE-S was described as impactful and convenient. The nondirective, patient-centered health coaching and mindfulness exercises were deemed most acceptable; text messages were less acceptable. Stakeholders suggested increased flexibility in format, frequency, timing, and length of participation, and additional tailored educational materials. Patients reported tangible health behavior changes, improved mood, and increased accountability and self-efficacy. CONCLUSIONS: SHARE-S is overall an acceptable and potentially effective intervention that may enhance survivors' self-management and well-being. Alterations to tailored content, timing, and dose should be tested to determine impact on acceptability and outcomes.

"For the first time I am seriously fighting burnout": clinician experiences with a challenging electronic health record transition.

Objectives: The Department of Veterans Affairs (VA) is transitioning from its legacy electronic health record (EHR) to a new commercial EHR in a nationwide, rolling-wave transition. We evaluated clinician and staff experiences to identify strategies to improve future EHR rollouts. Materials and Methods: We completed a convergent mixed-methods formative evaluation collecting survey and interview data to measure and describe clinician and staff experiences. Survey responses were analyzed using descriptive statistics; interview transcripts were coded using a combination of a priori and emergent codes followed by qualitative content analysis. Qualitative and quantitative findings were compared to provide a more comprehensive understanding of participant experience. Employees of specialty and primary care teams at the first nationwide EHR transition site agreed to participate in our study. We distributed surveys at 1-month pre-transition, 2 months post-transition, and 10 months post-transition to each of the 68 identified team members and completed longitudinal interviews with 30 of these individuals totaling 122 semi-structured interviews. Results: Interview participants reported profoundly disruptive experiences during the EHR transition that persisted at 1-year post implementation. Survey responses indicated training difficulties throughout the transition, and sharp declines (P ≤ .05) between pre- and post-go-live measures of EHR usability and increase in EHR burden that were perceived to be due in part to system inefficiencies, discordant positive messaging that initially ignored user challenges, and inadequate support for and attention to ongoing EHR issues. Participants described persistent high levels of stress associated with these disruptions. Discussion: Our findings highlight strategies to improve employee experiences during EHR transitions: (1) working with Oracle Cerner to resolve known issues and improve usability; (2) role-based training with opportunities for self-directed learning; (3) peer-led support systems and timely feedback on issues; (4) messaging that responds to challenges and successes; and (5) continuous efforts to support staff with issues and address clinician and staff stress and burnout. Conclusion: Our findings provide relevant strategies to navigate future EHR transitions while supporting clinical teams.

Understanding pathways from implementation to sustainment: a longitudinal, mixed methods analysis of promising practices implemented in the Veterans Health Administration.

BACKGROUND: The Veterans Health Administration (VHA) is the United States largest learning health system. The Diffusion of Excellence (DoE) program is a large-scale model of diffusion that identifies and diffuses evidence-informed practices across VHA. During the period of 2016-2021, 57 evidence-informed practices were implemented across 82 VHA facilities. This setting provides a unique opportunity to understand sustainment determinants and pathways. Our objective was to characterize the longitudinal pathways of practices as they transition from initial implementation to long-term sustainment at each facility. METHODS: A longitudinal, mixed-methods evaluation of 82 VHA facilities. Eighty-two facility representatives, chosen by leadership as points-of-contact for 57 DoE practices, were eligible for post-implementation interviews and annual sustainment surveys. Primary outcomes (implementation, sustainment), and secondary outcomes (institutionalization, effectiveness, anticipated sustainment) at four time-points were collected. We performed descriptive statistics and directed content analysis using Hailemariam et al.'s factors influencing sustainment. RESULTS: After approximately five years post-implementation (e.g., 2021 sustainment outcomes), of the 82 facilities, about one-third fully sustained their practice compared to one-third that did not fully sustain their practice because it was in a "liminal" stage (neither sustained nor discontinued) or permanently discontinued. The remaining one-third of facilities had missing 2021 sustainment outcomes. A higher percentage of facilities (70%) had inconsistent primary outcomes (changing over time) compared to facilities (30%) with consistent primary outcomes (same over time). Thirty-four percent of facilities with sustained practices reported resilience since they overcame implementation and sustainment barriers. Facilities with sustained practices reported more positive secondary outcomes compared to those that did not sustain their practice. Key factors facilitating practice sustainment included: demonstrating practice effectiveness/benefit, sufficient organizational leadership, sufficient workforce, and adaptation/alignment with local context. Key factors hindering practice sustainment included: insufficient workforce, not able to maintain practice fidelity/integrity, critical incidents related to the COVID-19 pandemic, organizational leadership did not support sustainment of practice, and no ongoing support. CONCLUSIONS: We identified diverse pathways from implementation to sustainment, and our data underscore that initial implementation outcomes may not determine long-term sustainment outcomes. This longitudinal evaluation contributes to understanding impacts of the DoE program, including return on investment, achieving learning health system goals, and insights into achieving high-quality healthcare in VHA.

Clinician and staff experiences with frustrated patients during an electronic health record transition: a qualitative case study.

BACKGROUND: Electronic health record (EHR) transitions are known to be highly disruptive, can drastically impact clinician and staff experiences, and may influence patients' experiences using the electronic patient portal. Clinicians and staff can gain insights into patient experiences and be influenced by what they see and hear from patients. Through the lens of an emergency preparedness framework, we examined clinician and staff reactions to and perceptions of their patients' experiences with the portal during an EHR transition at the Department of Veterans Affairs (VA). METHODS: This qualitative case study was situated within a larger multi-methods evaluation of the EHR transition. We conducted a total of 122 interviews with 30 clinicians and staff across disciplines at the initial VA EHR transition site before, immediately after, and up to 12 months after go-live (September 2020-November 2021). Interview transcripts were coded using a priori and emergent codes. The coded text segments relevant to patient experience and clinician interactions with patients were extracted and analyzed to identify themes. For each theme, recommendations were defined based on each stage of an emergency preparedness framework (mitigate, prepare, respond, recover). RESULTS: In post-go-live interviews participants expressed concerns about the reliability of communicating with their patients via secure messaging within the new EHR portal. Participants felt ill-equipped to field patients' questions and frustrations navigating the new portal. Participants learned that patients experienced difficulties learning to use and accessing the portal; when unsuccessful, some had difficulties obtaining medication refills via the portal and used the call center as an alternative. However, long telephone wait times provoked patients to walk into the clinic for care, often frustrated and without an appointment. Patients needing increased in-person attention heightened participants' daily workload and their concern for patients' well-being. Recommendations for each theme fit within a stage of the emergency preparedness framework. CONCLUSIONS: Application of an emergency preparedness framework to EHR transitions could help address the concerns raised by the participants, (1) mitigating disruptions by identifying at-risk patients before the transition, (2) preparing end-users by disseminating patient-centered informational resources, (3) responding by building capacity for disrupted services, and (4) recovering by monitoring integrity of the new portal function.

Longitudinal Monitoring of Clinician-Patient Video Visits During the Peak of the COVID-19 Pandemic: Adoption and Sustained Challenges in an Integrated Health Care Delivery System.

BACKGROUND: Numerous prior opinion papers, administrative electronic health record data studies, and cross-sectional surveys of telehealth during the pandemic have been published, but none have combined assessments of video visit success monitoring with longitudinal assessments of perceived challenges to the rapid adoption of video visits during the pandemic. OBJECTIVE: This study aims to quantify (1) the use of video visits (compared with in-person and telephone visits) over time during the pandemic, (2) video visit successful connection rates, and (3) changes in perceived video visit challenges. METHODS: A web-based survey was developed for the dual purpose of monitoring and improving video visit implementation in our health care system during the COVID-19 pandemic. The survey included questions regarding rates of in-person, telephone, and video visits for clinician-patient encounters; the rate of successful connection for video visits; and perceived challenges to video visits (eg, software, hardware, bandwidth, and technology literacy). The survey was distributed via email to physicians, advanced practice professionals, and clinicians in May 2020. The survey was repeated in March 2021. Differences between the 2020 and 2021 responses were adjusted for within-respondent correlation across surveys and tested using generalized estimating equations. RESULTS: A total of 1126 surveys were completed (511 surveys in 2020 and 615 surveys in 2021). In 2020, only 21.7% (73/336) of clinicians reported no difficulty connecting with patients during video visits and 28.6% (93/325) of clinicians reported no difficulty in 2021. The distribution of the percentage of successfully connected video visits ("Over the past two weeks of scheduled visits, what percentage did you successfully connect with patients by video?") was not significantly different between 2020 and 2021 (P=.74). Challenges in conducting video visits persisted over time. Poor connectivity was the most common challenge reported by clinicians. This response increased over time, with 30.5% (156/511) selecting it as a challenge in 2020 and 37.1% (228/615) in 2021 (P=.01). Patients not having access to their electronic health record portals was also a commonly reported challenge (109/511, 21.3% in 2020 and 137/615, 22.3% in 2021, P=.73). CONCLUSIONS: During the pandemic, our health care delivery system rapidly adopted synchronous patient-clinician communication using video visits. As experience with video visits increased, the reported failure rate did not significantly decline, and clinicians continued to report challenges related to general network connectivity and patient access to technology.

Improving Veteran Engagement with Virtual Care Technologies: a Veterans Health Administration State of the Art Conference Research Agenda.

Although the availability of virtual care technologies in the Veterans Health Administration (VHA) continues to expand, ensuring engagement with these technologies among Veterans remains a challenge. VHA Health Services Research & Development convened a Virtual Care State of The Art (SOTA) conference in May 2022 to create a research agenda for improving virtual care access, engagement, and outcomes. This article reports findings from the Virtual Care SOTA engagement workgroup, which comprised fourteen VHA subject matter experts representing VHA clinical care, research, administration, and operations. Workgroup members reviewed current evidence on factors and strategies that may affect Veteran engagement with virtual care technologies and generated key questions to address evidence gaps. The workgroup agreed that although extensive literature exists on factors that affect Veteran engagement, more work is needed to identify effective strategies to increase and sustain engagement. Workgroup members identified key priorities for research on Veteran engagement with virtual care technologies through a series of breakout discussion groups and ranking exercises. The top three priorities were to (1) understand the Veteran journey from active service to VHA enrollment and beyond, and when and how virtual care technologies can best be introduced along that journey to maximize engagement and promote seamless care; (2) utilize the meaningful relationships in a Veteran's life, including family, friends, peers, and other informal or formal caregivers, to support Veteran adoption and sustained use of virtual care technologies; and (3) test promising strategies in meaningful combinations to promote Veteran adoption and/or sustained use of virtual care technologies. Research in these priority areas has the potential to help VHA refine strategies to improve virtual care user engagement, and by extension, outcomes.

Factors Associated with Veteran Self-Reported Use of Digital Health Devices.

BACKGROUND: Digital health devices (DHDs), technologies designed to gather, monitor, and sometimes share data about health-related behaviors or symptoms, can support the prevention or management of chronic conditions. DHDs range in complexity and utility, from tracking lifestyle behaviors (e.g., pedometer) to more sophisticated biometric data collection for disease self-management (e.g., glucometers). Despite these positive health benefits, supporting adoption and sustained use of DHDs remains a challenge. OBJECTIVE: This analysis examined the prevalence of, and factors associated with, DHD use within the Veterans Health Administration (VHA). DESIGN: National survey. PARTICIPANTS: Veterans who receive VHA care and are active secure messaging users. MAIN MEASURES: Demographics, access to technology, perceptions of using health technologies, and use of lifestyle monitoring and self-management DHDs. RESULTS: Among respondents, 87% were current or past users of at least one DHD, and 58% were provided a DHD by VHA. Respondents 65 + years were less likely to use a lifestyle monitoring device (AOR 0.57, 95% CI [0.39, 0.81], P = .002), but more likely to use a self-management device (AOR 1.69, 95% [1.10, 2.59], P = .016). Smartphone owners were more likely to use a lifestyle monitoring device (AOR 2.60, 95% CI [1.42, 4.75], P = .002) and a self-management device (AOR 1.83, 95% CI [1.04, 3.23], P = .037). CONCLUSIONS: The current analysis describes the types of DHDs that are being adopted by Veterans and factors associated with their adoption. Results suggest that various factors influence adoption, including age, access to technology, and health status, and that these relationships may differ based on the functionalities of the device. VHA provision of devices was frequent among device users. Providing Veterans with DHDs and the training needed to use them may be important factors in facilitating device adoption. Taken together, this knowledge can inform future implementation efforts, and next steps to support patient-team decision making about DHD use.

Impacts of an Electronic Health Record Transition on Veterans Health Administration Health Professions Trainee Experience.

BACKGROUND: Adoption of electronic health care records (EHRs) has proliferated since 2000. While EHR transitions are widely understood to be disruptive, little attention has been paid to their effect on health professions trainees' (HPTs) ability to learn and conduct work. Veterans Health Administration's (VA) massive transition from its homegrown EHR (CPRS/Vista) to the commercial Oracle Cerner presents an unparalleled-in-scope opportunity to gain insight on trainee work functions and their ability to obtain requisite experience during transitions. OBJECTIVE: To identify how an organizational EHR transition affected HPT work and learning at the third VA go-live site. DESIGN: A formative mixed-method evaluation of HPT experiences with VHA's EHR transition including interviews with HPTs and supervisors at Chalmers P. Wylie VA Outpatient Clinic in Columbus, OH, before (~60 min), during (15-30 min), and after (~60 min) go-live (December 2021-July 2022). We also conducted pre- (March 2022-April 2022) and post-go live (May 2022-June 2022) HPT and employee surveys. PARTICIPANTS: We conducted 24 interviews with HPTs (n=4), site leaders (n=2), and academic affiliates (n=2) using snowball sampling. We recruited HPTs in pre- (n=13) and post-go-live (n=10) surveys and employees in pre- (n=408) and post-go-live (n=458) surveys. APPROACH: We conducted interviews using a semi-structured guide and grounded prompts. We coded interviews and survey free text data using a priori and emergent codes, subsequently conducting thematic analysis. We conducted descriptive statistical analysis of survey responses and merged interview and survey data streams. KEY RESULTS: Our preliminary findings indicate that the EHR transition comprehensively affected HPT experiences, disrupting processes from onboarding and training to clinical care contributions and training-to-career retention. CONCLUSIONS: Understanding HPTs' challenges during EHR transitions is critical to effective training. Mitigating the identified barriers to HPT training and providing patient care may lessen their dissatisfaction and ensure quality patient care during EHR transitions.

"Everything's so Role-Specific": VA Employee Perspectives' on Electronic Health Record (EHR) Transition Implications for Roles and Responsibilities.

BACKGROUND: Electronic health record (EHR) transitions are increasingly widespread and often highly disruptive. It is imperative we learn from past experiences to anticipate and mitigate such disruptions. Veterans Affairs (VA) is undergoing a large-scale transition from its homegrown EHR (CPRS/Vista) to a commercial EHR (Cerner), creating a unique opportunity of shedding light on large-scale EHR-to-EHR transition challenges. OBJECTIVE: To explore one facet of the organizational impact of VA's EHR transition: its implications for employees' roles and responsibilities at the first VA site to implement Cerner Millennium EHR. DESIGN: As part of a formative evaluation of frontline staff experiences with VA's EHR transition, we conducted brief (~ 15 min) and full-length interviews (~ 60 min) with clinicians and staff at Mann-Grandstaff VA Medical Center in Spokane, WA, before, during, and after transition (July 2020-November 2021). PARTICIPANTS: We conducted 111 interviews with 26 Spokane clinicians and staff, recruited via snowball sampling. APPROACH: We conducted audio interviews using a semi-structured guide with grounded prompts. We coded interview transcripts using a priori and emergent codes, followed by qualitative content analysis. KEY RESULTS: Unlike VA's previous EHR, Cerner imposes additional restrictions on access to its EHR functionality based upon "roles" assigned to users. Participants described a mismatch between established institutional duties and their EHR permissions, unanticipated changes in scope of duties brought upon by the transition, as well as impediments to communication and collaboration due to different role-based views. CONCLUSIONS: Health systems should anticipate substantive impacts on professional workflows when EHR role settings do not reflect prior workflows. Such changes may increase user error, dissatisfaction, and patient care disruptions. To mitigate employee dissatisfaction and safety risks, health systems should proactively plan for and communicate about expected modifications and monitor for unintended role-related consequences of EHR transitions, while vendors should ensure accurate role configuration and assignment.

"Like One Long Battle:" Employee Perspectives of the Simultaneous Impact of COVID-19 and an Electronic Health Record Transition.

BACKGROUND: Healthcare organizations regularly manage external stressors that threaten patient care, but experiences handling concurrent stressors are not well characterized. OBJECTIVE: To evaluate the experience of Veterans Affairs (VA) clinicians and staff navigating simultaneous organizational stressors-an electronic health record (EHR) transition and the COVID-19 pandemic-and identify potential strategies to optimize management of co-occurring stressors. DESIGN: Qualitative case study describing employee experiences at VA's initial EHR transition site. PARTICIPANTS: Clinicians, nurses, allied health professionals, and local leaders at VA's initial EHR transition site. APPROACH: We collected longitudinal qualitative interview data between July 2020 and November 2021 once before and 2-4 times after the date on which the health system transitioned; this timing corresponded with local surges of COVID-19 cases. Interviewers conducted coding and analysis of interview transcripts. For this study, we focused on quotes related to COVID-19 and performed content analysis to describe recurring themes describing the simultaneous impact of COVID-19 and an EHR transition. KEY RESULTS: We identified five themes related to participants' experiences: (1) efforts to mitigate COVID-19 transmission led to insufficient access to EHR training and support, (2) clinical practice changes in response to the pandemic impacted EHR workflows in unexpected ways, (3) lack of clear communication and inconsistent enforcement of COVID-19 policies intensified pre-existing frustrations with the EHR, (4) managing concurrent organizational stressors increased work dissatisfaction and feelings of burnout, and (5) participants had limited bandwidth to manage competing demands that arose from concurrent organizational stressors. CONCLUSION: The expected challenges of an EHR transition were compounded by co-occurrence of the COVID-19 pandemic, which had negative impacts on clinician experience and patient care. During simultaneous organizational stressors, health care facilities should be prepared to address the complex interplay of two stressors on employee experience.

Assessing Electronic Health Record (EHR) Use during a Major EHR Transition: An Innovative Mixed Methods Approach.

BACKGROUND: Electronic health record (EHR) transitions are inherently disruptive to healthcare workers who must rapidly learn a new EHR and adapt to altered clinical workflows. Healthcare workers' perceptions of EHR usability and their EHR use patterns following transitions are poorly understood. The Department of Veterans Affairs (VA) is currently replacing its homegrown EHR with a commercial Cerner EHR, presenting a unique opportunity to examine EHR use trends and usability perceptions. OBJECTIVE: To assess EHR usability and uptake up to 1-year post-transition at the first VA EHR transition site using a novel longitudinal, mixed methods approach. DESIGN: A concurrent mixed methods strategy using EHR use metrics and qualitative interview data. PARTICIPANTS: 141 clinicians with data from select EHR use metrics in Cerner Lights On Network®. Interviews with 25 healthcare workers in various clinical and administrative roles. APPROACH: We assessed changes in total EHR time, documentation time, and order time per patient post-transition. Interview transcripts (n = 90) were coded and analyzed for content specific to EHR usability. KEY RESULTS: Total EHR time, documentation time, and order time all decreased precipitously within the first four months after go-live and demonstrated gradual improvements over 12 months. Interview participants expressed ongoing concerns with the EHR's usability and functionality up to a year after go-live such as tasks taking longer than the old system and inefficiencies related to inadequate training and inherent features of the new system. These sentiments did not seem to reflect the observed improvements in EHR use metrics. CONCLUSIONS: The integration of quantitative and qualitative data yielded a complex picture of EHR usability. Participants described persistent challenges with EHR usability 1 year after go-live contrasting with observed improvements in EHR use metrics. Combining findings across methods can provide a clearer, contextualized understanding of EHR adoption and use patterns during EHR transitions.

Supporting Veteran's Administration Medical Center Directors' Decisions When Adopting Innovative Practices: Development and Implementation of the "QuickView" and "WishList" Tools.

Background Since 2015, the Veterans Health Administration (VHA) Diffusion of Excellence Program has supported spread of practices developed by frontline employees. Shark Tank-style competitions encourage "Sharks" nationwide (VHA medical center/regional directors) to bid for the opportunity to implement practices at their institutions. Methods The authors evaluated bidding strategies (2016-2020), developing the "QuickView" practice comparator to promote informed bidding. Program leaders distributed QuickView and revised versions in subsequent competitions. Our team utilized in-person observation, online chats after the competition, bidder interviews, and bid analysis to evaluate QuickView use. Bids were ranked based on demonstrated understanding of resources required for practice implementation. Results Sharks stated that QuickView supported preparation before the competition and suggested improvements. Our revised tool reported necessary staff time and incorporated a "WishList" from practice finalists detailing minimum requirements for successful implementation. Bids from later years reflected increased review of facilities' current states before the competition and increased understanding of the resources needed for implementation. Percentage of bids describing local need for the practice rose from 2016 to 2020: 4.7% (6/127); 62.1% (54/87); 78.3% (36/46); 80.6% (29/36); 89.7% (26/29). Percentage of bids committing specific resources rose following QuickView introduction: 81.1% (103/127) in 2016, 69.0% (60/87) in 2017, then 73.9% (34/46) in 2018, 88.9% (32/36) in 2019, and 89.7% (26/29) in 2020. Discussion In the years following QuickView/WishList implementation, bids reflected increased assessment before the competition of both local needs and available resources. Conclusion Selection of a new practice for implementation requires an understanding of local need, necessary resources, and fit. QuickView and WishList appear to support these determinations.

Pilot study of implementing the Shared Healthcare Actions & Reflections Electronic systems in Survivorship (SHARE-S) program in coordination with clinical care.

INTRODUCTION: Initial cancer survivorship care planning efforts focused on information sharing demonstrated limited impact on patient health outcomes. We designed the Shared Healthcare Actions & Reflections Electronic Systems in survivorship (SHARE-S) program to enhance survivorship guideline implementation by transitioning some effort from clinicians to technology and patients through supporting health self-management (e.g., healthy lifestyles). METHODS: We conducted a single-group hybrid implementation-effectiveness pilot study. SHARE-S incorporated three strategies: (1) e-referral from the clinical team for patient engagement, (2) three health self-management coach calls, and (3) text messages to enhance coaching. Our primary implementation measure was the proportion of patients e-referred who enrolled (target >30%). Secondary implementation measures assessed patient engagement. We also measured effectiveness by describing changes in patient health outcomes. RESULTS: Of the 118 cancer survivor patients e-referred, 40 engaged in SHARE-S (proportion enrolled = 34%). Participants had a mean age of 57.4 years (SD = 15.7), 73% were female, 23% were Black/African American, and 5 (12.5%) were from a rural location. Patient-level adherence to coach calls was >90%. Changes from baseline to follow-up showed at least a small effect (Cohen's d = 0.2) for improvements in: mindful attention, alcohol use, physical activity, fruit and vegetable intake, days of mindfulness practice, depressive symptoms, ability to participate in social roles and activities, cancer-specific quality of life, benefits of having cancer, and positive feelings. CONCLUSION: The SHARE-S program successfully engaged cancer survivor patients. Once enrolled, patients showed promising improvements in health outcomes. Supporting patient self-management is an important component of optimizing delivery of cancer survivorship care.

Monitoring the Implementation of Tobacco Cessation Support Tools: Using Novel Electronic Health Record Activity Metrics.

BACKGROUND: Clinical decision support (CDS) tools in electronic health records (EHRs) are often used as core strategies to support quality improvement programs in the clinical setting. Monitoring the impact (intended and unintended) of these tools is crucial for program evaluation and adaptation. Existing approaches for monitoring typically rely on health care providers' self-reports or direct observation of clinical workflows, which require substantial data collection efforts and are prone to reporting bias. OBJECTIVE: This study aims to develop a novel monitoring method leveraging EHR activity data and demonstrate its use in monitoring the CDS tools implemented by a tobacco cessation program sponsored by the National Cancer Institute's Cancer Center Cessation Initiative (C3I). METHODS: We developed EHR-based metrics to monitor the implementation of two CDS tools: (1) a screening alert reminding clinic staff to complete the smoking assessment and (2) a support alert prompting health care providers to discuss support and treatment options, including referral to a cessation clinic. Using EHR activity data, we measured the completion (encounter-level alert completion rate) and burden (the number of times an alert was fired before completion and time spent handling the alert) of the CDS tools. We report metrics tracked for 12 months post implementation, comparing 7 cancer clinics (2 clinics implemented the screening alert and 5 implemented both alerts) within a C3I center, and identify areas to improve alert design and adoption. RESULTS: The screening alert fired in 5121 encounters during the 12 months post implementation. The encounter-level alert completion rate (clinic staff acknowledged completion of screening in EHR: 0.55; clinic staff completed EHR documentation of screening results: 0.32) remained stable over time but varied considerably across clinics. The support alert fired in 1074 encounters during the 12 months. Providers acted upon (ie, not postponed) the support alert in 87.3% (n=938) of encounters, identified a patient ready to quit in 12% (n=129) of encounters, and ordered a referral to the cessation clinic in 2% (n=22) of encounters. With respect to alert burden, on average, both alerts fired over 2 times (screening alert: 2.7; support alert: 2.1) before completion; time spent postponing the screening alert was similar to completing (52 vs 53 seconds) the alert, and time spent postponing the support alert was more than completing (67 vs 50 seconds) the alert per encounter. These findings inform four areas where the alert design and use can be improved: (1) improving alert adoption and completion through local adaptation, (2) improving support alert efficacy by additional strategies including training in provider-patient communication, (3) improving the accuracy of tracking for alert completion, and (4) balancing alert efficacy with the burden. CONCLUSIONS: EHR activity metrics were able to monitor the success and burden of tobacco cessation alerts, allowing for a more nuanced understanding of potential trade-offs associated with alert implementation. These metrics can be used to guide implementation adaptation and are scalable across diverse settings.

Leveraging Organizational Conditions for Innovation: A Typology of Facility Engagement in the Veterans Health Administration Shark Tank-Style Competition.

Introduction The development and spread of innovation are known challenges in health care. The US Veterans Health Administration (VHA) created a "Shark Tank"-style competition directed at frontline employees. In this annual, systemwide competition, employees submit innovations to the competition, and winning innovations receive support for implementation in other facilities. Method A multiple case study design was used to understand facility engagement in the competition, and the relationship between engagement and organizational conditions. The authors created a typology to describe the relationship between facility engagement in the competition and organizational conditions for innovation. Results Overall, there was high participation in the VHA's competition across all 130 facilities. The authors identified 7 mutually exclusive types of facility engagement. Discussion As expected, facilities with the most established conditions for innovation were the most engaged in the competition. Additionally, other facilities had various ways to be involved. Consequently, there may be benefit to the VHA tailoring how they work with facilities, based on organizational conditions. Larger facilities with ongoing research and more resources may be more suited to develop innovations, whereas smaller facilities could benefit from a focus on adoption. Conclusion These insights are valuable to the VHA and can be used by other health care systems to tailor innovation programs and allocate resources based on diverse needs across a vast health care system.

Sustainment of diverse evidence-informed practices disseminated in the Veterans Health Administration (VHA): initial development and piloting of a pragmatic survey tool.

BACKGROUND: There are challenges associated with measuring sustainment of evidence-informed practices (EIPs). First, the terms sustainability and sustainment are often falsely conflated: sustainability assesses the likelihood of an EIP being in use in the future while sustainment assesses the extent to which an EIP is (or is not) in use. Second, grant funding often ends before sustainment can be assessed. The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program is one of few large-scale models of diffusion; it seeks to identify and disseminate practices across the VHA system. The DoE sponsors "Shark Tank" competitions, in which leaders bid on the opportunity to implement a practice with approximately 6 months of implementation support. As part of an ongoing evaluation of the DoE, we sought to develop and pilot a pragmatic survey tool to assess sustainment of DoE practices. METHODS: In June 2020, surveys were sent to 64 facilities that were part of the DoE evaluation. We began analysis by comparing alignment of quantitative and qualitative responses; some facility representatives reported in the open-text box of the survey that their practice was on a temporary hold due to COVID-19 but answered the primary outcome question differently. As a result, the team reclassified the primary outcome of these facilities to Sustained: Temporary COVID-Hold. Following this reclassification, the number and percent of facilities in each category was calculated. We used directed content analysis, guided by the Consolidated Framework for Implementation Research (CFIR), to analyze open-text box responses. RESULTS: A representative from forty-one facilities (64%) completed the survey. Among responding facilities, 29/41 sustained their practice, 1/41 partially sustained their practice, 8/41 had not sustained their practice, and 3/41 had never implemented their practice. Sustainment rates increased between Cohorts 1-4. CONCLUSIONS: The initial development and piloting of our pragmatic survey allowed us to assess sustainment of DoE practices. Planned updates to the survey will enable flexibility in assessing sustainment and its determinants at any phase after adoption. This assessment approach can flex with the longitudinal and dynamic nature of sustainment, including capturing nuances in outcomes when practices are on a temporary hold. If additional piloting illustrates the survey is useful, we plan to assess the reliability and validity of this measure for broader use in the field.

Characterizing Pandemic-Related Changes in Smoking Over Time in a Cohort of Current and Former Smokers.

INTRODUCTION: We used a longitudinal cohort of US adults who were current or former smokers to explore how three participant-reported factors-general stress, coronavirus disease of 2019 (COVID-19) distress, and perceived risk of complications from COVID-19 related to smoking-were associated with changes in smoking status. METHODS: Smoking status was assessed at three time points. Timepoint 1 status was assessed at a prior study completion (2018-2020). Timepoint 2 (start of the pandemic), and Timepoint 3 (early phase of the pandemic) statuses were assessed using an additional survey in 2020. After classifying participants into eight groups per these time points, we compared the means of participant-reported factors and used a linear regression model to adjust for covariates. RESULTS: Participants (n = 392) were mostly female (73.9%) and non-Hispanic White (70.1%). Between Timepoints 2 and 3, abstinence rates decreased by 11%, and 40% of participants reported a smoking status change. Among those reporting a change and the highest general stress levels, newly abstinent participants had higher perceived risk of complications from COVID-19 related to smoking than those who relapsed during pandemic (mean (SD): 14.2 (3.3) vs. 12.6 (3.8)). Compared to participants who sustained smoking, those who sustained abstinence, on average, scored 1.94 less on the general stress scale (ßeta Coefficient (ß): -1.94, p-value < .01) and 1.37 more on the perceived risk of complications from COVID-19 related to smoking scale (ß: 1.37, p-value .02). CONCLUSIONS: Decreased abstinence rates are concerning. Patterns of reported factors were as expected for individuals who sustained their smoking behavior but not for those who changed. IMPLICATIONS: We observed an increase in smoking rates during the COVID-19 pandemic. In exploring how combinations of general stress levels, COVID-19 distress levels, and perceived risk of complications from COVID-19 related to smoking were associated with changes in smoking, we observed expected patterns of these factors among individuals who sustained abstinence or smoking. Among individuals who changed smoking status and reported high stress levels, those who reported a higher perceived risk of complications from COVID-19 related to smoking abstained from smoking. In contrast, those who reported a lower perceived risk of complications from COVID-19 related to smoking, started smoking. An intersectional perspective may be needed to understand smokers' pandemic-related behavior changes.

Diffusion of excellence: evaluating a system to identify, replicate, and spread promising innovative practices across the Veterans health administration.

Introduction: The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program provides a system to identify, replicate, and spread promising practices across the largest integrated healthcare system in the United States. DoE identifies innovations that have been successfully implemented in the VHA through a Shark Tank style competition. VHA facility and regional directors bid resources needed to replicate promising practices. Winning facilities/regions receive external facilitation to aid in replication/implementation over the course of a year. DoE staff then support diffusion of successful practices across the nationwide VHA. Methods: Organized around the Reach, Effectiveness, Adoption, Implementation, and Maintenance (RE-AIM) Framework, we summarize results of an ongoing long-term mixed-methods implementation evaluation of DoE. Data sources include: Shark Tank application and bid details, tracking practice adoptions through a Diffusion Marketplace, characteristics of VHA facilities, focus groups with Shark Tank bidders, structured observations of DoE events, surveys of DoE program participants, and semi-structured interviews of national VHA program office leaders, VHA healthcare system/facility executives, practice developers, implementation teams and facilitators. Results: In the first eight Shark Tanks (2016-2022), 3,280 Shark Tank applications were submitted; 88 were designated DoE Promising Practices (i.e., practices receive facilitated replication). DoE has effectively spread practices across the VHA, with 1,440 documented instances of adoption/replication of practices across the VHA. This includes 180 adoptions/replications in facilities located in rural areas. Leadership decisions to adopt innovations are often based on big picture considerations such as constituency support and linkage to organizational goals. DoE Promising Practices that have the greatest national spread have been successfully replicated at new sites during the facilitated replication process, have close partnerships with VHA national program offices, and tend to be less expensive to implement. Two indicators of sustainment indicate that 56 of the 88 Promising Practices are still being diffused across the VHA; 56% of facilities originally replicating the practices have sustained them, even up to 6 years after the first Shark Tank. Conclusion: DoE has developed a sustainable process for the identification, replication, and spread of promising practices as part of a learning health system committed to providing equitable access to high quality care.

Understanding suboptimal e-consult requests: lessons from the VA.

OBJECTIVES: Electronic consultations, or e-consults, which are requests for specialist advice without direct patient interaction, are becoming increasingly common across health systems. We sought to identify clinicians' perspectives on the quality of e-consult requests that they send and receive. STUDY DESIGN: A qualitative research study at the US Department of Veterans Affairs (VA) New England Healthcare System. METHODS: We interviewed a total of 73 clinicians, including 38 specialists across 3 specialties (cardiology, neurology, pulmonology) and 35 primary care clinicians (PCCs), between March and June 2019. The interviews were analyzed using thematic analysis. RESULTS: VA specialists and PCCs generally agreed that e-consult requests should be focused and precise, not require lengthy chart review, and include adequate preliminary workup results. At the same time, specialists expressed frustration with what they perceived as suboptimal e-consult requests. Interviewees attributed this gap to 3 factors: limitations of the electronic health record user interface, divergence between PCCs and specialists in the areas of expertise, and organizational pressures on the 2 groups. CONCLUSIONS: VA clinicians' perspectives on suboptimal requests contain lessons that are broadly applicable to other health systems that seek to maximize the potential of e-consults to facilitate clinician collaboration and care coordination.

Association of neighborhood-level social determinants of health with psychosocial distress in patients newly diagnosed with lung cancer.

BACKGROUND AND AIM: Patients with lung cancer experience high rates of psychosocial distress. They are also more likely to have unresolved, unmet social needs which may contribute to psychosocial distress. Despite this, neighborhood-level social determinants of health (SDOH) in relation to psychosocial distress have not been adequately investigated in patients with lung cancer. The goal of this study is to examine the association between neighborhood-level SDOH and psychosocial distress among a sample of newly diagnosed patients with lung cancer. METHODS: This cross-sectional study included newly diagnosed, adult lung cancer patients from an accredited cancer center. Psychosocial distress was measured with the Distress Thermometer. Neighborhood-level SDOH indicators for income and education were used to create a composite SDOH variable categorized into low, medium, and high deprivation levels. Covariates were age, gender, race/ethnicity, comorbidity index, cancer stage, and insurance status. Using multivariate logistic regression modeling, the association of psychosocial distress with the neighborhood-level SDOH was examined. RESULTS: The prevalence of psychosocial distress in the sample was 58.4%. Neighborhood-level SDOH indicators were not significantly associated with psychosocial distress. Higher odds of psychosocial distress were significantly associated with being female and having distant or regional cancer versus localized cancer. The age group 66-75 years was significantly associated with lower distress compared with those aged <65 years. CONCLUSIONS: Psychosocial distress was consistently high across all the SDOH deprivation categories; but these neighborhood-level SDOH indicators do not appear to be predictive of psychosocial distress at the time of diagnosis of lung cancer.