Evaluating feedback reports to support documentation of veterans' care preferences in home based primary care.

BACKGROUND: To evaluate the effectiveness of delivering feedback reports to increase completion of LST notes among VA Home Based Primary Care (HBPC) teams. The Life Sustaining Treatment Decisions Initiative (LSTDI) was implemented throughout the Veterans Health Administration (VHA) in the United States in 2017 to ensure that seriously ill Veterans have care goals and LST decisions elicited and documented. METHODS: We distributed monthly feedback reports summarizing LST template completion rates to 13 HBPC intervention sites between October 2018 and February 2020 as the sole implementation strategy. We used principal component analyses to match intervention to 26 comparison sites and used interrupted time series/segmented regression analyses to evaluate the differences in LST template completion rates between intervention and comparison sites. Data were extracted from national databases for VA HBPC in addition to interviews and surveys in a mixed methods process evaluation. RESULTS: LST template completion rose from 6.3 to 41.9% across both intervention and comparison HBPC teams between March 1, 2018, and February 26, 2020. There were no statistically significant differences for intervention sites that received feedback reports. CONCLUSIONS: Feedback reports did not increase documentation of LST preferences for Veterans at intervention compared with comparison sites. Observed increases in completion rates across intervention and comparison sites can likely be attributed to implementation strategies used nationally as part of the national roll-out of the LSTDI. Our results suggest that feedback reports alone were not an effective implementation strategy to augment national implementation strategies in HBPC teams.

Randomized controlled trial of tailored audit with feedback in VHA long-term care settings.

BACKGROUND: The Long-Term Care QUERI program supported implementation of the Life-Sustaining Treatment Decisions Initiative in US Veterans Health Administration long-term care settings. The program worked with eleven Community Living Centers (CLCs) and twelve Home-Based Primary Care (HBPC) programs to increase rates of completed templates, using audit with feedback. We distributed monthly feedback reports to site champions showing the number of Veterans with appropriate documentation. Although feedback reports are a common implementation tool, little is known about the most effective ways to design, distribute, and support them. We sought to test tailoring reports with tips using site-specific data, as well as national comparator data. METHODS: We conducted a cluster randomized controlled trial of monthly feedback reports utilizing site-tailored tips and national comparator data compared to our original feedback reports that included only graphical and numerical data. CLC and HBPC team members were invited to participate in brief surveys each quarter to determine if they had received and used the feedback reports. The outcome for CLC residents was the percent with a completed LST template any time prior to the 14th day of their stay. The outcome for HBPC residents was the percent of Veterans with a completed LST template by their second HBPC visit. RESULTS: The response rate to the survey ranged between 6.8 and 19.3% of staff members across the CLC and HBPC sites with 12.8-25.5% of survey respondents reporting that they had seen the feedback reports. The linear regression models showed no significant association between receiving the enhanced feedback reports and having a higher documentation completion rate. CONCLUSIONS: Receiving feedback reports tailored to sites by including tips based on baseline context assessments and qualitative findings, and reports showing national comparator data, did not have an impact on the number of Veterans with a completed LST template. Having a higher proportion of CLC or HBPC team members view the reports was not associated with an increase in LST template completion. These findings suggest that tailored audit with feedback may not have been effective at the program level, although the proportion of respondents who reported seeing the reports was small.

Preferences Elicited and Respected for Seriously Ill Veterans through Enhanced Decision-Making (PERSIVED): a protocol for an implementation study in the Veterans Health Administration.

BACKGROUND: Empirical evidence supports the use of structured goals of care conversations and documentation of life-sustaining treatment (LST) preferences in durable, accessible, and actionable orders to improve the care for people living with serious illness. As the largest integrated healthcare system in the USA, the Veterans Health Administration (VA) provides an excellent environment to test implementation strategies that promote this evidence-based practice. The Preferences Elicited and Respected for Seriously Ill Veterans through Enhanced Decision-Making (PERSIVED) program seeks to improve care outcomes for seriously ill Veterans by supporting efforts to conduct goals of care conversations, systematically document LST preferences, and ensure timely and accurate communication about preferences across VA and non-VA settings. METHODS: PERSIVED encompasses two separate but related implementation projects that support the same evidence-based practice. Project 1 will enroll 12 VA Home Based Primary Care (HBPC) programs and Project 2 will enroll six VA Community Nursing Home (CNH) programs. Both projects begin with a pre-implementation phase during which data from diverse stakeholders are gathered to identify barriers and facilitators to adoption of the LST evidence-based practice. This baseline assessment is used to tailor quality improvement activities using audit with feedback and implementation facilitation during the implementation phase. Site champions serve as the lynchpin between the PERSIVED project team and site personnel. PERSIVED teams support site champions through monthly coaching sessions. At the end of implementation, baseline site process maps are updated to reflect new steps and procedures to ensure timely conversations and documentation of treatment preferences. During the sustainability phase, intense engagement with champions ends, at which point champions work independently to maintain and improve processes and outcomes. Ongoing process evaluation, guided by the RE-AIM framework, is used to monitor Reach, Adoption, Implementation, and Maintenance outcomes. Effectiveness will be assessed using several endorsed clinical metrics for seriously ill populations. DISCUSSION: The PERSIVED program aims to prevent potentially burdensome LSTs by consistently eliciting and documenting values, goals, and treatment preferences of seriously ill Veterans. Working with clinical operational partners, we will apply our findings to HBPC and CNH programs throughout the national VA healthcare system during a future scale-out period.

Evaluating implementation strategies to support documentation of veterans' care preferences.

OBJECTIVE: To evaluate the effectiveness of feedback reports and feedback reports + external facilitation on completion of life-sustaining treatment (LST) note the template and durable medical orders. This quality improvement program supported the national roll-out of the Veterans Health Administration (VA) LST Decisions Initiative (LSTDI), which aims to ensure that seriously-ill veterans have care goals and LST decisions elicited and documented. DATA SOURCES: Primary data from national databases for VA nursing homes (called Community Living Centers [CLCs]) from 2018 to 2020. STUDY DESIGN: In one project, we distributed monthly feedback reports summarizing LST template completion rates to 12 sites as the sole implementation strategy. In the second involving five sites, we distributed similar feedback reports and provided robust external facilitation, which included coaching, education, and learning collaboratives. For each project, principal component analyses matched intervention to comparison sites, and interrupted time series/segmented regression analyses evaluated the differences in LSTDI template completion rates between intervention and comparison sites. DATA COLLECTION METHODS: Data were extracted from national databases in addition to interviews and surveys in a mixed-methods process evaluation. PRINCIPAL FINDINGS: LSTDI template completion rose from 0% to about 80% throughout the study period in both projects' intervention and comparison CLCs. There were small but statistically significant differences for feedback reports alone (comparison sites performed better, coefficient estimate 3.48, standard error 0.99 for the difference between groups in change in trend) and feedback reports + external facilitation (intervention sites performed better, coefficient estimate -2.38, standard error 0.72). CONCLUSIONS: Feedback reports + external facilitation was associated with a small but statistically significant improvement in outcomes compared with comparison sites. The large increases in completion rates are likely due to the well-planned national roll-out of the LSTDI. This finding suggests that when dissemination and support for widespread implementation are present and system-mandated, significant enhancements in the adoption of evidence-based practices may require more intensive support.

Applying the Tailored Implementation in Chronic Diseases framework to inform implementation of the Preferences Elicited and Respected for Seriously Ill Veterans through enhanced decision-making program in the United States Veterans Health Administration.

In 2017, the National Center for Ethics in Health Care for the United States Department of Veterans Affairs (VA) commenced national roll-out of the Life-Sustaining Treatment Decisions Initiative. This national VA initiative aimed to promote personalized, proactive, patient-driven care for seriously ill Veterans by documenting Veterans' goals and preferences for life-sustaining treatments in a durable electronic health record note template known as the life-sustaining treatment template. The Preferences Elicited and Respected for Seriously Ill Veterans through Enhanced Decision-Making (PERSIVED) quality improvement program was created to address the high variation in life-sustaining treatment template completion in VA Home Based Primary Care (HBPC) and Community Nursing Home programs. This manuscript describes the program that focuses on improving life sustaining treatment template completion rates amongst HBPC programs. To increase life-sustaining treatment template completion for Veterans receiving care from HBPC programs, the PERSIVED team applies two implementation strategies: audit with feedback and implementation facilitation. The PERSIVED team conducts semi-structured interviews, needs assessments, and process mapping with HBPC programs in order to identify barriers and facilitators to life-sustaining treatment template completion to inform tailored facilitation. Our interview data is analyzed using the Tailored Implementation in Chronic Diseases (TICD) framework, which identifies 57 determinants that might influence practice or implementation of interventions. To quickly synthesize and use baseline data to inform the tailored implementation plan, we adapted a rapid analysis process for our purposes. This paper describes a six-step process for conducting and analyzing baseline interviews through applying the TICD that can be applied and adapted by implementation scientists to rapidly inform tailoring of implementation facilitation.

Mapping the organizational readiness to change assessment to the Consolidated Framework for Implementation Research.

BACKGROUND: Implementation researchers recognize the influential role of organizational factors and, thus, seek to assess these factors using quantitative measurement instruments. However, researchers are hindered by instruments that measure similar constructs but rely on different nomenclature and/or definitions. The Consolidated Framework for Implementation Research (CFIR) provides a taxonomy of constructs derived from prior frameworks and empirical studies of implementation-related constructs. The CFIR includes constructs based on the original Promoting Action on Research Implementation in Health Services (PARiHS) framework which highlights the key roles of strength of evidence for a specific evidence-based intervention (EBI), favorability of organizational context for change, and capacities to facilitate implementation of the EBI. Although the CFIR is among the most frequently used implementation frameworks, it does not include quantitative measures. The Organizational Resource and Context Assessment (ORCA) is a quantitative measurement instrument that was developed based on PARiHS, assessing its three domains. Factors within these three domains are conceptually similar to constructs in the CFIR but do not match directly. The aim of this work was to map ORCA survey items to CFIR constructs to enable direct comparisons and syntheses of findings across studies using the CFIR and/or ORCA. METHODS: Two distinct, independent research teams, each used rigorous constant comparative techniques with deliberation and consensus to map individual items from the ORCA to the five domains and 39 constructs of CFIR. RESULTS: ORCA items were mapped primarily to three of five CFIR domains: Inner Setting, Process, and Intervention Characteristics. The two research teams agreed on 88% of mappings at the higher domain level; at the lower construct level, their mappings aligned for 62.2% of the ORCA items. CONCLUSIONS: Mapping results reveal that the ORCA focuses measurement prominently on Inner Setting, Process, and Intervention Characteristics. This mapping guide can help improve consistency in measurement and reporting, enabling more efficient comparison and synthesis of findings that use either the ORCA instrument or the CFIR framework. The guide helps advance implementation science utilizing mixed methods by providing CFIR users with quantitative measures for selected constructs and enables ORCA users to map their findings to CFIR constructs.

Correction: Predictors and Consequences of Veterans Affairs Mental Health Provider Burnout: Protocol for a Mixed Methods Study.

[This corrects the article DOI: 10.2196/18345.].

Implementation of VA's Life-Sustaining Treatment Decisions Initiative: Facilitators and Barriers to Early Implementation Across Seven VA Medical Centers.

CONTEXT: In 2017, Veterans Health Administration (VHA) National Center for Ethics in Health Care began system-wide implementation of the Life-Sustaining Treatment Decisions Initiative (LSTDI). The LSTDI is a national VHA policy and practice to promote conducting goals of care conversations and documenting veterans' preferences for life-sustaining treatments (LSTs). OBJECTIVES: The aim of this article is to describe facilitators and barriers to early implementation of the LSTDI within one VHA Veterans Integrated Service Network. METHODS: From September 2016 to December 2018, we conducted site visits and semistructured phone interviews with implementation coordinators who championed the LSTDI rollout at seven VHA medical centers. We applied the Consolidated Framework for Implementation Research (CFIR) to assess facilitators and barriers to implementing the LSTDI and assigning interview data to specific CFIR constructs and CFIR valence ratings. We simultaneously benchmarked VHA medical centers' implementation progress as outlined by the National Center for Ethics in Health Care implementation guidebook. RESULTS: We divided sites into three descriptive groups based on implementation progress: successfully implemented (n = 2); moving forward, but delayed (n = 3); and implementation stalled (n = 2). Five CFIR constructs emerged as facilitators or barriers to implementation of the LSTDI: 1) self-efficacy of implementation coordinators; 2) leadership engagement; 3) compatibility with pre-existing workflows; 4) available resources; and 5) overall implementation climate. CONCLUSION: Although self-efficacy proved key to overcoming obstacles, degree of perceived workflow compatibility of the LSTDI policy, available resources, and leadership engagement must be adequate for successful implementation within the implementation time line. Without these components, successful implementation was hindered or delayed.

Predictors and Consequences of Veterans Affairs Mental Health Provider Burnout: Protocol for a Mixed Methods Study.

BACKGROUND: In the Veterans Health Administration (VHA), mental health providers (MHPs) report the second highest level of burnout after primary care physicians. Burnout is defined as increased emotional exhaustion and depersonalization and decreased sense of personal accomplishment at work. OBJECTIVE: This study aims to characterize variation in MHP burnout by VHA facility over time, identifying workplace characteristics and practices of high-performing facilities. METHODS: Using both qualitative and quantitative methods, we will evaluate factors that influence MHP burnout and their effects on patient outcomes. We will compile annual survey data on workplace conditions and annual staffing as well as productivity data to assess same and subsequent year provider and patient outcomes reflecting provider and patient experiences. We will conduct interviews with mental health leadership at the facility level and with frontline MHPs sampled based on our quantitative findings. We will present our findings to an expert panel of operational partners, Veterans Affairs clinicians, administrators, policy leaders, and experts in burnout. We will reengage with facilities that participated in the earlier qualitative interviews and will hold focus groups that share results based on our quantitative and qualitative work combined with input from our expert panel. We will broadly disseminate these findings to support the development of actionable policies and approaches to addressing MHP burnout. RESULTS: This study will assist in developing and testing interventions to improve MHP burnout and employee engagement. Our work will contribute to improvements within VHA and will generate insights for health care delivery, informing efforts to address burnout. CONCLUSIONS: This is the first comprehensive, longitudinal, national, mixed methods study that incorporates different types of MHPs. It will engage MHP leadership and frontline providers in understanding facilitators and barriers to effectively address burnout. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/18345.

Visual process maps to support implementation efforts: a case example.

BACKGROUND: Process mapping is often used in quality improvement work to examine current processes and workflow and to identify areas to intervene to improve quality. Our objective in this paper is to describe process maps as a visual means of understanding modifiable behaviors and activities, in this case example to ensure that goals of care conversations are part of admitting a veteran in long-term care settings. METHODS: We completed site visits to 6 VA nursing homes and reviewed their current admission processes. We conducted interviews to document behaviors and activities that occur when a veteran is referred to a long-term care setting, during admission, and during mandatory VA reassessments. We created visualizations of the data using process mapping approaches. Process maps for each site were created to document the admission activities for each VA nursing home and were reviewed by the research team to identify consistencies across sites and to identify potential opportunities for implementing goals of care conversations. RESULTS: We identified five consistent behaviors that take place when a veteran is referred and admitted in long-term care. These behaviors are assessing, discussing, decision-making, documenting, and re-assessing. CONCLUSIONS: Based on the process maps, it seems feasible that the LST note and order template could be completed along with other routine assessment processes. However, this will require more robust multi-disciplinary collaboration among both prescribing and non-prescribing health care providers. Completing the LST template during the current admission process would increase the likelihood that the template is completed in a timely manner, potentially alleviate the perceived time burden, and help with the provision of veteran-centered care.

Life-Sustaining Treatment Decisions Initiative: Early Implementation Results of a National Veterans Affairs Program to Honor Veterans' Care Preferences.

BACKGROUND: On July 1, 2018, the Veterans Health Administration (VA) National Center for Ethics in Health Care implemented the Life-Sustaining Treatment Decisions Initiative (LSTDI). Its goal is to identify, document, and honor LST decisions of seriously ill veterans. Providers document veterans' goals and decisions using a standardized LST template and order set. OBJECTIVE: Evaluate the first 7 months of LSTDI implementation and identify predictors of LST template completion. DESIGN: Retrospective observational study of clinical and administrative data. We identified all completed LST templates, defined as completion of four required template fields. Templates also include four non-required fields. Results were stratified by risk of hospitalization or death as estimated by the Care Assessment Need (CAN) score. SUBJECTS: All veterans with VA utilization between July 1, 2018, and January 31, 2019. MAIN MEASURES: Completed LST templates, goals and LST preferences, and predictors of documentation. RESULTS: LST templates were documented for 108,145 veterans, and 85% had one or more of the non-required fields completed in addition to the required fields. Approximately half documented a preference for cardiopulmonary resuscitation. Among those who documented specific goals, half wanted to improve or maintain function, independence, and quality of life while 28% had a goal of life prolongation irrespective of risk of hospitalization/death and 45% expressed a goal of comfort. Only 7% expressed a goal of being cured. Predictors of documentation included VA nursing home residence, older age, frailty, and comorbidity, while non-Caucasian race, rural residence, and receipt of care in a lower complexity medical center were predictive of no documentation. CONCLUSIONS: LST decisions were documented for veterans at high risk of hospitalization or death. While few expressed a preference for cure, half desire, cardiopulmonary resuscitation. Predictors of documentation were generally consistent with existing literature. Opportunities to reduce observed disparities exist by leveraging available VA resources and programs.

Designing clinical practice feedback reports: three steps illustrated in Veterans Health Affairs long-term care facilities and programs.

BACKGROUND: User-centered design (UCD) methods are well-established techniques for creating useful artifacts, but few studies illustrate their application to clinical feedback reports. When used as an implementation strategy, the content of feedback reports depends on a foundational audit process involving performance measures and data, but these important relationships have not been adequately described. Better guidance on UCD methods for designing feedback reports is needed. Our objective is to describe the feedback report design method for refining the content of prototype reports. METHODS: We propose a three-step feedback report design method (refinement of measures, data, and display). The three steps follow dependencies such that refinement of measures can require changes to data, which in turn may require changes to the display. We believe this method can be used effectively with a broad range of UCD techniques. RESULTS: We illustrate the three-step method as used in implementation of goals of care conversations in long-term care settings in the U.S. Veterans Health Administration. Using iterative usability testing, feedback report content evolved over cycles of the three steps. Following the steps in the proposed method through 12 iterations with 13 participants, we improved the usability of the feedback reports. CONCLUSIONS: UCD methods can improve feedback report content through an iterative process. When designing feedback reports, refining measures, data, and display may enable report designers to improve the user centeredness of feedback reports.