Using machine learning to mine mental health diagnostic groups from emergency department presentations before and during the COVID-19 pandemic.

PURPOSE: The COVID-19 pandemic had a profound negative effect on mental health worldwide. The hospital emergency department plays a pivotal role in responding to mental health crises. Understanding data trends relating to hospital emergency department usage is beneficial for service planning, particularly around preparing for future pandemics. Machine learning has been used to mine large volumes of unstructured data to extract meaningful data in relation to mental health presentations. This study aims to analyse trends in five mental health-related presentations to an emergency department before and during, the COVID-19 pandemic. METHODS: Data from 690,514 presentations to two Australian, public hospital emergency departments between April 2019 to February 2022 were assessed. A machine learning-based framework, Mining Emergency Department Records, Evolutionary Algorithm Data Search (MEDREADS), was used to identify suicidality, psychosis, mania, eating disorder, and substance use. RESULTS: While the mental health-related presentations to the emergency department increased during the COVID-19 pandemic compared to pre-pandemic levels, the proportion of mental health presentations relative to the total emergency department presentations decreased. Several troughs in presentation frequency were identified across the pandemic period, which occurred consistently during the public health lockdown and restriction periods. CONCLUSION: This study implemented novel machine learning techniques to analyse mental health presentations to an emergency department during the COVID-19 pandemic. Results inform understanding of the use of emergency mental health services during the pandemic, and highlight opportunities to further investigate patterns in presentation.

Interventional Response of Hospital and Health Services to the Mental Health Effects of Viral Outbreaks on Health Professionals.

The aim of this integrative review was to examine the impact of past viral epidemics on staff mental health interventional responses, with a specific focus on healthcare provider response in the context of the COVID-19 pandemic. Following PRISMA methodology, databases were searched for relevant articles. A total of 55 articles with a range of methodologies (e.g., commentary papers, cohort studies, qualitative studies) were included to ensure broad coverage of this rapidly emerging research area. The literature showed that many healthcare providers implemented a variety of wellbeing initiatives to support their staff during a viral outbreak. Most of these interventions, however, were not formally evaluated. Interventions included leadership/team support; online psychoeducational resources and updated information on the pandemic; respite spaces; peer support outreach; staff resilience training; telephone hotline support; staff support groups; and individual counseling. Staff were generally supportive of the initiatives offered by hospital and health services, with certain interventions being more appreciated (e.g., staff respite areas). Rapid, locally, and culturally appropriate workplace-based responses may counter the negative mental health impact on staff; but a stepped response is required for a smaller number of staff at risk of mental illness, or those with pre-existing mental illness. Systematic Review Registration: Unique Identifier: CRD42020222761.

A lived experience co-designed study protocol for a randomised control trial: the Attempted Suicide Short Intervention Program (ASSIP) or Brief Cognitive Behavioural Therapy as additional interventions after a suicide attempt compared to a standard Suicide Prevention Pathway (SPP).

BACKGROUND: Despite being preventable, suicide is a leading cause of death and a major global public health problem. For every death by suicide, many more suicide attempts are undertaken, and this presents as a critical risk factor for suicide. Currently, there are limited treatment options with limited underpinning research for those who present to emergency departments with suicidal behaviour. The aim of this study is to assess if adding one of two structured suicide-specific psychological interventions (Attempted Suicide Short Intervention Program [ASSIP] or Brief Cognitive Behavioural Therapy [CBT] for Suicide Prevention) to a standardised clinical care approach (Suicide Prevention Pathway [SPP]) improves the outcomes for consumers presenting to a Mental Health Service with a suicide attempt. METHODS: This is a randomised controlled trial with blinding of those assessing the outcomes. People who attempt suicide or experience suicidality after a suicide attempt, present to the Gold Coast Mental Health and Specialist Services, are placed on the Suicide Prevention Pathway (SPP), and meet the eligibility criteria, are offered the opportunity to participate. A total of 411 participants will be recruited for the study, with 137 allocated to each cohort (participants are randomised to SPP, ASSIP + SPP, or CBT + SPP). The primary outcomes of this study are re-presentation to hospitals with suicide attempts. Presentations with suicidal ideation will also be examined (in a descriptive analysis) to ascertain whether a rise in suicidal ideation is commensurate with a fall in suicide attempts (which might indicate an increase in help-seeking behaviours). Death by suicide rates will also be examined to ensure that representations with a suicide attempt are not due to participants dying, but due to a potential improvement in mental health. For participants without a subsequent suicide attempt, the total number of days from enrolment to the last assessment (24 months) will be calculated. Self-reported levels of suicidality, depression, anxiety, stress, resilience, problem-solving skills, and self- and therapist-reported level of therapeutic engagement are also being examined. Psychometric data are collected at baseline, end of interventions, and 6,12, and 24 months. DISCUSSION: This project will move both ASSIP and Brief CBT from efficacy to effectiveness research, with clear aims of assessing the addition of two structured psychological interventions to treatment as usual, providing a cost-benefit analysis of the interventions, thus delivering outcomes providing a clear pathway for rapid translation of successful interventions. TRIALS REGISTRATION: ClinicalTrials.gov NCT04072666 . Registered on 28 August 2019.

The impact of viral epidemics and pandemics on acute mental health service use: an integrative review.

The aim of this integrative review was to examine the impact of past viral epidemics on mental health, with a specific focus on changes in numbers of acute mental health presentations and mental health service recommendations in response to this, in the context of the COVID-19 pandemic. Following PRISMA methodology, databases were searched for relevant publications. A total of 83 articles with a range of methodologies were included to ensure broad coverage of this rapidly emerging research area. The literature supports an initial increase in mental health concerns which generally do not reach the threshold for diagnosis with a mental illness, but present to frontline telephone services. There is a potential delay before community and hospital-based mental health services see an increase in new or relapsing mental illness presentations. However vulnerable populations, such as people with pre-existing mental illness, are at increased risk of mental health issues during such public health crises. Many of the general recommendations distilled from the literature are closely aligned with existing strategic frameworks for mental health service provision. However, in review of these frameworks, gaps in the literature become more apparent, such as a failure to include people with lived experience, peer workers, and First Nations People in the COVID-19 mental health response.

The development of prospective memory across adolescence: an event-related potential analysis.

Prospective memory (PM) is an important cognitive function vital for day-to-day functioning. Although there has been extensive research into the decline of PM in older adulthood, little is known about its developmental trajectory throughout adolescence, a time of important brain maturation. In the present study, the development of PM was examined in 85 participants across the following groups: 12 to 13-year-olds (n = 19), 14 to 15-year-olds (n = 21), 16 to 17-year-olds (n = 19), and 18 to 19-year-olds (n = 26). A 30-cue (30 min) event-based PM task (with font-color stimuli as PM cues and a lexical-decision task as the ongoing task) was used while recording Event-Related Potentials (ERPs). The well-established neural correlates of PM, the N300 and parietal positivity, were examined across the age groups. In addition, hierarchical multiple regressions were used to examine the unique contribution of executive functioning measures (viz., the Self-Ordered Pointing Task [SOPT], the Stroop task, and Trail Making Test [TMT]) on the ERP components of PM (after controlling for age). First, the established components of ERPs associated with prospective remembering (i.e., N300 and parietal positivity) were detected for each age group. Second, although there were no significant age- group differences on the amplitude of the N300, the amplitude of the parietal positivity was found to be different between the 12 to 13-year-olds and 18 to 19-year-olds (viz., the 12 to 13-year-olds had the highest amplitude). Third, for the contribution of executive functioning measures on the amplitude of the ERP components of PM, the regression on the N300 was not significant, however, the SOPT beta weights were significant predictors of the amplitude of the parietal positivity. This relationship was found to be specific for the central and right electrode region. These findings are discussed within the context of brain development and executive functioning along with particular task demands, which may contribute to age-related PM differences across adolescence. Moreover, the findings suggest that cognitive processes associated with parietal positivity may continue to develop across adolescence.

A Qualitative Evaluation of Web-Based Cancer Care Quality Improvement Toolkit Use in the Veterans Health Administration.

BACKGROUND: Disease severity, complexity, and patient burden highlight cancer care as a target for quality improvement (QI) interventions. The Veterans Health Administration (VHA) implemented a series of disease-specific online cancer care QI toolkits. OBJECTIVES: To describe characteristics of the toolkits, target users, and VHA cancer care facilities that influenced toolkit access and use and assess whether such resources were beneficial for users. METHODS: Deductive content analysis of detailed notes from 94 telephone interviews with individuals from 48 VHA facilities. We evaluated toolkit access and use across cancer types, participation in learning collaboratives, and affiliation with VHA cancer care facilities. RESULTS: The presence of champions was identified as a strong facilitator of toolkit use, and learning collaboratives were important for spreading information about toolkit availability. Identified barriers included lack of personnel and financial resources and complicated approval processes to support tool use. CONCLUSIONS: Online cancer care toolkits are well received across cancer specialties and provider types. Clinicians, administrators, and QI staff may benefit from the availability of toolkits as they become more reliant on rapid access to strategies that support comprehensive delivery of evidence-based care. Toolkits should be considered as a complement to other QI approaches.

Assessing the impact of academic-practice partnerships on nursing staff.

BACKGROUND: The 'spillover effect' of academic-practice partnerships on hospital nursing staff has received limited attention. In 2007, the Department of Veterans Affairs (VA) created the VA Nursing Academy (VANA) to fund fifteen partnerships between schools of nursing and local VA healthcare facilities. In this paper, we examine the experiences of the VA staff nurses who worked on the units used for VANA clinical training. METHODS: We used survey methods to collect information from staff nurses at all active VANA sites on their characteristics, exposure to the program's clinical training activities, satisfaction with program components, and perspectives of the impact on their work and their own plans for education (N = 314). Our analyses utilized descriptive statistics and bivariate and multivariate regression. RESULTS: Results show that staff nurses working on VANA units had moderately high levels of exposure to the program's clinical education activities, and most reported positive experiences with those activities. The vast majority (80 %) did not perceive the presence of students as making their work more difficult. Among those who were enrolled or considering enrolling in a higher education program, over a quarter (28 %) said that their VA's participation in VANA had an influence on this decision. The majority of staff nurses were generally satisfied with their experience with the students. Their satisfaction with the program was related to the level or dose of their exposure to it. Those who were more involved were more satisfied. Greater interaction with the students, more information on the program, and a preceptor role were all independently associated with greater program satisfaction. CONCLUSIONS: Our study suggests that academic-practice partnerships may have positive spillover effects on staff nurses who work on clinical education units. Further, partnerships may be able to foster positive experiences for their unit nurses by focusing on informing and engaging them in clinical training activities. In particular, our results suggest that academic-practice partnerships should keep unit nurses well informed about program content and learning objectives, encourage frequent interaction with students, involve them in partnership-related unit-based activities, and urge them to become preceptors for the students.

Implementing a user-driven online quality improvement toolkit for cancer care.

PURPOSE: Peer-to-peer collaboration within integrated health systems requires a mechanism for sharing quality improvement lessons. The Veterans Health Administration (VA) developed online compendia of tools linked to specific cancer quality indicators. We evaluated awareness and use of the toolkits, variation across facilities, impact of social marketing, and factors influencing toolkit use. METHODS: A diffusion of innovations conceptual framework guided the collection of user activity data from the Toolkit Series SharePoint site and an online survey of potential Lung Cancer Care Toolkit users. RESULTS: The VA Toolkit Series site had 5,088 unique visitors in its first 22 months; 5% of users accounted for 40% of page views. Social marketing communications were correlated with site usage. Of survey respondents (n = 355), 54% had visited the site, of whom 24% downloaded at least one tool. Respondents' awareness of the lung cancer quality performance of their facility, and facility participation in quality improvement collaboratives, were positively associated with Toolkit Series site use. Facility-level lung cancer tool implementation varied widely across tool types. CONCLUSION: The VA Toolkit Series achieved widespread use and a high degree of user engagement, although use varied widely across facilities. The most active users were aware of and active in cancer care quality improvement. Toolkit use seemed to be reinforced by other quality improvement activities. A combination of user-driven tool creation and centralized toolkit development seemed to be effective for leveraging health information technology to spread disease-specific quality improvement tools within an integrated health care system.

Impact of VANA academic-practice partnership participation on educational mobility decisions and teaching aspirations of nurses.

This study reports findings assessing the influence of the Department of Veterans Affairs Nursing Academy (VANA) academic-practice partnership program on nurse decision making regarding educational mobility and teaching aspirations. We conducted national surveys with nursing faculty from VANA partnership sites in 2011 (N = 133) and 2012 (N = 74). Faculty who spent more hours per week in the VANA role and who reported an increase in satisfaction with their participation in VANA were more likely to have been influenced by their VANA experience in choosing to pursue a higher degree (p < .05). Sixty-nine percent of VANA faculty reported that they would be very interested in staying on as a VANA faculty member if the program should continue. Six measures were positively associated with VANA's influence on the desire to continue as faculty beyond the VANA pilot; support from VANA colleagues, quality of VANA students, amount of guidance with curriculum development, availability of administrative support, support for improving teaching methods, and overall satisfaction with VANA experience (p < .05). As the popularity of academic-practice partnerships grows and their list of benefits is further enumerated, motivating nurses to pursue both higher degrees and faculty roles should be listed among them based on results reported here.

Multimethod evaluation of the VA's peer-to-peer Toolkit for patient-centered medical home implementation.

BACKGROUND: Effective implementation of the patient-centered medical home (PCMH) in primary care practices requires training and other resources, such as online toolkits, to share strategies and materials. The Veterans Health Administration (VA) developed an online Toolkit of user-sourced tools to support teams implementing its Patient Aligned Care Team (PACT) medical home model. OBJECTIVE: To present findings from an evaluation of the PACT Toolkit, including use, variation across facilities, effect of social marketing, and factors influencing use. INNOVATION: The Toolkit is an online repository of ready-to-use tools created by VA clinic staff that physicians, nurses, and other team members may share, download, and adopt in order to more effectively implement PCMH principles and improve local performance on VA metrics. DESIGN: Multimethod evaluation using: (1) website usage analytics, (2) an online survey of the PACT community of practice's use of the Toolkit, and (3) key informant interviews. PARTICIPANTS: Survey respondents were PACT team members and coaches (n = 544) at 136 VA facilities. Interview respondents were Toolkit users and non-users (n = 32). MEASURES: For survey data, multivariable logistic models were used to predict Toolkit awareness and use. Interviews and open-text survey comments were coded using a "common themes" framework. The Consolidated Framework for Implementation Research (CFIR) guided data collection and analyses. KEY RESULTS: The Toolkit was used by 6,745 staff in the first 19 months of availability. Among members of the target audience, 80 % had heard of the Toolkit, and of those, 70 % had visited the website. Tools had been implemented at 65 % of facilities. Qualitative findings revealed a range of user perspectives from enthusiastic support to lack of sufficient time to browse the Toolkit. CONCLUSIONS: An online Toolkit to support PCMH implementation was used at VA facilities nationwide. Other complex health care organizations may benefit from adopting similar online peer-to-peer resource libraries.

The critical elements of effective academic-practice partnerships: a framework derived from the Department of Veterans Affairs Nursing Academy.

BACKGROUND: The nursing profession is exploring how academic-practice partnerships should be structured to maximize the potential benefits for each partner. As part of an evaluation of the U.S. Department of Veterans Affairs Nursing Academy (VANA) program, we sought to identify indicators of successful partnerships during the crucial first year. METHODS: We conducted a qualitative analysis of 142 individual interviews and 23 focus groups with stakeholders from 15 partnerships across the nation. Interview respondents typically included the nursing school Dean, the VA chief nurse, both VANA Program Directors (VA-based and nursing school-based), and select VANA faculty members. The focus groups included a total of 222 VANA students and the nursing unit managers and staff from units where VANA students were placed. An ethnographic approach was utilized to identify emergent themes from these data that underscored indicators of and influences on Launch Year achievement. RESULTS: We emphasize five key themes: the criticality of inter-organizational collaboration; challenges arising from blending different cultures; challenges associated with recruiting nurses to take on faculty roles; the importance of structuring the partnership to promote evidence-based practice and simulation-based learning in the clinical setting; and recognizing that stable relationships must be based on long-term commitments rather than short-term changes in the demand for nursing care. CONCLUSIONS: Developing an academic-clinical partnership requires identifying how organizations with different leadership and management structures, different responsibilities, goals and priorities, different cultures, and different financial models and accountability systems can bridge these differences to develop joint programs integrating activities across the organizations. The experience of the VANA sites in implementing academic-clinical partnerships provides a broad set of experiences from which to learn about how such partnerships can be effectively implemented, the barriers and challenges that will be encountered, and strategies and factors to overcome challenges and build an effective, sustainable partnership. This framework provides actionable guidelines for structuring and implementing effective academic-practice partnerships that support undergraduate nursing education.

A systematic review of strategies to address the clinical nursing faculty shortage.

This systematic review provides a comprehensive assessment of models used to expand the ranks of clinical nursing faculty. Nursing faculty shortages constrict the pipeline for educating nurses and make addressing the projected nursing shortage more difficult. Schools of nursing have denied admission to qualified applicants, citing insufficient numbers of nursing faculty as one major reason. Using key search terms in PubMed(®) and CINAHL(®), we identified 14 peer-reviewed articles published between 1980 and 2010 about models for expanding clinical faculty. Partnership models (n = 11) and expanded use of faculty resources (n = 9) were the most common strategies. Few (n = 8) studies assessed program efficacy. A need was identified for studies to assess the effect of alternative models on educational capacity and student performance and to examine the subcomponents of academic-practice partnerships and other innovative approaches to understand the essential factors necessary to implement successful programs.

The Department of Veterans Affairs Nursing Academy (VANA): forging strategic alliances with schools of nursing to address nursing's workforce needs.

In 2007, the Department of Veterans Affairs (VA) established the VA Nursing Academy (VANA), a 5-year, $60-million pilot program funding 15 partnerships between schools of nursing and local VA health care facilities nationwide, to expand nursing faculty, enhance clinical faculty development, increase nursing student enrollment, and promote educational innovations. VA is an ideal setting for educating nursing students owing to a well-educated registered nurse staff, an array of traditional and nontraditional settings, a state-of-the-art computerized electronic health record system, and a unique patient population. Challenges related to the complex nature of VANA partnerships, conceptualized as strategic alliances created between disparate subunits, each embedded in a larger organization, require careful governance to ensure smooth implementation. To ensure the program's aims are met, a 6-year national evaluation has been funded to help identify which strategies best achieve VANA's goals. The speed of economic recovery and the resulting changes in the nursing workforce are important determinants of VANA's future.

HIV screening among substance-abusing veterans in care.

Calls for screening for HIV infection among individuals with substance use disorders, including alcohol use, are increasing. We investigated HIV screening and its predictors in the Veterans Health Administration (VA) system among such individuals in care. Our primary outcome was retrospective evidence of screening for HIV infection, adjusting for patient demographics and important comorbid disease. Of the 371,749 sample patients with histories of substance use disorders using VA services, 20% had evidence of HIV screening. Screening was lowest among those with alcohol use disorders alone (11%) and highest among those treated in substance use programs (28%) or receiving inpatient care (28%). The findings suggest a low recognition of substance use disorders (especially alcohol use) as risk factors for HIV. Quality improvement initiatives to increase risk factor recognition and screening among patients with substance use disorders will yield benefits in the fight against HIV.

The impact of integrated HIV care on patient health outcomes.

BACKGROUND: Control of viral replication through combination antiretroviral therapy (cART) improves patient health outcomes. Yet many HIV-infected patients have comorbidities that pose social and clinical barriers to achieving viral suppression. Integration of subspecialty services into HIV primary care may overcome such barriers. OBJECTIVE: To evaluate effect of integrated HIV care (IHC) on suppression of HIV replication. RESEARCH DESIGN: A retrospective cohort study of HIV patients from 5 Veterans Affairs healthcare facilities 2000 to 2006. SUBJECTS: Patients with >3 months of follow-up, sufficient baseline HIV severity, on cART. MEASURES: We measured and ranked Integrated Care at the facilities. These rankings were applied to patient visits to form an index of IHC utilization. We evaluated effect of IHC utilization on likelihood of achieving viral suppression while on cART, controlling for demographic and clinical factors using survival analysis. RESULTS: : The 1018 HIV-infected patients eligible for analysis had substantial barriers to responding to cART: 93% had comorbidities with mean 3.2 comorbidities per patient (SD = 2.0); 52% achieved viral suppression in median 231 days (SD = 411.6). Patients visiting clinics that offered hepatitis, psychiatric, psychologic, and social services in addition to HIV primary care were 3.1 times more likely to achieve viral suppression than patients visiting clinics which offered only HIV primary care (hazard ratio = 3.1, P < 0.001). CONCLUSIONS: Patients who visited IHC clinics were more likely to achieve viral suppression while on cART. Future research should investigate which elements of Integrated Care are most associated with viral control and what role provider experience plays in this association.

Quality indicators for depression care in HIV patients.

HIV patients are at increased risk for depression. However, a comprehensive set of quality of care indicators for depression in the context of HIV does not exist. We report the results of a recent expert panel convened to develop a comprehensive set of HIV depression quality indicators. We systematically reviewed the depression and HIV depression literature to provide the evidence for the quality indicators. The HIV depression quality indicators were divided into six chapters: general, bereavement, substance abuse, viral hepatitis, cognitive impairment and drug interactions. A modified Delphi expert panel method was used to evaluate the validity and feasibility of the candidate quality indicators. The expert panel included HIV physicians, psychiatrists and a pharmacist, all with experience with HIV depression. The in-person eleven-member panel rated 80 candidate HIV-depression quality indicators. Indicators were evaluated using main and sensitivity analysis criteria for validity and feasibility. Seventy-four percent (59/80) of the quality indicators met a priori main analysis criteria for validity and feasibility and 61% met sensitivity analysis criteria. Approved indicators that are more specific to HIV depression include: depression screening frequency; depression screening and treatment in the context of hepatitis C; optimizing depression and HIV treatment in the context of cognitive impairment; and starting antidepressants at lower doses but advancing to doses typical for the general population. Most adverse medication interaction indicators did not meet main analysis validity criteria. The quality indicators identified in this study provide a useful tool for measuring and informing the quality of HIV-depression care.

Behind the scenes in health care improvement: the complex structures and emergent strategies of Implementation Science.

Implementation Science (IS) is a new branch of health services research (HSR) that strives to increase the efficiency and effectiveness of health care quality improvement (QI) efforts. Despite the fact that IS takes a systems approach, building contextual factors into its research designs, the complex systems context of IS itself-and the impact this context has on IS practice-has never been scrutinized. Using individual interviews and participant observation, the research described here characterizes key contextual factors affecting how implementation scientists in one large health care organization approach and conduct their research. Some of the organizational and professional system forces structuring their attitudes and actions were grant-related time lines, administrative burdens, and team turnover. The need for publications also figured highly. While such pressures (and related responses to them) may be rife in most grant-funded health care research settings, IS's particularly marginal position drove these implementation scientists to strategically highlight particular aspects of their work depending on which audience or part of the system they required favor from. Their narratives illuminate the contradictions and contests entailed within and engendered by organizational and professional structures, and the strategies used to negotiate these. They also reveal a great deal about the struggles underwriting disciplinary identity claims in a complex systems context.

A system-wide intervention to improve HIV testing in the Veterans Health Administration.

BACKGROUND: Although the benefits of identifying and treating asymptomatic HIV-infected individuals are firmly established, health care providers often miss opportunities to offer HIV-testing. OBJECTIVE: To evaluate whether a multi-component intervention increases the rate of HIV diagnostic testing. DESIGN: Pre- to post-quasi-experiment in 5 Veterans Health Administration facilities. Two facilities received the intervention; the other three facilities were controls. The intervention included a real-time electronic clinical reminder that encourages HIV testing, and feedback reports and a provider activation program. PATIENTS: Persons receiving health care between August 2004 and September 2006 who were at risk but had not been previously tested for HIV infection. MEASUREMENTS: Pre- to post-changes in the rates of HIV testing at the intervention and control facilities RESULTS: At the two intervention sites, the adjusted rate of testing increased from 4.8% to 10.8% and from 5.5% to 12.8% (both comparisons, p < .001). In addition, there were 15 new diagnoses of HIV in the pre-intervention year (0.46% of all tests) versus 30 new diagnoses in the post-intervention year (0.45% of all tests). No changes were observed at the control facilities. CONCLUSIONS: Use of clinical reminders and provider feedback, activation, and social marketing increased the frequency of HIV testing and the number of new HIV diagnoses. These findings support a multimodal approach toward achieving the Centers for Disease Control and Prevention's goal of having every American know their HIV status as a matter of routine clinical practice.

Measuring persistence of implementation: QUERI Series.

As more quality improvement programs are implemented to achieve gains in performance, the need to evaluate their lasting effects has become increasingly evident. However, such long-term follow-up evaluations are scarce in healthcare implementation science, being largely relegated to the "need for further research" section of most project write-ups. This article explores the variety of conceptualizations of implementation sustainability, as well as behavioral and organizational factors that influence the maintenance of gains. It highlights the finer points of design considerations and draws on our own experiences with measuring sustainability, framed within the rich theoretical and empirical contributions of others. In addition, recommendations are made for designing sustainability analyses. This article is one in a Series of articles documenting implementation science frameworks and approaches developed by the U.S. Department of Veterans Affairs Quality Enhancement Research Initiative (QUERI).