Evidence-Based Supported Employment for Individuals With Obsessive-Compulsive Disorder.

Individuals with obsessive-compulsive disorder (OCD), a chronic and disabling psychiatric disorder, experience high rates of occupational impairment. OCD symptoms commonly affect individuals' vocational aspirations and result in disability and the need for financial support, problems that are not addressed by current clinical practice guideline recommendations for treating OCD. This Open Forum highlights the need to address occupational impairment caused by OCD and makes the case for formally evaluating whether evidence-based supported employment can help individuals with OCD find and succeed in meaningful work.

Support From Intermediary Organizations for Evidence-Based Practices in Behavioral Health Care.

Intermediary and purveyor organizations play a key role in disseminating and implementing evidence-based behavioral health best practices (EBPs). The authors provide a case example to describe how state-funded intermediaries can enhance the implementation and sustainment of EBP. Benefits of using state-funded intermediaries include the ability to collaborate with state entities to address barriers to and then incentivize best practices, access to resources to develop a robust infrastructure to support EBP training and implementation, and enhanced capacity to support organizations beyond individual EBPs (e.g., developing an internal quality-improvement process, supporting cross-cutting competencies, and helping organizations to identify synergies across EBP and to prioritize what to implement first).

Integrated Treatment Group Curriculum for People with Serious Mental Health Conditions Who Use Substances.

Without proper treatment, people with co-occurring mental health and substance use problems are at great risk for poor outcomes and high treatment costs in multiple domains. Intermediary organizations can provide support to programs implementing integrated treatment and other evidence-based practices; this includes developing practical tools for programs built to encourage fidelity to a particular practice. In this paper, we describe a group curriculum workbook designed to help practitioners provide integrated treatment with fidelity and a pilot learning collaborative to evaluate whether this workbook is a helpful tool for programs to support people with serious mental health conditions and substance use in identifying and achieving personal goals. Results of the pilot found that nearly all participants demonstrated progress with respect to their identified goals, and group facilitators reported that the workbook was easy to use, that participants enjoyed the material, and that they intended to continue offering the group as part of their regular programming.

Staff Time in the Community: An Enduring Critical Component of Individual Placement and Support in the Digital Age.

OBJECTIVE: This study examined the relationship between individual placement and support (IPS) employment specialists' time spent in the community and employment outcomes in the current digital age, featuring increased technology use and online hiring practices. METHODS: The authors examined the relationship between employment outcomes and IPS employment specialists' time spent in the community at 78 sites in 2018 and 84 sites in 2019. RESULTS: The amount of time staff spent in the community was significantly and positively associated with better employment outcomes. CONCLUSIONS: These data support the continued importance of employment specialists' spending time in the community with employers and IPS recipients to achieve optimal outcomes for recipients.

Behavioral Health Workforce Development in the era of COVID-19: Examples From a State-Funded Intermediary Organization.

Intermediary and purveyor organizations (IPOs) play a key role in disseminating and implementing behavioral health evidence-based practices. The COVID-19 pandemic created a time of crisis and disruption to behavioral health care delivery. Using the conceptual framework of basic, targeted, and intensive technical assistance (TA) from the Training and Technology Transfer Centers, case studies are used to describe how programs at The Center for Practice Innovations a state funded-intermediary organization, adapted its training and technical assistance to be delivered entirely remotely, to include content related to COVID-19 and to provide guidance on telehealth-based behavioral health care.

Impact of the COVID-19 Pandemic on a Statewide Individual Placement and Support Employment Initiative.

The COVID-19 pandemic has had an enormous impact on the provision of behavioral health care services across the United States. This column examines this impact within the context of New York State's supported employment initiative, which involved 89 implementation sites before the start of the pandemic. The pandemic caused changes to the training and implementation supports provided, the number of sites providing these services, and the ways in which sites provided supported employment services. Although mean self-assessed implementation fidelity decreased modestly, employment outcomes that dipped early in the pandemic rebounded quickly to prepandemic levels.

Applying Implementation Drivers to Scale-up Evidence-Based Practices in New York State.

Mental health authorities in several states, often working with academic partners, have played important roles in disseminating evidence-based practices (EBPs) for adults diagnosed with serious mental illness. This work has been facilitated by intermediary organizations that work directly with providers to implement EBPs. This report uses two case studies to describe how the Center for Practice Innovations (CPI), an intermediary organization, has used the Active Implementation Research Network's nine implementation drivers to successfully implement EBPs across the large state of New York. One case study focuses on supported employment and the second on integrated treatment for co-occurring mental health and substance use conditions. We provide these case studies to illustrate how intermediary organizations can use implementation science to organize and select effective support strategies to disseminate and implement a range of EBPs within a state system.

An Online Training Module on the Cultural Formulation Interview: The Case of New York State.

Professional organizations and government guidelines recommend cultural competence training for providers, but the lack of a standardized cultural assessment has hindered research. Studies with the DSM-5 Cultural Formulation Interview (CFI) suggest that active learning during training improves perceptions of the CFI's usefulness as a cultural competence tool. This column reports demographic characteristics and evaluation scores among 423 providers who completed an online CFI training module developed through the New York State Office of Mental Health. Both the module, which uses the principle of active learning, and the CFI were associated with strong favorability ratings.

Using Instructional Design, Analyze, Design, Develop, Implement, and Evaluate, to Develop e-Learning Modules to Disseminate Supported Employment for Community Behavioral Health Treatment Programs in New York State.

BACKGROUND: Implementation science lacks a systematic approach to the development of learning strategies for online training in evidence-based practices (EBPs) that takes the context of real-world practice into account. The field of instructional design offers ecologically valid and systematic processes to develop learning strategies for workforce development and performance support. OBJECTIVE: This report describes the application of an instructional design framework-Analyze, Design, Develop, Implement, and Evaluate (ADDIE) model-in the development and evaluation of e-learning modules as one strategy among a multifaceted approach to the implementation of individual placement and support (IPS), a model of supported employment for community behavioral health treatment programs, in New York State. METHODS: We applied quantitative and qualitative methods to develop and evaluate three IPS e-learning modules. Throughout the ADDIE process, we conducted formative and summative evaluations and identified determinants of implementation using the Consolidated Framework for Implementation Research (CFIR). Formative evaluations consisted of qualitative feedback received from recipients and providers during early pilot work. The summative evaluation consisted of levels 1 and 2 (reaction to the training, self-reported knowledge, and practice change) quantitative and qualitative data and was guided by the Kirkpatrick model for training evaluation. RESULTS: Formative evaluation with key stakeholders identified a range of learning needs that informed the development of a pilot training program in IPS. Feedback on this pilot training program informed the design document of three e-learning modules on IPS: Introduction to IPS, IPS Job development, and Using the IPS Employment Resource Book. Each module was developed iteratively and provided an assessment of learning needs that informed successive modules. All modules were disseminated and evaluated through a learning management system. Summative evaluation revealed that learners rated the modules positively, and self-report of knowledge acquisition was high (mean range: 4.4-4.6 out of 5). About half of learners indicated that they would change their practice after watching the modules (range: 48-51%). All learners who completed the level 1 evaluation demonstrated 80% or better mastery of knowledge on the level 2 evaluation embedded in each module. The CFIR was used to identify implementation barriers and facilitators among the evaluation data which facilitated planning for subsequent implementation support activities in the IPS initiative. CONCLUSION: Instructional design approaches such as ADDIE may offer implementation scientists and practitioners a flexible and systematic approach for the development of e-learning modules as a single component or one strategy in a multifaceted approach for training in EBPs.

Relationship Between Self-Assessed Fidelity and Self-Reported Employment in the Individual Placement and Support Model of Supported Employment.

OBJECTIVE: A growing body of literature demonstrates that high-fidelity implementation of the individual placement and support (IPS) model of supported employment increases the chances of achieving desired outcomes. This study examined the relationship between IPS fidelity, as self-reported by program sites, and employment outcomes and determined whether this relationship was maintained over time. METHODS: A total of 78 outpatient programs in New York State provided data on self-reported IPS fidelity and employment outcomes. Pearson correlations were used to determine the relationship between fidelity scores and competitive employment rates. A mixed-effects model examined the relationship between repeated fidelity and employment measures over time. RESULTS: A significant positive relationship was found between better self-reported IPS fidelity and greater employment. The relationship between IPS fidelity and employment was sustained over time (up to one year). CONCLUSIONS: Higher-fidelity implementation of the IPS model, as self-assessed by program sites, was associated with higher employment rates, which were sustained over time.

Is There a Role for Fidelity Self-Assessment in the Individual Placement and Support Model of Supported Employment?

OBJECTIVE: Fidelity assessments help ensure that evidence-based practices are implemented properly. Although assessments are typically conducted by independent raters, some programs have implemented self-assessments because of resource constraints. Self-assessments were compared with independent assessments of programs implementing individual placement and support supported employment. METHODS: Eleven community-based outpatient programs in New York State completed both self- and independent assessments. Intraclass correlation coefficients and paired t tests were used to compare scores from self- and independent assessments. RESULTS: For both assessment methods, mean scores for all programs were within the range of fair fidelity. Self- and independent assessment total scores were not significantly different; however, significant differences were found on some scale items in this small sample. CONCLUSIONS: Self-assessment may be valid for examining a program's overall functioning and useful when resource constraints prevent independent assessment. Independent assessors may be able to identify nuances, particularly on individual assessment items, that can point to areas for program improvement.

Using incentives for training participation.

TOPIC: This column describes how public partners can help incentivize participation in training. Specifically, a state mental health agency and its implementation center applied financial and nonfinancial incentives to encourage participation in training and implementation supports. PURPOSE: Although training is not sufficient to change practice, it is a necessary first step in implementing evidence-based treatments. Finding ways to incentivize participation, particularly strategies with minimal resource involvement, is important for the psychiatric rehabilitation workforce and cash-strapped public systems. SOURCES USED: This description draws from published material and experiences from New York State. CONCLUSIONS AND IMPLICATIONS FOR PRACTICE: Engaging public partners to incentivize training can significantly increase participation in training. Incentive programs exist that do not require additional funding-an important consideration, given the fiscal climate for most public payers.

Using distance technologies to facilitate a learning collaborative to implement stagewise treatment.

OBJECTIVE: This report describes experiences and outcomes of an online learning collaborative focused on implementation of stagewise treatment. METHODS: Eleven participating programs convened online monthly for a year. Between meetings, program staff created an implementation plan and programs collected performance indicator data, including assessment of staff knowledge of integrated treatment for people with co-occurring disorders, whether a person's current stage of treatment was documented in his or her chart, and whether the treatments were appropriate for the stage of treatment. Descriptive statistics were used to characterize performance indicators and feedback. Wilcoxon matched-pairs signed-rank tests examined changes in performance indicators over time. RESULTS: Program staff generally demonstrated significant improvements in performance indicators over time and rated the distance learning collaborative favorably. CONCLUSIONS: Distance learning collaboratives can be structured to provide opportunities for program staff to interact and learn from one another and to implement and sustain changes.

Use of learning collaboratives by the center for practice innovations to bring IPS to scale in New York State.

This column focuses on use of learning collaboratives by the Center for Practice Innovations to help programs implement the evidence-based individual placement and support model of supported employment in New York State. These learning collaboratives use fidelity and performance indicator data to drive the development of program-specific individualized quality improvement plans. As of 2014, 59 (69%) of 86 eligible programs have joined the initiative. Programs are achieving employment outcomes for consumers on par with national benchmarks, along with improved fidelity.

Scaling up evidence-based behavioral health care practices in New York state.

This column describes the Center for Practice Innovations (CPI), which was created in 2007 by the New York State Office of Mental Health and the Department of Psychiatry at Columbia University. CPI uses innovative approaches to build stakeholder collaborations, develop and maintain practitioners' expertise, and build agency infrastructures that support implementing and sustaining evidence-based practices. CPI's five core initiatives provide training in co-occurring mental and substance use disorders, assertive community treatment, supported employment and education, wellness self-management, and treatment of first-episode psychosis. Central to CPI's activities are award-winning training modules, statewide learning collaboratives, and use of a learning management system.