Policy Recommendations for Coordinated and Sustainable Growth of the Behavioral Health Workforce.

Policy Points Demand for behavioral health services outpaces the capacity of the existing workforce, and the unmet need for behavioral health services is expected to grow. This paper summarizes research and policy evidence demonstrating that the long-standing challenges that impede behavioral health workforce development and retention (i.e., low wages, high workloads, training gaps) are being replicated by growing efforts to expand the workforce through task-sharing delivery to nonspecialist behavioral health providers (e.g., peer specialists, promotores de salud). In this paper, we describe policy opportunities to sustain behavioral health workforce growth to meet demand while supporting fair wages, labor protections, and rigorous training.

Defining the transition from new to normal: a qualitative investigation of the clinical change process.

Background: Understanding how and when a new evidence-based clinical intervention becomes standard practice is crucial to ensure that healthcare is delivered in alignment with the most up-to-date knowledge. However, rigorous methods are needed to determine when a new clinical practice becomes normalized to the standard of care. To address this gap, this study qualitatively explores how, when, and why a clinical practice change becomes normalized within healthcare organizations. Methods: We used purposive sampling to recruit clinical leaders who worked in implementation science across diverse health contexts. Enrolled participants completed semi-structured interviews. Qualitative data analysis was guided by a modified version of the Normalization Process Theory (NPT) framework to identify salient themes. Identified normalization strategies were mapped to the Expert Recommendations for Implementation Change (ERIC) project. Results: A total of 17 individuals were interviewed. Participants described four key signals for identifying when a novel clinical practice becomes the new normal: 1) integration into existing workflows; 2) scaling across the entire organizational unit; 3) staff buy-in and ownership; and 4) sustainment without ongoing monitoring. Participants identified salient strategies to normalize new clinical interventions: 1) taking a patient approach; 2) gaining staff buy-in and ownership; and 3) conducting ongoing measurement of progress towards normalization. Conclusions: The results offer valuable insight into the indicators that signify when a novel clinical practice becomes normalized, and the strategies employed to facilitate this transition. These findings can inform future research to develop instruments that implementation leaders can use to systematically measure the clinical change process.

Utility of the Leadership and Organizational Change for Implementation-Systems Level (LOCI-SL) strategy for a statewide substance use treatment implementation effort.

INTRODUCTION: Multi-level and cross-context implementation strategies are needed to support health systems, healthcare delivery organizations, and providers to adopt evidence-based practice (EBP) for substance use disorder (SUD) treatment. However, misalignment between state oversight agencies and healthcare organizations about which services to prioritize and which outcomes are reasonable to expect can hinder implementation success and widespread access to high-quality care. This study investigated the utility of the Leadership and Organizational Change for Implementation-System Level (LOCI-SL) strategy for supporting statewide EBP implementation for SUD treatment. METHODS: Nine community mental health centers (CMHCs) contracted by a state agency participated in a combined motivational-enhancement therapy and cognitive behavioral therapy (MET/CBT) implementation effort. Five of the CMHCs also received the LOCI-SL strategy to obtain ongoing implementation support. We conducted 21 individual interviews and three small group interviews with 30 participants across CMHCs and state health agencies to investigate the utility of LOCI-SL in supporting their EBP implementation efforts. Deductive thematic analysis was guided by the Exploration, Preparation, Implementation, Sustainment Framework. RESULTS: Five themes described CMHCs' LOCI-SL and broader contextual experiences implementing EBPs: (1) LOCI-SL supported executives in Preparation phase activities that holistically considered organizational needs and capacity to implement and sustain EBPs; (2) LOCI-SL facilitated trust and communication processes across Preparation, Implementation, and Sustainment phases to improve EBP uptake; (3) LOCI-SL increased CMHCs' use of implementation climate strengthening activities throughout the Implementation phase; (4) state contracts did not emphasize quality and thus were not sufficient bridging factors to enforce EBP fidelity during Implementation; and, (5) limited funding and low Medicaid reimbursement rates hindered EBP use throughout the Implementation and Sustainment phases. CONCLUSIONS: LOCI-SL was viewed as a favorable and useful implementation strategy for supporting statewide adoption of EBPs. However, outer context barriers, including limited financial investments in the treatment system, impeded implementation and sustainment efforts. While previous research suggests that contracts are viable alignment-promoting bridging factors, this study demonstrates the importance of articulating implementation outcome expectations to aid state-contracted organizations in achieving EBP implementation success. This study also highlights the need for multi-level implementation strategies to effectively align implementation expectations between outer- and inner-context entities.

Medicaid Inmate Exclusion Policy and Infectious Diseases Care for Justice-Involved Populations.

The Medicaid Inmate Exclusion Policy (MIEP) prohibits using federal funds for ambulatory care services and medications (including for infectious diseases) for incarcerated persons. More than one quarter of states, including California and Massachusetts, have asked the federal government for authority to waive the MIEP. To improve health outcomes and continuation of care, those states seek to cover transitional care services provided to persons in the period before release from incarceration. The Massachusetts Sheriffs' Association, Massachusetts Department of Correction, Executive Office of Health and Human Services, and University of Massachusetts Chan Medical School have collaborated to improve infectious disease healthcare service provision before and after release from incarceration. They seek to provide stakeholders working at the intersection of criminal justice and healthcare with tools to advance Medicaid policy and improve treatment and prevention of infectious diseases for persons in jails and prisons by removing MIEP barriers through Section 1115 waivers.

Implementation of state health insurance benefit mandates for cancer-related fertility preservation: following policy through a complex system.

BACKGROUND: A myriad of federal, state, and organizational policies are designed to improve access to evidence-based healthcare, but the impact of these policies likely varies due to contextual determinants of, reinterpretations of, and poor compliance with policy requirements throughout implementation. Strategies enhancing implementation and compliance with policy intent can improve population health. Critically assessing the multi-level environments where health policies and their related health services are implemented is essential to designing effective policy-level implementation strategies. California passed a 2019 health insurance benefit mandate requiring coverage of fertility preservation services for individuals at risk of infertility due to medical treatments, in order to improve access to services that are otherwise cost prohibitive. Our objective was to document and understand the multi-level environment, relationships, and activities involved in using state benefit mandates to facilitate patient access to fertility preservation services. METHODS: We conducted a mixed-methods study and used the policy-optimized exploration, preparation, implementation, and sustainment (EPIS) framework to analyze the implementation of California's fertility preservation benefit mandate (SB 600) at and between the state insurance regulator, insurer, and clinic levels. RESULTS: Seventeen publicly available fertility preservation benefit mandate-relevant documents were reviewed. Interviews were conducted with four insurers; 25 financial, administrative, and provider participants from 16 oncology and fertility clinics; three fertility pharmaceutical representatives; and two patient advocates. The mandate and insurance regulator guidance represented two "Big P" (system level) policies that gave rise to a host of "little p" (organizational) policies by and between the regulator, insurers, clinics, and patients. Many little p policies were bridging factors to support implementation across levels and fertility preservation service access. Characterizing the mandate's functions (i.e., policy goals) and forms (i.e., ways that policies were enacted) led to identification of (1) intended and unintended implementation, service, and patient outcomes, (2) implementation processes by level and EPIS phase, (3) actor-delineated key processes and heterogeneity among them, and (4) inner and outer context determinants that drove adaptations. CONCLUSIONS: Following the midstream and downstream implementation of a state health insurance benefit mandate, data generated will enable development of policy-level implementation strategies, evaluation of determinants and important outcomes of effective implementation, and design of future mandates to improve fit and fidelity.

Impact of U.S. Federal Loan Repayment Programs on the Behavioral Health Workforce: Scoping Review.

OBJECTIVE: Federal loan repayment programs (LRPs) are one strategy to address the shortage of behavioral health providers. This scoping review aimed to identify and characterize the federal LRPs' impact on the U.S. behavioral health workforce. METHODS: A scoping review was conducted in accordance with JBI (formerly known as the Joanna Briggs Institute) methodology for scoping reviews. The authors searched the Ovid MEDLINE, Web of Science, APA PsycInfo, EconLit, PAIS Index, and Embase databases, and gray literature was also reviewed. Two coders screened each article's abstract and full text and extracted study data. Findings were narratively synthesized and conceptually organized. RESULTS: The full-text screening identified 17 articles that met eligibility criteria. Of these, eight were peer-reviewed studies, and all but one evaluated the National Health Service Corps (NHSC) LRP. Findings were conceptually organized into five categories: descriptive studies of NHSC behavioral health needs and the NHSC workforce (k=4); providers' perceptions of, and experiences with, the NHSC (k=2); associations between NHSC funding and the number of NHSC behavioral health providers (k=4); NHSC behavioral health workforce productivity and capacity (k=3); and federal LRP recruitment and retention (k=4). CONCLUSIONS: The literature on federal LRPs and their impact on the behavioral health workforce is relatively limited. Although federal LRPs are an important and effective tool to address the behavioral health workforce shortage, additional federal policy strategies are needed to attract and retain behavioral health providers and to diversify the behavioral health workforce.

Elucidating the Perspectives of Autistic Youth About Their Health Care Experiences: A Qualitative Study.

OBJECTIVES: Autistic individuals have higher rates of co-occurring medical conditions and service use. However, autistic individuals and their families also reported lower satisfaction with health care service delivery. Previous studies described health care experiences of autistic adults, but less is known about those of autistic adolescents and young adults. This study aimed to qualitatively describe the health care experiences of autistic youth. METHODS: Four longitudinal/serial focus groups were conducted with 8 autistic adolescents and young adults. Participants were members of an autistic patient advisory board, which is part of a broader initiative at a large, urban, safety-net hospital to improve the health care experiences of autistic patients. Focus groups were conducted virtually and were audio-recorded. Audio recordings were transcribed and verified for accuracy. Transcripts were consensus-coded with an inductive approach using tenets of grounded theory. RESULTS: Findings included 4 recurring themes: accessibility and accommodations, barriers of health service use, patient involvement in health care decisions, and facilitators of patient-clinician relationship. Participants noted that visit preparation, sensory items, and repeated positive interactions with clinician were helpful to build a positive health care experience. CONCLUSION: Our findings support previous research that suggest the need to individualize care, ensure availability of accommodations, apply flexibility in practice whenever possible, and increase health care professional knowledge about this unique patient population.

The application of implementation science methods in correctional health intervention research: a systematic review.

BACKGROUND: Improving access to high-quality healthcare for individuals in correctional settings is critical to advancing health equity in the United States. Compared to the general population, criminal-legal involved individuals experience higher rates of chronic health conditions and poorer health outcomes. Implementation science frameworks and strategies offer useful tools to integrate health interventions into criminal-legal settings and to improve care. A review of implementation science in criminal-legal settings to date is necessary to advance future applications. This systematic review summarizes research that has harnessed implementation science to promote the uptake of effective health interventions in adult criminal-legal settings. METHODS: A systematic review of seven databases (Academic Search Premier, Cumulative Index to Nursing and Allied Health Literature, PsycINFO, Social Work Abstracts, ProQuest Criminal Justice Database, ProQuest Sociological Abstracts, MEDLINE/PubMed) was conducted. Eligible studies used an implementation science framework to assess implementation outcomes, determinants, and/or implementation strategies in adult criminal-legal settings. Qualitative synthesis was used to extract and summarize settings, study designs, sample characteristics, methods, and application of implementation science methods. Implementation strategies were further analyzed using the Pragmatic Implementation Reporting Tool. RESULTS: Twenty-four studies met inclusion criteria. Studies implemented interventions to address infectious diseases (n=9), substance use (n=6), mental health (n=5), co-occurring substance use and mental health (n=2), or other health conditions (n=2). Studies varied in their operationalization and description of guiding implementation frameworks/taxonomies. Sixteen studies reported implementation determinants and 12 studies measured implementation outcomes, with acceptability (n=5), feasibility (n=3), and reach (n=2) commonly assessed. Six studies tested implementation strategies. Systematic review results were used to generate recommendations for improving implementation success in criminal-legal contexts. CONCLUSIONS: The focus on implementation determinants in correctional health studies reflects the need to tailor implementation efforts to complex organizational and inter-agency contexts. Future studies should investigate policy factors that influence implementation success, design, and test implementation strategies tailored to determinants, and investigate a wider array of implementation outcomes relevant to criminal-legal settings, health interventions relevant to adult and juvenile populations, and health equity outcomes. TRIAL REGISTRATION: A study protocol (CRD42020114111) was registered with Prospero.

A structured approach to applying systems analysis methods for examining implementation mechanisms.

BACKGROUND: It is challenging to identify and understand the specific mechanisms through which an implementation strategy affects implementation outcomes, as implementation happens in the context of complex, multi-level systems. These systems and the mechanisms within each level have their own dynamic environments that change frequently. For instance, sequencing may matter in that a mechanism may only be activated indirectly by a strategy through another mechanism. The dosage or strength of a mechanism may vary over time or across different health care system levels. To elucidate the mechanisms relevant to successful implementation amidst this complexity, systems analysis methods are needed to model and manage complexity. METHODS: The fields of systems engineering and systems science offer methods-which we refer to as systems analysis methods-to help explain the interdependent relationships between and within systems, as well as dynamic changes to systems over time. When applied to studying implementation mechanisms, systems analysis methods can help (i) better identify and manage unknown conditions that may or may not activate mechanisms (both expected mechanisms targeted by a strategy and unexpected mechanisms that the methods help detect) and (ii) flexibly guide strategy adaptations to address contextual influences that emerge after the strategy is selected and used. RESULTS: In this paper, we delineate a structured approach to applying systems analysis methods for examining implementation mechanisms. The approach includes explicit steps for selecting, tailoring, and evaluating an implementation strategy regarding the mechanisms that the strategy is initially hypothesized to activate, as well as additional mechanisms that are identified through the steps. We illustrate the approach using a case example. We then discuss the strengths and limitations of this approach, as well as when these steps might be most appropriate, and suggest work to further the contributions of systems analysis methods to implementation mechanisms research. CONCLUSIONS: Our approach to applying systems analysis methods can encourage more mechanisms research efforts to consider these methods and in turn fuel both (i) rigorous comparisons of these methods to alternative mechanisms research approaches and (ii) an active discourse across the field to better delineate when these methods are appropriate for advancing mechanisms-related knowledge.

Implementation of state health insurance benefit mandates for cancer-related fertility preservation: Following policy through a complex system.

Background: A myriad of federal, state, and organizational policies are designed to improve access to evidence-based healthcare, but the impact of these policies likely varies due to contextual determinants, re-interpretations of and poor compliance with policy requirements throughout implementation. Strategies enhancing implementation and compliance with policy intent can improve population health. Critically assessing the multi-level environments where health policies and their related health services are implemented is essential to designing effective policy-level implementation strategies. California passed a 2019 health insurance benefit mandate requiring coverage of fertility preservation (FP) services for individuals at risk of infertility due to medical treatments to improve access to services that are otherwise cost-prohibitive. Our objective was to document and understand multi-level environment, relationships, and activities involved in using state benefit mandates to facilitate patient access to FP services. Methods: We conducted a mixed-methods study and used the policy-optimized Exploration, Preparation, Implementation, Sustainment (EPIS) framework to analyze implementation of California's fertility preservation benefit mandate (SB 600) at and between the state insurance regulator, insurer and clinic levels. Results: Seventeen publicly available FP benefit mandate-relevant documents were reviewed, and four insurers, 25 financial, administrative and provider participants from 16 oncology and fertility clinics, three fertility pharmaceutical representatives, and two patient advocates were interviewed. The mandate and insurance regulator guidance represented two "Big P" (system level) policies that gave rise to a host of "little p" (organizational) policies by and between the regulator, insurers, clinics, and patients. Many little p policies were bridging factors to support implementation across levels and FP service access. Characterizing the mandate's functions (i.e., policy goals) and forms (i.e., ways that policies were enacted) led to identification of (1) intended and unintended implementation, service, and patient outcomes; (2) implementation processes by level, EPIS phase; (3) actor-delineated key processes and heterogeneity among them; and (4) inner and outer context determinants that drove adaptations. Conclusions: Following the mid- and down-stream implementation of a state health insurance benefit mandate, data generated will enable development of policy level implementation strategies, evaluation of determinants and important outcomes of effective implementation, and design of future mandates to improve fit and fidelity.

Who's "in the room where it happens"? A taxonomy and five-step methodology for identifying and characterizing policy actors.

BACKGROUND: Engaging policy actors in research design and execution is critical to increasing the practical relevance and real-world impact of policy-focused dissemination and implementation science. Identifying and selecting which policy actors to engage, particularly actors involved in "Big P" public policies such as laws, is distinct from traditional engaged research methods. This current study aimed to develop a transparent, structured method for iteratively identifying policy actors involved in key policy decisions-such as adopting evidence-based interventions at systems-scale-and to guide implementation study sampling and engagement approaches. A flexible policy actor taxonomy was developed to supplement existing methods and help identify policy developers, disseminators, implementers, enforcers, and influencers. METHODS: A five-step methodology for identifying policy actors to potentially engage in policy dissemination and implementation research was developed. Leveraging a recent federal policy as a case study-The Family First Prevention Services Act (FFPSA)-publicly available documentation (e.g., websites, reports) were searched, retrieved, and coded using content analysis to characterize the organizations and individual policy actors in the "room" during policy decisions. RESULTS: The five steps are as follows: (1) clarify the policy implementation phase(s) of interest, (2) identify relevant proverbial or actual policymaking "rooms," (3) identify and characterize organizations in the room, (4) identify and characterize policy actors in the "room," and (5) quantify (e.g., count actors across groups), summarize, and compare "rooms" to develop or select engagement approaches aligned with the "room" and actors. The use and outcomes of each step are exemplified through the FFPSA case study. CONCLUSIONS: The pragmatic and transparent policy actor identification steps presented here can guide researchers' methods for continuous sampling and successful policy actor engagement. Future work should explore the utility of the proposed methods for guiding selection and tailoring of engagement and implementation strategies (e.g., research-policy actor partnerships) to improve both "Big P" and "little p" (administrative guidelines, procedures) policymaking and implementation in global contexts.

Tailoring dissemination strategies to increase evidence-informed policymaking for opioid use disorder treatment: study protocol.

BACKGROUND: Policy is a powerful tool for systematically altering healthcare access and quality, but the research to policy gap impedes translating evidence-based practices into public policy and limits widespread improvements in service and population health outcomes. The US opioid epidemic disproportionately impacts Medicaid members who rely on publicly funded benefits to access evidence-based treatment including medications for opioid use disorder (MOUD). A myriad of misaligned policies and evidence-use behaviors by policymakers across federal agencies, state Medicaid agencies, and managed care organizations limit coverage of and access to MOUD for Medicaid members. Dissemination strategies that improve policymakers' use of current evidence are critical to improving MOUD benefits and reducing health disparities. However, no research describes key determinants of Medicaid policymakers' evidence use behaviors or preferences, and few studies have examined data-driven approaches to developing dissemination strategies to enhance evidence-informed policymaking. This study aims to identify determinants and intermediaries that influence policymakers' evidence use behaviors, then develop and test data-driven tailored dissemination strategies that promote MOUD coverage in benefit arrays. METHODS: Guided by the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework, we will conduct a national survey of state Medicaid agency and managed care organization policymakers to identify determinants and intermediaries that influence how they seek, receive, and use research in their decision-making processes. We will use latent class methods to empirically identify subgroups of agencies with distinct evidence use behaviors. A 10-step dissemination strategy development and specification process will be used to tailor strategies to significant predictors identified for each latent class. Tailored dissemination strategies will be deployed to each class of policymakers and assessed for their acceptability, appropriateness, and feasibility for delivering evidence about MOUD benefit design. DISCUSSION: This study will illuminate key determinants and intermediaries that influence policymakers' evidence use behaviors when designing benefits for MOUD. This study will produce a critically needed set of data-driven, tailored policy dissemination strategies. Study results will inform a subsequent multi-site trial measuring the effectiveness of tailored dissemination strategies on MOUD benefit design and implementation. Lessons from dissemination strategy development will inform future research about policymakers' evidence use preferences and offer a replicable process for tailoring dissemination strategies.

Enhancing review criteria for dissemination and implementation science grants.

BACKGROUND: The existing grant review criteria do not consider unique methods and priorities of Dissemination and Implementation Science (DIS). The ImplemeNtation and Improvement Science Proposals Evaluation CriTeria (INSPECT) scoring system includes 10 criteria based on Proctor et al.'s "ten key ingredients" and was developed to support the assessment of DIS research proposals. We describe how we adapted INSPECT and used it in combination with the NIH scoring system to evaluate pilot DIS study proposals through our DIS Center. METHODS: We adapted INSPECT to broaden considerations for diverse DIS settings and concepts (e.g., explicitly including dissemination and implementation methods). Five PhD-level researchers with intermediate to advanced DIS knowledge were trained to conduct reviews of seven grant applications using both the INSPECT and NIH criteria. The INSPECT overall scores range from 0 to 30 (higher scores are better), and the NIH overall scores range from 1 to 9 (lower scores are better). Each grant was independently reviewed by two reviewers, then discussed in a group meeting to compare the experiences using both criteria to evaluate the proposal and to finalize scoring decisions. A follow-up survey was sent to grant reviewers to solicit further reflections on each scoring criterion. RESULTS: Averaged across reviewers, the INSPECT overall scores ranged from 13 to 24, while the NIH overall scores ranged from 2 to 5. Reviewer reflections highlighted the unique value and utility for each scoring criterion. The NIH criteria had a broad scientific purview and were better suited to evaluate more effectiveness-focused and pre-implementation proposals not testing implementation strategies. The INSPECT criteria were easier to rate in terms of the quality of integrating DIS considerations into the proposal and to assess the potential for generalizability, real-world feasibility, and impact. Overall, reviewers noted that INSPECT was a helpful tool to guide DIS research proposal writing. CONCLUSIONS: We confirmed complementarity in using both scoring criteria in our pilot study grant proposal review and highlighted the utility of INSPECT as a potential DIS resource for training and capacity building. Possible refinements to INSPECT include more explicit reviewer guidance on assessing pre-implementation proposals, providing reviewers with the opportunity to submit written commentary with each numerical rating, and greater clarity on rating criteria with overlapping descriptions.

A Standardized Radiology Template Improves Incidental Adrenal Mass Follow-Up: A Prospective Effectiveness and Implementation Study.

PURPOSE: Incidental adrenal masses (IAMs) are common but rarely evaluated. To improve this, we developed a standardized radiology report recommendation template and investigated its implementation and effectiveness. METHODS: We prospectively studied implementation of a standardized IAM reporting template as part of an ongoing quality improvement initiative, which also included primary care provider (PCP) notifications and a straightforward clinical algorithm. Data were obtained via medical record review and a survey of radiologists. Outcomes included template adoption rates and acceptability (implementation measures), as well as the proportion of patients evaluated and time to follow-up (effectiveness outcomes). RESULTS: Of 4,995 imaging studies, 200 (4.0%) detected a new IAM. The standardized template was used in 54 reports (27.0%). All radiologists surveyed were aware of the template, and 91% affirmed that standardized recommendations are useful. Patients whose reports included the template were more likely to have PCP follow-up after IAM discovery compared with those with no template (53.7% versus 36.3%, P = .03). After adjusting for sex, current or prior malignancy, and provider ordering the initial imaging (PCP, other outpatient provider, or emergency department or inpatient provider), odds of PCP follow-up remained 2.0 times higher (95% confidence interval 1.02-3.9). Patients whose reports included the template had a shorter time to PCP follow-up (log-rank P = .018). PCPs ultimately placed orders for biochemical testing (35.2% versus 18.5%, P = .01), follow-up imaging (40.7% versus 23.3%, P = .02), and specialist referral (22.2% versus 4.8%, P < .01) for a higher proportion of patients who received the template compared with those who did not. CONCLUSIONS: Use of a standardized template to communicate IAM recommendations was associated with improved IAM evaluation. Our template demonstrated high acceptability, but additional strategies are necessary to optimize adoption.

Embedding Big Qual and Team Science into Qualitative Research: Lessons from a Large-scale, Cross-site Research Study.

Background: A major part of the HEALing Communities Study (HCS), launched in 2019 to address the growing opioid epidemic, is evaluating the study's intervention implementation process through an implementation science (IS) approach. One component of the IS approach involves teams with more than 20 researchers collaborating across four research sites to conduct in-depth qualitative interviews with over 300 participants at four time points. After completion of the first two rounds of data collection, we reflect upon our qualitative data collection and analysis approach. We aim to share our lessons learned about designing and applying qualitative methods within an implementation science framework. Methods: The HCS evaluation is based on the RE-AIM/PRISM framework and incorporates interviews at four timepoints. At each timepoint, the core qualitative team of the Intervention Work Group drafts an interview guide based on the framework and insights from previous round(s) of data collection. Researchers then conduct interviews with key informants and coalition members within their respective states. Data analysis involves drafting, iteratively refining, and finalizing a codebook in a cross-site and within-site consensus processes. Interview transcripts are then individually coded by researchers within their respective states. Results: Successes in the evaluation process includes having structured procedures for communication, data collection, and analysis, all of which are critical for ensuring consistent data collection and for achieving consensus during data analysis. Challenges include recognizing and accommodating the diversity of training and knowledge between researchers, and establishing reliable ways to securely store, manage, and share the large volumes of data. Conclusion: Qualitative methods using a team science approach have been limited in their application in large, multi-site randomized controlled trials of health interventions. Our experience provides practical guidance for future studies with large, experientially and disciplinarily diverse teams, and teams seeking to incorporate qualitative or mixed-methods components for their evaluations.

Where is "policy" in dissemination and implementation science? Recommendations to advance theories, models, and frameworks: EPIS as a case example.

BACKGROUND: Implementation science aims to accelerate the public health impact of evidence-based interventions. However, implementation science has had too little focus on the role of health policy - and its inseparable politics, polity structures, and policymakers - in the implementation and sustainment of evidence-based healthcare. Policies can serve as determinants, implementation strategies, the evidence-based "thing" to be implemented, or another variable in the causal pathway to healthcare access, quality, and patient outcomes. Research describing the roles of policy in dissemination and implementation (D&I) efforts is needed to resolve persistent knowledge gaps about policymakers' evidence use, how evidence-based policies are implemented and sustained, and methods to de-implement policies that are ineffective or cause harm. Few D&I theories, models, or frameworks (TMF) explicitly guide researchers in conceptualizing where, how, and when policy should be empirically investigated. We conducted and reflected on the results of a scoping review to identify gaps of existing Exploration, Preparation, Implementation, and Sustainment (EPIS) framework-guided policy D&I studies. We argue that rather than creating new TMF, researchers should optimize existing TMF to examine policy's role in D&I. We describe six recommendations to help researchers optimize existing D&I TMF. Recommendations are applied to EPIS, as one example for advancing TMF for policy D&I. RECOMMENDATIONS: (1) Specify dimensions of a policy's function (policy goals, type, contexts, capital exchanged). (2) Specify dimensions of a policy's form (origin, structure, dynamism, outcomes). (3) Identify and define the nonlinear phases of policy D&I across outer and inner contexts. (4) Describe the temporal roles that stakeholders play in policy D&I over time. (5) Consider policy-relevant outer and inner context adaptations. (6) Identify and describe bridging factors necessary for policy D&I success. CONCLUSION: Researchers should use TMF to meaningfully conceptualize policy's role in D&I efforts to accelerate the public health impact of evidence-based policies or practices and de-implement ineffective and harmful policies. Applying these six recommendations to existing D&I TMF advances existing theoretical knowledge, especially EPIS application, rather than introducing new models. Using these recommendations will sensitize researchers to help them investigate the multifaceted roles policy can play within a causal pathway leading to D&I success.

The impact of clinical decision support systems on provider behavior in the inpatient setting: A systematic review and meta-analysis.

BACKGROUND: Clinical decision support systems (CDSS) are used to improve processes of care. CDSS proliferation may have unintended consequences impacting effectiveness. OBJECTIVE: To evaluate the effectiveness of CDSS in altering clinician behavior. DESIGN: Electronic searches were performed in EMBASE, PubMed, and Cochrane Central Register of Control Trials for randomized controlled trials testing the impacted of CDSS on clinician behavior from 2000-2021. Extracted data included study design, CDSS attributed and outcomes, user characteristics, settings, and risk of bias. Eligible studies were analyzed qualitatively to describe CDSS types. Studies with sufficient outcome data were included in the meta-analysis. SETTING AND PARTICIPANTS: Adult inpatients in the United States. INTERVENTION: Clinical decision support system versus non-clinical decision support system. MAIN OUTCOME AND MEASURE: A random-effects model measured the pooled risk difference (RD) and odds ratio of clinicians' adherence to CDSS; subgroup analyses tested differences in CDSS effectiveness over time and by CDSS type. RESULTS: Qualitative synthesis included 22 studies. Eleven studies reported sufficient outcome data for inclusion in the meta-analysis. CDSS did not result in a statistically significant increase in clinician adoption of desired practicies (RD = 0.04 [95% confidence interval {CI} 0.00, 0.07]). CDSS from 2010-2015 (n = 5) did not increase clinician adoption of desired practice [RD -0.01, (95% CI -0.04, 0.02)].CDSS from 2016-2021 (n = 6) were associated with an increase in targeted practices [RD 0.07 (95% CI0.03, 0.12)], pInteraction = 0.004. EHR [RD 0.04 (95% CI 0.00, 0.08)] vs. non-EHR [RD 0.01 (95% CI -0.01, 0.04)] based CDSS interventions did not result in different adoption of desired practices (pInteraction = 0.27). The meta-analysis did not find an overall positive impact of CDSS on clinician behavior in the inpatient setting.

Community coalition and key stakeholder perceptions of the community opioid epidemic before an intensive community-level intervention.

INTRODUCTION: Opioid overdoses are a major public health emergency in the United States. Despite effective treatments that can save lives, access to and utilization of such treatments are limited. Community context plays an important role in addressing treatment barriers and increasing access. The HEALing Communities Study (HCS) is a multisite community-level cluster-randomized trial that will study implementation and outcomes of a community coalition-based intervention (Communities that HEAL [CTH]) that implements evidence-based practices (EBPs) to reduce opioid overdose deaths in four states. To examine contextual factors critical to understanding implementation, we assessed the perspectives of community members about their communities, current substance use-related services, and other important issues that could impact intervention implementation. METHODS: Researchers conducted 382 semi-structured qualitative interviews in the HCS communities. Interviews were audio-recorded and transcribed; researchers subsequently analyzed data using directed content analysis based on the constructs of the RE-AIM/PRISM implementation science framework to identify key themes within the external community context. RESULTS: Despite the diversity in states and communities, four similar themes related to the external community context emerged across communities: These themes included the importance of understanding: 1) community risk perceptions, 2) levels of stigma, 3) the health services environment and the availability of substance use services, and 4) funding for substance use services. CONCLUSION: Understanding and addressing the external community context in which the CTH intervention and EBPs are implemented are crucial for successful health services-related and community engaged interventions. While implementing EBPs is a challenging undertaking, doing so will help us to understand if and how a community-based intervention can successfully reduce opioid overdose deaths and influence both community beliefs and the community treatment landscape.

How Do Medicaid Agencies Improve Substance Use Treatment Benefits? Lessons from Three States' 1115 Waiver Experiences.

CONTEXT: In 2015, the Centers for Medicare and Medicaid Services (CMS) urged state Medicaid programs to use 1115 waiver demonstrations to expand substance use treatment benefits. We analyzed four critical points in states' decision-making processes before expanding benefits. METHODS: We conducted qualitative cross-case comparison of three states that were early adopters of the 1115 waiver request. We conducted 44 interviews with key informants from CMS, Medicaid, and other state agencies, providers, and managed care organizations. FINDINGS: Policy makers expanded substance use treatment in response to "fragmented" care systems and unsustainable funding streams. Medicaid staff had mixed preferences for implementing new benefits via 1115 waivers or state plan amendments. The 1115 waiver process enabled states to provide coverage for residential benefits, but state plan amendments made other services permanent parts of the benefit. Medicaid agencies relied on interorganizational networks to identify evidence-based practices. Medicaid staff secured legislative support for reform by focusing on program integrity concerns and downstream effects of substance use rather than Medicaid beneficiaries' needs. CONCLUSIONS: Decision-making processes were influenced by Medicaid agency characteristics and interorganizational partnerships, not federal executive branch influence. Lessons from early-adopter states provide a road map for other state Medicaid agencies considering similar reform.