Correction to: A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia.

.

A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia.

BACKGROUND: Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static. METHOD: Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia's public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised. RESULTS: Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048). CONCLUSIONS: System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.

Concordance Between Administrator and Clinician Ratings of Organizational Culture and Climate.

Organizational culture and climate are important determinants of behavioral health service delivery for youth. The Organizational Social Context measure is a well validated assessment of organizational culture and climate that has been developed and extensively used in public sector behavioral health service settings. The degree of concordance between administrators and clinicians in their reports of organizational culture and climate may have implications for research design, inferences, and organizational intervention. However, the extent to which administrators' and clinicians' reports demonstrate concordance is just beginning to garner attention in public behavioral health settings in the United States. We investigated the concordance between 73 administrators (i.e., supervisors, clinical directors, and executive directors) and 247 clinicians in 28 child-serving programs in a public behavioral health system. Findings suggest that administrators, compared to clinicians, reported more positive cultures and climates. Organizational size moderated this relationship such that administrators in small programs (<466 youth clients served annually) provided more congruent reports of culture and climate in contrast to administrators in large programs (≥466 youth clients served annually) who reported more positive cultures and climates than clinicians. We propose a research agenda that examines the effect of concordance between administrators and clinicians on organizational outcomes in public behavioral health service settings.

The relationship between consumer, clinician, and organizational characteristics and use of evidence-based and non-evidence-based therapy strategies in a public mental health system.

We investigated the relationship between consumer, clinician, and organizational factors and clinician use of therapy strategies within a system-wide effort to increase the use of cognitive-behavioral therapy. Data from 247 clinicians in 28 child-serving organizations were collected. Clinicians participating in evidence-based practice training initiatives were more likely to report using cognitive-behavioral therapy when they endorsed more clinical experience, being salaried clinicians, and more openness to evidence-based practice. Clinicians participating in evidence-based practice initiatives were more likely to use psychodynamic techniques when they had older clients, less knowledge about evidence-based practice, more divergent attitudes toward EBP, higher financial strain, and worked in larger organizations. In clinicians not participating in evidence-based training initiatives; depersonalization was associated with higher use of cognitive-behavioral; whereas clinicians with less knowledge of evidence-based practices were more likely to use psychodynamic techniques. This study suggests that clinician characteristics are important when implementing evidence-based practices; and that consumer, clinician, and organizational characteristics are important when de-implementing non evidence-based practices. This work posits potential characteristics at multiple levels to target with implementation and deimplementation strategies.

Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study.

BACKGROUND: Examining the role of modifiable barriers and facilitators is a necessary step toward developing effective implementation strategies. This study examines whether both general (organizational culture, organizational climate, and transformational leadership) and strategic (implementation climate and implementation leadership) organizational-level factors predict therapist-level determinants of implementation (knowledge of and attitudes toward evidence-based practices). METHODS: Within the context of a system-wide effort to increase the use of evidence-based practices (EBPs) and recovery-oriented care, we conducted an observational, cross-sectional study of 19 child-serving agencies in the City of Philadelphia, including 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators. Organizational variables included characteristics such as EBP initiative participation, program size, and proportion of independent contractor therapists; general factors such as organizational culture and climate (Organizational Social Context Measurement System) and transformational leadership (Multifactor Leadership Questionnaire); and strategic factors such as implementation climate (Implementation Climate Scale) and implementation leadership (Implementation Leadership Scale). Therapist-level variables included demographics, attitudes toward EBPs (Evidence-Based Practice Attitudes Scale), and knowledge of EBPs (Knowledge of Evidence-Based Services Questionnaire). We used linear mixed-effects regression models to estimate the associations between the predictor (organizational characteristics, general and strategic factors) and dependent (knowledge of and attitudes toward EBPs) variables. RESULTS: Several variables were associated with therapists' knowledge of EBPs. Clinicians in organizations with more proficient cultures or higher levels of transformational leadership (idealized influence) had greater knowledge of EBPs; conversely, clinicians in organizations with more resistant cultures, more functional organizational climates, and implementation climates characterized by higher levels of financial reward for EBPs had less knowledge of EBPs. A number of organizational factors were associated with the therapists' attitudes toward EBPs. For example, more engaged organizational cultures, implementation climates characterized by higher levels of educational support, and more proactive implementation leadership were all associated with more positive attitudes toward EBPs. CONCLUSIONS: This study provides evidence for the importance of both general and strategic organizational determinants as predictors of knowledge of and attitudes toward EBPs. The findings highlight the need for longitudinal and mixed-methods studies that examine the influence of organizational factors on implementation.

Predictors of clinician use of exposure therapy in community mental health settings.

Exposure therapy is recognized as the key component of cognitive-behavioral treatment for anxiety. However, exposure is the least used evidence-based treatment in community mental health settings and is the most challenging technique for clinicians to adopt within the context of effectiveness and implementation trials. Little work has examined clinician and organizational characteristics that predict use of exposure, which is important for identifying implementation strategies that may increase its use. In a large sample of community health clinicians (N=335) across 31 clinical practice sites, this study characterized clinician and organizational predictors of exposure use and relaxation for anxiety. Mixed effects regression analyses indicated that both clinician attitudes and an organization's implementation climate may be important levers for interventions seeking to increase clinician exposure use. Greater clinician use of relaxation strategies was also associated with less exposure use. Results point to important implications for implementing cognitive-behavioral therapy for anxiety, including de-emphasizing relaxation and attending to organizational climate.

Non-participants in policy efforts to promote evidence-based practices in a large behavioral health system.

BACKGROUND: System-wide training initiatives to support and implement evidence-based practices (EBPs) in behavioral health systems have become increasingly widespread. Understanding more about organizations who do not participate in EBP training initiatives is a critical piece of the dissemination and implementation puzzle if we endeavor to increase access in community settings. METHODS: We conducted 30 1-h semi-structured interviews with leaders in non-participating agencies who did not formally participate in system-wide training initiatives to implement EBPs in the City of Philadelphia, with the goal to understand why they did not participate. RESULTS: We found that despite not participating in training initiatives, most agencies were adopting (and self-financing) some EBP implementation. Leadership from agencies that were implementing EBPs reported relying on previously trained staff to implement EBPs and acknowledged a lack of emphasis on fidelity. Most leaders at agencies not adopting EBPs did not have a clear understanding of what EBP is. Those familiar with EBPs in agencies not adopting EBPs reported philosophical objections to EBPs. When asked about quality assurance and treatment selection, leaders reported being guided by system audits. CONCLUSIONS: While it is highly encouraging that many agencies are adopting EBPs on their own, significant questions about fidelity and implementation success more broadly remain.

The Price per Prospective Consumer of Providing Therapist Training and Consultation in Seven Evidence-Based Treatments within a Large Public Behavioral Health System: An Example Cost-Analysis Metric.

OBJECTIVE: Public-sector behavioral health systems seeking to implement evidence-based treatments (EBTs) may face challenges selecting EBTs given their limited resources. This study describes and illustrates one method to calculate cost related to training and consultation to assist system-level decisions about which EBTs to select. METHODS: Training, consultation, and indirect labor costs were calculated for seven commonly implemented EBTs. Using extant literature, we then estimated the diagnoses and populations for which each EBT was indicated. Diagnostic and demographic information from Medicaid claims data were obtained from a large behavioral health payer organization and used to estimate the number of covered people with whom the EBT could be used and to calculate implementation-associated costs per consumer. RESULTS: Findings suggest substantial cost to therapists and service systems related to EBT training and consultation. Training and consultation costs varied by EBT, from Dialectical Behavior Therapy at $238.07 to Cognitive Behavioral Therapy at $0.18 per potential consumer served. Total cost did not correspond with the number of prospective consumers served by an EBT. CONCLUSION: A cost-metric that accounts for the prospective recipients of a given EBT within a given population may provide insight into how systems should prioritize training efforts. Future policy should consider the financial burden of EBT implementation in relation to the context of the population being served and begin a dialog in creating incentives for EBT use.

Synchronizing Watches: The Challenge of Aligning Implementation Science and Public Systems.

This special issue of Administration and Policy in Mental Health explores the complexities of the outer system context in implementation science research. In this commentary, we highlight areas of asynchrony between implementation science research and policy realities of public systems. Timing is a critical factor for many aspects of system-level implementation including when and how evidence-based practice initiatives are launched, short and inconsistent timeframes for funding and support, need for early indicators of success and demonstrating return on investment. Greater consideration for the timing that drives change in public systems will strengthen efforts to implement and sustain EBPs in community settings.

Applying the Policy Ecology Framework to Philadelphia's Behavioral Health Transformation Efforts.

Raghavan et al. (Implement Sci 3(26):1-9, 2008) proposed that effective implementation of evidence-based practices requires implementation strategies deployed at multiple levels of the "policy ecology," including the organizational, regulatory or purchaser agency, political, and social levels. However, much of implementation research and practice targets providers without accounting for contextual factors that may influence provider behavior. This paper examines Philadelphia's efforts to work toward an evidence-based and recovery-oriented behavioral health system, and uses the policy ecology framework to illustrate how multifaceted, multilevel implementation strategies can facilitate the widespread implementation of evidence-based practices. Ongoing challenges and implications for research and practice are discussed.

Independent Contractors in Public Mental Health Clinics: Implications for Use of Evidence-Based Practices.

OBJECTIVES: Community mental health clinics are increasingly utilizing independent contractors to provide clinical services. At the same time, many organizations are participating in initiatives intended to increase implementation of evidence-based practices (EBPs). The primary aim of this study was to understand the associations of utilizing independent contractors with clinician knowledge and attitudes toward EBPs and organizational culture and climate. The study also sought to understand the potential impact of using independent contractors on mental health services delivery from the perspective of organizational leadership. METHODS: Quantitative data were collected from 130 therapists in 23 organizations; qualitative data were collected from executive administrators in nine of the 16 organizations participating in EBP initiatives sponsored by the City of Philadelphia. Regression with random effects was used to estimate the associations between worker status (contractor or employee) and clinician attitudes toward EBPs, knowledge of EBPs, and organizational culture and climate. Qualitative inquiry was used to understand the impact of reliance on independent contractors on organizational participation in EBP initiatives. RESULTS: Independent contractors endorsed less positive attitudes toward EBPs and scored lower on knowledge of EBPs. Interviews revealed four main themes: reasons for using independent contractors, general consequences of using independent contractors, specific impact of independent contractors on participation in EBP initiatives, and suggestions for alternatives. CONCLUSIONS: A growing number of community mental health clinics rely on independent contractors. There may be consequences of this shift that deserve exploration.

The Perfect Storm: Collision of the Business of Mental Health and the Implementation of Evidence-Based Practices.

Financing has been hypothesized to be an important driver of the implementation of evidence-based practices (EBPs), yet there has been little systematic investigation of financing as a factor in EBP implementation. This column presents findings from a qualitative study of the effects of financial factors on the implementation of EBPs in a large urban publicly funded mental health system. Interviews with 33 agency leaders and 16 policy makers identified financial distress in community mental health agencies, leading to concerns about complex and expensive implementation of EBPs. Stakeholders agreed that the cost of EBP implementation should be shared between the agencies and the system; however, the stakeholders did not agree on how EBPs should be financed.

A Multi-Level Examination of Stakeholder Perspectives of Implementation of Evidence-Based Practices in a Large Urban Publicly-Funded Mental Health System.

Our goal was to identify barriers and facilitators to the implementation of evidence-based practices from the perspectives of multiple stakeholders in a large publicly funded mental health system. We completed 56 interviews with three stakeholder groups: treatment developers (n = 7), agency administrators (n = 33), and system leadership (n = 16). The three stakeholder groups converged on the importance of inner (e.g., agency competing resources and demands, therapist educational background) and outer context (e.g., funding) factors as barriers to implementation. Potential threats to implementation and sustainability included the fiscal landscape of community mental health clinics and an evolving workforce. Intervention characteristics were rarely endorsed as barriers. Inner context, outer context, and intervention characteristics were all seen as important facilitators. All stakeholders endorsed the importance of coordinated collaboration across stakeholder groups within the system to successfully implement evidence-based practices.

Use of latent profile analysis to assess the validity of a peer-rejected group of children.

The goal of this article was to validate the existence and qualities of a peer-rejected group of children using latent profile analysis (LPA). Two separate racially/ethnically diverse samples (Study 1: N = 2,052 second graders; Study 2: N = 594 fourth and fifth graders) completed peer nominations of liking and disliking, from which we calculated Social Preference and Social Impact scores. These scores served as indicators in the LPAs to form LPA groups. In addition, we collected self-, teacher-, and peer-report report data on aggression, depressive symptoms, peer victimization, and social competence. In each sample, an LPA group emerged in which most children were classified as rejected using the Coie, Dodge, and Coppotelli ( 1982 ; CDC) approach (Study 1: 95%; Study 2: 86%). However, in both samples, only a minority of children classified as rejected using the CDC approach fell into this LPA group (Study 1: 46%; Study 2: 36%). The LPA group that mirrored the CDC rejected group received more maladjusted scores than all other LPA groups on aggression, depressive symptoms, peer victimization, and social competence. Furthermore, when compared to children classified as rejected using only the CDC approach, children classified as rejected under both the LPA and CDC approaches were more maladjusted in terms of all sociometric and socioemotional variables. LPA analyses across two developmental levels validated the existence of an empirically derived group of children who overlapped closely with the CDC rejected group. However, this group was considerably smaller and more maladjusted than the CDC rejected group.

Reactive and proactive aggression: stability of constructs and relations to correlates.

The authors examined short-term temporal stability of reactive and proactive aggression, as well as short-term consistency of differential relations of reactive versus proactive aggression to 4 correlates. The authors used parent, teacher, peer, and self-report measures twice across 1 year to assess reactive aggression, proactive aggression, hyperactivity, social skills, anger expression, and depressive symptoms of 2nd-grade boys and girls (N = 57). Both subtypes of aggression remained stable across the year, even when the other subtype of aggression was explained at each assessment. Reactive aggression, but not proactive aggression, was consistently positively related to hyperactivity, poor social skills, and anger expression at each assessment.

Children's verbalizations and cheating behavior during game playing: the role of sociometric status, aggression, and gender.

The first goal of this study was to investigate sociometric status, aggression, and gender differences in children's verbalizations and cheating behavior during game playing using a fine-grained observational coding system. The second goal was to control for the effects of differential peer treatment and bias on children's behavior by observing children in a standardized procedure with unfamiliar peer confederates. Participants were 111 second-grade African American children, half average and half rejected sociometric status, half aggressive and half nonaggressive based on peer nominations, and half boys and half girls. Rejected children engaged in more cheating behavior and made more negative and argumentative verbalizations than average status children. Boys made more negative and argumentative verbalizations than girls. Aggressive children did not differ from nonaggressive children, in terms of either verbalizations or cheating behavior.