Correction to: A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia.

.

A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia.

BACKGROUND: Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static. METHOD: Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia's public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised. RESULTS: Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048). CONCLUSIONS: System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.

Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study.

BACKGROUND: Examining the role of modifiable barriers and facilitators is a necessary step toward developing effective implementation strategies. This study examines whether both general (organizational culture, organizational climate, and transformational leadership) and strategic (implementation climate and implementation leadership) organizational-level factors predict therapist-level determinants of implementation (knowledge of and attitudes toward evidence-based practices). METHODS: Within the context of a system-wide effort to increase the use of evidence-based practices (EBPs) and recovery-oriented care, we conducted an observational, cross-sectional study of 19 child-serving agencies in the City of Philadelphia, including 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators. Organizational variables included characteristics such as EBP initiative participation, program size, and proportion of independent contractor therapists; general factors such as organizational culture and climate (Organizational Social Context Measurement System) and transformational leadership (Multifactor Leadership Questionnaire); and strategic factors such as implementation climate (Implementation Climate Scale) and implementation leadership (Implementation Leadership Scale). Therapist-level variables included demographics, attitudes toward EBPs (Evidence-Based Practice Attitudes Scale), and knowledge of EBPs (Knowledge of Evidence-Based Services Questionnaire). We used linear mixed-effects regression models to estimate the associations between the predictor (organizational characteristics, general and strategic factors) and dependent (knowledge of and attitudes toward EBPs) variables. RESULTS: Several variables were associated with therapists' knowledge of EBPs. Clinicians in organizations with more proficient cultures or higher levels of transformational leadership (idealized influence) had greater knowledge of EBPs; conversely, clinicians in organizations with more resistant cultures, more functional organizational climates, and implementation climates characterized by higher levels of financial reward for EBPs had less knowledge of EBPs. A number of organizational factors were associated with the therapists' attitudes toward EBPs. For example, more engaged organizational cultures, implementation climates characterized by higher levels of educational support, and more proactive implementation leadership were all associated with more positive attitudes toward EBPs. CONCLUSIONS: This study provides evidence for the importance of both general and strategic organizational determinants as predictors of knowledge of and attitudes toward EBPs. The findings highlight the need for longitudinal and mixed-methods studies that examine the influence of organizational factors on implementation.

The Price per Prospective Consumer of Providing Therapist Training and Consultation in Seven Evidence-Based Treatments within a Large Public Behavioral Health System: An Example Cost-Analysis Metric.

OBJECTIVE: Public-sector behavioral health systems seeking to implement evidence-based treatments (EBTs) may face challenges selecting EBTs given their limited resources. This study describes and illustrates one method to calculate cost related to training and consultation to assist system-level decisions about which EBTs to select. METHODS: Training, consultation, and indirect labor costs were calculated for seven commonly implemented EBTs. Using extant literature, we then estimated the diagnoses and populations for which each EBT was indicated. Diagnostic and demographic information from Medicaid claims data were obtained from a large behavioral health payer organization and used to estimate the number of covered people with whom the EBT could be used and to calculate implementation-associated costs per consumer. RESULTS: Findings suggest substantial cost to therapists and service systems related to EBT training and consultation. Training and consultation costs varied by EBT, from Dialectical Behavior Therapy at $238.07 to Cognitive Behavioral Therapy at $0.18 per potential consumer served. Total cost did not correspond with the number of prospective consumers served by an EBT. CONCLUSION: A cost-metric that accounts for the prospective recipients of a given EBT within a given population may provide insight into how systems should prioritize training efforts. Future policy should consider the financial burden of EBT implementation in relation to the context of the population being served and begin a dialog in creating incentives for EBT use.

Synchronizing Watches: The Challenge of Aligning Implementation Science and Public Systems.

This special issue of Administration and Policy in Mental Health explores the complexities of the outer system context in implementation science research. In this commentary, we highlight areas of asynchrony between implementation science research and policy realities of public systems. Timing is a critical factor for many aspects of system-level implementation including when and how evidence-based practice initiatives are launched, short and inconsistent timeframes for funding and support, need for early indicators of success and demonstrating return on investment. Greater consideration for the timing that drives change in public systems will strengthen efforts to implement and sustain EBPs in community settings.

Applying the Policy Ecology Framework to Philadelphia's Behavioral Health Transformation Efforts.

Raghavan et al. (Implement Sci 3(26):1-9, 2008) proposed that effective implementation of evidence-based practices requires implementation strategies deployed at multiple levels of the "policy ecology," including the organizational, regulatory or purchaser agency, political, and social levels. However, much of implementation research and practice targets providers without accounting for contextual factors that may influence provider behavior. This paper examines Philadelphia's efforts to work toward an evidence-based and recovery-oriented behavioral health system, and uses the policy ecology framework to illustrate how multifaceted, multilevel implementation strategies can facilitate the widespread implementation of evidence-based practices. Ongoing challenges and implications for research and practice are discussed.

Use of latent profile analysis to assess the validity of a peer-rejected group of children.

The goal of this article was to validate the existence and qualities of a peer-rejected group of children using latent profile analysis (LPA). Two separate racially/ethnically diverse samples (Study 1: N = 2,052 second graders; Study 2: N = 594 fourth and fifth graders) completed peer nominations of liking and disliking, from which we calculated Social Preference and Social Impact scores. These scores served as indicators in the LPAs to form LPA groups. In addition, we collected self-, teacher-, and peer-report report data on aggression, depressive symptoms, peer victimization, and social competence. In each sample, an LPA group emerged in which most children were classified as rejected using the Coie, Dodge, and Coppotelli ( 1982 ; CDC) approach (Study 1: 95%; Study 2: 86%). However, in both samples, only a minority of children classified as rejected using the CDC approach fell into this LPA group (Study 1: 46%; Study 2: 36%). The LPA group that mirrored the CDC rejected group received more maladjusted scores than all other LPA groups on aggression, depressive symptoms, peer victimization, and social competence. Furthermore, when compared to children classified as rejected using only the CDC approach, children classified as rejected under both the LPA and CDC approaches were more maladjusted in terms of all sociometric and socioemotional variables. LPA analyses across two developmental levels validated the existence of an empirically derived group of children who overlapped closely with the CDC rejected group. However, this group was considerably smaller and more maladjusted than the CDC rejected group.

Reactive and proactive aggression: stability of constructs and relations to correlates.

The authors examined short-term temporal stability of reactive and proactive aggression, as well as short-term consistency of differential relations of reactive versus proactive aggression to 4 correlates. The authors used parent, teacher, peer, and self-report measures twice across 1 year to assess reactive aggression, proactive aggression, hyperactivity, social skills, anger expression, and depressive symptoms of 2nd-grade boys and girls (N = 57). Both subtypes of aggression remained stable across the year, even when the other subtype of aggression was explained at each assessment. Reactive aggression, but not proactive aggression, was consistently positively related to hyperactivity, poor social skills, and anger expression at each assessment.

Children's verbalizations and cheating behavior during game playing: the role of sociometric status, aggression, and gender.

The first goal of this study was to investigate sociometric status, aggression, and gender differences in children's verbalizations and cheating behavior during game playing using a fine-grained observational coding system. The second goal was to control for the effects of differential peer treatment and bias on children's behavior by observing children in a standardized procedure with unfamiliar peer confederates. Participants were 111 second-grade African American children, half average and half rejected sociometric status, half aggressive and half nonaggressive based on peer nominations, and half boys and half girls. Rejected children engaged in more cheating behavior and made more negative and argumentative verbalizations than average status children. Boys made more negative and argumentative verbalizations than girls. Aggressive children did not differ from nonaggressive children, in terms of either verbalizations or cheating behavior.