A Randomized Trial to Identify Accurate Measurement Methods for Adherence to Cognitive-Behavioral Therapy.

Clinician fidelity to cognitive behavioral therapy (CBT) is an important mechanism by which desired clinical outcomes are achieved and is an indicator of care quality. Despite its importance, there are few fidelity measurement methods that are efficient and have demonstrated reliability and validity. Using a randomized trial design, we compared three methods of assessing CBT adherence-a core component of fidelity-to direct observation, the gold standard. Clinicians recruited from 27 community mental health agencies (n = 126; M age = 37.69 years, SD = 12.84; 75.7% female) were randomized 1:1:1 to one of three fidelity conditions: self-report (n = 41), chart-stimulated recall (semistructured interviews with the chart available; n = 42), or behavioral rehearsal (simulated role-plays; n = 43). All participating clinicians completed fidelity assessments for up to three sessions with three different clients that were recruited from clinicians' caseloads (n = 288; M age = 13.39 years SD = 3.89; 41.7% female); sessions were also audio-recorded and coded for comparison to determine the most accurate method. All fidelity measures had parallel scales that yielded an adherence maximum score (i.e., the highest-rated intervention in a session), a mean of techniques observed, and a count total of observed techniques. Results of three-level mixed effects regression models indicated that behavioral rehearsal produced comparable scores to observation for all adherence scores (all ps > .01), indicating no difference between behavioral rehearsal and observation. Self-report and chart-stimulated recall overestimated adherence compared to observation (ps < .01). Overall, findings suggested that behavioral rehearsal indexed CBT adherence comparably to direct observation, the gold-standard, in pediatric populations. Behavioral rehearsal may at times be able to replace the need for resource-intensive direct observation in implementation research and practice.

Performance of a Supervisor Observational Coding System and an Audit and Feedback Intervention.

Workplace-based clinical supervision is common in community based mental health care for youth and families and could be a leveraged to scale and improve the implementation of evidence-based treatment (EBTs). Accurate methods are needed to measure, monitor, and support supervisor performance with limited disruption to workflow. Audit and Feedback (A&F) interventions may offer some promise in this regard. The study-a randomized controlled trial with 60 clinical supervisors measured longitudinally for 7 months-had two parts: (1) psychometric evaluation of an observational coding system for measuring adherence and competence of EBT supervision and (2) evaluation of an experimental Supervisor Audit and Feedback (SAF) intervention on outcomes of supervisor adherence and competence. All supervisors recorded and uploaded weekly supervision sessions for 7 months, and those in the experimental condition were provided a single, monthly web-based feedback report. Psychometric performance was evaluated using measurement models based in Item Response Theory, and the effect of the SAF intervention was evaluated using mixed-effects regression models. The observational instrument performed well across psychometric indicators of dimensionality, rating scale functionality, and item fit; however, coder reliability was lower for competence than for adherence. Statistically significant A&F effects were largely in the expected directions and consistent with hypotheses. The observational coding system performed well, and a monthly electronic feedback report showed promise in maintaining or improving community-based clinical supervisors' adherence and, to a lesser extent, competence. Limitations discussed include unknown generalizability to the supervision of other EBTs.

Stakeholder intention to engage in fidelity measurement methods in community mental health settings: A mixed methods study.

Background: The current gold standard for measuring fidelity (specifically, adherence) to cognitive behavioral therapy (CBT) is direct observation, a costly, resource-intensive practice that is not feasible for many community organizations to implement regularly. Recent research indicates that behavioral rehearsal (i.e., role-play between clinician and individual with regard to session delivery) and chart-stimulated recall (i.e., brief structured interview between clinician and individual about what they did in session; clinicians use the client chart to prompt memory) may provide accurate and affordable alternatives for measuring adherence to CBT in such settings, with behavioral rehearsal yielding greater correspondence with direct observation. Methods: Drawing on established causal theories from social psychology and leading implementation science frameworks, this study evaluates stakeholders' intention to use behavioral rehearsal and chart-stimulated recall. Specifically, we measured attitudes, self-efficacy, and subjective norms toward using each, and compared these factors across the two methods. We also examined the relationship between attitudes, self-efficacy, subjective norms, and intention to use each method. Finally, using an integrated approach we asked stakeholders to discuss their perception of contextual factors that may influence beliefs about using each method. These data were collected from community-based supervisors (n = 17) and clinicians (n = 66). Results: Quantitative analyses suggest moderately strong intention to use both methods across stakeholders. There were no differences in supervisors' or clinicians' attitudes, self-efficacy, subjective norms, or intention across methods. More positive attitudes and greater reported subjective norms were associated with greater reported intention to use either measure. Qualitative analyses identified participants' specific beliefs about using each fidelity measure in their organization, and results were organized using the Consolidated Framework for Implementation Research. Conclusions: Strategies are warranted to overcome or minimize potential barriers to using fidelity measurement methods and to further increase the strength of intention to use them.Plain Language Summary: The best way to measure fidelity, or how closely a clinician follows the protocol, to Cognitive Behavioral Therapy (CBT) is watching the session. This is an expensive practice that is not feasible for many community organizations to do regularly. Recent research indicates that behavioral rehearsal, or a role-play between the clinician and individual with regard to session delivery, and chart-stimulated recall, or a brief discussion between an individual and the clinician about what they did in session with the clinician having access to the chart to help them remember, may provide accurate and affordable alternatives for measuring fidelity to CBT. We just completed a study demonstrating that both methods are promising, with behavioral rehearsal offering scores that are the most similar to watching the session. Drawing on established theories from social psychology and leading implementation science frameworks, this study evaluates future supervisor and clinician motivation to use these fidelity measurement methods. Specifically, we measured supervisor (n = 17) and clinician (n = 66) attitudes, norms, self-efficacy, intentions, and anticipated barriers and facilitators to using each of these fidelity measurement tools. Quantitative and qualitative analyses suggest similar intention to use both methods, and concerns about barriers to using each method. Further research is warranted to minimize the burden associated with implementing fidelity measurement methods and deploying strategies to increase use.

Improving the feasibility of fidelity measurement for community-based quality assurance: Partial- versus full-session observations of supervisor adherence and competence.

Background: Clinical supervision is a common quality assurance method for supporting the implementation and sustainment of evidence-based interventions (EBIs) in community mental health settings. However, assessing and supporting supervisor fidelity requires efficient and effective measurement methods. This study evaluated two observational coding approaches that are potentially more efficient than coding full sessions: a randomly selected 15-min segment and the first case discussion of the session. Method: Data were leveraged from a randomized trial of an Audit and Feedback (A&F) intervention for supervisor Adherence and Competence. Supervisors (N = 57) recorded and uploaded weekly group supervision sessions for 7 months, with one session observationally coded each month (N = 374). Of the coded sessions, one was randomly selected for each supervisor, and a random 15-min segment was coded. Additionally, the first case discussion was coded for the full sample of sessions. Results: Across all models (and controlling for the proportion of the session covered by the partial observation), Adherence and Competence scores from partial observations were positively and significantly associated with scores from full sessions. In all cases, partial observations were most accurate when the level of Adherence and Competence was moderate. At lower levels, partial observations were underestimates, and at higher levels, they were overestimates. Conclusions: The results suggest that efficient observational measurement can be achieved while retaining a general level of measurement effectiveness. Practically, first-case discussions are easier to implement, whereas 15-min segments have fewer potential threats to validity. Evaluation of resource requirements is needed, along with determining whether A&F effects are retained if feedback is based on partial observations. Nevertheless, more efficient observational coding could increase the feasibility of routine fidelity monitoring and quality assurance strategies, including A&F, which ultimately could support the implementation and sustainment of effective supervision practices and EBIs in community practice settings.Plain Language Summary: When delivering evidence-based mental health interventions in community-based practice settings, a common quality assurance method is clinical supervision. To support supervisors, assessment methods are needed, and those methods need to be both efficient and effective. Ideally, supervision sessions would be recorded, and trained coders would rate the supervisor's use of specific strategies. In most settings, though, this requires too many resources. The present study evaluated a more efficient approach. The data came from an existing randomized trial of an Audit and Feedback intervention for enhancing supervisor Adherence and Competence. This included 57 supervisors and 374 sessions across seven months of monitoring. Instead of rating full supervision sessions, a more efficient approach was to have coders rate partial sessions. Two types of partial observations were considered: a randomly selected 15-minute segment of the session and the first case discussion of the session. The aim was to see if partial observations and full observations led to similar conclusions about Adherence and Competence. In all cases, they did. The scores were most similar for sessions with moderate levels of Adherence and Competence. If Adherence and Competence were low, partial observations were underestimates, but if they were high, partial observations were overestimates. Observing partial sessions is more efficient, but in terms of accuracy, the benefits and limitations should be evaluated in light of how the scores will be used. Additionally, future research should consider whether Audit and Feedback interventions have the same effect if feedback is based on observations of partial sessions.

The TPOCS-self-reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY): A case study of pragmatic measure development.

Background: Developing pragmatic assessment tools to measure clinician use of evidence-based practices is critical to advancing implementation of evidence-based practices in mental health. This case study details our community-partnered process of developing the Therapy Process Observation Coding Scale-Self-Reported Therapist Intervention Fidelity for Youth (TPOCS-SeRTIFY), a pragmatic, clinician-report instrument to measure cognitive behavioral therapy (CBT) delivery. Approach: We describe a five-step community-partnered development process. Initial goals were to create a self-report instrument that paralleled an existing direct observation measure of clinician delivery of CBT use to facilitate later assessment of measure performance. Cognitive interviews with community clinicians (n = 6) and consultation with CBT experts (n = 6) were used to enhance interpretability and usability as part of an iterative refinement process. The instrument was administered to 247 community clinicians along with an established self-reported measure of clinician delivery of CBT and other treatments to assess preliminary psychometric performance. Preliminary psychometrics were promising. Conclusion: Our community-partnered development process showed promising success and can guide future development of pragmatic implementation measures both to facilitate measurement of ongoing implementation efforts and future research aimed at building learning mental health systems. Plain language summary: Developing brief, user-friendly, and accurate tools to measure how therapists deliver cognitive behavioral therapy (CBT) in routine practice is important for advancing the reach of CBT into community settings. To date, developing such "pragmatic" measures has been difficult. There is little known about how researchers can best develop these types of assessment tools so that they (1) are easy for clinicians in practice to use and (2) provide valid and useful information about implementation outcomes. As a result, there are few well-validated measures in existence that measure therapist use of CBT that are feasible for use in community practice. This paper contributes to the literature by describing our community-partnered process for developing a measure of therapist use of CBT (Therapy Process Observation Coding Scale -Self-Reported Therapist Intervention Fidelity for Youth; TPOCS-SeRTIFY). This descriptive case study outlines the community-partnered approach we took to develop this measure. This case study will contribute to future research by serving as a guide to others aiming to develop pragmatic implementation measures. In addition, the TPOCS-SeRTIFY is a pragmatic measure of clinician use of CBT that holds promise for its use by both researchers and clinicians to measure the success of CBT implementation efforts.

Festschrift for Leonard Bickman: Introduction to The Future of Children's Mental Health Services Special Issue.

This introductory article describes the genesis of the Festschrift for Leonard Bickman and of this Festschrift special issue entitled, The Future of Children's Mental Health Services. The special issue includes a collection of 11 original children's mental health services research articles, broadly organized in accordance with three themes (i.e., Improving Precision and Use of Service Data to Guide Policy and Practice, Implementation and Dissemination, and Preparing for Innovation), followed by an interview-style article with Bickman. Then follows a featured manuscript by Bickman himself, three invited commentaries, and a compilation of letters and notes in which colleagues reflect on his career and on their experiences of him. The introduction concludes with a few thoughts about the future of children's mental health services portended by the extraordinary scholarly contributions of Bickman and those who have been inspired by him.

Sustainment of Trauma-Focused and Evidence-Based Practices Following Learning Collaborative Implementation.

Given the need to develop and validate effective implementation models that lead to sustainable improvements, we prospectively examined changes in attitudes, behaviors, and perceived organizational support during and after statewide Community-Based Learning Collaboratives (CBLCs) promoting trauma-focused evidence-based practices (EBPs). Participants (N = 857; i.e., 492 clinicians, 218 brokers, and 139 senior leaders) from 10 CBLCs completed surveys pre- and post-CBLC; a subsample (n = 146) completed a follow-up survey approximately two years post-CBLC. Results indicated (a) medium, sustained increases in clinician-reported use of trauma-focused EBPs, (b) medium to large, sustained increases in perceived organizational support for trauma-focused EBPs, and (c) trivial to small, sustained increases in perceived organizational support for EBPs broadly. In contrast, clinician-reported overall attitudes towards EBPs decreased to a trivial degree pre- to post-CBLC, but then increased to a small, statistically significant degree from post-CBLC to follow-up. Notably, the degree of perceived improvements in organizational support for general and trauma-focused EBPs varied by professional role. Findings suggest the CBLC implementation strategies may both increase and sustain provider practices and organizational support towards EBPs, particularly those EBPs a CBLC explicitly targets.

Editors' welcome to Implementation Research and Practice.

This editorial introduces the new, online-only, open-access, international journal, Implementation Research and Practice (IRP), jointly published by SAGE and the Society for Implementation Research Collaboration (SIRC). IRP provides rapid publication of interdisciplinary research that advances the implementation in diverse contexts of effective approaches to assess, prevent, and treat mental health, substance use, or other addictive behaviors, in the general population or among those at risk or suffering from these disorders. IRP welcomes a variety of paper types designed to publish empirical, methodological, conceptual, and practical advances, as well as syntheses and perspectives intended to inspire new directions for future research. Consistent with the title of the journal, IRP welcomes manuscripts that stimulate, curate, and align research, policy, and practice.

Correction to: A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia.

.

A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia.

BACKGROUND: Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static. METHOD: Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia's public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised. RESULTS: Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048). CONCLUSIONS: System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.

A randomized trial to identify accurate and cost-effective fidelity measurement methods for cognitive-behavioral therapy: project FACTS study protocol.

BACKGROUND: This randomized trial will compare three methods of assessing fidelity to cognitive-behavioral therapy (CBT) for youth to identify the most accurate and cost-effective method. The three methods include self-report (i.e., therapist completes a self-report measure on the CBT interventions used in session while circumventing some of the typical barriers to self-report), chart-stimulated recall (i.e., therapist reports on the CBT interventions used in session via an interview with a trained rater, and with the chart to assist him/her) and behavioral rehearsal (i.e., therapist demonstrates the CBT interventions used in session via a role-play with a trained rater). Direct observation will be used as the gold-standard comparison for each of the three methods. METHODS/DESIGN: This trial will recruit 135 therapists in approximately 12 community agencies in the City of Philadelphia. Therapists will be randomized to one of the three conditions. Each therapist will provide data from three unique sessions, for a total of 405 sessions. All sessions will be audio-recorded and coded using the Therapy Process Observational Coding System for Child Psychotherapy-Revised Strategies scale. This will enable comparison of each measurement approach to direct observation of therapist session behavior to determine which most accurately assesses fidelity. Cost data associated with each method will be gathered. To gather stakeholder perspectives of each measurement method, we will use purposive sampling to recruit 12 therapists from each condition (total of 36 therapists) and 12 supervisors to participate in semi-structured qualitative interviews. DISCUSSION: Results will provide needed information on how to accurately and cost-effectively measure therapist fidelity to CBT for youth, as well as important information about stakeholder perspectives with regard to each measurement method. Findings will inform fidelity measurement practices in future implementation studies as well as in clinical practice. TRIAL REGISTRATION: NCT02820623 , June 3rd, 2016.

Redesigning community mental health services for urban children: Supporting schooling to promote mental health.

OBJECTIVE: This study examined a school- and home-based mental health service model, Links to Learning, focused on empirical predictors of learning as primary goals for services in high-poverty urban communities. METHOD: Teacher key opinion leaders were identified through sociometric surveys and trained, with mental health providers and parent advocates, on evidence-based practices to enhance children's learning. Teacher key opinion leaders and mental health providers cofacilitated professional development sessions for classroom teachers to disseminate 2 universal (Good Behavior Game, peer-assisted learning) and 2 targeted (Good News Notes, Daily Report Card) interventions. Group-based and home-based family education and support were delivered by mental health providers and parent advocates for children in kindergarten through 4th grade diagnosed with 1 or more disruptive behavior disorders. Services were Medicaid-funded through 4 social service agencies (N = 17 providers) in 7 schools (N = 136 teachers, 171 children) in a 2 (Links to Learning vs. services as usual) × 6 (pre- and posttests for 3 years) longitudinal design with random assignment of schools to conditions. Services as usual consisted of supported referral to a nearby social service agency. RESULTS: Mixed effects regression models indicated significant positive effects of Links to Learning on mental health service use, classroom observations of academic engagement, teacher report of academic competence and social skills, and parent report of social skills. Nonsignificant between-groups effects were found on teacher and parent report of problem behaviors, daily hassles, and curriculum-based measures. Effects were strongest for young children, girls, and children with fewer symptoms. CONCLUSION: Community mental health services targeting empirical predictors of learning can improve school and home behavior for children living in high-poverty urban communities.

From Practice to Evidence in Child Welfare: Model Specification and Fidelity Measurement of Team Decisionmaking.

Fidelity measurement methods have traditionally been used to develop and evaluate the effects of psychosocial treatments and, more recently, their implementation in practice. The fidelity measurement process can also be used to operationally define and specify components of emerging but untested practices outside the realm of conventional treatment. Achieving optimal fidelity measurement effectiveness (scientific validity and reliability) and efficiency (feasibility and relevance in routine care contexts) is challenging. The purpose of this paper is to identify strategies to address these challenges in child welfare system practices. To illustrate the challenges, and operational steps to address them, we present a case example using the "Team Decisionmaking" (TDM; Annie E. Casey Foundation) intervention. This intervention has potential utility for decreasing initial entry into and time spent in foster care and increasing rates of reunification and relative care. While promising, the model requires rigorous research to refine knowledge regarding the relationship between intervention components and outcomes-research that requires fidelity measurement. The intent of this paper is to illustrate how potentially generalizable steps for developing effective and efficient fidelity measurement methods can be used to more clearly define and test the effects of child welfare system practices.

Long-term outcomes for the Child STEPs randomized effectiveness trial: a comparison of modular and standard treatment designs with usual care.

OBJECTIVE: This article reports outcomes from the Child STEPs randomized effectiveness trial conducted over a 2-year period to gauge the longer term impact of protocol design on the effectiveness of evidence-based treatment procedures. METHOD: An ethnoracially diverse sample of 174 youths ages 7- 13 (N = 121 boys) whose primary clinical concerns involved diagnoses or clinical elevations related to anxiety, depression, or disruptive behavior were treated by community therapists randomly assigned to 1 of 3 conditions: (a) standard, which involved the use of 1 or more of 3 manualized evidence-based treatments, (b) modular, which involved a single modular protocol (Modular Approach to Treatment of Children With Anxiety, Depression, or Conduct Problems; MATCH) having clinical procedures similar to the standard condition but flexibly selected and sequenced using a guiding clinical algorithm, and (c) usual care. RESULTS: As measured with combined Child Behavior Checklist and Youth Self-Report Total Problems, Internalizing, and Externalizing scales, the rate of improvement for youths in the modular condition was significantly better than for those in usual care. On a measure of functional impairment (Brief Impairment Scale), no significant differences were found among the 3 conditions. Analysis of service utilization also showed no significant differences among conditions, with almost half of youths receiving some additional services in the 1st year after beginning treatment, and roughly one third of youths in the 2nd year. CONCLUSIONS: Overall, these results extend prior findings, supporting incremental benefits of MATCH over usual care over a 2-year period.

A review of treatment adherence measurement methods.

Fidelity measurement is critical for testing the effectiveness and implementation in practice of psychosocial interventions. Adherence is a critical component of fidelity. The purposes of this review were to catalogue adherence measurement methods and assess existing evidence for the valid and reliable use of the scores that they generate and the feasibility of their use in routine care settings. A systematic literature search identified articles published between 1980-2008 reporting studies of evidence-based psychosocial treatments for child or adult mental health problems, including mention of adherence or fidelity assessment. Coders abstracted data on the measurement methods and clinical contexts of their use. Three hundred forty-one articles were reviewed, in which 249 unique adherence measurement methods were identified. These methods assessed many treatment models, although more than half (59%) assessed cognitive behavioral treatments. The measurement methods were used in studies with diverse clientele and clinicians. The majority (71.5%) of methods were observational. Information about psychometric properties was reported for 35% of the measurement methods, but adherence-outcomes relationships were reported for only 10%. Approximately 1/3 of the measures were used in community-based settings. Many adherence measurement methods have been used in treatment research; however, little reliability and validity evidence exists for the use of these methods. That some methods were used in routine care settings suggests the feasibility of their use in practice; however, information about the operational details of measurement, scoring, and reporting is sorely needed to inform and evaluate strategies to embed fidelity measurement in implementation support and monitoring systems.

Taking Effective Treatments to Scale: Organizational Effects on Outcomes of Multisystemic Therapy for Youth with Co-occurring Substance Use.

A prospective multi-site study examined organizational climate and structure effects on the behavior and functioning of delinquent youth with and without co-occurring substance treated with an evidence-based treatment for serious antisocial behavior (i.e., Multisystemic Therapy). Participants were 1979 youth treated by 429 therapists across 45 provider organizations in North America. Results of Mixed Effects Regression Models showed some aspects of climate and structure had no effects, some had similar effects, and some had slightly differential and sometimes counter-intuitive effects on the outcomes of these youth. Implications are considered for research to increase the array and availability of effective treatments for youth with co-occurring substance use across service sectors.

Teacher consultation and coaching within mental health practice: classroom and child effects in urban elementary schools.

OBJECTIVE: To examine effects of a teacher consultation and coaching program delivered by school and community mental health professionals on change in observed classroom interactions and child functioning across one school year. METHOD: Thirty-six classrooms within 5 urban elementary schools (87% Latino, 11% Black) were randomly assigned to intervention (training + consultation/coaching) and control (training only) conditions. Classroom and child outcomes (n = 364; 43% girls) were assessed in the fall and spring. RESULTS: Random effects regression models showed main effects of intervention on teacher-student relationship closeness, academic self-concept, and peer victimization. Results of multiple regression models showed levels of observed teacher emotional support in the fall moderated intervention impact on emotional support at the end of the school year. CONCLUSIONS: Results suggest teacher consultation and coaching can be integrated within existing mental health activities in urban schools and impact classroom effectiveness and child adaptation across multiple domains.

Testing standard and modular designs for psychotherapy treating depression, anxiety, and conduct problems in youth: a randomized effectiveness trial.

CONTEXT: Decades of randomized controlled trials have produced separate evidence-based treatments for depression, anxiety, and conduct problems in youth, but these treatments are not often used in clinical practice, and they produce mixed results in trials with the comorbid, complex youths seen in practice. An integrative, modular redesign may help. OBJECTIVE: Standard/separate and modular/integrated arrangements of evidence-based treatments for depression, anxiety, and conduct problems in youth were compared with usual care treatment, with the modular design permitting a multidisorder focus and a flexible application of treatment procedures. DESIGN: Randomized effectiveness trial. SETTING: Ten outpatient clinical service organizations in Massachusetts and Hawaii. PARTICIPANTS: A total of 84 community clinicians were randomly assigned to 1 of 3 conditions for the treatment of 174 clinically referred youths who were 7 to 13 years of age (70% of these youths were boys, and 45% were white). The study was conducted during the period from January 12, 2005 to May 8, 2009. INTERVENTIONS: Standard manual treatment (59 youths [34% of the sample]; cognitive behavioral therapy for depression, cognitive behavioral therapy for anxiety, and behavioral parent training for conduct problems), modular treatment (62 youths [36%]; integrating the procedures of the 3 separate treatments), and usual care (53 youths [30%]). MAIN OUTCOME MEASURES: Outcomes were assessed using weekly youth and parent assessments. These assessments relied on a standardized Brief Problem Checklist and a patient-generated Top Problems Assessment (ie, the severity ratings on the problems that the youths and parents had identified as most important). We also conducted a standardized diagnostic assessment before and after treatment. RESULTS: Mixed effects regression analyses showed that modular treatment produced significantly steeper trajectories of improvement than usual care and standard treatment on multiple Brief Problem Checklist and Top Problems Assessment measures. Youths receiving modular treatment also had significantly fewer diagnoses than youths receiving usual care after treatment. In contrast, outcomes of standard manual treatment did not differ significantly from outcomes of usual care. CONCLUSIONS: The modular approach outperformed usual care and standard evidence-based treatments on multiple clinical outcome measures. The modular approach may be a promising way to build on the strengths of evidence-based treatments, improving their utility and effectiveness with referred youths in clinical practice settings. Trial Registration  clinicaltrials.gov Identifier: NCT01178554.

It's a Bird, It's A Plane, It's Fidelity Measurement In the Real World.

In psychotherapy research, fidelity instruments were originally developed as manipulation checks in experimental tests of treatment efficacy. The purposes of fidelity measurement are expanding as consumers, administrators, and payers seek to determine the extent to which the interventions purchased are actually received. Emerging purposes for fidelity measurement are described, as are challenges to developing a single instrument that can adequately meet multiple purposes, and that is both effective (psychometrically sound) and efficient (feasibly used in routine care). Examples are provided of efforts to balance these attributes of fidelity measurement, to measure fidelity at multiple levels of the practice context, and to index and evaluate the effects of additional program parameters on client outcomes in routine care.