ABSTRACT
Background: Fidelity measurement is critical for developing, evaluating, and implementing evidence-based treatments (EBTs). However, traditional fidelity measurement tools are often not feasible for community-based settings. We developed a short fidelity rating form for the Collaborative Problem Solving (CPS) approach from an existing manualized coding system that requires extensive training. We examined the reliability and accuracy of this short form when completed by trained observers, untrained observers, and self-reporting providers to evaluate multiple options for reducing barriers to fidelity measurement in community-based settings. Methods: Community-based treatment providers submitted recordings of youth service sessions in which they did, or did not, use CPS. For 60 recordings, we compared short-form fidelity ratings assigned by trained observers and untrained observers to those provided by trained observers on the manualized coding system. For 141 recordings, we compared providers' self-reported fidelity on the short form to ratings provided by trained observers on the manualized coding system and examined providers' accuracy as a function of their global fidelity. Results & Conclusions: The short form was reliable and accurate for trained observers. An assigned global integrity score and a calculated average of component scores on the short form, but not component scores themselves, were reliable and accurate for observers who had CPS expertise but no specific training on rating CPS fidelity. When providers self-reported fidelity on the short form, their global integrity score was a reliable estimate of their CPS integrity; however, providers with better CPS fidelity were most accurate in their self-reports. We discuss the costs and benefits of these more pragmatic fidelity measurement options in community-based settings.
Developing brief, easy-to-use, and reliable tools to measure how well providers deliver evidence-based treatments (EBTs) in community clinical settings is critical to ensure the benefits of EBTs. However, reliable tools are often too time-consuming and not feasible to use in community settings because they require independent observers to receive intensive training on a coding system and to observe live or recorded treatment sessions for reliable and accurate evaluation. This paper describes steps we took to develop a more practical measure of how well providers deliver one EBT, Collaborative Problem Solving (CPS), based on a previously validated measure, to explore whether the quality of the measure can be maintained while reducing the need for training independent observers and the need for recording treatment sessions. This work contributes to the growing efforts of developing more pragmatic fidelity measures and introduces a new tool, the CPS Practice Integrity Form (CPS-PIF), as a promising measure for community-based clinical settings using CPS.
ABSTRACT
OBJECTIVE: This study examines the influence of therapist and youth characteristics on post-discharge outcomes from intensive in-home therapy. METHOD: Data for 1,416 youth and 412 therapists were obtained from a behavioral health services provider. The Huber-White method was used to account for nested data; ordered logistic regression was employed to assess outcomes. RESULTS: Therapist gender and employment stability were significantly associated with youth outcomes. The likelihood of an undesirable outcome was significantly less for cases with female therapists. CONCLUSION: Findings underscore the need for additional study concerning the impact of therapist characteristics and stability on youth outcomes, and to improve the understanding of the relationship between the two. Future studies in these areas would advance social work practice in family-based treatment programs.
ABSTRACT
This study compares outcomes for behaviorally troubled children receiving intensive in-home therapy (IIHT) and those receiving residential care (RC). Propensity score matching is used to identify matched pairs of youth (n = 786) with equivalent propensity for IIHT. The majority of pretreatment differences between the IIHT and RC groups are eliminated following matching. Logistic regression is then conducted on outcome differences at 1 year postdischarge. Results show that IIHT recipients had a greater tendency (.615) toward living with family, making progress in school, not experiencing trouble with the law, and placement stability compared with RC youth (.558; p < .10). This suggests that IIHT is at least as effective for achieving positive outcomes. Given IIHT's reduced restrictiveness and cost, intensive in-home services should be the preferred treatment over RC in most cases.
