In pursuit of truth: A critical examination of meta-analyses of cognitive behavior therapy.

OBJECTIVE: Three recent meta-analyses have made the claim, albeit with some caveats, that cognitive-behavioral treatments (CBT) are superior to other psychotherapies, in general or for specific disorders (e.g., social phobia). METHOD: The purpose of the present article was to examine four issues in meta-analysis that mitigate claims of CBT superiority: (a) effect size, power, and statistical significance, (b) focusing on disorder-specific symptom measures and ignoring other important indicators of psychological functioning, (c) problems inherent in classifying treatments provided in primary studies into classes of treatments, and (d) the inclusion of problematic trials, which biases the results, and the exclusion of trials that fail to find differences among treatments. RESULTS: When these issues are examined, the effects demonstrating the superiority of CBT are small, nonsignificant for the most part, limited to targeted symptoms, or are due to flawed primary studies. CONCLUSION: Meta-analytic evidence for the superiority of CBT in the three meta-analysis are nonexistent or weak.

The effectiveness of evidence-based treatments for personality disorders when comparing treatment-as-usual and bona fide treatments.

OBJECTIVE: The purpose of Study 1 was to examine the relative efficacy of evidence-based treatments (EBTs) when compared to treatment-as-usual (TAU) for adults diagnosed with a personality disorder (PD). The purpose of Study 2 was to investigate the strength of the differences between bona fide psychotherapeutic treatments for PDs. METHOD: Two separate computerized searches were conducted of: (a) studies that directly compared an EBT with a TAU for treatment of PDs, or (b) studies that compared at least two bona fide treatments for PDs. Meta-analytic methods were used to estimate the effectiveness of the treatments when compared to one another and to model how various confounding variables impacted the results of this comparative research. RESULTS: A total of 30 studies (Study 1; N=1662) were included in the meta-analysis comparing EBTs to TAU. A total of 12 studies (Study 2; N=723) were included in the meta-analysis comparing bona fide treatments. Study 1 found that EBTs were superior to TAU, although the TAU conditions were not comparable in many respects (e.g., not psychotherapy, lacking supervision, lacking training, etc.) to the EBT and there was significant heterogeneity in the effects. Study 2 found that some bona fide treatments were superior to others.

Cognitive-behavioral therapy versus other therapies: redux.

Despite the evidence suggesting that all treatments intended to be therapeutic are equally efficacious, the conjecture that one form of treatment, namely cognitive-behavioral therapy (CBT), is superior to all other treatment persists. The purpose of the current study was to (a) reanalyze the clinical trials from an earlier meta-analysis that compared CBT to 'other therapies' for depression and anxiety (viz., Tolin, 2010) and (b) conduct a methodologically rigorous and comprehensive meta-analysis to determine the relative efficacy of CBT and bona fide non-CBT treatments for adult anxiety disorders. Although the reanalysis was consistent with the earlier meta-analysis' findings of small to medium effect sizes for disorder-specific symptom measures, the reanalysis revealed no evidence for the superiority of CBT for depression and anxiety for outcomes that were not disorder-specific. Following the reanalysis, a comprehensive anxiety meta-analysis that utilized a survey of 91 CBT experts from the Association of Behavioral and Cognitive Therapists (ABCT) to consensually identify CBT treatments was conducted. Thirteen clinical trials met the inclusion criteria. There were no differences between CBT treatments and bona fide non-CBT treatments across disorder-specific and non-disorder specific symptom measures. These analyses, in combination with previous meta-analytic findings, fail to provide corroborative evidence for the conjecture that CBT is superior to bona fide non-CBT treatments.

Evidence-based treatments for depression and anxiety versus treatment-as-usual: a meta-analysis of direct comparisons.

OBJECTIVE: The aim of this study was to examine the relative efficacy of evidence-based treatments (EBTs) versus treatment-as-usual (TAU) in routine care for anxiety and depression in adults. METHOD: A computerized search of studies that directly compared an EBT with a TAU was conducted. Meta-analytic methods were used to estimate effectiveness of EBTs relative to TAU and to model how various confounding variables impacted the results of this comparative research. RESULTS: A total of 14 studies were included in the final meta-analysis. There was significant heterogeneity in the TAU conditions, which ranged from unknown and/or minimal mental health treatment to psychotherapeutic interventions provided by trained professionals. Although the effect for EBT vs. TAU was significantly greater than zero, the effect for EBT vs. TAUs that were psychotherapeutic interventions was not statistically different from zero. CONCLUSIONS: Heterogeneity of TAU conditions in this meta-analysis highlight the importance of clarifying the research questions being asked when investigating and drawing conclusions from EBT-TAU comparisons. Researchers need to clarify if they are comparing an EBT to psychotherapeutic services in routine care or to minimal mental health services. Extant research on EBT versus TAU reveals that there is insufficient evidence to recommend the transportation of EBTs for anxiety and depression to routine care, particularly when the routine care involves psychotherapeutic services.

Determining what works in the treatment of PTSD.

Many researchers accept that trauma-focused treatments are superior to non-trauma focused treatments for Post-Traumatic Stress Disorder (PTSD). However, Benish, Imel, and Wampold (2008) recently published a meta-analysis of clinical trials directly comparing 'bona fide' PTSD treatments that failed to reject the null hypothesis that PTSD treatments are similarly effective. They concluded that the results of previous meta-analysis may have been influenced by several confounds, including the use of control treatments, to make conclusions about the relative efficacy of specific PTSD treatments. Ehlers et al. (2010) claim that the selection procedures of the Benish et al. meta-analysis were biased and cite results from individual studies and previous meta-analyses that suggest trauma-focused psychological treatments are superior to non-trauma focused treatments. We first offer a review and justification of the coding criteria and procedure used in Benish et al. In addition, we discuss the appropriateness of utilizing treatments designed to control for non-specifics or common factors such as 'supportive therapy' for determining the relative efficacy of specific PTSD treatments. Finally, we note several additional confounds, such as therapist effects, allegiance, and alteration of legitimate protocols, in PTSD research and describe conceptual problems involved in the classification scheme used to determine the "trauma focus" of interventions, which lead to inappropriate conclusions about what works in the treatment of PTSD.