Evaluating psychotherapist competence: Testing the generalizability of clinical competence assessments of graduate trainees.

Health service psychology (HSP) graduate programs are shifting from knowledge- to competency-based assessments of trainees' psychotherapy skills. This study used Generalizability Theory to test the dependability of psychotherapy competence assessments based on video observation of trainees. A 10-item rating form was developed from a collection of forms used by graduate programs (n = 102) in counseling and clinical psychology, and a review of the common factors research literature. This form was then used by 11 licensed psychologists to rate eight graduate trainees while viewing 129, approximately 5-min video clips from their psychotherapy sessions with clients (n = 22) at a graduate program's training clinic. Generalizability analyses were used to forecast how the number of raters and clients, and length of observation time impact the dependability of ratings in various rating designs. Raters were the primary source of error variance in ratings, with rater main effects (leniency bias) and dyadic effects (rater-target interactions) contributing 24% and 7% of variance, respectively. Variance due to segments (video clips) was also substantial, suggesting that therapist performance varies within the same counseling session. Generalizability coefficients (G) were highest for crossed rating designs and reached maximum levels (G > .50) after four raters watched each therapist working with three clients and observed 15 min per dyad. These findings suggest that expert raters show consensus in ratings even without rater training and only limited direct observation. Future research should investigate the validity of competence ratings as predictors of outcome. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Comparative clinical trials in psychotherapy: Have large effects been replicated?

AIMS: The purpose of this review is to examine the replication attempts of psychotherapy clinical trials for depression and anxiety. We focus specifically on replications of trials that exhibit large differences between psychotherapies. The replicability of these trials is especially important for meta-analysis, where the inclusion of false-positive trials can lead to erroneous conclusions about treatment efficacy. METHODS: Standard replication criteria were developed to distinguish direct from conceptual replication methodologies. Next, an exhaustive literature search was conducted for published meta-analyses of psychotherapy comparisons. Trials that exhibited large effects (d > 0.8) were culled from these meta-analyses. For each trial, a cited replication was conducted to determine if the trial had been subsequently replicated by either 'direct' or 'conceptual' methods. Finally, a broader search was conducted to examine the extent of replication efforts in the psychotherapy literature overall. RESULTS: In the meta-analytic search, a total of N = 10 meta-analyses met the inclusion criteria. From these meta-analyses, N = 12 distinct trials exhibited large effect sizes. The meta-analyses containing more than two large effect trials reported evidence for treatment superiority. A cited replication search yielded no direct replication attempts (N = 0) for the trials with large effects, and N = 4 conceptual replication attempts of average or above average quality. However, of these four attempts, only two partially corroborated the results from their original trial. CONCLUSION: Meta-analytic reviews are influenced by trials with large effects, and it is not uncommon for these reviews to contain several such trials. Since we find no evidence that trials with such large effects are directly replicable, treatment superiority conclusions from these reviews are highly questionable. To enhance the quality of clinical science, the development of authoritative replication criteria for clinical trials is needed. Moreover, quality benchmarks should be considered before trials are included in a meta-analysis, or replications are attempted.

Depression and family arguments: disentangling reciprocal effects for women and men.

BACKGROUND: Depression is a debilitating condition that affects the individual and the family. OBJECTIVE: This study sought to identify potential reciprocal influences between family arguments and depressive symptoms among clinically depressed patients over a 23-year span. METHODS: The present study employed a longitudinal, observational design with 424 depressed patients. Separate cross-lagged path models examined longitudinal associations for women and men over 23 years while adjusting for age, income, and marital and parental status. RESULTS: Among depressed men, more severe baseline depressive symptoms predicted more family arguments 10 years later. Among depressed women, more severe baseline depressive symptoms predicted fewer family arguments 1 year later, while more severe depressive symptoms at 10-year follow-up predicted more family arguments at 23-year follow-up. More family arguments predicted more severe depressive symptoms among women and men, with some variation in the time intervals of these associations. CONCLUSION: These findings suggest that while depressive symptoms may temporarily diminish family arguments among women, such symptoms were associated with more family arguments over longer time intervals. Moreover, family arguments put depressed men and women at risk for more severe depressive symptoms. These results support the use of screening for family arguments and interventions to help depressed individuals develop skills to manage interpersonal conflict.

The importance of problem-focused treatments: A meta-analysis of anxiety treatments.

One explanation for differences in treatment effectiveness for targeted symptoms is that more-effective treatments are more focused on patients' problems than are less-effective treatments. This conjecture was examined meta-analytically. Comparisons of two treatments of adults with anxiety disorders were included. Effect sizes for targeted symptoms, nontargeted symptoms, and global outcomes (e.g., quality of life and well-being) as well as the relative focus on patients' problems and researcher allegiance were coded. Metaregressions were conducted to predict effect sizes from (a) variables related to the focus on patients' problems and (b) researcher allegiance. For symptom measures, the relative focus on patients' problems predicted the relative effectiveness of the treatments, with the expectations created by explanation appearing more predictive than specific therapeutic actions focused on patients' problems, although conclusions about relative importance were difficult to determine given collinearity of predictors. Researcher allegiance also predicted the effects of the comparisons. For global outcomes, both the focus on patients' problems and researcher allegiance seemed to have smaller roles. A focus on patients' problems appears to be important for the reductions of symptoms. Clinical trials comparing treatments need to balance the focus on patients' problems and reduce researcher allegiance. (PsycINFO Database Record

In pursuit of truth: A critical examination of meta-analyses of cognitive behavior therapy.

OBJECTIVE: Three recent meta-analyses have made the claim, albeit with some caveats, that cognitive-behavioral treatments (CBT) are superior to other psychotherapies, in general or for specific disorders (e.g., social phobia). METHOD: The purpose of the present article was to examine four issues in meta-analysis that mitigate claims of CBT superiority: (a) effect size, power, and statistical significance, (b) focusing on disorder-specific symptom measures and ignoring other important indicators of psychological functioning, (c) problems inherent in classifying treatments provided in primary studies into classes of treatments, and (d) the inclusion of problematic trials, which biases the results, and the exclusion of trials that fail to find differences among treatments. RESULTS: When these issues are examined, the effects demonstrating the superiority of CBT are small, nonsignificant for the most part, limited to targeted symptoms, or are due to flawed primary studies. CONCLUSION: Meta-analytic evidence for the superiority of CBT in the three meta-analysis are nonexistent or weak.

The evidence for present-centered therapy as a treatment for posttraumatic stress disorder.

To examine the evidence for present-centered therapy (PCT) as a treatment for posttraumatic stress disorder (PTSD), 5 randomized clinical trials that compared PCT to an existing evidence-based treatment for PTSD were reviewed. A meta-analysis was used to estimate between-treatment differences on targeted measures, secondary measures, and dropout. PCT was found to be as efficacious as the comparison evidence-based treatment in 3 of the 5 trials, and in the 2 cases where a no-treatment condition was included, PCT was superior, with large effect sizes for targeted variables (d = 0.88, 0.74, and 1.27). When results were aggregated using meta-analysis, effects for PCT versus an evidence-based treatment for both targeted and secondary measures were small and nonsignificant (d = 0.13 and d = 0.09, respectively). As well, the dropout rate for PCT was significantly less than for the comparison evidence-based treatments (14.3% and 31.3%, respectively). It appears that PCT is an efficacious and acceptable treatment for PTSD.