How do cognitive behavioral therapy and interpersonal psychotherapy improve youth depression? Applying meta-analytic structural equation modeling to three decades of randomized trials.

Investigating the mechanisms through which psychotherapy brings about desired change can inform efforts to improve therapies. We applied meta-analytic structural equation modeling (MASEM) to assess putative change mechanisms for cognitive behavioral therapy (CBT) as mediators of youth depression treatment outcome. Then, we tested whether candidate mediators (CMs) showed evidence of treatment-specificity to CBT versus interpersonal psychotherapy (IPT). Literature searches identified 34 randomized trials (27 CBT, 6 IPT, 1 CBT/IPT, 3,868 participants, published 1982-2020) that measured seven CMs: negative cognition, social engagement, family functioning, pleasant activity engagement, problem solving, reframing, or avoidance. We assessed mediational pathways and whether posttreatment CMs mediated treatment effects on posttreatment depression symptoms, covarying pretreatment CMs, and symptoms. Treatment type was tested as a moderator of mediational pathways. Results show that negative cognition (24 trials) and pleasant activities (3 trials) mediated depression symptom outcome in CBT. Social engagement and family functioning showed stronger mediation in IPT (5 and 6 trials) than in CBT (14 and 13 trials). We conclude that negative cognition is a robust mediator of CBT but may not be treatment-specific; pleasant activities may also be a mediator of CBT. However, the lack of treatment or mediation effects involving problem solving and reframing contradicts CBT theory. In contrast, social and family mechanisms appear to be IPT-specific mediators. These conclusions are provisional due to small samples examining IPT and several CMs, limitations in CM measurement (i.e., posttreatment retrospective report), and assumptions of MASEM-and will need to be confirmed when more and better evidence accumulates. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Effects of a statewide pre-kindergarten program on children's achievement and behavior through sixth grade.

As state-funded pre-kindergarten (pre-K) programs expand, it is critical to investigate their short- and long-term effects. This article presents the results through sixth grade of a longitudinal randomized control study of the effects of a scaled-up, state-supported pre-K program. The analytic sample includes 2,990 children from low-income families who applied to oversubscribed pre-K program sites across the state and were randomly assigned to offers of admission or a wait list control. Data through sixth grade from state education records showed that the children randomly assigned to attend pre-K had lower state achievement test scores in third through sixth grades than control children, with the strongest negative effects in sixth grade. A negative effect was also found for disciplinary infractions, attendance, and receipt of special education services, with null effects on retention. The implications of these findings for pre-K policies and practices are discussed. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Dealing with effect size multiplicity in systematic reviews and meta-analyses.

Systematic reviews often encounter primary studies that report multiple effect sizes based on data from the same participants. These have the potential to introduce statistical dependency into the meta-analytic data set. In this paper, we provide a tutorial on dealing with effect size multiplicity within studies in the context of meta-analyses of intervention and association studies, recommending a three-step approach. The first step is to define the research question and consider the extent to which it mainly reflects interest in mean effect sizes (which we term a convergent approach) or an interest in exploring heterogeneity (which we term a divergent approach). A second step is to identify the types of multiplicities that appear in the initial database of effect sizes relevant to the research question, and we propose a categorization scheme to differentiate them. The third step is to select a strategy for dealing with each type of multiplicity. The researcher can choose between a reductionist meta-analytic approach, which is characterized by inclusion of a single effect size per study, and an integrative approach, characterized by inclusion of multiple effect sizes per study. We present an overview of available analysis strategies for dealing with effect size multiplicity within studies and provide recommendations intended to help researchers decide which strategy might be preferable in particular situations. Last, we offer caveats and cautions about addressing the challenges multiplicity poses for systematic reviews and meta-analyses.

Can Propensity Score Analysis Approximate Randomized Experiments Using Pretest and Demographic Information in Pre-K Intervention Research?

BACKGROUND: It is unclear whether propensity score analysis (PSA) based on pretest and demographic covariates will meet the ignorability assumption for replicating the results of randomized experiments. PURPOSE: This study applies within-study comparisons to assess whether pre-Kindergarten (pre-K) treatment effects on achievement outcomes estimated using PSA based on a pretest and demographic covariates can approximate those found in a randomized experiment. METHODS: Data-Four studies with samples of pre-K children each provided data on two math achievement outcome measures with baseline pretests and child demographic variables that included race, gender, age, language spoken at home, and mother's highest education. Research Design and Data Analysis-A randomized study of a pre-K math curriculum provided benchmark estimates of effects on achievement measures. Comparison samples from other pre-K studies were then substituted for the original randomized control and the effects were reestimated using PSA. The correspondence was evaluated using multiple criteria. RESULTS AND CONCLUSIONS: The effect estimates using PSA were in the same direction as the benchmark estimates, had similar but not identical statistical significance, and did not differ from the benchmarks at statistically significant levels. However, the magnitude of the effect sizes differed and displayed both absolute and relative bias larger than required to show statistical equivalence with formal tests, but those results were not definitive because of the limited statistical power. We conclude that treatment effect estimates based on a single pretest and demographic covariates in PSA correspond to those from a randomized experiment on the most general criteria for equivalence.

Assessing meta-regression methods for examining moderator relationships with dependent effect sizes: A Monte Carlo simulation.

Dependent effect sizes are ubiquitous in meta-analysis. Using Monte Carlo simulation, we compared the performance of 2 methods for meta-regression with dependent effect sizes-robust variance estimation (RVE) and 3-level modeling-with the standard meta-analytic method for independent effect sizes. We further compared bias-reduced linearization and jackknife estimators as small-sample adjustments for RVE and Wald-type and likelihood ratio tests for 3-level models. The bias in the slope estimates, width of the confidence intervals around those estimates, and empirical type I error and statistical power rates of the hypothesis tests from these different methods were compared for mixed-effects meta-regression analysis with one moderator either at the study or at the effect size level. All methods yielded nearly unbiased slope estimates under most scenarios, but as expected, the standard method ignoring dependency provided inflated type I error rates when testing the significance of the moderators. Robust variance estimation methods yielded not only the best results in terms of type I error rate but also the widest confidence intervals and the lowest power rates, especially when using the jackknife adjustments. Three-level models showed a promising performance with a moderate to large number of studies, especially with the likelihood ratio test, and yielded narrower confidence intervals around the slope and higher power rates than those obtained with the RVE approach. All methods performed better when the moderator was at the effect size level, the number of studies was moderate to large, and the between-studies variance was small. Our results can help meta-analysts deal with dependency in their data.

Fitting meta-analytic structural equation models with complex datasets.

A modification of the first stage of the standard procedure for two-stage meta-analytic structural equation modeling for use with large complex datasets is presented. This modification addresses two common problems that arise in such meta-analyses: (a) primary studies that provide multiple measures of the same construct and (b) the correlation coefficients that exhibit substantial heterogeneity, some of which obscures the relationships between the constructs of interest or undermines the comparability of the correlations across the cells. One component of this approach is a three-level random effects model capable of synthesizing a pooled correlation matrix with dependent correlation coefficients. Another component is a meta-regression that can be used to generate covariate-adjusted correlation coefficients that reduce the influence of selected unevenly distributed moderator variables. A non-technical presentation of these techniques is given, along with an illustration of the procedures with a meta-analytic dataset. Copyright © 2016 John Wiley & Sons, Ltd.

Can brief alcohol interventions for youth also address concurrent illicit drug use? results from a meta-analysis.

Brief interventions aimed at reducing alcohol use among youth may interrupt a possible developmental progression to more serious substance use if they can also affect the use of other illicit drugs. This meta-analysis examined the findings of recent research on the effects of brief alcohol interventions for adolescents and young adults on both alcohol and illicit drug use. Eligible studies were those using randomized or controlled quasi-experimental designs to examine the effects of brief alcohol interventions on illicit drug use outcomes among youth. A comprehensive literature search identified 30 eligible study samples that, on average, included participants age 17, with 57 % male participants and 56 % White youth. Three-level random-effects meta-analyses were used to estimate mean effect sizes and explore variability in effects. Overall, brief interventions targeting both alcohol and other drugs were effective in reducing both of these substances. However, the brief interventions that targeted only alcohol had no significant secondary effects on untargeted illicit drug use. The evidence from current research, therefore, shows modest beneficial effects on outcomes that are targeted by brief interventions for youth, but does not show that those effects generalize to untargeted illicit drug use outcomes.

Brief alcohol interventions for adolescents and young adults: a systematic review and meta-analysis.

This study reports findings from a meta-analysis summarizing the effectiveness of brief alcohol interventions for adolescents (age 11-18) and young adults (age 19-30). We identified 185 eligible study samples using a comprehensive literature search and synthesized findings using random-effects meta-analyses with robust standard errors. Overall, brief alcohol interventions led to significant reductions in alcohol consumption and alcohol-related problems among adolescents (g = 0.27 and g = 0.19) and young adults (g = 0.17 and g = 0.11). These effects persisted for up to 1 year after intervention and did not vary across participant demographics, intervention length, or intervention format. However, certain intervention modalities (e.g., motivational interviewing) and components (e.g., decisional balance, goal-setting exercises) were associated with larger effects. We conclude that brief alcohol interventions yield beneficial effects on alcohol-related outcomes for adolescents and young adults that are modest but potentially worthwhile given their brevity and low cost.

Identifying Baseline Covariates for Use in Propensity Scores: A Novel Approach Illustrated for a Non-randomized Study of Recovery High Schools.

There are many situations where random assignment of participants to treatment and comparison conditions may be unethical or impractical. This article provides an overview of propensity score techniques that can be used for estimating treatment effects in non-randomized quasi-experimental studies. After reviewing the logic of propensity score methods, we call attention to the importance of the strong ignorability assumption and its implications. We then discuss the importance of identifying and measuring a sufficient set of baseline covariates upon which to base the propensity scores and illustrate approaches to that task in the design of a study of recovery high schools for adolescents treated for substance abuse. One novel approach for identifying important covariates that we suggest and demonstrate is to draw on the predictor-outcome correlations compiled in meta-analyses of prospective longitudinal correlations.

The effects of CenteringPregnancy group prenatal care on gestational age, birth weight, and fetal demise.

We examined the effects of CenteringPregnancy group prenatal care versus individually delivered prenatal care on gestational age, birth weight, and fetal demise. We conducted a retrospective chart review and used propensity score matching to form a sample of 6,155 women receiving prenatal care delivered in a group or individual format at five sites in Tennessee. Compared to the matched group of women receiving prenatal care in an individual format, women in CenteringPregnancy group prenatal care had longer weeks of gestation (b = .35, 95 % CI [.29, .41]), higher birth weight in grams (b = 28.6, 95 % CI [4.8, 52.3]), lower odds of very low birth weight (OR = .21, 95 % CI [.06, .70]), and lower odds of fetal demise (OR = .12, 95 % CI [.02, .92]). Results indicated no evidence of differences in the odds of preterm birth or low birth weight for participants in group versus individual prenatal care. CenteringPregnancy group prenatal care had statistically and clinically significant beneficial effects on very low birth weight and fetal demise outcomes relative to traditional individually delivered prenatal care. Group prenatal care had statistically significant beneficial effects on gestational age and birth weight, although the effects were relatively small in clinical magnitude.

Effects of CenteringPregnancy group prenatal care on breastfeeding outcomes.

INTRODUCTION: The purpose of this study was to examine the effects of CenteringPregnancy group prenatal care on breastfeeding outcomes, relative to traditional prenatal care delivered in an individual format. METHODS: A quasi-experimental research design was conducted with 794 women receiving prenatal care delivered in a group or individual format at 4 sites in Tennessee. Propensity scores were used to create groups of women statistically matched on background demographics and medical history. Outcomes included breastfeeding at discharge and breastfeeding at postpartum follow-up. RESULTS: Compared with the matched comparison group of women receiving prenatal care in an individual format, women in CenteringPregnancy group prenatal care had significantly higher odds of any breastfeeding at discharge (odds ratio [OR], 2.08; 95% confidence interval [CI], 1.32-3.26; P < .001). Across the 4 sites, there were no consistent differences in the odds of any breastfeeding at follow-up or exclusive breastfeeding at discharge or postpartum follow-up. DISCUSSION: CenteringPregnancy group prenatal care may have beneficial effects on initial rates of breastfeeding relative to individually delivered care. However, there is not sufficient evidence to conclude that CenteringPregnancy group prenatal care has robust effects on exclusive breastfeeding at discharge or postpartum follow-up.

The comparative effectiveness of outpatient treatment for adolescent substance abuse: a meta-analysis.

Meta-analysis was used to synthesize research on the effects of outpatient treatment on substance use outcomes for adolescents with substance use disorders. An extensive literature search located 45 eligible experimental or quasi-experimental studies reporting 73 treatment-comparison group pairs, with many of the comparison groups also receiving some treatment. The first analysis examined 250 effect sizes for the substance use outcomes of adolescents receiving different types of treatment relative to the respective comparison groups. As a category, family therapy programs were found to be more effective than their comparison conditions, whereas no treatment programs were less effective. However, not all treatment types were compared with each other in the available research, making it difficult to assess the comparative effectiveness of the different treatments. To provide a more differentiated picture of the relative improvement in substance use outcomes for different treatments, a second analysis examined 311 pre-post effect sizes measuring changes in substance use for adolescents in the separate treatment and comparison arms of the studies. The adolescents in almost all types of treatment showed reductions in substance use. The greatest improvements were found for family therapy and mixed and group counseling. Longer treatment duration was associated with smaller improvements, but other treatment characteristics and participant characteristics had little relationship to the pre-post changes in substance use. Based on these findings family therapy is the treatment with the strongest evidence of comparative effectiveness, although most types of treatment appear to be beneficial in helping adolescents reduce their substance use.

Language abilities of children who stutter: a meta-analytical review.

PURPOSE: To identify, integrate, and summarize evidence from empirical studies of the language abilities of children who stutter (CWS) and children who do not stutter (CWNS). METHOD: Candidate studies were identified through electronic databases, the tables of contents of speech-language journals, and reference lists of relevant articles and literature reviews. The 22 included studies met the following criteria: studied both children who did and did not stutter between ages 2;0 (years;months) and 8;0, and reported norm-referenced language measures and/or measures from spontaneous language samples amenable to effect size calculation. Data were extracted using a coding manual and were assessed by application of general and specialized analytical software. Mean difference effect size was estimated using Hedges's g (Hedges, 1982). RESULTS: Findings indicated that CWS scored significantly lower than CWNS on norm-referenced measures of overall language (Hedges's g = -0.48), receptive (Hedges's g = -0.52) and expressive vocabulary (Hedges's g = -0.41), and mean length of utterance (Hedges's g = -0.23). CONCLUSIONS: Present findings were taken to suggest that children's language abilities are potentially influential variables associated with childhood stuttering.

School-based interventions for aggressive and disruptive behavior: update of a meta-analysis.

BACKGROUND: Research about the effectiveness of school-based psychosocial prevention programs for reducing aggressive and disruptive behavior was synthesized using meta-analysis. This work updated previous work by the authors and further investigated which program and student characteristics were associated with the most positive outcomes. METHODS: Two hundred forty-nine experimental and quasi-experimental studies of school-based programs with outcomes representing aggressive and/or disruptive behavior were obtained. Effect sizes and study characteristics were coded from these studies and analyzed. RESULTS: Positive overall intervention effects were found on aggressive and disruptive behavior and other relevant outcomes. The most common and most effective approaches were universal programs and targeted programs for selected/indicated children. The mean effect sizes for these types of programs represent a decrease in aggressive/disruptive behavior that is likely to be of practical significance to schools. Multicomponent comprehensive programs did not show significant effects and those for special schools or classrooms were marginal. Different treatment modalities (e.g., behavioral, cognitive, social skills) produced largely similar effects. Effects were larger for better-implemented programs and those involving students at higher risk for aggressive behavior. CONCLUSIONS: Schools seeking prevention programs may choose from a range of effective programs with some confidence that whatever they pick will be effective. Without the researcher involvement that characterizes the great majority of programs in this meta-analysis, schools might be well-advised to give priority to those that will be easiest to implement well in their settings.

The effects of school-based intervention programs on aggressive behavior: a meta-analysis.

Research on the effectiveness of school-based programs for preventing or reducing aggressive behavior was synthesized with a meta-analysis. Changes in aggressive behavior between pretest and posttest were analyzed for developmental patterns and characteristics associated with differential effects. Control groups showed little change in aggressive behavior, but there were significant reductions among intervention groups. Most studies were conducted on demonstration programs; the few studies of routine practice programs showed much smaller effects. Among demonstration programs, positive outcomes were associated with a variety of study, subject, and intervention characteristics. Most notably, higher risk youth showed greater reductions in aggressive behavior, poorly implemented programs produced smaller effects, and different types of programs were generally similar in their effectiveness, other things equal.