The evolution of feedback: Toward a multicultural orientation.

There have been great strides in psychology regarding diversity, equity, inclusion, and multicultural competence, but a need remains to translate these values into actionable practices in psychotherapy. While the case has been made that measurement-based care is an evidence-based intervention that improves outcomes and reduces dropouts (de Jong et al., 2021) and recently that it provides a transparent collaborative process to engage clients in treatment (Boswell et al., 2023), it has not been widely considered as a methodology for multicultural competence. We trace the evolution of what was once called "patient-focused research" (Lambert, 2001) and identify a significant change in recent writings to include important clinical and collaborative processes, a transition from a strictly normative or nomothetic understanding of the value of feedback to an appreciation of its communicative or idiographic processes. We propose that systematic client feedback promotes a "multicultural orientation" (Owen, 2013) at the individual therapist-client level and that client responses to outcome and process measures can foster cultural humility and create cultural opportunities (Hook et al., 2017) to address marginalization and other sociocultural factors relevant to treatment. Using one system to illustrate what is possible for all feedback approaches, we present client examples that demonstrate an integration of a multicultural orientation. We suggest that systematic client feedback can provide a structure to address diversity, marginalization, and privilege in psychotherapy. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

The person of the therapist: therapists' personal characteristics as predictors of alliance and treatment outcomes.

Objective Studies examining the unique contribution that the therapist has on outcomes (i.e., "therapist effects") have increased in recent years. Therapist effects are believed to occur via how the therapist's interpersonal and intrapersonal qualities influence the therapeutic relationship, which in turn influences outcomes. The current study focused on the extent to which professional self-doubt, counseling self-efficacy, and humility are therapist qualities that influence treatment outcomes. Methods: Data were collected from 46 therapists at a multi-site community behavioral health organization. Therapists completed measures of therapist characteristics. Therapists' responses were matched with de-identified archival client data (N = 1, 817) that contained an outcome and alliance measure administered every session. Multilevel modeling was used to determine the extent to which therapists' personal characteristics predicted client outcomes. Results: Results of the unconditional model for the alliance measure indicated a lack of growth in alliance scores across treatment. Approximately 5% of the variance in rate of growth for treatment outcome was between therapists. When controlling for the effects of counseling self-efficacy, professional self-doubt was marginally significant. When counseling-self-efficacy was removed from the model, professional self-doubt was no longer marginally significant. Conclusion: The findings highlight the complexity of therapist effects and the need for future research on this topic.

Measurement-based care in integrated health care: A randomized clinical trial.

Introduction: Many suggest that the next step for integrated care is widespread implementation of measurement-based care (MBC). Although the measures most associated with MBC are standardized, no randomized clinical trial has demonstrated their use to improve psychotherapeutic outcomes with embedded behavioral health providers in integrated care. Two evidence-based MBC systems have been studied in a variety of behavioral health environments, but neither system has been investigated in integrated health care. Addressing this gap in the literature, the present study evaluated the use of MBC, specifically the Partners for Change Outcome Management System, in three integrated care sites. Method: Using a randomized design within routine care, treatment as usual (TAU; n = 133) was compared using the Outcome Rating Scale (ORS) and Patient Health Questionnaire-9 (PHQ-9) with a feedback condition (n = 147) in which behavioral health providers had access to patient-generated outcome (ORS only) and alliance information at each session. Results: Patients in the feedback condition demonstrated significantly more improvement than those in the TAU condition posttreatment on the ORS. Patients in the feedback condition also achieved significantly more clinically significant change as measured by both the ORS and PHQ-9. Feedback condition patients also attended significantly more sessions and dropped out significantly less that TAU patients. Discussion: Although our findings need to be replicated, this study offers evidence that the improved outcomes and reduced dropouts associated with MBC in traditional behavioral health centers also occur in integrated care settings. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Benchmarking Community-Based Couple Therapy: Considering Measurement Reactivity.

Couple therapy has been shown to be effective in randomized clinical trials; however, results from naturalistic couple therapy have been less consistent. This study utilized a benchmarking approach to compare the effectiveness of couple therapy in a community-based setting with findings from efficacy treatments, such as treatment within randomized clinical trials. The current study is the largest couple therapy sample published to date (N = 3,347 couples). Clients in couple therapy were asked to provide initial and weekly ratings of symptomology on the Outcome Questionnaire (OQ-45.2). We found that treatment effect sizes found at community clinics were smaller than efficacy studies (i.e., the benchmark). However, when taking into account measurement reactivity, the effect sizes were comparable. This is the first benchmarking study for community-based couple therapy, allowing for meaningful comparisons and understanding of outcomes in real-world couple therapy. Implications for the field are offered in terms of evaluating community-based psychotherapy studies with benchmarking for couple therapy. Results of this study provide clinicians and researchers a way to meaningfully compare couple therapy outcomes, accounting for differences in community-based practices and randomized clinical trials. This benchmark also underscores the impact of measurement sensitivity, an issue commonly overlooked in psychotherapy research and practice.

Se ha demostrado que la terapia de pareja es eficaz en los ensayos clínicos aleatorizados, sin embargo, los resultados de la terapia de pareja naturalista han sido menos consecuentes. Este estudio utilizó un enfoque comparativo para comparar la eficacia de la terapia de pareja en un entorno basado en la comunidad con hallazgos de los tratamientos de eficacia, como los tratamientos dentro de los ensayos clínicos aleatorizados El presente estudio es la muestra más grande de terapia de pareja publicado hasta la fecha (N = 3347 parejas). Se pidió a los pacientes en terapia de pareja que proporcionen calificaciones iniciales y semanales de la sintomatología en el cuestionario para evaluación de resultados (OQ-45.2).Descubrimos que los tamaños del efecto de los tratamientos hallados en las clínicas comunitarias fueron más pequeños que los de los estudios de eficacia (p. ej.: el estudio comparativo). Sin embargo, cuando se tomó en cuenta la reactividad a la medición, los tamaños del efecto fueron comparables. Este es el primer estudio comparativo para la terapia de pareja basada en la comunidad, el cual permite comparaciones significativas y comprensión de los resultados en la terapia de pareja en el mundo real. Se ofrecen las implicancias para el área en cuanto a la evaluación de los estudios de la psicoterapia basada en la comunidad con evaluación comparativa de la terapia de pareja. Los resultados de este estudio proporcionan a los clínicos y a los investigadores una manera de comparar significativamente los resultados de la terapia de pareja, así como de explicar las diferencias en las prácticas basadas en la comunidad y en los ensayos clínicos aleatorizados. Esta evaluación comparativa también subraya el efecto de la sensibilidad a la medición, una cuestión comúnmente pasada por alto en la investigación y la práctica de la psicoterapia.

Psychotherapy for depressed youth in poverty: Benchmarking outcomes in a public behavioral health setting.

Research demonstrating the effectiveness of treatment with youth from low socioeconomic backgrounds is limited. To address this limitation, we evaluated pre-post psychotherapy treatment outcomes with youth presenting with depression-related diagnoses (N = 469) at a public behavioral health agency after they implemented a systematic client feedback approach as a quality improvement strategy. Clients were ethnically diverse youth at or under the poverty line. Treatment outcome was measured with the Outcome Rating Scale (Miller, Duncan, Brown, Sparks, & Claud, 2003) and the Child Outcome Rating Scale (Duncan, Sparks, Miller, Bohanske, & Claud, 2006). Benchmark methodology was used to compare effect size estimates to those achieved in randomized clinical trials. Average treatment effect sizes for the public behavioral health depression samples of children and adolescents (d = 1.39 and d = 1.69, respectively) were clinically superior to a waitlist benchmark drawn from clinical trials of youth depression, and clinically equivalent to a treatment benchmark drawn from youth depression clinical trials. Findings demonstrate that mental health services for depressed youth in poverty across an agency can be effective, and systematic client feedback may be a useful strategy to improve treatment outcomes. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Health-related quality of life and the physician-patient alliance: a preliminary investigation of ultra-brief, real-time measures for primary care.

PURPOSE: Many psychometrically sound measures exist but feasibility makes real-time use difficult. This study validates two ultra-brief, patient-rated instruments, the Wellness Rating Scale (WRS) and the Provider Alliance Scale (PAS). METHODS: The WRS and the EuroQol visual analogue scale (EQ VAS) were completed by patients in a primary care practice (n = 97) and a non-clinical sample of graduate students (n = 122). The WRS and the Patient-Completed Health Outcome Measures Information System-Global 10 (PROMIS) were completed by patients in a primary care setting (n = 305). The WRS and PROMIS were also administered to graduate students (n = 158). The PAS and the Patient Physician Working Alliance were administered to a primary care sample of 40 and a retrospective sample of students (n = 228). RESULTS: The WRS generated reliable scores, with coefficient alphas ranging from .83 to .91. Bivariate correlations between the WRS and the EQ VAS (r = .55-.75) and PROMIS (r = .64-73) indicate moderate-to-strong concurrent validity. The larger coefficients were with patient samples. Construct validity was evidenced by higher levels of distress for chronic conditions as well as for clinical samples. The PAS achieved an alpha of .94 for the primary care sample and .87 for the retrospective sample and bivariate correlations (r = .61-.72) indicate moderate-to-strong evidence of concurrent validity. CONCLUSIONS: The WRS and PAS demonstrate sufficient reliability and validity to move to the next phase of research: a randomized clinical trial comparing the use of real-time feedback from the two measures to treatment as usual targeting outcomes of chronic disease patients.

Client feedback in China: A randomized clinical trial in a college counseling center.

Although client feedback has been demonstrated to improve psychotherapy outcomes in over a dozen randomized clinical trials, no studies to date have investigated the feedback effect outside of the United States or Europe. This study examined the impact of a client feedback intervention, the Partners for Change Outcome Management System, in a college counseling center in Wuhan, China (N = 186). Using a randomized design within routine care, treatment as usual (TAU; n = 85) was compared with a feedback condition (n = 101) in which therapists had access to client-generated outcome and alliance information at each session. Clients in the feedback condition demonstrated significantly greater improvement than those in the TAU condition at posttreatment. Not-on-track (n = 60) clients also demonstrated significantly more improvement at 6 times the rate of reliable change compared with the TAU condition. Survival analysis revealed that 66.7% of the clients in the feedback condition achieved reliable and clinically significant change after a median of 4 sessions whereas 57.0% of the clients in the TAU condition achieved reliable and clinically significant change after a median of 6 sessions. Alliance scores improved significantly more across treatment and were higher at posttreatment in the feedback condition. Although preliminary, this study suggests that the positive effects of improved outcomes and increased efficiency associated with systematic client feedback can also occur in a college counseling setting in China. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Patient feedback as a quality improvement strategy in an acute care, inpatient unit: An investigation of outcome and readmission rates.

High psychiatric readmission rates continue while evidence suggests that care is not perceived by patients as "patient centered." Research has focused on aftercare strategies with little attention to the inpatient treatment itself as an intervention to reduce readmission rates. Quality improvement strategies based on patient-centered care may offer an alternative. We evaluated outcomes and readmission rates using a benchmarking methodology with a naturalistic data set from an inpatient psychiatric facility (N = 2,247) that used a quality-improvement strategy called systematic patient feedback. Benchmarks were constructed using randomized clinical trials (RCTs) from inpatient treatment for depression, RCTs from patient feedback in outpatient settings, and national data on psychiatric hospital readmission rates. A systematic patient feedback system, the Partners for Change Outcome Management System (PCOMS), was used. Overall pre-post effect sizes were d = 1.33 and d = 1.38 for patients diagnosed with a mood disorder. These effect sizes were statistically equivalent to RCT benchmarks for feedback and depression. Readmission rates were 6.1% (30 days), 9.5% (60 days), and 16.4% (180 days), all lower than national benchmarks. We also found that patients who achieved clinically significant treatment outcomes were less likely to be readmitted. We tentatively suggest that a focus on real-time patient outcomes as well as care that is "patient centered" may provide a path toward lower readmission rates in addition to other evidence-based strategies after discharge. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

COMPASS for Hope: Evaluating the Effectiveness of a Parent Training and Support Program for Children with ASD.

Despite the growing number of studies that demonstrate the importance of empowering parents with knowledge and skills to act as intervention agents for their children with autism spectrum disorder (ASD), there are limited examples of parent-mediated interventions that focus on problem behaviors. Additionally, access to ASD-trained clinicians and research supported delivery options for families in rural areas is severely limited. COMPASS for Hope (C-HOPE) is an 8-week parent intervention program that was developed with the option of telehealth or face-to-face delivery. Parents who received C-HOPE intervention reported a reduction in parenting stress and an increase in competence. Parents also reported significant reductions in child behavior problems, both when compared to pre-intervention levels and to a waitlist control condition.

Casting a wider net in behavioral health screening in primary care: a preliminary study of the Outcome Rating Scale.

Introduction The integration of behavioral health services into primary care has led to enhanced use of brief screening measures to identify mental health problems. Although useful, such instruments are largely symptom based and diagnosis specific. This narrow focus can potentially limit the identification of broader social or relational distress in patients that affect medical outcomes, as well as present feasibility challenges using a multi-measure approach in identifying mental health comorbidities. METHOD: This exploratory study of adult primary care patients compared an ultra-brief, and widely used measure of global distress across life functioning, the Outcome Rating Scale (ORS), with the Patient Health Questionnaire (PHQ-9 and PHQ-2). RESULTS: Correlations between the ORS and the PHQ-9 and PHQ-2 indicated agreement between the measures in classifying patients, and the ORS identified significantly more patients in the clinical range. Discussion Although results are preliminary, the ORS may cast a wider net in identifying patients with significant distress in primary care.

Does psychotherapy work with school-aged youth? A meta-analytic examination of moderator variables that influence therapeutic outcomes.

The present study is a quantitative synthesis of the available literature to investigate the efficacy of psychotherapy for children's mental health outcomes. In particular, this study focuses on potential moderating variables-study design, treatment, client, and therapist characteristics-that may influence therapeutic outcomes for youth but have not been thoroughly accounted for in prior meta-analytic studies. An electronic search of relevant databases resulted in 190 unpublished and published studies that met criteria for inclusion in the analysis. Effect sizes differed by study design. Pre-post-test designs resulted in absolute magnitudes of treatment effects ranging from |-0.02| to |-0.76| while treatment versus control group comparison designs resulted in absolute magnitudes of treatment effects ranging from |-0.14| to |-2.39|. Changes in youth outcomes larger than 20% were found, irrespective of study design, for outcomes focused on psychosomatization (29% reduction), school attendance (25% increase), and stress (48% reduction). The magnitude of changes after psychotherapy ranged from 6% (externalizing problems) to 48% (stress). Several moderator variables significantly influenced psychotherapy treatment effect sizes, including frequency and length of treatment as well as treatment format. However, results did not support the superiority of a single type of intervention for most outcomes. Implications for therapy with school-aged youth and future research are discussed.

Good-enough level and dose-effect models: Variation among outcomes and therapists.

OBJECTIVE: The current study examined the good-enough level (GEL) and dose-effect model on three outcome variables: well-being, symptom distress, and life functioning, while accounting for therapist effects. The dose-effect model assumes the rate of change is consistent across clients, and the GEL model assumes that the rate of change will vary according to the total length of treatment. METHOD: The sample included 13,664 clients who completed 2-100 sessions of therapy. RESULTS: The GEL model was a better fit to the data compared to the dose-effect model for all outcomes. There were fewer changes in life functioning compared to well-being and symptom distress for clients. There were significant therapist effects for changes in symptom distress and life functioning, but not for the rates of change in well-being. CONCLUSION: Therapists should consider the rates of change for their own clients as well as by outcome.

The Partners for Change Outcome Management System (PCOMS) revisiting the client's frame of reference.

Despite overall psychotherapy efficacy (Lambert, 2013), many clients do not benefit (Reese, Duncan, Bohanske, Owen, & Minami, 2014), dropouts are a problem (Swift & Greenberg, 2012), and therapists vary significantly in success rates (Baldwin & Imel, 2013), are poor judges of negative outcomes (Chapman et al., 2012), and grossly overestimate their effectiveness (Walfish, McAlister, O'Donnell, & Lambert, 2012). Systematic client feedback offers 1 solution (Duncan, 2014). Several feedback systems have emerged (Castonguay, Barkham, Lutz, & McAleavey, 2013), but only 2 have randomized clinical trial support and are included in the Substance Abuse and Mental Health Administration's National Registry of Evidence-Based Programs and Practices: The Outcome Questionnaire-45.2 System (Lambert, 2010) and the Partners for Change Outcome Management System (PCOMS; Duncan, 2012). This article presents the current status of PCOMS, the psychometrics of the PCOMS measures, its empirical support, and its clinical and training applications. Future directions and implications of PCOMS research, training, and practice are detailed. Finally, we propose that systematic feedback offers a way, via large-scale data collection, to reprioritize what matters to psychotherapy outcome, reclaim our empirically validated core values and identity, and change the conversation from a medical model dominated discourse to a more scientific, relational perspective.

Using telepsychology to provide a group parenting program: A preliminary evaluation of effectiveness.

Telepsychology offers the potential to reach rural and underserved children and families with mental health concerns. The current study evaluated the effects of using videoconferencing technology to deliver an evidence-based parenting program, the Group Triple P Positive Parenting Program (Group Triple P; Turner, Markie-Dadds, & Sanders, 2002), with families who had a child experiencing behavioral problems. Using a pre/post design, families (N = 13) from low socioeconomic backgrounds in Kentucky completed the Group Triple P via a videoconferencing delivery format. A benchmarking strategy (Weersing & Hamilton, 2005) indicated that treatment effect sizes for the videoconferencing format were generally comparable to treatment effect sizes for Group Triple P studies conducted in-person. Specifically, child behavior and parenting outcomes were similar across delivery formats. Implications of the study's findings and future directions for telepsychology research and practice with underserved families and children are discussed.

Efficacy of client feedback in group psychotherapy with soldiers referred for substance abuse treatment.

This study investigated whether routine monitoring of client progress, often called "client feedback," via an abbreviated version of the Partners for Change Outcome Management System (PCOMS) resulted in improved outcomes for soldiers receiving group treatment at an Army Substance Abuse Outpatient Treatment Program (ASAP). Participants (N = 263) were active-duty male and female soldiers randomized into a group feedback condition (n = 137) or a group treatment-as-usual (TAU) condition (n = 126). Results indicated that clients in the feedback condition achieved significantly more improvement on the outcome rating scale (d = 0.28), higher rates of clinically significant change, higher percentage of successful ratings by both clinicians and commanders, and attended significantly more sessions compared to the TAU condition. Despite a reduced PCOMS protocol and a limited duration of intervention, preliminary results suggest that the benefits of client feedback appear to extend to group psychotherapy with clients in the military struggling with substance abuse.

Validation of a self-administered computerized system to detect cognitive impairment in older adults.

There is increasing interest in the development of economical and accurate approaches to identifying persons in the community who have mild, undetected cognitive impairments. Computerized assessment systems have been suggested as a viable approach to identifying these persons. The validity of a computerized assessment system for identification of memory and executive deficits in older individuals was evaluated in the current study. Volunteers (N = 235) completed a 3-hr battery of neuropsychological tests and a computerized cognitive assessment system. Participants were classified as impaired (n = 78) or unimpaired (n = 157) on the basis of the Mini Mental State Exam, Wechsler Memory Scale-III and the Trail Making Test (TMT), Part B. All six variables (three memory variables and three executive variables) derived from the computerized assessment differed significantly between groups in the expected direction. There was also evidence of temporal stability and concurrent validity. Application of computerized assessment systems for clinical practice and for identification of research participants is discussed in this article.

Benchmarking outcomes in a public behavioral health setting: feedback as a quality improvement strategy.

OBJECTIVE: The purpose of this study was to evaluate the effectiveness of a large public behavioral health (PBH) agency serving only clients at or below the federal poverty level that had implemented continuous outcome feedback as a quality improvement strategy. METHOD: The authors investigated the post treatment outcomes of 5,168 individuals seeking treatment for a broad range of diagnoses who completed at least 2 psychotherapy sessions. The Outcome Rating Scale (ORS; Duncan, 2011; Miller & Duncan, 2004) was used to measure outcomes. Clients had a mean age of 36.7 years and were predominantly female (60.7%) and White (67.8%), with 17.7% being Hispanic, 9.3% being African American, and 2.8% being Native American. Forty-six percent were diagnosed with depression, mood, and anxiety disorders; 18.8% were diagnosed with substance abuse disorders; and 14.4% were diagnosed with bipolar disorder and schizophrenia. A subset of clients with a primary diagnosis of a depressive disorder was compared to treatment efficacy benchmarks derived from clinical trials of major depression. Given that the PBH agency had also implemented an outcome management system, the total sample was also compared to benchmarks derived from clinical trials of continuous outcome feedback. RESULTS: Treatment effect sizes of psychotherapy delivered at the PBH agency were comparable to effect size estimates of clinical trials of depression and feedback. Observed effect sizes were smaller, however, when compared to feedback benchmarks that used the ORS. CONCLUSIONS: Services to the poor and disabled can be effective, and continuous outcome feedback may be a viable means both to improve outcomes and to narrow the gap between research and practice.

Using client feedback in psychotherapy from an interpersonal process perspective.

The process of monitoring treatment outcome, also known as "client feedback," is increasingly becoming a recommended practice for psychotherapy. One concern, however, is how to integrate such a process into the work that psychotherapists typically do. Three clinical examples are presented, illustrating how a client feedback system can be used in conjunction with a specific theoretical framework, interpersonal process therapy (Teyber, 2006). The examples highlight that client feedback not only can be of minimal disruption to the psychotherapy process, but may also offer the potential to augment a clinician's approach to helping. Theoretical and research support are provided for each example.

The influence of demand characteristics and social desirability on clients' ratings of the therapeutic alliance.

OBJECTIVE: To examine demand characteristics, social desirability on clients' rating of working alliance using the Session Rating Scale (SRS; Miller, Duncan, & Johnson, 2000). METHOD: Clients (N = 102) at two sites were randomly assigned to one of three alliance feedback conditions: (a) IF--SRS completed in presence of therapist and the results discussed immediately afterward; (b) Next Session Feedback--SRS completed alone and results discussed next session; or (c) No Feedback--SRS completed alone and results not available to therapist. Clients completed the SRS for the first three sessions of treatment. RESULTS: No statistically significant differences in SRS scores across the feedback conditions were found. Additionally, the analysis showed that SRS scores were not correlated with a measure of social desirability but were correlated with an established alliance measure. CONCLUSIONS: The results indicate that alliance scores were not inflated due to the presence of a therapist or knowing that the scores would be observed by the therapist.

Alliance in action: a new measure of clients' perceptions of therapists' alliance activity.

We developed a new measure, Alliance in Action (AiA), which assesses clients' perceptions of therapist behavior related to fostering and maintaining the alliance. Clients (N=170) were treated by 42 therapists. All clients were currently in therapy. The results of a factor analysis revealed four subscales to the AiA, which reflected clients' perceptions of their therapists' behavior to monitor the therapeutic relationship, the goals for therapy, and progress towards client goals. A fourth subscale emerged that reflected clients' perceptions of therapist avoidance of eliciting feedback. The AiA subscales demonstrated alphas above .70 and they were associated with client-rated alliance and session outcomes in univariate correlation tests. In multilevel models, three of the four subscales were associated with alliance and session outcomes. The AiA may be helpful in understanding how the therapeutic alliance functions in therapy.