Impact of racial microaggressions in psychotherapy vignettes with african american clients: An experimental analogue design.

Mental health researchers have focused on promoting culturally sensitive clinical care (Herman et al., 2007; Whaley & Davis, 2007), emphasizing the need to understand how biases may impact client well-being. Clients report that their therapists commit racial microaggressions-subtle, sometimes unintentional, racial slights-during treatment (Owen et al., 2014). Yet, existing studies often rely on retrospective evaluations of clients and cannot establish the causal impact of varying ambiguity of microaggressions on clients. This study uses an experimental analogue design to examine offensiveness, emotional reactions, and evaluations of the interaction across three distinct levels of microaggression statements: subtle, moderate, and overt. We recruited 158 adult African American participants and randomly assigned them to watch a brief counseling vignette. We found significant differences between the control and three microaggression statements on all outcome variables. We did not find significant differences between the microaggression conditions. This study, in conjunction with previous correlational research, highlights the detrimental impact of microaggressions within psychotherapy, regardless of racially explicit content. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Automating the assessment of multicultural orientation through machine learning and natural language processing.

Recent scholarship has highlighted the value of therapists adopting a multicultural orientation (MCO) within psychotherapy. A newly developed performance-based measure of MCO capacities exists (MCO-performance task [MCO-PT]) in which therapists respond to video-based vignettes of clients sharing culturally relevant information in therapy. The MCO-PT provides scores related to the three aspects of MCO: cultural humility (i.e., adoption of a nonsuperior and other-oriented stance toward clients), cultural opportunities (i.e., seizing or making moments in session to ask about clients' cultural identities), and cultural comfort (i.e., therapists' comfort in cultural conversations). Although a promising measure, the MCO-PT relies on labor-intensive human coding. The present study evaluated the ability to automate the scoring of the MCO-PT transcripts using modern machine learning and natural language processing methods. We included a sample of 100 participants (n = 613 MCO-PT responses). Results indicated that machine learning models were able to achieve near-human reliability on the average across all domains (Spearman's ρ = .75, p < .0001) and opportunity (ρ = .81, p < .0001). Performance was less robust for cultural humility (ρ = .46, p < .001) and was poorest for cultural comfort (ρ = .41, p < .001). This suggests that we may be on the cusp of being able to develop machine learning-based training paradigms that could allow therapists opportunities for feedback and deliberate practice of some key therapist behaviors, including aspects of MCO. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Improving the quality of counseling and clinical supervision in opioid treatment programs: how can technology help?

BACKGROUND: The opioid epidemic has resulted in expanded substance use treatment services and strained the clinical workforce serving people with opioid use disorder. Focusing on evidence-based counseling practices like motivational interviewing may be of interest to counselors and their supervisors, but time-intensive adherence tasks like recording and feedback are aspirational in busy community-based opioid treatment programs. The need to improve and systematize clinical training and supervision might be addressed by the growing field of machine learning and natural language-based technology, which can promote counseling skill via self- and supervisor-monitoring of counseling session recordings. METHODS: Counselors in an opioid treatment program were provided with an opportunity to use an artificial intelligence based, HIPAA compliant recording and supervision platform (Lyssn.io) to record counseling sessions. We then conducted four focus groups-two with counselors and two with supervisors-to understand the integration of technology with practice and supervision. Questions centered on the acceptability of the clinical supervision software and its potential in an OTP setting; we conducted a thematic coding of the responses. RESULTS: The clinical supervision software was experienced by counselors and clinical supervisors as beneficial to counselor training, professional development, and clinical supervision. Focus group participants reported that the clinical supervision software could help counselors learn and improve motivational interviewing skills. Counselors said that using the technology highlights the value of counseling encounters (versus paperwork). Clinical supervisors noted that the clinical supervision software could help meet national clinical supervision guidelines and local requirements. Counselors and clinical supervisors alike talked about some of the potential challenges of requiring session recording. CONCLUSIONS: Implementing evidence-based counseling practices can help the population served in OTPs; another benefit of focusing on clinical skills is to emphasize and hold up counselors' roles as worthy. Machine learning technology can have a positive impact on clinical practices among counselors and clinical supervisors in opioid treatment programs, settings whose clinical workforce continues to be challenged by the opioid epidemic. Using technology to focus on clinical skill building may enhance counselors' and clinical supervisors' overall experiences in their places of work.

Mental Health Counseling From Conversational Content With Transformer-Based Machine Learning.

Importance: Use of asynchronous text-based counseling is rapidly growing as an easy-to-access approach to behavioral health care. Similar to in-person treatment, it is challenging to reliably assess as measures of process and content do not scale. Objective: To use machine learning to evaluate clinical content and client-reported outcomes in a large sample of text-based counseling episodes of care. Design, Setting, and Participants: In this quality improvement study, participants received text-based counseling between 2014 and 2019; data analysis was conducted from September 22, 2022, to November 28, 2023. The deidentified content of messages was retained as a part of ongoing quality assurance. Treatment was asynchronous text-based counseling via an online and mobile therapy app (Talkspace). Therapists were licensed to provide mental health treatment and were either independent contractors or employees of the product company. Participants were self-referred via online sign-up and received services via their insurance or self-pay and were assigned a diagnosis from their health care professional. Exposure: All clients received counseling services from a licensed mental health clinician. Main Outcomes and Measures: The primary outcomes were client engagement in counseling (number of weeks), treatment satisfaction, and changes in client symptoms, measured via the 8-item version of Patient Health Questionnaire (PHQ-8). A previously trained, transformer-based, deep learning model automatically categorized messages into types of therapist interventions and summaries of clinical content. Results: The total sample included 166 644 clients treated by 4973 therapists (20 600 274 messages). Participating clients were predominantly female (75.23%), aged 26 to 35 years (55.4%), single (37.88%), earned a bachelor's degree (59.13%), and were White (61.8%). There was substantial variability in intervention use and treatment content across therapists. A series of mixed-effects regressions indicated that collectively, interventions and clinical content were associated with key outcomes: engagement (multiple R = 0.43), satisfaction (multiple R = 0.46), and change in PHQ-8 score (multiple R = 0.13). Conclusions and Relevance: This quality improvement study found associations between therapist interventions, clinical content, and client-reported outcomes. Consistent with traditional forms of counseling, higher amounts of supportive counseling were associated with improved outcomes. These findings suggest that machine learning-based evaluations of content may increase the scale and specificity of psychotherapy research.

The Silent Treatment?: Changes in patient emotional expression after silence.

Psychotherapy can be an emotionally laden conversation, where both verbal and non-verbal interventions may impact the therapeutic process. Prior research has postulated mixed results in how clients emotionally react following a silence after the therapist is finished talking, potentially due to studying a limited range of silences with primarily qualitative and self-report methodologies. A quantitative exploration may illuminate new findings. Utilizing research and automatic data processing from the field of linguistics, we analysed the full range of silence lengths (0.2 to 24.01 seconds), and measures of emotional expression - vocally encoded arousal and emotional valence from the works spoken - of 84 audio recordings Motivational Interviewing sessions. We hypothesized that both the level and the variance of client emotional expression would change as a function of silence length, however, due to the mixed results in the literature the direction of emotional change was unclear. We conducted a multilevel linear regression to examine how the level of client emotional expression changed across silence length, and an ANOVA to examine the variability of client emotional expression across silence lengths. Results indicated in both analyses that as silence length increased, emotional expression largely remained the same. Broadly, we demonstrated a weak connection between silence length and emotional expression, indicating no persuasive evidence that silence leads to client emotional processing and expression.

Facilitating dyadic synchrony in psychotherapy sessions: Systematic review and meta-analysis.

Objective: This paper highlights the facilitation of dyadic synchrony as a core psychotherapist skill that occurs at the non-verbal level and underlies many other therapeutic methods. We define dyadic synchrony, differentiate it from similar constructs, and provide an excerpt illustrating dyadic synchrony in a psychotherapy session. Method: We then present a systematic review of 17 studies that have examined the associations between dyadic synchrony and psychotherapy outcomes. We also conduct a meta-analysis of 8 studies that examined whether there is more synchrony between clients and therapists than would be expected by chance. Results: Weighted box score analysis revealed that the overall association of synchrony and proximal as well as distal outcomes was neutral to mildly positive. The results of the meta-analysis indicated that real client-therapist dyad pairs exhibited synchronized behavioral patterns to a much greater extent than a sample of randomly paired people who did not actually speak. Conclusion: Our discussion revolves around how synchrony can be facilitated in a beneficial way, as well as situations in which it may not be beneficial. We conclude with training implications and therapeutic practices.

Threat alert: The effect of outliers on the alliance-outcome correlation.

Meta-analyses have established the alliance as the most robust predictor of outcome in psychotherapy. A growing number of studies have evaluated potential threats to the conclusion that alliance is a causal factor in psychotherapy. One potential threat that has not been systematically examined is the possibility that the alliance-outcome association is driven by low alliance outliers. We examined the influence of removing low alliance outliers on the alliance-outcome association using data drawn from two large-scale, naturalistic psychotherapy data sets (Ns = 1,052; 11,029). These data sets differed in setting (university counseling center, community mental health center), country (United States and Canada), alliance measure (four-item Working Alliance Inventory Short Form Revised, 10-item Session Rating Scale), and outcome measure (Counseling Center Assessment of Psychological Symptoms-34, Outcome Questionnaire-45). We examined the impact of treating outliers in five different ways: retaining them, removing values three or two standard deviations from the mean, and winsorizing values three or two standard deviations from the mean. We also examined the effect of outliers after disaggregating alliance ratings into within-therapist and between-therapist components. The alliance-outcome correlation and the proportion of variance in posttest outcomes explained by alliance when controlling for pretest outcomes were similar regardless of how low alliance outliers were treated (change in r ≤ .04, change in R² ≤ 1%). Results from the disaggregation were similar. Thus, it appears that the alliance-outcome association is not an artifact of the influence of low alliance outliers. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Knowledge and Attitudes Toward an Artificial Intelligence-Based Fidelity Measurement in Community Cognitive Behavioral Therapy Supervision.

To capitalize on investments in evidence-based practices, technology is needed to scale up fidelity assessment and supervision. Stakeholder feedback may facilitate adoption of such tools. This evaluation gathered stakeholder feedback and preferences to explore whether it would be fundamentally feasible or possible to implement an automated fidelity-scoring supervision tool in community mental health settings. A partially mixed, sequential research method design was used including focus group discussions with community mental health therapists (n = 18) and clinical leadership (n = 12) to explore typical supervision practices, followed by discussion of an automated fidelity feedback tool embedded in a cloud-based supervision platform. Interpretation of qualitative findings was enhanced through quantitative measures of participants' use of technology and perceptions of acceptability, appropriateness, and feasibility of the tool. Initial perceptions of acceptability, appropriateness, and feasibility of automated fidelity tools were positive and increased after introduction of an automated tool. Standard supervision was described as collaboratively guided and focused on clinical content, self-care, and documentation. Participants highlighted the tool's utility for supervision, training, and professional growth, but questioned its ability to evaluate rapport, cultural responsiveness, and non-verbal communication. Concerns were raised about privacy and the impact of low scores on therapist confidence. Desired features included intervention labeling and transparency about how scores related to session content. Opportunities for asynchronous, remote, and targeted supervision were particularly valued. Stakeholder feedback suggests that automated fidelity measurement could augment supervision practices. Future research should examine the relations among use of such supervision tools, clinician skill, and client outcomes.

Automated evaluation of psychotherapy skills using speech and language technologies.

With the growing prevalence of psychological interventions, it is vital to have measures which rate the effectiveness of psychological care to assist in training, supervision, and quality assurance of services. Traditionally, quality assessment is addressed by human raters who evaluate recorded sessions along specific dimensions, often codified through constructs relevant to the approach and domain. This is, however, a cost-prohibitive and time-consuming method that leads to poor feasibility and limited use in real-world settings. To facilitate this process, we have developed an automated competency rating tool able to process the raw recorded audio of a session, analyzing who spoke when, what they said, and how the health professional used language to provide therapy. Focusing on a use case of a specific type of psychotherapy called "motivational interviewing", our system gives comprehensive feedback to the therapist, including information about the dynamics of the session (e.g., therapist's vs. client's talking time), low-level psychological language descriptors (e.g., type of questions asked), as well as other high-level behavioral constructs (e.g., the extent to which the therapist understands the clients' perspective). We describe our platform and its performance using a dataset of more than 5000 recordings drawn from its deployment in a real-world clinical setting used to assist training of new therapists. Widespread use of automated psychotherapy rating tools may augment experts' capabilities by providing an avenue for more effective training and skill improvement, eventually leading to more positive clinical outcomes.

Feature Fusion Strategies for End-to-End Evaluation of Cognitive Behavior Therapy Sessions.

Cognitive Behavioral Therapy (CBT) is a goal-oriented psychotherapy for mental health concerns implemented in a conversational setting. The quality of a CBT session is typically assessed by trained human raters who manually assign pre-defined session-level behavioral codes. In this paper, we develop an end-to-end pipeline that converts speech audio to diarized and transcribed text and extracts linguistic features to code the CBT sessions automatically. We investigate both word-level and utterance-level features and propose feature fusion strategies to combine them. The utterance level features include dialog act tags as well as behavioral codes drawn from another well-known talk psychotherapy called Motivational Interviewing (MI). We propose a novel method to augment the word-based features with the utterance level tags for subsequent CBT code estimation. Experiments show that our new fusion strategy outperforms all the studied features, both when used individually and when fused by direct concatenation. We also find that incorporating a sentence segmentation module can further improve the overall system given the preponderance of multi-utterance conversational turns in CBT sessions.

How do you feel? Using natural language processing to automatically rate emotion in psychotherapy.

Emotional distress is a common reason for seeking psychotherapy, and sharing emotional material is central to the process of psychotherapy. However, systematic research examining patterns of emotional exchange that occur during psychotherapy sessions is often limited in scale. Traditional methods for identifying emotion in psychotherapy rely on labor-intensive observer ratings, client or therapist ratings obtained before or after sessions, or involve manually extracting ratings of emotion from session transcripts using dictionaries of positive and negative words that do not take the context of a sentence into account. However, recent advances in technology in the area of machine learning algorithms, in particular natural language processing, have made it possible for mental health researchers to identify sentiment, or emotion, in therapist-client interactions on a large scale that would be unattainable with more traditional methods. As an attempt to extend prior findings from Tanana et al. (2016), we compared their previous sentiment model with a common dictionary-based psychotherapy model, LIWC, and a new NLP model, BERT. We used the human ratings from a database of 97,497 utterances from psychotherapy to train the BERT model. Our findings revealed that the unigram sentiment model (kappa = 0.31) outperformed LIWC (kappa = 0.25), and ultimately BERT outperformed both models (kappa = 0.48).

Automated rating of patient and physician emotion in primary care visits.

OBJECTIVE: Train machine learning models that automatically predict emotional valence of patient and physician in primary care visits. METHODS: Using transcripts from 353 primary care office visits with 350 patients and 84 physicians (Cook, 2002 [1], Tai-Seale et al., 2015 [2]), we developed two machine learning models (a recurrent neural network with a hierarchical structure and a logistic regression classifier) to recognize the emotional valence (positive, negative, neutral) (Posner et al., 2005 [3]) of each utterance. We examined the agreement of human-generated ratings of emotional valence with machine learning model ratings of emotion. RESULTS: The agreement of emotion ratings from the recurrent neural network model with human ratings was comparable to that of human-human inter-rater agreement. The weighted-average of the correlation coefficients for the recurrent neural network model with human raters was 0.60, and the human rater agreement was also 0.60. CONCLUSIONS: The recurrent neural network model predicted the emotional valence of patients and physicians in primary care visits with similar reliability as human raters. PRACTICE IMPLICATIONS: As the first machine learning-based evaluation of emotion recognition in primary care visit conversations, our work provides valuable baselines for future applications that might help monitor patient emotional signals, supporting physicians in empathic communication, or examining the role of emotion in patient-centered care.

Determining the therapist's contribution in therapist multicultural competence process and outcome.

Efforts to help therapists improve their multicultural competence (MCC) rely on measures that can distinguish between different levels of competence. MCC is often assessed by asking clients to rate their experiences with their therapists. However, differences in client ratings of therapist MCC do not necessarily provide information about the relative performance of therapists and can be influenced by other factors including the client's own characteristics. In this study, we used a repeated measures design of 8,497 observations from 1,458 clients across 35 therapists to clarify the proportion of variability in MCC ratings attributed to the therapist versus the client and better understand the extent that an MCC measure detects therapist differences. Overall, we found that a small amount of variability in MCC ratings was attributed to the therapist (2%) and substantial amount attributed to the client (70%). These findings suggest that our measure of MCC primarily detected differences at the client level versus therapist level, indicating that therapist MCC scores were largely dependent on the client. Clinical implications and recommendations for future MCC research and measurement are discussed. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Can a computer detect interpersonal skills? Using machine learning to scale up the Facilitative Interpersonal Skills task.

Objective: Therapist interpersonal skills are foundational to psychotherapy. However, assessment is labor intensive and infrequent. This study evaluated if machine learning (ML) tools can automatically assess therapist interpersonal skills. Method: Data were drawn from a previous study in which 164 undergraduate students (i.e., not clinical trainees) completed the Facilitative Interpersonal Skills (FIS) task. This task involves responding to video vignettes depicting interpersonally challenging moments in psychotherapy. Trained raters scored the responses. We used an elastic net model on top of a term frequency-inverse document frequency representation to predict FIS scores. Results: Models predicted FIS total and item-level scores above chance (rhos = .27-.53, ps < .001), achieving 31-60% of human reliability. Models explained 13-24% of the variance in FIS total and item-level scores on a held out set of data (R2), with the exception of the two items most reliant on vocal cues (verbal fluency, emotional expression), for which models explained ≤1% of variance. Conclusion: ML may be a promising approach for automating assessment of constructs like interpersonal skill previously coded by humans. ML may perform best when the standardized stimuli limit the "space" of potential responses (vs. naturalistic psychotherapy) and when models have access to the same data available to raters (i.e., transcripts).

Machine learning and natural language processing in psychotherapy research: Alliance as example use case.

Artificial intelligence generally and machine learning specifically have become deeply woven into the lives and technologies of modern life. Machine learning is dramatically changing scientific research and industry and may also hold promise for addressing limitations encountered in mental health care and psychotherapy. The current paper introduces machine learning and natural language processing as related methodologies that may prove valuable for automating the assessment of meaningful aspects of treatment. Prediction of therapeutic alliance from session recordings is used as a case in point. Recordings from 1,235 sessions of 386 clients seen by 40 therapists at a university counseling center were processed using automatic speech recognition software. Machine learning algorithms learned associations between client ratings of therapeutic alliance exclusively from session linguistic content. Using a portion of the data to train the model, machine learning algorithms modestly predicted alliance ratings from session content in an independent test set (Spearman's ρ = .15, p < .001). These results highlight the potential to harness natural language processing and machine learning to predict a key psychotherapy process variable that is relatively distal from linguistic content. Six practical suggestions for conducting psychotherapy research using machine learning are presented along with several directions for future research. Questions of dissemination and implementation may be particularly important to explore as machine learning improves in its ability to automate assessment of psychotherapy process and outcome. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Coregulation of therapist and client emotion during psychotherapy.

OBJECTIVE: Close interpersonal relationships are fundamental to emotion regulation. Clinical theory suggests that one role of therapists in psychotherapy is to help clients regulate emotions, however, if and how clients and therapists serve to regulate each other's emotions has not been empirically tested. Emotion coregulation - the bidirectional emotional linkage of two people that promotes emotional stability - is a specific, temporal process that provides a framework for testing the way in which therapists' and clients' emotions may be related on a moment to moment basis in clinically relevant ways. METHOD: Utilizing 227 audio recordings from a relationally oriented treatment (Motivational Interviewing), we estimated continuous values of vocally encoded emotional arousal via mean fundamental frequency. We used dynamic systems models to examine emotional coregulation, and tested the hypothesis that each individual's emotional arousal would be significantly associated with fluctuations in the other's emotional state over the course of a psychotherapy session. RESULTS: Results indicated that when clients became more emotionally labile over the course of the session, therapists became less so. When changes in therapist arousal increased, the client's tendency to become more aroused during session slowed. Alternatively, when changes in client arousal increased, the therapist's tendency to become less aroused slowed.

Provider Contributions to Disparities in Mental Health Care.

OBJECTIVE: Disparities in diagnosis of mental health problems and in access to treatment among racial-ethnic groups are apparent across different behavioral conditions, particularly in the quality of treatment for depression. This study aimed to determine how much disparities differ across providers. METHODS: Bayesian mixed-effects models were used to estimate whether disparities in patient adherence to antidepressant medication (N=331,776) or psychotherapy (N=275,095) were associated with specific providers. Models also tested whether providers who achieved greater adherence to treatment, on average, among non-Hispanic white patients than among patients from racial-ethnic minority groups attained lower disparities and whether the percentage of patients from racial-ethnic minority groups in a provider caseload was associated with disparities. RESULTS: Disparities in adherence to both antidepressant medication and psychotherapy were associated with the provider. Provider performance with non-Hispanic white patients was negatively correlated with provider-specific disparities in adherence to psychotherapy but not to antidepressants. A higher proportion of patients from racial-ethnic minority groups in a provider's caseload was associated with lower adherence among non-Hispanic white patients, lower disparities in adherence to psychotherapy, and greater disparities in adherence to antidepressant medication. CONCLUSIONS: Adherence to depression treatment among a provider's patients from racial-ethnic minority groups was related to adherence among that provider's non-Hispanic white patients, but evidence also suggested provider-specific disparities. Efforts among providers to decrease disparities might focus on improving the general skill of providers who treat more patients from racial-ethnic minority groups as well as offering culturally based training to providers with notable disparities.

The Structure of Competence: Evaluating the Factor Structure of the Cognitive Therapy Rating Scale.

The Cognitive Therapy Rating Scale (CTRS) is an observer-rated measure of cognitive behavioral therapy (CBT) treatment fidelity. Although widely used, the factor structure and psychometric properties of the CTRS are not well established. Evaluating the factorial validity of the CTRS may increase its utility for training and fidelity monitoring in clinical practice and research. The current study used multilevel exploratory factor analysis to examine the factor structure of the CTRS in a large sample of therapists (n = 413) and observations (n = 1,264) from community-based CBT training. Examination of model fit and factor loadings suggested that three within-therapist factors and one between-therapist factor provided adequate fit and the most parsimonious and interpretable factor structure. The three within-therapist factors included items related to (a) session structure, (b) CBT-specific skills and techniques, and (c) therapeutic relationship skills, although three items showed some evidence of cross-loading. All items showed moderate to high loadings on the single between-therapist factor. Results support continued use of the CTRS and suggest factors that may be a relevant focus for therapists, trainers, and researchers.

Detecting conversation topics in primary care office visits from transcripts of patient-provider interactions.

OBJECTIVE: Amid electronic health records, laboratory tests, and other technology, office-based patient and provider communication is still the heart of primary medical care. Patients typically present multiple complaints, requiring physicians to decide how to balance competing demands. How this time is allocated has implications for patient satisfaction, payments, and quality of care. We investigate the effectiveness of machine learning methods for automated annotation of medical topics in patient-provider dialog transcripts. MATERIALS AND METHODS: We used dialog transcripts from 279 primary care visits to predict talk-turn topic labels. Different machine learning models were trained to operate on single or multiple local talk-turns (logistic classifiers, support vector machines, gated recurrent units) as well as sequential models that integrate information across talk-turn sequences (conditional random fields, hidden Markov models, and hierarchical gated recurrent units). RESULTS: Evaluation was performed using cross-validation to measure 1) classification accuracy for talk-turns and 2) precision, recall, and F1 scores at the visit level. Experimental results showed that sequential models had higher classification accuracy at the talk-turn level and higher precision at the visit level. Independent models had higher recall scores at the visit level compared with sequential models. CONCLUSIONS: Incorporating sequential information across talk-turns improves the accuracy of topic prediction in patient-provider dialog by smoothing out noisy information from talk-turns. Although the results are promising, more advanced prediction techniques and larger labeled datasets will likely be required to achieve prediction performance appropriate for real-world clinical applications.