RESUMO
Background: Emotions play a key role in psychotherapy. However, a problem with examining emotional states via self-report questionnaires is that the assessment usually takes place after the actual emotion has been experienced which might lead to biases and continuous human ratings are time and cost intensive. Using the AI-based software package Non-Verbal Behavior Analyzer (NOVA), video-based emotion recognition of arousal and valence can be applied in naturalistic psychotherapeutic settings. In this study, four emotion recognition models (ERM) each based on specific feature sets (facial: OpenFace, OpenFace-Aureg; body: OpenPose-Activation, OpenPose-Energy) were developed and compared in their ability to predict arousal and valence scores correlated to PANAS emotion scores and processes of change (interpersonal experience, coping experience, affective experience) as well as symptoms (depression and anxiety in HSCL-11). Materials and methods: A total of 183 patient therapy videos were divided into a training sample (55 patients), a test sample (50 patients), and a holdout sample (78 patients). The best ERM was selected for further analyses. Then, ERM based arousal and valence scores were correlated with patient and therapist estimates of emotions and processes of change. Furthermore, using regression models arousal and valence were examined as predictors of symptom severity in depression and anxiety. Results: The ERM based on OpenFace produced the best agreement to the human coder rating. Arousal and valence correlated significantly with therapists' ratings of sadness, shame, anxiety, and relaxation, but not with the patient ratings of their own emotions. Furthermore, a significant negative correlation indicates that negative valence was associated with higher affective experience. Negative valence was found to significantly predict higher anxiety but not depression scores. Conclusion: This study shows that emotion recognition with NOVA can be used to generate ERMs associated with patient emotions, affective experiences and symptoms. Nevertheless, limitations were obvious. It seems necessary to improve the ERMs using larger databases of sessions and the validity of ERMs needs to be further investigated in different samples and different applications. Furthermore, future research should take ERMs to identify emotional synchrony between patient and therapists into account.
RESUMO
Early change is an increasing area of investigation in psychotherapy research. In this study, we analyzed patterns of early change in interpersonal problems and their relationship to nonverbal synchrony and multiple outcome measures for the first time. We used growth mixture modeling to identify different latent classes of early change in interpersonal problems with 212 patients who underwent cognitive-behavioral treatment including interpersonal and emotion-focused elements. Furthermore, videotaped sessions were analyzed using motion energy analysis, providing values for the calculation of nonverbal synchrony to predict early change in interpersonal problems. The relationship between early change patterns and symptoms as well as overall change in interpersonal problems was also investigated. Three latent subgroups were identified: 1 class with slow improvement (n = 145), 1 class with fast improvement (n = 12), and 1 early deterioration class (n = 55). Lower levels of early nonverbal synchrony were significantly related to fast improvement in interpersonal change patterns. Furthermore, such patterns predicted treatment outcome in symptoms and interpersonal problems. The results suggest that nonverbal synchrony is associated with early change patterns in interpersonal problems, which are also predictive of treatment outcome. Limitations of the applied methods as well as possible applications in routine care are discussed. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
