Personalized treatment selection in routine care: Integrating machine learning and statistical algorithms to recommend cognitive behavioral or psychodynamic therapy.

Objective: This study aims at developing a treatment selection algorithm using a combination of machine learning and statistical inference to recommend patients' optimal treatment based on their pre-treatment characteristics. Methods: A disorder-heterogeneous, naturalistic sample of N = 1,379 outpatients treated with either cognitive behavioral therapy or psychodynamic therapy was analyzed. Based on a combination of random forest and linear regression, differential treatment response was modeled in the training data (n = 966) to indicate each individual's optimal treatment. A separate holdout dataset (n = 413) was used to evaluate personalized recommendations. Results: The difference in outcomes between patients treated with their optimal vs. non-optimal treatment was significant in the training data, but non-significant in the holdout data (b = -0.043, p = .280). However, for the 50% of patients with the largest predicted benefit of receiving their optimal treatment, the average percentage of change on the BSI in the holdout data was 52.6% for their optimal and 38.4% for their non-optimal treatment (p = .017; d = 0.33 [0.06, 0.61]). Conclusion: A treatment selection algorithm based on a combination of ML and statistical inference might improve treatment outcome for some, but not all outpatients and could support therapists' clinical decision-making.

[Testing the Psychometric Properties of the Helping Alliance Questionnaire (HAQ) in Different Health Care Settings]. / Psychometrische Überprüfung des Helping Alliance Questionnaire (HAQ) in unterschiedlichen Versorgungssettings.

Introduction The Helping Alliance Questionnaire developed by Luborsky was psychometricly examined for the first time and translated into German by Bassler et al. in the mid-1990s. It consists of 11 Items, which are summarized to the scales "relation to the therapist" and "satisfaction with therapeutic outcome". HAQ is now one of the most used instruments to measure therapeutic alliance. The goal of this study was to test the psychometric properties based on three large patient samples in different treatment settings. Material and methods Analyses were conducted based on 2 samples of patients in inpatient psychosomatic/psychotherapeutic rehabilitation (n=655, n=2494) and one sample in outpatient psychotherapy (n=1477). Exploratory factor analyses and for verification confirmatory factor analyses were applied. Furthermore reliability and validity analysis were conducted. Results The 2-factorial structure found in literature was replicated with an increasing stability at the end of treatment. Item 2 and 3 had inconsistent factor loadings at different points of measurement and study. Reliability and validity indices were satisfying to good. The fit of the model, on the other hand, was less satisfying and suggests a solution without Item 2 and 3. For reasons of content as well as for reasons of dissemination, it is pleaded for the maintenance of the existing item assignments in the HAQ for the present.

The Success of Linear Bootstrapping Models: Decision Domain-, Expertise-, and Criterion-Specific Meta-Analysis.

The success of bootstrapping or replacing a human judge with a model (e.g., an equation) has been demonstrated in Paul Meehl's (1954) seminal work and bolstered by the results of several meta-analyses. To date, however, analyses considering different types of meta-analyses as well as the potential dependence of bootstrapping success on the decision domain, the level of expertise of the human judge, and the criterion for what constitutes an accurate decision have been missing from the literature. In this study, we addressed these research gaps by conducting a meta-analysis of lens model studies. We compared the results of a traditional (bare-bones) meta-analysis with findings of a meta-analysis of the success of bootstrap models corrected for various methodological artifacts. In line with previous studies, we found that bootstrapping was more successful than human judgment. Furthermore, bootstrapping was more successful in studies with an objective decision criterion than in studies with subjective or test score criteria. We did not find clear evidence that the success of bootstrapping depended on the decision domain (e.g., education or medicine) or on the judge's level of expertise (novice or expert). Correction of methodological artifacts increased the estimated success of bootstrapping, suggesting that previous analyses without artifact correction (i.e., traditional meta-analyses) may have underestimated the value of bootstrapping models.

Feedback and therapist effects in the context of treatment outcome and treatment length.

OBJECTIVE: This study estimates feedback and therapist effects and tests the predictive value of therapists' and patient attitudes toward psychometric feedback for treatment outcome and length. METHODS: Data of 349 outpatients and 44 therapists in private practices were used. Separate multilevel analyses were conducted to estimate predictors and feedback and therapist effects. RESULTS: Around 5.88% of the variability in treatment outcome and 8.89% in treatment length were attributed to therapists. There was no relationship between the average effectiveness of therapists and the average length of their treatments. Initial impairment, early alliance, number of diagnoses, feedback as well as therapists' and patients' attitudes toward feedback were significant predictors of treatment outcome. Treatments tended to be longer for patients with a higher number of approved sessions by the insurance company, with higher levels of interpersonal distress at intake, and for those who developed negatively (negative feedback) over the course of their treatment. CONCLUSIONS: Therapist effects on treatment outcome and treatment length in routine care seem to be relevant predictors in the context of feedback studies. Therapists' attitudes toward and use of feedback as well as patients' attitudes toward feedback should be further investigated in future research on psychometric feedback.

[Typical patterns of depressive disorders during outpatient psychotherapy and their prediction]. / Verlaufsmuster depressiver Störungen bei ambulanten psychotherapeutischen Behandlungen und deren Vorhersage.

OBJECTIVES: Are there typical patterns of outpatient psychotherapy among depressed patients? What characterizes patients with different patterns? METHODS: We examined N= 548 patients with primary depressive disorders using a naturalistic design. Using a latent-state-mixture model and depression measures at baseline, therapy end and 1-year follow-up we found a total of five patterns. Subgroups were compared with respect to sociodemographic and treatment-related variables. RESULTS: Responders with moderate depressive symptoms at baseline and responders with severe symptoms at baseline were most common (54% and 25% of the sample, respectively) compared to late responders (9 %), small-response patients (9 %) and recidivists (4 %). Patterns of change were related to symptom intensity at baseline and ratings of perceived helpfulness at the end of treatment. CONCLUSIONS: Since psychometric scales better predicted change pattern than sociodemographic characteristics, primary and secondary diagnoses, psychometric assessments and feedback systems could be a useful supplement to traditional quality assurance procedures.

Benefits and challenges in practice-oriented psychotherapy research in Germany: the TK and the QS-PSY-BAY projects of quality assurance in outpatient psychotherapy.

OBJECTIVE: Two patient-focused long-term research projects performed in the German outpatient psychotherapy system are focused on in this article. The TK (Techniker Krankenkasse) project is the first study to evaluate a quality assurance and feedback system with regard to its practical feasibility in German routine care. The other study ("Quality Assurance in Outpatient Psychotherapy in Bavaria"; QS-PSY-BAY) was designed to test a new approach for quality assurance in outpatient psychotherapy using electronic documentation of patient characteristics and outcome parameters. In addition this project provides the opportunity to analyze data on health-related costs for the patients undergoing outpatient psychotherapy. METHOD: Both projects and their results indicating high effect sizes are briefly described. RESULTS: From the perspectives of the research teams, advisory boards and other stakeholders, the experiences with these projects are discussed focusing on obstacles, challenges, difficulties, and benefits in developing and implementing the studies. The triangle collaboration of therapists, researchers, and health insurance companies/health service institutions turned out to be fruitful in both studies. CONCLUSIONS: Despite some controversies between the partners the experiences indicate the importance of practiced-research collaborations to provide relevant information about the delivery of outpatient psychotherapy in the health system.

[Non-completion of insurance-funded outpatient psychotherapy: frequency, risk factors, outcome]. / Abbrüche antragspflichtiger ambulanter Psychotherapien: Häufigkeit, Risikofaktoren, Outcome.

OBJECTIVE: Because premature discontinuation of psychotherapy limits the effectiveness of the interventions, in a naturalistic design we examined the prevalence, predictors, and outcome of premature discontinuation. METHODS: The sample included N = 584 patients with various mental disorders. Risk factors were identified using regression analysis. As outcome Patient Health Questionnaire scales were considered. We compared pre and post averages as well as post averages of premature discontinuation versus regular termination. RESULTS: Risk factors were: female and/or unemployed patient, low patient and/or therapist ratings regarding therapy success, and extraordinarily high therapist ratings of the therapeutic alliance. Despite premature discontinuation we found significant reductions of depression, anxiety, somatic symptoms, and stress (ES = 0.30, …, 0.44). Compared to regular termination though, patients with premature discontinuation were more impaired at last measurement (ES = 0.17, …, 0.37). CONCLUSION: Not each premature discontinuation is a psychotherapeutic failure. Warning signals for looming premature discontinuation are low ratings of therapy success while psychotherapeutic treatment.

[Extensions of the outpatient psychotherapy: a study about patient, therapist, treatment, and therapy course characteristics]. / Verlängerungen ambulanter Psychotherapien: Eine Studie zu Patienten-, Therapeuten-, Behandlungs- und Verlaufsmerkmalen.

Due to the treatment costs, extensions of the standard therapy duration are a matter of critical examination. This study investigates which factors characterize patients with treatment extensions in the German health system and how effective these extensions are for a reduction of the patients' symptoms. We analysed a disorder heterogeneous sample of 810 patients. We found that therapy extensions are more common among with more severe mental disorders, when the therapeutic relationship is positive, and little therapy success has been achieved during the standard duration. Overall, the findings suggest that outpatient psychotherapy is mainly extended for patients with a low symptom reduction after the standard therapy duration and that therapy extension allows a symptom reduction that could not have been achieved otherwise.

A critical meta-analysis of lens model studies in human judgment and decision-making.

Achieving accurate judgment ('judgmental achievement') is of utmost importance in daily life across multiple domains. The lens model and the lens model equation provide useful frameworks for modeling components of judgmental achievement and for creating tools to help decision makers (e.g., physicians, teachers) reach better judgments (e.g., a correct diagnosis, an accurate estimation of intelligence). Previous meta-analyses of judgment and decision-making studies have attempted to evaluate overall judgmental achievement and have provided the basis for evaluating the success of bootstrapping (i.e., replacing judges by linear models that guide decision making). However, previous meta-analyses have failed to appropriately correct for a number of study design artifacts (e.g., measurement error, dichotomization), which may have potentially biased estimations (e.g., of the variability between studies) and led to erroneous interpretations (e.g., with regards to moderator variables). In the current study we therefore conduct the first psychometric meta-analysis of judgmental achievement studies that corrects for a number of study design artifacts. We identified 31 lens model studies (N = 1,151, k = 49) that met our inclusion criteria. We evaluated overall judgmental achievement as well as whether judgmental achievement depended on decision domain (e.g., medicine, education) and/or the level of expertise (expert vs. novice). We also evaluated whether using corrected estimates affected conclusions with regards to the success of bootstrapping with psychometrically-corrected models. Further, we introduce a new psychometric trim-and-fill method to estimate the effect sizes of potentially missing studies correct psychometric meta-analyses for effects of publication bias. Comparison of the results of the psychometric meta-analysis with the results of a traditional meta-analysis (which only corrected for sampling error) indicated that artifact correction leads to a) an increase in values of the lens model components, b) reduced heterogeneity between studies, and c) increases the success of bootstrapping. We argue that psychometric meta-analysis is useful for accurately evaluating human judgment and show the success of bootstrapping.

[Results from the pilot project of the Techniker Krankenkasse (TK) 'quality monitoring in outpatient psychotherapy': the evaluators' perspective]. / Zu den Ergebnissen des Modellprojektes der Techniker-Krankenkasse zum Qualitätsmonitoring in der ambulanten Psychotherapie aus Sicht des wissenschaftlichen Evaluationsteams.

This is the first study to evaluate a quality assurance and feedback tool with regard to its practical feasibility in routine care and within the German health insurance system. This paper summarizes the evaluators' perspective on issues regarding the representativeness of the sample, consequences for therapy length and central assumptions and concepts of the project. The evaluation team comments on the paper "Pilot project of the Techniker-Krankenkasse - A comment from the perspective of the academic advisory council".

Assessment of tinnitus-related impairments and disabilities using the German THI-12: sensitivity and stability of the scale over time.

OBJECTIVE: To evaluate the reliability, dimensionality, predictive validity, construct validity, and sensitivity to change of the THI-12 total and sub-scales as diagnostic aids to describe and quantify tinnitus-evoked reactions and evaluate treatment efficacy. DESIGN: Explorative analysis of the German tinnitus handicap inventory (THI-12) to assess potential sensitivity to tinnitus therapy in placebo-controlled randomized studies. Correlation analysis, including Cronbach's coefficient α and explorative common factor analysis (EFA), was conducted within and between assessments to demonstrate the construct validity, dimensionality, and factorial structure of the THI-12. STUDY SAMPLE: N = 618 patients suffering from subjective tinnitus who were to be screened to participate in a randomized, placebo-controlled, 16-week, longitudinal study. RESULTS: The THI-12 can reliably diagnose tinnitus-related impairments and disabilities and assess changes over time. The test-retest coefficient for neighboured visits was r > 0.69, the internal consistency of the THI-12 total score was α ≤ 0.79 and α ≤ 0.89 at subsequent visits. Predictability of THI-12 total score and overall variance increased with successive measurements. The three-factorial structure allowed for evaluation of factors that affect aspects of patients' health-related quality of life. CONCLUSIONS: The THI-12, with its three-factorial structure, is a simple, reliable, and valid instrument for the diagnosis and assessment of tinnitus and associated impairment over time.

Adult age differences in inhibitory processes and their predictive validity for fluid intelligence.

According to the inhibition-deficit hypothesis age differences in working memory capacity and fluid intelligence have been attributed to a decline in inhibitory efficiency. Conceptualizing inhibition as multifaceted, 88 participants (49 younger and 39 elderly) completed two versions of the negative priming paradigm (identification and localization), and two variants of the directed forgetting paradigm (listwise and itemwise). Two tasks of the Wechsler Intelligence Test with high loadings on general fluid intelligence (Gf) served as validation criteria. Results revealed task-specific and speed-independent inhibitory deficits in the elderly (lower negative priming in both paradigms; more intrusions in the directed forgetting tasks), as well as higher levels of repetition priming. Significant correlations between measures of inhibition and the Wechsler scores were found in both age groups. Results support the view of multiple inhibitory-like capabilities that play a central role in the decline of higher-order cognitive functions in old age.

Individual differences in components of reaction time distributions and their relations to working memory and intelligence.

The authors bring together approaches from cognitive and individual differences psychology to model characteristics of reaction time distributions beyond measures of central tendency. Ex-Gaussian distributions and a diffusion model approach are used to describe individuals' reaction time data. The authors identified common latent factors for each of the 3 ex-Gaussian parameters and for 3 parameters central to the diffusion model using structural equation modeling for a battery of choice reaction tasks. These factors had differential relations to criterion constructs. Parameters reflecting the tail of the distribution (i.e., tau in the ex-Gaussian and drift rate in the diffusion model) were the strongest unique predictors of working memory, reasoning, and psychometric speed. Theories of controlled attention and binding are discussed as potential theoretical explanations.