Model Specification Searches in Structural Equation Modeling Using Bee Swarm Optimization.

Metaheuristics are optimization algorithms that efficiently solve a variety of complex combinatorial problems. In psychological research, metaheuristics have been applied in short-scale construction and model specification search. In the present study, we propose a bee swarm optimization (BSO) algorithm to explore the structure underlying a psychological measurement instrument. The algorithm assigns items to an unknown number of nested factors in a confirmatory bifactor model, while simultaneously selecting items for the final scale. To achieve this, the algorithm follows the biological template of bees' foraging behavior: Scout bees explore new food sources, whereas onlooker bees search in the vicinity of previously explored, promising food sources. Analogously, scout bees in BSO introduce major changes to a model specification (e.g., adding or removing a specific factor), whereas onlooker bees only make minor changes (e.g., adding an item to a factor or swapping items between specific factors). Through this division of labor in an artificial bee colony, the algorithm aims to strike a balance between two opposing strategies diversification (or exploration) versus intensification (or exploitation). We demonstrate the usefulness of the algorithm to find the underlying structure in two empirical data sets (Holzinger-Swineford and short dark triad questionnaire, SDQ3). Furthermore, we illustrate the influence of relevant hyperparameters such as the number of bees in the hive, the percentage of scouts to onlookers, and the number of top solutions to be followed. Finally, useful applications of the new algorithm are discussed, as well as limitations and possible future research opportunities.

Ant colony optimization for parallel test assembly.

Ant colony optimization (ACO) algorithms have previously been used to compile single short scales of psychological constructs. In the present article, we showcase the versatility of the ACO to construct multiple parallel short scales that adhere to several competing and interacting criteria simultaneously. Based on an initial pool of 120 knowledge items, we assembled three 12-item tests that (a) adequately cover the construct at the domain level, (b) follow a unidimensional measurement model, (c) allow reliable and (d) precise measurement of factual knowledge, and (e) are gender-fair. Moreover, we aligned the test characteristic and test information functions of the three tests to establish the equivalence of the tests. We cross-validated the assembled short scales and investigated their association with the full scale and covariates that were not included in the optimization procedure. Finally, we discuss potential extensions to metaheuristic test assembly and the equivalence of parallel knowledge tests in general.

Predicting Lifetime Suicide Attempts in a Community Sample of Adolescents Using Machine Learning Algorithms.

Suicide is a major global health concern and a prominent cause of death in adolescents. Previous research on suicide prediction has mainly focused on clinical or adult samples. To prevent suicides at an early stage, however, it is important to screen for risk factors in a community sample of adolescents. We compared the accuracy of logistic regressions, elastic net regressions, and gradient boosting machines in predicting suicide attempts by 17-year-olds in the Millennium Cohort Study (N = 7,347), combining a large set of self- and other-reported variables from different categories. Both machine learning algorithms outperformed logistic regressions and achieved similar balanced accuracies (.76 when using data 3 years before the self-reported lifetime suicide attempts and .85 when using data from the same measurement wave). We identified essential variables that should be considered when screening for suicidal behavior. Finally, we discuss the usefulness of complex machine learning models in suicide prediction.

Public events knowledge in an age-heterogeneous sample: Reminiscence bump or bummer?

The reminiscence bump describes an increased recollection of autobiographic experiences made in adolescence and early adulthood. It is unclear if this phenomenon can also be found in declarative knowledge of past public events. To answer this question, we assessed public events knowledge (PEK) about the past 6 decades with a 120-item knowledge test across six domains in a sample of 1,012 Germans that were sampled uniformly across the ages of 30-80 years. General and domain-specific PEK scores were analyzed as a function of age-at-event. Scores were lower for public events preceding participants' birth and stayed stable from the age-at-event of 5-10 years onward. There was no significant peak in PEK in adolescence or early adulthood, arguing against an extension of the reminiscence effect to factual knowledge. We examined associations between PEK and relevant variables such as crystallized intelligence (Gc), news consumption, and openness to experience with structural equation models. Strong associations between PEK and Gc were established, whereas the associations of PEK with news consumption and openness were mainly driven by their link to declarative knowledge. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Accuracy and precision of fixed and random effects in meta-analyses of randomized control trials for continuous outcomes.

Meta-analyses of treatment effects in randomized control trials are often faced with the problem of missing information required to calculate effect sizes and their sampling variances. Particularly, correlations between pre- and posttest scores are frequently not available. As an ad-hoc solution, researchers impute a constant value for the missing correlation. As an alternative, we propose adopting a multivariate meta-regression approach that models independent group effect sizes and accounts for the dependency structure using robust variance estimation or three-level modeling. A comprehensive simulation study mimicking realistic conditions of meta-analyses in clinical and educational psychology suggested that imputing a fixed correlation 0.8 or adopting a multivariate meta-regression with robust variance estimation work well for estimating the pooled effect but lead to slightly distorted between-study heterogeneity estimates. In contrast, three-level meta-regressions resulted in largely unbiased fixed effects but more inconsistent prediction intervals. Based on these results recommendations for meta-analytic practice and future meta-analytic developments are provided.

Predicting treatment response using machine learning: A registered report.

OBJECTIVE: Previous research on psychotherapy treatment response has mainly focused on outpatients or clinical trial data which may have low ecological validity regarding naturalistic inpatient samples. To reduce treatment failures by proactively screening for patients at risk of low treatment response, gain more knowledge about risk factors and to evaluate treatments, accurate insights about predictors of treatment response in naturalistic inpatient samples are needed. METHODS: We compared the performance of different machine learning algorithms in predicting treatment response, operationalized as a substantial reduction in symptom severity as expressed in the Patient Health Questionnaire Anxiety and Depression Scale. To achieve this goal, we used different sets of variables-(a) demographics, (b) physical indicators, (c) psychological indicators and (d) treatment-related variables-in a naturalistic inpatient sample (N = 723) to specify their joint and unique contribution to treatment success. RESULTS: There was a strong link between symptom severity at baseline and post-treatment (R2 = .32). When using all available variables, both machine learning algorithms outperformed the linear regressions and led to an increment in predictive performance of R2 = .12. Treatment-related variables were the most predictive, followed psychological indicators. Physical indicators and demographics were negligible. CONCLUSIONS: Treatment response in naturalistic inpatient settings can be predicted to a considerable degree by using baseline indicators. Regularization via machine learning algorithms leads to higher predictive performances as opposed to including nonlinear and interaction effects. Heterogenous aspects of mental health have incremental predictive value and should be considered as prognostic markers when modelling treatment processes.

The Nomological Net of Knowledge, Self-Reported Knowledge, and Overclaiming in Children.

Research on self-reported knowledge and overclaiming in children is sparse. With the current study, we aim to close this gap by developing an overclaiming questionnaire measuring self-reported knowledge and overclaiming that is tailored to children. Moreover, we examine the nomological net of self-reported knowledge and overclaiming in childhood discussing three perspectives: Overclaiming as (a) a result of deliberate self-enhancement tendencies, (b) a proxy for declarative knowledge, and (c) an indicator of creative engagement. We juxtaposed overclaiming, as indicated by claiming familiarity with non-existent terms, and self-reported knowledge with fluid and crystallized intelligence, creativity, and personality traits in a sample of 897 children attending third grade. The results of several latent variable analyses were similar to findings known from adult samples: We found no strong evidence for any of the competing perspectives on overclaiming. Just like in adults, individual differences in self-reported knowledge were strongly inflated by overclaiming, and only weakly related to declarative knowledge.

The Road to Hell Is Paved With Good Intentions: How Common Practices in Scale Construction Hurt Validity.

Sound scale construction is pivotal to the measurement of psychological constructs. Common item sampling procedures emphasize aspects of reliability to the disadvantage of aspects of validity, which are less tangible. We use a health knowledge test as an example to demonstrate how item sampling strategies that focus on either factor saturation or construct coverage influence scale composition and demonstrate how to find a trade-off between these two opposing needs. More specifically, we compile three 75-item health knowledge scales using Ant Colony Optimization, a metaheuristic algorithm that is inspired by the foraging behavior of ants, to optimize factor saturation, construct coverage, or a compromise of both. We demonstrate that our approach is well suited to balance out construct coverage and factor saturation when constructing a health knowledge test. Finally, we discuss conceptual problems with the modeling of declarative knowledge and provide recommendations for the assessment of health knowledge.

Mapping established psychopathology scales onto the Hierarchical Taxonomy of Psychopathology (HiTOP).

The Hierarchical Taxonomy of Psychopathology (HiTOP) organizes phenotypes of mental disorder based on empirical covariation, offering a comprehensive organizational framework from narrow symptoms to broader patterns of psychopathology. We argue that established self-report measures of psychopathology from the pre-HiTOP era should be systematically integrated into HiTOP to foster cumulative research and further the understanding of psychopathology structure. Hence, in this study, we mapped 92 established psychopathology (sub)scales onto the current HiTOP working model using data from an extensive battery of self-report assessments that was completed by community participants and outpatients (N = 909). Content validity ratings of the item pool were used to select indicators for a bifactor-(S-1) model of the p factor and five HiTOP spectra (i.e., internalizing, thought disorder, detachment, disinhibited externalizing, and antagonistic externalizing). The content-based HiTOP scales were validated against personality disorder diagnoses as assessed by standardized interviews. We then located established scales within the taxonomy by estimating the extent to which scales reflected higher-level HiTOP dimensions. The analyses shed light on the location of established psychopathology scales in HiTOP, identifying pure markers and blends of HiTOP spectra, as well as pure markers of the p factor (i.e., scales assessing mentalizing impairment and suspiciousness/epistemic mistrust).

Dynamic interplay of cognitive functioning and depressive symptoms in patients with Parkinson's disease.

OBJECTIVE: We examine the trajectories of and the dynamic interplay between cognitive functioning and depressive symptoms in patients with Parkinson's disease (PD) in comparison to healthy controls (HC) from an intraindividual perspective. METHOD: The DeNoPa study is a single-center, observational, longitudinal study with biennial follow-ups over 8 years. The present analyses are based on 123 PD (79 male) and 107 HC (64 male) with a mean age of 64.1 years (SD = 8.3). PD and HC completed a battery of neuropsychological tests and scales assessing depressive symptoms. We used a random-intercept cross-lagged panel model (RI-CLPM) to study their trajectories and the dynamic interplay. RESULTS: Cognitive abilities of PD were on average d = -0.67 worse at baseline and d = -1.22 at 8-years follow-up in comparison to HC. Depressive symptoms in PD showed large variability and followed a U-shaped trajectory. From an intraindividual perspective, greater impairments in cognitive abilities were subsequently associated with increased depressive symptoms (b = -0.60, p = .03), whereas the effect in the opposite direction was not significant. CONCLUSIONS: We found indication that a decline on a global composite scale of cognition can be seen as a precursor of depressive symptoms in patients with PD. To counter cognitive losses and the subsequent mood deterioration, patient education and early cognitive (and behavioral) enrichment seem promising candidates for treatment. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Detecting Careless Responding in Survey Data Using Stochastic Gradient Boosting.

Careless responding is a bias in survey responses that disregards the actual item content, constituting a threat to the factor structure, reliability, and validity of psychological measurements. Different approaches have been proposed to detect aberrant responses such as probing questions that directly assess test-taking behavior (e.g., bogus items), auxiliary or paradata (e.g., response times), or data-driven statistical techniques (e.g., Mahalanobis distance). In the present study, gradient boosted trees, a state-of-the-art machine learning technique, are introduced to identify careless respondents. The performance of the approach was compared with established techniques previously described in the literature (e.g., statistical outlier methods, consistency analyses, and response pattern functions) using simulated data and empirical data from a web-based study, in which diligent versus careless response behavior was experimentally induced. In the simulation study, gradient boosting machines outperformed traditional detection mechanisms in flagging aberrant responses. However, this advantage did not transfer to the empirical study. In terms of precision, the results of both traditional and the novel detection mechanisms were unsatisfactory, although the latter incorporated response times as additional information. The comparison between the results of the simulation and the online study showed that responses in real-world settings seem to be much more erratic than can be expected from the simulation studies. We critically discuss the generalizability of currently available detection methods and provide an outlook on future research on the detection of aberrant response patterns in survey research.

The Structure of the Toronto Alexithymia Scale (TAS-20): A Meta-Analytic Confirmatory Factor Analysis.

Alexithymia is defined as the inability of persons to describe their emotional states, to identify the feelings of others, and a utilitarian type of thinking. The most popular instrument to assess alexithymia is the Toronto Alexithymia Scale (TAS-20). Despite its widespread use, an ongoing controversy pertains to its internal structure. The TAS-20 was originally constructed to capture three different factors, but several studies suggested different factor solutions, including bifactor models and models with a method factor for the reversely keyed items. The present study examined the dimensionality of the TAS-20 using summary data of 88 samples from 62 studies (total N = 69,722) with meta-analytic structural equation modeling. We found support for the originally proposed three-dimensional solution, whereas more complex models produced inconsistent factor loadings. Because a major source of misfit stems from translated versions, the results are discussed with respect to generalizations across languages and cultural contexts.

Structural invariance of declarative knowledge across the adult lifespan.

The differentiation-dedifferentiation hypothesis of general cognitive ability has been widely studied, but comparable research on crystallized intelligence is scarce. To close this gap, we conducted an empirical test of the age differentiation hypothesis of declarative knowledge as proposed in Cattell's investment theory, which predicts that knowledge differentiates into diverse forms after compulsory education ends. Thereto, a cross-sectional sample of 1,629 participants aged 18-70 years (M = 45.3) completed a comprehensive knowledge test comprising 120 broadly sampled questions from 12 knowledge domains, as well as a measure of openness/intellect. To investigate age-related differences in the level and structure of knowledge, we performed invariance tests in local structural equation models. The results did not provide any evidence for age-related differentiation of declarative knowledge but indicated age-related differences in the mean structure. Higher levels in openness/intellect were associated with higher levels in knowledge but not with more differentiated structure of knowledge. Contrary to predictions of the investment theory, our results suggest that declarative knowledge is age invariant across major parts of the adult lifespan. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Caught in the Act: Predicting Cheating in Unproctored Knowledge Assessment.

Cheating is a serious threat in unproctored ability assessment, irrespective of countermeasures taken, anticipated consequences (high vs. low stakes), and test modality (paper-pencil vs. computer-based). In the present study, we examined the power of (a) self-report-based indicators (i.e., Honesty-Humility and Overclaiming scales), (b) test data (i.e., performance with extremely difficult items), and (c) para data (i.e., reaction times, switching between browser tabs) to predict participants' cheating behavior. To this end, 315 participants worked on a knowledge test in an unproctored online assessment and subsequently in a proctored lab assessment. We used multiple regression analysis and an extended latent change score model to assess the potential of the different indicators to predict cheating. In summary, test data and para data performed best, while traditional self-report-based indicators were not predictive. We discuss the findings with respect to unproctored testing in general and provide practical advice on cheating detection in online ability assessments.

A Reappraisal of the Threshold Hypothesis of Creativity and Intelligence.

Intelligence has been declared as a necessary but not sufficient condition for creativity, which was subsequently (erroneously) translated into the so-called threshold hypothesis. This hypothesis predicts a change in the correlation between creativity and intelligence at around 1.33 standard deviations above the population mean. A closer inspection of previous inconclusive results suggests that the heterogeneity is mostly due to the use of suboptimal data analytical procedures. Herein, we applied and compared three methods that allowed us to handle intelligence as a continuous variable. In more detail, we examined the threshold of the creativity-intelligence relation with (a) scatterplots and heteroscedasticity analysis, (b) segmented regression analysis, and (c) local structural equation models in two multivariate studies (N1 = 456; N2 = 438). We found no evidence for the threshold hypothesis of creativity across different analytical procedures in both studies. Given the problematic history of the threshold hypothesis and its unequivocal rejection with appropriate multivariate methods, we recommend the total abandonment of the threshold.

Cognitive Abilities Explain Wording Effects in the Rosenberg Self-Esteem Scale.

There is consensus that the 10 items of the Rosenberg Self-Esteem Scale (RSES) reflect wording effects resulting from positively and negatively keyed items. The present study examined the effects of cognitive abilities on the factor structure of the RSES with a novel, nonparametric latent variable technique called local structural equation models. In a nationally representative German large-scale assessment including 12,437 students competing measurement models for the RSES were compared: a bifactor model with a common factor and a specific factor for all negatively worded items had an optimal fit. Local structural equation models showed that the unidimensionality of the scale increased with higher levels of reading competence and reasoning, while the proportion of variance attributed to the negatively keyed items declined. Wording effects on the factor structure of the RSES seem to represent a response style artifact associated with cognitive abilities.

Knowledge Is Power for Medical Assistants: Crystallized and Fluid Intelligence As Predictors of Vocational Knowledge.

Medical education research has focused almost entirely on the education of future physicians. In comparison, findings on other health-related occupations, such as medical assistants, are scarce. With the current study, we wanted to examine the knowledge-is-power hypothesis in a real life educational setting and add to the sparse literature on medical assistants. Acquisition of vocational knowledge in vocational education and training (VET) was examined for medical assistant students (n = 448). Differences in domain-specific vocational knowledge were predicted by crystallized and fluid intelligence in the course of VET. A multiple matrix design with 3 year-specific booklets was used for the vocational knowledge tests of the medical assistants. The unique and joint contributions of the predictors were investigated with structural equation modeling. Crystallized intelligence emerged as the strongest predictor of vocational knowledge at every stage of VET, while fluid intelligence only showed weak effects. The present results support the knowledge-is-power hypothesis, even in a broad and more naturalistic setting. This emphasizes the relevance of general knowledge for occupations, such as medical assistants, which are more focused on learning hands-on skills than the acquisition of academic knowledge.

A confirmatory examination of age-associated personality differences: Deriving age-related measurement-invariant solutions using ant colony optimization.

OBJECTIVE: The goal of this study was to examine age-associated personality differences using a measurement-invariant representation of the higher-order structure of the Five-Factor Model. METHOD: We reanalyzed the German NEO-PI-R norm sample (N = 11,724) and applied ant colony optimization in a multigroup confirmatory factor analysis setting in order to select three items per first-order factor that would optimize model fit and measurement invariance across 18 age groups ranging from 16 to 65 years of age. RESULTS: Ant colony optimization substantially improved absolute and relative model fit under measurement invariance constraints. However, the results showed that even when selecting items, measurement invariance across a large age span could not be guaranteed. Strong measurement invariance for Extraversion and Agreeableness could not be established. The age-associated mean-level differences of the first-order factors of Neuroticism and Conscientiousness supported the maturity hypothesis. The mean levels of the first-order factors of Openness varied substantially from each other across age. CONCLUSIONS: Findings on age differences in personality can be particularly distorted in older age groups. Testing for and ensuring measurement invariance with item selection procedures can help solve this problem. The higher-order structure of personality should be accounted for when personality development is examined.

Meta-Heuristics in Short Scale Construction: Ant Colony Optimization and Genetic Algorithm.

The advent of large-scale assessment, but also the more frequent use of longitudinal and multivariate approaches to measurement in psychological, educational, and sociological research, caused an increased demand for psychometrically sound short scales. Shortening scales economizes on valuable administration time, but might result in inadequate measures because reducing an item set could: a) change the internal structure of the measure, b) result in poorer reliability and measurement precision, c) deliver measures that cannot effectively discriminate between persons on the intended ability spectrum, and d) reduce test-criterion relations. Different approaches to abbreviate measures fare differently with respect to the above-mentioned problems. Therefore, we compare the quality and efficiency of three item selection strategies to derive short scales from an existing long version: a Stepwise COnfirmatory Factor Analytical approach (SCOFA) that maximizes factor loadings and two metaheuristics, specifically an Ant Colony Optimization (ACO) with a tailored user-defined optimization function and a Genetic Algorithm (GA) with an unspecific cost-reduction function. SCOFA compiled short versions were highly reliable, but had poor validity. In contrast, both metaheuristics outperformed SCOFA and produced efficient and psychometrically sound short versions (unidimensional, reliable, sensitive, and valid). We discuss under which circumstances ACO and GA produce equivalent results and provide recommendations for conditions in which it is advisable to use a metaheuristic with an unspecific out-of-the-box optimization function.