Measurement invariance, selection invariance, and fair selection revisited.

This note contains a corrective and a generalization of results by Borsboom et al. (2008), based on Heesen and Romeijn (2019). It highlights the relevance of insights from psychometrics beyond the context of psychological testing. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Investigating data-driven biological subtypes of psychiatric disorders using specification-curve analysis.

BACKGROUND: Cluster analyses have become popular tools for data-driven classification in biological psychiatric research. However, these analyses are known to be sensitive to the chosen methods and/or modelling options, which may hamper generalizability and replicability of findings. To gain more insight into this problem, we used Specification-Curve Analysis (SCA) to investigate the influence of methodological variation on biomarker-based cluster-analysis results. METHODS: Proteomics data (31 biomarkers) were used from patients (n = 688) and healthy controls (n = 426) in the Netherlands Study of Depression and Anxiety. In SCAs, consistency of results was evaluated across 1200 k-means and hierarchical clustering analyses, each with a unique combination of the clustering algorithm, fit-index, and distance metric. Next, SCAs were run in simulated datasets with varying cluster numbers and noise/outlier levels to evaluate the effect of data properties on SCA outcomes. RESULTS: The real data SCA showed no robust patterns of biological clustering in either the MDD or a combined MDD/healthy dataset. The simulation results showed that the correct number of clusters could be identified quite consistently across the 1200 model specifications, but that correct cluster identification became harder when the number of clusters and noise levels increased. CONCLUSION: SCA can provide useful insights into the presence of clusters in biomarker data. However, SCA is likely to show inconsistent results in real-world biomarker datasets that are complex and contain considerable levels of noise. Here, the number and nature of the observed clusters may depend strongly on the chosen model-specification, precluding conclusions about the existence of biological clusters among psychiatric patients.

What's in a model? Network models as tools instead of representations of what psychiatric disorders really are.

Network models block reductionism about psychiatric disorders only if models are interpreted in a realist manner - that is, taken to represent "what psychiatric disorders really are." A flexible and more instrumentalist view of models is needed to improve our understanding of the heterogeneity and multifactorial character of psychiatric disorders.

Intervention and Identifiability in Latent Variable Modelling.

We consider the use of interventions for resolving a problem of unidentified statistical models. The leading examples are from latent variable modelling, an influential statistical tool in the social sciences. We first explain the problem of statistical identifiability and contrast it with the identifiability of causal models. We then draw a parallel between the latent variable models and Bayesian networks with hidden nodes. This allows us to clarify the use of interventions for dealing with unidentified statistical models. We end by discussing the philosophical and methodological import of our result.

PSYCHIATRIC COMORBIDITY DOES NOT ONLY DEPEND ON DIAGNOSTIC THRESHOLDS: AN ILLUSTRATION WITH MAJOR DEPRESSIVE DISORDER AND GENERALIZED ANXIETY DISORDER.

BACKGROUND: High rates of psychiatric comorbidity are subject of debate: To what extent do they depend on classification choices such as diagnostic thresholds? This paper investigates the influence of different thresholds on rates of comorbidity between major depressive disorder (MDD) and generalized anxiety disorder (GAD). METHODS: Point prevalence of comorbidity between MDD and GAD was measured in 74,092 subjects from the general population (LifeLines) according to Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) criteria. Comorbidity rates were compared for different thresholds by varying the number of necessary criteria from ≥ 1 to all nine symptoms for MDD, and from ≥ 1 to all seven symptoms for GAD. RESULTS: According to DSM thresholds, 0.86% had MDD only, 2.96% GAD only, and 1.14% both MDD and GAD (odds ratio (OR) 42.6). Lower thresholds for MDD led to higher rates of comorbidity (1.44% for ≥ 4 of nine MDD symptoms, OR 34.4), whereas lower thresholds for GAD hardly influenced comorbidity (1.16% for ≥ 3 of seven GAD symptoms, OR 38.8). Specific patterns in the distribution of symptoms within the population explained this finding: 37.3% of subjects with core criteria of MDD and GAD reported subthreshold MDD symptoms, whereas only 7.6% reported subthreshold GAD symptoms. CONCLUSIONS: Lower thresholds for MDD increased comorbidity with GAD, but not vice versa, owing to specific symptom patterns in the population. Generally, comorbidity rates result from both empirical symptom distributions and classification choices and cannot be reduced to either of these exclusively. This insight invites further research into the formation of disease concepts that allow for reliable predictions and targeted therapeutic interventions.

Psychiatric comorbidity: fact or artifact?

The frequent occurrence of comorbidity has brought about an extensive theoretical debate in psychiatry. Why are the rates of psychiatric comorbidity so high and what are their implications for the ontological and epistemological status of comorbid psychiatric diseases? Current explanations focus either on classification choices or on causal ties between disorders. Based on empirical and philosophical arguments, we propose a conventionalist interpretation of psychiatric comorbidity instead. We argue that a conventionalist approach fits well with research and clinical practice and resolves two problems for psychiatric diseases: experimenter's regress and arbitrariness.

Psychiatric comorbidity and causal disease models.

In psychiatry, comorbidity is the rule rather than the exception. Up to 45% of all patients are classified as having more than one psychiatric disorder. These high rates of comorbidity have led to a debate concerning the interpretation of this phenomenon. Some authors emphasize the problematic character of the high rates of comorbidity because they indicate absent zones of rarities. Others consider comorbid conditions to be a validator for a particular reclassification of diseases. In this paper we will show that those at first sight contrasting interpretations of comorbidity are based on similar assumptions about disease models. The underlying ideas are that firstly high rates of comorbidity are the result of the absence of causally defined diseases in psychiatry, and second that causal disease models are preferable to non-causal disease models. We will argue that there are good reasons to seek after causal understanding of psychiatric disorders, but that causal disease models will not rule out high rates of comorbidity--neither in psychiatry, nor in medicine in general. By bringing to the fore these underlying assumptions, we hope to clear the ground for a different understanding of comorbidity, and of models for psychiatric diseases.

The humble Bayesian: model checking from a fully Bayesian perspective.

Gelman and Shalizi (2012) criticize what they call the 'usual story' in Bayesian statistics: that the distribution over hypotheses or models is the sole means of statistical inference, thus excluding model checking and revision, and that inference is inductivist rather than deductivist. They present an alternative hypothetico-deductive approach to remedy both shortcomings. We agree with Gelman and Shalizi's criticism of the usual story, but disagree on whether Bayesian confirmation theory should be abandoned. We advocate a humble Bayesian approach, in which Bayesian confirmation theory is the central inferential method. A humble Bayesian checks her models and critically assesses whether the Bayesian statistical inferences can reasonably be called upon to support real-world inferences.

Data-driven subtypes of major depressive disorder: a systematic review.

BACKGROUND: According to current classification systems, patients with major depressive disorder (MDD) may have very different combinations of symptoms. This symptomatic diversity hinders the progress of research into the causal mechanisms and treatment allocation. Theoretically founded subtypes of depression such as atypical, psychotic, and melancholic depression have limited clinical applicability. Data-driven analyses of symptom dimensions or subtypes of depression are scarce. In this systematic review, we examine the evidence for the existence of data-driven symptomatic subtypes of depression. METHODS: We undertook a systematic literature search of MEDLINE, PsycINFO and Embase in May 2012. We included studies analyzing the depression criteria of the Diagnostic and Statistical Manual of Mental Disorders, fourth edition (DSM-IV) of adults with MDD in latent variable analyses. RESULTS: In total, 1176 articles were retrieved, of which 20 satisfied the inclusion criteria. These reports described a total of 34 latent variable analyses: 6 confirmatory factor analyses, 6 exploratory factor analyses, 12 principal component analyses, and 10 latent class analyses. The latent class techniques distinguished 2 to 5 classes, which mainly reflected subgroups with different overall severity: 62 of 71 significant differences on symptom level were congruent with a latent class solution reflecting severity. The latent class techniques did not consistently identify specific symptom clusters. Latent factor techniques mostly found a factor explaining the variance in the symptoms depressed mood and interest loss (11 of 13 analyses), often complemented by psychomotor retardation or fatigue (8 of 11 analyses). However, differences in found factors and classes were substantial. CONCLUSIONS: The studies performed to date do not provide conclusive evidence for the existence of depressive symptom dimensions or symptomatic subtypes. The wide diversity of identified factors and classes might result either from the absence of patterns to be found, or from the theoretical and modeling choices preceding analysis.

Evaluating expectations about negative emotional states of aggressive boys using Bayesian model selection.

Researchers often have expectations about the research outcomes in regard to inequality constraints between, e.g., group means. Consider the example of researchers who investigated the effects of inducing a negative emotional state in aggressive boys. It was expected that highly aggressive boys would, on average, score higher on aggressive responses toward other peers than moderately aggressive boys, who would in turn score higher than nonaggressive boys. In most cases, null hypothesis testing is used to evaluate such hypotheses. We show, however, that hypotheses formulated using inequality constraints between the group means are generally not evaluated properly. The wrong hypotheses are tested, i.e.. the null hypothesis that group means are equal. In this article, we propose an innovative solution to these above-mentioned issues using Bayesian model selection, which we illustrate using a case study.

The all-too-flexible abductive method: ATOM's normative status.

The author discusses the abductive theory of method (ATOM) by Brian Haig from a philosophical perspective, connecting his theory with a number of issues and trends in contemporary philosophy of science. It is argued that as it stands, the methodology presented by Haig is too permissive. Both the use of analogical reasoning and the application of exploratory factor analysis leave us with too many candidate theories to choose from, and explanatory coherence cannot be expected to save the day. The author ends with some suggestions to remedy the permissiveness and lack of normative force in ATOM, deriving from the experimental practice within which psychological data are produced.

Measurement invariance versus selection invariance: is fair selection possible?

This article shows that measurement invariance (defined in terms of an invariant measurement model in different groups) is generally inconsistent with selection invariance (defined in terms of equal sensitivity and specificity across groups). In particular, when a unidimensional measurement instrument is used and group differences are present in the location but not in the variance of the latent distribution, sensitivity and positive predictive value will be higher in the group at the higher end of the latent dimension, whereas specificity and negative predictive value will be higher in the group at the lower end of the latent dimension. When latent variances are unequal, the differences in these quantities depend on the size of group differences in variances relative to the size of group differences in means. The effect originates as a special case of Simpson's paradox, which arises because the observed score distribution is collapsed into an accept-reject dichotomy. Simulations show the effect can be substantial in realistic situations. It is suggested that the effect may be partly responsible for overprediction in minority groups as typically found in empirical studies on differential academic performance. A methodological solution to the problem is suggested, and social policy implications are discussed.