Measurement invariance in the social sciences: Historical development, methodological challenges, state of the art, and future perspectives.

This review summarizes the current state of the art of statistical and (survey) methodological research on measurement (non)invariance, which is considered a core challenge for the comparative social sciences. After outlining the historical roots, conceptual details, and standard procedures for measurement invariance testing, the paper focuses in particular on the statistical developments that have been achieved in the last 10 years. These include Bayesian approximate measurement invariance, the alignment method, measurement invariance testing within the multilevel modeling framework, mixture multigroup factor analysis, the measurement invariance explorer, and the response shift-true change decomposition approach. Furthermore, the contribution of survey methodological research to the construction of invariant measurement instruments is explicitly addressed and highlighted, including the issues of design decisions, pretesting, scale adoption, and translation. The paper ends with an outlook on future research perspectives.

Approximate Measurement Invariance of Willingness to Sacrifice for the Environment Across 30 Countries: The Importance of Prior Distributions and Their Visualization.

Nationwide opinions and international attitudes toward climate and environmental change are receiving increasing attention in both scientific and political communities. An often used way to measure these attitudes is by large-scale social surveys. However, the assumption for a valid country comparison, measurement invariance, is often not met, especially when a large number of countries are being compared. This makes a ranking of countries by the mean of a latent variable potentially unstable, and may lead to untrustworthy conclusions. Recently, more liberal approaches to assessing measurement invariance have been proposed, such as the alignment method in combination with Bayesian approximate measurement invariance. However, the effect of prior variances on the assessment procedure and substantive conclusions is often not well understood. In this article, we tested for measurement invariance of the latent variable "willingness to sacrifice for the environment" using Maximum Likelihood Multigroup Confirmatory Factor Analysis and Bayesian approximate measurement invariance, both with and without alignment optimization. For the Bayesian models, we used multiple priors to assess the impact on the rank order stability of countries. The results are visualized in such a way that the effect of different prior variances and models on group means and rankings becomes clear. We show that even when models appear to be a good fit to the data, there might still be an unwanted impact on the rank ordering of countries. From the results, we can conclude that people in Switzerland and South Korea are most motivated to sacrifice for the environment, while people in Latvia are less motivated to sacrifice for the environment.

Necessary but Insufficient: Why Measurement Invariance Tests Need Online Probing as a Complementary Tool.

Cross-national data production in social science research has increased dramatically in recent decades. Assessing the comparability of data is necessary before drawing substantive conclusions that are based on cross-national data. Researchers assessing data comparability typically use either quantitative methods such as multigroup confirmatory factor analysis or qualitative methods such as online probing. Because both methods have complementary strengths and weaknesses, this study applies both multigroup confirmatory factor analysis and online probing in a mixed-methods approach to assess the comparability of constructive patriotism and nationalism, two important concepts in the study of national identity. Previous measurement invariance tests failed to achieve scalar measurement invariance, which prohibits a cross-national comparison of latent means (Davidov 2009). The arrival of the 2013 ISSP Module on National Identity has encouraged a reassessment of both constructs and a push to understand why scalar invariance cannot be achieved. Using the example of constructive patriotism and nationalism, this study demonstrates how the combination of multigroup confirmatory factor analysis and online probing can uncover and explain issues related to cross-national comparability.