Enhancing Interpretability in Factor Analysis by Means of Mathematical Optimization.

Exploratory Factor Analysis (EFA) is a widely used statistical technique to discover the structure of latent unobserved variables, called factors, from a set of observed variables. EFA exploits the property of rotation invariance of the factor model to enhance factors' interpretability by building a sparse loading matrix. In this paper, we propose an optimization-based procedure to give meaning to the factors arising in EFA by means of an additional set of variables, called explanatory variables, which may include in particular the set of observed variables. A goodness-of-fit criterion is introduced which quantifies the quality of the interpretation given this way. Our methodology also exploits the rotational invariance of EFA to obtain the best orthogonal rotation of the factors, in terms of the goodness-of-fit, but making them match to some of the explanatory variables, thus going beyond traditional rotation methods. Therefore, our approach allows the analyst to interpret the factors not only in terms of the observed variables, but in terms of a broader set of variables. Our experimental results demonstrate how our approach enhances interpretability in EFA, first in an empirical dataset, concerning volumes of reservoirs in California, and second in a synthetic data example.

The Infinitesimal Jackknife and Moment Structure Analysis Using Higher Order Moments.

Mean corrected higher order sample moments are asymptotically normally distributed. It is shown that both in the literature and popular software the estimates of their asymptotic covariance matrices are incorrect. An introduction to the infinitesimal jackknife is given and it is shown how to use it to correctly estimate the asymptotic covariance matrices of higher order sample moments. Another advantage in using the infinitesimal jackknife is the ease with which it may be used when stacking or sub-setting estimators. The estimates given are used to test the goodness of fit of a non-linear factor analysis model. A computationally accelerated form for infinitesimal jackknife estimates is given.

A Theorem on the Rank of a Product of Matrices with Illustration of Its Use in Goodness of Fit Testing.

This paper develops a theorem that facilitates computing the degrees of freedom of Wald-type chi-square tests for moment restrictions when there is rank deficiency of key matrices involved in the definition of the test. An if and only if (iff) condition is developed for a simple rule of difference of ranks to be used when computing the desired degrees of freedom of the test. The theorem is developed exploiting basics tools of matrix algebra. The theorem is shown to play a key role in proving the asymptotic chi-squaredness of a goodness of fit test in moment structure analysis, and in finding the degrees of freedom of this chi-square statistic.

The nonsingularity of γ in covariance structure analysis of nonnormal data.

Covariance structure analysis of nonnormal data is important because in practice all data are nonnormal. When applying covariance structure analysis to nonnormal data, it is generally assumed that the asymptotic covariance matrix Γ for the nonredundant terms in the sample covariance matrix S is nonsingular. It is shown this need not be the case, which raises a question of how restrictive this assumption may be and how difficult it may be to verify it. It is shown that Γ is nonsingular whenever sampling is from a nonsingular distribution, including any distribution defined by a density function. In the discrete case necessary and sufficient conditions are given for the nonsingularity of Γ, and it is shown how to demonstrate Γ is nonsingular with high probability. Thus, the nonsingularity of Γ assumption is mild and one should feel comfortable about making it. These observations also apply to the asymptotic covariance matrix Γ that arises in structural equation modeling.

Continuous orthogonal complement functions and distribution-free goodness of fit tests in moment structure analysis.

It is shown that for any full column rank matrix X 0 with more rows than columns there is a neighborhood [Formula: see text] of X 0 and a continuous function f on [Formula: see text] such that f(X) is an orthogonal complement of X for all X in [Formula: see text]. This is used to derive a distribution free goodness of fit test for covariance structure analysis. This test was proposed some time ago and is extensively used. Unfortunately, there is an error in the proof that the proposed test statistic has an asymptotic χ (2) distribution. This is a potentially serious problem, without a proof the test statistic may not, in fact, be asymptoticly χ (2). The proof, however, is easily fixed using a continuous orthogonal complement function. Similar problems arise in other applications where orthogonal complements are used. These can also be resolved by using continuous orthogonal complement functions.

Generating Nonnormal Multivariate Data Using Copulas: Applications to SEM.

This article develops a procedure based on copulas to simulate multivariate nonnormal data that satisfy a prespecified variance-covariance matrix. The covariance matrix used can comply with a specific moment structure form (e.g., a factor analysis or a general structural equation model). Thus, the method is particularly useful for Monte Carlo evaluation of structural equation models within the context of nonnormal data. The new procedure for nonnormal data simulation is theoretically described and also implemented in the widely used R environment. The quality of the method is assessed by Monte Carlo simulations. A 1-sample test on the observed covariance matrix based on the copula methodology is proposed. This new test for evaluating the quality of a simulation is defined through a particular structural model specification and is robust against normality violations.

Ensuring Positiveness of the Scaled Difference Chi-square Test Statistic.

A scaled difference test statistic [Formula: see text] that can be computed from standard software of structural equation models (SEM) by hand calculations was proposed in Satorra and Bentler (2001). The statistic [Formula: see text] is asymptotically equivalent to the scaled difference test statistic T̄(d) introduced in Satorra (2000), which requires more involved computations beyond standard output of SEM software. The test statistic [Formula: see text] has been widely used in practice, but in some applications it is negative due to negativity of its associated scaling correction. Using the implicit function theorem, this note develops an improved scaling correction leading to a new scaled difference statistic T̄(d) that avoids negative chi-square values.

Testing model nesting and equivalence.

When using existing technology, it can be hard or impossible to determine whether two structural equation models that are being considered may be nested. There is also no routine technology for evaluating whether two very different structural models may be equivalent. A simple nesting and equivalence testing (NET) procedure is proposed that uses random sample and model-reproduced moment matrices to evaluate both model nesting and equivalence. The analysis is "local" rather than "global" in nature, but its use with simulation or bootstrapping can imply global conclusions. Two standard applications of NET are to verify whether or not two proposed models are equivalent and whether a baseline model used in an incremental fit index is appropriately nested.

Smoking and Cancers: Case-robust Analysis of a Classic Data Set.

A typical structural equation model is intended to reproduce the means, variances, and correlations or covariances among a set of variables based on parameter estimates of a highly restricted model. It is not widely appreciated that the sample statistics being modeled can be quite sensitive to outliers and influential observations leading to bias in model parameter estimates. A classic public epidemiological data set on the relation between cigarette purchases and rates of four types of cancer among states in the USA is studied with case-weighting methods that reduce the influence of a few cases on the overall results. The results support and extend the original conclusions; the standardized effect of smoking on a factor underlying deaths from bladder and lung cancer is .79.