Incorporating Uncertainty Into Parallel Analysis for Choosing the Number of Factors via Bayesian Methods.

A number of psychometricians have suggested that parallel analysis (PA) tends to yield more accurate results in determining the number of factors in comparison with other statistical methods. Nevertheless, all too often PA can suggest an incorrect number of factors, particularly in statistically unfavorable conditions (e.g., small sample sizes and low factor loadings). Because of this, researchers have recommended using multiple methods to make judgments about the number of factors to extract. Implicit in this recommendation is that, when the number of factors is chosen based on PA, uncertainty nevertheless exists. We propose a Bayesian parallel analysis (B-PA) method to incorporate the uncertainty with decisions about the number of factors. B-PA yields a probability distribution for the various possible numbers of factors. We implement and compare B-PA with a frequentist approach, revised parallel analysis (R-PA), in the contexts of real and simulated data. Results show that B-PA provides relevant information regarding the uncertainty in determining the number of factors, particularly under conditions with small sample sizes, low factor loadings, and less distinguishable factors. Even if the indicated number of factors with the highest probability is incorrect, B-PA can show a sizable probability of retaining the correct number of factors. Interestingly, when the mode of the distribution of the probabilities associated with different numbers of factors was treated as the number of factors to retain, B-PA was somewhat more accurate than R-PA in a majority of the conditions.

An SEM Assessment of the Internal Structure and Predictive Validity of the Abbreviated Early Adolescent HOME Inventory.

The Home Observation for Measurement of the Environment (HOME) Inventory is designed to assess the quality and quantity of support, stimulation, and structure provided to children in the home environment. HOME has been widely used for research and applied purposes. We focused on an abbreviated version of the Early Adolescent HOME (EA-HOME-A) that was administered to 15-year-old adolescents and their parents (N = 958) as part of the NICHD (National Institute of Child Health and Human Development) Study of Early Child Care and Youth Development. Our study had two objectives. First, we hypothesized and tested a bifactor model that specified a general factor in support of the use of the HOME total score and group factors for subsets of items in support of the content domain scores. Second, we applied structural equation modeling to relate the EA-HOME-A factors to outcome factors assessing maladaptive behaviors, autonomy, self-control, and cognitive-academic performance. The results supported the construct validity of the EA-HOME-A with respect to its internal structure as well as its correlates.

Proportion of Indicator Common Variance Due to a Factor as an Effect Size Statistic in Revised Parallel Analysis.

Past research suggests revised parallel analysis (R-PA) tends to yield relatively accurate results in determining the number of factors in exploratory factor analysis. R-PA can be interpreted as a series of hypothesis tests. At each step in the series, a null hypothesis is tested that an additional factor accounts for zero common variance among measures in the population. Integration of an effect size statistic-the proportion of common variance (PCV)-into this testing process should allow for a more nuanced interpretation of R-PA results. In this article, we initially assessed the psychometric qualities of three PCV statistics that can be used in conjunction with principal axis factor analysis: the standard PCV statistic and two modifications of it. Based on analyses of generated data, the modification that considered only positive eigenvalues ( π ^ SMC : k ' + Λ ^ ) overall yielded the best results. Next, we examined PCV using minimum rank factor analysis, a method that avoids the extraction of negative eigenvalues. PCV with minimum rank factor analysis generally did not perform as well as π ^ SMC : k ' + Λ ^ , even with a relatively large sample size of 5,000. Finally, we investigated the use of π ^ SMC : k ' + Λ ^ in combination with R-PA and concluded that practitioners can gain additional information from π ^ SMC : k ' + Λ ^ and make more nuanced decision about the number of factors when R-PA fails to retain the correct number of factors.

The Problem with Having Two Watches: Assessment of Fit When RMSEA and CFI Disagree.

The root mean square error of approximation (RMSEA) and the comparative fit index (CFI) are two widely applied indices to assess fit of structural equation models. Because these two indices are viewed positively by researchers, one might presume that their values would yield comparable qualitative assessments of model fit for any data set. When RMSEA and CFI offer different evaluations of model fit, we argue that researchers are likely to be confused and potentially make incorrect research conclusions. We derive the necessary as well as the sufficient conditions for inconsistent interpretations of these indices. We also study inconsistency in results for RMSEA and CFI at the sample level. Rather than indicating that the model is misspecified in a particular manner or that there are any flaws in the data, the two indices can disagree because (a) they evaluate, by design, the magnitude of the model's fit function value from different perspectives; (b) the cutoff values for these indices are arbitrary; and (c) the meaning of "good" fit and its relationship with fit indices are not well understood. In the context of inconsistent judgments of fit using RMSEA and CFI, we discuss the implications of using cutoff values to evaluate model fit in practice and to design SEM studies.

Use of internal consistency coefficients for estimating reliability of experimental task scores.

Reliabilities of scores for experimental tasks are likely to differ from one study to another to the extent that the task stimuli change, the number of trials varies, the type of individuals taking the task changes, the administration conditions are altered, or the focal task variable differs. Given that reliabilities vary as a function of the design of these tasks and the characteristics of the individuals taking them, making inferences about the reliability of scores in an ongoing study based on reliability estimates from prior studies is precarious. Thus, it would be advantageous to estimate reliability based on data from the ongoing study. We argue that internal consistency estimates of reliability are underutilized for experimental task data and in many applications could provide this information using a single administration of a task. We discuss different methods for computing internal consistency estimates with a generalized coefficient alpha and the conditions under which these estimates are accurate. We illustrate use of these coefficients using data for three different tasks.

Accuracy of Revised and Traditional Parallel Analyses for Assessing Dimensionality with Binary Data.

Parallel analysis (PA) is a useful empirical tool for assessing the number of factors in exploratory factor analysis. On conceptual and empirical grounds, we argue for a revision to PA that makes it more consistent with hypothesis testing. Using Monte Carlo methods, we evaluated the relative accuracy of the revised PA (R-PA) and traditional PA (T-PA) methods for factor analysis of tetrachoric correlations between items with binary responses. We manipulated five data generation factors: number of observations, type of factor model, factor loadings, correlation between factors, and distribution of thresholds. The R-PA method tended to be more accurate than T-PA, although not uniformly across conditions. R-PA tended to perform better relative to T-PA if the underlying model (a) was unidimensional but had some unique items, (b) had highly correlated factors, or (c) had a general factor as well as a group factor. In addition, R-PA tended to outperform T-PA if items had higher factor loadings and sample size was large. A major disadvantage of the T-PA method was that it frequently yielded inflated Type I error rates.

Further discussion on reliability: the art of reliability estimation.

Sijtsma and van der Ark (2015) focused in their lead article on three frameworks for reliability estimation in nursing research: classical test theory (CTT), factor analysis (FA), and generalizability theory. We extend their presentation with particular attention to CTT and FA methods. We first consider the potential of yielding an overly negative or an overly positive assessment of reliability based on coefficient alpha. Next, we discuss other CTT methods for estimating reliability and how the choice of methods affects the interpretation of the reliability coefficient. Finally, we describe FA methods, which not only permit an understanding of a measure's underlying structure but also yield a variety of reliability coefficients with different interpretations. On a more general note, we discourage reporting reliability as a two-choice outcome--unsatisfactory or satisfactory; rather, we recommend that nursing researchers make a conceptual and empirical argument about when a measure might be more or less reliable, depending on its use.

Psychometric Evaluation of Lexical Diversity Indices: Assessing Length Effects.

PURPOSE: Several novel techniques have been developed recently to assess the breadth of a speaker's vocabulary exhibited in a language sample. The specific aim of this study was to increase our understanding of the validity of the scores generated by different lexical diversity (LD) estimation techniques. Four techniques were explored: D, Maas, measure of textual lexical diversity, and moving-average type-token ratio. METHOD: Four LD indices were estimated for language samples on 4 discourse tasks (procedures, eventcasts, story retell, and recounts) from 442 adults who are neurologically intact. The resulting data were analyzed using structural equation modeling. RESULTS: The scores for measure of textual lexical diversity and moving-average type-token ratio were stronger indicators of the LD of the language samples. The results for the other 2 techniques were consistent with the presence of method factors representing construct-irrelevant sources. CONCLUSION: These findings offer a deeper understanding of the relative validity of the 4 estimation techniques and should assist clinicians and researchers in the selection of LD measures of language samples that minimize construct-irrelevant sources.

Type I and Type II Error Rates and Overall Accuracy of the Revised Parallel Analysis Method for Determining the Number of Factors.

Traditional parallel analysis (T-PA) estimates the number of factors by sequentially comparing sample eigenvalues with eigenvalues for randomly generated data. Revised parallel analysis (R-PA) sequentially compares the kth eigenvalue for sample data to the kth eigenvalue for generated data sets, conditioned on k- 1 underlying factors. T-PA and R-PA are conceptualized as stepwise hypothesis-testing procedures and, thus, are alternatives to sequential likelihood ratio test (LRT) methods. We assessed the accuracy of T-PA, R-PA, and LRT methods using a Monte Carlo approach. Although no method was uniformly more accurate across all 180 conditions, the PA approaches outperformed LRT methods overall. Relative to T-PA, R-PA tended to perform better within the framework of hypothesis testing and to evidence greater accuracy in conditions with higher factor loadings.

Confirmatory factor analysis: an introduction for psychosomatic medicine researchers.

We present an introduction to the basic concepts essential to understanding confirmatory factor analysis (CFA). We initially discuss the underlying mathematical model and its graphical representation. We then show how parameters are estimated for the CFA model based on the maximum likelihood function. Finally, we discuss several ways in which model fit is evaluated as well as introduce the concept of model identification. In our presentation, we use an example to illustrate the application of CFA to psychosomatic research and touch on the more general role of structural equation modeling in psychosomatic research.

Structural equation modeling for conducting tests of differences in multiple means.

Multivariate methods for analyzing group differences in means on dependent variables include multivariate analysis of variance, discriminant analysis, and multivariate analysis of factor means. To make appropriate choices among these methods, researchers should understand the statistical models underlying them. We present these models using path diagrams within a structural equation modeling framework. Results for the different methods are presented for an example concerning coping with asthma.

A coefficient alpha for test-retest data.

Transient errors are caused by variations in feelings, moods, and mental states over time. If these errors are present, coefficient alpha is an inflated estimate of reliability. A true-score model is presented that incorporates transient errors for test-retest data, and a reliability estimate is derived. This estimate, referred to as the test-retest alpha, is less than coefficient alpha if transient error is present and is less susceptible to effects due to item recall than a test-retest correlation. An assumption underlying the test-retest alpha is essential tau equivalency of items. A test-retest split-half coefficient is presented as an alternative to the test-retest alpha when this assumption is violated. The test-retest alpha is the mean of all possible test-retest split-half coefficients.