Anchor Selection Using the Wald Test Anchor-All-Test-All Procedure.

Methods for testing differential item functioning (DIF) require that the reference and focal groups are linked on a common scale using group-invariant anchor items. Several anchor-selection strategies have been introduced in an item response theory framework. However, popular strategies often utilize likelihood ratio testing with all-others-as-anchors that requires multiple model fittings. The current study explored alternative anchor-selection strategies based on a modified version of the Wald χ2 test that is implemented in flexMIRT and IRTPRO, and made comparisons with methods based on the popular likelihood ratio test. Accuracies of anchor identification of four different strategies (two testing methods combined with two selection criteria), along with the power and Type I error associated with respective follow-up DIF tests, will be presented. Implications for applied researchers and suggestions for future research will be discussed.

How Item Residual Heterogeneity Affects Tests for Differential Item Functioning.

Differential item functioning (DIF) occurs when people with the same proficiency have different probabilities of giving a certain response to an item. The present study focused on an assumption implicit in popular methods for DIF testing that has received little attention in published literature (item residual homogeneity). The assumption is explained, a strategy for detecting violations of it (i.e., item residual heterogeneity) is illustrated with empirical data, and simulations are carried out to evaluate the performance of binary logistic regression, two-group item response theory (IRT), and the Mantel-Haenszel (MH) test in the presence of item residual heterogeneity. Results indicated that heterogeneity inflated Type I error and attenuated power for logistic regression, and attenuated power and produced biased estimates of the latent focal group mean and standard deviation for two-group IRT. The MH test was robust to item residual heterogeneity, probably because it does not use the logistic function.

Assessing the Straightforwardly-Worded Brief Fear of Negative Evaluation Scale for Differential Item Functioning Across Gender and Ethnicity.

The Brief Fear of Negative Evaluation Scale (BFNE; Leary Personality and Social Psychology Bulletin, 9, 371-375, 1983) assesses fear and worry about receiving negative evaluation from others. Rodebaugh et al. Psychological Assessment, 16, 169-181, (2004) found that the BFNE is composed of a reverse-worded factor (BFNE-R) and straightforwardly-worded factor (BFNE-S). Further, they found the BFNE-S to have better psychometric properties and provide more information than the BFNE-R. Currently there is a lack of research regarding the measurement invariance of the BFNE-S across gender and ethnicity with respect to item thresholds. The present study uses item response theory (IRT) to test the BFNE-S for differential item functioning (DIF) related to gender and ethnicity (White, Asian, and Black). Six data sets consisting of clinical, community, and undergraduate participants were utilized (N=2,109). The factor structure of the BFNE-S was confirmed using categorical confirmatory factor analysis, IRT model assumptions were tested, and the BFNE-S was evaluated for DIF. Item nine demonstrated significant non-uniform DIF between White and Black participants. No other items showed significant uniform or non-uniform DIF across gender or ethnicity. Results suggest the BFNE-S can be used reliably with men and women and Asian and White participants. More research is needed to understand the implications of using the BFNE-S with Black participants.

How bandwidth selection algorithms impact exploratory data analysis using kernel density estimation.

Exploratory data analysis (EDA) can reveal important features of underlying distributions, and these features often have an impact on inferences and conclusions drawn from data. Graphical analysis is central to EDA, and graphical representations of distributions often benefit from smoothing. A viable method of estimating and graphing the underlying density in EDA is kernel density estimation (KDE). This article provides an introduction to KDE and examines alternative methods for specifying the smoothing bandwidth in terms of their ability to recover the true density. We also illustrate the comparison and use of KDE methods with 2 empirical examples. Simulations were carried out in which we compared 8 bandwidth selection methods (Sheather-Jones plug-in [SJDP], normal rule of thumb, Silverman's rule of thumb, least squares cross-validation, biased cross-validation, and 3 adaptive kernel estimators) using 5 true density shapes (standard normal, positively skewed, bimodal, skewed bimodal, and standard lognormal) and 9 sample sizes (15, 25, 50, 75, 100, 250, 500, 1,000, 2,000). Results indicate that, overall, SJDP outperformed all methods. However, for smaller sample sizes (25 to 100) either biased cross-validation or Silverman's rule of thumb was recommended, and for larger sample sizes the adaptive kernel estimator with SJDP was recommended. Information is provided about implementing the recommendations in the R computing language.

Evaluation of dominance-based ordinal multiple regression for variables with few categories.

Dominance-based ordinal multiple regression (DOR) is designed to answer ordinal questions about relationships among ordinal variables. Only one parameter per predictor is estimated, and the number of parameters is constant for any number of outcome levels. The majority of existing simulation evaluations of DOR use predictors that are continuous or ordinal with many categories, so the performance of the method is not well understood for ordinal variables with few categories. This research evaluates DOR in simulations using three-category ordinal variables for the outcome and predictors, with a comparison to the cumulative logits proportional odds model (POC). Although ordinary least squares (OLS) regression is inapplicable for theoretical reasons, it was also included in the simulations because of its popularity in the social sciences. Most simulation outcomes indicated that DOR performs well for variables with few categories, and is preferable to the POC for smaller samples and when the proportional odds assumption is violated. Nevertheless, confidence interval coverage for DOR was not flawless and possibilities for improvement are suggested.

Cross-ethnic measurement equivalence of measures of depression, social anxiety, and worry.

Although study of clinical phenomena in individuals from different ethnic backgrounds has improved over the years, African American and Asian American individuals continue to be underrepresented in research samples. Without adequate psychometric data about how questionnaires perform in individuals from different ethnic samples, findings from both within and across groups are arguably uninterpretable. Analyses based on item response theory (IRT) allow us to make fine-grained comparisons of the ways individuals from different ethnic groups respond to clinical measures. This study compared response patterns of African American and Asian American undergraduates to White undergraduates on measures of depression, social anxiety, and worry. On the Beck Depression Inventory-II, response patterns for African American participants were roughly equivalent to the response patterns of White participants. On measures of worry and social anxiety, there were substantial differences, suggesting that the use of these measures in African American and Asian American populations may lead to biased conclusions.

Development and initial validation of an abbreviated Spider Phobia Questionnaire using item response theory.

An abbreviated Spider Phobia Questionnaire (SPQ) was developed using methods based in item response theory. Fifteen of the 31 SPQ items that demonstrated good to excellent discrimination along the spider fear continuum were retained in Study 1 that consisted of 1,555 nonclinical and clinical participants. The SPQ-15 demonstrated good internal consistency and correlated highly with the full SPQ. Structural equation modeling revealed that the SPQ-15 demonstrated excellent convergent validity, with strong associations with small animal disgust and other phobic symptoms. Supportive evidence was also found for divergent validity in relation to panic-related symptoms. The SPQ-15 was uniquely predictive of avoidance behavior and fear and disgust responding towards spiders in nonclinical, analogue, and treatment-seeking samples in Studies 2, 3, and 4. Lastly, in Study 5, the SPQ-15 was sensitive to the effects of exposure-based treatment. These findings suggest that the SPQ-15 has considerable strengths, including decreased assessment and scoring time while retaining high reliability, validity, and sensitivity.

Illustration of MIMIC-Model DIF Testing with the Schedule for Nonadaptive and Adaptive Personality.

This research provides an example of testing for differential item functioning (DIF) using multiple indicator multiple cause (MIMIC) structural equation models. True/False items on five scales of the Schedule for Nonadaptive and Adaptive Personality (SNAP) were tested for uniform DIF in a sample of Air Force recruits with groups defined by gender and ethnicity. Uniform DIF exists when an item is more easily endorsed for one group than the other, controlling for group mean differences on the variable under study. Results revealed significant DIF for many SNAP items and some effects were quite large. Differentially-functioning items can produce measurement bias and should be either deleted or modeled as if separate items were administered to different groups. Future research should aim to determine whether the DIF observed here holds for other samples.

Consistent Small-Sample Variances for Six Gamma-Family Measures of Ordinal Association.

Gamma-family measures are bivariate ordinal correlation measures that form a family because they all reduce to Goodman and Kruskal's gamma in the absence of ties (1954) . For several gamma-family indices, more than one variance estimator has been introduced. In previous research, the "consistent" variance estimator described by Cliff and colleagues produced more accurate confidence intervals (CIs) than 3 other types of variances. However, the Cliff consistent (CC) variance is available for only 4 of 10 gamma-family measures. In the present research, a CC variance is derived for the remaining 6 gamma-family measures, and CIs constructed with the CC variance are compared with CIs constructed using other available variance estimators. The CIs are evaluated and compared in simulations and illustrated with scores on the Disinhibition and Avoidance scales of the Schedule for Nonadaptive and Adaptive Personality. Computer code for calculating all 10 gamma-family measures and their CC variances (using the R program) is available on a Web site ( http://www.artsci.wustl.edu/~cwoods/CCvarR.html ).

Evaluation of MIMIC-Model Methods for DIF Testing With Comparison to Two-Group Analysis.

Differential item functioning (DIF) occurs when an item on a test or questionnaire has different measurement properties for 1 group of people versus another, irrespective of mean differences on the construct. This study focuses on the use of multiple-indicator multiple-cause (MIMIC) structural equation models for DIF testing, parameterized as item response models. The accuracy of these methods, and the sample size requirements, are not well established. This study examines the accuracy of MIMIC methods for DIF testing when the focal group is small and compares results with those obtained using 2-group item response theory (IRT). Results support the utility of the MIMIC approach. With small focal-group samples, tests of uniform DIF with binary or 5-category ordinal responses were more accurate with MIMIC models than 2-group IRT. Recommendations are offered for the application of MIMIC methods for DIF testing.

Detection of aberrant responding on a personality scale in a military sample: an application of evaluating person fit with two-level logistic regression.

Person-fit assessment is used to identify persons who respond aberrantly to a test or questionnaire. In this study, S. P. Reise's (2000) method for evaluating person fit using 2-level logistic regression was applied to 13 personality scales of the Schedule for Nonadaptive and Adaptive Personality (SNAP; L. Clark, 1996) that had been administered to military recruits (N = 2,026). Results revealed significant person-fit heterogeneity and indicated that for 5 SNAP scales (Disinhibition, Entitlement, Exhibitionism, Negative Temperament, and Workaholism), the scale was more discriminating for some people than for others. Possible causes of aberrant responding were explored with several covariates. On all 5 scales, severe pathology emerged as a key influence on responses, and there was evidence of differential test functioning with respect to gender, ethnicity, or both. Other potential sources of aberrancy were carelessness, haphazard responding, or uncooperativeness. Social desirability was not as influential as expected.

Monte Carlo Evaluation of Two-Level Logistic Regression for Assessing Person Fit.

Person fit is the degree to which an item response model fits for individual examinees. Reise (2000) described how two-level logistic regression can be used to detect heterogeneity in person fit, evaluate potential predictors of person fit heterogeneity, and identify potentially aberrant individuals. The method has apparently never been applied to empirical data or evaluated in a simulation study. The present research applies Reise's method to empirical data obtained from university undergraduates measured on the Fear of Negative Evaluation scale. Additionally, Reise's method is evaluated under conditions varying according to the type of aberrancy, level of test reliability, and scale length. Statistical power to detect aberrancy was highly dependent on manipulated variables, and some results were affected by bias in model parameters that was due to the aberrant responders. Nevertheless, Reise's method generally performed well and detected aberrant individuals either as well as, or better than, the well-established l z person-fit statistic.

Overdiagnosis and underdiagnosis of personality disorders in older adults.

OBJECTIVE: Recent evidence suggests that some of the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition personality disorder (PD) criteria contain measurement bias across age groups. Specifically, this research showed that younger and older adults were differentially likely to endorse certain PD criteria, even when both groups were statistically matched using mechanisms of item response theory (IRT) for degree of PD pathology. For the analyses presented here, the authors used data from a large epidemiological study (N = 43,093), the National Epidemiologic Survey on Alcohol and Related Conditions, to examine the influence of this item-level measurement bias for reaching accurate algorithmic PD diagnoses of older adults. METHODS: Joint probability analyses were used to determine the net effect of the item-level bias on the possible over- or underdiagnosis of six PDs in older adults. RESULTS: When older adults were compared to younger adults at equivalent levels of PD pathology, they were more likely to receive diagnoses of obsessive-compulsive and schizoid PDs. In contrast, they were less likely to receive diagnoses of avoidant and dependent PDs. Younger and older adults were equally likely to receive diagnoses of histrionic and paranoid PDs. Of the seven PDs assessed in this dataset, only these six lend themselves to this type of analysis; antisocial PD differs because a diagnosis depends upon the presence of conduct disorder. CONCLUSION: These findings raise concerns regarding the interpretation of existing older adult PD prevalence data.

Confidence intervals for gamma-family measures of ordinal association.

This research focused on confidence intervals (CIs) for 10 measures of monotonic association between ordinal variables. Standard errors (SEs) were also reviewed because more than 1 formula was available per index. For 5 indices, an element of the formula used to compute an SE is given that is apparently new. CIs computed with different SEs were compared in simulations with small samples (N = 25, 50, 75, or 100) for variables with 4 or 5 categories. With N > 25, many CIs performed well. Performance was best for consistent CIs due to N. Cliff and colleagues (N. Cliff, 1996; N. Cliff & V. Charlin, 1991; J. D. Long & N. Cliff, 1997). CIs for Spearman's rank correlation were also examined: Parameter coverage was erratic and sometimes egregiously underestimated.

Model stability of the 15-item Geriatric Depression Scale across cognitive impairment and severe depression.

The 15-item Geriatric Depression Scale (GDS) is used in a wide variety of clinical and research settings. The study's purpose was to further establish the validity of the 15-item GDS by exploring the underlying factor structure in a healthy, nondemented sample of older adults and then analyzing whether this factor structure remained stable across a sample of demented individuals and a sample of individuals with a history of depression 6 months after discharge from an inpatient psychiatric setting. A 2-factor model fit the data best in the exploratory analyses. The 2 factors, Life Satisfaction and General Depressive Affect, found in the nondemented sample (r = .39) remained stable across cognitive impairment (r = .12) but merged into a 1-factor model in the psychiatric sample (r = .93). The results indicate that nondepressed older adults with poor life satisfaction may be identified as depressed on screening instruments such as the 15-item GDS.

The reverse of social anxiety is not always the opposite: the reverse-scored items of the social interaction anxiety scale do not belong.

Although well-used and empirically supported, the Social Interaction Anxiety Scale (SIAS) has a questionable factor structure and includes reverse-scored items with questionable utility. Here, using samples of undergraduates and a sample of clients with social anxiety disorder, we extend previous work that opened the question of whether the reverse-scored items belong on the scale. First, we successfully confirmed the factor structure obtained in previous samples. Second, we found the reverse-scored items to show consistently weaker relationships with a variety of comparison measures. Third, we demonstrated that removing the reverse-scored questions generally helps rather than hinders the psychometric performance of the SIAS total score. Fourth, we found that the reverse-scored items show a strong relationship with the normal personality characteristic of extraversion, suggesting that the reverse-scored items may primarily assess extraversion. Given the above results, we suggest investigators consider performing data analyses using only the straightforwardly worded items of the SIAS.

An item response theory analysis of DSM-IV personality disorder criteria across younger and older age groups.

Many of the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV; American Psychiatric Association, 1994) personality disorder (PD) diagnostic criteria focus on a younger social and occupational context. The absence of age-appropriate criteria for older adults forces researchers and clinicians to draw conclusions based on existing criteria, which are likely inadequate. To explore which DSM-IV PD criteria contain age group measurement bias, the authors report 2 analyses of data on nearly 37,000 participants, ages 18-98 years, taken from a public data set that includes 7 of the 10 PDs (antisocial, avoidant, dependent, histrionic, obsessive-compulsive, paranoid, and schizoid). The 1st analysis revealed that older age groups tend to endorse fewer PD criteria than younger age groups. The 2nd analysis revealed that 29% of the criteria contain measurement bias. Although the latent variable structure for each PD was quite similar across younger and older age groups, some individual criteria were differentially endorsed by younger and older adults with equivalent PD pathology. The presence of measurement bias for these criteria raises questions concerning the assessment of PDs in older adults and the interpretation of existing data.

Ramsay-curve item response theory (RC-IRT) to detect and correct for nonnormal latent variables.

Popular methods for fitting unidimensional item response theory (IRT) models to data assume that the latent variable is normally distributed in the population of respondents, but this can be unreasonable for some variables. Ramsay-curve IRT (RC-IRT) was developed to detect and correct for this nonnormality. The primary aims of this article are to introduce RC-IRT less technically than it has been described elsewhere; to evaluate RC-IRT for ordinal data via simulation, including new approaches for model selection; and to illustrate RC-IRT with empirical examples. The empirical examples demonstrate the utility of RC-IRT for real data, and the simulation study indicates that when the latent distribution is skewed, RC-IRT results can be more accurate than those based on the normal model. Along with a plot of candidate curves, the Hannan-Quinn criterion is recommended for model selection.

The factor structure and screening utility of the Social Interaction Anxiety Scale.

The widely used Social Interaction Anxiety Scale (SIAS; R. P. Mattick & J. C. Clarke, 1998) possesses favorable psychometric properties, but questions remain concerning its factor structure and item properties. Analyses included 445 people with social anxiety disorder and 1,689 undergraduates. Simple unifactorial models fit poorly, and models that accounted for differences due to item wording (i.e., reverse scoring) provided superior fit. It was further found that clients and undergraduates approached some items differently, and the SIAS may be somewhat overly conservative in selecting analogue participants from an undergraduate sample. Overall, this study provides support for the excellent properties of the SIAS's straightforwardly worded items, although questions remain regarding its reverse-scored items.

Disgust sensitivity and obsessive-compulsive symptoms in a non-clinical sample.

Disgust sensitivity has been posited to play a role in the etiology and/or maintenance of obsessive-compulsive disorder (OCD); however, results of studies in this area have been mixed. We examined the relationship between specific domains of disgust sensitivity and specific OCD symptom patterns. One thousand and five undergraduate volunteers completed an internet battery of questionnaires including measures of OCD symptoms, depression, anxiety, and disgust sensitivity. Results indicated that even when controlling for depression and anxiety, several OCD symptom groups (checking, ordering, and washing) were associated with disgust sensitivity. Analysis of residuals, in which we controlled for every other OCD and disgust sensitivity domain for each paired comparison, indicated that the clearest relationship was between washing symptoms of OCD and disgust sensitivity toward hygiene-related stimuli. Examination of these items, however, raises questions about whether commonly accepted measures of disgust sensitivity might confound disgust with other forms of aversion. We discuss possible strategies for clarifying the degree to which fear and disgust are involved in OCD symptoms.