ABSTRACT
The authors modeled sources of error variance in job specification ratings collected from 3 levels of raters across 5 organizations (N=381). Variance components models were used to estimate the variance in ratings attributable to true score (variance between knowledge, skills, abilities, and other characteristics [KSAOs]) and error (KSAO-by-rater and residual variance). Subsequent models partitioned error variance into components related to the organization, position level, and demographic characteristics of the raters. Analyses revealed that the differential ordering of KSAOs by raters was not a function of these characteristics but rather was due to unexplained rating differences among the raters. The implications of these results for job specification and validity transportability are discussed.
Subject(s)
Task Performance and Analysis , Adult , Analysis of Variance , Female , Humans , Likelihood Functions , Male , Models, Statistical , Observer Variation , Psychometrics/methods , Reproducibility of Results , United StatesABSTRACT
Previous studies of standardized ethnic group differences in the employment interview have shown differences to be relatively small. Unfortunately, many researchers conducting interview studies have not considered the issue of range restriction in research design. This omission is likely to lead to underestimates of standardized ethnic group differences (d) when the interview is considered as an initial screening device or used in combination with other initial screening devices. The authors found that 2 forms of a behavioral interview were associated with standardized ethnic group differences of .36 and .56 when corrected for range restriction. These differences are substantially larger than previously thought and demonstrate the importance of considering a variety of study design characteristics in obtaining the appropriate parameter estimates.
