The effect of measurement error on the positive predictive value of PSW methods for SLD identification: How buffer zones dispel the illusion of inaccuracy.

Patterns of Strengths and Weaknesses (PSW) methods are widely used for identifying specific learning disabilities (SLDs). Several researchers, however, have reported that the diagnostic accuracy of PSW methods is unacceptably low when strict thresholds were used to identify students with SLDs. We believe these findings give a misleading impression of the magnitude of the diagnostic errors that are likely to arise in PSW assessments. In a simulation study of 10 million cases using a simplified PSW method for demonstration, most of what have been called diagnostic errors were cases in which observed scores and true scores fell on opposite sides of a strict threshold but were still within a buffer zone the size of a typical measurement error. Because small score differences do not result in meaningfully different case conceptualizations, the use of buffer zones reveals that previous estimates of the diagnostic accuracy of PSW methods are misleadingly low. We also demonstrate that diagnostic decisions become increasingly reliable when observed scores are comfortably distant from diagnostic thresholds. For practitioners, we present a flowchart and practical guidelines to improve the accuracy and stability of SLD identification decisions.

A cross-battery, reference variable, confirmatory factor analytic investigation of the CHC taxonomy.

The Cattell-Horn-Carroll (CHC) taxonomy has been used to classify and describe human cognitive abilities. The ability factors derived from the CHC taxonomy are often assumed to be invariant across multiple populations and intelligence batteries, which is an important assumption for research and assessment. In this study, data from five different test batteries that were collected during separate Kaufman Assessment Battery for Children-Second Edition (KABC-II; Kaufman & Kaufman, 2004) concurrent validity studies were factor-analyzed jointly. Because the KABC-II was administered to everyone in the validity studies, it was used as a reference battery to link the separate test batteries in a "cross-battery" confirmatory factor analysis. Some findings from this analysis were that CHC-based test classifications based on theory and prior research were straightforward and accurate, a first-order Fluid/Novel Reasoning (Gf) factor was equivalent to a second-order g factor, and sample heterogeneity related to SES and sex influenced factor loadings. It was also shown that a reference variable approach, used in studies that incorporate planned missingness into data collection, may be used successfully to analyze data from several test batteries and studies. One implication from these findings is that CHC theory should continue to serve as a useful guide that can be used for intelligence research, assessment, and test development.