Cross-validating the Dot Counting Test Among an Adult ADHD Clinical Sample and Analyzing the Effect of ADHD Subtype and Comorbid Psychopathology.

This study cross-validated the dot counting test (DCT) as a performance validity test (PVT) in an adult attention-deficit/hyperactivity disorder (ADHD) clinical population and examined the effect of ADHD subtype and psychiatric comorbidity on accuracy for detecting invalidity. DCT performance was assessed among 210 consecutive adult ADHD referrals who underwent neuropsychological evaluation and were classified into valid (n = 175) or invalid (n = 35) groups based on seven independent criterion PVTs. The invalid group had significantly worse DCT performance than the valid group using both the standard and unrounded scoring procedure (ηp2=.28). Classification accuracy was excellent, with 54.3% sensitivity/92% specificity at optimal cut-scores of ≥14 (rounded) and ≥13.38 (unrounded). Nonsignificant DCT performance differences emerged based on ADHD subtype or the presence/absence of comorbid psychopathology. The DCT functions well as a nonmemory-based PVT in an ethnoracially diverse ADHD population, supporting its clinical utility for detecting invalid neurocognitive performance during ADHD evaluations.

Examining independent and combined accuracy of embedded performance validity tests in the California Verbal Learning Test-II and Brief Visuospatial Memory Test-Revised for detecting invalid performance.

The California Verbal Learning Test-Second Edition (CVLT-II) Forced Choice Recognition (FC) and Brief Visuospatial Memory Test-Revised (BVMT-R) Recognition Discrimination Index (RD) are embedded performance validity tests (PVTs) assessing material-specific neuropsychological processes (i.e., verbal and visual memory, respectively). Prior research demonstrated the utility of these PVTs independently; however, no study has compared their diagnostic accuracy for identifying invalid performance relative to each other and in combination within a single sample. This cross-sectional study included an adult neuropsychiatric sample who underwent neuropsychological evaluation. Validity groups were determined via independent criterion PVT performance, and consisted of 103 participants with valid and 25 with invalid neurocognitive performance. FC and RD were not significantly correlated (r = .154), yet both differed between validity groups (ηp2 = .14-.19). Previously established FC (≤14) and RD (≤4) cutoffs evidenced 32-40% sensitivity/90-98% specificity, though receiver operating characteristic (ROC) analyses indicated a more liberal FC cutoff (≤15) was optimal. Logistic regression models utilizing both embedded PVTs indicated that FC did not significantly improve classification accuracy above and beyond RD. Results support the clinical utility of existing cutoffs for FC and RD for independently identifying invalid performance, though the latter showed relatively better ability to detect invalid performance when both are used together.

Impaired or invalid? Limitations of assessing performance validity using the Boston Naming Test.

The Boston Naming Test (BNT) has been proposed as an embedded performance validity test (PVT), though replication is needed to provide further empirical support of its simultaneous use as a cognitive ability measure and embedded PVT. This cross-sectional study examined BNT performance in a mixed neuropsychiatric sample of 137 patients with/without cognitive impairment. Four independent criterion PVTs classified 109 (80%) as valid and 28 (20%) as invalid. BNT raw and demographically-corrected T-scores were significantly higher among the valid group with small effect sizes (ηp2 = 0.04-0.05). Raw/T-scores differentiated valid/invalid groups, but with low classification accuracy (areas under the curve [AUCs] = 0.68/0.63), and unacceptably weak sensitivities (i.e. 7%/18%). When separated by impairment status, raw score accuracy appreciably increased (AUC = 0.87; 61% sensitivity/89% specificity) among unimpaired patients, whereas T-score accuracy, while significant, remained low (AUC = 0.68; 21% sensitivity/89% specificity). Conversely, among impaired patients, neither the raw (AUC = 0.59) nor T-score (AUC = 0.60) accurately identified invalid performance. In sum, BNT scores were not able to differentiate valid from invalid performance when cognitive impairment was present, and therefore showed limited overall utility as embedded PVTs. These findings further caution against inferring performance validity from measures in which a single score is used to assess both cognitive ability and validity.

Every second counts: A comparison of four dot counting test scoring procedures for detecting invalid neuropsychological test performance.

Although performance validity tests (PVTs) are an integral element of neuropsychological assessment, most PVTs have historically been restricted to the memory domain. The Dot Counting Test (DCT) is a nonmemory PVT shown to reliably identify invalid performance. Although several traditional and abbreviated scoring methods have been derived, no study to date has directly compared the available scoring approaches within a single sample. This cross-sectional study cross-validated 4 different DCT scoring approaches, including the traditional rounded E-score proposed within the manual, an unrounded E-score, and 2 abbreviated scoring procedures based on 4- and 6-card versions (DCT-4 and DCT-6, respectively) in a diverse mixed clinical neuropsychiatric sample (N = 132). Validity groups were established by 5 independent criterion PVTs (102 valid and 30 invalid). Receiver operating characteristic curve analyses yielded significant areas under the curve (AUCs = .84-.86) for the overall sample, with sensitivities of 50%-67% at ≥ 89% specificity. The DCT scores had outstanding classification accuracy (AUCs ≥ .92; sensitivities = 80%-83%) in the unimpaired group and excellent classification accuracy in the impaired group (AUCs = .79-.81; sensitivities = 43%-60%). Whereas negligible differences emerged between the 4 scoring methods for the cognitively intact group, the DCT-4 showed notably stronger psychometric properties among the overall sample in general and the mild cognitive impairment group in particular. Results corroborate previous findings suggesting that the DCT is a robust PVT, regardless of the employed scoring procedure, and replicate support for the abbreviated DCT-4 as the recommended validity indicator. (PsycInfo Database Record (c) 2021 APA, all rights reserved).