Attorney demands for protected psychological test information: Is access necessary for cross examination or does it lead to misinformation? An interorganizational* position paper.

Objective: Some attorneys claim that to adequately cross examine neuropsychological experts, they require direct access to protected test information, rather than having test data analyzed by retained neuropsychological experts. The objective of this paper is to critically examine whether direct access to protected test materials by attorneys is indeed necessary, appropriate, and useful to the trier-of-fact. Method: Examples are provided of the types of nonscientific misinformation that occur when attorneys, who lack adequate training in testing, attempt to independently interpret neurocognitive/psychological test data. Results: Release of protected test information to attorneys introduces inaccurate information to the trier of fact, and jeopardizes future use of tests because non-psychologists are not ethically bound to protect test content. Conclusion: The public policy underlying the right of attorneys to seek possibly relevant documents should not outweigh the damage to tests and resultant misinformation that arise when protected test information is released directly to attorneys. The solution recommended by neuropsychological/psychological organizations and test publishers is to have protected psychological test information exchanged directly and only between clinical psychologist/neuropsychologist experts.

Cross-validation of RAVLT performance validity indicators and the RAVLT/RO discriminant function in a large known groups sample.

OBJECTIVE: To cross-validate RAVLT performance validity cut-offs and the RAVLT/RO discriminant function in a large neuropsychological sample. METHOD: RAVLT scores and the RAVLT/RO discriminant function were compared in credible (n = 100) and noncredible (n = 353) neuropsychology referrals. RESULTS: Noncredible patients scored lower than credible patients on RAVLT scores and the RAVLT/RO discriminant function. With cut-offs set to ≥90% specificity, highest sensitivities were observed for the discriminant function (cut-off ≤.064; 55.8%), recognition total (cut-off ≤9; 53.1%), the recognition combination score (≤10; 47.7%), and total learning across trials (cut-off ≤31; 45.3%). Individuals with histories of learning difficulties were over-represented in the 10% of credible patients exceeding cut-offs. When these individuals were removed, cut-offs could be tightened while still maintaining at least 90% specificity, and thereby increasing sensitivity (e.g., recognition total cut-off ≤10, 65% sensitivity; RAVLT/RO discriminant function cut-off ≤.176, 58% sensitivity). When three of the most sensitive, non-overlapping scores were considered in combination, 17% of credible patients failed ≥1 of the three cut-offs, while 3% failed two, and only 1% failed all three. In contrast, in the noncredible sample, more than two-thirds failed one or more of the three cut-offs, nearly half failed ≥2, and nearly a quarter failed all three. CONCLUSIONS: RAVLT PVT cut-offs and the RAVLT/RO discriminant function achieve approximately 50% sensitivity, and approach 65% sensitivity when cut-offs specific to samples without histories of learning problems are employed, confirming that RAVLT cut-offs and the RAVLT/RO discriminant function continue to be valuable techniques in the identification of performance invalidity.

Wait, There's a Baby in this Bath Water! Update on Quantitative and Qualitative Cut-Offs for Rey 15-Item Recall and Recognition.

OBJECTIVE: Evaluate the effectiveness of Rey 15-item plus recognition data in a large neuropsychological sample. METHOD: Rey 15-item plus recognition scores were compared in credible (n = 138) and noncredible (n = 353) neuropsychology referrals. RESULTS: Noncredible patients scored significantly worse than credible patients on all Rey 15-item plus recognition scores. When cut-offs were selected to maintain at least 89.9% specificity, cut-offs could be made more stringent, with the highest sensitivity found for recognition correct (cut-off ≤11; 62.6% sensitivity) and the combination score (recall + recognition - false positives; cut-off ≤22; 60.6% sensitivity), followed by recall correct (cut-off ≤11; 49.3% sensitivity), and recognition false positive errors (≥3; 17.9% sensitivity). A cut-off of ≥4 applied to a summed qualitative error score for the recall trial resulted in 19.4% sensitivity. Approximately 10% of credible subjects failed either recall correct or recognition correct, whereas two-thirds of noncredible patients (67.7%) showed this pattern. Thirteen percent of credible patients failed either recall correct, recognition correct, or the recall qualitative error score, whereas nearly 70% of noncredible patients failed at least one of the three. Some individual qualitative recognition errors had low false positive rates (<2%) indicating that their presence was virtually pathognomonic for noncredible performance. Older age (>50) and IQ < 80 were associated with increased false positive rates in credible patients. CONCLUSIONS: Data on a larger sample than that available in the 2002 validation study show that Rey 15-item plus recognition cut-offs can be made more stringent, and thereby detect up to 70% of noncredible test takers, but the test should be used cautiously in older individuals and in individuals with lowered IQ.

Cross-validation of the Dot Counting Test in a large sample of credible and non-credible patients referred for neuropsychological testing.

OBJECTIVE: To cross-validate the Dot Counting Test in a large neuropsychological sample. METHOD: Dot Counting Test scores were compared in credible (n = 142) and non-credible (n = 335) neuropsychology referrals. RESULTS: Non-credible patients scored significantly higher than credible patients on all Dot Counting Test scores. While the original E-score cut-off of ≥17 achieved excellent specificity (96.5%), it was associated with mediocre sensitivity (52.8%). However, the cut-off could be substantially lowered to ≥13.80, while still maintaining adequate specificity (≥90%), and raising sensitivity to 70.0%. Examination of non-credible subgroups revealed that Dot Counting Test sensitivity in feigned mild traumatic brain injury (mTBI) was 55.8%, whereas sensitivity was 90.6% in patients with non-credible cognitive dysfunction in the context of claimed psychosis, and 81.0% in patients with non-credible cognitive performance in depression or severe TBI. Thus, the Dot Counting Test may have a particular role in detection of non-credible cognitive symptoms in claimed psychiatric disorders. Alternative to use of the E-score, failure on ≥1 cut-offs applied to individual Dot Counting Test scores (≥6.0″ for mean grouped dot counting time, ≥10.0″ for mean ungrouped dot counting time, and ≥4 errors), occurred in 11.3% of the credible sample, while nearly two-thirds (63.6%) of the non-credible sample failed one of more of these cut-offs. CONCLUSIONS: An E-score cut-off of 13.80, or failure on ≥1 individual score cut-offs, resulted in few false positive identifications in credible patients, and achieved high sensitivity (64.0-70.0%), and therefore appear appropriate for use in identifying neurocognitive performance invalidity.

Examination of the Modified Somatic Perception Questionnaire (MSPQ) in a large sample of credible and noncredible patients referred for neuropsychological testing.

OBJECTIVE: The current study evaluated MSPQ sensitivity to noncredible PVT performance in the context of external incentive, and examined MSPQ false positive rates in noncompensation-seeking neuropsychology patients; and investigated effects of ethnicity/culture, gender, and somatoform diagnosis on MSPQ scores, and relationships with PVT and MMPI-2-RF data. METHOD: MSPQ scores were compared in credible (n = 110) and noncredible (n = 153) neuropsychology referrals. RESULTS: Noncredible patients scored higher than credible patients. When the credible group was divided into those with somatoform orientation (n = 39) versus those without (n = 71), the credible nonsomatoform group scored lower than the other two groups, who did not differ from each other. MSPQ elevations were found in ethnic minorities, and in individuals who learned English as a second language or concurrently with another language. MSPQ elevations were also associated with chronic systemic diseases, neurologic illness, and substance abuse. Women scored higher than men, but men and women were equally represented among those patients scoring beyond cut-offs. MSPQ scores were minimally related to PVT data but were more strongly correlated with MMPI-2-RF scales, particularly over-report validity scales, RC1, and Somatic/Cognitive scales, with more widespread relationships observed in noncredible patients. CONCLUSIONS: A cut-off of 18 resulted in few false positives in credible nonsomatoform patients, and appears appropriate for identifying physical symptom over-report (due to malingering or somatoform orientations), with associated sensitivity of 29%. However, clinicians are cautioned regarding using the MSPQ in patients with systemic, neurologic, and substance abuse conditions, and in ethnic minorities and non-monolingual English-speakers.

Apparent effect of type of compensation seeking (disability versus litigation) on performance validity test scores may be due to other factors.

Neuropsychologists use performance validity tests (PVTs; Larrabee, 2012 ) to ensure that results of testing are reflective of the test taker's true neurocognitive ability, and their use is recommended in all compensation-seeking settings. However, whether the type of compensation context (e.g., personal injury litigation versus disability seeking) impacts the nature and extent of neurocognitive symptom feigning has not been adequately investigated. PVT performance was compared in an archival data set of noncredible individuals in either a personal injury litigation (n = 163) or a disability-seeking context (n = 201). Individuals were deemed noncredible based on meeting Slick, Sherman, and Iverson's ( 1999 ) criteria including failure on at least two PVTs and a lack of congruency between their low cognitive scores and normal function in activities of daily living (ADLs). In general, disability seekers tended to perform in a less sophisticated manner than did litigants (i.e., they failed more indicators and did so more extensively). Upon further investigation, these differences were in part accounted for by type of diagnoses feigned; those seeking compensation for mental health diagnoses were more likely to feign or exaggerate a wide variety of cognitive deficits, whereas those with claimed medical diagnoses (i.e., traumatic brain injury) were more targeted in their attempts to feign and/or exaggerate neurocognitive compromise.

Effectiveness of the Comalli Stroop Test as a measure of negative response bias.

Practice guidelines recommend the use of multiple performance validity tests (PVTs) to detect noncredible performance during neuropsychological evaluations, and PVTs embedded in standard cognitive tests achieve this goal most efficiently. The present study examined the utility of the Comalli version of the Stroop Test as a measure of response bias in a large sample of "real world" noncredible patients (n = 129) as compared with credible neuropsychology clinic patients (n=233). The credible group performed significantly better than the noncredible group on all trials, but particularly on word-reading (Stroop A) and color-naming (Stroop B); cut-scores for Stroop A and Stroop B trials were associated with moderate sensitivity (49-53%) as compared to the low sensitivity found for the color interference trial (29%). Some types of diagnoses (including learning disability, severe traumatic brain injury, psychosis, and depression), very advanced age (⩾80), and lowered IQ were associated with increased rates of false positive identifications, suggesting the need for some adjustments to cut-offs in these subgroups. Despite some previous reports of an inverted Stroop effect (i.e., color-naming worse than color interference) in noncredible subjects, individual Stroop word reading and color naming trials were much more effective in identifying response bias.

Cross validation of the Lu and colleagues (2003) Rey-Osterrieth Complex Figure Test effort equation in a large known-group sample.

A Rey-Osterrieth Complex Figure Test (ROCFT) equation incorporating copy and recognition was found to be useful in detecting negative response bias in neuropsychological assessments (ROCFT Effort Equation; Lu, P. H., Boone, K. B., Cozolino, L., & Mitchell, C. (2003). Effectiveness of the Rey-Osterrieth Complex Figure Test and the Meyers and Meyers recognition trial in the detection of suspect effort. Clinical Neuropsychologist, 17, 426-440). In the current cross validation of this validity, the credible patient group (n = 146; 124 with equation data) outperformed the noncredible group (n = 157; 115 with equation data) on copy, 3-min recall, total recognition correct and the Effort Equation, but the latter was most effective in classifying subjects. A cut-off of ≤50 maintained specificity of 90% and achieved sensitivity of 80%. Results of the current cross validation provide corroboration that the ROCFT Effort Equation is an effective measure of neurocognitive response bias.

Examination of various WMS-III logical memory scores in the assessment of response bias.

The assessment of response validity during neuropsychological evaluation is an integral part of the testing process. Research has increasingly focused on the use of "embedded" effort measures (derived from standard neuropsychological tasks) because they do not require additional administration time and are less likely to be identified as effort indicators by test takers because of their primary focus as measures of cognitive function. The current study examined the clinical utility of various WMS-III Logical Memory scores in detecting response bias, as well as the Rarely Missed Index, an embedded effort indicator derived from the WMS-III Logical Memory Delayed Recognition subtest. The Rarely Missed Index cut-off only identified 24.1% of 63 non-credible participants (at >/=90% specificity in 125 credible patients), and cut-offs for other Logical Memory variables were in fact found to be more sensitive to non-credible performance. A new indicator, consisting of the weighted combination of the two most sensitive Logical Memory subtest scores (Logical Memory II raw score and Logical Memory Delayed Recognition raw score), was associated with 53% to 60% sensitivity, and thus may be an effective adjunct when utilized in conjunction with other validated effort indicators and collateral information in identifying non-credible performance.

Interpreting the meaning of multiple symptom validity test failure.

While it is recommended that judgments regarding the credibility of test performance be based on the results of more than one effort indicator, and recent efforts have been made to improve interpretation of multiple effort test failure, the field currently lacks adequate guidelines for using multiple measures of effort in concert with one another. A total of 103 patients were referred for outpatient neuropsychological evaluation, which included multiple measures of negative response bias embedded in standard test batteries. Using any pairwise failure combination to predict diagnostic classification was superior (sensitivity = 83.8%, specificity = 93.9%, overall hit rate = 90.3%) to using any one test by itself and to using any three-test failure combination. Further, the results were comparable to the results of logistical regression analyses using the embedded indicators as continuous predictors. Given its parsimony and clinical utility, the pairwise failure model is therefore a recommended criterion for identifying non-credible performance; however, there are of course other important contextual factors and influences to consider, which are also discussed.

Dementia and effort test performance.

Research on the performance of patients with dementia on tests of effort is particularly limited. We examined archival data from 214 non-litigating patients with dementia on 18 effort indices derived from 12 tests (WAIS-III/WAIS-R Digit Span and Vocabulary, Dot Counting Test, Warrington Recognition Memory Test-Words, WMS-III Logical Memory, Rey Word Recognition Memory Test, Finger Tapping, b-Test, Rey 15-Item, Test of Memory Malingering, Rey Auditory Verbal Learning Test, and Rey Complex Figure Test). Results indicated that recommended cut-offs for Digit Span indicators (Vocabulary Minus Digit Span and four-digit forward span time score) provided > or =90% specificity across participants, while the majority of other effort tests displayed specificities in the 30-70% range. Analyses of test specificity as a function of Mini Mental Status Examination (MMSE) score and specific dementia diagnosis are provided, as well as adjustments to cut-offs to maintain specificity where feasible.

Examination of the impact of ethnicity on the Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Fake Bad Scale.

The Minnesota Multiphasic Personality Inventory-2 (MMPI-2) Fake Bad Scale (FBS; Butcher, Dahlstrom, Graham, Tellegen, & Kaemmer, 1989) has been shown to be sensitive to somatic over-endorsement. However, the impact of ethnicity has not been examined on the FBS, which is of concern given some studies that show increased rates of somatic endorsement in particular ethnic groups. We evaluated the FBS performance of 190 Caucasian American, Hispanic, and African American outpatients who were obtained from two different clinical settings, excluding those who were applying for disability or in litigation. We failed to find significant ethnic differences in mean FBS performance or in cut-off specificity rates. We did find evidence of a gender effect, supporting continued use of gender-specific FBS cutoffs.

The relationship of IQ to effort test performance.

The relationship between IQ and nine effort indicators was examined in a sample of 189 neuropsychology clinic outpatients who were not in litigation or attempting to obtain disability. Participants with the lowest IQ (50-59) failed approximately 60% of the effort tests, while patients with an IQ of 60 to 69 failed 44% of effort indicators, and individuals with borderline IQ (70 to 79) exhibited a 17% failure rate. All patients with IQ < 70 failed at least one effort test. Cutoffs for the Warrington Recognition Memory Test (Words) and Finger Tapping maintained the highest specificities in low IQ samples.

The association between neuropsychological scores and ethnicity, language, and acculturation variables in a large patient population.

The relationship between ethnicity and cognitive test performance was examined in a sample of 161 patients referred for evaluation at a public hospital-affiliated neuropsychology clinic; 83 patients were Caucasian (non-Hispanic), 31 were African-American, 30 were Hispanic, and 17 were Asian. Significant group differences were present on some measures of language (Boston Naming Test), attention (Digit Span ACSS), constructional ability (Rey-Osterrieth [RO] copy), nonverbal processing speed (Trails A), and executive skills (Wisconsin Card Sorting Test [WCST]). Comparison of those who spoke English as a first language (or who learned English concurrently with a second language) versus those who spoke English as a second language (ESL) revealed significantly higher performance in the non-ESL group for Digit Span, Boston Naming Test, and FAS, and a higher score in the ESL group for RO copy. Boston Naming Test scores were significantly related to years educated in the United States; Boston Naming Test and Digit Span scores were significantly correlated with age at which conversational English was first learned and number of years in the United States; and finally, FAS scores were also significantly related to number of years in the United States. These findings are consistent with data from published literature on ethnic differences and the effects of acculturation on cognitive test performance in nonpatients, and also indicate that these observations are not attenuated by the presence of psychiatric or neurologic illness. The results further caution that normative data derived on Caucasian samples may not be appropriate for use with other ethnic groups.