The impact of race and other demographic factors on the false positive rates of five embedded Performance Validity Tests (PVTs) in a Veteran sample.

INTRODUCTION: It is common to use normative adjustments based on race to maintain accuracy when interpreting cognitive test results during neuropsychological assessment. However, embedded performance validity tests (PVTs) do not adjust for these racial differences and may result in elevated rates of false positives in African American/Black (AA) samples compared to European American/White (EA) samples. METHODS: Veterans without Major Neurocognitive Disorder completed an outpatient neuropsychological assessment and were deemed to be performing in a valid manner (e.g., passing both the Test of Memory Malingering Trial 1 (TOMM1) and the Medical Symptom Validity Test (MSVT), (n = 531, EA = 473, AA = 58). Five embedded PVTs were administered to all patients: WAIS-III/IV Processing Speed Index (PSI), Brief Visuospatial Memory Test-Revised: Discrimination Index (BVMT-R), TMT-A (secs), California Verbal Learning Test-II (CVLT-II) Forced Choice, and WAIS-III/IV Digit Span Scaled Score. Individual PVT false positive rates, as well as the rate of failing two or more embedded PVTs, were calculated. RESULTS: Failure rates of two embedded PVTs (PSI, TMT-A), and the total number of PVTs failed, were higher in the AA sample. The PSI and TMT-A remained significantly impacted by race after accounting for age, education, sex, and presence of Mild Neurocognitive Disorder. There were PVT failure rates greater than 10% (and considered false positives) in both groups (AA: PSI, TMT-A, and BVMT-R, 12-24%; EA: BVMT-R, 17%). Failing 2 or more PVTs (AA = 9%, EA = 4%) was impacted by education and Mild Neurocognitive Disorder but not by race. CONCLUSIONS: Individual (timed) PVTs showed higher false positive rates in the AA sample even after accounting for demographic factors and diagnosis of Mild Neurocognitive Disorder. Requiring failure on 2 or more embedded PVTs reduced false positive rates to acceptable levels across both groups (10% or less) and was not significantly influenced by race.

When Failing One Performance Validity Test Matters.

There appears to be a lack of consensus regarding how best to interpret cognitive test findings when there is a failure on only one Performance Validity Test (PVT). The current study examined the impact of failing one freestanding, forced-choice, memory-based (Fr-FC-MB) PVT across two memory measures in a large sample of veterans (N = 1,353). The impact of failing zero, one, or two Fr-FC-MB PVTs (Test of Memory Malingering Trial 1 or the Medical Symptom Validity Test) on subsequent memory measures was examined (California Verbal Learning Test-II [CVLT-II], Brief Visuospatial Memory Test-R [BVMT-R]). Compared to those failing zero PVTs, those failing one PVT showed significant declines across all memory indices with large average effect sizes (BVMT-R, d = -0.9, CVLT-II, d = -1.0). Those failing one PVT had memory scores more similar to those failing two PVTs. There is a need for greater nuance and flexibility when determining invalid test performance. The current findings, along with a brief review of the literature, find that failing even one Fr-FC-MB PVT dramatically (negatively) impacts memory performance. Results suggest that including individuals failing one Fr-FC-MB PVT into a credible group should be more closely scrutinized.

The TOMM1 discrepancy index (TDI): A new performance validity test (PVT) that differentiates between invalid cognitive testing and those diagnosed with dementia.

There is a need to develop performance validity tests (PVTs) that accurately identify those with severe cognitive decline but also remain sensitive to those suspected of invalid cognitive testing. The TOMM1 Discrepancy Index (TDI) attempts to address both of these issues. Veterans diagnosed with dementia (n = 251) were administered TOMM1 and the MSVT in order to develop the TDI (TOMM1 percent correct minus MSVT Free Recall percent correct). Cut offs based on the dementia sample were then used to identify those in the non-dementia sample (n = 1,226) suspected of invalid test performance (n = 401). Combining TOMM1 and the TDI in the dementia sample greatly reduced the false positive rate (specificity = 0.97) at a cut off of 28 points or less on the TDI. Those suspected of invalid testing were identified at much higher rates (sensitivity = 0.75) compared to the MSVT genuine memory impairment profile (GMIP, sensitivity = 0.49). By utilizing a neurologically plausible pattern of scores across two PVTs, the TDI correctly classified those with dementia and identified a large percentage with invalid test performance. PVTs utilizing a complex pattern of performance may help reduce one's ability to fabricate cognitive deficits.

Self-reported disability-seeking predicts PVT failure in veterans undergoing clinical neuropsychological evaluation.

Objective: This study examined disability-related factors as predictors of PVT performance in Veterans who underwent neuropsychological evaluation for clinical purposes, not for determination of disability benefits. Method: Participants were 1,438 Veterans who were seen for clinical evaluation in a VA Medical Center's Neuropsychology Clinic. All were administered the TOMM, MSVT, or both. Predictors of PVT performance included (1) whether Veterans were receiving VA disability benefits ("service connection") for psychiatric or neurological conditions at the time of evaluation, and (2) whether Veterans reported on clinical interview that they were in the process of applying for disability benefits. Data were analyzed using binary logistic regression, with PVT performance as the dependent variable in separate analyses for the TOMM and MSVT. Results: Veterans who were already receiving VA disability benefits for psychiatric or neurological conditions were significantly more likely to fail both the TOMM and the MSVT, compared to Veterans who were not receiving benefits for such conditions. Independently of receiving such benefits, Veterans who reported that they were applying for disability benefits were significantly more likely to fail the TOMM and MSVT than were Veterans who denied applying for benefits at the time of evaluation. Conclusions: These findings demonstrate that simply being in the process of applying for disability benefits increases the likelihood of noncredible performance. The presence of external incentives can predict the validity of neuropsychological performance even in clinical, non-forensic settings.

When 10 is enough: Errors on the first 10 items of the Test of Memory Malingering (TOMMe10) and administration time predict freestanding performance validity tests (PVTs) and underperformance on memory measures.

It is critical that we develop more efficient performance validity tests (PVTs). A shorter version of the Test of Memory Malingering (TOMM) that utilizes errors on the first 10 items (TOMMe10) has shown promise as a freestanding PVT. Retrospective review included 397 consecutive veterans administered TOMM trial 1 (TOMM1), the Medical Symptom Validity Test (MSVT), and the Brief Visuospatial Memory Test-Revised (BVMT-R). TOMMe10 accuracy and administration time were used to predict performance on freestanding PVTs (TOMM1, MSVT). The impact of failing TOMMe10 (2 or more errors) on independent memory measures was also explored. TOMMe10 was a robust predictor of TOMM1 (area under the curve [AUC] = 0.97) and MSVT (AUC = 0.88) with sensitivities = 0.76 to 0.89 and specificities = 0.89 to 0.96. Administration time predicted PVT performance but did not improve accuracy compared to TOMMe10 alone. Failing TOMMe10 was associated with clinically and statistically significant declines on the BVMT-R and MSVT Paired Associates and Free Recall memory tests (d = -0.32 to -1.31). Consistent with prior research, TOMMe10 at 2 or more errors was highly accurate in predicting performance on other well-validated freestanding PVTs. Failing just 1 freestanding PVT (TOMMe10) significantly impacted memory measures and likely reflects invalid test performance.

Effects of Distraction on Performance Validity: A Pilot Study with Veterans.

OBJECTIVE: The purpose of this experimental pilot study was to evaluate whether distraction can affect results of performance validity testing. METHOD: Thirty-three veterans who have served in the US military since 09/11/2001 (Mage = 38.60, SD = 10.85 years) completed the Test of Memory Malingering (TOMM), Trail Making Test, and Medical Symptom Validity Test (MSVT). Subjects were randomly assigned to complete the MSVT in one of three experimental conditions: standard administration, while performing serial 2 s (Cognitive Distraction), and while submerging a hand in ice water (Physical Distraction). RESULTS: All participants included in primary analyses passed the TOMM (n = 30). Physical distraction did not affect performance on the MSVT. Cognitive distraction negatively affected MSVT performance. CONCLUSIONS: Cognitive distraction can substantially affect MSVT performance in a subgroup of individuals. Physical distraction did not significantly affect MSVT performance.

Cost of malingering mild traumatic brain injury-related cognitive deficits during compensation and pension evaluations in the veterans benefits administration.

Given the high rates of exaggeration in those claiming long-term cognitive deficits as a result of mild traumatic brain injury (mTBI), the aim of this study was to evaluate the rates of malingering in those seeking disability through the Veterans Benefits Administration and estimate the financial burden of disability payments for those receiving compensation despite exaggerated mTBI-related cognitive deficits. Retrospective review included 74 veterans seen for Compensation and Pension evaluations for mTBI. Rates of malingering were based on failure of the Medical Symptom Validity Test (MSVT) and/or the Test of Memory Malingering (TOMM) trial 1 ≤ 40. Total estimated compensation was based on the level of disability awarded and the number of individuals found to be malingering cognitive deficits. Overall, 33-52% of the sample was found to be malingering mTBI-related cognitive deficits. The malingering groups were receiving approximately $71,000-$121,000/year ($6,390-$7,063 per year, per veteran on average). Estimated nationwide disability payments for those possibly malingering mTBI-related cognitive deficits would be $136-$235 million/year (projected costs from 2015-2020 = $700 million-$1.2 billion). It is critical that providers and administrative officials identify those exaggerating disability claims attributed to mTBI. The cost of malingering impacts society in general as well as veterans themselves, as it diverts needed funds/resources away from those legitimately impaired by their military service.

Symptom and performance validity with veterans assessed for attention-deficit/hyperactivity disorder (ADHD).

Little is known about attention-deficit/hyperactivity disorder (ADHD) in veterans. Practice standards recommend the use of both symptom and performance validity measures in any assessment, and there are salient external incentives associated with ADHD evaluation (stimulant medication access and academic accommodations). The purpose of this study was to evaluate symptom and performance validity measures in a clinical sample of veterans presenting for specialty ADHD evaluation. Patients without a history of a neurocognitive disorder and for whom data were available on all measures (n = 114) completed a clinical interview structured on DSM-5 ADHD symptoms, the Minnesota Multiphasic Personality Inventory-2-Restructured Form (MMPI-2-RF), and the Test of Memory Malingering Trial 1 (TOMM1) as part of a standardized ADHD diagnostic evaluation. Veterans meeting criteria for ADHD were not more likely to overreport symptoms on the MMPI-2-RF nor to fail TOMM1 (score ≤ 41) compared with those who did not meet criteria. Those who overreported symptoms did not endorse significantly more ADHD symptoms; however, those who failed TOMM1 did report significantly more ADHD symptoms (g = 0.90). In the total sample, 19.3% failed TOMM1, 44.7% overreported on the MMPI-2-RF, and 8.8% produced both an overreported MMPI-2-RF and invalid TOMM1. F-r had the highest correlation to TOMM1 scores (r = -.30). These results underscore the importance of assessing both symptom and performance validity in a clinical ADHD evaluation with veterans. In contrast to certain other conditions (e.g., mild traumatic brain injury), ADHD as a diagnosis is not related to higher rates of invalid report/performance in veterans. (PsycINFO Database Record

An Intervention to Decrease the Occurrence of Invalid Data on Neuropsychological Evaluation.

OBJECTIVE: This study tested whether patients who were given a handout based on deterrence theory, immediately prior to evaluation, would provide invalid data less frequently than patients who were simply given an informational handout. METHOD: All outpatients seen for clinical evaluation in a VA Neuropsychology Clinic were randomly given one of the two handouts immediately prior to evaluation. The "Intervention" handout emphasized the importance of trying one's hardest, explicitly listed consequences of valid and invalid responding and asked patients to sign and initial it. The "Control" handout provided general information about neuropsychological evaluation. Examiners were blinded to condition. Patients were excluded from analyses if they were diagnosed with major neurocognitive disorder or could not read the handout. Medical Symptom Validity Test (MSVT) was used to determine performance validity. RESULTS: Groups did not differ on age, education, or litigation status. For the entire sample (N = 251), there was no effect of handout on passing versus failing MSVT. However, among patients who were seeking disability benefits at the time of evaluation (n = 70), the Intervention handout was associated with lower frequency of failing MSVT than the Control handout. CONCLUSIONS: This brief, theory-based, cost-free intervention was associated with lower frequency of invalid data among patients seeking disability benefits at the time of clinical evaluation. We suggest methodological modifications that might produce a more potent intervention that could be effective with additional subsets of patients.

TOMM Trial 1 as a performance validity indicator in a criminal forensic sample.

OBJECTIVE: To determine the effectiveness of the Test of Memory Malingering Trial 1 (TOMM1) as a freestanding Performance Validity Test (PVT) as compared to the full TOMM in a criminal forensic sample. METHOD: Participants included 119 evaluees in a Midwestern forensic hospital. Criterion groups were formed based on passing/failing scores on other freestanding PVTs. This resulted in three groups: +MND (Malingered Neurocognitive Dysfunction), who failed two or more freestanding PVTs; possible MND (pMND), who failed one freestanding PVT; and -MND, who failed no other freestanding PVTs. All three groups were compared initially, but only +MND and -MND groups were retained for final analyses. TOMM1 performance was compared to standard TOMM performance using Receiver Operating Characteristic (ROC) analyses. RESULTS: TOMM1 was highly predictive of the standard TOMM decision rules (AUC = .92). Overall accuracy rate for TOMM1 predicting failure on 2 PVTs was quite robust as well (AUC = .80), and TOMM1 ≤ 39 provided acceptable diagnostic statistics (Sensitivity = .68, Specificity = .89). These results were essentially no different from the standard TOMM accuracy statistics. In addition, by adjusting for those strongly suspected of being inaccurately placed into the -MND group (e.g. false negatives), TOMM1 diagnostics slightly improved (AUC = .84) at a TOMM1 ≤ 40 (sensitivity = .71, specificity = .94). CONCLUSIONS: Results support use of TOMM1 in a criminal forensic setting where accuracy, shorter evaluation times, and more efficient use of resources are often critical in informing legal decision-making.

Embedded Performance Validity Measures with Postdeployment Veterans: Cross-Validation and Efficiency with Multiple Measures.

Embedded validity measures support comprehensive assessment of performance validity. The purpose of this study was to evaluate the accuracy of individual embedded measures and to reduce them to the most efficient combination. The sample included 212 postdeployment veterans (average age = 35 years, average education = 14 years). Thirty embedded measures were initially identified as predictors of Green's Word Memory Test (WMT) and were derived from the California Verbal Learning Test-Second Edition (CVLT-II), Conners' Continuous Performance Test-Second Edition (CPT-II), Trail Making Test, Stroop, Wisconsin Card Sorting Test-64, the Wechsler Adult Intelligence Scale-Third Edition Letter-Number Sequencing, Rey Complex Figure Test (RCFT), Brief Visuospatial Memory Test-Revised, and the Finger Tapping Test. Eight nonoverlapping measures with the highest area-under-the-curve (AUC) values were retained for entry into a logistic regression analysis. Embedded measure accuracy was also compared to cutoffs found in the existing literature. Twenty-one percent of the sample failed the WMT. Previously developed cutoffs for individual measures showed poor sensitivity (SN) in the current sample except for the CPT-II (Total Errors, SN = .41). The CVLT-II (Trials 1-5 Total) showed the best overall accuracy (AUC = .80). After redundant measures were statistically eliminated, the model included the RCFT (Recognition True Positives), CPT-II (Total Errors), and CVLT-II (Trials 1-5 Total) and increased overall accuracy compared with the CVLT-II alone (AUC = .87). The combination of just 3 measures from the CPT-II, CVLT-II, and RCFT was the most accurate/efficient in predicting WMT performance.

The efficiency and accuracy of the Test of Memory Malingering trial 1, errors on the first 10 items of the test of memory malingering, and five embedded measures in predicting invalid test performance.

The current study attempted to improve upon the efficiency and accuracy of one of the most frequently administered measures of test validity, the Test of Memory Malingering (TOMM) by utilizing two short forms (TOMM trial 1 or TOMM1; and errors on the first 10 items of TOMM1 or TOMMe10). In addition, we cross-validated the accuracy of five embedded measures frequently used in malingering research. TOMM1 and TOMMe10 were highly accurate in predicting test validity (area under the curve [AUC]=92% and 87%, respectively; TOMM1≤40 and TOMMe10≥1; sensitivities>70% and specificities>90%). A logistic regression of five embedded measures showed better accuracy compared with any individual embedded measure alone or in combination (AUC=87%). TOMM1 and TOMMe10 provide evidence of greater sensitivity to invalid test performance compared with the standard TOMM administration and the use of regression improved the accuracy of the five embedded cognitive measures.

Hormonal mechanisms underlying aberrant sexual differentiation in male rats prenatally exposed to alcohol, stress, or both.

The male offspring of rats exposed to restraint stress, alcohol, or both during late pregnancy show normally masculinized genitalia; however, sexual differentiation of behavior is dissociated from the external morphology. In contrast to controls, males exposed prenatally to stress, alcohol, or a combination of these factors exhibited the female lordotic pattern. Thus, all 3 prenatal treatments led to incomplete behavioral defeminization. Behavioral masculinization was not altered by fetal alcohol exposure alone, but a significant number of males that experienced prenatal stress alone failed to copulate. A more severe disruption of behavioral masculinization occurred when stress and alcohol were combined. Very few males exposed to the combination treatment mated with females. This study attempted to relate the effects of these treatments on sexual behavior to the postparturitional surge in plasma testosterone (T) that is known to influence the process of sexual differentiation. Prenatally stressed males, like control males showed a large, brief surge in plasma T that peaked 1 hr after delivery. Altered defeminization and masculinization were seen in prenatally stressed males, despite a normal postparturitional T surge. Fetal alcohol exposure, with or without concomitant stress, depressed T to the same extent right after birth and led to a similarly blunted T surge 1 hr later. Thus, equal disruption of the neonatal T pattern occurred in alcohol-alone males, who showed normal male copulatory behavior, and in alcohol-plus-stress males, whose behavior was severely attenuated. The results suggest that consideration of abnormal exposure to T during prenatal ontogeny may be required to understand the atypical sexual behaviors associated with these treatments.

Postparturitional testosterone surge in male offspring of rats stressed and/or fed ethanol during late pregnancy.

Male offspring of rats exposed to restraint stress and/or alcohol during late pregnancy show aberrant patterns of sexual behavior masculinization and defeminization that vary as a function of treatment. The impact of these treatments on the postparturitional testosterone (T) surge that contributes to sexual behavior differentiation was investigated. Plasma T was measured using radioimmunoassay in individual males sampled on day 21 of gestation within 10 min of cesarean delivery or 1, 2, or 4 h thereafter. Neonatal T in the group exposed only to stress did not differ from that in the control group. T was lower than control levels at birth in both alcohol groups. The magnitude of the T surge that occurred during the first hour of birth in the control group was diminished by 50% in both alcohol groups, whose T pattern was very similar. There was no common alteration in postparturitional T associated with the increased lordotic behavior potential that males in all three treatment groups typically share, nor were there idiosyncratic endocrine abnormalities linked to the very different male copulatory pattern each exhibits. Exposure to an abnormal T milieu during fetal as well as neonatal ontogeny may underlie the etiology of the different sexual behavior patterns exhibited by males exposed to stress and/or alcohol. Possible unique effects each treatment exerts on perinatal plasma T and it's aromatization to estradiol in hypothalamic targets are discussed.