Remote self-administration of digital cognitive tests using the Brief Assessment of Cognition: Feasibility, reliability, and sensitivity to subjective cognitive decline.

Cognitive impairment is a common and pervasive feature of etiologically diverse disorders of the central nervous system, and a target indication for a growing number of symptomatic and disease modifying drugs. Remotely acquired digital endpoints have been recognized for their potential in providing frequent, real-time monitoring of cognition, but their ultimate value will be determined by the reliability and sensitivity of measurement in the populations of interest. To this end, we describe initial validation of remote self-administration of cognitive tests within a regulatorily compliant tablet-based platform. Participants were 61 older adults (age 55+), including 20 individuals with subjective cognitive decline (SCD). To allow comparison between remote (in-home) and site-based testing, participants completed 2 testing sessions 1 week apart. Results for three of four cognitive domains assessed demonstrated equivalence between remote and site-based tests, with high cross-modality ICCs (absolute agreement) for Symbol Coding (ICC = 0.75), Visuospatial Working Memory (ICC = 0.70) and Verbal Fluency (ICC > 0.73). Group differences in these domains were significant and reflected sensitivity to objective cognitive impairment in the SCD group for both remote and site-based testing (p < 0.05). In contrast, performance on tests of verbal episodic memory suggested inflated performance during unmonitored testing and indicate reliable use of remote cognitive assessments may depend on the construct, as well as the population being tested.

Factor structure of cognitive performance and functional capacity in schizophrenia: Evidence for differences across functional capacity measures.

BACKGROUND: Cognition and functional capacity predict functional outcomes in mental illness. Traditional approaches conceptualize cognition as comprised of domains, but many studies support a unifactorial structure. Some functional capacity measures may share a single-factor structure with cognition. In this study, we examined the factor structure of two measures of functional capacity, a conventional assessment and a newer computerized assessment, testing for a shared factor structure with cognition. METHODS: Patients with schizophrenia and healthy controls were examined with the MATRICS Consensus Cognitive Battery (MCCB), the UCSD Performance Based Skills Assessment (UPSA), and the Virtual Reality Functional Capacity Assessment Tool (VRFCAT). Models of the factor structures of the MCCB, UPSA, and VRFCAT were calculated, as were correlations between MCCB scores and individual VRFCAT objectives. RESULTS: The MCCB, VRFCAT, and UPSA all had unifactorial structures. The best fitting model of the correlations between MCCB and UPSA was a shared single factor, while the best fit for the relationship between MCCB and VRFCAT had two factors. Correlations between the MCCB domain and composite scores and the VRFCAT objectives suggested global rather than specific patterns of correlation. DISCUSSION: The relationship between cognitive performance and functional capacity was found to vary across functional capacity assessments. The UPSA and MCCB were not differentiated into separate factors, suggesting that the UPSA may overlap with neurocognitive performance. However, the VRFCAT appears to measure functional abilities that are separable from, yet correlated with, neurocognitive performance. It may provide a more distinctive assessment of the functional capacity construct.

Serial position-dependent false memory effects.

Evidence for false recognition within seconds of encoding suggests that semantic-associative influences are not restricted to long-term memory, consistent with unitary memory accounts but contrary to dual store models. The present study sought further relevant evidence using a modified free recall converging associates task where participants studied 12-item lists composed of 3 semantically distinct quartets (sublists) related to a separate, non-presented theme word (i.e., words 1-4/theme1, 5-8/theme2, and 9-12/theme3). This list construction permits assessment of false recall errors from each sublist, and, particularly, the primacy and recency sublists that have been linked to long- and short-term memory stores. Experiment 1 tested immediate free recall for items. Associative false memories were evident from all sublists, however, significantly less so from the recent sublist, which also showed the highest levels of veridical memory. By inserting a brief (3 s) distractor prior to recall, Experiment 2 selectively reduced veridical memory and increased false memory for the recent sublist while leaving the primacy sublist unaffected. These recall results converge with prior evidence indicating the immediacy of false recognition, and can be understood within a unitary framework where the differential availability of verbatim features and gist-based cues affect memory for primacy and recency sublists.

TOMMORROW neuropsychological battery: German language validation and normative study.

INTRODUCTION: Assessment of preclinical Alzheimer's disease (AD) requires reliable and validated methods to detect subtle cognitive changes. The battery of standardized cognitive assessments that is used for diagnostic criteria for mild cognitive impairment due to AD in the TOMMORROW study have only been fully validated in English-speaking countries. We conducted a validation and normative study of the German language version of the TOMMORROW neuropsychological test battery, which tests episodic memory, language, visuospatial ability, executive function, and attention. METHODS: German-speaking cognitively healthy controls (NCs) and subjects with AD were recruited from a memory clinic at a Swiss medical center. Construct validity, test-retest, and alternate form reliability were assessed in NCs. Criterion and discriminant validities of the cognitive measures were tested using logistic regression and discriminant analysis. Cross-cultural equivalency of performance of the German language tests was compared with English language tests. RESULTS: A total of 198 NCs and 25 subjects with AD (aged 65-88 years) were analyzed. All German language tests discriminated NCs from persons with AD. Episodic memory tests had the highest potential to discriminate with almost twice the predictive power of any other domain. Test-retest reliability of the test battery was adequate, and alternate form reliability for episodic memory tests was supported. For most tests, age was a significant predictor of group effect sizes; therefore, normative data were stratified by age. Validity and reliability results were similar to those in the published US cognitive testing literature. DISCUSSION: This study establishes the reliability and validity of the German language TOMMORROW test battery, which performed similarly to the English language tests. Some variations in test performance underscore the importance of regional normative values. The German language battery and normative data will improve the precision of measuring cognition and diagnosing incident mild cognitive impairment due to AD in clinical settings in German-speaking countries.

Placebo Response and Practice Effects in Schizophrenia Cognition Trials.

Importance: Patients' previous experience with performance-based cognitive tests in clinical trials for cognitive impairment associated with schizophrenia can create practice-related improvements. Placebo-controlled trials for cognitive impairment associated with schizophrenia are at risk for these practice effects, which can be difficult to distinguish from placebo effects. Objectives: To conduct a systematic evaluation of the magnitude of practice effects on the Measurement and Treatment Research to Improve Cognition in Schizophrenia Consensus Cognitive Battery (MCCB) in cognitive impairment associated with schizophrenia and to examine which demographic, clinical, and cognitive characteristics were associated with improvement in placebo conditions. Design, Setting, and Participants: A blinded review was conducted of data from 813 patients with schizophrenia who were treated with placebo in 12 randomized placebo-controlled clinical trials conducted mostly in outpatient clinics in North America, Europe, Asia, and Latin America from February 22, 2007, to March 1, 2014. A total of 779 patients provided data for the primary outcome measure at baseline and at least 1 follow-up. Seven trials had prebaseline assessments wherein the patients knew that they were not receiving treatment, allowing a comparison of practice and placebo effects in the same patients. Interventions: Placebo compared with various experimental drug treatments. Main Outcomes and Measures: Composite score on the MCCB. Results: Of the 813 patients in the study (260 women and 553 men; mean [SD] age, 41.2 [11.5] years), the mean MCCB composite score at baseline was 22.8 points below the normative mean, and the mean (SEM) total change in the MCCB during receipt of placebo was 1.8 (0.2) T-score points (95% CI, 1.40-2.18), equivalent to a change of 0.18 SD. Practice effects in the 7 studies in which there was a prebaseline assessment were essentially identical to the postbaseline placebo changes. Baseline factors associated with greater improvements in the MCCB during receipt of placebo included more depression/anxiety (F1,438 = 5.41; P = .02), more motivation (F1,272 = 4.63; P = .03), and less improvement from screening to baseline (F1,421 = 59.32; P < .001). Conclusions and Relevance: Placebo effects were minimal and associated with the number of postbaseline assessments and several patient characteristics. Given that the patients performed 2.28 SDs below normative standards on average at baseline, a mean placebo-associated improvement of less than 0.2 SD provides evidence that ceiling effects do not occur in these trials. These minimal changes in the MCCB could not be responsible for effective active treatments failing to separate from placebo.

Psychometric characteristics of the MATRICS Consensus Cognitive Battery in a large pooled cohort of stable schizophrenia patients.

The MATRICS Consensus Cognitive Battery (MCCB) was developed to assess cognitive treatment effects in schizophrenia clinical trials, and is considered the FDA gold standard outcome measure for that purpose. The aim of the present study was to establish pre-treatment psychometric characteristics of the MCCB in a large pooled sample. The dataset included 2616 stable schizophrenia patients enrolled in 15 different clinical trials between 2007 and 2016 within the United States (94%) and Canada (6%). The MCCB was administered twice prior to the initiation of treatment in 1908 patients. Test-retest reliability and practice effects of the cognitive composite score, the neurocognitive composite score, which excludes the domain Social Cognition, and the subtests/domains were examined using Intra-Class Correlations (ICC) and Cohen's d. Simulated regression models explored which domains explained the greatest portion of variance in composite scores. Test-retest reliability was high (ICC=0.88) for both composite scores. Practice effects were small for the cognitive (d=0.15) and neurocognitive (d=0.17) composites. Simulated bootstrap regression analyses revealed that 3 of the 7 domains explained 86% of the variance for both composite scores. The domains that entered most frequently in the top 3 positions of the regression models were Speed of Processing, Working Memory, and Visual Learning. Findings provide definitive psychometric characteristics and a benchmark comparison for clinical trials using the MCCB. The test-retest reliability of the MCCB composite scores is considered excellent and the learning effects are small, fulfilling two of the key criteria for outcome measures in cognition clinical trials.

Validation of the tablet-administered Brief Assessment of Cognition (BAC App).

Computerized tests benefit from automated scoring procedures and standardized administration instructions. These methods can reduce the potential for rater error. However, especially in patients with severe mental illnesses, the equivalency of traditional and tablet-based tests cannot be assumed. The Brief Assessment of Cognition in Schizophrenia (BACS) is a pen-and-paper cognitive assessment tool that has been used in hundreds of research studies and clinical trials, and has normative data available for generating age- and gender-corrected standardized scores. A tablet-based version of the BACS called the BAC App has been developed. This study compared performance on the BACS and the BAC App in patients with schizophrenia and healthy controls. Test equivalency was assessed, and the applicability of paper-based normative data was evaluated. Results demonstrated the distributions of standardized composite scores for the tablet-based BAC App and the pen-and-paper BACS were indistinguishable, and the between-methods mean differences were not statistically significant. The discrimination between patients and controls was similarly robust. The between-methods correlations for individual measures in patients were r>0.70 for most subtests. When data from the Token Motor Test was omitted, the between-methods correlation of composite scores was r=0.88 (df=48; p<0.001) in healthy controls and r=0.89 (df=46; p<0.001) in patients, consistent with the test-retest reliability of each measure. Taken together, results indicate that the tablet-based BAC App generates results consistent with the traditional pen-and-paper BACS, and support the notion that the BAC App is appropriate for use in clinical trials and clinical practice.

Validation of a Computerized test of Functional Capacity.

Regulatory guidance for schizophrenia cognition clinical trials requires that the assessment of cognitive change is accompanied by a functionally meaningful endpoint. However, currently available measures are challenged by resistance to change, psychometric weaknesses, and for interview-based assessments, dependence upon the presence of an informant. The aims of the current study were to: 1) assess the validity, sensitivity, and reliability of the Virtual Reality Functional Capacity Assessment Tool (VRFCAT) as a measure of functional capacity; 2) determine the association between performance on the VRFCAT and performance on the MATRICS Consensus Cognitive Battery (MCCB); and 3) compare the metrics of the VRFCAT with the UCSD Performance-based Skills Assessment (UPSA). 167 patients with schizophrenia and 166 healthy controls completed the VRFCAT, UPSA, and the MCCB at baseline. The VRFCAT and UPSA were completed again at follow-up. The VRFCAT, MCCB, and UPSA were very sensitive to impairment in schizophrenia (d=1.16 to 1.22). High test-retest reliability was demonstrated for VRFCAT total completion time and the UPSA total score in patients (ICC=0.81 and 0.78, respectively). The UPSA demonstrated significant practice effects in patients (d=0.35), while the VRFCAT did not (d=-0.04). VRFCAT total completion time was correlated with both UPSA (r=-0.56, p<0.0001 for patients and -0.58, p<0.0001 for controls) and MCCB Composite (r=-0.57, p<0.0001 for patients and -0.68, p<0.0001 for controls). The VRFCAT is a highly reliable and sensitive measure of functional capacity with associations to the UPSA and MCCB. These results provide encouraging support for a computerized functional capacity assessment for use in schizophrenia.

Virtual Reality Functional Capacity Assessment In Schizophrenia: Preliminary Data Regarding Feasibility and Correlations with Cognitive and Functional Capacity Performance.

INTRODUCTION: Assessment of functional capacity is an intrinsic part of determining the functional relevance of response to treatment of cognitive impairment in schizophrenia. Existing methods are highly and consistently correlated with performance on neuropsychological tests, but most current assessments of functional capacity are still paper and pencil simulations. We developed a computerized virtual reality assessment that contains all of the components of a shopping trip. METHODS: We administered the Virtual Reality Functional Capacity Assessment Tool (VRFCAT) to 54 healthy controls and to 51 people with schizophrenia to test its feasibility. Dependent variables for the VRFCAT included time to completion and errors on 12 objectives and the number of times that an individual failed to complete an objective. The MATRICS Consensus Cognitive Battery (MCCB) and a standard functional capacity measure, the UCSD Performance-Based Skills Assessment-Brief (UPSA-B) were administered to the patients with schizophrenia. RESULTS: Patients with schizophrenia performed more poorly than healthy controls on 10/11 of the time variables, as well as 2/12 error scores and 2/12 failed objectives. Pearson correlations for 7 of 15 VRFCAT variables with MCCB composite scores were statistically significant. CONCLUSION: These results provide support for the possibility of computerized functional capacity assessment, but more substantial studies are required.

Development of a virtual reality assessment of everyday living skills.

Cognitive impairments affect the majority of patients with schizophrenia and these impairments predict poor long term psychosocial outcomes.  Treatment studies aimed at cognitive impairment in patients with schizophrenia not only require demonstration of improvements on cognitive tests, but also evidence that any cognitive changes lead to clinically meaningful improvements.  Measures of "functional capacity" index the extent to which individuals have the potential to perform skills required for real world functioning.  Current data do not support the recommendation of any single instrument for measurement of functional capacity.  The Virtual Reality Functional Capacity Assessment Tool (VRFCAT) is a novel, interactive gaming based measure of functional capacity that uses a realistic simulated environment to recreate routine activities of daily living. Studies are currently underway to evaluate and establish the VRFCAT's sensitivity, reliability, validity, and practicality. This new measure of functional capacity is practical, relevant, easy to use, and has several features that improve validity and sensitivity of measurement of function in clinical trials of patients with CNS disorders.

Memory distortion in Alzheimer's disease: deficient monitoring of short- and long-term memory.

OBJECTIVE: This study measured distortions of memory during short-term memory (STM) and long-term memory (LTM) versions of a semantically associated word list learning paradigm. Performance of patients with mild-to-moderate Alzheimer's disease (AD; MMSE ≥16) was compared with performance of age-matched, healthy older adult participants. METHOD: In a STM version of the Deese-Roediger-McDermott (DRM) task, participants viewed four-word lists and were prompted for recall after a brief interval. The LTM task tested recall memory for 12-word lists. RESULTS: Compared with the healthy group, the AD participants show greater impairment on the LTM task than on the STM task, although veridical recall is significantly reduced on both tasks. Furthermore, on both memory tasks, (1) participants with AD generate more nonsemantic intrusions than healthy older adult participants, and (2) semantic intrusion rate, when computed as a proportion of total recall, does not differ between groups. Notably, nonsemantic intrusions are consistently high for AD participants across both STM and LTM despite a marked difference in recall accuracy (65% and 23%, respectively). CONCLUSIONS: STM impairment with some preserved semantic processing is evident in AD. The extent and variety of intrusions reported by AD participants indicates a breakdown in their ability to monitor and constrain their recall responses, even within seconds of initial learning.

When less is heard than meets the ear: change deafness in a telephone conversation.

During a conversation, we hear the sound of the talker as well as the intended message. Traditional models of speech perception posit that acoustic details of a talker's voice are not encoded with the message whereas more recent models propose that talker identity is automatically encoded. When shadowing speech, listeners often fail to detect a change in talker identity. The present study was designed to investigate whether talker changes would be detected when listeners are actively engaged in a normal conversation, and visual information about the speaker is absent. Participants were called on the phone, and during the conversation the experimenter was surreptitiously replaced by another talker. Participants rarely noticed the change. However, when explicitly monitoring for a change, detection increased. Voice memory tests suggested that participants remembered only coarse information about both voices, rather than fine details. This suggests that although listeners are capable of change detection, voice information is not continuously monitored at a fine-grain level of acoustic representation during natural conversation and is not automatically encoded. Conversational expectations may shape the way we direct attention to voice characteristics and perceive differences in voice.

Resolving semantic and proactive interference in memory over the short-term.

Interference is a major source of short-term errors of memory. The present investigation explores the relationship between two important forms of interference: proactive interference (PI), induced by the need to reject recently studied items no longer relevant to task performance, and semantic interference (SI), induced by the need to reject lures sharing a meaningful relationship with current memoranda. We explore the possibility that shared cognitive control processes are recruited to resolve both forms of interference. In Experiment 1, we find that the requirement to engage in articulatory suppression during the retention interval of tasks that induce either PI or SI increases both forms of interference similarly and selectively. In Experiment 2, we develop a task to examine PI and SI within the same experimental context. The results show interactive effects between factors that lead to the two forms of interference. Taken together, these findings support contextual-cuing models of short-term remembering (Nairne, Annual Review of Psychology, 53, 53-81 2002), where the context in which retrieval occurs can influence susceptibility to interference. Lastly, we discuss several theoretical hypotheses concerning the cognitive control processes that are recruited to resolve SI and PI in short-term remembering.

Neural mechanisms of semantic interference and false recognition in short-term memory.

Decades of research using the Deese-Roediger-McDermott (DRM) paradigm have demonstrated that episodic memory is vulnerable to semantic distortion, and neuroimaging investigations of this phenomenon have shown dissociations between the neural mechanisms subserving true and false retrieval from long-term memory. Recently, false short-term memories have also been demonstrated, with false recognition of items related in meaning to memoranda encoded less than 5s earlier. Semantic interference is also evident in short-term memory, such that correct rejection of related lures is slowed relative to correct rejection of unrelated lures. The present research constitutes the first fMRI investigation of false recognition and semantic interference in short-term memory using a short-term DRM paradigm in which participants retained 4 semantic associates over a short 4-s filled retention interval. Results showed increased activation in the left mid-ventrolateral prefrontal cortex (BA45) associated with semantic interference, and significant correlations between these increases and behavioral measures of interference across subjects. Furthermore, increases in dorsolateral PFC occurred when related lures were correctly rejected versus falsely remembered. Compared with false recognition, true recognition was associated with increases in left fusiform gyrus, a finding consistent with the notion that increased perceptual processing may distinguish true from false recognition over both short and long retention intervals. Findings are discussed in relation to current models of interference resolution in short-term memory, and suggest that false short-term recognition occurs as a consequence of the failure of frontally mediated cognitive control processes which adjudicate semantic familiarity in support of accurate mnemonic retrieval.

False memories seconds later: the rapid and compelling onset of illusory recognition.

Distortions of long-term memory (LTM) in the converging associates task are thought to arise from semantic associative processes and monitoring failures due to degraded verbatim and/or contextual memory. Sensory-based coding is traditionally considered more prevalent than meaning-based coding in short-term memory (STM), whereas the converse is true of LTM, leading to the expectation that false memory phenomena should be less robust in a canonical STM task. These expectations were violated in 2 experiments in which participants were shown lists of 4 semantically related words and were probed immediately following a filled 3- to 4-s retention interval or approximately 20 min later in a surprise recognition test. Corrected false recognition rates, confidence ratings, and Remember/Know judgments reveal similar false memory effects across STM and LTM conditions. These results indicate that compelling false memory illusions can be rapidly instantiated and that, consistent with unitary models of memory, they originate from processes that are not specific to LTM tasks.

False working memories? Semantic distortion in a mere 4 seconds.

False memories are well-established, episodic memory phenomena: Semantically related associates are confidently and erroneously remembered as studied items. We report four experiments yielding similar effects in a working memory paradigm. Four semantically related words were retained over a brief interval. Whether or not the interval was filled with a math verification task, semantically related lures were mistakenly recognized as members of the memory set and took longer to reject than did unrelated negative probes. In a short-term recall task, semantic intrusions exceeded other errors (e.g., phonemic). Our results demonstrate false memory effects for a subspan list when a mere 4 sec was given between study and test. Such rapid semantic errors presumably result from associative processing, may be related to familiarity-based proactive interference in working memory, and are consistent with recent models that integrate short- and long-term memory processes.