The influence of imagery vividness and internally-directed attention on the neural mechanisms underlying the encoding of visual mental images into episodic memory.

Attention can be directed externally toward sensory information or internally toward self-generated information. Using electroencephalography (EEG), we investigated the attentional processes underlying the formation and encoding of self-generated mental images into episodic memory. Participants viewed flickering words referring to common objects and were tasked with forming visual mental images of the objects and rating their vividness. Subsequent memory for the presented object words was assessed using an old-new recognition task. Internally-directed attention during image generation was indexed as a reduction in steady-state visual evoked potentials (SSVEPs), oscillatory EEG responses at the frequency of a flickering stimulus. The results yielded 3 main findings. First, SSVEP power driven by the flickering word stimuli decreased as subjects directed attention internally to form the corresponding mental image. Second, SSVEP power returned to pre-imagery baseline more slowly for low- than high-vividness later remembered items, suggesting that longer internally-directed attention is required to generate subsequently remembered low-vividness images. Finally, the event-related-potential difference due to memory was more sustained for subsequently remembered low- versus high-vividness items, suggesting that additional conceptual processing may have been needed to remember the low-vividness visual images. Taken together, the results clarify the neural mechanisms supporting the encoding of self-generated information.

Age-related dedifferentiation and hyperdifferentiation of perceptual and mnemonic representations.

Preliminary evidence indicates that occipito-temporal activation patterns for different visual stimuli are less distinct in older (OAs) than younger (YAs) adults, suggesting a dedifferentiation of visual representations with aging. Yet, it is unclear if this deficit (1) affects only sensory or also categorical aspects of representations during visual perception (perceptual representations), and (2) affects only perceptual or also mnemonic representations. To investigate these issues, we fMRI-scanned YAs and OAs viewing and then remembering visual scenes. First, using representational similarity analyses, we distinguished sensory vs. categorical features of perceptual representations. We found that, compared to YAs, sensory features in early visual cortex were less differentiated in OAs (i.e., age-related dedifferentiation), replicating previous research, whereas categorical features in anterior temporal lobe (ATL) were more differentiated in OAs. This is, to our knowledge, the first report of an age-related hyperdifferentiation. Second, we assessed the quality of mnemonic representations by measuring encoding-retrieval similarity (ERS) in activation patterns. We found that aging impaired mnemonic representations in early visual cortex and hippocampus but enhanced mnemonic representations in ATL. Thus, both perceptual and mnemonic representations in ATL were enhanced by aging. In sum, our findings suggest that aging impairs visual and mnemonic representations in posterior brain regions but enhances them in anterior regions.

The visual and semantic features that predict object memory: Concept property norms for 1,000 object images.

Humans have a remarkable fidelity for visual long-term memory, and yet the composition of these memories is a longstanding debate in cognitive psychology. While much of the work on long-term memory has focused on processes associated with successful encoding and retrieval, more recent work on visual object recognition has developed a focus on the memorability of specific visual stimuli. Such work is engendering a view of object representation as a hierarchical movement from low-level visual representations to higher level categorical organization of conceptual representations. However, studies on object recognition often fail to account for how these high- and low-level features interact to promote distinct forms of memory. Here, we use both visual and semantic factors to investigate their relative contributions to two different forms of memory of everyday objects. We first collected normative visual and semantic feature information on 1,000 object images. We then conducted a memory study where we presented these same images during encoding (picture target) on Day 1, and then either a Lexical (lexical cue) or Visual (picture cue) memory test on Day 2. Our findings indicate that: (1) higher level visual factors (via DNNs) and semantic factors (via feature-based statistics) make independent contributions to object memory, (2) semantic information contributes to both true and false memory performance, and (3) factors that predict object memory depend on the type of memory being tested. These findings help to provide a more complete picture of what factors influence object memorability. These data are available online upon publication as a public resource.

Visual and Semantic Representations Predict Subsequent Memory in Perceptual and Conceptual Memory Tests.

It is generally assumed that the encoding of a single event generates multiple memory representations, which contribute differently to subsequent episodic memory. We used functional magnetic resonance imaging (fMRI) and representational similarity analysis to examine how visual and semantic representations predicted subsequent memory for single item encoding (e.g., seeing an orange). Three levels of visual representations corresponding to early, middle, and late visual processing stages were based on a deep neural network. Three levels of semantic representations were based on normative observed ("is round"), taxonomic ("is a fruit"), and encyclopedic features ("is sweet"). We identified brain regions where each representation type predicted later perceptual memory, conceptual memory, or both (general memory). Participants encoded objects during fMRI, and then completed both a word-based conceptual and picture-based perceptual memory test. Visual representations predicted subsequent perceptual memory in visual cortices, but also facilitated conceptual and general memory in more anterior regions. Semantic representations, in turn, predicted perceptual memory in visual cortex, conceptual memory in the perirhinal and inferior prefrontal cortex, and general memory in the angular gyrus. These results suggest that the contribution of visual and semantic representations to subsequent memory effects depends on a complex interaction between representation, test type, and storage location.

Age-Related Compensatory Reconfiguration of PFC Connections during Episodic Memory Retrieval.

During demanding cognitive tasks, older adults (OAs) frequently show greater prefrontal cortex (PFC) activity than younger adults (YAs). This age-related increase in PFC activity is often associated with enhanced cognitive performance, suggesting functional compensation. However, the brain is a complex network of interconnected regions, and it is unclear how network connectivity of PFC regions differs for OAs versus YAs. To investigate this, we examined the age-related difference on the functional brain networks mediating episodic memory retrieval. YAs and OAs participants encoded and recalled visual scenes, and age-related differences in network topology during memory retrieval were investigated as a function of memory performance. We measured both changes in functional integration and reconfiguration in connectivity patterns. The study yielded three main findings. First, PFC regions were more functionally integrated with the rest of the brain network in OAs. Critically, this age-related increase in PFC integration was associated with better retrieval performance. Second, PFC regions showed stronger performance-related reconfiguration of connectivity patterns in OAs. Finally, the PFC reconfiguration increases in OAs tracked reconfiguration reductions in the medial temporal lobe (MTL)-a core episodic memory region, suggesting that PFC connectivity in OAs may be compensating for MTL deficits.

Cortical Overlap and Cortical-Hippocampal Interactions Predict Subsequent True and False Memory.

The declarative memory system allows us to accurately recognize a countless number of items and events, particularly those strengthened by repeated exposure. However, increased familiarity due to repetition can also lead to false recognition of related but new items, particularly when mechanisms supporting fine-grain mnemonic discrimination fail. The hippocampus is thought to be particularly important in separating overlapping cortical inputs during encoding so that similar experiences can be differentiated. In the current study of male and female human subjects, we examine how neural pattern similarity between repeated exemplars of a given concept (e.g., apple) influences true and false memory for target or lure images. Consistent with past work, we found that subsequent true recognition was related to pattern similarity between concept exemplars and the entire encoding set (global encoding similarity), particularly in ventral visual stream. In addition, memory for an individual target exemplar (a specific apple) could be predicted solely by the degree of pattern overlap between the other exemplars (different apple pictures) of that concept (concept-specific encoding similarity). Critically, subsequent false memory for lures was mitigated when high concept-specific similarity in cortical areas was accompanied by differentiated hippocampal representations of the corresponding exemplars. Furthermore, both true and false memory entailed the reinstatement of concept-related information at varying levels of specificity. These results link both true and false memory to a measure of concept strength expressed in the overlap of cortical representations, and importantly, illustrate how the hippocampus serves to separate concurrent cortical overlap in the service of detailed memory.SIGNIFICANCE STATEMENT In some instances, the same processes that help promote memory for a general idea or concept can also hinder more detailed memory judgments, which may involve differentiating between closely related items. The current study shows that increased overlap in cortical representations for conceptually-related pictures is associated with increased recognition of repeated concept pictures. Whether similar lure items were falsely remembered as old further depended on the hippocampus, where the presence of more distinct representations protected against later false memory. This work suggests that the differentiability of brain patterns during perception is related to the differentiability of items in memory, but that fine-grain discrimination depends on the interaction between cortex and hippocampus.

A key role for stimulus-specific updating of the sensory cortices in the learning of stimulus-reward associations.

Successful adaptive behavior requires the learning of associations between stimulus-specific choices and rewarding outcomes. Most research on the mechanisms underlying such processes has focused on subcortical reward-processing regions, in conjunction with frontal circuits. Given the extensive stimulus-specific coding in the sensory cortices, we hypothesized they would play a key role in the learning of stimulus-specific reward associations. We recorded electrical brain activity (using electroencephalogram) during a learning-based decision-making gambling task where, on each trial, participants chose between a face and a house and then received feedback (gain or loss). Within each 20-trial set, either faces or houses were more likely to predict a gain. Results showed that early feedback processing (~200-1200 ms) was independent of the choice made. In contrast, later feedback processing (~1400-1800 ms) was stimulus-specific, reflected by decreased alpha power (reflecting increased cortical activity) over face-selective regions, for winning-vs-losing after a face choice but not after a house choice. Finally, as the reward association was learned in a set, there was an increasingly stronger attentional bias towards the more likely winning stimulus, reflected by increasing attentional orienting-related brain activity and increasing likelihood of choosing that stimulus. These results delineate the processes underlying the updating of stimulus-reward associations during feedback-guided learning, which then guide future attentional allocation and decision-making.

Functional networks underlying item and source memory: shared and distinct network components and age-related differences.

Although the medial temporal lobes (MTLs) are critical for both item memory (IM) and source memory (SM), the lateral prefrontal cortex and posterior parietal cortex play a greater role during SM than IM. It is unclear, however, how these differences translate into shared and distinct IM versus SM network components and how these network components vary with age. Within a sample of younger adults (YAs; n = 15, Mage = 19.5 years) and older adults (OAs; n = 40, Mage = 68.6 years), we investigated the functional networks underlying IM and SM. Before functional MRI scanning, participants encoded nouns while making either pleasantness or size judgments. During functional MRI scanning, participants completed IM and SM retrieval tasks. We found that MTL nodes were similarly interconnected among each other during both IM and SM (shared network components) but maintained more intermodule connections during SM (distinct network components). Also, during SM, OAs (compared to YAs) had MTL nodes with more widespread connections. These findings provide a novel viewpoint on neural mechanism differences underlying IM versus SM in YAs and OAs.

Functional modular architecture underlying attentional control in aging.

Previous research suggests that age-related differences in attention reflect the interaction of top-down and bottom-up processes, but the cognitive and neural mechanisms underlying this interaction remain an active area of research. Here, within a sample of community-dwelling adults 19-78 years of age, we used diffusion reaction time (RT) modeling and multivariate functional connectivity to investigate the behavioral components and whole-brain functional networks, respectively, underlying bottom-up and top-down attentional processes during conjunction visual search. During functional MRI scanning, participants completed a conjunction visual search task in which each display contained one item that was larger than the other items (i.e., a size singleton) but was not informative regarding target identity. This design allowed us to examine in the RT components and functional network measures the influence of (a) additional bottom-up guidance when the target served as the size singleton, relative to when the distractor served as the size singleton (i.e., size singleton effect) and (b) top-down processes during target detection (i.e., target detection effect; target present vs. absent trials). We found that the size singleton effect (i.e., increased bottom-up guidance) was associated with RT components related to decision and nondecision processes, but these effects did not vary with age. Also, a modularity analysis revealed that frontoparietal module connectivity was important for both the size singleton and target detection effects, but this module became central to the networks through different mechanisms for each effect. Lastly, participants 42 years of age and older, in service of the target detection effect, relied more on between-frontoparietal module connections. Our results further elucidate mechanisms through which frontoparietal regions support attentional control and how these mechanisms vary in relation to adult age.

From hippocampus to whole-brain: The role of integrative processing in episodic memory retrieval.

Multivariate functional connectivity analyses of neuroimaging data have revealed the importance of complex, distributed interactions between disparate yet interdependent brain regions. Recent work has shown that topological properties of functional brain networks are associated with individual and group differences in cognitive performance, including in episodic memory. After constructing functional whole-brain networks derived from an event-related fMRI study of memory retrieval, we examined differences in functional brain network architecture between forgotten and remembered words. This study yielded three main findings. First, graph theory analyses showed that successfully remembering compared to forgetting was associated with significant changes in the connectivity profile of the left hippocampus and a corresponding increase in efficient communication with the rest of the brain. Second, bivariate functional connectivity analyses indicated stronger interactions between the left hippocampus and a retrieval assembly for remembered versus forgotten items. This assembly included the left precuneus, left caudate, bilateral supramarginal gyrus, and the bilateral dorsolateral superior frontal gyrus. Integrative properties of the retrieval assembly were greater for remembered than forgotten items. Third, whole-brain modularity analyses revealed that successful memory retrieval was marginally significantly associated with a less segregated modular architecture in the network. The magnitude of the decreases in modularity between remembered and forgotten conditions was related to memory performance. These findings indicate that increases in integrative properties at the nodal, retrieval assembly, and whole-brain topological levels facilitate memory retrieval, while also underscoring the potential of multivariate brain connectivity approaches for providing valuable new insights into the neural bases of memory processes. Hum Brain Mapp 38:2242-2259, 2017. © 2017 Wiley Periodicals, Inc.

Hippocampal Contributions to the Large-Scale Episodic Memory Network Predict Vivid Visual Memories.

A common approach in memory research is to isolate the function(s) of individual brain regions, such as the hippocampus, without addressing how those regions interact with the larger network. To investigate the properties of the hippocampus embedded within large-scale networks, we used functional magnetic resonance imaging and graph theory to characterize complex hippocampal interactions during the active retrieval of vivid versus dim visual memories. The study yielded 4 main findings. First, the right hippocampus displayed greater communication efficiency with the network (shorter path length) and became a more convergent structure for information integration (higher centrality measures) for vivid than dim memories. Second, vivid minus dim differences in our graph theory measures of interest were greater in magnitude for the right hippocampus than for any other region in the 90-region network. Moreover, the right hippocampus significantly reorganized its set of direct connections from dim to vivid memory retrieval. Finally, beyond the hippocampus, communication throughout the whole-brain network was more efficient (shorter global path length) for vivid than dim memories. In sum, our findings illustrate how multivariate network analyses can be used to investigate the roles of specific regions within the large-scale network, while also accounting for global network changes.

A multimodal analysis of antipsychotic effects on brain structure and function in first-episode schizophrenia.

IMPORTANCE: Recent data suggest that treatment with antipsychotics is associated with reductions in cortical gray matter in patients with schizophrenia. These findings have led to concerns about the effect of antipsychotic treatment on brain structure and function; however, no studies to date have measured cortical function directly in individuals with schizophrenia and shown antipsychotic-related reductions of gray matter. OBJECTIVE: To examine the effects of antipsychotics on brain structure and function in patients with first-episode schizophrenia, using cortical thickness measurements and administration of the AX version of the Continuous Performance Task (AX-CPT) during event-related functional magnetic resonance imaging. DESIGN, SETTING, AND PARTICIPANTS: This case-control cross-sectional study was conducted at the Imaging Research Center of the University of California, Davis, from November 2004 through July 2012. Participants were recruited on admission into the Early Diagnosis and Preventive Treatment Clinic, an outpatient clinic specializing in first-episode psychosis. Patients with first-episode schizophrenia who received atypical antipsychotics (medicated patient group) (n = 23) and those who received no antipsychotics (unmedicated patient group) (n = 22) and healthy control participants (n = 37) underwent functional magnetic resonance imaging using a 1.5-T scanner. MAIN OUTCOMES AND MEASURES: Behavioral performance was measured by trial accuracy, reaction time, and d'-context score. Voxelwise statistical parametric maps tested differences in functional activity during the AX-CPT, and vertexwise maps of cortical thickness tested differences in cortical thickness across the whole brain. RESULTS: Significant cortical thinning was identified in the medicated patient group relative to the control group in prefrontal (mean reduction [MR], 0.27 mm; P < .001), temporal (MR, 0.34 mm; P = .02), parietal (MR, 0.21 mm; P = .001), and occipital (MR, 0.24 mm; P = .001) cortices. The unmedicated patient group showed no significant cortical thickness differences from the control group after clusterwise correction. The medicated patient group showed thinner cortex compared with the unmedicated patient group in the dorsolateral prefrontal cortex (DLPFC) (MR, 0.26 mm; P = .001) and temporal cortex (MR, 0.33 mm; P = .047). During the AX-CPT, both patient groups showed reduced DLPFC activity compared with the control group (P = .02 compared with the medicated group and P < .001 compared with the unmedicated group). However, the medicated patient group demonstrated higher DLPFC activation (P = .02) and better behavioral performance (P = .02) than the unmedicated patient group. CONCLUSIONS AND RELEVANCE: These findings highlight the complex relationship between antipsychotic treatment and the structural, functional, and behavioral deficits repeatedly identified in schizophrenia. Although short-term treatment with antipsychotics was associated with prefrontal cortical thinning, treatment was also associated with better cognitive control and increased prefrontal functional activity. This study adds important context to the growing literature on the effects of antipsychotics on the brain and suggests caution in interpreting neuroanatomical changes as being related to a potentially adverse effect on brain function.