Working memory signals in early visual cortex are present in weak and strong imagers.

It has been suggested that visual images are memorized across brief periods of time by vividly imagining them as if they were still there. In line with this, the contents of both working memory and visual imagery are known to be encoded already in early visual cortex. If these signals in early visual areas were indeed to reflect a combined imagery and memory code, one would predict them to be weaker for individuals with reduced visual imagery vividness. Here, we systematically investigated this question in two groups of participants. Strong and weak imagers were asked to remember images across brief delay periods. We were able to reliably reconstruct the memorized stimuli from early visual cortex during the delay. Importantly, in contrast to the prediction, the quality of reconstruction was equally accurate for both strong and weak imagers. The decodable information also closely reflected behavioral precision in both groups, suggesting it could contribute to behavioral performance, even in the extreme case of completely aphantasic individuals. Our data thus suggest that working memory signals in early visual cortex can be present even in the (near) absence of phenomenal imagery.

Nonfrontal Control of Working Memory.

Items held in visual working memory can be quickly updated, replaced, removed, and even manipulated in accordance with current behavioral goals. Here, we use multivariate pattern analyses to identify the patterns of neuronal activity that realize the executive control processes supervising these flexible stores. We find that portions of the middle temporal gyrus and the intraparietal sulcus represent what item is cued for continued memorization independently of representations of the item itself. Importantly, this selection-specific activity could not be explained by sensory representations of the cue and is only present when control is exerted. Our results suggest that the selection of memorized items might be controlled in a distributed and decentralized fashion. This evidence provides an alternative perspective to the notion of "domain general" central executive control over memory function.

Categorical working memory codes in human visual cortex.

Working memory contents are represented in neural activity patterns across multiple regions of the cortical hierarchy. A division of labor has been proposed where more anterior regions harbor increasingly abstract and categorical representations while the most detailed representations are held in primary sensory cortices. Here, using fMRI and multivariate encoding modeling, we demonstrate that for color stimuli categorical codes are already present at the level of extrastriate visual cortex (V4 and VO1), even when subjects are neither implicitly nor explicitly encouraged to categorize the stimuli. Importantly, this categorical coding was observed during working memory, but not during perception. Thus, visual working memory is likely to rely at least in part on categorical representations. SIGNIFICANCE STATEMENT: Working memory is the representational basis for human cognition. Recent work has demonstrated that numerous regions across the human brain can represent the contents of working memory. We use fMRI brain scanning and machine learning methods to demonstrate that different regions can represent the same content differently during working memory. Reading out the neural codes used to store working memory contents, we show that already in sensory cortex, areas V4 and VO1 represent color in a categorical format rather than a purely sensory fashion. Thereby, we provide a better understanding of how different regions of the brain might serve working memory and cognition.

Decoding verbal working memory representations of Chinese characters from Broca's area.

Representations of sensory working memory can be found across the entire neocortex. But how are verbal working memory (VWM) contents retained in the human brain? Here we used fMRI and multi-voxel pattern analyses to study Chinese native speakers (15 males, 13 females) memorizing Chinese characters. Chinese characters are uniquely suitable to study VWM because verbal encoding is encouraged by their complex visual appearance and monosyllabic pronunciation. We found that activity patterns in Broca's area and left premotor cortex carried information about the memorized characters. These language-related areas carried (1) significantly more information about cued characters than those not cued for memorization, (2) significantly more information on the left than the right hemisphere and (3) significantly more information about Chinese symbols than complex visual patterns which are hard to verbalize. In contrast, early visual cortex carries a comparable amount of information about cued and uncued stimuli and is thus unlikely to be involved in memory retention. This study provides evidence for verbal working memory maintenance in a distributed network of language-related brain regions, consistent with distributed accounts of WM. The results also suggest that Broca's area and left premotor cortex form the articulatory network which serves articulatory rehearsal in the retention of verbal working memory contents.

No evidence for mnemonic modulation of interocularly suppressed visual input.

Visual working memory (VWM) allows for keeping visual information available for upcoming goal-directed behavior, while new visual input is processed concurrently. Interactions between the mnemonic and perceptual systems cause VWM to affect the processing of visual input in a content-specific manner: visual input that is initially suppressed from consciousness is detected faster when it matches rather than mismatches the content of VWM. It is currently under debate whether such mnemonic influences on perception occur prior to or after conscious access. To address this issue, we investigated whether VWM content modulates the neural response to visual input that remains suppressed from consciousness. We measured fMRI responses to interocularly suppressed stimuli in 20 human participants performing a delayed match-to-sample task: Participants were retro-cued to memorize one of two geometrical shapes for subsequent recognition. During retention, an interocularly suppressed peripheral stimulus (the probe) was briefly presented, which was either of the cued (memorized) or uncued (not memorized) shape category. We found no evidence that VWM content modulated the neural response to the probe. Substantial evidence for the absence of this modulation was found despite leveraging a highly liberal analysis approach: (1) selecting regions of interest that were particularly prone to detecting said modulation, and (2) using directional Bayesian tests favoring the presence of the hypothesized modulation. We did observe faster detection of memory-matching compared to memory-mismatching probes in a behavioral control experiment, thus validating the stimulus set. We conclude that VWM impacts the processing of visual input only once suppression is mostly alleviated.

Cortical specialization for attended versus unattended working memory.

Items held in working memory can be either attended or not, depending on their current behavioral relevance. It has been suggested that unattended contents might be solely retained in an activity-silent form. Instead, we demonstrate here that encoding unattended contents involves a division of labor. While visual cortex only maintains attended items, intraparietal areas and the frontal eye fields represent both attended and unattended items.

View-Independent Working Memory Representations of Artificial Shapes in Prefrontal and Posterior Regions of the Human Brain.

Traditional views of visual working memory postulate that memorized contents are stored in dorsolateral prefrontal cortex using an adaptive and flexible code. In contrast, recent studies proposed that contents are maintained by posterior brain areas using codes akin to perceptual representations. An important question is whether this reflects a difference in the level of abstraction between posterior and prefrontal representations. Here, we investigated whether neural representations of visual working memory contents are view-independent, as indicated by rotation-invariance. Using functional magnetic resonance imaging and multivariate pattern analyses, we show that when subjects memorize complex shapes, both posterior and frontal brain regions maintain the memorized contents using a rotation-invariant code. Importantly, we found the representations in frontal cortex to be localized to the frontal eye fields rather than dorsolateral prefrontal cortices. Thus, our results give evidence for the view-independent storage of complex shapes in distributed representations across posterior and frontal brain regions.

Visual Working Memory Enhances the Neural Response to Matching Visual Input.

Visual working memory (VWM) is used to maintain visual information available for subsequent goal-directed behavior. The content of VWM has been shown to affect the behavioral response to concurrent visual input, suggesting that visual representations originating from VWM and from sensory input draw upon a shared neural substrate (i.e., a sensory recruitment stance on VWM storage). Here, we hypothesized that visual information maintained in VWM would enhance the neural response to concurrent visual input that matches the content of VWM. To test this hypothesis, we measured fMRI BOLD responses to task-irrelevant stimuli acquired from 15 human participants (three males) performing a concurrent delayed match-to-sample task. In this task, observers were sequentially presented with two shape stimuli and a retro-cue indicating which of the two shapes should be memorized for subsequent recognition. During the retention interval, a task-irrelevant shape (the probe) was briefly presented in the peripheral visual field, which could either match or mismatch the shape category of the memorized stimulus. We show that this probe stimulus elicited a stronger BOLD response, and allowed for increased shape-classification performance, when it matched rather than mismatched the concurrently memorized content, despite identical visual stimulation. Our results demonstrate that VWM enhances the neural response to concurrent visual input in a content-specific way. This finding is consistent with the view that neural populations involved in sensory processing are recruited for VWM storage, and it provides a common explanation for a plethora of behavioral studies in which VWM-matching visual input elicits a stronger behavioral and perceptual response.SIGNIFICANCE STATEMENT Humans heavily rely on visual information to interact with their environment and frequently must memorize such information for later use. Visual working memory allows for maintaining such visual information in the mind's eye after termination of its retinal input. It is hypothesized that information maintained in visual working memory relies on the same neural populations that process visual input. Accordingly, the content of visual working memory is known to affect our conscious perception of concurrent visual input. Here, we demonstrate for the first time that visual input elicits an enhanced neural response when it matches the content of visual working memory, both in terms of signal strength and information content.

The Distributed Nature of Working Memory.

Studies in humans and non-human primates have provided evidence for storage of working memory contents in multiple regions ranging from sensory to parietal and prefrontal cortex. We discuss potential explanations for these distributed representations: (i) features in sensory regions versus prefrontal cortex differ in the level of abstractness and generalizability; and (ii) features in prefrontal cortex reflect representations that are transformed for guidance of upcoming behavioral actions. We propose that the propensity to produce persistent activity is a general feature of cortical networks. Future studies may have to shift focus from asking where working memory can be observed in the brain to how a range of specialized brain areas together transform sensory information into a delayed behavioral response.

Increasing the spatial resolution and sensitivity of magnetic resonance elastography by correcting for subject motion and susceptibility-induced image distortions.

PURPOSE: To improve the resolution of elasticity maps by adapting motion and distortion correction methods for phase-based magnetic resonance imaging (MRI) contrasts such as magnetic resonance elastography (MRE), a technique for measuring mechanical tissue properties in vivo. MATERIALS AND METHODS: MRE data of the brain were acquired with echo-planar imaging (EPI) at 3T (n = 14) and 7T (n = 18). Motion and distortion correction parameters were estimated using the magnitude images. The real and imaginary part of the complex MRE data were corrected separately and recombined. The width of the point-spread function (PSF) and the position variability were calculated. The images were normalized to the Montreal Neurological Institute (MNI) anatomical template. The gray-to-white matter separability of the elasticity maps was tested. RESULTS: Motion correction sharpened the |G*| maps as demonstrated by a narrowing of the PSF by 0.78 ± 0.51 mm at 7T and 0.52 ± 0.63 mm at 3T. The amount of individual head motion during MRE acquisition correlated with the decrease in the width of the PSF at 7T (r = 0.53, P = 0.025) and at 3T (r = 0.69, P = 0.006) and with the increase of gray-to-white matter separability after motion correction at 7T (r = 0.64, P = 0.0039) and at 3T (r = 0.57, P = 0.0319). Improved spatial accuracy after distortion correction results in a significant increase in separability of gray and white matter stiffness (P = 0.0067), especially in inferior parts of the brain suffering from strong B0 inhomogeneities. CONCLUSION: We demonstrate that our method leads to sharper images and higher spatial accuracy, raising the prospect of the investigation of smaller brain areas with increased sensitivity in studies using MRE. LEVEL OF EVIDENCE: 1 Technical Efficacy: Stage 1 J. MAGN. RESON. IMAGING 2017;46:134-141.

Parietal and early visual cortices encode working memory content across mental transformations.

Active and flexible manipulations of memory contents "in the mind's eye" are believed to occur in a dedicated neural workspace, frequently referred to as visual working memory. Such a neural workspace should have two important properties: The ability to store sensory information across delay periods and the ability to flexibly transform sensory information. Here we used a combination of functional MRI and multivariate decoding to indentify such neural representations. Subjects were required to memorize a complex artificial pattern for an extended delay, then rotate the mental image as instructed by a cue and memorize this transformed pattern. We found that patterns of brain activity already in early visual areas and posterior parietal cortex encode not only the initially remembered image, but also the transformed contents after mental rotation. Our results thus suggest that the flexible and general neural workspace supporting visual working memory can be realized within posterior brain regions.

Decoding pattern motion information in V1.

When two gratings drifting in different directions are superimposed, the resulting stimulus can be perceived as two overlapping component gratings moving in different directions or as a single pattern moving in one direction. Whilst the motion direction of component gratings is already represented in visual area V1, the majority of previous studies have found processing of pattern motion direction only from visual area V2 onwards. Here, we question these findings using multi-voxel pattern analysis (MVPA). In Experiment 1, we presented superimposed sinusoidal gratings with varying angles between the two component motions. These stimuli were perceived as patterns moving in one of two possible directions. We found that linear support vector machines (SVMs) could generalise across stimuli composed of different component motions to successfully discriminate pattern motion direction from brain activity in V1, V3A and hMT+/V5. This demonstrates the representation of pattern motion information present in these visual areas. This conclusion was verified in Experiment 2, where we manipulated similar plaid stimuli to induce the perception of either a single moving pattern or two separate component gratings. While a classifier could again generalise across stimuli composed of different component motions when they were perceived as a single moving pattern, its performance dropped substantially in the case where components were perceived. Our results indicate that pattern motion direction information is present in V1.

Decoding complex flow-field patterns in visual working memory.

There has been a long history of research on visual working memory. Whereas early studies have focused on the role of lateral prefrontal cortex in the storage of sensory information, this has been challenged by research in humans that has directly assessed the encoding of perceptual contents, pointing towards a role of visual and parietal regions during storage. In a previous study we used pattern classification to investigate the storage of complex visual color patterns across delay periods. This revealed coding of such contents in early visual and parietal brain regions. Here we aim to investigate whether the involvement of visual and parietal cortex is also observable for other types of complex, visuo-spatial pattern stimuli. Specifically, we used a combination of fMRI and multivariate classification to investigate the retention of complex flow-field stimuli defined by the spatial patterning of motion trajectories of random dots. Subjects were trained to memorize the precise spatial layout of these stimuli and to retain this information during an extended delay. We used a multivariate decoding approach to identify brain regions where spatial patterns of activity encoded the memorized stimuli. Content-specific memory signals were observable in motion sensitive visual area MT+ and in posterior parietal cortex that might encode spatial information in a modality independent manner. Interestingly, we also found information about the memorized visual stimulus in somatosensory cortex, suggesting a potential crossmodal contribution to memory. Our findings thus indicate that working memory storage of visual percepts might be distributed across unimodal, multimodal and even crossmodal brain regions.

Multiple routes to mental animation: language and functional relations drive motion processing for static images.

When looking at static visual images, people often exhibit mental animation, anticipating visual events that have not yet happened. But what determines when mental animation occurs? Measuring mental animation using localized brain function (visual motion processing in the middle temporal and middle superior temporal areas, MT+), we demonstrated that animating static pictures of objects is dependent both on the functionally relevant spatial arrangement that objects have with one another (e.g., a bottle above a glass vs. a glass above a bottle) and on the linguistic judgment to be made about those objects (e.g., "Is the bottle above the glass?" vs. "Is the bottle bigger than the glass?"). Furthermore, we showed that mental animation is driven by functional relations and language separately in the right hemisphere of the brain but conjointly in the left hemisphere. Mental animation is not a unitary construct; the predictions humans make about the visual world are driven flexibly, with hemispheric asymmetry in the routes to MT+ activation.

Decoding the contents of visual short-term memory from human visual and parietal cortex.

How content is stored in the human brain during visual short-term memory (VSTM) is still an open question. Different theories postulate storage of remembered stimuli in prefrontal, parietal, or visual areas. Aiming at a distinction between these theories, we investigated the content-specificity of BOLD signals from various brain regions during a VSTM task using multivariate pattern classification. To participate in memory maintenance, candidate regions would need to have information about the different contents held in memory. We identified two brain regions where local patterns of fMRI signals represented the remembered content. Apart from the previously established storage in visual areas, we also discovered an area in the posterior parietal cortex where activity patterns allowed us to decode the specific stimuli held in memory. Our results demonstrate that storage in VSTM extends beyond visual areas, but no frontal regions were found. Thus, while frontal and parietal areas typically coactivate during VSTM, maintenance of content in the frontoparietal network might be limited to parietal cortex.

Coupling between wrist flexion-extension and radial-ulnar deviation.

BACKGROUND: Conventional wrist joint goniometry evaluates range of motion in isolated directions. The coupling between wrist flexion-extension and radial-ulnar deviation was investigated. METHODS: Ten healthy young male subjects performed wrist flexion-extension, radial-ulnar deviation, and circumduction motions. Flexion-extension and radial-ulnar deviation angles were computed from the coordinates of surface markers attached to the forearm and hand. A motion analysis system recorded marker motion. FINDINGS: During radial-ulnar deviation, the amount of accompanying flexion-extension movement was linearly related to the amount of radial-ulnar deviation. The secondary (flexion-extension) range of motion (48.3 degrees) was about 75% of the primary (radial-ulnar deviation) range of motion (55.1 degrees). During the flexion-extension task, the coupling was less linear. The motion range in radial-ulnar deviation (21.2 degrees) was about 20% of the primary (flexion-extension) range of motion (108.3 degrees). The radial-ulnar deviation and flexion-extension motions combined extension with radial deviation, and flexion with ulnar deviation. The convex hull of the flexion-extension and radial-ulnar deviation angles during circumduction was "egg-shaped" and asymmetric with respect to the anatomically defined flexion-extension and radial-ulnar deviation axes. Wrist position in one direction strongly influenced the range of motion in the other. Maximum range of motion in flexion-extension occurred with the wrist near the neutral radial-ulnar deviation position, and vice versa. Wrist deviation from neutral position in one direction diminished wrist range of motion in the other. INTERPRETATION: Wrist movements in flexion-extension and radial-ulnar deviation are coupled. Maximal wrist range of motion is near the neutral position. To account for the naturally coupled wrist motion in work station design and rehabilitation, the wrist should be placed at a neutral position.