Large-scale neural dynamics in a shared low-dimensional state space reflect cognitive and attentional dynamics.

Cognition and attention arise from the adaptive coordination of neural systems in response to external and internal demands. The low-dimensional latent subspace that underlies large-scale neural dynamics and the relationships of these dynamics to cognitive and attentional states, however, are unknown. We conducted functional magnetic resonance imaging as human participants performed attention tasks, watched comedy sitcom episodes and an educational documentary, and rested. Whole-brain dynamics traversed a common set of latent states that spanned canonical gradients of functional brain organization, with global desynchronization among functional networks modulating state transitions. Neural state dynamics were synchronized across people during engaging movie watching and aligned to narrative event structures. Neural state dynamics reflected attention fluctuations such that different states indicated engaged attention in task and naturalistic contexts, whereas a common state indicated attention lapses in both contexts. Together, these results demonstrate that traversals along large-scale gradients of human brain organization reflect cognitive and attentional dynamics.

Automatic compensation enhances the orientation perception in chronic astigmatism.

Astigmatism is a prevalent optical problem in which two or more focal points blur the retinal image at a particular meridian. Although many features of astigmatic vision, including orientation perception, are impaired at the retinal image level, the visual system appears to partly restore perceptual impairment after an extended period of astigmatism. However, the mechanism of orientation perception restoration in chronic astigmatism has not yet been clarified. We investigated the notable reduction of perceptual error in chronic astigmatism by comparing the orientation perception of a chronic astigmatism group with the perception of a normal-vision group, in which astigmatism was transiently induced. We found that orientation perception in the chronic group was more accurate than in the normal vision group. Interestingly, the reduction of perceptual errors was automatic; it remained even after the optical refractive errors were fully corrected, and the orientation perception was much more stable across different orientations, despite the uneven noise levels of the retinal images across meridians. We provide here a mechanistic explanation for how the compensation of astigmatic orientation perception occurred, using neural adaptation to the biased distribution of orientations.

Neural representations of ensemble coding in the occipital and parietal cortices.

The human visual system is able to extract summary statistics from sets of similar items, but the underlying neural mechanism remains poorly understood. Using functional magnetic resonance imaging (fMRI) and an encoding model, we examined how the neural representation of ensemble coding is constructed by manipulating the task-relevance of ensemble features. We found a gradual increase in orientation-selective responses to the mean orientation of multiple stimuli along the visual hierarchy only when these orientations were task-relevant. Such responses to the ensemble orientation were present in the extrastriate area, V3, even when the mean orientation was not task-relevant, indicating that the ensemble representation can co-exist with the task-relevant individual feature representation. Ensemble orientations were also represented in frontal regions, but those representations were robust only when each mean orientation was linked to a motor response dimension. Together, our findings suggest that the neural representation of the ensemble percept is formed by pooling signals at multiple levels of the visual processing stream.

Cognitive and Neural State Dynamics of Narrative Comprehension.

Narrative comprehension involves a constant interplay of the accumulation of incoming events and their integration into a coherent structure. This study characterizes cognitive states during narrative comprehension and the network-level reconfiguration occurring dynamically in the functional brain. We presented movie clips of temporally scrambled sequences to human participants (male and female), eliciting fluctuations in the subjective feeling of comprehension. Comprehension occurred when processing events that were highly causally related to the previous events, suggesting that comprehension entails the integration of narratives into a causally coherent structure. The functional neuroimaging results demonstrated that the integrated and efficient brain state emerged during the moments of narrative integration with the increased level of activation and across-modular connections in the default mode network. Underlying brain states were synchronized across individuals when comprehending novel narratives, with increased occurrences of the default mode network state, integrated with sensory processing network, during narrative integration. A model based on time-resolved functional brain connectivity predicted changing cognitive states related to comprehension that are general across narratives. Together, these results support adaptive reconfiguration and interaction of the functional brain networks on causal integration of the narratives.SIGNIFICANCE STATEMENT The human brain can integrate temporally disconnected pieces of information into coherent narratives. However, the underlying cognitive and neural mechanisms of how the brain builds a narrative representation remain largely unknown. We showed that comprehension occurs as the causally related events are integrated to form a coherent situational model. Using fMRI, we revealed that the large-scale brain states and interaction between brain regions dynamically reconfigure as comprehension evolves, with the default mode network playing a central role during moments of narrative integration. Overall, the study demonstrates that narrative comprehension occurs through a dynamic process of information accumulation and causal integration, supported by the time-varying reconfiguration and brain network interaction.

Feasibility of head-tilted brain scan to reduce susceptibility-induced signal loss in the prefrontal cortex in gradient echo-based imaging.

Susceptibility-induced static field (B0) inhomogeneity near the nasal cavity degrades high-field MRI image quality. Many studies have addressed this problem by hardware- or sequence-based methods to improve local B0 shimming or minimize the impact of inhomogeneity. Here, we investigate the feasibility of the head-tilted brain scan as an easily accessible way to reduce B0 inhomogeneity and associated gradient echo signal loss in the prefrontal cortex (PFC). We exploit the fact that the region of intense local B0 gradient can be steered away from the PFC by head reorientation with respect to the main magnetic field. We found that the required chin-up head tilting by a substantial angle (> 30°) can be readily achieved for a group of healthy subjects when their back was raised by about 10  cm. Eleven subjects were scanned at 3T, using a standard 20 channel head-neck coil, for whole-head B0 mapping and gradient-echo EPI-based functional MRI (fMRI) performing a reward-punishment task in normal and tilted head orientations. Additionally, multi-echo gradient echo and resting-state fMRI scans were performed on six subjects in both orientations. Head-tilted sessions, which lasted for at least 20 min, were well-tolerated by all subjects and demonstrated a marked reduction of localized signal loss in the gradient echo-based images and EPI images in the PFC compared to normal orientation scans. Imaging in tilted orientation reduced the group-averaged B0 standard deviation and peak B0 gradient in the orbital gyrus beyond what was possible with simulated 3rd order shimming. The behavioral performance in the head-tilted fMRI scans indicated that the subjects were able to perform a cognitive task with little difficulty, and the tilted fMRI scans successfully produced a robust whole-brain functional activation map consistent with the literature. Our study proposes that the back-raised, head-tilted imaging can benefit the shimming of the prefrontal brain regions while being compatible with moderate-length neuroimaging scans on healthy, cooperating subjects.

Neural representations of perceptual color experience in the human ventral visual pathway.

Color is a perceptual construct that arises from neural processing in hierarchically organized cortical visual areas. Previous research, however, often failed to distinguish between neural responses driven by stimulus chromaticity versus perceptual color experience. An unsolved question is whether the neural responses at each stage of cortical processing represent a physical stimulus or a color we see. The present study dissociated the perceptual domain of color experience from the physical domain of chromatic stimulation at each stage of cortical processing by using a switch rivalry paradigm that caused the color percept to vary over time without changing the retinal stimulation. Using functional MRI (fMRI) and a model-based encoding approach, we found that neural representations in higher visual areas, such as V4 and VO1, corresponded to the perceived color, whereas responses in early visual areas V1 and V2 were modulated by the chromatic light stimulus rather than color perception. Our findings support a transition in the ascending human ventral visual pathway, from a representation of the chromatic stimulus at the retina in early visual areas to responses that correspond to perceptually experienced colors in higher visual areas.

Possible links between the lag structure in visual cortex and visual streams using fMRI.

Conventional functional connectivity analysis using functional magnetic resonance imaging (fMRI) measures the correlation of temporally synchronized brain activities between brain regions. Lag structure analysis relaxes the synchronicity constraint of fMRI signals, and thus, this approach might be better at explaining functional connectivity. However, the sources of the lag structure in fMRI are primarily unknown. Here, we applied lag structure analysis to the human visual cortex to identify the possible sources of lag structure. A total of 1,250 fMRI data from two independent databases were considered. We explored the temporal lag patterns between the central and peripheral visual fields in early visual cortex and those in two visual pathways of dorsal and ventral streams. We also compared the lag patterns with effective connectivity obtained with dynamic causal modeling. We found that the lag structure in early visual cortex flows from the central to peripheral visual fields and the order of the lag structure flow was consistent with the order of signal flows in visual pathways. The effective connectivity computed by dynamic causal modeling exhibited similar patterns with the lag structure results. This study suggests that signal flows in visual streams are possible sources of the lag structure in human visual cortex.

Temporal-Order-Based Attentional Priority Modulates Mnemonic Representations in Parietal and Frontal Cortices.

The respective roles of occipital, parietal, and frontal cortices in visual working memory maintenance have long been under debate. Previous work on whether parietal and frontal regions convey mnemonic information has yielded mixed findings. One possibility for this variability is that the mnemonic representations in high-level frontoparietal regions are modulated by attentional priority, such as temporal order. To test this hypothesis, we examined whether the most recent item, which has a higher attentional priority in terms of temporal order, is preferentially encoded in frontoparietal regions. On each trial, participants viewed 2 gratings with different orientations in succession, and were cued to remember one of them. Using fMRI and an inverted encoding model, we reconstructed population-level, orientation representations in occipital (V1-V3), parietal (IPS), and frontal (FEF) areas during memory maintenance. Unlike early visual cortex where robust orientation representations were observed regardless of serial order, parietal, and frontal cortices showed stronger representations when participants remembered the second grating. A subsequent experiment using a change detection task on color rings excluded the possibilities of residual stimulus-driven signals or motor preparative signals for responses. These results suggest that mnemonic representations in parietal and frontal cortices are modulated by temporal-order-based attentional priority signals.

Functional connectivity based parcellation of early visual cortices.

Human brain can be divided into multiple brain regions based on anatomical and functional properties. Recent studies showed that resting-state connectivity can be utilized for parcellating brain regions and identifying their distinctive roles. In this study, we aimed to parcellate the primary and secondary visual cortices (V1 and V2) into several subregions based on functional connectivity and to examine the functional characteristics of each subregion. We used resting-state data from a research database and also acquired resting-state data with retinotopy results from a local site. The long-range connectivity profile and three different algorithms (i.e., K-means, Gaussian mixture model distribution, and Ward's clustering algorithms) were adopted for the parcellation. We compared the parcellation results within V1 and V2 with the eccentric map in retinotopy. We found that the boundaries between subregions within V1 and V2 were located in the parafovea, indicating that the anterior and posterior subregions within V1 and V2 corresponded to peripheral and central visual field representations, respectively. Next, we computed correlations between each subregion within V1 and V2 and intermediate and high-order regions in ventral and dorsal visual pathways. We found that the anterior subregions of V1 and V2 were strongly associated with regions in the dorsal stream (V3A and inferior parietal gyrus), whereas the posterior subregions of V1 and V2 were highly related to regions in the ventral stream (V4v and inferior temporal gyrus). Our findings suggest that the anterior and posterior subregions of V1 and V2, parcellated based on functional connectivity, may have distinct functional properties.

Occipital, parietal, and frontal cortices selectively maintain task-relevant features of multi-feature objects in visual working memory.

Previous studies have shown that information held in visual working memory is represented in the occipital, parietal, and frontal cortices. However, less is known about whether the mnemonic information of multi-feature objects is modulated by task demand in the parietal and frontal regions. To address this question, we asked participants to remember either color or orientation of one of the two colored gratings for a delay. Using fMRI and an inverted encoding model, we reconstructed population-level, feature-selective responses in the occipital, parietal and frontal cortices during memory maintenance. We found that not only orientation but also color information can be maintained in higher-order parietal and frontal cortices as well as the early visual cortex when it was cued to be remembered. Conversely, neither the task-irrelevant feature of the cued object, nor any feature of the uncued object was maintained in the occipital, parietal, or frontal cortices. These results suggest a highly selective mechanism of visual working memory that maintains task-relevant features only.

Modulating foveal representation can influence visual discrimination in the periphery.

A previous study by Williams et al. (2008) provided evidence for a novel form of feedback in the visual system, whereby peripheral information is contained in foveal retinotopic cortex. Beyond its possible implication for peripheral object recognition, few studies have examined the effect of a direct behavioral manipulation of the foveal feedback representation. To address this question, we measured participants' peripheral visual discrimination performance while modulating their foveal representation in a series of psychophysical experiments. On each trial, participants discriminated the identities of briefly presented novel, three-dimensional objects or the orientations of gratings in a peripheral location while fixating at the center. Besides the peripheral target, another stimulus (foil) was also presented and masked at the fovea. Our results showed that for objects, when the foveal foil that was identical to the peripheral target was presented 150 ms after the onset of the peripheral target, visual discrimination of the peripheral target was improved. This congruency effect occurred even though participants did not consciously perceive the foveal stimulus. No such effect was observed when the foveal foil was presented simultaneously with the peripheral target, or when the foil was presented in a parafoveal location. The foil effect in gratings was different from that in objects in terms of its effective timing and foveal specificity, suggesting that foveal feedback may be specific to high-level objects. These results indicate that modulating foveal information can affect individuals' ability to discriminate peripheral objects, suggesting a functional role of foveal representations in peripheral visual perception.

When audiovisual correspondence disturbs visual processing.

Multisensory integration is known to create a more robust and reliable perceptual representation of one's environment. Specifically, a congruent auditory input can make a visual stimulus more salient, consequently enhancing the visibility and detection of the visual target. However, it remains largely unknown whether a congruent auditory input can also impair visual processing. In the current study, we demonstrate that temporally congruent auditory input disrupts visual processing, consequently slowing down visual target detection. More importantly, this cross-modal inhibition occurs only when the contrast of visual targets is high. When the contrast of visual targets is low, enhancement of visual target detection is observed, consistent with the prediction based on the principle of inverse effectiveness (PIE) in cross-modal integration. The switch of the behavioral effect of audiovisual interaction from benefit to cost further extends the PIE to encompass the suppressive cross-modal interaction.

Reconstructing representations of dynamic visual objects in early visual cortex.

As raw sensory data are partial, our visual system extensively fills in missing details, creating enriched percepts based on incomplete bottom-up information. Despite evidence for internally generated representations at early stages of cortical processing, it is not known whether these representations include missing information of dynamically transforming objects. Long-range apparent motion (AM) provides a unique test case because objects in AM can undergo changes both in position and in features. Using fMRI and encoding methods, we found that the "intermediate" orientation of an apparently rotating grating, never presented in the retinal input but interpolated during AM, is reconstructed in population-level, feature-selective tuning responses in the region of early visual cortex (V1) that corresponds to the retinotopic location of the AM path. This neural representation is absent when AM inducers are presented simultaneously and when AM is visually imagined. Our results demonstrate dynamic filling-in in V1 for object features that are interpolated during kinetic transformations.

Seeing the world through target-tinted glasses: Positive mood broadens perceptual tuning.

Research shows that positive mood can serve to broaden the scope of attention at both the perceptual and conceptual level (e.g., increasing the size of spatial attentional focus and semantic access to remote associates). We investigated whether this relaxation of attentional filters by positive affect reduces their selectivity for basic visual features. We induced positive, neutral, or negative affect and asked observers to identify a target motion direction in a series of rapid random moving dot displays. Using a reverse correlation method, we examined the differential effects of emotion on observers' perceptual tuning curves for motion direction. Here we find that positive affect reduces selectivity for motion direction by broadening observers' perceptual tuning relative to both neutral and negative affect conditions. These findings provide the first behavioral evidence that positive emotion influences selectivity for basic visual features through modulation of tuning properties.

Color updating on the apparent motion path.

When a static stimulus appears successively at two distant locations, we perceive illusory motion of the stimulus across them-long-range apparent motion (AM). Previous studies have shown that when the apparent motion stimuli differ in shape, interpolation between the two shapes is perceived across the AM path. In contrast, the perceived color during AM has been shown to abruptly change from the color of the first stimulus into that of the second, suggesting interpolation does not occur for color during AM. Here, we report the first evidence to our knowledge, that an interpolated color, distinct from the colors of either apparent motion stimulus, is represented as the intermediate percept on the path of apparent motion. Using carefully chosen target colors-cyan, pink, and lime-that are perceptually and neurally intermediate between blue and green, orange and magenta, and green and orange respectively, we show that detection of a target presented on the apparent motion path was impaired when the color of the target was "in-between" the initial and terminal stimulus colors. Furthermore, we show that this feature-specific masking effect for the intermediate color cannot be accounted for by color similarity between the intermediate color and the color of the terminal inducer. Our findings demonstrate that intermediate colors can be interpolated over the apparent motion trajectory as in the case of shape, possibly involving similar interpolation processes for shape and color during apparent motion.

Backward position shift in apparent motion.

We investigated the perceived position of visual targets in apparent motion. A disc moved horizontally through three positions from -10° to +10° in the far periphery (20° above fixation), generating a compelling impression of apparent motion. In the first experiment, observers compared the position of the middle of the three discs to a subsequently presented reference. Unexpectedly, observers judged its position to be shifted backward, in the direction opposite that of the motion. We then tested the middle disc in sequences of 3, 5, and 7 discs, each covering the same spatial and temporal extents (similar speeds). The backwards shift was only found for the three-disc sequence. With the extra discs approaching more continuous motion, the perceived shift was in the same direction as the apparent motion. Finally, using a localization task with constant static references, we measured the position shifts of all the disc locations for two-disc, three-disc and four-disc apparent motion sequences. The backward shift was found for the second location of all sequences. We suggest that the backward shift of the second element along an apparent motion path is due to an attraction effect induced by the initial point of the motion.

Redundancy gains in retinotopic cortex.

It is widely claimed that interactions among simultaneously presented visual stimuli are suppressive and that these interactions primarily occur when stimuli fall within the same receptive field (Desimone and Duncan 1995). Here, we show evidence for a novel form of interaction between simultaneously presented but distant stimuli that does not fit either pattern. To examine interactions between simultaneously presented stimuli, we measure the response to a single stimulus as a function of whether or not other stimuli are also presented simultaneously, and we further ask how the response to a given stimulus is affected by whether the simultaneously present stimuli are identical or different from each other. Our method reveals a new phenomenon of "redundancy gain:" the visual response to a stimulus is higher when accompanied by identical stimuli than when that stimulus is presented alone, even though the stimuli are displayed in separate visual quadrants. This pattern is observed throughout the visual hierarchy, including V1 and V2, and we show that it is distinct from the well-known simultaneous suppression effect (Kastner et al. 1998). We propose that the redundancy gain in early retinotopic cortex results from feedback from higher visual areas and may underlie perceptual averaging and other ensemble coding phenomena observed behaviorally.

The number of attentional foci and their precision are dissociated in the posterior parietal cortex.

Many everyday tasks require us to track moving objects with attention. The demand for attention increases both when more targets are tracked and when the targets move faster. These 2 aspects of attention-assigning multiple attentional foci (or indices) to targets and monitoring each focus with precision-may tap into different cognitive and brain mechanisms. In this study, we used functional magnetic resonance imaging to quantify the response profile of dorsal attentional areas to variations in the number of attentional foci and their spatiotemporal precision. Subjects were asked to track a specific spoke of either 1 or 2 pinwheels that rotated at various speeds. Their tracking performance declined both when more pinwheels were tracked and when the tracked pinwheels rotated faster. However, posterior parietal activity increased only when subjects tracked more pinwheels but remained flat when they tracked faster moving pinwheels. The frontal eye fields and early visual areas increased activity when there were more targets and when the targets rotated faster. These results suggest that the posterior parietal cortex is specifically involved in indexing independently moving targets with attention but not in monitoring each focus with precision.

Supercrowding: weakly masking a target expands the range of crowding.

Crowding is impairment of peripheral object identification by nearby objects. Critical spacing (the minimum target-flanker distance that does not produce crowding) scales with target eccentricity and is consistently reported as roughly equal to or less than 50% of target eccentricity (0.5e). This study demonstrates that crowding occurs far beyond the typical critical spacing when the target is weakly masked by a surrounding contour or backwards pattern mask. A target was presented at a peripheral location on every trial and participants reported its orientation. Flankers appeared at target-flanker distances of 0.3-0.7e, or were absent. The target was presented with or without a mask. When flankers were absent, the masks only mildly impaired performance. When flankers were present but the mask was absent, target identification was nearly perfect at wide target-flanker distances (0.5e-0.7e). However, when flankers were present and the target was masked, performance dropped significantly, even when target-flanker distances far exceeded the typical crowding range. This phenomenon ("supercrowding") shares critical features with standard crowding: flankers similar to the target impair performance more than dissimilar flankers, and the characteristic anisotropic profile of crowding is preserved. Supercrowding may reflect a general interaction between crowding and other forms of masking.

Visual working memory for line orientations and face identities.

Previous studies have shown that the number of objects we can actively hold in visual working memory is smaller for more complex objects. However, complex objects are not just more complex but are often more similar to other complex objects used as test probes. To separate effects of complexity from effects of similarity, we measured visual memory following a 1-sec delay for complex and simple objects at several levels of memory-to-test similarity. When memory load was one object, memory accuracy for a face (a complex attribute) was similar to a line orientation (a simple attribute) when the face changed in steps of 10% along a morphing continuum and the line changed in steps of 5 degrees in orientation. Performance declined with increasing memory load and increasing memory-to-test similarity. Remarkably, when memory load was three or four objects, face memory was better than orientation memory at similar changed steps. These results held when memory for line orientations was compared with that for inverted faces. We conclude that complex objects do not always exhaust visual memory more quickly than simple objects do.