Influence of colour vision on attention to, and impression of, complex aesthetic images.

Humans exhibit colour vision variations due to genetic polymorphisms, with trichromacy being the most common, while some people are classified as dichromats. Whether genetic differences in colour vision affect the way of viewing complex images remains unknown. Here, we investigated how people with different colour vision focused their gaze on aesthetic paintings by eye-tracking while freely viewing digital rendering of paintings and assessed individual impressions through a decomposition analysis of adjective ratings for the images. Gaze-concentrated areas among trichromats were more highly correlated than those among dichromats. However, compared with the brief dichromatic experience with the simulated images, there was little effect of innate colour vision differences on impressions. These results indicate that chromatic information is instructive as a cue for guiding attention, whereas the impression of each person is generated according to their own sensory experience and normalized through one's own colour space.

Optimal policy for multi-alternative decisions.

Everyday decisions frequently require choosing among multiple alternatives. Yet the optimal policy for such decisions is unknown. Here we derive the normative policy for general multi-alternative decisions. This strategy requires evidence accumulation to nonlinear, time-dependent bounds that trigger choices. A geometric symmetry in those boundaries allows the optimal strategy to be implemented by a simple neural circuit involving normalization with fixed decision bounds and an urgency signal. The model captures several key features of the response of decision-making neurons as well as the increase in reaction time as a function of the number of alternatives, known as Hick's law. In addition, we show that in the presence of divisive normalization and internal variability, our model can account for several so-called 'irrational' behaviors, such as the similarity effect as well as the violation of both the independence of irrelevant alternatives principle and the regularity principle.

Locally embedded presages of global network bursts.

Spontaneous, synchronous bursting of neural population is a widely observed phenomenon in nervous networks, which is considered important for functions and dysfunctions of the brain. However, how the global synchrony across a large number of neurons emerges from an initially nonbursting network state is not fully understood. In this study, we develop a state-space reconstruction method combined with high-resolution recordings of cultured neurons. This method extracts deterministic signatures of upcoming global bursts in "local" dynamics of individual neurons during nonbursting periods. We find that local information within a single-cell time series can compare with or even outperform the global mean-field activity for predicting future global bursts. Moreover, the intercell variability in the burst predictability is found to reflect the network structure realized in the nonbursting periods. These findings suggest that deterministic local dynamics can predict seemingly stochastic global events in self-organized networks, implying the potential applications of the present methodology to detecting locally concentrated early warnings of spontaneous seizure occurrence in the brain.

Task-dependent recurrent dynamics in visual cortex.

The capacity for flexible sensory-action association in animals has been related to context-dependent attractor dynamics outside the sensory cortices. Here, we report a line of evidence that flexibly modulated attractor dynamics during task switching are already present in the higher visual cortex in macaque monkeys. With a nonlinear decoding approach, we can extract the particular aspect of the neural population response that reflects the task-induced emergence of bistable attractor dynamics in a neural population, which could be obscured by standard unsupervised dimensionality reductions such as PCA. The dynamical modulation selectively increases the information relevant to task demands, indicating that such modulation is beneficial for perceptual decisions. A computational model that features nonlinear recurrent interaction among neurons with a task-dependent background input replicates the key properties observed in the experimental data. These results suggest that the context-dependent attractor dynamics involving the sensory cortex can underlie flexible perceptual abilities.

Integrated information and dimensionality in continuous attractor dynamics.

There has been increasing interest in the integrated information theory (IIT) of consciousness, which hypothesizes that consciousness is integrated information within neuronal dynamics. However, the current formulation of IIT poses both practical and theoretical problems when empirically testing the theory by computing integrated information from neuronal signals. For example, measuring integrated information requires observing all the elements in a considered system at the same time, but this is practically very difficult. Here, we propose that some aspects of these problems are resolved by considering the topological dimensionality of shared attractor dynamics as an indicator of integrated information in continuous attractor dynamics. In this formulation, the effects of unobserved nodes on the attractor dynamics can be reconstructed using a technique called delay embedding, which allows us to identify the dimensionality of an embedded attractor from partial observations. We propose that the topological dimensionality represents a critical property of integrated information, as it is invariant to general coordinate transformations. We illustrate this new framework with simple examples and discuss how it fits with recent findings based on neural recordings from awake and anesthetized animals. This topological approach extends the existing notions of IIT to continuous dynamical systems and offers a much-needed framework for testing the theory with experimental data by substantially relaxing the conditions required for evaluating integrated information in real neural systems.

Gradual Development of Visual Texture-Selective Properties Between Macaque Areas V2 and V4.

Complex shape and texture representations are known to be constructed from V1 along the ventral visual pathway through areas V2 and V4, but the underlying mechanism remains elusive. Recent study suggests that, for processing of textures, a collection of higher-order image statistics computed by combining V1-like filter responses serves as possible representations of textures both in V2 and V4. Here, to gain a clue for how these image statistics are processed in the extrastriate visual areas, we compared neuronal responses to textures in V2 and V4 of macaque monkeys. For individual neurons, we adaptively explored their preferred textures from among thousands of naturalistic textures and fitted the obtained responses using a combination of V1-like filter responses and higher-order statistics. We found that, while the selectivity for image statistics was largely comparable between V2 and V4, V4 showed slightly stronger sensitivity to the higher-order statistics than V2. Consistent with that finding, V4 responses were reduced to a greater extent than V2 responses when the monkeys were shown spectrally matched noise images that lacked higher-order statistics. We therefore suggest that there is a gradual development in representation of higher-order features along the ventral visual hierarchy.

Optimal policy for value-based decision-making.

For decades now, normative theories of perceptual decisions, and their implementation as drift diffusion models, have driven and significantly improved our understanding of human and animal behaviour and the underlying neural processes. While similar processes seem to govern value-based decisions, we still lack the theoretical understanding of why this ought to be the case. Here, we show that, similar to perceptual decisions, drift diffusion models implement the optimal strategy for value-based decisions. Such optimal decisions require the models' decision boundaries to collapse over time, and to depend on the a priori knowledge about reward contingencies. Diffusion models only implement the optimal strategy under specific task assumptions, and cease to be optimal once we start relaxing these assumptions, by, for example, using non-linear utility functions. Our findings thus provide the much-needed theory for value-based decisions, explain the apparent similarity to perceptual decisions, and predict conditions under which this similarity should break down.

Population Code Dynamics in Categorical Perception.

Categorical perception is a ubiquitous function in sensory information processing, and is reported to have important influences on the recognition of presented and/or memorized stimuli. However, such complex interactions among categorical perception and other aspects of sensory processing have not been explained well in a unified manner. Here, we propose a recurrent neural network model to process categorical information of stimuli, which approximately realizes a hierarchical Bayesian estimation on stimuli. The model accounts for a wide variety of neurophysiological and cognitive phenomena in a consistent framework. In particular, the reported complexity of categorical effects, including (i) task-dependent modulation of neural response, (ii) clustering of neural population representation, (iii) temporal evolution of perceptual color memory, and (iv) a non-uniform discrimination threshold, are explained as different aspects of a single model. Moreover, we directly examine key model behaviors in the monkey visual cortex by analyzing neural population dynamics during categorization and discrimination of color stimuli. We find that the categorical task causes temporally-evolving biases in the neuronal population representations toward the focal colors, which supports the proposed model. These results suggest that categorical perception can be achieved by recurrent neural dynamics that approximates optimal probabilistic inference in the changing environment.

Untangling Brain-Wide Dynamics in Consciousness by Cross-Embedding.

Brain-wide interactions generating complex neural dynamics are considered crucial for emergent cognitive functions. However, the irreducible nature of nonlinear and high-dimensional dynamical interactions challenges conventional reductionist approaches. We introduce a model-free method, based on embedding theorems in nonlinear state-space reconstruction, that permits a simultaneous characterization of complexity in local dynamics, directed interactions between brain areas, and how the complexity is produced by the interactions. We demonstrate this method in large-scale electrophysiological recordings from awake and anesthetized monkeys. The cross-embedding method captures structured interaction underlying cortex-wide dynamics that may be missed by conventional correlation-based analysis, demonstrating a critical role of time-series analysis in characterizing brain state. The method reveals a consciousness-related hierarchy of cortical areas, where dynamical complexity increases along with cross-area information flow. These findings demonstrate the advantages of the cross-embedding method in deciphering large-scale and heterogeneous neuronal systems, suggesting a crucial contribution by sensory-frontoparietal interactions to the emergence of complex brain dynamics during consciousness.

Saliency-based color accessibility.

Perception of color varies markedly between individuals because of differential expression of photopigments in retinal cones. However, it has been difficult to quantify the individual cognitive variation in colored scene and to predict its complex impacts on the behaviors. We developed a method for quantifying and visualizing information loss and gain resulting from individual differences in spectral sensitivity based on visual salience. We first modeled the visual salience for color-deficient observers, and found that the predicted losses and gains in local image salience derived from normal and color-blind models were correlated with the subjective judgment of image saliency in psychophysical experiments, i.e., saliency loss predicted reduced image preference in color-deficient observers. Moreover,saliency-guided image manipulations sufficiently compensated for individual differences in saliency. This visual saliency approach allows for quantification of information extracted from complex visual scenes and can be used as an image compensation to enhance visual accessibility by color-deficient individuals.

Image statistics underlying natural texture selectivity of neurons in macaque V4.

Our daily visual experiences are inevitably linked to recognizing the rich variety of textures. However, how the brain encodes and differentiates a plethora of natural textures remains poorly understood. Here, we show that many neurons in macaque V4 selectively encode sparse combinations of higher-order image statistics to represent natural textures. We systematically explored neural selectivity in a high-dimensional texture space by combining texture synthesis and efficient-sampling techniques. This yielded parameterized models for individual texture-selective neurons. The models provided parsimonious but powerful predictors for each neuron's preferred textures using a sparse combination of image statistics. As a whole population, the neuronal tuning was distributed in a way suitable for categorizing textures and quantitatively predicts human ability to discriminate textures. Together, we suggest that the collective representation of visual image statistics in V4 plays a key role in organizing the natural texture perception.

Defining statistical perceptions with an empirical Bayesian approach.

Extracting statistical structures (including textures or contrasts) from a natural stimulus is a central challenge in both biological and engineering contexts. This study interprets the process of statistical recognition in terms of hyperparameter estimations and free-energy minimization procedures with an empirical Bayesian approach. This mathematical interpretation resulted in a framework for relating physiological insights in animal sensory systems to the functional properties of recognizing stimulus statistics. We applied the present theoretical framework to two typical models of natural images that are encoded by a population of simulated retinal neurons, and demonstrated that the resulting cognitive performances could be quantified with the Fisher information measure. The current enterprise yielded predictions about the properties of human texture perception, suggesting that the perceptual resolution of image statistics depends on visual field angles, internal noise, and neuronal information processing pathways, such as the magnocellular, parvocellular, and koniocellular systems. Furthermore, the two conceptually similar natural-image models were found to yield qualitatively different predictions, striking a note of warning against confusing the two models when describing a natural image.

Whether dots moving in two directions appear coherent or transparent depends on directional biases induced by surrounding motion.

When two random-dot patterns moving in different directions are superimposed, motion appears coherent or transparent depending on the directional difference. In addition, when a pattern is surrounded by another pattern that is moving, the perceived motion of the central stimulus is biased away from the direction of the surrounding motion. That phenomenon is known as induced motion. How is the perception of motion coherence and transparency modulated by surrounding motion? It was found that two random-dot horizontal motions surrounded by another stimulus in downward motion appeared to move in two oblique directions: left-up and right-up. Consequently, when motion transparency occurs, each of the two motions interacts independently with the induced motion direction. Furthermore, for a central stimulus consisting of two physical motions in left-up and right-up directions, the presence of the surrounding stimulus in a vertical motion modulated the perceptual solution of motion coherence/transparency such that if interactions with an induced motion signal narrow the apparent directional difference between the two central motions, then motion coherence is preferred over motion transparency. Therefore, whether a moving stimulus is perceived as coherent or transparent is determined based on the internal representation of motion directions, which can be altered by spatial interactions between adjacent regions.

Acquisition of nonlinear forward optics in generative models: two-stage "downside-up" learning for occluded vision.

We propose a two-stage learning method which implements occluded visual scene analysis into a generative model, a type of hierarchical neural network with bi-directional synaptic connections. Here, top-down connections simulate forward optics to generate predictions for sensory driven low-level representation, whereas bottom-up connections function to send the prediction error, the difference between the sensory based and the predicted low-level representation, to higher areas. The prediction error is then used to update the high-level representation to obtain better agreement with the visual scene. Although the actual forward optics is highly nonlinear and the accuracy of simulated forward optics is crucial for these types of models, the majority of previous studies have only investigated linear and simplified cases of forward optics. Here we take occluded vision as an example of nonlinear forward optics, where an object in front completely masks out the object behind. We propose a two-staged learning method inspired by the staged development of infant visual capacity. In the primary learning stage, a minimal set of object basis is acquired within a linear generative model using the conventional unsupervised learning scheme. In the secondary learning stage, an auxiliary multi-layer neural network is trained to acquire nonlinear forward optics by supervised learning. The important point is that the high-level representation of the linear generative model serves as the input and the sensory driven low-level representation provides the desired output. Numerical simulations show that occluded visual scene analysis can indeed be implemented by the proposed method. Furthermore, considering the format of input to the multi-layer network and analysis of hidden-layer units leads to the prediction that whole object representation of partially occluded objects, together with complex intermediate representation as a consequence of nonlinear transformation from non-occluded to occluded representation may exist in the low-level visual system of the brain.

Neuronal population decoding explains the change in signal detection sensitivity caused by task-irrelevant perceptual bias.

Spatiotemporal context in sensory stimulus has profound effects on neural responses and perception, and it sometimes affects task difficulty. Recently reported experimental data suggest that human detection sensitivity to motion in a target stimulus can be enhanced by adding a slow surrounding motion in an orthogonal direction, even though the illusory motion component caused by the surround is not relevant to the task. It is not computationally clear how the task-irrelevant component of motion modulates the subject's sensitivity to motion detection. In this study, we investigated the effects of encoding biases on detection performance by modeling the stochastic neural population activities. We modeled two types of modulation on the population activity profiles caused by a contextual stimulus: one type is identical to the activity evoked by a physical change in the stimulus, and the other is expressed more simply in terms of response gain modulation. For both encoding schemes, the motion detection performance of the ideal observer is enhanced by a task-irrelevant, additive motion component, replicating the properties observed for real subjects. The success of these models suggests that human detection sensitivity can be characterized by a noisy neural encoding that limits the resolution of information transmission in the cortical visual processing pathway. On the other hand, analyses of the neuronal contributions to the task predict that the effective cell populations differ between the two encoding schemes, posing a question concerning the decoding schemes that the nervous system used during illusory states.

Discriminating natural image statistics from neuronal population codes.

The power law provides an efficient description of amplitude spectra of natural scenes. Psychophysical studies have shown that the forms of the amplitude spectra are clearly related to human visual performance, indicating that the statistical parameters in natural scenes are represented in the nervous system. However, the underlying neuronal computation that accounts for the perception of the natural image statistics has not been thoroughly studied. We propose a theoretical framework for neuronal encoding and decoding of the image statistics, hypothesizing the elicited population activities of spatial-frequency selective neurons observed in the early visual cortex. The model predicts that frequency-tuned neurons have asymmetric tuning curves as functions of the amplitude spectra falloffs. To investigate the ability of this neural population to encode the statistical parameters of the input images, we analyze the Fisher information of the stochastic population code, relating it to the psychophysically measured human ability to discriminate natural image statistics. The nature of discrimination thresholds suggested by the computational model is consistent with experimental data from previous studies. Of particular interest, a reported qualitative disparity between performance in fovea and parafovea can be explained based on the distributional difference over preferred frequencies of neurons in the current model. The threshold shows a peak at a small falloff parameter when the neuronal preferred spatial frequencies are narrowly distributed, whereas the threshold peak vanishes for a neural population with a more broadly distributed frequency preference. These results demonstrate that the distributional property of neuronal stimulus preference can play a crucial role in linking microscopic neurophysiological phenomena and macroscopic human behaviors.

Opposing effects of contextual surround in human early visual cortex revealed by functional magnetic resonance imaging with continuously modulated visual stimuli.

Spatial context in vision has profound effects on neural responses and perception. Recent animal studies suggest that the effect of surround on a central stimulus can dramatically change its character depending on the contrast of the center stimulus, but such a drastic change has not been demonstrated in the human visual cortex. To examine the dependency of the surround effect on the contrast of the center stimulus, we conducted an functional magnetic resonance imaging experiment by using a low or a high contrast in the center region while the surround contrast was sinusoidally modulated between the two contrasts. We found that the blood oxygen level-dependent response in human V1 corresponding to the center region was differentially modulated by the surround contrast, depending crucially on the center contrast: whereas a suppressive effect was observed in conditions in which the center contrast was high, a facilitative effect was seen in conditions where the center contrast was low.