Dynamic divisive normalization circuits explain and predict change detection in monkey area MT.

Sudden changes in visual scenes often indicate important events for behavior. For their quick and reliable detection, the brain must be capable to process these changes as independently as possible from its current activation state. In motion-selective area MT, neurons respond to instantaneous speed changes with pronounced transients, often far exceeding the expected response as derived from their speed tuning profile. We here show that this complex, non-linear behavior emerges from the combined temporal dynamics of excitation and divisive inhibition, and provide a comprehensive mathematical analysis. A central prediction derived from this investigation is that attention increases the steepness of the transient response irrespective of the activation state prior to a stimulus change, and irrespective of the sign of the change (i.e. irrespective of whether the stimulus is accelerating or decelerating). Extracellular recordings of attention-dependent representation of both speed increments and decrements confirmed this prediction and suggest that improved change detection derives from basic computations in a canonical cortical circuitry.

Visual Stimulus Content in V4 Is Conveyed by Gamma-Rhythmic Information Packages.

Selective visual attention allows the brain to focus on behaviorally relevant information while ignoring irrelevant signals. As a possible mechanism, routing-by-synchronization was proposed: neural populations receiving attended signals align their gamma-rhythmic activity to that of the sending populations, such that incoming spikes arrive at excitability peaks of receiving populations, enhancing signal transfer. Conversely, non-attended signals arrive unaligned to the receiver's oscillation, reducing signal transfer. Therefore, visual signals should be transferred through gamma-rhythmic bursts of information, resulting in a modulation of the stimulus content within the receiving population's activity by its gamma phase and amplitude. To test this prediction, we quantified gamma-phase-dependent stimulus content within neural activity from area V4 of two male macaques performing a visual attention task. For the attended stimulus, we find highest stimulus information content near excitability peaks, an effect that increases with oscillation amplitude, establishing a functional link between selective processing and gamma-activity.SIGNIFICANCE STATEMENT The ability to focus on the behaviorally relevant signals is essential for the brain to cope with the continuous, high-dimensional stream of sensory information it receives. What are the neural mechanisms which allow such selective processing in the visual system? We analyzed data from area V4 and found that the amount of visual signal information content is tightly linked to the phase of local gamma-rhythmic activity, with maximal signal content occurring near peaks of neural excitability. Our investigations provide direct evidence that selective attention relies on rhythmic temporal coordination between visual areas, and establish novel methods for pinpointing pulsed transmission schemes in neural data.

Constrained inference in sparse coding reproduces contextual effects and predicts laminar neural dynamics.

When probed with complex stimuli that extend beyond their classical receptive field, neurons in primary visual cortex display complex and non-linear response characteristics. Sparse coding models reproduce some of the observed contextual effects, but still fail to provide a satisfactory explanation in terms of realistic neural structures and cortical mechanisms, since the connection scheme they propose consists only of interactions among neurons with overlapping input fields. Here we propose an extended generative model for visual scenes that includes spatial dependencies among different features. We derive a neurophysiologically realistic inference scheme under the constraint that neurons have direct access only to local image information. The scheme can be interpreted as a network in primary visual cortex where two neural populations are organized in different layers within orientation hypercolumns that are connected by local, short-range and long-range recurrent interactions. When trained with natural images, the model predicts a connectivity structure linking neurons with similar orientation preferences matching the typical patterns found for long-ranging horizontal axons and feedback projections in visual cortex. Subjected to contextual stimuli typically used in empirical studies, our model replicates several hallmark effects of contextual processing and predicts characteristic differences for surround modulation between the two model populations. In summary, our model provides a novel framework for contextual processing in the visual system proposing a well-defined functional role for horizontal axons and feedback projections.

Causally Investigating Cortical Dynamics and Signal Processing by Targeting Natural System Attractors With Precisely Timed (Electrical) Stimulation.

Electrical stimulation is a promising tool for interacting with neuronal dynamics to identify neural mechanisms that underlie cognitive function. Since effects of a single short stimulation pulse typically vary greatly and depend on the current network state, many experimental paradigms have rather resorted to continuous or periodic stimulation in order to establish and maintain a desired effect. However, such an approach explicitly leads to forced and "unnatural" brain activity. Further, continuous stimulation can make it hard to parse the recorded activity and separate neural signal from stimulation artifacts. In this study we propose an alternate strategy: by monitoring a system in realtime, we use the existing preferred states or attractors of the network and apply short and precise pulses in order to switch between those states. When pushed into one of its attractors, one can use the natural tendency of the system to remain in such a state to prolong the effect of a stimulation pulse, opening a larger window of opportunity to observe the consequences on cognitive processing. To elaborate on this idea, we consider flexible information routing in the visual cortex as a prototypical example. When processing a stimulus, neural populations in the visual cortex have been found to engage in synchronized gamma activity. In this context, selective signal routing is achieved by changing the relative phase between oscillatory activity in sending and receiving populations (communication through coherence, CTC). In order to explore how perturbations interact with CTC, we investigate a network of interneuronal gamma (ING) oscillators composed of integrate-and-fire neurons exhibiting similar synchronization and signal routing phenomena. We develop a closed-loop stimulation paradigm based on the phase-response characteristics of the network and demonstrate its ability to establish desired synchronization states. By measuring information content throughout the model, we evaluate the effect of signal contamination caused by the stimulation in relation to the magnitude of the injected pulses and intrinsic noise in the system. Finally, we demonstrate that, up to a critical noise level, precisely timed perturbations can be used to artificially induce the effect of attention by selectively routing visual signals to higher cortical areas.

Attention Selectively Gates Afferent Signal Transmission to Area V4.

Selective attention allows focusing on only part of the incoming sensory information. Neurons in the extrastriate visual cortex reflect such selective processing when different stimuli are simultaneously present in their large receptive fields. Their spiking response then resembles the response to the attended stimulus when presented in isolation. Unclear is where in the neuronal pathway attention intervenes to achieve such selective signal routing and processing. To investigate this question, we tagged two equivalent visual stimuli by independent broadband luminance noise and used the spectral coherence of these behaviorally irrelevant signals with the field potential of a local neuronal population in male macaque monkeys' area V4 as a measure for their respective causal influences. This new experimental paradigm revealed that signal transmission was considerably weaker for the not-attended stimulus. Furthermore, our results show that attention does not need to modulate responses in the input populations sending signals to V4 to selectively represent a stimulus, nor do they suggest a change of the V4 neurons' output gain depending on their feature similarity with the stimuli. Our results rather imply that selective attention uses a gating mechanism comprising the synaptic "inputs" that transmit signals from upstream areas into the V4 neurons. A minimal model implementing attention-dependent routing by gamma-band synchrony replicated the attentional gating effect and the signals' spectral transfer characteristics. It supports the proposal that selective interareal gamma-band synchrony subserves signal routing and explains our experimental finding that attention selectively gates signals already at the level of afferent synaptic input.SIGNIFICANCE STATEMENT Depending on the behavioral context, the brain needs to channel the flow of information through its networks of massively interconnected neurons. We designed an experiment that allows to causally assess routing of information originating from an attended object. We found that attention "gates" signals at the interplay between afferent fibers and the local neurons. A minimal model demonstrated that coherent gamma-rhythmic activity (∼60 Hz) between local neurons and their afferent-providing input neurons can realize the gating. Importantly, the attended signals did not need to be amplified already in an earlier processing stage, nor did they get amplified by a simple output response modulation. The method provides a useful tool to study mechanisms of dynamic network configuration underlying cognitive processes.

Contour Integration in Dynamic Scenes: Impaired Detection Performance in Extended Presentations.

Since scenes in nature are highly dynamic, perception requires an on-going and robust integration of local information into global representations. In vision, contour integration (CI) is one of these tasks, and it is performed by our brain in a seemingly effortless manner. Following the rule of good continuation, oriented line segments are linked into contour percepts, thus supporting important visual computations such as the detection of object boundaries. This process has been studied almost exclusively using static stimuli, raising the question of whether the observed robustness and "pop-out" quality of CI carries over to dynamic scenes. We investigate contour detection in dynamic stimuli where targets appear at random times by Gabor elements aligning themselves to form contours. In briefly presented displays (230 ms), a situation comparable to classical paradigms in CI, performance is about 87%. Surprisingly, we find that detection performance decreases to 67% in extended presentations (about 1.9-3.8 s) for the same target stimuli. In order to observe the same reduction with briefly presented stimuli, presentation time has to be drastically decreased to intervals as short as 50 ms. Cueing a specific contour position or shape helps in partially compensating this deterioration, and only in extended presentations combining a location and a shape cue was more efficient than providing a single cue. Our findings challenge the notion of CI as a mainly stimulus-driven process leading to pop-out percepts, indicating that top-down processes play a much larger role in supporting fundamental integration processes in dynamic scenes than previously thought.

Contextual Interactions in Grating Plaid Configurations Are Explained by Natural Image Statistics and Neural Modeling.

Processing natural scenes requires the visual system to integrate local features into global object descriptions. To achieve coherent representations, the human brain uses statistical dependencies to guide weighting of local feature conjunctions. Pairwise interactions among feature detectors in early visual areas may form the early substrate of these local feature bindings. To investigate local interaction structures in visual cortex, we combined psychophysical experiments with computational modeling and natural scene analysis. We first measured contrast thresholds for 2 × 2 grating patch arrangements (plaids), which differed in spatial frequency composition (low, high, or mixed), number of grating patch co-alignments (0, 1, or 2), and inter-patch distances (1° and 2° of visual angle). Contrast thresholds for the different configurations were compared to the prediction of probability summation (PS) among detector families tuned to the four retinal positions. For 1° distance the thresholds for all configurations were larger than predicted by PS, indicating inhibitory interactions. For 2° distance, thresholds were significantly lower compared to PS when the plaids were homogeneous in spatial frequency and orientation, but not when spatial frequencies were mixed or there was at least one misalignment. Next, we constructed a neural population model with horizontal laminar structure, which reproduced the detection thresholds after adaptation of connection weights. Consistent with prior work, contextual interactions were medium-range inhibition and long-range, orientation-specific excitation. However, inclusion of orientation-specific, inhibitory interactions between populations with different spatial frequency preferences were crucial for explaining detection thresholds. Finally, for all plaid configurations we computed their likelihood of occurrence in natural images. The likelihoods turned out to be inversely related to the detection thresholds obtained at larger inter-patch distances. However, likelihoods were almost independent of inter-patch distance, implying that natural image statistics could not explain the crowding-like results at short distances. This failure of natural image statistics to resolve the patch distance modulation of plaid visibility remains a challenge to the approach.

A model for attentional information routing through coherence predicts biased competition and multistable perception.

Selective attention allows to focus on relevant information and to ignore distracting features of a visual scene. These principles of information processing are reflected in response properties of neurons in visual area V4: if a neuron is presented with two stimuli in its receptive field, and one is attended, it responds as if the nonattended stimulus was absent (biased competition). In addition, when the luminance of the two stimuli is temporally and independently varied, local field potentials are correlated with the modulation of the attended stimulus and not, or much less, correlated with the nonattended stimulus (information routing). To explain these results in one coherent framework, we present a two-layer spiking cortical network model with distance-dependent lateral connectivity and converging feed-forward connections. With oscillations arising inherently from the network structure, our model reproduces both experimental observations. Hereby, lateral interactions and shifts of relative phases between sending and receiving layers (communication through coherence) are identified as the main mechanisms underlying both biased competition as well as selective routing. Exploring the parameter space, we show that the effects are robust and prevalent over a broad range of parameters. In addition, we identify the strength of lateral inhibition in the first model layer as crucial for determining the working regime of the system: increasing lateral inhibition allows a transition from a network configuration with mixed representations to one with bistable representations of the competing stimuli. The latter is discussed as a possible neural correlate of multistable perception phenomena such as binocular rivalry.

Marginally subcritical dynamics explain enhanced stimulus discriminability under attention.

Recent experimental and theoretical work has established the hypothesis that cortical neurons operate close to a critical state which describes a phase transition from chaotic to ordered dynamics. Critical dynamics are suggested to optimize several aspects of neuronal information processing. However, although critical dynamics have been demonstrated in recordings of spontaneously active cortical neurons, little is known about how these dynamics are affected by task-dependent changes in neuronal activity when the cortex is engaged in stimulus processing. Here we explore this question in the context of cortical information processing modulated by selective visual attention. In particular, we focus on recent findings that local field potentials (LFPs) in macaque area V4 demonstrate an increase in γ-band synchrony and a simultaneous enhancement of object representation with attention. We reproduce these results using a model of integrate-and-fire neurons where attention increases synchrony by enhancing the efficacy of recurrent interactions. In the phase space spanned by excitatory and inhibitory coupling strengths, we identify critical points and regions of enhanced discriminability. Furthermore, we quantify encoding capacity using information entropy. We find a rapid enhancement of stimulus discriminability with the emergence of synchrony in the network. Strikingly, only a narrow region in the phase space, at the transition from subcritical to supercritical dynamics, supports the experimentally observed discriminability increase. At the supercritical border of this transition region, information entropy decreases drastically as synchrony sets in. At the subcritical border, entropy is maximized under the assumption of a coarse observation scale. Our results suggest that cortical networks operate at such near-critical states, allowing minimal attentional modulations of network excitability to substantially augment stimulus representation in the LFPs.

The role of eye movements in a contour detection task.

Vision combines local feature integration with active viewing processes, such as eye movements, to perceive complex visual scenes. However, it is still unclear how these processes interact and support each other. Here, we investigated how the dynamics of saccadic eye movements interact with contour integration, focusing on situations in which contours are difficult to find or even absent. We recorded observers' eye movements while they searched for a contour embedded in a background of randomly oriented elements. Task difficulty was manipulated by varying the contour's path angle. An association field model of contour integration was employed to predict potential saccade targets by identifying stimulus locations with high contour salience. We found that the number and duration of fixations increased with the increasing path angle of the contour. In addition, fixation duration increased over the course of a trial, and the time course of saccade amplitude depended on the percept of observers. Model fitting revealed that saccades fully compensate for the reduced saliency of peripheral contour targets. Importantly, our model predicted fixation locations to a considerable degree, indicating that observers fixated collinear elements. These results show that contour integration actively guides eye movements and determines their spatial and temporal parameters.

Dynamics of dual prism adaptation: relating novel experimental results to a minimalistic neural model.

In everyday life, humans interact with a dynamic environment often requiring rapid adaptation of visual perception and motor control. In particular, new visuo-motor mappings must be learned while old skills have to be kept, such that after adaptation, subjects may be able to quickly change between two different modes of generating movements ('dual-adaptation'). A fundamental question is how the adaptation schedule determines the acquisition speed of new skills. Given a fixed number of movements in two different environments, will dual-adaptation be faster if switches ('phase changes') between the environments occur more frequently? We investigated the dynamics of dual-adaptation under different training schedules in a virtual pointing experiment. Surprisingly, we found that acquisition speed of dual visuo-motor mappings in a pointing task is largely independent of the number of phase changes. Next, we studied the neuronal mechanisms underlying this result and other key phenomena of dual-adaptation by relating model simulations to experimental data. We propose a simple and yet biologically plausible neural model consisting of a spatial mapping from an input layer to a pointing angle which is subjected to a global gain modulation. Adaptation is performed by reinforcement learning on the model parameters. Despite its simplicity, the model provides a unifying account for a broad range of experimental data: It quantitatively reproduced the learning rates in dual-adaptation experiments for both direct effect, i.e. adaptation to prisms, and aftereffect, i.e. behavior after removal of prisms, and their independence on the number of phase changes. Several other phenomena, e.g. initial pointing errors that are far smaller than the induced optical shift, were also captured. Moreover, the underlying mechanisms, a local adaptation of a spatial mapping and a global adaptation of a gain factor, explained asymmetric spatial transfer and generalization of prism adaptation, as observed in other experiments.

Toward high performance, weakly invasive brain computer interfaces using selective visual attention.

Brain-computer interfaces have been proposed as a solution for paralyzed persons to communicate and interact with their environment. However, the neural signals used for controlling such prostheses are often noisy and unreliable, resulting in a low performance of real-world applications. Here we propose neural signatures of selective visual attention in epidural recordings as a fast, reliable, and high-performance control signal for brain prostheses. We recorded epidural field potentials with chronically implanted electrode arrays from two macaque monkeys engaged in a shape-tracking task. For single trials, we classified the direction of attention to one of two visual stimuli based on spectral amplitude, coherence, and phase difference in time windows fixed relative to stimulus onset. Classification performances reached up to 99.9%, and the information about attentional states could be transferred at rates exceeding 580 bits/min. Good classification can already be achieved in time windows as short as 200 ms. The classification performance changed dynamically over the trial and modulated with the task's varying demands for attention. For all three signal features, the information about the direction of attention was contained in the γ-band. The most informative feature was spectral amplitude. Together, these findings establish a novel paradigm for constructing brain prostheses as, for example, virtual spelling boards, promising a major gain in performance and robustness for human brain-computer interfaces.

Optimality of human contour integration.

For processing and segmenting visual scenes, the brain is required to combine a multitude of features and sensory channels. It is neither known if these complex tasks involve optimal integration of information, nor according to which objectives computations might be performed. Here, we investigate if optimal inference can explain contour integration in human subjects. We performed experiments where observers detected contours of curvilinearly aligned edge configurations embedded into randomly oriented distractors. The key feature of our framework is to use a generative process for creating the contours, for which it is possible to derive a class of ideal detection models. This allowed us to compare human detection for contours with different statistical properties to the corresponding ideal detection models for the same stimuli. We then subjected the detection models to realistic constraints and required them to reproduce human decisions for every stimulus as well as possible. By independently varying the four model parameters, we identify a single detection model which quantitatively captures all correlations of human decision behaviour for more than 2000 stimuli from 42 contour ensembles with greatly varying statistical properties. This model reveals specific interactions between edges closely matching independent findings from physiology and psychophysics. These interactions imply a statistics of contours for which edge stimuli are indeed optimally integrated by the visual system, with the objective of inferring the presence of contours in cluttered scenes. The recurrent algorithm of our model makes testable predictions about the temporal dynamics of neuronal populations engaged in contour integration, and it suggests a strong directionality of the underlying functional anatomy.

Perceptual inference predicts contextual modulations of sensory responses.

Sensory receptive fields (RFs) vary as a function of stimulus properties and measurement methods. Previous stimuli or surrounding stimuli facilitate, suppress, or change the selectivity of sensory neurons' responses. Here, we propose that these spatiotemporal contextual dependencies are signatures of efficient perceptual inference and can be explained by a single neural mechanism, input targeted divisive inhibition. To respond both selectively and reliably, sensory neurons should behave as active predictors rather than passive filters. In particular, they should remove input they can predict ("explain away") from the synaptic inputs to all other neurons. This implies that RFs are constantly and dynamically reshaped by the spatial and temporal context, while the true selectivity of sensory neurons resides in their "predictive field." This approach motivates a reinvestigation of sensory representations and particularly the role and specificity of surround suppression and adaptation in sensory areas.

Attention improves object representation in visual cortical field potentials.

Selective attention improves perception and modulates neuronal responses, but how attention-dependent changes of cortical activity improve the processing of attended objects is an open question. Changes in total signal strength or enhancements in signal-to-noise ratio have been proposed as putative mechanisms. However, it is still not clear whether, and to what extent, these processes contribute to the large perceptual improvements. We studied the ability to discriminate states of activity in visual cortex evoked by differently shaped objects depending on selective attention in monkeys. We found that gamma-band activity from V4 and V1 contains a high amount of information about stimulus shape, which increases for V4 recordings considerably with attention in successful trials, but not in case of behavioral errors. This effect resulted from enhanced differences between the stimulus-specific distributions of power spectral amplitudes. It could be explained neither by enhancements of signal-to-noise ratios, nor by changes in total signal power. Instead our results indicate that attention causes underlying cortical network states to become more distinct for different stimuli, providing a new neurophysiological explanation for improvements of behavioral performance by attention. The absence of the enhancement in discriminability in trials with behavioral errors demonstrates the relevance of this novel neural mechanism for perception.

Modeling spatial and temporal aspects of visual backward masking.

Visual backward masking is a versatile tool for understanding principles and limitations of visual information processing in the human brain. However, the mechanisms underlying masking are still poorly understood. In the current contribution, the authors show that a structurally simple mathematical model can explain many spatial and temporal effects in visual masking, such as spatial layout effects on pattern masking and B-type masking. Specifically, the authors show that lateral excitation and inhibition on different length scales, in combination with the typical time scales, are capable of producing a rich, dynamic behavior that explains this multitude of masking phenomena in a single, biophysically motivated model.

Visual backward masking: Modeling spatial and temporal aspects.

In modeling visual backward masking, the focus has been on temporal effects. More specifically, an explanation has been sought as to why strongest masking can occur when the mask is delayed with respect to the target. Although interesting effects of the spatial layout of the mask have been found, only a few attempts have been made to model these phenomena. Here, we elaborate a structurally simple model which employs lateral excitation and inhibition together with different neural time scales to explain many spatial and temporal aspects of backward masking. We argue that for better understanding of visual masking, it is vitally important to consider the interplay of spatial and temporal factors together in one single model.

Efficient computation based on stochastic spikes.

The speed and reliability of mammalian perception indicate that cortical computations can rely on very few action potentials per involved neuron. Together with the stochasticity of single-spike events in cortex, this appears to imply that large populations of redundant neurons are needed for rapid computations with action potentials. Here we demonstrate that very fast and precise computations can be realized also in small networks of stochastically spiking neurons. We present a generative network model for which we derive biologically plausible algorithms that perform spike-by-spike updates of the neuron's internal states and adaptation of its synaptic weights from maximizing the likelihood of the observed spike patterns. Paradigmatic computational tasks demonstrate the online performance and learning efficiency of our framework. The potential relevance of our approach as a model for cortical computation is discussed.

Self-organized critical noise amplification in human closed loop control.

When humans perform closed loop control tasks like in upright standing or while balancing a stick, their behavior exhibits non-Gaussian fluctuations with long-tailed distributions. The origin of these fluctuations is not known. Here, we investigate if they are caused by self-organized critical noise amplification which emerges in control systems when an unstable dynamics becomes stabilized by an adaptive controller that has finite memory. Starting from this theory, we formulate a realistic model of adaptive closed loop control by including constraints on memory and delays. To test this model, we performed psychophysical experiments where humans balanced an unstable target on a screen. It turned out that the model reproduces the long tails of the distributions together with other characteristic features of the human control dynamics. Fine-tuning the model to match the experimental dynamics identifies parameters characterizing a subject's control system which can be independently tested. Our results suggest that the nervous system involved in closed loop motor control nearly optimally estimates system parameters on-line from very short epochs of past observations.