Population coding with motion energy filters: the impact of correlations.

The codes obtained from the responses of large populations of neurons are known as population codes. Several studies have shown that the amount of information conveyed by such codes, and the format of this information, is highly dependent on the pattern of correlations. However, very little is known about the impact of response correlations (as found in actual cortical circuits) on neural coding. To address this problem, we investigated the properties of population codes obtained from motion energy filters, which provide one of the best models for motion selectivity in early visual areas. It is therefore likely that the correlations that arise among energy filters also arise among motion-selective neurons. We adopted an ideal observer approach to analyze filter responses to three sets of images: noisy sine gratings, random dots kinematograms, and images of natural scenes. We report that in our model, the structure of the population code varies with the type of image. We also show that for all sets of images, correlations convey a large fraction of the information: 40% to 90% of the total information. Moreover, ignoring those correlations when decoding leads to considerable information loss-from 50% to 93%, depending on the image type. Finally we show that it is important to consider a large population of motion energy filters in order to see the impact of correlations. Study of pairs of neurons, as is often done experimentally, can underestimate the effect of correlations.

Probabilistic population codes and the exponential family of distributions.

Many experiments have shown that human behavior is nearly Bayes optimal in a variety of tasks. This implies that neural activity is capable of representing both the value and uncertainty of a stimulus, if not an entire probability distribution, and can also combine such representations in an optimal manner. Moreover, this computation can be performed optimally despite the fact that observed neural activity is highly variable (noisy) on a trial-by-trial basis. Here, we argue that this observed variability is actually expected in a neural system which represents uncertainty. Specifically, we note that Bayes' rule implies that a variable pattern of activity provides a natural representation of a probability distribution, and that the specific form of neural variability can be structured so that optimal inference can be executed using simple operations available to neural circuits.

Decoding M1 neurons during multiple finger movements.

We tested several techniques for decoding the activity of primary motor cortex (M1) neurons during movements of single fingers or pairs of fingers. We report that single finger movements can be decoded with >99% accuracy using as few as 30 neurons randomly selected from populations of task-related neurons recorded from the M1 hand representation. This number was reduced to 20 neurons or less when the neurons were not picked randomly but selected on the basis of their information content. We extended techniques for decoding single finger movements to the problem of decoding the simultaneous movement of two fingers. Movements of pairs of fingers were decoded with 90.9% accuracy from 100 neurons. The techniques we used to obtain these results can be applied, not only to movements of single fingers and pairs of fingers as reported here, but also to movements of arbitrary combinations of fingers. The remarkably small number of neurons needed to decode a relatively large repertoire of movements involving either one or two effectors is encouraging for the development of neural prosthetics that will control hand movements.

MSTd neuronal basis functions for the population encoding of heading direction.

Basis functions have been extensively used in models of neural computation because they can be combined linearly to approximate any nonlinear functions of the encoded variables. We investigated whether dorsal medial superior temporal (MSTd) area neurons use basis functions to simultaneously encode heading direction, eye position, and the velocity of ocular pursuit. Using optimal linear estimators, we first show that the head-centered and eye-centered position of a focus of expansion (FOE) in optic flow, pursuit direction, and eye position can all be estimated from the single-trial responses of 144 MSTd neurons with an average accuracy of 2-3 degrees, a value consistent with the discrimination thresholds measured in humans and monkeys. We then examined the format of the neural code for the head-centered position of the FOE, eye position, and pursuit direction. The basis function hypothesis predicts that a large majority of cells in MSTd should encode two or more signals simultaneously and combine these signals nonlinearly. Our analysis shows that 95% of the neurons encode two or more signals, whereas 76% code all three signals. Of the 95% of cells encoding two or more signals, 90% show nonlinear interactions between the encoded variables. These findings support the notion that MSTd may use basis functions to represent the FOE in optic flow, eye position, and pursuit.

Simulating a lesion in a basis function model of spatial representations: comparison with hemineglect.

The basis function theory of spatial representations explains how neurons in the parietal cortex can perform nonlinear transformations from sensory to motor coordinates. The authors present computer simulations showing that unilateral parietal lesions leading to a neuronal gradient in basis function maps can account for the behavior of patients with hemineglect, including (a) neglect in line cancellation and line bisection experiments; (b) neglect in multiple frames of reference simultaneously; (c) relative neglect, a form of what is sometime called object-centered neglect; and (d) neglect without optic ataxia. Contralateral neglect arises in the model because the lesion produces an imbalance in the salience of stimuli that is modulated by the orientation of the body in space. These results strongly support the basis function theory for spatial representations in humans and provide a computational model of hemineglect at the single-cell level.

Efficient computation and cue integration with noisy population codes.

The brain represents sensory and motor variables through the activity of large populations of neurons. It is not understood how the nervous system computes with these population codes, given that individual neurons are noisy and thus unreliable. We focus here on two general types of computation, function approximation and cue integration, as these are powerful enough to handle a range of tasks, including sensorimotor transformations, feature extraction in sensory systems and multisensory integration. We demonstrate that a particular class of neural networks, basis function networks with multidimensional attractors, can perform both types of computation optimally with noisy neurons. Moreover, neurons in the intermediate layers of our model show response properties similar to those observed in several multimodal cortical areas. Thus, basis function networks with multidimensional attractors may be used by the brain to compute efficiently with population codes.

Computational approaches to sensorimotor transformations.

Behaviors such as sensing an object and then moving your eyes or your hand toward it require that sensory information be used to help generate a motor command, a process known as a sensorimotor transformation. Here we review models of sensorimotor transformations that use a flexible intermediate representation that relies on basis functions. The use of basis functions as an intermediate is borrowed from the theory of nonlinear function approximation. We show that this approach provides a unifying insight into the neural basis of three crucial aspects of sensorimotor transformations, namely, computation, learning and short-term memory. This mathematical formalism is consistent with the responses of cortical neurons and provides a fresh perspective on the issue of frames of reference in spatial representations.

Information processing with population codes.

Information is encoded in the brain by populations or clusters of cells, rather than by single cells. This encoding strategy is known as population coding. Here we review the standard use of population codes for encoding and decoding information, and consider how population codes can be used to support neural computations such as noise removal and nonlinear mapping. More radical ideas about how population codes may directly represent information about stimulus uncertainty are also discussed.

Relating unilateral neglect to the neural coding of space.

Neuropsychological findings on the human neglect syndrome after parietal damage may relate to the physiological properties of single cells that have been studied in monkey parietal cortex and in related brain areas. Human neglect may reflect partial loss or dysfunction of similar cell populations, producing a pathological gradient in the numbers of cells representing particular lateral positions in space, for particular functions. This can explain the graded deficits seen in patients. We relate the patient deficits to cellular properties for several current issues: spatial frames-of-reference; multimodal integration; effective treatments for neglect; motor components to parietal function; and residual unconscious processing. A neural perspective may resolve traditional debates in the neglect literature and outline directions for future research.

Reading population codes: a neural implementation of ideal observers.

Many sensory and motor variables are encoded in the nervous system by the activities of large populations of neurons with bell-shaped tuning curves. Extracting information from these population codes is difficult because of the noise inherent in neuronal responses. In most cases of interest, maximum likelihood (ML) is the best read-out method and would be used by an ideal observer. Using simulations and analysis, we show that a close approximation to ML can be implemented in a biologically plausible model of cortical circuitry. Our results apply to a wide range of nonlinear activation functions, suggesting that cortical areas may, in general, function as ideal observers of activity in preceding areas.

Narrow versus wide tuning curves: What's best for a population code?

Neurophysiologists are often faced with the problem of evaluating the quality of a code for a sensory or motor variable, either to relate it to the performance of the animal in a simple discrimination task or to compare the codes at various stages along the neuronal pathway. One common belief that has emerged from such studies is that sharpening of tuning curves improves the quality of the code, although only to a certain point; sharpening beyond that is believed to be harmful. We show that this belief relies on either problematic technical analysis or improper assumptions about the noise. We conclude that one cannot tell, in the general case, whether narrow tuning curves are better than wide ones; the answer depends critically on the covariance of the noise. The same conclusion applies to other manipulations of the tuning curve profiles such as gain increase.

Eye position encoding in the macaque posterior parietal cortex.

In two previous studies, we had demonstrated the influence of eye position on neuronal discharges in the middle temporal area, medial superior temporal area, lateral intraparietal area and area 7A of the awake monkey (Bremmer et al., 1997a,b). Eye position effects also have been found in visual cortical areas V3A and V6 and even in the premotor cortex and the supplementary eye field. These effects are generally discussed in light of a coordinate transformation of visual signals into a non-retinocentric frame of reference. Neural network studies dealing with the eye position effect succeeded in constructing such non-retinocentric representations by using model neurones whose response characteristics resembled those of 'real' neurones. However, to our knowledge, response properties of real neurones never acted as input into these neural networks. In the present study, we thus investigated whether, theoretically, eye position could be estimated from the population discharge of the (previously) recorded neurones and, if so, we intended to develop an encoding algorithm for the position of the eyes in the orbit. The optimal linear estimator proved the capability of the ensemble activity for determining correctly eye position. We then developed the so-called subpopulation encoding of eye position. This algorithm is based on the partition of the ensemble of neurones into two pairs of subpopulations. Eye position is represented by the differences of activity levels within each pair of subpopulations. Considering this result, encoding of the location of an object relative to the head could easily be accomplished by combining eye position information with the intrinsic knowledge about the retinal location of a visual stimulus. Taken together, these results show that throughout the monkey's visual cortical system information is available which can be used in a fairly simple manner in order to generate a non-retinocentric representation of visual information.

Reliable disparity estimation through selective integration.

A network model of disparity estimation was developed based on disparity-selective neurons, such as those found in the early stages of processing in the visual cortex. The model accurately estimated multiple disparities in regions, which may be caused by transparency or occlusion. The selective integration of reliable local estimates enabled the network to generate accurate disparity estimates on normal and transparent random-dot stereograms. The model was consistent with human psychophysical results on the effects of spatial-frequency filtering on disparity sensitivity. The responses of neurons in macaque area V2 to random-dot stereograms are consistent with the prediction of the model that a subset of neurons responsible for disparity selection should be sensitive to disparity gradients.

Interactions between natural and electrically evoked saccades. III. Is the nonstationarity the result of an integrator not instantaneously reset?

In the monkey, fixed-vector saccades evoked by superior colliculus (SC) stimulation when the animal fixates can be dramatically modified if the stimulation is applied during or immediately after an initial natural saccade. The vector is then deviated in the direction opposite to the displacement just accomplished as if it were compensating for part of the preceding trajectory. Recently, it was suggested that the amplitude of the compensatory deviation is related to the amplitude of the initial saccade linearly, and that the ratio between the two decreases exponentially as stimulation is applied later. These two findings (spatial linearity and temporal nonstationarity) were invoked as evidence for the noninstantaneous resetting of a feedback integrator. Such an integrator is included in most models of saccade generation for the specific purpose of terminating a saccade when it has reached its intended goal. However, the hypothesis of a feedback integrator in the process of being reset implies that the exponential decay of the compensatory deviation is temporally linked to the end of the initial saccade. We analyzed the time course of this decay in stimulation experiments performed at 24 SC sites in two monkeys. The results show that if the start of the exponential decay of compensation is assumed to be linked to the end of the initial saccade, then the relation between the amount of compensatory deviation and the amplitude of the initial saccade is not linear. On the other hand, it is possible to show a linear relation if the measurements of compensatory deviation are made in terms of delay of stimulation from the saccade beginning. We conclude that stimulating the SC just after a visually guided saccade does not seem to test the properties of a feedback integrator. Whether such an integrator is or is not resettable is not likely to be decided by this approach. Conversely, as the nonstationarity of compensation is linked to the beginning of the saccade, the nonstationarity seems to represent a property of an event occurring at saccade onset. We suggest that this event, close to the input of the oculomotor apparatus, is the summation of the visual signal with a damped signal of eye position or displacement.

Statistically efficient estimation using population coding.

Coarse codes are widely used throughout the brain to encode sensory and motor variables. Methods designed to interpret these codes, such as population vector analysis, are either inefficient (the variance of the estimate is much larger than the smallest possible variance) or biologically implausible, like maximum likelihood. Moreover, these methods attempt to compute a scalar or vector estimate of the encoded variable. Neurons are faced with a similar estimation problem. They must read out the responses of the presynaptic neurons, but, by contrast, they typically encode the variable with a further population code rather than as a scalar. We show how a nonlinear recurrent network can be used to perform estimation in a near-optimal way while keeping the estimate in a coarse code format. This work suggests that lateral connections in the cortex may be involved in cleaning up uncorrelated noise among neurons representing similar variables.

Probabilistic interpretation of population codes.

We present a general encoding-decoding framework for interpreting the activity of a population of units. A standard population code interpretation method, the Poisson model, starts from a description as to how a single value of an underlying quantity can generate the activities of each unit in the population. In casting it in the encoding-decoding framework, we find that this model is too restrictive to describe fully the activities of units in population codes in higher processing areas, such as the medial temporal area. Under a more powerful model, the population activity can convey information not only about a single value of some quantity but also about its whole distribution, including its variance, and perhaps even the certainty the system has in the actual presence in the world of the entity generating this quantity. We propose a novel method for forming such probabilistic interpretations of population codes and compare it to the existing method.

A new view of hemineglect based on the response properties of parietal neurones.

Lesion studies of the parietal cortex have led to a wide range of conclusions regarding the coordinate reference frame in which hemineglect is expressed. A model of spatial representation in the parietal cortex has recently been developed in which the position of an object is not encoded in a particular frame of reference, but instead involves neurones computing basis functions of sensory inputs. In this type of representation, a nonlinear sensorimotor transformation of an object is represented in a population of units having the response properties of neurones that are observed in the parietal cortex. A simulated lesion in a basis-function representation was found to replicate three of the most important aspects of hemineglect: (i) the model behaved like parietal patients in line-cancellation and line-bisection experiments; (ii) the deficit affected multiple frames of reference; and (iii) the deficit could be object-centred. These results support the basis-function hypothesis for spatial representations and provide a testable computational theory of hemineglect at the level of single cells.

Perceived geometrical relationships affected by eye-movement signals.

To determine the location of visual objects relative to the observer, the visual system must take account not only of the location of the stimulus on the retina, but also of the direction of gaze. In contrast, the perceived spatial relationship between visual stimuli is normally assumed to depend on retinal information alone, and not to require information about eye position. We now show, however, that the perceived alignment of three dots-tested by a vernier alignment task-is systematically altered in the period immediately preceding a saccade. Thus, information about eye position can modify not only the perceived relationship of the entire retinal image to the observer, but also the relations between elements within the image. The processing of relative position and of egocentric (observer-centred) position may therefore be less distinct than previously believed.