Variance as a signature of neural computations during decision making.

Traditionally, insights into neural computation have been furnished by averaged firing rates from many stimulus repetitions or trials. We pursue an analysis of neural response variance to unveil neural computations that cannot be discerned from measures of average firing rate. We analyzed single-neuron recordings from the lateral intraparietal area (LIP), during a perceptual decision-making task. Spike count variance was divided into two components using the law of total variance for doubly stochastic processes: (1) variance of counts that would be produced by a stochastic point process with a given rate, and loosely (2) the variance of the rates that would produce those counts (i.e., "conditional expectation"). The variance and correlation of the conditional expectation exposed several neural mechanisms: mixtures of firing rate states preceding the decision, accumulation of stochastic "evidence" during decision formation, and a stereotyped response at decision end. These analyses help to differentiate among several alternative decision-making models.

Limits to the temporal fidelity of cortical spike rate signals.

The cerebral cortex processes information primarily through changes in the spike rates of neurons within local ensembles. To evaluate how reliably the average spike rate of a group of cortical neurons can represent a time-varying signal, we simulated an ensemble with realistic spike discharge behavior. We found that weak interneuronal correlation, or synchrony, allows the variability in spike rates of individual neurons to compromise the ensemble representation of time-varying signals. Brief cycles of sinusoidal modulation at frequencies above 115 Hz could not be represented by an ensemble of hundreds of neurons whose interneuronal correlation mimics that of the visual cortex. The spike variability and correlation assumed in our simulations are likely to apply to many areas of cortex and therefore may constrain the fidelity of neural computations underlying higher brain function.

Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey.

We recorded the activity of single neurons in the posterior parietal cortex (area LIP) of two rhesus monkeys while they discriminated the direction of motion in random-dot visual stimuli. The visual task was similar to a motion discrimination task that has been used in previous investigations of motion-sensitive regions of the extrastriate cortex. The monkeys were trained to decide whether the direction of motion was toward one of two choice targets that appeared on either side of the random-dot stimulus. At the end of the trial, the monkeys reported their direction judgment by making an eye movement to the appropriate target. We studied neurons in LIP that exhibited spatially selective persistent activity during delayed saccadic eye movement tasks. These neurons are thought to carry high-level signals appropriate for identifying salient visual targets and for guiding saccadic eye movements. We arranged the motion discrimination task so that one of the choice targets was in the LIP neuron's response field (RF) while the other target was positioned well away from the RF. During motion viewing, neurons in LIP altered their firing rate in a manner that predicted the saccadic eye movement that the monkey would make at the end of the trial. The activity thus predicted the monkey's judgment of motion direction. This predictive activity began early in the motion-viewing period and became increasingly reliable as the monkey viewed the random-dot motion. The neural activity predicted the monkey's direction judgment on both easy and difficult trials (strong and weak motion), whether or not the judgment was correct. In addition, the timing and magnitude of the response was affected by the strength of the motion signal in the stimulus. When the direction of motion was toward the RF, stronger motion led to larger neural responses earlier in the motion-viewing period. When motion was away from the RF, stronger motion led to greater suppression of ongoing activity. Thus the activity of single neurons in area LIP reflects both the direction of an impending gaze shift and the quality of the sensory information that instructs such a response. The time course of the neural response suggests that LIP accumulates sensory signals relevant to the selection of a target for an eye movement.

Neural computations that underlie decisions about sensory stimuli.

Decision-making behavior has been studied extensively, but the neurophysiological mechanisms responsible for this remarkable cognitive ability are just beginning to be understood. Here we propose neural computations that can account for the formation of categorical decisions about sensory stimuli by accumulating information over time into a single quantity: the logarithm of the likelihood ratio favoring one alternative over another. We also review electrophysio-logical studies that have identified brain structures that may be involved in computing this sort of decision variable. The ideas presented constitute a framework for understanding how and where perceptual decisions are formed in the brain.

Representation of a perceptual decision in developing oculomotor commands.

Behaviour often depends on the ability to make categorical judgements about sensory information acquired over time. Such judgements require a comparison of the evidence favouring the alternatives, but how the brain forms these comparisons is unknown. Here we show that in a visual discrimination task, the accumulating balance of sensory evidence favouring one interpretation over another is evident in the neural circuits that generate the behavioural response. We trained monkeys to make a direction judgement about dynamic random-dot motions and to indicate their judgement with an eye movement to a visual target. We interrupted motion viewing with electrical microstimulation of the frontal eye field and analysed the resulting, evoked eye movements for evidence of ongoing activity associated with the oculomotor response. Evoked eye movements deviated in the direction of the monkey's judgement. The magnitude of the deviation depended on motion strength and viewing time. The oculomotor signals responsible for these deviations reflected the accumulated motion information that informed the monkey's choices on the discrimination task. Thus, for this task, decision formation and motor preparation appear to share a common level of neural organization.

Effect of expected reward magnitude on the response of neurons in the dorsolateral prefrontal cortex of the macaque.

The dorsolateral prefrontal cortex plays a critical role in guiding actions that ensue seconds after an instruction. We recorded from neurons in area 46 and the frontal eye field (FEF) while monkeys performed a memory-guided eye movement task. A visual cue signaled whether a small or large liquid reward would accompany a correct response. Many neurons in area 46 responded more when the monkey expected a larger reward. Reward-related enhancement was evident throughout the memory period and was most pronounced when the remembered target appeared in the neuron's response field. Enhancement was not present in the FEF. The mixture of neural signals representing spatial working memory and reward expectation appears to be a distinct feature of area 46.

Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque.

To make a visual discrimination, the brain must extract relevant information from the retina, represent appropriate variables in the visual cortex and read out this representation to decide which of two or more alternatives is more likely. We recorded from neurons in the dorsolateral prefrontal cortex (areas 8 and 46) of the rhesus monkey while it performed a motion discrimination task. The monkey indicated its judgment of direction by making appropriate eye movements. As the monkey viewed the motion stimulus, the neural response predicted the monkey's subsequent gaze shift, hence its judgment of direction. The response comprised a mixture of high-level oculomotor signals and weaker visual sensory signals that reflected the strength and direction of motion. This combination of sensory integration and motor planning could reflect the conversion of visual motion information into a categorical decision about direction and thus give insight into the neural computations behind a simple cognitive act.

The variable discharge of cortical neurons: implications for connectivity, computation, and information coding.

Cortical neurons exhibit tremendous variability in the number and temporal distribution of spikes in their discharge patterns. Furthermore, this variability appears to be conserved over large regions of the cerebral cortex, suggesting that it is neither reduced nor expanded from stage to stage within a processing pathway. To investigate the principles underlying such statistical homogeneity, we have analyzed a model of synaptic integration incorporating a highly simplified integrate and fire mechanism with decay. We analyzed a "high-input regime" in which neurons receive hundreds of excitatory synaptic inputs during each interspike interval. To produce a graded response in this regime, the neuron must balance excitation with inhibition. We find that a simple integrate and fire mechanism with balanced excitation and inhibition produces a highly variable interspike interval, consistent with experimental data. Detailed information about the temporal pattern of synaptic inputs cannot be recovered from the pattern of output spikes, and we infer that cortical neurons are unlikely to transmit information in the temporal pattern of spike discharge. Rather, we suggest that quantities are represented as rate codes in ensembles of 50-100 neurons. These column-like ensembles tolerate large fractions of common synaptic input and yet covary only weakly in their spike discharge. We find that an ensemble of 100 neurons provides a reliable estimate of rate in just one interspike interval (10-50 msec). Finally, we derived an expression for the variance of the neural spike count that leads to a stable propagation of signal and noise in networks of neurons-that is, conditions that do not impose an accumulation or diminution of noise. The solution implies that single neurons perform simple algebra resembling averaging, and that more sophisticated computations arise by virtue of the anatomical convergence of novel combinations of inputs to the cortical column from external sources.

A computational analysis of the relationship between neuronal and behavioral responses to visual motion.

We have documented previously a close relationship between neuronal activity in the middle temporal visual area (MT or V5) and behavioral judgments of motion (Newsome et al., 1989; Salzman et al., 1990; Britten et al., 1992; Britten et al., 1996). We have now used numerical simulations to try to understand how neural signals in area MT support psychophysical decisions. We developed a model that pools neuronal responses drawn from our physiological data set and compares average responses in different pools to produce psychophysical decisions. The structure of the model allows us to assess the relationship between "neuronal" input signals and simulated psychophysical performance using the same methods we have applied to real experimental data. We sought to reconcile three experimental observations: psychophysical performance (threshold sensitivity to motion stimuli embedded in noise), a trial-by-trial covariation between the neural response and the monkey's choices, and a modest correlation between pairs of MT neurons in their variable responses to identical visual stimuli. Our results can be most accurately simulated if psychophysical decisions are based on pools of at least 100 weakly correlated sensory neurons. The neurons composing the pools must include a broader range of sensitivities than we encountered in our MT recordings, presumably because of the inclusion of neurons whose optimal stimulus is different from the one being discriminated. Central sources of noise degrade the signal-to-noise ratio of the pooled signal, but this degradation is relatively small compared with the noise typically carried by single cortical neurons. This suggests that our monkeys base near-threshold psychophysical judgments on signals carried by populations of weakly interacting neurons; these populations include many neurons that are not tuned optimally for the particular stimuli being discriminated.

Motion perception: seeing and deciding.

The primate visual system offers unprecedented opportunities for investigating the neural basis of cognition. Even the simplest visual discrimination task requires processing of sensory signals, formation of a decision, and orchestration of a motor response. With our extensive knowledge of the primate visual and oculomotor systems as a base, it is now possible to investigate the neural basis of simple visual decisions that link sensation to action. Here we describe an initial study of neural responses in the lateral intraparietal area (LIP) of the cerebral cortex while alert monkeys discriminated the direction of motion in a visual display. A subset of LIP neurons carried high-level signals that may comprise a neural correlate of the decision process in our task. These signals are neither sensory nor motor in the strictest sense; rather they appear to reflect integration of sensory signals toward a decision appropriate for guiding movement. If this ultimately proves to be the case, several fascinating issues in cognitive neuroscience will be brought under rigorous physiological scrutiny.

A relationship between behavioral choice and the visual responses of neurons in macaque MT.

We have previously documented the exquisite motion sensitivity of neurons in extrastriate area MT by studying the relationship between their responses and the direction and strength of visual motion signals delivered to their receptive fields. These results suggested that MT neurons might provide the signals supporting behavioral choice in visual discrimination tasks. To approach this question from another direction, we have now studied the relationship between the discharge of MT neurons and behavioral choice, independently of the effects of visual stimulation. We found that trial-to-trial variability in neuronal signals was correlated with the choices the monkey made. Therefore, when a directionally selective neuron in area MT fires more vigorously, the monkey is more likely to make a decision in favor of the preferred direction of the cell. The magnitude of the relationship was modest, on average, but was highly significant across a sample of 299 cells from four monkeys. The relationship was present for all stimuli (including those without a net motion signal), and for all but the weakest responses. The relationship was reduced or eliminated when the demands of the task were changed so that the directional signal carried by the cell was less informative. The relationship was evident within 50 ms of response onset, and persisted throughout the stimulus presentation. On average, neurons that were more sensitive to weak motion signals had a stronger relationship to behavior than those that were less sensitive. These observations are consistent with the idea that neuronal signals in MT are used by the monkey to determine the direction of stimulus motion. The modest relationship between behavioral choice and the discharge of any one neuron, and the prevalence of the relationship across the population, make it likely that signals from many neurons are pooled to form the data on which behavioral choices are based.

Noise, neural codes and cortical organization.

Cortical circuitry must facilitate information transfer in accordance with a neural code. In this article we examine two candidate neural codes: information is represented in the spike rate of neurons, or information is represented in the precise timing of individual spikes. These codes can be distinguished by examining the physiological basis of the highly irregular interspike intervals typically observed in cerebral cortex. Recent advances in our understanding of cortical microcircuitry suggest that the timing of neuronal spikes conveys little, if any, information. The cortex is likely to propagate a noisy rate code through redundant, patchy interconnections.

Correlated neuronal discharge rate and its implications for psychophysical performance.

Single neurons can signal subtle changes in the sensory environment with surprising fidelity, often matching the perceptual sensitivity of trained psychophysical observers. This similarity poses an intriguing puzzle: why is psychophysical sensitivity not greater than that of single neurons? Pooling responses across neurons should average out noise in the activity of single cells, leading to substantially improved psychophysical performance. If, however, noise is correlated among these neurons, the beneficial effects of pooling would be diminished. To assess correlation within a pool, the responses of pairs of neurons were recorded simultaneously during repeated stimulus presentations. We report here that the observed covariation in spike count was relatively weak, the correlation coefficient averaging 0.12. A theoretical analysis revealed, however, that weak correlation can limit substantially the signalling capacity of the pool. In addition, theory suggests a relationship between neuronal responses and psychophysical decisions which may prove useful for identifying cell populations underlying specific perceptual capacities.

Responses of neurons in macaque MT to stochastic motion signals.

Dynamic random-dot stimuli have been widely used to explore central mechanisms of motion processing. We have measured the responses of neurons in area MT of the alert monkey while we varied the strength and direction of the motion signal in such displays. The strength of motion is controlled by the proportion of spatiotemporally correlated dots, which we term the correlation of the stimulus. For many MT cells, responses varied approximately linearly with stimulus correlation. When they occurred, nonlinearities were equally likely to be either positively or negatively accelerated. We also explored the relationship between response magnitude and response variance for these cells and found, in general agreement with other investigators, that this relationship conforms to a power law with an exponent slightly greater than 1. The variance of the cells' discharge is little influenced by the trial-to-trial fluctuations inherent in our stochastic display, and is therefore likely to be of neural origin. Linear responses to these stochastic motion stimuli are predicted by simple, low-level motion models incorporating sensors having relatively broad spatial and temporal frequency tuning.

Dichoptic activation of the early motion system.

The short range or early motion system has long been considered incapable of binocular integration. We have developed dichoptic motion stimuli which are based upon the decomposition of traveling sinewave gratings into the sum of two standing waves in spatial and temporal quadrature. The monocular views of such displays appear as counterphase flicker but when presented dichoptically the perception is of movement in a unique direction. Two lines of evidence are presented for the binocularity of early motion mechanisms in human vision. First, adaptation to dichoptic motion sinewave gratings is found to result in a motion aftereffect. Second, random texture motion displays based on the quadrature decomposition are found to support dichoptic perception of motion direction, but not figure/ground. Unlike random dot kinematograms, these displays do not necessitate alternating the direction of motion during dichoptic presentation. This encumbrance, and the reliance on figure/ground discrimination, may have been responsible for prior failure to achieve dichoptic motion perception with short range stimuli.

The analysis of visual motion: a comparison of neuronal and psychophysical performance.

We compared the ability of psychophysical observers and single cortical neurons to discriminate weak motion signals in a stochastic visual display. All data were obtained from rhesus monkeys trained to perform a direction discrimination task near psychophysical threshold. The conditions for such a comparison were ideal in that both psychophysical and physiological data were obtained in the same animals, on the same sets of trials, and using the same visual display. In addition, the psychophysical task was tailored in each experiment to the physiological properties of the neuron under study; the visual display was matched to each neuron's preference for size, speed, and direction of motion. Under these conditions, the sensitivity of most MT neurons was very similar to the psychophysical sensitivity of the animal observers. In fact, the responses of single neurons typically provided a satisfactory account of both absolute psychophysical threshold and the shape of the psychometric function relating performance to the strength of the motion signal. Thus, psychophysical decisions in our task are likely to be based upon a relatively small number of neural signals. These signals could be carried by a small number of neurons if the responses of the pooled neurons are statistically independent. Alternatively, the signals may be carried by a much larger pool of neurons if their responses are partially intercorrelated.