Form representation in monkey inferotemporal cortex is virtually unaltered by free viewing.

How are objects represented in the brain during natural behavior? Visual object recognition in primates is thought to depend on the inferotemporal cortex (IT). In most neurophysiological studies of IT, monkeys hold their direction of gaze fixed while isolated visual stimuli are presented (controlled viewing). However, during natural behavior, primates visually explore cluttered environments by changing gaze direction several times each second (free viewing). We examined the effect of free viewing on IT neuronal responses in monkeys engaged in a form-recognition task. By making small, real-time stimulus adjustments, we produced nearly identically retinal stimulation during controlled and free viewing. Nearly 90% of neuronal responses were unaffected by free viewing, and average stimulus selectivity was unchanged. Thus, neuronal representations that likely underlie form recognition are virtually unaltered by free viewing.

Attention to both space and feature modulates neuronal responses in macaque area V4.

Attention is the mechanism with which we select specific aspects of our environment for processing. Psychological experiments have shown that attention can be directed to a spatial location or to a particular object. Electrophysiological studies in trained macaque monkeys have found that attention can strengthen the responses of neurons in cortical area V4. Some of these studies have attributed these effects to spatial attention, whereas others have suggested that feature-directed attention may modulate the neuronal response. Here we report that neuronal correlates for both spatial and feature-directed attention exist in individual neurons in area V4 of behaving rhesus monkeys.

Effects of attention on the reliability of individual neurons in monkey visual cortex.

To determine the physiological mechanisms underlying the enhancement of performance by attention, we examined how attention affects the ability of isolated neurons to discriminate orientation by investigating the reliability of responses with and without attention. Recording from 262 neurons in cortical area V4 while two rhesus macaques did a delayed match-to-sample task with oriented stimuli, we found that attention did not produce detectable changes in the variability of neuronal responses but did improve the orientation discriminability of the neurons. We also found that attention did not change the relationship between burst rate and response rate. Our results are consistent with the idea that attention selects groups of neurons for a multiplicative enhancement in response strength.

Effects of attention on the processing of motion in macaque middle temporal and medial superior temporal visual cortical areas.

The visual system is continually inundated with information received by the eyes. Only a fraction of this information appears to reach visual awareness. This process of selection is one of the functions ascribed to visual attention. Although many studies have investigated the role of attention in shaping neuronal representations in cortical areas, few have focused on attentional modulation of neuronal signals related to visual motion. We recorded from 89 direction-selective neurons in middle temporal (MT) and medial superior temporal (MST) visual cortical areas of two macaque monkeys using identical sensory stimulation under various attentional conditions. Neural responses in both areas were greatly influenced by attention. When attention was directed to a stimulus inside the receptive field of a neuron, responses in MT and MST were enhanced an average of 20 and 40% compared with a condition in which attention was directed outside the receptive field. Even stronger average enhancements (70% in MT and 100% in MST) were observed when attention was switched from a stimulus moving in the nonpreferred direction inside the receptive field to another stimulus in the receptive field that was moving in the preferred direction. These findings show that attention modulates motion processing from stages early in the dorsal visual pathway by selectively enhancing the representation of attended stimuli and simultaneously reducing the influence of unattended stimuli.

Visual response latencies of magnocellular and parvocellular LGN neurons in macaque monkeys.

Signals relayed through the magnocellular layers of the LGN travel on axons with faster conduction speeds than those relayed through the parvocellular layers. As a result, magnocellular signals might reach cerebral cortex appreciably before parvocellular signals. The relative speed of these two channels cannot be accurately predicted based solely on axon conduction speeds, however. Other factors, such as different degrees of convergence in the magnocellular and parvocellular channels and the retinal circuits that feed them, can affect the time it takes for magnocellular and parvocellular signals to activate cortical neurons. We have investigated the relative timing of visual responses mediated by the magnocellular and parvocellular channels. We recorded individually from 78 magnocellular and 80 parvocellular neurons in the LGN of two anesthetized monkeys. Visual response latencies were measured for small spots of light of various intensities. Over a wide range of stimulus intensities the fastest magnocellular response latencies preceded the fastest parvocellular response latencies by about 10 ms. Because parvocellular neurons are far more numerous than magnocellular neurons, convergence in cortex could reduce the magnocellular advantage by allowing parvocellular signals to generate detectable responses sooner than expected based on the responses of individual parvocellular neurons. An analysis based on a simple model using neurophysiological data collected from the LGN shows that convergence in cortex could eliminate or reverse the magnocellular advantage. This observation calls into question inferences that have been made about ordinal relationships of neurons based on timing of responses.

Effects of attention on orientation-tuning functions of single neurons in macaque cortical area V4.

We examined how attention affected the orientation tuning of 262 isolated neurons in extrastriate area V4 and 135 neurons in area V1 of two rhesus monkeys. The animals were trained to perform a delayed match-to-sample task in which oriented stimuli were presented in the receptive field of the neuron being recorded. On some trials the animals were instructed to pay attention to those stimuli, and on other trials they were instructed to pay attention to other stimuli outside the receptive field. In this way, orientation-tuning curves could be constructed from neuronal responses collected in two behavioral states: one in which those stimuli were attended by the animal and one in which those stimuli were ignored by the animal. We fit Gaussians to the neuronal responses to twelve different orientations for each behavioral state. Although attention enhanced the responses of V4 neurons (median 26% increase) and V1 neurons (median 8% increase), selectivity, as measured by the width of its orientation-tuning curve, was not systematically altered by attention. The effects of attention were consistent with a multiplicative scaling of the driven response to all orientations. We also found that attention did not cause systematic changes in the undriven activity of the neurons.

Shape selectivity in primate lateral intraparietal cortex.

The extrastriate visual cortex can be divided into functionally distinct temporal and parietal regions, which have been implicated in feature-related ('what') and spatial ('where') vision, respectively. Neuropsychological studies of patients with damage to either the temporal or the parietal regions provide support for this functional distinction. Given the prevailing modular theoretical framework and the fact that prefrontal cortex receives inputs from both temporal and parietal streams, recent studies have focused on the role of prefrontal cortex in understanding where and how information about object identity is integrated with (or remains segregated from) information about object location. Here we show that many neurons in primate posterior parietal cortex (the 'where' pathway) show sensory shape selectivities to simple, two-dimensional geometric shapes while the animal performs a simple fixation task. In a delayed match-to-sample paradigm, many neuronal units also show significant differences in delay-period activity, and these differences depend on the shape of the sample. These results indicate that units in posterior parietal cortex contribute to attending to and remembering shape features in a way that is independent of eye movements, reaching, or object manipulation. These units show shape selectivity equivalent to any shown in the ventral pathway.

Sensory modality specificity of neural activity related to memory in visual cortex.

Previous studies have shown that when monkeys perform a delayed match-to-sample (DMS) task, some neurons in inferotemporal visual cortex are activated selectively during the delay period when the animal must remember particular visual stimuli. This selective delay activity may be involved in short-term memory. It does not depend on visual stimulation: both auditory and tactile stimuli can trigger selective delay activity in inferotemporal cortex when animals expect to respond to visual stimuli in a DMS task. We have examined the overall modality specificity of delay period activity using a variety of auditory/visual cross-modal and unimodal DMS tasks. The cross-modal DMS tasks involved making specific long-term memory associations between visual and auditory stimuli, whereas the unimodal DMS tasks were standard identity matching tasks. Delay activity existed in auditory/visual cross-modal DMS tasks whether the animal anticipated responding to visual or auditory stimuli. No evidence of selective delay period activation was seen in a purely auditory DMS task. Delay-selective cells were relatively common in one animal where they constituted up to 53% neurons tested with a given task. This was only the case for up to 9% of cells in a second animal. In the first animal, a specific long-term memory representation for learned cross-modal associations was observed in delay activity, indicating that this type of representation need not be purely visual. Furthermore, in this same animal, delay activity in one cross-modal task, an auditory-to-visual task, predicted correct and incorrect responses. These results suggest that neurons in inferotemporal cortex contribute to abstract memory representations that can be activated by input from other sensory modalities, but these representations are specific to visual behaviors.

On the relationship between synaptic input and spike output jitter in individual neurons.

What is the relationship between the temporal jitter in the arrival times of individual synaptic inputs to a neuron and the resultant jitter in its output spike? We report that the rise time of firing rates of cells in striate and extrastriate visual cortex in the macaque monkey remain equally sharp at different stages of processing. Furthermore, as observed by others, multiunit recordings from single units in the primate frontal lobe reveal a strong peak in their cross-correlation in the 10-150 msec range with very small temporal jitter (on the order of 1 msec). We explain these results using numerical models to study the relationship between the temporal jitter in excitatory and inhibitory synaptic input and the variability in the spike output timing in integrate-and-fire units and in a biophysically and anatomically detailed model of a cortical pyramidal cell. We conclude that under physiological circumstances, the standard deviation in the output jitter is linearly related to the standard deviation in the input jitter, with a constant of less than one. Thus, the timing jitter in successive layers of such neurons will converge to a small value dictated by the jitter in axonal propagation times.

Attentional modulation of visual motion processing in cortical areas MT and MST.

The visual system is constantly inundated with information received by the eyes, only a fraction of which seems to reach visual awareness. This selection process is one of the functions ascribed to visual attention. Although many studies have investigated the role of attention in shaping neuronal representations in the visual cortex, few have focused on attentional modulation of neuronal signals related to visual motion. Here we report that the responses of direction-selective neurons in monkey visual cortex are greatly influenced by attention, and that this modulation occurs as early in the cortical hierarchy as the level of the middle temporal visual area (MT). Our finding demonstrates a stronger and earlier influence of attention on motion processing along the dorsal visual pathway than previously recognized.

No binocular rivalry in the LGN of alert macaque monkeys.

Orthogonal drifting gratings were presented binocularly to alert macaque monkeys in an attempt to find neural correlates of binocular rivalry. Gratings were centered over lateral geniculate nucleus (LGN) receptive fields and the corresponding points for the opposite eye. The only task of the monkey was to fixate. We found no difference between the responses of LGN neurons under rivalrous and nonrivalrous conditions, as determined by examining the ratios of their respective power spectra. There was, however, a curious "temporal afterimage" effect in which cell responses continued to be modulated at the drift frequency of the grating for several seconds after the grating disappeared.

The brain's visual world: representation of visual targets in cerebral cortex.

Microelectrode recordings from behaving monkeys have shown that neuronal responses in the visual cerebral cortex can depend greatly on which aspect of the scene is the target of the animal's attention. Accumulating evidence suggests that while the early stages of the visual pathway provide a faithful representation of the retinal image, later stages of processing in the visual cortex hold representations that emphasize the viewer's current interest. By filtering out irrelevant signals and adding information about objects whose presence is remembered or inferred, the cortex creates an edited representation of the visual world that is dynamically modified to suit the immediate goals of the viewer.

Neuronal correlates of inferred motion in primate posterior parietal cortex.

For many types of behaviours, it is necessary to monitor the position or movement of objects that are temporarily occluded. The primate posterior parietal cortex contains neurons that are active during visual guidance tasks: in some cases, even if the visual target disappears transiently. It has been proposed that activity of this sort could be related to current or planned eye movements, but it might also provide a more generalized abstract representation of the spatial disposition of targets, even when they are not visible. We have recorded from monkey posterior parietal cortex while the animal viewed a visual stimulus that disappeared, and then, depending on experimental context, could be inferred to be either moving or stationary. During this temporary absence of the stimulus, about half of the neurons were found to be significantly more active on those trials in which the stimulus could be presumed to be moving rather than stationary. The activity was thus present in the absence of either sensory input or motor output, suggesting that it may indeed constitute a generalized representation of target motion.

Responses of neurons in the parietal and temporal visual pathways during a motion task.

The visual cortex of macaque monkeys has been divided into two functional streams that have been characterized in terms of sensory processing (color/form vs motion) and in terms of behavioral goals (object recognition vs spatial orientation). As a step toward unifying these two views of cortical processing, we compared the behavioral modulation of sensory signals across the two streams in monkeys trained to do a visual short-term memory task. We recorded from individual neurons in areas MT, MST, 7a, and V4 while monkeys performed a delayed match-to-sample task using direction of motion as the matching criterion. This task allowed us to determine if sensory responses were modulated by extraretinal signals related to the direction of the remembered sample. We sorted neuronal responses as a function of the remembered direction and calculated a modulation index, MI = (maximum response--minimum response)/(maximum response + minimum response). In the motion pathway, we found virtually no extraretinal signals in MT (average MI = 0.11 +/- 0.01 SE, 66 cells), but progressively stronger extraretinal signals in later stages, that is, MST (average MI = 0.17 +/- 0.01 SE, 57 cells) and 7a (average MI = 0.23 +/- 0.02 SE, 46 cells). In contrast to MT, strong extraretinal signals for direction matching were found in V4 (average MI = 0.28 +/- 0.02 SE, 94 cells), a relatively early stage of the color/form pathway, even though this pathway is not generally viewed as playing a major role in motion processing. Some cells in V4 were also tested while the animals performed a color matching task. These cells showed memory-related modulation of their response when either color or direction was used as the matching criterion. We conclude that extraretinal signals related to the match-to-sample task may be stronger in the temporal (color/form) pathway than in the parietal (motion) pathway, regardless of the stimulus dimension involved. Furthermore, our results indicate that the temporal pathway is capable of making a significant contribution to motion processing in tasks where motion can be considered as a cue for the identification of object attributes.

Magnocellular and parvocellular contributions to the responses of neurons in macaque striate cortex.

Anatomical and physiological studies of the primate visual system have suggested that the signals relayed by the magnocellular and parvocellular subdivisions of the LGN remain segregated in visual cortex. It has been suggested that this segregation may account for the known differences in visual function between the parietal and temporal cortical processing streams in extrastriate visual cortex. To test directly the hypothesis that the temporal stream of processing receives predominantly parvocellular signals, we recorded visual responses from the superficial layers of V1 (striate cortex), which give rise to the temporal stream, while selectively inactivating either the magnocellular or parvocellular subdivisions of the LGN. Inactivation of the parvocellular subdivision reduced neuronal responses in the superficial layers of V1, but the effects of magnocellular blocks were generally as pronounced or slightly stronger. Individual neurons were found to receive contributions from both pathways. We furthermore found no evidence that magnocellular contributions were restricted to either the cytochrome oxidase blobs or interblobs in V1. Instead, magnocellular signals made substantial contributions to responses throughout the superficial layers. Thus, the regions within V1 that constitute the early stages of the temporal processing stream do not appear to contain isolated parvocellular signals. These results argue against a direct mapping of the subcortical magnocellular and parvocellular pathways onto the parietal and temporal streams of processing in cortex.

Responses in macaque visual area V4 following inactivation of the parvocellular and magnocellular LGN pathways.

A substantial body of evidence has suggested that signals transmitted through the magnocellular and parvocellular subdivisions of the LGN remain largely segregated in visual cortex. This hypothesis can be tested directly by selectively blocking transmission through either the magnocellular or parvocellular layers with small injections of lidocaine or GABA while recording cortical responses to a visual stimulus. In a previous study, we found that responses in the middle temporal visual area (MT) were almost always greatly reduced by blocks of magnocellular LGN, but that few MT neurons were affected by parvocellular blocks. In the present study, we have examined magnocellular and parvocellular contributions to area V4, which lies at the same level of processing in the cortical hierarchy as does MT and is thought to be a major recipient of parvocellular input. We found that inactivation of parvocellular layers usually resulted in a moderate reduction of visual responses (median reduction, 36%). However, comparable reductions in V4 responses were also seen following magnocellular blocks (median reduction, 47%). Directionally selective responses in V4 were not found to depend specifically on either subdivision. We conclude that area V4, unlike MT, receives strong input from both subdivisions of the LGN. These results suggest that the relationship between the subcortical magnocellular and parvocellular pathways and the parietal and temporal streams of processing in cortex is not one-to-one.

Magnocellular or parvocellular lesions in the lateral geniculate nucleus of monkeys cause minor deficits of smooth pursuit eye movements.

The effects of unilateral LGN lesions, made with ibotenic acid, on smooth pursuit eye movements were studied in two monkeys (Macaca nemestrina). Both monkeys received unilateral magnocellular (M-) layer lesions 18 months before the study and one monkey received a parvocellular (P-) lesion during the study on the side opposite the magnocellular lesion. The lesions did not affect the accuracy of saccades to stationary or moving targets, but the latencies of saccades to targets in the M-layer lesioned hemifields were significantly longer. Neither M- nor P-layer lesions affected the earliest interval (0-50 msec) of pursuit initiation, but during later intervals (50-150 msec), eye acceleration was less for pursuit initiation in the lesioned hemifield compared to the control hemifield. M-layer lesions created larger deficits in ocular acceleration than P-layer lesions. All deficits, however, were relatively small and accurate pursuit speeds were achieved near the time of the initial "catch-up" saccade. If both M and P layers representing the same part of the visual field were destroyed, the monkey was unable to locate the target or initiate smooth pursuit eye movements. We conclude that smooth pursuit initiation receives contributions from both the M- and P-layers of the LGN and either of these inputs can support pursuit initiation.

Visual effects of lesions of cortical area V2 in macaques.

Ibotenic acid lesions were placed in two monkeys in a portion of cortical area V2 that corresponds to a lower quadrant of the visual field extending approximately 3-7 degrees from the fovea. For purposes of comparison, another lesion was placed in area V1 in one animal. A wide range of visual capacities were then measured, using a discrimination between vertical and horizontal orientation, in and near the affected regions of the visual field. Visual acuity declined sharply as the test stimulus approached the visual field location corresponding to the V1 lesion, and no threshold could be measured at its center. In contrast, lesions of area V2 caused no measurable decrease in acuity, nor was there any substantial effect on several measures of contrast sensitivity. Subsequently, two types of more complex visual discriminations were measured (also using a vertical-horizontal discrimination), and these discriminations were severely disrupted by V2 lesions. The first discrimination was of the orientation of two parallel lines of five colinear dots each. We measured the number of background dots that would bring the discrimination to threshold, and this number of dots was greatly decreased by a V2 lesion. The second discrimination was of the orientation of a group of three distinctive texture elements embedded in a six by six element texture. This task could not be done in the visual field region affected by the V2 lesion when the distinctive elements differed in orientation from the others. Control experiments showed that the discrimination could be done when the three distinctive elements differed in size or color. These results suggest that cortical area V2 is not needed for some low-level discriminations, but may be essential for tasks involving complex spatial discriminations.