Effect of feature-selective attention on neuronal responses in macaque area MT.

Attention influences visual processing in striate and extrastriate cortex, which has been extensively studied for spatial-, object-, and feature-based attention. Most studies exploring neural signatures of feature-based attention have trained animals to attend to an object identified by a certain feature and ignore objects/displays identified by a different feature. Little is known about the effects of feature-selective attention, where subjects attend to one stimulus feature domain (e.g., color) of an object while features from different domains (e.g., direction of motion) of the same object are ignored. To study this type of feature-selective attention in area MT in the middle temporal sulcus, we trained macaque monkeys to either attend to and report the direction of motion of a moving sine wave grating (a feature for which MT neurons display strong selectivity) or attend to and report its color (a feature for which MT neurons have very limited selectivity). We hypothesized that neurons would upregulate their firing rate during attend-direction conditions compared with attend-color conditions. We found that feature-selective attention significantly affected 22% of MT neurons. Contrary to our hypothesis, these neurons did not necessarily increase firing rate when animals attended to direction of motion but fell into one of two classes. In one class, attention to color increased the gain of stimulus-induced responses compared with attend-direction conditions. The other class displayed the opposite effects. Feature-selective activity modulations occurred earlier in neurons modulated by attention to color compared with neurons modulated by attention to motion direction. Thus feature-selective attention influences neuronal processing in macaque area MT but often exhibited a mismatch between the preferred stimulus dimension (direction of motion) and the preferred attention dimension (attention to color).

Neural correlates of chromatic motion perception.

A variety of psychophysical and neurophysiological studies suggest that chromatic motion perception in the primate brain may be performed outside the classical motion processing pathway. We addressed this provocative proposal directly by assessing the sensitivity of neurons in motion area MT to moving colored stimuli while simultaneously determining perceptual sensitivity in nonhuman primate observers. The results of these studies demonstrate a strong correspondence between neuronal and perceptual measures. Our findings testify that area MT is indeed a principal component of the neuronal substrate for color-based motion processing.

Neuronal representations of stimulus associations develop in the temporal lobe during learning.

Visual stimuli that are frequently seen together become associated in long-term memory, such that the sight of one stimulus readily brings to mind the thought or image of the other. It has been hypothesized that acquisition of such long-term associative memories proceeds via the strengthening of connections between neurons representing the associated stimuli, such that a neuron initially responding only to one stimulus of an associated pair eventually comes to respond to both. Consistent with this hypothesis, studies have demonstrated that individual neurons in the primate inferior temporal cortex tend to exhibit similar responses to pairs of visual stimuli that have become behaviorally associated. In the present study, we investigated the role of these areas in the formation of conditional visual associations by monitoring the responses of individual neurons during the learning of new stimulus pairs. We found that many neurons in both area TE and perirhinal cortex came to elicit more similar neuronal responses to paired stimuli as learning proceeded. Moreover, these neuronal response changes were learning-dependent and proceeded with an average time course that paralleled learning. This experience-dependent plasticity of sensory representations in the cerebral cortex may underlie the learning of associations between objects.

Progress in the neural sciences in the the century after Cajal (and the mysteries that remain).

One hundred years after Santiago Ramon y Cajal provided critical evidence for the "neuron doctrine," his cellular view of the brain remains the basis of modern neural science. This article begins with a review of how the early work of Ramon y Cajal, Charles Sherrington, and John Eccles and their contemporaries laid the groundwork for our current understanding of he information processing of neural systems and for understanding the task faced by studies of how the brain develops. The visual system is examined in some detail as a model for experimental investigation into the structure, operational mechanisms, and functions of large neural systems. Discussion of the phenomena of visual awareness and consciousness, links between the visual system and other brain systems, and disorders that disrupt voluntary control of cognition and emotion lead to a broader consideration of the problem of consciousness.

Nonlocal interactions in color perception: nonlinear processing of chromatic signals from remote inducers.

The perceived color of an object depends on the chromaticity of its immediate background. But color appearance is also influenced by remote chromaticities. To quantify these influences, the effects of remote color fields on the appearance of a fixated 2 degrees test field were measured using a forced-choice method. Changes in the appearance of the test field were induced by chromaticity changes of the background and of 2 degrees color fields not adjacent to the test field. The appearance changes induced by the color of the background corresponded to a fraction of between 0.5 and 0.95 of the cone contrast of the background change, depending on the observer. The magnitude of induction by the background color was modulated on average by 7.6% by chromaticity changes in the remote color fields. Chromaticity changes in the remote fields had virtually no inducing effect when they occurred without a change in background color. The spatial range of these chromatic interactions extended over at least 10 degrees from the fovea. They were established within the first few hundred milliseconds after the change of background color and depended only weakly on the number of inducing fields. These results may be interpreted as reflecting rapid chromatic interactions that support robustness of color vision under changing viewing conditions.

Cognitive neuroscience.

The last decade of the 20th century has seen the development of cognitive neuroscience as an effort to understand how the brain represents mental events. We review the areas of emotional and motor memory, vision, and higher mental processes as examples of this new understanding. Progress in all of these areas has been swift and impressive, but much needs to be done to reveal the mechanisms of cognition at the local circuit and molecular levels. This work will require new methods for controlling gene expression in higher animals and in studying the interactions between neurons at multiple levels.

Occlusion and the interpretation of visual motion: perceptual and neuronal effects of context.

Visual motion can be represented in terms of the dynamic visual features in the retinal image or in terms of the moving surfaces in the environment that give rise to these features. For natural images, the two types of representation are necessarily quite different because many moving features are only spuriously related to the motion of surfaces in the visual scene. Such "extrinsic" features arise at occlusion boundaries and may be detected by virtue of the depth-ordering cues that exist at those boundaries. Although a number of studies have provided evidence of the impact of depth ordering on the perception of visual motion, few attempts have been made to identify the neuronal substrate of this interaction. To address this issue, we devised a simple contextual manipulation that decouples surface motion from the motions of visual image features. By altering the depth ordering between a moving pattern and abutting static regions, the perceived direction of motion changes dramatically while image motion remains constant. When stimulated with these displays, many neurons in the primate middle temporal visual area (area MT) represent the implied surface motion rather than the motion of retinal image features. These neurons thus use contextual depth-ordering information to achieve a representation of the visual scene consistent with perceptual experience.

Neural correlates of contrast detection at threshold.

Human psychophysical studies have demonstrated that, for stimuli near the threshold of visibility, detection of motion in one direction is unaffected by the superimposition of motion in the opposite direction. To investigate the neural basis for this perceptual phenomenon, we recorded from directionally selective neurons in macaque visual area MT (middle temporal visual area). Contrast thresholds obtained for single gratings moving in a neuron's preferred direction were compared with those obtained for motion presented simultaneously in the neuron's preferred and antipreferred directions. A simple model based on probability summation between neurons tuned to opposite directions could sufficiently account for contrast thresholds revealed psychophysically, suggesting that area MT is likely to provide the neural basis for contrast detection of stimuli modulated in time.

Comparison of red-green equiluminance points in humans and macaques: evidence for different L:M cone ratios between species.

The human spectral luminosity function (V(lambda)) can be modeled as the linear sum of signals from long-wavelength-selective (L) and middle-wavelength-selective (M) cones, with L cones being weighted by a factor of approximately 2. This factor of approximately 2 is thought to reflect an approximate 2:1 ratio of L:M cones in the human retina, which has been supported by studies that allow for more direct counting of different cone types in the retina. In contrast to humans, several lines of retinally based evidence in macaques suggest an L:M ratio closer to 1:1. To investigate the consequences of differences in L:M cone ratios between humans and macaques, red-green equiluminance matches obtained psychophysically in humans (n = 11) were compared with those obtained electrophysiologically from single neurons in the extrastriate middle temporal visual area of macaques (M. mulatta, n = 5). Neurons in the middle temporal visual area were tested with sinusoidal red-green moving gratings across a range of luminance contrasts, with equiluminance being defined as the red-green contrast yielding a response minimum. Human subjects were tested under analogous conditions, by a minimally distinct motion technique, to establish psychophysical equiluminance. Although red-green equiluminance points in both humans and macaques were found to vary across individuals, the means across species differed significantly; compared with humans, macaque equiluminance points reflected relatively greater sensitivity to green. By means of a simple model based on equating the weighted sum of L and M cone signals, the observed red-green equiluminance points were found to be consistent with L:M cone ratios of approximately 2:1 in humans and 1:1 in macaques. These data thus support retinally based estimates of L:M cone ratios and further demonstrate that the information carried in the cone mosaic has functional consequences for red-green spectral sensitivity revealed perceptually and in the dorsal stream of visual cortex.

Covert visual search: a comparison of performance by humans and macaques (Macaca mulatta).

The duration of the visual search by human participants for visual features is independent of the number of targets being viewed. In contrast, search for targets formed by conjunction of features is characterized by reaction times that increase as a linear function of the number of items viewed, suggesting that the target detection requires scrutiny of the search array by focal attention. Macaque (Macaca mulatta) and human performance on feature and conjunction search tasks was compared by using color or motion, or by conjunctions of color and motion. Like human participants, monkeys exhibited a dichotomy between feature and conjunction search performance. This finding suggests that humans and macaques engage similar brain mechanisms for representation of feature and conjunction targets. This behavioral paradigm can thus be used in neurophysiological experiments directed at the mechanisms of feature integration and target selection.

The contribution of color to motion processing in Macaque middle temporal area.

The chromatic properties of an image yield strong cues for object boundaries and thus hold the potential to facilitate the detection of object motion. The extent to which cortical motion detectors exploit chromatic information, however, remains a matter of debate. To address this further, we quantified the strength of chromatic input to directionally selective neurons in the middle temporal area (MT) of macaque cerebral cortex using an equivalent luminance contrast (EqLC) paradigm. This paradigm, in which two sinusoidal gratings, one heterochromatic and the other achromatic, are superimposed and moved in opposite directions, allows the sensitivity of motion detectors to heterochromatic stimuli to be quantified and expressed relative to the benchmark of sensitivity for a luminance-defined stimulus. The results of these experiments demonstrate that the chromatic contrast in a moving red-green heterochromatic grating strongly influences directional responses in MT when the luminance contrast in that same grating is relatively low; for such stimuli, EqLC is at least 5%. When luminance contrast is added to the heterochromatic grating, however, EqLC wanes sharply and becomes negative (-4%) when luminance contrast is sufficiently high (>17-23%). Thus, the chromatic properties of an object appear to confer little or no benefit to motion processing by MT neurons when sufficient luminance contrast concurrently exists. These data support a simple model in which chromatic motion processing in MT is almost exclusively determined by magnocellular input. Additionally, a comparison of neuronal and psychophysical data suggests that MT may not be the sole contributor to the perceptual experience elicited by motion of heterochromatic patterns, or that only a subset of MT neurons serve this function.

Gauging sensory representations in the brain.

The stream of information that enters a sensory system is a product of the ecological niche of an organism and the way in which the information is sampled. The most salient characteristic of this sensory stream is the rich temporal structure that is caused by changes in the environment and self motion of sensors (for example, rapid eye or whisker movements). In recent years, substantial progress has been made in understanding how such rapidly varying stimuli are represented in the responses of sensory neurons of a large variety of sensory systems. The crucial observation that has emerged from these studies is that individual action potentials convey substantial amounts of information, which permits the discrimination of rapidly varying stimuli with high temporal precision.

Segmentation by color influences responses of motion-sensitive neurons in the cortical middle temporal visual area.

We previously showed that human subjects are better able to discriminate the direction of a motion signal in dynamic noise when the signal is distinguished (segmented) from the noise by color. This finding suggested a hitherto unexplored avenue of interaction between motion and color pathways in the primate visual system. To examine whether chromatic segmentation exerts a similar influence on cortical neurons that contribute to motion direction discrimination, we have now compared the discriminative capacity of single MT neurons and psychophysical observers viewing motion signals with and without chromatic segmentation. All data were obtained from rhesus monkeys trained to discriminate motion direction in dynamic stimuli containing varying proportions of coherently moving (signal) and randomly moving (noise) dots. We obtained psychophysical and neurophysiological data in the same animals, on the same trials, and using the same visual display. Chromatic segmentation of the signal from the noise enhanced both neuronal and psychophysical sensitivity to the motion signal but had a smaller influence on neuronal than on psychophysical sensitivity. Hence the ratio of neuronal to psychophysical thresholds, one measure of the relation between neuronal and psychophysical performance, depended on chromatic segmentation. Increased neuronal sensitivity to chromatically segmented displays stemmed from larger and less noisy responses to motion in the preferred directions of the neurons, suggesting that specialized mechanisms influence responses in the motion pathway when color segments motion signal in visual scenes. These findings lead us to reevaluate potential mechanisms for pooling of MT responses and the role of MT in motion perception.

Efficient discrimination of temporal patterns by motion-sensitive neurons in primate visual cortex.

Although motion-sensitive neurons in macaque middle temporal (MT) area are conventionally characterized using stimuli whose velocity remains constant for 1-3 s, many ecologically relevant stimuli change on a shorter time scale (30-300 ms). We compared neuronal responses to conventional (constant-velocity) and time-varying stimuli in alert primates. The responses to both stimulus ensembles were well described as rate-modulated Poisson processes but with very high precision (approximately 3 ms) modulation functions underlying the time-varying responses. Information-theoretic analysis revealed that the responses encoded only approximately 1 bit/s about constant-velocity stimuli but up to 29 bits/s about the time-varying stimuli. Analysis of local field potentials revealed that part of the residual response variability arose from "noise" sources extrinsic to the neuron. Our results demonstrate that extrastriate neurons in alert primates can encode the fine temporal structure of visual stimuli.

Luminance contrast affects motion coherency in plaid patterns by acting as a depth-from-occlusion cue.

Moving plaid patterns composed of component gratings that differ in luminance contrast tend not to cohere perceptually. Plaid patterns configured to mimic one occlusive grating overlying another also fail to cohere. We hypothesized that plaids constructed of components with different luminance contrasts fail to cohere because these components are interpreted as occlusive surfaces lying in different depth planes. It is known that when depth-from-occlusion and depth-from-binocular disparity cues support the same depth-ordering, both segregation in depth and motion non-coherency are more likely to be perceived than when these two cues conflict. We exploited this interaction and tested our hypothesis by introducing horizontal binocular disparity between two superimposed component gratings of different luminance contrasts. We found that both depth segregation and motion non-coherency were much more likely when the high-contrast grating was stereoscopically in front of the low-contrast grating. From these results we infer that luminance contrast acts as a depth-cue in plaid patterns, with higher contrast gratings appearing to lie in front of lower contrast gratings. Perceptual motion coherency parallels these depth-ordering judgments. We conclude that luminance contrast affects motion coherency by acting as a depth-from-occlusion cue.

Perceptual, oculomotor, and neural responses to moving color plaids.

Moving plaids constructed from two achromatic gratings of identical luminance contrast are known to yield a percept of coherent pattern motion, as are plaids constructed from two identical chromatic (e.g. isoluminant red/green) gratings. To examine the interactive influences of chromatic and luminance contrast on the integration of visual motion signals, we constructed plaids with gratings that possessed both forms of contrast. We used plaids of two basic types, which differed with respect to the phase relationship between chromatic and luminance modulations (after Kooi et al, 1992 Perception 21 583-598). One plaid type ('symmetric') was made from component gratings that had identical chromatic/luminance phase relationships (e.g. both components were red-bright/green-dark modulation). The second plaid type ('asymmetric') was made from components that had complimentary phase relationships (i.e. one red-bright/green-dark grating and one green-bright/red-dark grating). Human subjects reported that the motion of symmetric plaids was perceptually coherent, while the components of asymmetric plaids failed to cohere. We also recorded eye movements elicited by both types of plaids to determine if they are similarly affected by these image cues for motion coherence. Results demonstrate that, under many conditions, eye movements elicited by perceptually coherent vs noncoherent plaids are distinguishable from one another. To reveal the neural bases of these perceptual and oculomotor phenomena, we also recorded the responses of neurons in the middle temporal visual area (area MT) of macaque visual cortex. Here we found that individual neurons exhibited differential tuning to symmetric vs asymmetric plaids. These neurophysiological results demonstrate that the neural mechanism for motion coherence is sensitive to the phase relationship between chromatic and luminance contrast, a finding which has implications for interactions between 'color' and 'motion' processing streams in the primate visual system.