The invariance of directional tuning with contrast and coherence.

The responses of motion mechanisms depend not only on the direction of a stimulus, but also on its contrast, coherence and speed. We examined how contrast, coherence and directional selectivity interact by measuring directional tuning psychophysically across a wide range of coherence and contrast levels. We fit data with a simple model that estimated directional tuning bandwidth using contrast and coherence gain parameters that were based on neurophysiological estimates. This model estimated a bandwidth of approximately 90 degrees for directionally selective mechanisms. Bandwidth was invariant across a wide range of contrasts and coherences, as predicted by models of contrast normalization.

Visual stimuli activate auditory cortex in the deaf.

Previous brain imaging studies have demonstrated responses to tactile and auditory stimuli in visual cortex of blind subjects, suggesting that removal of one sensory modality leads to neural reorganization of the remaining modalities. To investigate whether similar 'cross-modal' plasticity occurs in human auditory cortex, we used functional magnetic resonance imaging (fMRI) to measure visually evoked activity in auditory areas of both early-deafened and hearing individuals. Here we find that deaf subjects exhibit activation in a region of the right auditory cortex, corresponding to Brodmann's areas 42 and 22, as well as in area 41 (primary auditory cortex), demonstrating that early deafness results in the processing of visual stimuli in auditory cortex.

Neural correlates of chromatic motion perception.

A variety of psychophysical and neurophysiological studies suggest that chromatic motion perception in the primate brain may be performed outside the classical motion processing pathway. We addressed this provocative proposal directly by assessing the sensitivity of neurons in motion area MT to moving colored stimuli while simultaneously determining perceptual sensitivity in nonhuman primate observers. The results of these studies demonstrate a strong correspondence between neuronal and perceptual measures. Our findings testify that area MT is indeed a principal component of the neuronal substrate for color-based motion processing.

Effects of set-size and selective spatial attention on motion processing.

In order to investigate the effects of divided attention and selective spatial attention on motion processing, we obtained direction-of-motion thresholds using a stochastic motion display under various attentional manipulations and stimulus durations (100-600 ms). To investigate divided attention, we compared motion thresholds obtained when a single motion stimulus was presented in the visual field (set-size=1) to those obtained when the motion stimulus was presented amongst three confusable noise distractors (set-size=4). The magnitude of the observed detriment in performance with an increase in set-size from 1 to 4 could be accounted for by a simple decision model based on signal detection theory, which assumes that attentional resources are not limited in capacity. To investigate selective attention, we compared motion thresholds obtained when a valid pre-cue alerted the subject to the location of the to-be-presented motion stimulus to those obtained when no pre-cue was provided. As expected, the effect of pre-cueing was large when the visual field contained noise distractors, an effect we attribute to "noise reduction" (i.e. the pre-cue allows subjects to exclude irrelevant distractors that would otherwise impair performance). In the single motion stimulus display, we found a significant benefit of pre-cueing only at short durations (< or =150 ms), a result that can potentially be explained by a "time-to-orient" hypothesis (i.e. the pre-cue improves performance by eliminating the time it takes to orient attention to a peripheral stimulus at its onset, thereby increasing the time spent processing the stimulus). Thus, our results suggest that the visual motion system can analyze several stimuli simultaneously without limitations on sensory processing per se, and that spatial pre-cueing serves to reduce the effects of distractors and perhaps increase the effective processing time of the stimulus.

Development of psychophysically-derived detection contours in L- and M-cone contrast space.

In order to investigate the development of color mechanisms in infants we fitted elliptical detection contours to psychophysically-derived contrast thresholds plotted in L- and M-cone contrast space. Detection ellipses were obtained for 47 infants (ages 2-5 months of age), and were compared to those of six adults tested under nearly identical conditions. The parameters of the fitted ellipses allowed us to address several aspects of color development. First, the lengths and widths were used to assess the relative development of chromatic, with respect to luminance, sensitivity. The results of these analyses revealed a sharp increase in chromatic sensitivity between 3 and 4 months of age, suggesting an accelerated development of chromatic mechanisms around this time. Second, the angles of the ellipses provided estimates of individual red/green isoluminance points. In line with previous reports, we found that isoluminance points do not vary significantly with age. Finally, our ellipse-fitting procedures were used to assess whether color sensitivity is best described by a model that assumes independence between post-receptoral chromatic and luminance mechanisms. Similar to previous results of Kelly and Chang [Kelly, J. P. & Chang, S. (2000). Vision Research 40, 1887-1906] obtained using steady-state visually evoked potentials, only a proportion (approximately half) of our infants exhibited detection contours that were consistent with independent mechanisms, a finding that most likely results from statistical noise in the infant data sets.

Visual contrast sensitivity in deaf versus hearing populations: exploring the perceptual consequences of auditory deprivation and experience with a visual language.

Early deafness in humans provides a unique opportunity to examine the perceptual consequences of altered sensory experience. In particular, visual perception in the deaf may be altered as a result of their auditory deprivation and/or because the deaf rely heavily upon a visual language (American Sign Language, or ASL, in the US). Recently, we found that deaf, but not hearing, subjects exhibit a right visual field/left hemisphere advantage on a low-level direction of motion task, a finding that has been attributed to the deaf's experience with ASL [Psychol. Sci. 10 (1999) 256; Brain Res. 405 (1987) 268]. In order to determine whether this visual field asymmetry generalizes to other low-level visual functions, in this study we measured contrast sensitivity in deaf and hearing subjects to moving stimuli over a range of speeds (0.125-64 degrees /s). We hypothesized that if ASL use drives differences between hearing and deaf subjects, such differences may occur over a restricted range of speeds most commonly found in ASL. In addition, we tested a third group, hearing native signers who learned ASL early from their deaf parents, to further assess whether potential differences between groups results from ASL use. These experiments reveal no overall differences in contrast sensitivity, nor differences in visual field asymmetries, across subject groups at any speed tested. Thus, differences previously observed between deaf and hearing subjects for discriminating the direction of moving stimuli do not generalize to contrast sensitivity for moving stimuli, a result that has implications for the neural level at which plastic changes occur in the visual system of deaf subjects.

Neural correlates of contrast detection at threshold.

Human psychophysical studies have demonstrated that, for stimuli near the threshold of visibility, detection of motion in one direction is unaffected by the superimposition of motion in the opposite direction. To investigate the neural basis for this perceptual phenomenon, we recorded from directionally selective neurons in macaque visual area MT (middle temporal visual area). Contrast thresholds obtained for single gratings moving in a neuron's preferred direction were compared with those obtained for motion presented simultaneously in the neuron's preferred and antipreferred directions. A simple model based on probability summation between neurons tuned to opposite directions could sufficiently account for contrast thresholds revealed psychophysically, suggesting that area MT is likely to provide the neural basis for contrast detection of stimuli modulated in time.

What covariance mechanisms underlie green/red equiluminance, luminance contrast sensitivity and chromatic (green/red) contrast sensitivity?

In order to investigate the mechanisms underlying green/red equiluminance matches in human observers and their relationship to mechanisms subserving luminance and/or chromatic (green/red) contrast sensitivity, we tested 21 human subjects along these dimensions at 16 different spatial and temporal frequencies (spatial frequency, 0.25-2 c/deg; temporal frequency, 2-16 Hz) and applied factor analysis to extract mechanisms underlying the data set. The results from our factor analysis revealed separate sources of variability for green/red equiluminance, luminance sensitivity and chromatic sensitivity, thus suggesting separate mechanisms underlying each of the three main conditions. When factor analysis was applied separately to green/red equiluminance data, two temporally-tuned factors were revealed (factor 1, 2-4 Hz; factor 2, 8-16 Hz), suggesting the existence of separate mechanisms underlying equiluminance settings at low versus high temporal frequencies. In addition, although the three main conditions remained separate in our factor analysis of the entire data set, our correlation matrix nonetheless revealed systematic correlations between equiluminance settings and luminance sensitivity at high temporal frequencies, and between equiluminance settings and chromatic sensitivity at low temporal frequencies. Taken together, these data suggest that the high temporal frequency factor underlying green/red equiluminance is governed predominantly by luminance mechanisms, while the low temporal frequency factor receives contribution from chromatic mechanisms.

Comparison of red-green equiluminance points in humans and macaques: evidence for different L:M cone ratios between species.

The human spectral luminosity function (V(lambda)) can be modeled as the linear sum of signals from long-wavelength-selective (L) and middle-wavelength-selective (M) cones, with L cones being weighted by a factor of approximately 2. This factor of approximately 2 is thought to reflect an approximate 2:1 ratio of L:M cones in the human retina, which has been supported by studies that allow for more direct counting of different cone types in the retina. In contrast to humans, several lines of retinally based evidence in macaques suggest an L:M ratio closer to 1:1. To investigate the consequences of differences in L:M cone ratios between humans and macaques, red-green equiluminance matches obtained psychophysically in humans (n = 11) were compared with those obtained electrophysiologically from single neurons in the extrastriate middle temporal visual area of macaques (M. mulatta, n = 5). Neurons in the middle temporal visual area were tested with sinusoidal red-green moving gratings across a range of luminance contrasts, with equiluminance being defined as the red-green contrast yielding a response minimum. Human subjects were tested under analogous conditions, by a minimally distinct motion technique, to establish psychophysical equiluminance. Although red-green equiluminance points in both humans and macaques were found to vary across individuals, the means across species differed significantly; compared with humans, macaque equiluminance points reflected relatively greater sensitivity to green. By means of a simple model based on equating the weighted sum of L and M cone signals, the observed red-green equiluminance points were found to be consistent with L:M cone ratios of approximately 2:1 in humans and 1:1 in macaques. These data thus support retinally based estimates of L:M cone ratios and further demonstrate that the information carried in the cone mosaic has functional consequences for red-green spectral sensitivity revealed perceptually and in the dorsal stream of visual cortex.

The contribution of color to motion processing in Macaque middle temporal area.

The chromatic properties of an image yield strong cues for object boundaries and thus hold the potential to facilitate the detection of object motion. The extent to which cortical motion detectors exploit chromatic information, however, remains a matter of debate. To address this further, we quantified the strength of chromatic input to directionally selective neurons in the middle temporal area (MT) of macaque cerebral cortex using an equivalent luminance contrast (EqLC) paradigm. This paradigm, in which two sinusoidal gratings, one heterochromatic and the other achromatic, are superimposed and moved in opposite directions, allows the sensitivity of motion detectors to heterochromatic stimuli to be quantified and expressed relative to the benchmark of sensitivity for a luminance-defined stimulus. The results of these experiments demonstrate that the chromatic contrast in a moving red-green heterochromatic grating strongly influences directional responses in MT when the luminance contrast in that same grating is relatively low; for such stimuli, EqLC is at least 5%. When luminance contrast is added to the heterochromatic grating, however, EqLC wanes sharply and becomes negative (-4%) when luminance contrast is sufficiently high (>17-23%). Thus, the chromatic properties of an object appear to confer little or no benefit to motion processing by MT neurons when sufficient luminance contrast concurrently exists. These data support a simple model in which chromatic motion processing in MT is almost exclusively determined by magnocellular input. Additionally, a comparison of neuronal and psychophysical data suggests that MT may not be the sole contributor to the perceptual experience elicited by motion of heterochromatic patterns, or that only a subset of MT neurons serve this function.

Infants code the direction of chromatic quadrature motion.

The present experiment uses a quadrature motion paradigm to investigate the motion correspondence cues used by young infants for coding the direction of motion of red/green isoluminant gratings. Three-month-old infants and adults were tested with 0.25 c/d luminance-modulated or red/green isoluminant gratings, either moving continuously or shifted in spatial quadrature. Both direction-of-motion and detection thresholds were measured, and motion:detection (M:D) threshold ratios were examined. Infants, like adults, could code the direction of motion of red/green quadrature-shifted gratings. In adults, M:D ratios were similar for continuous and quadrature motion. In infants, M:D ratios were higher for quadrature than for continuous motion, but elevations of similar magnitude were seen for both luminance-modulated and red/green gratings. The results suggest that frequency-doubled signals, such as those often seen in the magnocellular (M-cell) pathway, are not necessary for coding the direction of motion of isoluminant gratings in infant subjects. Two other theoretical options--mediation by the scatter of isoluminance points in the M-cell population, and parvocellular (P-cell) mediation--are discussed.

Infant temporal contrast sensitivity functions (tCSFs) mature earlier for luminance than for chromatic stimuli: evidence for precocious magnocellular development?

In order to investigate the development of luminance and chromatic temporal contrast sensitivity functions (tCSFs), we obtained chromatic and luminance contrast thresholds from individual 3- and 4-month old infants, and compared them to previously obtained functions in adults. Stimuli were moving sinusoidal gratings of 0.27 cyc/deg, presented at one of five temporal frequencies: 1.0, 2.1, 4.2, 9.4 or 19 Hz (corresponding speeds: 3.8, 7.7, 15, 34, 69 deg/s). Previous studies, including our own, have shown that adult tCSFs are bandpass for luminance stimuli (peaking at 5-10 Hz), yet lowpass for chromatic stimuli (sensitivity falling at > 2 Hz), and that the two functions cross one another near 4-5 Hz when plotted in terms of cone contrast. In the present study, we find that the shapes and peaks of the luminance tCSF in both 3- and 4-months-olds appear quite similar to those of adults. By contrast, chromatic tCSFs in infants are markedly different from those of adults. In agreement with our earlier report (Dobkins, K. R., Lia, B., & Teller, D. Y. (1997). Vision Research, 37(19), 2699-2716), the chromatic function in 3-month-olds is rather flat, lacking the sharp high temporal frequency fall-off characteristic of the adult function. In addition, the luminance tCSF in 3-month-olds is elevated above the chromatic tCSF, and the two functions do not exhibit an adult-like cross-over within the range of temporal frequencies tested. By 4 months of age, substantial development of chromatic contrast sensitivity takes place at the lowest temporal frequencies. Although still immature, the 4-month-old chromatic tCSF has begun to adopt a more adult-like shape. In addition, similar to adults, luminance and chromatic tCSFs in 4-month-olds cross one another near 5 Hz. In adults, magnocellular (M) and parvocellular (P) pathways are thought to underlie the bandpass luminance and lowpass chromatic tCSF, respectively (e.g. Lee, B. B., Pokorny, J., Smith, V. C., Martin, P. R., & Valberg, A. (1990). Journal of the Optical Society of America (a), 7(12), 2223-2236). Based on this correspondence between psychophysical and neural responses in adults, our results suggest that the relatively slow development of the chromatic tCSF in infants may reflect immature chromatic responses in the P pathway and/or reliance on chromatic responses originating in the M pathway.

Perceptual, oculomotor, and neural responses to moving color plaids.

Moving plaids constructed from two achromatic gratings of identical luminance contrast are known to yield a percept of coherent pattern motion, as are plaids constructed from two identical chromatic (e.g. isoluminant red/green) gratings. To examine the interactive influences of chromatic and luminance contrast on the integration of visual motion signals, we constructed plaids with gratings that possessed both forms of contrast. We used plaids of two basic types, which differed with respect to the phase relationship between chromatic and luminance modulations (after Kooi et al, 1992 Perception 21 583-598). One plaid type ('symmetric') was made from component gratings that had identical chromatic/luminance phase relationships (e.g. both components were red-bright/green-dark modulation). The second plaid type ('asymmetric') was made from components that had complimentary phase relationships (i.e. one red-bright/green-dark grating and one green-bright/red-dark grating). Human subjects reported that the motion of symmetric plaids was perceptually coherent, while the components of asymmetric plaids failed to cohere. We also recorded eye movements elicited by both types of plaids to determine if they are similarly affected by these image cues for motion coherence. Results demonstrate that, under many conditions, eye movements elicited by perceptually coherent vs noncoherent plaids are distinguishable from one another. To reveal the neural bases of these perceptual and oculomotor phenomena, we also recorded the responses of neurons in the middle temporal visual area (area MT) of macaque visual cortex. Here we found that individual neurons exhibited differential tuning to symmetric vs asymmetric plaids. These neurophysiological results demonstrate that the neural mechanism for motion coherence is sensitive to the phase relationship between chromatic and luminance contrast, a finding which has implications for interactions between 'color' and 'motion' processing streams in the primate visual system.

Infant color vision: temporal contrast sensitivity functions for chromatic (red/green) stimuli in 3-month-olds.

In order to investigate the development of temporal contrast sensitivity functions (tCSFs) for chromatic (red/green) stimuli, we obtained chromatic contrast thresholds from 3-month-old infants and adults using behavioral techniques. Stimuli were moving or counterphase-reversing sinusoidal gratings of 0.25 c/deg. Five temporal frequencies were used: 0.7, 2.1, 5.6, 11 and 17 Hz (corresponding speeds = 2.8, 8.4, 22, 44 and 67 deg/sec). In order to compare chromatic results with those obtained under luminance-defined conditions, luminance tCSFs were also obtained from adults, and previously obtained infant luminance tCSFs were used (from Dobkins & Teller, 1996a). In accordance with previous studies, adults exhibited bandpass luminance tCSFs with peaks near 5 Hz and lowpass chromatic tCSFs that declined rapidly at temporal frequencies greater than 2 Hz, and the two curves crossed one another near 4 Hz. By contrast, infants exhibited bandpass rather than lowpass chromatic tCSFs with peaks near 5 Hz. These chromatic curves were quite similar in peak frequency and general shape to previously obtained infant tCSFs for luminance stimuli. Moreover, both chromatic and luminance tCSFs in infants were found to be quite similar in peak and shape to luminance tCSFs observed in adults. These findings point to the possibility that, for 3-month-old infants, both chromatic and luminance stimuli are detected by the same underlying mechanism under these conditions. We propose that such a mechanism is probably a physiological pathway dominated by magnocellular input. Earlier studies of infant color vision are discussed in this context.

Infant motion: detection (M:D) ratios for chromatically defined and luminance-defined moving stimuli.

In order to assess the relative contributions of chromatic vs luminance information to motion processing in infants, we employed a motion:detection (M:D) paradigm. Stimuli consisted of 27 deg by 40 deg, 0.25 c/deg sinusoidal gratings moving at 22 deg/sec (5.6 Hz), and were either chromatically defined or luminance-defined. Contrast thresholds for direction-of-motion (M) were obtained using a directional eye movement technique. Contrast thresholds for detection (D) were obtained using forced-choice preferential looking. M:D threshold ratios were obtained for individual infant subjects, and results were compared to those of adults. As expected, adult M:D threshold ratios were near 1:1 for luminance-defined stimuli, but greater than 1:1 for chromatically defined stimuli. This suggests that, for adults, luminance-defined, but not chromatically defined, stimuli are detected by mechanisms labeled for direction of motion. By contrast, infant M:D ratios for chromatically and luminance-defined stimuli were approximately equal and close to 1:1, suggesting that, for infants, luminance- as well as chromatically defined stimuli are detected by mechanisms that are labeled for direction of motion.

Infant contrast detectors are selective for direction of motion.

In order to investigate the presence of directionally selective mechanisms in 3-month-old infants, we employed a summation-near-threshold paradigm previously developed for studies of adult vision (Levinson & Sekuler, 1975 Journal of Physiology (London), 250, 347-366); Watson, Thompson, Murphy & Nachmias, 1980 Vision Research, 20, 341-347). The degree of contrast summation occurring between two sinusoidal gratings moving in opposite direction was determined by comparing the contrast threshold for a compound stimulus (a counterphase-reversing grating) with the contrast threshold for one of its components (a single moving grating). Using the forced-choice preferential looking (FPL) technique, contrast thresholds were obtained for both counterphase and single moving gratings within individual infant subjects. Data were collected at several speeds, ranging from 2.8 to 66.8 degrees/sec (temporal frequency range: 0.7-16.7 Hz). At slow speeds, infants' thresholds were approximately equal for counterphase and moving gratings, indicating that non-directional mechanisms were responsible for detection. At an intermediate speed (22.3 degrees/sec), thresholds were nearly twice as high for counterphase gratings as for single moving gratings, indicating the existence of directionally selective mechanisms at detection threshold for this speed. For faster speeds, relative thresholds for the two types of stimuli fell between the two extremes; a model incorporating probability summation between directionally selective mechanisms was sufficient to account for the data. These results demonstrate that, at speeds greater than or equal to 22.3 degrees/sec (t.f. = 5.6 Hz), 3-month-old infants possess directionally selective mechanisms at threshold.

Behavioral and neural effects of chromatic isoluminance in the primate visual motion system.

We have previously reported that the responses of individual neurons in macaque visual area MT elicited by movement of contrast-reversing heterochromatic red/green borders are largest when the two hues are "balanced" or isoluminant (Dobkins & Albright, 1994). This "neural" isoluminant point was found to vary somewhat across the sample of neurons. Here, we compare the average neural isoluminant point in area MT to a behavioral measure of isoluminance, obtained using a modification of an oculomotor procedure developed by Chaudhuri and Albright (1992). These behavioral estimates of isoluminance closely parallel the neuronal data obtained from area MT. In accordance with previous evidence (e.g. Lee et al., 1988; Kaiser et al., 1990; Valberg et al., 1992), this correlation suggests that activity within the dorsal/magnocellular stream underlies behavioral expression of chromatic isoluminance.

What happens if it changes color when it moves?: the nature of chromatic input to macaque visual area MT.

Neurons in the middle temporal visual area (MT) of macaque cerebral cortex are highly selective for the direction of motion but not the color of a moving stimulus. Recent experiments have shown, however, that the directional selectivity of many MT neurons persists even when a moving stimulus is defined solely by chromatic variation (Charles and Logothetis, 1989; Saito et al., 1989; Dobkins and Albright, 1991 a, b; Movshon et al., 1991; Gegenfurtner et al., 1994). To illuminate the mechanisms by which area MT uses color as a cue for motion correspondence, we recorded from MT neurons while rhesus monkeys viewed an "apparent motion" stimulus in which red/green sine wave gratings underwent contrast reversal each time they were displaced in a particular direction. Under such conditions, correspondence based upon chromatically defined borders conflicts with correspondence based upon conservation of chromatic sign. When our heterochromatic stimuli possessed sufficient luminance modulation, MT neurons responded best to motion in the direction for which the sign of luminance (and chromatic) contrast was preserved. At isoluminance, however, two different chromatic influences were revealed. First, when stimuli underwent small spatial displacements, directional selectivity was elicited by movement of the stimulus in the direction of the nearest chromatically defined border, even though the sign of chromatic contrast at that border alternated over time. Under these conditions, MT neurons apparently exploited information about image borders defined by chromatic contrast while sacrificing information about the colors that make up those borders. By contrast, when chromatically defined borders provided only ambiguous information about direction of motion, MT neurons were capable of using information about the sign of chromatic contrast to detect direction of motion. The results from these experiments suggest the existence of a hybrid mechanism, one in which both signed and unsigned chromatic signals contribute to motion processing in visual area MT.

What happens if it changes color when it moves?: psychophysical experiments on the nature of chromatic input to motion detectors.

Several lines of evidence indicate that the processing of motion by the primate visual system continues even when a moving stimulus differs from its surroundings by color alone. To illuminate the mechanisms by which our visual system uses color as a token for motion correspondence, we have developed an "apparent motion" paradigm in which red/green sine-wave gratings undergo reversal of chromatic contrast sign each time they are displaced in a particular direction. Under such conditions, correspondence based upon conservation of chromatic sign conflicts with correspondence based upon chromatically-defined borders. When these heterochromatic stimuli also possess luminance modulation, motion is always perceived in the direction in which the sign of luminance contrast is preserved. At isoluminance, however, two very different chromatic influences on motion detection are revealed. First, when stimuli undergo small spatial displacements, motion is perceived in the direction of the nearest chromatically-defined border even when the sign of chromatic contrast at that border alternates over time. Under these conditions, motion detectors apparently exploit information about image borders defined by color while sacrificing information about the colors that make up those borders. By contrast, when spatial displacement is large, motion is more apt to be perceived in the direction for which sign of chromatic contrast is preserved. In this instance, information about the polarity of chromatic contrast facilitates motion detection. These results suggest that chromatic signals contributing to motion detection are of two distinct types. This conclusion has implications for the degree of crosstalk between magnocellular and parvocellular processing streams in the primate visual system and it reinforces our understanding of how image features affect the way we see things move.