Glass pattern studies of local and global processing of contrast variations.

Using Glass patterns [Nature 223 (1969) 578; Nature 246 (1973) 360; Perception 5 (1976) 67], we have studied the role of contrast differences in local and global processes of form perception. The virtue of these patterns (composed of a set of randomly distributed elements combined with a geometrically transformed copy) for studying object formation is that they allow ready isolation of local processes, the combination of dots to form a perceptual pair, from global processes, the combination of dipoles into the percept of an overall rotational or translational pattern. We find that a contrast difference within dot-pairs reduces the ability to resolve local features; large differences totally abolish the perception of the pattern. Contrast differences between dot-pairs lessen, but do not abolish, the global integration among local features. In both cases the effect is proportional to the ratio of the two contrast levels employed. Effects which differ for rotations and translations, are consistent with the greater areal integration required to resolve rotational patterns.

Cartesian and non-Cartesian responses in LGN, V1, and V2 cells.

Cell responses to drifting Cartesian (parallel) and non-Cartesian (concentric, radial, and hyperbolic) stimuli were recorded in and beyond the classical receptive field (CRF) in the lateral geniculate nucleus (LGN), V1, and V2 of anesthetized monkeys. Many cells were equally responsive to Cartesian and non-Cartesian, especially concentric, gratings. Around 15% of cells in each area were significantly more responsive to concentric compared to parallel gratings; however, cells significantly more responsive to parallel compared to concentric gratings were more numerous in the cortex. While many cells responded to hyperbolic and radial gratings, few were most responsive to these gratings. Cell selectivity decreased for Cartesian and increased for non-Cartesian gratings from V1 to V2 and the relative response varied as a function of stimulus extent with respect to the CRE. Complex, nonoriented, nondirectional cells with a low aspect ratio (AR) responded best to non-Cartesian gratings. These results cannot be fully explained using Gabor linear/energy models of simple and complex receptive fields (RFs) although such models predict some cells to respond equally to Cartesian and non-Cartesian gratings. Cells significantly more responsive to non-Cartesian gratings can be accounted for by CRF selectivity influenced by modulation from the nonclassical receptive field (nCRF). The present study shows that Cartesian/non-Cartesian selectivity is not an emergent property of V4 cells but is present at all levels of early visual processing being subserved by a subset of cells with specific tuning properties.

Spatial and temporal receptive fields of geniculate and cortical cells and directional selectivity.

The spatio-temporal receptive fields (RFs) of cells in the macaque monkey lateral geniculate nucleus (LGN) and striate cortex (V1) have been examined and two distinct sub-populations of non-directional V1 cells have been found: those with a slow largely monophasic temporal RF, and those with a fast very biphasic temporal response. These two sub-populations are in temporal quadrature, the fast biphasic cells crossing over from one response phase to the reverse just as the slow monophasic cells reach their peak response. The two sub-populations also differ in the spatial phases of their RFs. A principal components analysis of the spatio-temporal RFs of directional V1 cells shows that their RFs could be constructed by a linear combination of two components, one of which has the temporal and spatial characteristics of a fast biphasic cell, and the other the temporal and spatial characteristics of a slow monophasic cell. Magnocellular LGN cells are fast and biphasic and lead the fast-biphasic V1 subpopulation by 7 ms; parvocellular LGN cells are slow and largely monophasic and lead the slow monophasic V1 sub-population by 12 ms. We suggest that directional V1 cells get inputs in the approximate temporal and spatial quadrature required for motion detection by combining signals from the two non-directional cortical sub-populations which have been identified, and that these sub-populations have their origins in magno and parvo LGN cells, respectively.

Some transformations of color information from lateral geniculate nucleus to striate cortex.

We have recorded the responses of single cells in the lateral geniculate nucleus (LGN) and striate cortex of the macaque monkey. The response characteristics of neurons at these successive visual processing levels were examined with isoluminant gratings, cone-isolating gratings, and luminance-varying gratings. The main findings were: (i) Whereas almost all parvo- and konio-cellular LGN cells are of just two opponent-cell types, either differencing the L and M cones (L(o) and M(o) cells), or the S vs. L + M cones (S(o) cells), relatively few striate cortex simple cells show chromatic responses along these two cardinal LGN axes. Rather, most are shifted away from these LGN chromatic axes as a result of combining the outputs (or the transformed outputs) of S(o) with those of L(o) and/or M(o) cells. (ii) LGN cells on average process color information linearly, exhibiting sinusoidal changes in firing rate to isoluminant stimuli that vary sinusoidally in cone contrast as a function of color angle. Some striate cortex simple cells also give linear responses, but most show an expansive response nonlinearity, resulting in narrower chromatic tuning on average at this level. (iii) There are many more +S(o) than -S(o) LGN cells, but at the striate cortex level -S(o) input to simple cells is as common as +S(o) input. (iv) Overall, the contribution of the S-opponent path is doubled at the level of the striate cortex, relative to that at the LGN.

Contribution of S opponent cells to color appearance.

We measured the regions in isoluminant color space over which observers perceive red, yellow, green, and blue and examined the extent to which the colors vary in perceived amount within these regions. We compared color scaling of various isoluminant stimuli by using large spots, which activate all cone types, to that with tiny spots in the central foveola, where S cones, and thus S opponent (S(o)) cell activity, are largely or entirely absent. The addition of S(o) input to that from the L and M opponent cells changes the chromatic appearance of all colors, affecting each primary color in different chromatic regions in the directions and by the amount predicted by our color model. Shifts from white to the various chromatic stimuli we used produced sinusoidal variations in cone activation as a function of color angle for each cone type and in the responses of lateral geniculate cells. However, psychophysical color-scaling functions for 2 degrees spots were nonsinusoidal, being much more peaked. The color-scaling functions are well fit by sine waves raised to exponents between 1 and 3. The same is true for the color responses of a large subpopulation of striate cortex cells. The narrow color tuning, the discrepancies between the spectral loci of the peaks of the color-scaling curves and those of lateral geniculate cells, and the changes in color appearance produced by eliminating S(o) input provide evidence for a cortical processing stage at which the color axes are rotated by a combination of the outputs of S(o) cells with those of L and M opponent cells in the manner that we postulated earlier. There seems to be an expansive response nonlinearity at this stage.

Inputs to directionally selective simple cells in macaque striate cortex.

It is clear that the initial analysis of visual motion takes place in the striate cortex, where directionally selective cells are found that respond to local motion in one direction but not in the opposite direction. Widely accepted motion models postulate as inputs to directional units two or more cells whose spatio-temporal receptive fields (RFs) are approximately 90 degrees out of phase (quadrature) in space and in time. Simple cells in macaque striate cortex differ in their spatial phases, but evidence is lacking for the varying time delays required for two inputs to be in temporal quadrature. We examined the space-time RF structure of cells in macaque striate cortex and found two subpopulations of (nondirectional) simple cells, some that show strongly biphasic temporal responses, and others that are weakly biphasic if at all. The temporal impulse responses of these two classes of cells are very close to 90 degrees apart, with the strongly biphasic cells having a shorter latency than the weakly biphasic cells. A principal component analysis of the spatio-temporal RFs of directionally selective simple cells shows that their RFs could be produced by a linear combination of two components; these two components correspond closely in their respective latencies and biphasic characters to those of strongly biphasic and weakly biphasic nondirectional simple cells, respectively. This finding suggests that the motion system might acquire the requisite temporal quadrature by combining inputs from these two classes of nondirectional cells (or from their respective lateral geniculate inputs, which appear to be from magno and parvo lateral geniculate cells, respectively).

Temporal dynamics of chromatic tuning in macaque primary visual cortex.

The ability to distinguish colour from intensity variations is a difficult computational problem for the visual system because each of the three cone photoreceptor types absorb all wavelengths of light, although their peak sensitivities are at relatively short (S cones), medium (M cones), or long (L cones) wavelengths. The first stage in colour processing is the comparison of the outputs of different cone types by spectrally opponent neurons in the retina and upstream in the lateral geniculate nucleus. Some neurons receive opponent inputs from L and M cones, whereas others receive input from S cones opposed by combined signals from L and M cones. Here we report how the outputs of the L/M- and S-opponent geniculate cell types are combined in time at the next stage of colour processing, in the macaque primary visual cortex (V1). Some V1 neurons respond to a single chromatic region, with either a short (68-95 ms) or a longer (96-135 ms) latency, whereas others respond to two chromatic regions with a difference in latency of 20-30 ms. Across all types, short latency responses are mostly evoked by L/M-opponent inputs whereas longer latency responses are evoked mostly by S-opponent inputs. Furthermore, neurons with late S-cone inputs exhibit dynamic changes in the sharpness of their chromatic tuning over time. We propose that the sparse, S-opponent signal in the lateral geniculate nucleus is amplified in area V1, possibly through recurrent excitatory networks. This results in a delayed, sluggish cortical S-cone signal which is then integrated with L/M-opponent signals to rotate the lateral geniculate nucleus chromatic axes.

Hue scaling of isoluminant and cone-specific lights.

Using a hue scaling technique, we have examined the appearance of colored spots produced by shifts from white to isoluminant stimuli along various color vectors in order to examine color appearance without the complications of the combined luminance and chromatic stimulation involved in most previous hue scaling studies, which have used flashes of monochromatic light. We also used spots lying along cone-isolating vectors in order to determine what hues would be reported with a change in activation of only single cone types or of only single geniculate opponent-cell types, an issue of direct relevance to any model of color vision. We find that: 1. Unique hues do not correspond either to the change in activation of single cone types or of single geniculate opponent-cell types. This is well known to be the case for yellow and blue, but we find it to be true for red and green as well. 2. These conclusions are not limited to the particular white (Illuminant C) used as an adapting background in most of the experiments. Shifts along the same cone-contrast vectors relative to different backgrounds lead to much the same hue percepts, independent of the starting white used. 3. The shifts of the perceptual colors from the geniculate axes are in the directions, and close to the absolute amounts, predicted by our [De Valois & De Valois (1993). Vision Research, 33, 1053-1065] multi-stage color model in which we postulate that the S-opponent cells are added to or subtracted from the M- and L-opponent cells to form the four perceptual color systems. 4. There are distinct asymmetries with respect to the extent to which various hues within each perceptual opponent system deviate from the geniculate opponent-cell axes. Blue is shifted more from the S-LM axis than is yellow; green is shifted more from the L-M axis than is red. There are also asymmetries in the angular extent of opponent color regions. Blue is seen over a larger range of color vectors than is yellow, and red over a slightly larger range than green. 5. Such asymmetries are not accounted for by any model that treats red-green and yellow-blue each as unitary, mirror-image opponent-color systems. Although red and green are perceptually opponent, the red and green perceptual systems do not appear to be constructed in a mirror-image fashion with respect to input from different cone types or from different geniculate opponent-cell types. The same is true for yellow and blue.

Figural aftereffects and spatial attention.

Distracting attention away from the location of an adaptation figure reduces the positional shift of a displaced test figure in the figural aftereffect (FAE). Participants performed an alignment task after adaptation involving various manipulations of spatial attention. In 1 condition, participants counted how often numbers occurred in an alphanumeric sequence presented during adaptation. (The sequence also appeared in a comparison condition, but no attention was required.) The FAE was reduced when the alphanumeric sequence attended to was in the center of the display while the adaptation figure was 3 degrees eccentric but not when the pattern was superimposed on the adaptation figure. Forced attention to 1 feature of the adaptation figure, its orientation, did not reduce the FAE (Experiment 3). To obtain a maximum FAE, the span of attention must cover the adaptation figure.

A multi-stage color model.

The first stage of our model has three cone types, with L:M:S cones in ratios of 10:5:1. In the second stage, retinal connectivity leads to three pairs of cone-opponent, and one pair of cone-nonopponent systems. At a third (cortical) stage of color processing, the S-opponent cells are added to or subtracted from the L- and M-opponent units to split and rotate the one effective parvo geniculate response axis into separate RG and YB color axes, and separate luminance from color. We also discuss changes with eccentricity, and connectivity based on correlated neural activity.

Properties of the recombination of one-dimensional motion signals into a pattern motion signal.

We have examined the human ability to determine the direction of movement of a variety of plaid patterns. The plaids were composed of two orthogonal sine-wave gratings. When the plaid components are of unequal spatial frequency or sometimes of unequal contrast, observers judge the direction of movement incorrectly. In terms of the two-stage model of Adelson and Movshon (1982), these errors may result from either a misjudgment in the perceived speeds of each of the components or a failure in the combination of one-dimensional component movements into a coherent direction of motion of the two-dimensional plaid pattern, or both. A comparison of the perceived direction of motion of plaids with the relative perceived speeds of the plaid component gratings suggest that both failures occur, but in different circumstances. The relative perceived speed of the plaid components was measured with a spatial and a temporal forced-choice technique, the former leading to larger differences. Our results support the notion that the visual system decomposes a moving plaid into oriented components and subsequently recombines the component motions.

Vernier acuity with stationary moving Gabors.

We examined the ability of observers to determine the vertical alignment of three Gabor patches (cosine gratings tapered in X and Y by Gaussians) when the grating within the middle patch was moving right or left. The comparison patches were flickered in counterphase, as was the test patch in a control condition. In all conditions, the Gabor patch itself (the envelope) was stationary. Vernier acuity (i.e. sensitivity) was almost as good with the moving as with the flickering Gabors, but there was a very pronounced positional bias in the case of the patterns in which the internal gratings were moving. The (stationary) patches appeared to be displaced in the direction of the grating movement. Thus if the grating were drifting rightwards, the observer would see the patches as being aligned only when the test patch position in fact was shifted far over to the left. This movement-related bias increased rapidly with retinal eccentricity, reaching 15 min at 8 deg eccentricity. The bias was greatest at 4-8 Hz temporal frequency, and at low spatial frequencies. Whether the patterns were on the horizontal or the vertical meridian was largely irrelevant, but larger biases were found with patterns moving towards or away from the fovea than with those moving in a tangential direction.

Spatial localization across channels.

We have studied vernier acuity for patterns in which the stimuli to be aligned either are similar in their spatial and color characteristics or differ in these properties. The question which we address is whether spatial localization is independent of the channels being stimulated by the patterns to be aligned. We found that the precision of vernier alignment of Gabor patches was very similar irrespective of whether the patches were the same or different in spatial frequency, orientation, or color. It appears that the visual system extracts very precise location information independent of the similarity or dissimilarity of the spatio-chromatic selectivity of the channels carrying that information.

Classifying simple and complex cells on the basis of response modulation.

Hubel and Wiesel (1962; Journal of Physiology, London, 160, 106-154) introduced the classification of cortical neurons as simple and complex on the basis of four tests of their receptive field structure. These tests are partly subjective and no one of them unequivocally places neurons into distinct classes. A simple, objective classification criterion based on the form of the response to drifting sinusoidal gratings has been used by several laboratories, although it has been criticized by others. We review published and unpublished evidence which indicates that this simple and objective criterion reliability divides neurons of the striate cortex in both cats and monkeys into two groups that correspond closely to the classically-described simple and complex classes.

Spatial-frequency organization in primate striate cortex.

We measured the spatial-frequency tuning of cells at regular intervals along tangential probes through the monkey striate cortex and correlated the recording sites with the cortical cytochrome oxidase (CytOx) patterns to address three questions with regard to the cortical spatial-frequency organization. (i) Is there a periodic anatomical arrangement of cells tuned to different spatial-frequency ranges? We found there is, because the spatial-frequency tuning of cells along tangential probes changed systematically, varying from a low frequency to a middle range to high frequencies and back again repeatedly over distances of about 0.6-0.7 mm. (ii) Are there just two populations of cells, low-frequency and high-frequency units, at a given eccentricity (perhaps corresponding to the magno- and parvocellular geniculate pathways) or is there a continuum of spatial-frequency peaks? We found a continuum of peak tuning. Most cells are tuned to intermediate spatial frequencies and form a unimodal rather than a bimodal distribution of cell peaks. Furthermore, the cells with different peak frequencies were found to be continuously and smoothly distributed across a module. (iii) What is the relation between the physiological spatial-frequency organization and the regions of high CytOx concentration ("blobs")? We found a systematic correlation between the topographical variation in spatial-frequency tuning and the modular CytOx pattern, which also varied continuously in density. Low-frequency cells are at the center of the blobs, and cells tuned to increasingly higher spatial frequencies are at increasing radial distances.

Responses of simple and complex cells to random dot patterns: a quantitative comparison.

1. There are several reports that random dot patterns are potent stimuli for cortical complex cells but not for simple cells. This finding is regarded as evidence against Hubel and Wiesel's hierarchical model of cortical circuitry, in which simple cells are the principal input to complex cells. We have reinvestigated the question quantitatively by recording responses to dot patterns from 106 cells in area 17 and the 17/18 border region of normal adult cats. 2. The cells were classified as simple (n = 62) or complex (n = 40) (4 were end stopped or hypercomplex) on the basis of whether they gave modulated (AC) or unmodulated (DC) responses to drifting sine gratings. 3. Although there are large within-group differences, we found both simple and complex cells that respond to bright random dots on a dark background, drifted across the receptive field at 3 degrees/s. The responses at the optimal direction averaged 6.2 and 18.1 spikes/s (spontaneous activity subtracted) for simple and complex cells, respectively. 4. We also recorded responses to drifting sine gratings. Complex cells were also found to respond more than simple cells to these stimuli. For each cell, we calculated a dot index expressing the dot response relative to grating response. The dot index averaged 0.43 for simple cells and 0.55 for complex cells. It therefore appears that much of the difference in response to dot patterns reflects a difference in general responsivity. 5. In subsamples of cells, we examined the effects of varying dot density, dot size, and drift velocity. These variables affect different cells in a manner largely independent of cell class. Most simple cells in our sample responded well to random dot patterns at several velocities, at two different dot sizes and at both 3 and 50% dot densities. 6. Our results agree with previous studies in showing that complex cells respond more vigorously than simple cells to dot patterns, but the fact that many simple cells also respond to these stimuli makes our results consistent with a hierarchical model of cortical circuitry.

Functional anatomy of macaque striate cortex. III. Color.

Using spatially diffuse stimuli (or sinusoidal gratings of very low spatial frequency), levels of 14C-2-deoxy-d-glucose (DG) uptake produced by color-varying stimuli are much greater than those produced by luminance-varying stimuli in macaque striate cortex. Such a difference in DG results is consistent with previous psychophysical and electrophysiological results from man and monkey. In DG experiments with color-varying gratings of low and middle spatial frequencies, or with spatially diffuse color variations, DG uptake was highest in the cytochrome oxidase blobs, as was also seen with low-spatial-frequency luminance gratings. High-spatial-frequency, color-varying uptake patterns were shifted to cover both blob and interblob regions in a manner similar to that of the patterns obtained with middle-spatial-frequency luminance stimuli. However, in no instance did chromatic gratings produce uptake restricted to the interblob regions, as with the pattern seen with the highest-spatial-frequency luminance gratings. Thus, DG uptake is relatively higher in the interblob regions when comparing luminance with color-varying gratings that are otherwise similar. It was also possible to show DG evidence for receptive-field double-opponency in the upper-layer blobs, but color sensitivity in layer 4Cb appears single-opponent. The DG results suggest that color sensitivity is also high in the lower-layer (layers 5 + 6) blobs, and that many layer 5 receptive fields are double-opponent. Striate layers 4Ca and 4B-appeared color-insensitive in a wide variety of DG tests; this supports the idea of a color-insensitive stream running from the magnocellular LGN layers through striate layers 4Ca and 4B to extrastriate areas MT and V3. There was also a major effect due to wavelength: long and short wavelengths produced much more uptake than did middle wavelengths, even when all colors were equated for luminance and saturation. No variation with eccentricity was seen in cortical color sensitivity, at least between 0 degrees and 10 degrees.

Functional anatomy of macaque striate cortex. V. Spatial frequency.

When macaque monkeys view achromatic, sinusoidal gratings of a single spatial frequency, the pattern of 14C-2-deoxy-d-glucose (DG) uptake produced by the gratings is shown to depend on the spatial frequency chosen. When a relatively high (5-7 cycles/deg) spatial frequency is shown binocularly at systematically varied orientations, uptake in parafoveal striate cortex is highest between the cytochrome oxidase blobs (that is, in the interblobs) in layers 1, 2, and 3. In layers 4B, 5, and 6, where the cytochrome oxidase blobs are faint or absent, DG uptake is highest in a periodic pattern that lies in register with the interblobs of layers 2 + 3. When the grating is, instead, of relatively low (1-1.5 cycles/deg) spatial frequency, DG uptake is highest in the blobs, in the blob-aligned portions of layers 1-4B, and in the lower-layer blobs as well. These variations in DG topography are confirmed in stimulus comparisons within a single hemisphere. Presumably, this shift in functional topography within the extra-granular layer is the primate homolog of "spatial frequency columns" shown earlier in the cat (Tootell et al., 1981; Silverman, 1984). In the well-differentiated architecture of primate striate cortex, laminar differences produced by high- versus low-spatial-frequency gratings are visible as well. Gratings of very high spatial frequency produce much higher uptake in 4Cb (which receives input from the parvocellular LGN layers) than in 4Ca (which gets its input from the magnocellular LGN layers). Gratings of low spatial frequency produce the converse result. Presumably, cells in the magnocellular LGN layers and/or in the magnocellular-dominated layer 4Ca have lower average spatial frequency tuning (larger receptive fields) than their counterparts in the parvocellular LGN and/or in striate layer 4Cb. The DG patterns produced by various spatial frequencies also vary with eccentricity, in a manner consistent with known, eccentricity-dependent variations of receptive-field size and spatial frequency tuning. Thus, gratings of a "middle"-spatial-frequency range (4-5 cycles/deg) produce high uptake in the blobs near the foveal representation and high uptake in the interblobs at more peripheral eccentricities, including 5 degrees. This shift in DG topography also includes the transition zone near 3 degrees, where the level of stimulus-driven uptake is as high in the blob regions as it is in interblob regions. Variations in uptake between layers 4Ca and 4Cb, as a function of eccentricity, shift in parallel with the changes in the upper-layer topography.

Temporal properties of brightness and color induction.

With a matching procedure, we studied the temporal properties of direct brightness (or lightness) and chromatic changes (produced by modulation of the region being matched) and induced brightness and chromatic changes (produced by modulation of the surround of the region being matched). The amount of direct brightness and color change was found to vary only slightly with temporal frequency over the 0.5-8 Hz range studied, whereas induced changes were found to occur only at low temporal frequencies, below about 2.5 Hz. With high temporal-frequency modulation of the surround, the induced patterns appeared to flicker but not to change in brightness or color. Despite the fact that chrominance and luminance temporal contrast sensitivity functions are very different, the temporal induction curves for color and brightness were very similar. However, brightness induction was found to increase approximately linearly with increasing surround modulation up to very high levels, whereas the amount of color induction was much less dependent on the modulation depth of the surround.