Visual and auditory cue integration for the generation of saccadic eye movements in monkeys and lever pressing in humans.

This study examined how effectively visual and auditory cues can be integrated in the brain for the generation of motor responses. The latencies with which saccadic eye movements are produced in humans and monkeys form, under certain conditions, a bimodal distribution, the first mode of which has been termed express saccades. In humans, a much higher percentage of express saccades is generated when both visual and auditory cues are provided compared with the single presentation of these cues [H. C. Hughes et al. (1994) J. Exp. Psychol. Hum. Percept. Perform., 20, 131-153]. In this study, we addressed two questions: first, do monkeys also integrate visual and auditory cues for express saccade generation as do humans and second, does such integration take place in humans when, instead of eye movements, the task is to press levers with fingers? Our results show that (i) in monkeys, as in humans, the combined visual and auditory cues generate a much higher percentage of express saccades than do singly presented cues and (ii) the latencies with which levers are pressed by humans are shorter when both visual and auditory cues are provided compared with the presentation of single cues, but the distribution in all cases is unimodal; response latencies in the express range seen in the execution of saccadic eye movements are not obtained with lever pressing.

New methods devised specify the size and color of the spots monkeys see when striate cortex (area V1) is electrically stimulated.

Creating a prosthetic device for the blind is a central future task. Our research examines the feasibility of producing a prosthetic device based on electrical stimulation of primary visual cortex (area V1), an area that remains intact for many years after loss of vision attributable to damage to the eyes. As an initial step in this effort, we believe that the research should be carried out in animals, as it has been in the creation of the highly successful cochlear implant. We chose the rhesus monkey, whose visual system is similar to that of man. We trained monkeys on two tasks to assess the size, contrast, and color of the percepts created when single sites in area V1 are stimulated through microelectrodes. Here, we report that electrical stimulation within the central 5° of the visual field representation creates a small spot that is between 9 and 26 min of arc in diameter and has a contrast ranging between 2.6% and 10%. The dot generated by the stimulation in the majority of cases was darker than the background viewed by the animal and was composed of a variety of low-contrast colors. These findings can be used as inputs to models of electrical stimulation in area V1. On the basis of these findings, we derive what kinds of images would be expected when implanted arrays of electrodes are stimulated through a camera attached to the head whose images are converted into electrical stimulation using appropriate algorithms.

The integration of disparity, shading and motion parallax cues for depth perception in humans and monkeys.

A visual stimulus display was created that enabled us to examine how effectively the three depth cues of disparity, motion parallax and shading can be integrated in humans and monkeys. The display was designed to allow us to present these three depth cues separately and in various combinations. Depth was processed most effectively and most rapidly when all three cues were presented together indicating that these separate cues are integrated at yet unknown sites in the brain. Testing in humans and monkeys yielded similar results suggesting that monkeys are a good animal model for the study of the underlying neural mechanisms of depth perception.

Parallel information processing channels created in the retina.

In the retina, several parallel channels originate that extract different attributes from the visual scene. This review describes how these channels arise and what their functions are. Following the introduction four sections deal with these channels. The first discusses the "ON" and "OFF" channels that have arisen for the purpose of rapidly processing images in the visual scene that become visible by virtue of either light increment or light decrement; the ON channel processes images that become visible by virtue of light increment and the OFF channel processes images that become visible by virtue of light decrement. The second section examines the midget and parasol channels. The midget channel processes fine detail, wavelength information, and stereoscopic depth cues; the parasol channel plays a central role in processing motion and flicker as well as motion parallax cues for depth perception. Both these channels have ON and OFF subdivisions. The third section describes the accessory optic system that receives input from the retinal ganglion cells of Dogiel; these cells play a central role, in concert with the vestibular system, in stabilizing images on the retina to prevent the blurring of images that would otherwise occur when an organism is in motion. The last section provides a brief overview of several additional channels that originate in the retina.

Conditions that alter saccadic eye movement latencies and affect target choice to visual stimuli and to electrical stimulation of area V1 in the monkey.

In this study, we examined procedures that alter saccadic latencies and target selection to visual stimuli and electrical stimulation of area V1 in the monkey. It has been shown that saccadic eye movement latencies to singly presented visual targets form a bimodal distribution when the fixation spot is turned off a number of milliseconds prior to the appearance of the target (the gap period); the first mode has been termed express saccades and the second regular saccades. When the termination of the fixation spot is coincident with the appearance of the target (0 ms gap), express saccades are rarely generated. We show here that a bimodal distribution of saccadic latencies can also be obtained when an array of visual stimuli is presented prior to the appearance of the visual target, provided the elements of the array overlap spatially with the visual target. The overall latency of the saccadic eye movements elicited by electrical stimulation of area V1 is significantly shortened both when a gap is introduced between the termination of the fixation spot and the stimulation and when an array is presented. However, under these conditions, the distribution of saccadic latencies is unimodal. When two visual targets are presented after the fixation spot, introducing a gap has no effect on which target is chosen. By contrast, when electrical stimulation is paired with a visual target, introducing a gap greatly increases the frequency with which the electrical stimulation site is chosen.

Visual prosthesis.

There are more than forty million blind individuals in the world whose plight would be greatly ameliorated by creating a visual prosthesis. We begin by outlining the basic operational characteristics of the visual system, as this knowledge is essential for producing a prosthetic device based on electrical stimulation through arrays of implanted electrodes. We then list a series of tenets that we believe need to be followed in this effort. Central among these is our belief that the initial research in this area, which is in its infancy, should first be carried out on animals. We suggest that implantation of area V1 holds high promise as the area is of a large volume and can therefore accommodate extensive electrode arrays. We then proceed to consider coding operations that can effectively convert visual images viewed by a camera to stimulate electrode arrays to yield visual impressions that can provide shape, motion, and depth information. We advocate experimental work that mimics electrical stimulation effects non-invasively in sighted human subjects with a camera from which visual images are converted into displays on a monitor akin to those created by electrical stimulation.

The effect of overall stimulus velocity on motion parallax.

This study examined the effectiveness with which motion parallax information can be utilized by rhesus monkeys for depth perception. A visual display comprised of random-dots that mimicked a rigid, three-dimensional object rocking back and forth was used. Differential depth was produced by presenting sub-regions of the dots moving at different velocities from the rest of dots in the display. The tasks for the monkeys were to detect or discriminate a target region that was protruding the furthest from the background plane. To understand the role of stimulus movement, we examined the accuracy and the rapidity of the saccadic responses as a function of rocking velocity of the entire three-dimensional object. The results showed that performance accuracy improved and reaction times decreased with increasing rocking velocities. The monkeys can process the motion parallax information with remarkable rapidity such that the average reaction time ranged between 212 and 246 milliseconds. The data collected suggest that the successive activation of just two sets of cones is sufficient to perform the task.

How the parallel channels of the retina contribute to depth processing.

Reconstructing the third dimension in the visual scene from the two dimensional images that impinge on the retinal surface is one of the major tasks of the visual system. We have devised a visual display that makes it possible to study stereoscopic depth cues and motion parallax cues separately or in concert using rhesus macaques. By varying the spatial frequency of the display and its luminance and chrominance, it is possible to selectively activate channels that originate in the primate retina. Our results show that (i) the parasol system plays a central role in processing motion parallax cues; (ii) the midget system plays a central role in stereoscopic depth perception at high spatial frequencies, and (iii) red/green colour selective neurons can effectively process both cues but blue/yellow neurons cannot do so.

Depth from shading and disparity in humans and monkeys.

A stimulus display was devised that enabled us to examine how effectively monkeys and humans can process shading and disparity cues for depth perception. The display allowed us to present these cues separately, in concert and in conflict with each other. An oddities discrimination task was used. Humans as well as monkeys were able to utilize both shading and disparity cues but shading cues were more effectively processed by humans. Humans and monkeys performed better and faster when the two cues were presented conjointly rather than singly. Performance was significantly degraded when the two cues were presented in conflict with each other suggesting that these cues are processed interactively at higher levels in the visual system. The fact that monkeys can effectively utilize depth information derived from shading and disparity indicates that they are a good animal model for the study of the neural mechanisms that underlie the processing of these two depth cues.

Demonstrations of spatiotemporal integration and what they tell us about the visual system.

Five sets of displays are presented on the journal website to be viewed in conjunction with the text. We concentrate on the factors that give rise to the integration and disruption of the direction of apparent motion in two-dimensional and three-dimensional space. In the first set of displays we examine what factors contribute to the integration and disruption of apparent motion in the Ramachandran/Anstis clustered bistable quartets. In the second set we examine what factors give rise to the perception of the direction of motion in rotating two-dimensional wheels and dots. In the third and fourth sets we examine how the depth cues of shading and disparity contribute to the perception of apparent motion of opaque displays, and to the perception of rotating unoccluded displays, respectively. In the fifth set we examine how the depth cue of motion parallax influences the perception of apparent motion. Throughout, we make inferences about the roles which various parallel pathways and cortical areas play in the perceptions produced by the displays shown.

The Hermann grid illusion revisited.

The Hermann grid illusion consists of smudges perceived at the intersections of a white grid presented on a black background. In 1960 the effect was first explained by a theory advanced by Baumgartner suggesting the illusory effect is due to differences in the discharge characteristics of retinal ganglion cells when their receptive fields fall along the intersections versus when they fall along non-intersecting regions of the grid. Since then, others have claimed that this theory might not be adequate, suggesting that a model based on cortical mechanisms is necessary [Lingelbach et al, 1985 Perception 14(1) A7; Spillmann, 1994 Perception 23 691 708; Geier et al, 2004 Perception 33 Supplement, 53; Westheimer, 2004 Vision Research 44 2457 2465]. We present in this paper the following evidence to show that the retinal ganglion cell theory is untenable: (i) varying the makeup of the grid in a manner that does not materially affect the putative differential responses of the ganglion cells can reduce or eliminate the illusory effect; (ii) varying the grid such as to affect the putative differential responses of the ganglion cells does not eliminate the illusory effect; and (iii) the actual spatial layout of the retinal ganglion cell receptive fields is other than that assumed by the theory. To account for the Hermann grid illusion we propose an alternative theory according to which the illusory effect is brought about by the manner in which S1 type simple cells (as defined by Schiller et al, 1976 Journal of Neurophysiology 39 1320-1333) in primary visual cortex respond to the grid. This theory adequately handles many of the facts delineated in this paper.

Mapping cortical activity elicited with electrical microstimulation using FMRI in the macaque.

Over the last two centuries, electrical microstimulation has been used to demonstrate causal links between neural activity and specific behaviors and cognitive functions. However, to establish these links it is imperative to characterize the cortical activity patterns that are elicited by stimulation locally around the electrode and in other functionally connected areas. We have developed a technique to record brain activity using the blood oxygen level dependent (BOLD) signal while applying electrical microstimulation to the primate brain. We find that the spread of activity around the electrode tip in macaque area V1 was larger than expected from calculations based on passive spread of current and therefore may reflect functional spread by way of horizontal connections. Consistent with this functional transynaptic spread we also obtained activation in expected projection sites in extrastriate visual areas, demonstrating the utility of our technique in uncovering in vivo functional connectivity maps.

Delaying visually guided saccades by microstimulation of macaque V1: spatial properties of delay fields.

Electrical microstimulation of macaque primary visual cortex (area V1) is known to delay the execution of saccadic eye movements made to a punctate visual target placed into the receptive field of the stimulated neurons. We examined the spatial extent of this delay effect, which we call a delay field, by placing a 0.2 degrees visual target at various locations relative to the receptive field of the stimulated neurons and by stimulating different sites within the operculum of V1. A 100-ms train of stimulation consisting of current pulses at or less than 100 microA was delivered immediately before monkeys generated a saccadic eye movement to the visual target. The region of tissue activated was within 0.5 mm from the electrode tip. The depth of stimulation for a given site ranged from 0.9 to 2.0 mm below the cortical surface. The location of the receptive fields of the stimulated neurons ranged from 1.8 to 4.4 degrees of eccentricity from the center of gaze. Within this range, the size of the delay field increased from 0.1 to 0.55 degrees of visual angle. The shape of the field was roughly circular. The size of the delay field increased as the stimulation site was located further from the foveal representation of V1. These results are consistent with the finding that phosphenes evoked by electrical stimulation of human V1 are circular and increase in size as the stimulating electrode is placed more distant from the foveal representation of V1.

Neural mechanisms underlying target selection with saccadic eye movements.

In exploring the visual scene we make about three saccadic eye movements per second. During each fixation, in addition to analyzing the object at which we are looking, a decision has to be made as to where to look next. Although we perform this task with the greatest of ease, the computations to perform the task are complex and involve numerous brain structures. We have applied several investigative tools that include single-cell recordings, microstimulation, pharmacological manipulations and lesions to learn more about the neural control of visually guided eye saccadic movements. Electrical stimulation of the superior colliculus (SC), areas V1 and V2, the lateral intraparietal sulcus (LIP), the frontal eye fields (FEF) and the medial eye fields (MEF) produces saccadic eye movements at low current levels. After ablation of the SC, electrical microstimulation of V1, V2, and LIP no longer elicits saccadic eye movements whereas stimulation of the FEF and MEF continues to be effective. Ablation of the SC but not of the FEF eliminates short-latency saccadic eye movements to visual targets called "express saccades," whereas lesions of the FEF selectively interfere with target selection. Bilateral removal of both the SC and the FEF causes major, long lasting deficits: all visually elicited saccadic eye movements are eliminated. In intact monkeys, subthreshold electrical microstimulation of the FEF and MEF as well as the lower layers of V1 and V2 and of some subregions of LIP greatly facilitates the choice of targets presented in the receptive fields of the stimulated neurons. By contrast, stimulation of the upper layers of V1 and V2 and other sub-regions of LIP produces a dramatic interference in target selection. Examination of the role of inhibitory circuits in eye-movement generation reveals that local infusion of muscimol, a GABA (gamma-aminobutyric acid) agonist, or bicuculline, a GABA antagonist, interferes with target selection in V1. On the other hand, infusion of bicuculline into the FEF produces facilitation in target choice and irrepressible saccades. It appears therefore that inhibitory circuits play a central role in visual analysis in V1 and in the generation of saccadic eye movements in the FEF. It is proposed that two major streams can be discerned in visually guided eye-movement control, the posterior from occipital and parietal cortex that reaches the brainstem via the SC and the anterior from the FEF and MEF that has direct access to the brainstem oculomotor centers.

What is the coordinate frame utilized for the generation of express saccades in monkeys?

The latencies of saccades to suddenly appearing eccentric targets can have a bimodal distribution, with an early, express peak, and a late, regular peak (Fischer and Boch 1983, Brain Res 260: 21-26). Express saccades usually are a product of learning. The purpose of this study was to determine whether this learning is specific to the relative position of the target in space, the orbital position of the eye, or the vector of the saccade to be produced. Further, it was asked whether and how the frequency with which express saccades are generated is influenced by the immediately preceding saccadic vector and the familiarity of the targets. To this end, rhesus monkeys were trained to make saccadic eye movements to single targets and to two sequential targets that appeared at various positions relative to the head, relative to the initial fixation spot and relative to each other. The results show that the frequency with which express saccades are generated is determined by the saccadic vector that has to be generated and not by the relative position of a target in space, the orbital position of the eye, the immediately preceding saccadic vector, or the familiarity of the targets.

Are express saccades generated under natural viewing conditions?

To assess whether express saccades are generated under everyday conditions, we collected eye movement data from Rhesus monkeys engaged in free viewing under a variety of conditions. The durations of the fixation periods that occurred between saccades were calculated. The results show that while short-duration fixations within the range of express saccades occur quite commonly, the overall distribution is unimodal. This is the case even when all the object elements in the visual scene have the same contrast. The findings suggest that while saccades that fall within the express range occur commonly under natural viewing conditions, bimodal distributions of saccadic latencies are obtainable only under laboratory conditions.

Microstimulation of V1 delays the execution of visually guided saccades.

Electrical stimulation delivered to V1 concurrently with the presentation of a visual target interferes with both the selection and the detection of targets positioned in the receptive field of the stimulated neurons. In the present study, we examined the temporal course of this effect by delivering electrical stimulation to V1 of rhesus monkeys at various times before the appearance of a visual target. Each trial was initiated by the appearance of a fixation spot that, once acquired, was followed by the presentation of a visual target in the receptive field of the stimulated neurons. A monkey was reward after making a saccadic eye movement to the target. A delay in saccade generation was obtained when stimulation was delivered while an animal maintained fixation on the fixation spot. No delay occurred when the visual target was placed outside the receptive field of the stimulated neurons. The best parameters for inducing the saccadic delay were: (i). anode-first pulses (as opposed to cathode-first pulses) and (ii). train durations greater than 40 ms and frequencies greater than 100 Hz. The lowest current threshold for producing a saccadic delay occurred at 1.5 mm below the top of superficial V1. The chronaxies of the directly stimulated elements mediating the delay ranged from 0.13 to 0.24 ms. These values overlap with those that have been described for phosphene induction in human V1. We discuss how the elements mediating the saccadic delay might interrupt a visual signal as it passes along the geniculostriate pathway.

An examination of the variables that affect express saccade generation.

The frequency with which express saccades are generated under a variety of conditions in rhesus monkeys was examined. Increasing the gap time between fixation spot termination and target onset increased express saccade frequency but was progressively less effective in doing so as the number of target positions in the sample was increased. Express saccades were rarely produced when two targets were presented simultaneously and the choice of either of which was rewarded; a temporal asynchrony of only 17 ms between the targets reinstated express saccade generation. Express saccades continued to be generated when the vergence or pursuit systems was coactivated with the saccadic system.

How do target predictability and precueing affect the production of express saccades in monkeys?

The extent to which target predictability and precueing affect express saccade generation was determined in Rhesus monkeys. Target predictability, as manipulated by the probability with which targets appeared at various locations, had a strong influence on express saccade generation. Pre-cueing the location of the appearance of an impending single target with an identical stimulus was effective in increasing express saccade generation when there was a gap of 50-150 ms between fixation spot termination and target onset. However, precueing was not effective when the gap time was set to 0 ms in the single target task, when several simultaneous targets appeared requiring a visual discrimination to be made using an oddity task, or when the precue was not identical to the target. These findings indicate that express saccades are facilitated by a restricted set of conditions that increase the predictability of target location and identity.