Standardized automated training of rhesus monkeys for neuroscience research in their housing environment.

Teaching nonhuman primates the complex cognitive behavioral tasks that are central to cognitive neuroscience research is an essential and challenging endeavor. It is crucial for the scientific success that the animals learn to interpret the often complex task rules and reliably and enduringly act accordingly. To achieve consistent behavior and comparable learning histories across animals, it is desirable to standardize training protocols. Automatizing the training can significantly reduce the time invested by the person training the animal. In addition, self-paced training schedules with individualized learning speeds based on automatic updating of task conditions could enhance the animals' motivation and welfare. We developed a training paradigm for across-task unsupervised training (AUT) of successively more complex cognitive tasks to be administered through a stand-alone housing-based system optimized for rhesus monkeys in neuroscience research settings (Calapai A, Berger M, Niessing M, Heisig K, Brockhausen R, Treue S, Gail A. Behav Res Methods 5: 1-11, 2016). The AUT revealed interindividual differences in long-term learning progress between animals, helping to characterize learning personalities, and commonalities, helping to identify easier and more difficult learning steps in the training protocol. Our results demonstrate that 1) rhesus monkeys stay engaged with the AUT over months despite access to water and food outside the experimental sessions but with lower numbers of interaction compared with conventional fluid-controlled training; 2) with unsupervised training across sessions and task levels, rhesus monkeys can learn tasks of sufficient complexity for state-of-the-art cognitive neuroscience in their housing environment; and 3) AUT learning progress is primarily determined by the number of interactions with the system rather than the mere exposure time. NEW & NOTEWORTHY We demonstrate that highly structured training of behavioral tasks, as used in neuroscience research, can be achieved in an unsupervised fashion over many sessions and task difficulties in a monkey housing environment. Employing a predefined training strategy allows for an observer-independent comparison of learning between animals and of training approaches. We believe that self-paced standardized training can be utilized for pretraining and animal selection and can contribute to animal welfare in a neuroscience research environment.

A cage-based training, cognitive testing and enrichment system optimized for rhesus macaques in neuroscience research.

In neurophysiological studies with awake non-human primates (NHP), it is typically necessary to train the animals over a prolonged period of time on a behavioral paradigm before the actual data collection takes place. Rhesus monkeys (Macaca mulatta) are the most widely used primate animal models in system neuroscience. Inspired by existing joystick- or touch-screen-based systems designed for a variety of monkey species, we built and successfully employed a stand-alone cage-based training and testing system for rhesus monkeys (eXperimental Behavioral Intrument, XBI). The XBI is mobile and easy to handle by both experts and non-experts; animals can work with only minimal physical restraints, yet the ergonomic design successfully encourages stereotypical postures with a consistent positioning of the head relative to the screen. The XBI allows computer-controlled training of the monkeys with a large variety of behavioral tasks and reward protocols typically used in systems and cognitive neuroscience research.

Neural correlates of attention in primate visual cortex.

The processing of visual information combines bottom-up sensory aspects with top-down influences, most notably attentional processes. Attentional influences have now been demonstrated throughout visual cortex, and their influence on the processing of visual information is profound. Neuronal responses to attended locations or stimulus features are enhanced, whereas those from unattended locations or features are suppressed. This influence of attention increases as one ascends the hierarchy of visual areas in primate cortex, ultimately resulting in a neural representation of the visual world that is dominated by the behavioral relevance of the information, rather than designed to provide an accurate and complete description of it. This realization has led to a rethinking of the role of areas that have previously been considered to be "purely sensory".

Different populations of neurons contribute to the detection and discrimination of visual motion.

The signal-to-noise ratio of a direction-selective neuron for 'detecting' visual motion is highest when the motion direction is close to the neuron's preferred direction. But because these neurons show a bell-shaped tuning for direction, they have the highest signal-to-noise ratio for 'discriminating' the direction of motion when their preferred direction is off the direction to be discriminated. In this paper, we demonstrate with an adaptation paradigm that the visual system shows a corresponding task-specific ability to select neurons depending on whether it is performing a detection or a discrimination task, relying preferentially on different neuronal populations in the two tasks. Detection is based on neuronal populations tuned to the test direction, while direction discrimination is based on neurons preferring directions 40-60 degrees off the test direction.

Image segmentation: a tug-of-war for the eyeball.

Separating objects from their background is one of the central abilities of the visual system. Recent evidence has revealed how populations of neurons, some of which have receptive fields with an antagonistic center-surround structure, and some of which do not, might contribute to this ability.

Seeing multiple directions of motion-physiology and psychophysics.

Dot patterns sliding transparently across one another are normally perceived as independently moving surfaces. Recordings from direction-selective neurons in area MT of the macaque suggested that this perceptual segregation did not depend on the presence of two peaks in the population activity. Rather, the visual system seemed to use overall shape of the population response to determine the number and directions of motion components. This approach explained a number of perceptual phenomena, including susceptibility of the motion system to direction metamers, motion patterns combining three or five directions incorrectly perceived by subjects as comprising only two directions. Our findings offer insights into the coding of multi-valued sensory signals and provide constraints for biologically based computational models.

Absence of a common functional denominator of visual disturbances in cerebellar disease.

Several studies have demonstrated disturbances of visual perception in patients suffering from cerebellar disease. In an attempt to determine the cause of these visual disturbances and thereby the cerebellar contribution to vision, we designed two sets of experiments in which we tested (i) the possibility of a general magnocellular deficit in cerebellar disease and (ii) the alternative possibility of impaired spatial attention underlying visual disturbances in cerebellar patients. The first set of experiments consisted of a test of position discrimination, a parvocellular function and tests tapping different aspects of motion perception including speed discrimination, direction discrimination and the ability to extract a coherent motion signal embedded in noise. The second set of experiments compared the performance on two different classes of texture discrimination. The first one required fast and precise shifts of focal spatial attention ('serial search'), the second one, testing preattentive texture discrimination ('pop-out'), did not. In the first set of experiments cerebellar patients were impaired on the position discrimination task as well as several, albeit not all, tests of motion perception. The pattern of disturbances obtained was neither compatible with the notion of a selective magnocellular deficit nor the idea, originally put forward by Ivry and Diener (J Cogn Neurosci 1991; 3: 355-66) that visual deficits are secondary to an impaired measurement of time. In the second set of experiments, cerebellar patients showed normal performance on pop-out tasks and normal performance on all variants of the serial search task except for the one requiring comparison of a single element presented with a sample of the target in short-term memory. In summary, our results support the existence of visual disturbances in cerebellar disease, but provide evidence against a common, simple denominator such as a timing deficit, deficient cerebellar modulation of magnocellular circuitry, deficits of spatial attention or visual working memory.

Effects of attention on the processing of motion in macaque middle temporal and medial superior temporal visual cortical areas.

The visual system is continually inundated with information received by the eyes. Only a fraction of this information appears to reach visual awareness. This process of selection is one of the functions ascribed to visual attention. Although many studies have investigated the role of attention in shaping neuronal representations in cortical areas, few have focused on attentional modulation of neuronal signals related to visual motion. We recorded from 89 direction-selective neurons in middle temporal (MT) and medial superior temporal (MST) visual cortical areas of two macaque monkeys using identical sensory stimulation under various attentional conditions. Neural responses in both areas were greatly influenced by attention. When attention was directed to a stimulus inside the receptive field of a neuron, responses in MT and MST were enhanced an average of 20 and 40% compared with a condition in which attention was directed outside the receptive field. Even stronger average enhancements (70% in MT and 100% in MST) were observed when attention was switched from a stimulus moving in the nonpreferred direction inside the receptive field to another stimulus in the receptive field that was moving in the preferred direction. These findings show that attention modulates motion processing from stages early in the dorsal visual pathway by selectively enhancing the representation of attended stimuli and simultaneously reducing the influence of unattended stimuli.

Feature-based attention influences motion processing gain in macaque visual cortex.

Changes in neural responses based on spatial attention have been demonstrated in many areas of visual cortex, indicating that the neural correlate of attention is an enhanced response to stimuli at an attended location and reduced responses to stimuli elsewhere. Here we demonstrate non-spatial, feature-based attentional modulation of visual motion processing, and show that attention increases the gain of direction-selective neurons in visual cortical area MT without narrowing the direction-tuning curves. These findings place important constraints on the neural mechanisms of attention and we propose to unify the effects of spatial location, direction of motion and other features of the attended stimuli in a 'feature similarity gain model' of attention.

Revisiting motion repulsion: evidence for a general phenomenon?

Previous studies have found large misperceptions when subjects are reporting the perceived angle between two directions of motion moving transparently at an acute angle, the so called motion repulsion. While these errors have been assumed to be caused by interactions between the two directions present, we reassessed these earlier measurements taking into account recent findings about directional misperceptions affecting the perception of single motion (reference repulsion). While our measurements confirm that errors in directional judgments of transparent motions can indeed be as big as 22 degrees we find that motion repulsion, i.e. the interaction between two directions, contributes at most about 7 degrees to these errors. This value is comparable to similar repulsion effects in orientation perception and stereoscopic depth perception, suggesting that they share a common neural basis. Our data further suggest that fast time scale adaptation and/or more general interactions between neurons contribute to motion repulsion while tracking eye movements play little or no role. These findings should serve as important constraints for models of motion perception.

Reference repulsion when judging the direction of visual motion.

While humans are very reliable (i.e. give highly reproducible answers) when repeatedly judging the direction of a moving random-dot pattern (RDP) we find that their accuracy (i.e. the direction they so reliably report) shows systematic errors. To quantify these errors, we presented a complete set of closely spaced directions and mapped the directional misjudgments by asking subjects to compare the perceived direction of a moving RDP with the direction of a test line. The results show misjudgments of up to 9 degrees, which are best accounted for by a tendency of the subjects to overestimate the angle between the observed motion and an internal reference direction. A control experiment in which subjects had to judge the spatial distance between a point and a line demonstrates that these misjudgments are not confined to motion stimuli but rather seem to reflect a general tendency to overestimate the distance between a stimulus and a reference when they are close to each other.

Attentional modulation of visual motion processing in cortical areas MT and MST.

The visual system is constantly inundated with information received by the eyes, only a fraction of which seems to reach visual awareness. This selection process is one of the functions ascribed to visual attention. Although many studies have investigated the role of attention in shaping neuronal representations in the visual cortex, few have focused on attentional modulation of neuronal signals related to visual motion. Here we report that the responses of direction-selective neurons in monkey visual cortex are greatly influenced by attention, and that this modulation occurs as early in the cortical hierarchy as the level of the middle temporal visual area (MT). Our finding demonstrates a stronger and earlier influence of attention on motion processing along the dorsal visual pathway than previously recognized.

Neural responses to velocity gradients in macaque cortical area MT.

Visual motion, i.e. the pattern of changes on the retinae caused by the motion of objects or the observer through the environment, contains important cues for the accurate perception of the three-dimensional layout of the visual scene. In this study, we investigate if neurons in the visual system, specifically in area MT of the macaque monkey, are able to differentiate between various velocity gradients. Our stimuli were random dot patterns designed to eliminate stimulus variables other than the orientation of a velocity gradient. We develop a stimulus space ("deformation space") that allows us to easily parameterize our stimuli. We demonstrate that a substantial proportion of MT cells show tuned responses to our various velocity gradients, often exceeding the response evoked by an optimized flat velocity profile. This suggests that MT cells are able to represent complex aspects of the visual environment and that their properties make them well suited as building blocks for the complex receptive field properties encountered in higher areas, such as area MST to which many cells in area MT project.

Recovering three-dimensional structure from motion with surface reconstruction.

This paper addresses the computational role that the construction of a complete surface representation may play in the recovery of 3-D structure from motion. We first discuss the need to integrate surface reconstruction with the structure-from-motion process, both on computational and perceptual grounds. We then present a model that combines a feature-based structure-from-motion algorithm with a smooth surface interpolation mechanism. This model allows multiple surfaces to be represented in a given viewing direction, incorporates constraints on surface structure from object boundaries, and segregates image features onto multiple surfaces on the basis of their 2-D image motion. We present the results of computer simulations that relate the qualitative behavior of this model to psychophysical observations. In a companion paper, we discuss further perceptual observations regarding the possible role of surface reconstruction in the human recovery of 3-D structure from motion.

Structure-from-motion: perceptual evidence for surface interpolation.

Dynamic random-dot displays representing a rotating cylinder were used to investigate surface interpolation in the perception of structure-from-motion (SFM) in humans. Surface interpolation refers to a process in which a complete surface in depth is reconstructed from the object depth values extracted at the stimulus features. Surface interpolation will assign depth values even in parts of the object that contain no features. Such a "fill-in" process should make the detection of featureless stimulus areas ("holes") difficult. Indeed, we demonstrate that such holes in our rotating cylinder can be as wide as one-quarter of the stimulus before subjects can reliably detect their presence. Subjects were presented with a variation on the rotating cylinder in which all dots were oscillating either in synchrony or asynchronously. Subjects perceive a rigidly rotating cylinder even when such a percept is not in agreement with the physical stimulus. To reconcile this discrepancy between actual and perceived stimulus we propose that individual points contribute to a surface based object representation and that in this process the visual system looses access to the identity of the individual features that make up the surface. Finally we are able to explain a variety of previously documented perceptual peculiarities in the perception of structure-from-motion by arguing that the perceptual interpretation of the object's boundaries influences the surface interpolation process. These findings offer strong perceptual evidence for a process of surface interpolation and are also physiologically plausible given results from recordings in awake behaving monkey cortical areas V1 and MT. The companion paper demonstrates how such a surface interpolation process can be incorporated into a structure-from-motion algorithm and how object boundaries can influence the perception of structure-from-motion as has been demonstrated before and in this paper.

The effect of transiency on perceived velocity of visual patterns: a case of "temporal capture".

We measured the points of subjective equality of velocity for dynamic unidirectionally moving random-dot patterns with different amounts of transiency. The transiency was changed by varying the time a dot would move before being randomly replotted within the stimulus. The perceived velocity of patterns moving at intermediate velocities (4 or 6 deg/sec) was increased by decreasing the point lifetime while no speedup was observed at high velocities (12 deg/sec). A speedup was also observed when a few stationary points of short lifetime were introduced into a stimulus. The non-directional transiency generated by these flickering points seems to be captured by the moving pattern and biases the velocity estimate. We term this phenomenon "temporal capture". The results are in agreement with models that determine velocity by comparing the activity in lower and higher temporal frequency channels. Our stimuli would selectively increase activity in high temporal frequency channels and thus lead to an increase in perceived velocity.

The response of neurons in areas V1 and MT of the alert rhesus monkey to moving random dot patterns.

We studied the response of single units to moving random dot patterns in areas V1 and MT of the alert macaque monkey. Most cells could be driven by such patterns; however, many cells in V1 did not give a consistent response but fired at a particular point during stimulus presentation. Thus different dot patterns can produce a markedly different response at any particular time, though the time averaged response is similar. A comparison of the directionality of cells in both V1 and MT using random dot patterns shows the cells of MT to be far more directional. In addition our estimates of the percentage of directional cells in both areas are consistent with previous reports using other stimuli. However, we failed to find a bimodality of directionality in V1 which has been reported in some other studies. The variance associated with response was determined for individual cells. In both areas the variance was found to be approximately equal to the mean response, indicating little difference between extrastriate and striate cortex. These estimates are in broad agreement (though the variance appears a little lower) with those of V1 cells of the anesthetized cat. The response of MT cells was simulated on a computer from the estimates derived from the single unit recordings. While the direction tuning of MT cells is quite wide (mean half-width at half-height approximately 50 degrees) it is shown that the cells can reliably discriminate much smaller changes in direction, and the performance of the cells with the smallest discriminanda were comparable to thresholds measured with human subjects using the same stimuli (approximately 1.1 degrees). Minimum discriminanda for individual cells occurred not at the preferred direction, that is, the peak of their tuning curves, but rather on the steep flanks of their tuning curves. This result suggests that the cells which may mediate the discrimination of motion direction may not be the cells most sensitive to that direction.

The response of area MT and V1 neurons to transparent motion.

An important use of motion information is to segment a complex visual scene into surfaces and objects. Transparent motions present a particularly difficult problem for segmentation because more than one velocity vector occurs at each local region in the image, and current machine vision systems fail in these circumstances. The fact that motion transparency is prevalent in natural scenes, and yet artificial systems display an inability to analyze it, suggests that the primate visual system has developed specialized methods for perceiving transparent motion. Also, the currently prevalent model of physiological mechanisms for motion-direction selectivity employs inhibitory interactions between neurons; such interactions would silence neurons under transparent conditions and render the visual system blind to transparent motion. To examine how the primate visual system solves this transparency problem, we recorded the activity of direction-selective cells in the first (area V1) and in a later (area MT) stage in the cortical motion-processing pathway in behaving monkeys. The visual stimuli consisted of random dot patterns forming single moving surfaces, transparent surfaces, and motion discontinuities. We found that area V1 cells responded to their preferred direction of movement even under transparent conditions, whereas area MT cells were suppressed under the transparent condition. These data suggest a simple solution to the transparency problem at the level of area V1. More than one motion vector can be represented at a single retinal location by different subpopulations of neurons tuned to different directions of motion; these subpopulations may represent the early stage for segmenting different, transparent surfaces. The results also suggest that facilitatory mechanisms, which unlike inhibitory interactions are largely unaffected by transparent conditions, play an important role in direction selectivity in area V1. The inhibitory interactions for different motion directions for area MT neurons may contribute to a mechanism for smoothing or averaging the velocity field, computations thought to be necessary for reducing noise and interpolating moving surfaces from sparse information.