Deep Learning with Dynamic Spiking Neurons and Fixed Feedback Weights.

Recent work in computer science has shown the power of deep learning driven by the backpropagation algorithm in networks of artificial neurons. But real neurons in the brain are different from most of these artificial ones in at least three crucial ways: they emit spikes rather than graded outputs, their inputs and outputs are related dynamically rather than by piecewise-smooth functions, and they have no known way to coordinate arrays of synapses in separate forward and feedback pathways so that they change simultaneously and identically, as they do in backpropagation. Given these differences, it is unlikely that current deep learning algorithms can operate in the brain, but we that show these problems can be solved by two simple devices: learning rules can approximate dynamic input-output relations with piecewise-smooth functions, and a variation on the feedback alignment algorithm can train deep networks without having to coordinate forward and feedback synapses. Our results also show that deep spiking networks learn much better if each neuron computes an intracellular teaching signal that reflects that cell's nonlinearity. With this mechanism, networks of spiking neurons show useful learning in synapses at least nine layers upstream from the output cells and perform well compared to other spiking networks in the literature on the MNIST digit recognition task.

Random synaptic feedback weights support error backpropagation for deep learning.

The brain processes information through multiple layers of neurons. This deep architecture is representationally powerful, but complicates learning because it is difficult to identify the responsible neurons when a mistake is made. In machine learning, the backpropagation algorithm assigns blame by multiplying error signals with all the synaptic weights on each neuron's axon and further downstream. However, this involves a precise, symmetric backward connectivity pattern, which is thought to be impossible in the brain. Here we demonstrate that this strong architectural constraint is not required for effective error propagation. We present a surprisingly simple mechanism that assigns blame by multiplying errors by even random synaptic weights. This mechanism can transmit teaching signals across multiple layers of neurons and performs as effectively as backpropagation on a variety of tasks. Our results help reopen questions about how the brain could use error signals and dispel long-held assumptions about algorithmic constraints on learning.

Adapting to inversion of the visual field: a new twist on an old problem.

While sensorimotor adaptation to prisms that displace the visual field takes minutes, adapting to an inversion of the visual field takes weeks. In spite of a long history of the study, the basis of this profound difference remains poorly understood. Here, we describe the computational issue that underpins this phenomenon and presents experiments designed to explore the mechanisms involved. We show that displacements can be mastered without altering the updated rule used to adjust the motor commands. In contrast, inversions flip the sign of crucial variables called sensitivity derivatives-variables that capture how changes in motor commands affect task error and therefore require an update of the feedback learning rule itself. Models of sensorimotor learning that assume internal estimates of these variables are known and fixed predicted that when the sign of a sensitivity derivative is flipped, adaptations should become increasingly counterproductive. In contrast, models that relearn these derivatives predict that performance should initially worsen, but then improve smoothly and remain stable once the estimate of the new sensitivity derivative has been corrected. Here, we evaluated these predictions by looking at human performance on a set of pointing tasks with vision perturbed by displacing and inverting prisms. Our experimental data corroborate the classic observation that subjects reduce their motor errors under inverted vision. Subjects' accuracy initially worsened and then improved. However, improvement was jagged rather than smooth and performance remained unstable even after 8 days of continually inverted vision, suggesting that subjects improve via an unknown mechanism, perhaps a combination of cognitive and implicit strategies. These results offer a new perspective on classic work with inverted vision.

Adaptive optimal control without weight transport.

Many neural control systems are at least roughly optimized, but how is optimal control learned? There are algorithms for this purpose, but in their current forms, they are not suited for biological neural networks because they rely on a type of communication that is not available in the brain, namely, weight transport-transmitting the strengths, or "weights," of individual synapses to other synapses and neurons. Here we show how optimal control can be learned without weight transport. Our method involves a set of simple mechanisms that can compensate for the absence of weight transport in the brain and so may be useful for neural computation generally.

Computational advantages of reverberating loops for sensorimotor learning.

When we learn something new, our brain may store the information in synapses or in reverberating loops of electrical activity, but current theories of motor learning focus almost entirely on the synapses. Here we show that loops could also play a role and would bring advantages: loop-based algorithms can learn complex control tasks faster, with exponentially fewer neurons, and avoid the problem of weight transport. They do all this at a cost: in the presence of long feedback delays, loop algorithms cannot control very fast movements, but in this case, loop and synaptic mechanisms can complement each other-mixed systems quickly learn to make accurate but not very fast motions and then gradually speed up. Loop algorithms explain aspects of consolidation, the role of attention, and the relapses that are sometimes seen after a task has apparently been learned, and they make further predictions.

Learning course adjustments during arm movements with reversed sensitivity derivatives.

BACKGROUND: To learn, a motor system needs to know its sensitivity derivatives, which quantify how its neural commands affect motor error. But are these derivatives themselves learned, or are they known solely innately? Here we test a recent theory that the brain's estimates of sensitivity derivatives are revisable based on sensory feedback. In its simplest form, the theory says that each control system has a single, adjustable estimate of its sensitivity derivatives which affects all aspects of its task, e.g. if you learn to reach to mirror-reversed targets then your revised estimate should reverse not only your initial aiming but also your online course adjustments when the target jumps in mid-movement. METHODS: Human subjects bent a joystick to move a cursor to a target on a computer screen, but the cursor's motion was reversed relative to the joystick's. The target jumped once during each movement. Subjects had up to 4000 trials to practice aiming and responding to target jumps. RESULTS: All subjects learned to reverse both initial aiming and course adjustments. CONCLUSIONS: Our study confirms that sensitivity derivatives can be relearned. It is consistent with the idea of a single, all-purpose estimate of those derivatives; and it suggests that the estimate is a function of context, as one would expect given that the true sensitivity derivatives may vary with the state of the controlled system, the target, and the motor commands.

Optimal inference explains dimension-specific contractions of spatial perception.

It is known that people misperceive scenes they see during rapid eye movements called saccades. It has been suggested that some of these misperceptions could be an artifact of neurophysiological processes related to the internal remapping of spatial coordinates during saccades. Alternatively, we have recently suggested, based on a computational model, that transsaccadic misperceptions result from optimal inference. As one of the properties of the model, sudden object displacements that occur in sync with a saccade should be perceived as contracted in a non-linear fashion. To explore this model property, here we use computer simulations and psychophysical methods first to test how robust the model is to close-to-optimal approximations and second to test two model predictions: (a) contracted transsaccadic perception should be dimension-specific with more contraction for jumps parallel to the saccade than orthogonal to it, and (b) contraction should rise as a function of visuomotor noise. Our results are consistent with these predictions. They support the idea that human transsaccadic integration is governed by close-to-optimal inference.

The extended horopter: quantifying retinal correspondence across changes of 3D eye position.

The theoretical horopter is an interesting qualitative tool for conceptualizing binocular correspondence, but its quantitative applications have been limited because they have ignored ocular kinematics and vertical binocular sensory fusion. Here we extend the mathematical definition of the horopter to a full surface over visual space, and we use this extended horopter to quantify binocular alignment and visualize its dependence on eye position. We reproduce the deformation of the theoretical horopter into a spiral shape in tertiary gaze as first described by Helmholtz (1867). We also describe a new effect of ocular torsion, where the Vieth-Müller circle rotates out of the visual plane for symmetric vergence conditions in elevated or depressed gaze. We demonstrate how these deformations are reduced or abolished when the eyes follow the modification of Listing's law during convergence called L2, which enlarges the extended horopter and keeps its location and shape constant across gaze directions.

A contralateral preference in the lateral occipital area: sensory and attentional mechanisms.

Here we examined the level of the lateral occipital (LO) area within the processing stream of the ventral visual cortex. An important determinant of an area's level of processing is whether it codes visual elements on both sides of the visual field, as do higher visual areas, or prefers those in the contralateral visual field, as do early visual areas. The former would suggest that LO, on one side, combines bilateral visual elements into a whole, while the latter suggests that it codes only the parts of forms. We showed that LO has a relative preference for visual objects in the contralateral visual field. LO responses were influenced by attention. However, relative changes in LO activity caused by changes in object location were preserved even when attention was shifted away from the objects to moving random dot patterns on the opposite side. Our data offer a new view on LO as an intermediate, but not a high-level, visual area in which neurons are driven by visual input and spatial attention in a multiplicative fashion.

Static ocular counterroll is implemented through the 3-D neural integrator.

Static head roll about the naso-occipital axis is known to produce an opposite ocular counterroll with a gain of approximately 10%, but the purpose and neural mechanism of this response remain obscure. In theory counterroll could be maintained either by direct tonic vestibular inputs to motoneurons, or by a neurally integrated pulse, as observed in the saccade generator and vestibulo-ocular reflex. When simulated together with ocular drift related to torsional integrator failure, the direct tonic input model predicted that the pattern of drift would shift torsionally as in ordinary counterroll, but the integrated pulse model predicted that the equilibrium position of torsional drift would be unaffected by head roll. This was tested experimentally by measuring ocular counterroll in 2 monkeys after injection of muscimol into the mesencephalic interstitial nucleus of Cajal. Whereas 90 degrees head roll produced a mean ocular counterroll of 8.5 degrees (+/-0.7 degrees SE) in control experiments, the torsional equilibrium position observed during integrator failure failed to counterroll, showing a torsional shift of only 0.3 degrees (+/-0.6 degrees SE). This result contradicted the direct tonic input model, but was consistent with models that implement counterroll by a neurally integrated pulse.

Motion parallax is computed in the updating of human spatial memory.

As we move through space, stationary objects around us show motion parallax: their directions relative to us change at different rates, depending on their distance. Does the brain incorporate parallax when it updates its stored representations of space? We had subjects fixate a distant target and then we flashed lights, at different distances, onto the retinal periphery. Subjects translated sideways while keeping their gaze on the distant target, and then they looked to the remembered location of the flash. Their responses corrected almost perfectly for parallax: they turned their eyes farther for nearer targets, in the predicted nonlinear patterns. Computer simulations suggest a neural mechanism in which feedback about self-motion updates remembered locations of objects within an internal map of three-dimensional visual space.

Interaction of retinal image and eye velocity in motion perception.

When we move our eyes, why does the world look stable even as its image flows across our retinas, and why do afterimages, which are stationary on the retinas, appear to move? Current theories say this is because we perceive motion by summation: if an object slips across the retina at r degrees/s while the eye turns at e degrees/s, the object's perceived velocity in space should be r + e. We show that activity in MT+, the visual-motion complex in human cortex, does reflect a mix of r and e rather than r alone. But we show also that, for optimal perception, r and e should not summate; rather, the signals coding e interact multiplicatively with the spatial gradient of illumination.

Influence of eye position on stereo matching.

In animals with binocular depth vision, or stereopsis, the visual fields of the two eyes overlap, shrinking the overall field of view. Eye movements increase the field of view, but they also complicate the first stage of stereopsis: the search for corresponding images on the two retinas. If the eyes were stationary in the head, corresponding images would always lie on retina-fixed bands called epipolar lines. Because the eyes rotate, the epipolar lines move on the retinas. Therefore, the stereoptic system has a choice: it may monitor eye position to keep track of the epipolar lines, or it may give up on tracking epipolar lines and instead search for matches over retina-fixed regions, but in that case the search regions must be 2-D patches, large enough to encompass all possible locations of the epipolar lines in all usual eye positions. We use a new type of random-dot stereogram to show that human stereopsis uses large, retina-fixed search zones. We show that the brain somewhat reduces the size of these search zones by rotating the eyes about their lines of sight in a way that reduces the motion of the epipolar lines. These findings show the link between sensory and motor processes: by considering eye motion we can understand why the brain searches for matching images over 2-D retinal regions rather than along epipolar lines; and by considering retinal correspondence we appreciate why the eyes rotate as they do about their lines of sight.

Optimal transsaccadic integration explains distorted spatial perception.

We scan our surroundings with quick eye movements called saccades, and from the resulting sequence of images we build a unified percept by a process known as transsaccadic integration. This integration is often said to be flawed, because around the time of saccades, our perception is distorted and we show saccadic suppression of displacement (SSD): we fail to notice if objects change location during the eye movement. Here we show that transsaccadic integration works by optimal inference. We simulated a visuomotor system with realistic saccades, retinal acuity, motion detectors and eye-position sense, and programmed it to make optimal use of these imperfect data when interpreting scenes. This optimized model showed human-like SSD and distortions of spatial perception. It made new predictions, including tight correlations between perception and motor action (for example, more SSD in people with less-precise eye control) and a graded contraction of perceived jumps; we verified these predictions experimentally. Our results suggest that the brain constructs its evolving picture of the world by optimally integrating each new piece of sensory or motor information.

Rotational remapping in human spatial memory during eye and head motion.

The brain uses vision and other senses to compute the locations of objects relative to the body, and then must update these locations when the body moves. How geometrically sophisticated is this internal updating? It has been suggested that updating simply shifts the stored locations of all objects uniformly, by a common vector, when the eye or head turns. For horizontal and vertical turns, a uniform shift would often approximate the real changes in location of objects in front of the subject. But for torsional rotations, a shift would be inadequate: accurate updating would call for a more geometrically exact remapping, not shifting but rotating the stored locations through the inverse of the rotation of the eye in space. Here we asked human subjects to make eye saccades to remembered targets after torsional head rotations. Their accuracy showed that spatial updating works in the torsional dimension and operates by rotation rather than shifting.