Reconfigurations of cortical manifold structure during reward-based motor learning.

Adaptive motor behavior depends on the coordinated activity of multiple neural systems distributed across the brain. While the role of sensorimotor cortex in motor learning has been well established, how higher-order brain systems interact with sensorimotor cortex to guide learning is less well understood. Using functional MRI, we examined human brain activity during a reward-based motor task where subjects learned to shape their hand trajectories through reinforcement feedback. We projected patterns of cortical and striatal functional connectivity onto a low-dimensional manifold space and examined how regions expanded and contracted along the manifold during learning. During early learning, we found that several sensorimotor areas in the dorsal attention network exhibited increased covariance with areas of the salience/ventral attention network and reduced covariance with areas of the default mode network (DMN). During late learning, these effects reversed, with sensorimotor areas now exhibiting increased covariance with DMN areas. However, areas in posteromedial cortex showed the opposite pattern across learning phases, with its connectivity suggesting a role in coordinating activity across different networks over time. Our results establish the neural changes that support reward-based motor learning and identify distinct transitions in the functional coupling of sensorimotor to transmodal cortex when adapting behavior.

Eye-hand coordination during online reach corrections is task dependent.

To produce accurate movements, the human motor system needs to deal with errors that can occur due to inherent noise, changes in the body, or disturbances in the environment. Here, we investigated the temporal coupling of rapid corrections of the eye and hand in response to a change in visual target location during the movement. In addition to a "classic" double-step task in which the target stepped to a new position, participants performed a set of modified double-step tasks in which the change in movement goal was indicated by the appearance of an additional target, or by a spatial or symbolic cue. We found that both the absolute correction latencies of the eye and hand and the relative eye-hand correction latencies were dependent on the visual characteristics of the target change, with increasingly longer latencies in tasks that required more visual and cognitive processing. Typically, the hand started correcting slightly earlier than the eye, especially when the target change was indicated by a symbolic cue, and in conditions where visual feedback of the hand position was provided during the reach. Our results indicate that the oculomotor and limb-motor system can be differentially influenced by processing requirements of the task and emphasize that temporal eye-hand coupling is flexible rather than rigid.NEW & NOTEWORTHY Eye movements support hand movements in many situations. Here, we used variations of a double-step task to investigate temporal coupling of corrective hand and eye movements in response to target displacements. Correction latency coupling depended on the visual and cognitive processing demands of the task. The hand started correcting before the eye, especially when the task required decoding a symbolic cue. These findings highlight the flexibility and task dependency of eye-hand coordination.

Human Variation in Error-Based and Reinforcement Motor Learning Is Associated With Entorhinal Volume.

Error-based and reward-based processes are critical for motor learning and are thought to be mediated via distinct neural pathways. However, recent behavioral work in humans suggests that both learning processes can be bolstered by the use of cognitive strategies, which may mediate individual differences in motor learning ability. It has been speculated that medial temporal lobe regions, which have been shown to support motor sequence learning, also support the use of cognitive strategies in error-based and reinforcement motor learning. However, direct evidence in support of this idea remains sparse. Here we first show that better overall learning during error-based visuomotor adaptation is associated with better overall learning during the reward-based shaping of reaching movements. Given the cognitive contribution to learning in both of these tasks, these results support the notion that strategic processes, associated with better performance, drive intersubject variation in both error-based and reinforcement motor learning. Furthermore, we show that entorhinal cortex volume is larger in better learning individuals-characterized across both motor learning tasks-compared with their poorer learning counterparts. These results suggest that individual differences in learning performance during error and reinforcement learning are related to neuroanatomical differences in entorhinal cortex.

Functional Use of Eye Movements for an Acting System.

Movements of the eyes assist vision and support hand and body movements in a cooperative way. Despite their strong functional coupling, different types of movements are usually studied independently. We integrate knowledge from behavioral, neurophysiological, and clinical studies on how eye movements are coordinated with goal-directed hand movements and how they facilitate motor learning. Understanding the coordinated control of eye and hand movements can provide important insights into brain functions that are essential for performing or learning daily tasks in health and disease. This knowledge can also inform applications such as robotic manipulation and clinical rehabilitation.

Correcting for natural visuo-proprioceptive matching errors based on reward as opposed to error feedback does not lead to higher retention.

When asked to move their unseen hand-to-visual targets, people exhibit idiosyncratic but reliable visuo-proprioceptive matching errors. Unsurprisingly, vision and proprioception quickly align when these errors are made apparent by providing visual feedback of the position of the hand. However, retention of this learning is limited, such that the original matching errors soon reappear when visual feedback is removed. Several recent motor learning studies have shown that reward feedback can improve retention relative to error feedback. Here, using a visuo-proprioceptive position-matching task, we examined whether binary reward feedback can be effectively exploited to reduce matching errors and, if so, whether this learning leads to improved retention relative to learning based on error feedback. The results show that participants were able to adjust the visuo-proprioceptive mapping with reward feedback, but that the level of retention was similar to that observed when the adjustment was accomplished with error feedback. Therefore, similar to error feedback, reward feedback allows for temporary recalibration, but does not support long-lasting retention of this recalibration.

Visuomotor feedback gains are modulated by gaze position.

During goal-directed reaching, people typically direct their gaze to the target before the start of the hand movement and maintain fixation until the hand arrives. This gaze strategy improves reach accuracy in two ways. It enables the use of central vision at the end of movement, and it allows the use of extraretinal information in guiding the hand to the target. Here we tested whether fixating the reach target further facilitates reach accuracy by optimizing the use of peripheral vision in detecting, and rapidly responding to, reach errors during the ongoing movement. We examined automatic visuomotor corrections in response to displacements of the cursor representing the hand position as a function of gaze fixation location during unimanual goal-directed reaching. Eight fixation targets were positioned either in line with, or at different angles relative to, the straight-ahead movement direction (manipulation of fixation angle), and at different distances from the location of the visual perturbation (manipulation of fixation distance). We found that corrections were fastest and strongest when gaze was directed at the reach target compared with when gaze was directed to a different location in the workspace. We found that the gain of the visuomotor response was strongly affected by fixation angle, and to a smaller extent by fixation distance, with lower gains as the angle or distance increased. We submit that fixating the reach target improves reach accuracy by facilitating rapid visuomotor responses to reach errors viewed in peripheral vision. NEW & NOTEWORTHY It is well known that directing gaze to the reach target allows the use of foveal visual feedback and extraretinal information to improve the accuracy of reaching movements. Here we demonstrate that target fixation also optimizes rapid visuomotor corrections to reach errors viewed in peripheral vision, with the angle of gaze relative to the hand movement being a critical determinant in the gain of the visuomotor response.

Using gaze behavior to parcellate the explicit and implicit contributions to visuomotor learning.

Successful motor performance relies on our ability to adapt to changes in the environment by learning novel mappings between motor commands and sensory outcomes. Such adaptation is thought to involve two distinct mechanisms: an implicit, error-based component linked to slow learning and an explicit, strategic component linked to fast learning and savings (i.e., faster relearning). Because behavior, at any given moment, is the resultant combination of these two processes, it has remained a challenge to parcellate their relative contributions to performance. The explicit component to visuomotor rotation (VMR) learning has recently been measured by having participants verbally report their aiming strategy used to counteract the rotation. However, this procedure has been shown to magnify the explicit component. Here we tested whether task-specific eye movements, a natural component of reach planning, but poorly studied in motor learning tasks, can provide a direct readout of the state of the explicit component during VMR learning. We show, by placing targets on a visible ring and including a delay between target presentation and reach onset, that individual differences in gaze patterns during sensorimotor learning are linked to participants' rates of learning and their expression of savings. Specifically, we find that participants who, during reach planning, naturally fixate an aimpoint rotated away from the target location, show faster initial adaptation and readaptation 24 h later. Our results demonstrate that gaze behavior cannot only uniquely identify individuals who implement cognitive strategies during learning but also how their implementation is linked to differences in learning. NEW & NOTEWORTHY Although it is increasingly well appreciated that sensorimotor learning is driven by two separate components, an error-based process and a strategic process, it has remained a challenge to identify their relative contributions to performance. Here we demonstrate that task-specific eye movements provide a direct read-out of explicit strategies during sensorimotor learning in the presence of visual landmarks. We further show that individual differences in gaze behavior are linked to learning rate and savings.

Dynamic representations of visual space for perception and action.

The division of labor in visual processing between two anatomically relatively separate cortical pathways, a ventral and a dorsal stream, has been hotly debated in the last decades. One influential model is the What & How pathway model, suggesting that the separation is along ventral perception versus dorsal action, although the degree of functional separation between the two streams is controversial. An implication of this model is that perception and memory-guided movements are highly sensitive to visual contextual illusions, whereas visually-guided movements are largely immune to them. Here, we summarize our recent behavioral and imaging data obtained in single and double saccade paradigms that test this proposal, with a focus on the role of time in visuomotor processing and updating. We describe results showing that presentation time of the illusion affects both saccade amplitude and perceptual judgments in a similar way. We also discuss behavioral findings showing that visuomotor updating is affected by illusory context. Complementary neuroimaging data suggest a neural correlate of these findings in dorsal stream areas. Taken together, these results are suggestive of a dynamic, common visual representation that drives both perception and action, or - at least - that there is no absolute functional specialization of the two visual processing streams.

Parallel Specification of Visuomotor Feedback Gains during Bimanual Reaching to Independent Goals.

During goal-directed reaching, rapid visuomotor feedback processes enable the human motor system to quickly correct for errors in the trajectory of the hand that arise from motor noise and, in some cases, external perturbations. To date, these visuomotor responses, the gain of which is sensitive to features of the task and environment, have primarily been examined in the context of unimanual reaching movements toward a single target. However, many natural tasks involve moving both hands together, often to separate targets, such that errors can occur in parallel and at different spatial locations. Here, we examined the resource capacity of automatic visuomotor corrective mechanisms by comparing feedback gains during bimanual reaches, toward two targets, to feedback gains during unimanual reaches toward single targets. To investigate the sensitivity of the feedback gains and their relation to visual-spatial processing, we manipulated the widths of the targets and participants' gaze location. We found that the gain of corrective responses to cursor displacements, while strongly modulated by target width and gaze position, were only slightly reduced during bimanual control. Our results show that automatic visuomotor corrective mechanisms can efficiently operate in parallel across multiple spatial locations.

A Bayesian Account of Visual-Vestibular Interactions in the Rod-and-Frame Task.

Panoramic visual cues, as generated by the objects in the environment, provide the brain with important information about gravity direction. To derive an optimal, i.e., Bayesian, estimate of gravity direction, the brain must combine panoramic information with gravity information detected by the vestibular system. Here, we examined the individual sensory contributions to this estimate psychometrically. We asked human subjects to judge the orientation (clockwise or counterclockwise relative to gravity) of a briefly flashed luminous rod, presented within an oriented square frame (rod-in-frame). Vestibular contributions were manipulated by tilting the subject's head, whereas visual contributions were manipulated by changing the viewing distance of the rod and frame. Results show a cyclical modulation of the frame-induced bias in perceived verticality across a 90° range of frame orientations. The magnitude of this bias decreased significantly with larger viewing distance, as if visual reliability was reduced. Biases increased significantly when the head was tilted, as if vestibular reliability was reduced. A Bayesian optimal integration model, with distinct vertical and horizontal panoramic weights, a gain factor to allow for visual reliability changes, and ocular counterroll in response to head tilt, provided a good fit to the data. We conclude that subjects flexibly weigh visual panoramic and vestibular information based on their orientation-dependent reliability, resulting in the observed verticality biases and the associated response variabilities.

Contributions of gaze-centered and object-centered coding in a double-step saccade task.

The position of a saccade target can be encoded in gaze-centered coordinates, that is, relative to the current gaze position, or in object-centered coordinates, that is, relative to an object in the environment. We tested the role of gaze-centered and object-centered coding in a double-step saccade task involving the Brentano version of the Müller-Lyer illusion. The two visual targets were presented either sequentially, requiring gaze-centered coding of the second saccade target, or simultaneously, thereby providing additional object-centered information about the location of the second target relative to the first. We found that the endpoint of the second saccade was affected by the illusion, irrespective of whether the targets were presented sequentially or simultaneously, suggesting that participants used a gaze-centered updating strategy. We found that variability in saccade endpoints was reduced when object-centered information was consistently available but not when its presence varied from trial to trial. Our results suggest that gaze-centered coding is dominant in the planning of sequential saccades, whereas object-centered information plays a relatively small role.

How Heavy Is an Illusory Length?

The perception of object properties, such as size and weight, can be subject to illusions. Could a visual size illusion influence perceived weight? Here, we tested whether the size-weight illusion occurs when lifting two physically identical but perceptually different objects, by using an illusion of size. Participants judged the weight and length of 11 to 17 cm brass bars with equal density to which cardboard arrowheads were attached to create a Müller-Lyer illusion. We found that these stimuli induced an illusion in which the bar that was visually perceived as being shorter was also perceived as feeling heavier. In fact, a 5-mm increase in illusory length corresponded to a decrease in illusory weight of 15 g.

Keeping a target in memory does not increase the effect of the Müller-Lyer illusion on saccades.

The effects of visual contextual illusions on motor behaviour vary largely between experimental conditions. Whereas it has often been reported that the effects of illusions on pointing and grasping are largest when the movement is performed some time after the stimulus has disappeared, the effect of a delay has hardly been studied for saccadic eye movements. In this experiment, participants viewed a briefly presented Müller-Lyer illusion with a target at its endpoint and made a saccade to the remembered position of this target after a delay of 0, 0.6, 1.2 or 1.8 s. We found that horizontal saccade amplitudes were shorter for the perceptually shorter than for the perceptually longer configuration of the illusion. Most importantly, although the delay clearly affected saccade amplitude, resulting in shorter saccades for longer delays, the illusion effect did not depend on the duration of the delay. We argue that visually guided and memory-guided saccades are likely based on a common visual representation.

The Müller-Lyer illusion affects visuomotor updating in the dorsal visual stream.

To guide our actions, the brain has developed mechanisms to code target locations in egocentric coordinates (i.e., with respect to the observer), and to update these when the observer moves. The latter mechanism, called visuomotor updating, is implemented in the dorsal visual stream of the brain. In contrast, the ventral visual stream is assumed to transform target locations into an allocentric reference frame that is highly sensitive to visual contextual illusions. Here, we tested the effect of the Müller-Lyer illusion on visuomotor updating in a double-step saccade task. Using the same paradigm in a 3T fMRI scanner, we investigated the effect of the illusion on the neural correlate of the updating process. Participants briefly viewed the Brentano version of the Müller-Lyer illusion with a target at its middle vertex, while fixating at one of the two endpoints of the illusion. Shortly after the disappearance of the stimulus, the eyes' fixation point moved to a position outside the illusion. After a delay, participants made a saccade to the remembered position of the target. The landing position of this saccade was systematically displaced in a manner congruent with the perceptual illusion, showing that visuomotor updating is affected by the illusion. fMRI results showed that the BOLD response in the occipito-parietal cortex (area V7) and the intraparietal sulcus related to planning of the saccade to the updated target was also modulated by the configuration of the illusion. This suggests that the dorsal visual stream represents perceived rather than physical locations of remembered saccade targets.

Time course of the effect of the Muller-Lyer illusion on saccades and perceptual judgments.

The amplitude of saccadic eye movements is affected by size illusions such as the Müller-Lyer illusion, but this effect varies highly between studies. Here we examine the origin of this variability by testing the influence of three temporal factors on the effect of the Müller-Lyer illusion: presentation time, response delay, and saccade latency. Subjects performed reflexive saccades, deferred saccades, and memory-guided saccades along the shaft of the illusion. We evaluated the time course of the saccadic illusion effects. We compared it to the influence of presentation time on the illusion effect in a perceptual judgment task. According to the "two visual systems hypothesis", visual perception and visual memory rely on a perceptual representation coded along the ventral "perception" pathway, which is affected by visual contextual illusions. Visuomotor actions, such as saccades, depend on the dorsal "action" pathway that is largely immune to illusions. In contrast with this hypothesis, our results show that the illusion affected both saccade amplitude and perceptual judgments with a similar time course. Presentation time of the Müller-Lyer illusion, not response delay or saccade latency, was the major factor in determining the size of the illusion effect. Longer presentation times resulted in smaller effects, suggesting that our visual representation is dynamic and becomes more accurate when we look at an object for a longer time before we act on it.

Don't rock the boat: how antiphase crew coordination affects rowing.

It is generally accepted that crew rowing requires perfect synchronization between the movements of the rowers. However, a long-standing and somewhat counterintuitive idea is that out-of-phase crew rowing might have benefits over in-phase (i.e., synchronous) rowing. In synchronous rowing, 5 to 6% of the power produced by the rower(s) is lost to velocity fluctuations of the shell within each rowing cycle. Theoretically, a possible way for crews to increase average boat velocity is to reduce these fluctuations by rowing in antiphase coordination, a strategy in which rowers perfectly alternate their movements. On the other hand, the framework of coordination dynamics explicates that antiphase coordination is less stable than in-phase coordination, which may impede performance gains. Therefore, we compared antiphase to in-phase crew rowing performance in an ergometer experiment. Nine pairs of rowers performed a two-minute maximum effort in-phase and antiphase trial at 36 strokes min(-1) on two coupled free-floating ergometers that allowed for power losses to velocity fluctuations. Rower and ergometer kinetics and kinematics were measured during the trials. All nine pairs easily acquired antiphase rowing during the warm-up, while one pair's coordination briefly switched to in-phase during the maximum effort trial. Although antiphase interpersonal coordination was indeed less accurate and more variable, power production was not negatively affected. Importantly, in antiphase rowing the decreased power loss to velocity fluctuations resulted in more useful power being transferred to the ergometer flywheels. These results imply that antiphase rowing may indeed improve performance, even without any experience with antiphase technique. Furthermore, it demonstrates that although perfectly synchronous coordination may be the most stable, it is not necessarily equated with the most efficient or optimal performance.