Interaction between decision-making and motor learning when selecting reach targets in the presence of bias and noise.

Motor errors can have both bias and noise components. Bias can be compensated for by adaptation and, in tasks in which the magnitude of noise varies across the environment, noise can be reduced by identifying and then acting in less noisy regions of the environment. Here we examine how these two processes interact when participants reach under a combination of an externally imposed visuomotor bias and noise. In a center-out reaching task, participants experienced noise (zero-mean random visuomotor rotations) that was target-direction dependent with a standard deviation that increased linearly from a least-noisy direction. They also experienced a constant bias, a visuomotor rotation that varied (across groups) from 0 to 40 degrees. Critically, on each trial, participants could select one of three targets to reach to, thereby allowing them to potentially select targets close to the least-noisy direction. The group who experienced no bias (0 degrees) quickly learned to select targets close to the least-noisy direction. However, groups who experienced a bias often failed to identify the least-noisy direction, even though they did partially adapt to the bias. When noise was introduced after participants experienced and adapted to a 40 degrees bias (without noise) in all directions, they exhibited an improved ability to find the least-noisy direction. We developed two models-one for reach adaptation and one for target selection-that could explain participants' adaptation and target-selection behavior. Our data and simulations indicate that there is a trade-off between adaptation and selection. Specifically, because bias learning is local, participants can improve performance, through adaptation, by always selecting targets that are closest to a chosen direction. However, this comes at the expense of improving performance, through selection, by reaching toward targets in different directions to find the least-noisy direction.

Fast Feedback Responses to Categorical Sensorimotor Errors That Do Not Indicate Error Magnitude Are Optimized Based on Short- and Long-Term Memory.

Skilled motor performance depends critically on rapid corrective responses that act to preserve the goal of the movement in the face of perturbations. Although it is well established that the gain of corrective responses elicited while reaching toward objects adapts to different contexts, little is known about the adaptability of corrective responses supporting the manipulation of objects after they are grasped. Here, we investigated the adaptability of the corrective response elicited when an object being lifted is heavier than expected and fails to lift off when predicted. This response involves a monotonic increase in vertical load force triggered, within ∼90 ms, by the absence of expected sensory feedback signaling lift off and terminated when actual lift off occurs. Critically, because the actual weight of the object cannot be directly sensed at the moment the object fails to lift off, any adaptation of the corrective response would have to be based on memory from previous lifts. We show that when humans, including men and women, repeatedly lift an object that on occasional catch trials increases from a baseline weight to a fixed heavier weight, they scale the gain of the response (i.e., the rate of force increase) to the heavier weight within two to three catch trials. We also show that the gain of the response scales, on the first catch trial, with the baseline weight of the object. Thus, the gain of the lifting response can be adapted by both short- and long-term experience. Finally, we demonstrate that this adaptation preserves the efficacy of the response across contexts.SIGNIFICANCE STATEMENT Here, we present the first investigation of the adaptability of the corrective lifting response elicited when an object is heavier than expected and fails to lift off when predicted. A striking feature of the response, which is driven by a sensory prediction error arising from the absence of expected sensory feedback, is that the magnitude of the error is unknown. That is, the motor system only receives a categorical error indicating that the object is heavier than expected but not its actual weight. Although the error magnitude is not known at the moment the response is elicited, we show that the response can be scaled to predictions of error magnitude based on both recent and long-term memories.

Visual search for reach targets in actionable space is influenced by movement costs imposed by obstacles.

Real world search tasks often involve action on a target object once it has been located. However, few studies have examined whether movement-related costs associated with acting on located objects influence visual search. Here, using a task in which participants reached to a target object after locating it, we examined whether people take into account obstacles that increase movement-related costs for some regions of the reachable search space but not others. In each trial, a set of 36 objects (4 targets and 32 distractors) were displayed on a vertical screen and participants moved a cursor to a target after locating it. Participants had to fixate on an object to determine whether it was a target or distractor. A rectangular obstacle, of varying length, location, and orientation, was briefly displayed at the start of the trial. Participants controlled the cursor by moving the handle of a robotic manipulandum in a horizontal plane. The handle applied forces to simulate contact between the cursor and the unseen obstacle. We found that search, measured using eye movements, was biased to regions of the search space that could be reached without moving around the obstacle. This result suggests that when deciding where to search, people can incorporate the physical structure of the environment so as to reduce the movement-related cost of subsequently acting on the located target.

Memory, perceptual, and motor costs affect the strength of categorical encoding during motor learning of object properties.

Nearly all tasks of daily life involve skilled object manipulation, and successful manipulation requires knowledge of object dynamics. We recently developed a motor learning paradigm that reveals the categorical organization of motor memories of object dynamics. When participants repeatedly lift a constant-density "family" of cylindrical objects that vary in size, and then an outlier object with a greater density is interleaved into the sequence of lifts, they often fail to learn the weight of the outlier, persistently treating it as a family member despite repeated errors. Here we examine eight factors (Similarity, Cardinality, Frequency, History, Structure, Stochasticity, Persistence, and Time Pressure) that could influence the formation and retrieval of category representations in the outlier paradigm. In our web-based task, participants (N = 240) anticipated object weights by stretching a virtual spring attached to the top of each object. Using Bayesian t-tests, we analyze the relative impact of each manipulated factor on categorical encoding (strengthen, weaken, or no effect). Our results suggest that category representations of object weight are automatic, rigid, and linear and, as a consequence, the key determinant of whether an outlier is encoded as a member of the family is its discriminability from the family members.

Object weight can be rapidly predicted, with low cognitive load, by exploiting learned associations between the weights and locations of objects.

Weight prediction is critical for dexterous object manipulation. Previous work has focused on lifting objects presented in isolation and has examined how the visual appearance of an object is used to predict its weight. Here we tested the novel hypothesis that when interacting with multiple objects, as is common in everyday tasks, people exploit the locations of objects to directly predict their weights, bypassing slower and more demanding processing of visual properties to predict weight. Using a three-dimensional robotic and virtual reality system, we developed a task in which participants were presented with a set of objects. In each trial a randomly chosen object translated onto the participant's hand and they had to anticipate the object's weight by generating an equivalent upward force. Across conditions we could control whether the visual appearance and/or location of the objects were informative as to their weight. Using this task, and a set of analogous web-based experiments, we show that when location information was predictive of the objects' weights participants used this information to achieve faster prediction than observed when prediction is based on visual appearance. We suggest that by "caching" associations between locations and weights, the sensorimotor system can speed prediction while also lowering working memory demands involved in predicting weight from object visual properties.NEW & NOTEWORTHY We use a novel object support task using a three-dimensional robotic interface and virtual reality system to provide evidence that the locations of objects are used to predict their weights. Using location information, rather than the visual appearance of the objects, supports fast prediction, thereby avoiding processes that can be demanding on working memory.

The influence of movement-related costs when searching to act and acting to search.

Real-world search behavior often involves limb movements, either during search or after search. Here we investigated whether movement-related costs influence search behavior in two kinds of search tasks. In our visual search tasks, participants made saccades to find a target object among distractors and then moved a cursor, controlled by the handle of a robotic manipulandum, to the target. In our manual search tasks, participants moved the cursor to perform the search, placing it onto objects to reveal their identity as either a target or a distractor. In all tasks, there were multiple targets. Across experiments, we manipulated either the effort or time costs associated with movement such that these costs varied across the search space. We varied effort by applying different resistive forces to the handle, and we varied time costs by altering the speed of the cursor. Our analysis of cursor and eye movements during manual and visual search, respectively, showed that effort influenced manual search but did not influence visual search. In contrast, time costs influenced both visual and manual search. Our results demonstrate that, in addition to perceptual and cognitive factors, movement-related costs can also influence search behavior.NEW & NOTEWORTHY Numerous studies have investigated the perceptual and cognitive factors that influence decision making about where to look, or move, in search tasks. However, little is known about how search is influenced by movement-related costs associated with acting on an object once it has been visually located or acting during manual search. In this article, we show that movement time costs can bias visual and manual search and that movement effort costs bias manual search.

Whole-brain dynamics of human sensorimotor adaptation.

Humans vary greatly in their motor learning abilities, yet little is known about the neural processes that underlie this variability. We identified distinct profiles of human sensorimotor adaptation that emerged across 2 days of learning, linking these profiles to the dynamics of whole-brain functional networks early on the first day when cognitive strategies toward sensorimotor adaptation are believed to be most prominent. During early learning, greater recruitment of a network of higher-order brain regions, involving prefrontal and anterior temporal cortex, was associated with faster learning. At the same time, greater integration of this "cognitive network" with a sensorimotor network was associated with slower learning, consistent with the notion that cognitive strategies toward adaptation operate in parallel with implicit learning processes of the sensorimotor system. On the second day, greater recruitment of a network that included the hippocampus was associated with faster learning, consistent with the notion that declarative memory systems are involved with fast relearning of sensorimotor mappings. Together, these findings provide novel evidence for the role of higher-order brain systems in driving variability in adaptation.

Distinct patterns of cortical manifold expansion and contraction underlie human sensorimotor adaptation.

Sensorimotor learning is a dynamic, systems-level process that involves the combined action of multiple neural systems distributed across the brain. Although much is known about the specialized cortical systems that support specific components of action (such as reaching), we know less about how cortical systems function in a coordinated manner to facilitate adaptive behavior. To address this gap, our study measured human brain activity using functional MRI (fMRI) while participants performed a classic sensorimotor adaptation task and used a manifold learning approach to describe how behavioral changes during adaptation relate to changes in the landscape of cortical activity. During early adaptation, areas in the parietal and premotor cortices exhibited significant contraction along the cortical manifold, which was associated with their increased covariance with regions in the higher-order association cortex, including both the default mode and fronto-parietal networks. By contrast, during Late adaptation, when visuomotor errors had been largely reduced, a significant expansion of the visual cortex along the cortical manifold was associated with its reduced covariance with the association cortex and its increased intraconnectivity. Lastly, individuals who learned more rapidly exhibited greater covariance between regions in the sensorimotor and association cortices during early adaptation. These findings are consistent with a view that sensorimotor adaptation depends on changes in the integration and segregation of neural activity across more specialized regions of the unimodal cortex with regions in the association cortex implicated in higher-order processes. More generally, they lend support to an emerging line of evidence implicating regions of the default mode network (DMN) in task-based performance.

Reach adaption to a visuomotor gain with terminal error feedback involves reinforcement learning.

Motor adaptation can be achieved through error-based learning, driven by sensory prediction errors, or reinforcement learning, driven by reward prediction errors. Recent work on visuomotor adaptation has shown that reinforcement learning leads to more persistent adaptation when visual feedback is removed, compared to error-based learning in which continuous visual feedback of the movement is provided. However, there is evidence that error-based learning with terminal visual feedback of the movement (provided at the end of movement) may be driven by both sensory and reward prediction errors. Here we examined the influence of feedback on learning using a visuomotor adaptation task in which participants moved a cursor to a single target while the gain between hand and cursor movement displacement was gradually altered. Different groups received either continuous error feedback (EC), terminal error feedback (ET), or binary reinforcement feedback (success/fail) at the end of the movement (R). Following adaptation we tested generalization to targets located in different directions and found that generalization in the ET group was intermediate between the EC and R groups. We then examined the persistence of adaptation in the EC and ET groups when the cursor was extinguished and only binary reward feedback was provided. Whereas performance was maintained in the ET group, it quickly deteriorated in the EC group. These results suggest that terminal error feedback leads to a more robust form of learning than continuous error feedback. In addition our findings are consistent with the view that error-based learning with terminal feedback involves both error-based and reinforcement learning.

Neural excursions from manifold structure explain patterns of learning during human sensorimotor adaptation.

Humans vary greatly in their motor learning abilities, yet little is known about the neural mechanisms that underlie this variability. Recent neuroimaging and electrophysiological studies demonstrate that large-scale neural dynamics inhabit a low-dimensional subspace or manifold, and that learning is constrained by this intrinsic manifold architecture. Here, we asked, using functional MRI, whether subject-level differences in neural excursion from manifold structure can explain differences in learning across participants. We had subjects perform a sensorimotor adaptation task in the MRI scanner on 2 consecutive days, allowing us to assess their learning performance across days, as well as continuously measure brain activity. We find that the overall neural excursion from manifold activity in both cognitive and sensorimotor brain networks is associated with differences in subjects' patterns of learning and relearning across days. These findings suggest that off-manifold activity provides an index of the relative engagement of different neural systems during learning, and that subject differences in patterns of learning and relearning are related to reconfiguration processes occurring in cognitive and sensorimotor networks.

Human Variation in Error-Based and Reinforcement Motor Learning Is Associated With Entorhinal Volume.

Error-based and reward-based processes are critical for motor learning and are thought to be mediated via distinct neural pathways. However, recent behavioral work in humans suggests that both learning processes can be bolstered by the use of cognitive strategies, which may mediate individual differences in motor learning ability. It has been speculated that medial temporal lobe regions, which have been shown to support motor sequence learning, also support the use of cognitive strategies in error-based and reinforcement motor learning. However, direct evidence in support of this idea remains sparse. Here we first show that better overall learning during error-based visuomotor adaptation is associated with better overall learning during the reward-based shaping of reaching movements. Given the cognitive contribution to learning in both of these tasks, these results support the notion that strategic processes, associated with better performance, drive intersubject variation in both error-based and reinforcement motor learning. Furthermore, we show that entorhinal cortex volume is larger in better learning individuals-characterized across both motor learning tasks-compared with their poorer learning counterparts. These results suggest that individual differences in learning performance during error and reinforcement learning are related to neuroanatomical differences in entorhinal cortex.

Motor memories of object dynamics are categorically organized.

The ability to predict the dynamics of objects, linking applied force to motion, underlies our capacity to perform many of the tasks we carry out on a daily basis. Thus, a fundamental question is how the dynamics of the myriad objects we interact with are organized in memory. Using a custom-built three-dimensional robotic interface that allowed us to simulate objects of varying appearance and weight, we examined how participants learned the weights of sets of objects that they repeatedly lifted. We find strong support for the novel hypothesis that motor memories of object dynamics are organized categorically, in terms of families, based on covariation in their visual and mechanical properties. A striking prediction of this hypothesis, supported by our findings and not predicted by standard associative map models, is that outlier objects with weights that deviate from the family-predicted weight will never be learned despite causing repeated lifting errors.

Human somatosensory cortex is modulated during motor planning.

Recent data and motor control theory argues that movement planning involves preparing the neural state of primary motor cortex (M1) for forthcoming action execution. Theories related to internal models, feedback control, and predictive coding also emphasize the importance of sensory prediction (and processing) prior to (and during) the movement itself, explaining why motor-related deficits can arise from damage to primary somatosensory cortex (S1). Motivated by this work, here we examined whether motor planning, in addition to changing the neural state of M1, changes the neural state of S1, preparing it for the sensory feedback that arises during action. We tested this idea in two human functional MRI studies (N=31, 16 female) involving delayed object manipulation tasks, focusing our analysis on pre-movement activity patterns in M1 and S1. We found that the motor effector to be used in the upcoming action could be decoded, well before movement, from neural activity in M1 in both studies. Critically, we found that this effector information was also present, well before movement, in S1. In particular, we found that the encoding of effector information in area 3b (S1 proper) was linked to the contralateral hand, similarly to that found in M1, whereas in areas 1 and 2 this encoding was present in both the contralateral and ipsilateral hemispheres. Together, these findings suggest that motor planning not only prepares the motor system for movement, but also changes the neural state of the somatosensory system, presumably allowing it to anticipate the sensory information received during movement.SIGNIFICANCE STATEMENT:Whereas recent work on motor cortex has emphasized the critical role of movement planning in preparing neural activity for movement generation, it has not investigated the extent to which planning also modulates the activity in adjacent primary somatosensory cortex (S1). This reflects a key gap in knowledge, given that recent motor control theories emphasize the importance of sensory feedback processing in effective movement generation. Here we find, through a convergence of experiments and analyses, that the planning of object manipulation tasks, in addition to modulating the activity in motor cortex, changes the state of neural activity in different subfields of human S1. We suggest that this modulation prepares S1 for the sensory information it will receive during action execution.

Functional Use of Eye Movements for an Acting System.

Movements of the eyes assist vision and support hand and body movements in a cooperative way. Despite their strong functional coupling, different types of movements are usually studied independently. We integrate knowledge from behavioral, neurophysiological, and clinical studies on how eye movements are coordinated with goal-directed hand movements and how they facilitate motor learning. Understanding the coordinated control of eye and hand movements can provide important insights into brain functions that are essential for performing or learning daily tasks in health and disease. This knowledge can also inform applications such as robotic manipulation and clinical rehabilitation.

Motor Planning Modulates Neural Activity Patterns in Early Human Auditory Cortex.

It is well established that movement planning recruits motor-related cortical brain areas in preparation for the forthcoming action. Given that an integral component to the control of action is the processing of sensory information throughout movement, we predicted that movement planning might also modulate early sensory cortical areas, readying them for sensory processing during the unfolding action. To test this hypothesis, we performed 2 human functional magnetic resonance imaging studies involving separate delayed movement tasks and focused on premovement neural activity in early auditory cortex, given the area's direct connections to the motor system and evidence that it is modulated by motor cortex during movement in rodents. We show that effector-specific information (i.e., movements of the left vs. right hand in Experiment 1 and movements of the hand vs. eye in Experiment 2) can be decoded, well before movement, from neural activity in early auditory cortex. We find that this motor-related information is encoded in a separate subregion of auditory cortex than sensory-related information and is present even when movements are cued visually instead of auditorily. These findings suggest that action planning, in addition to preparing the motor system for movement, involves selectively modulating primary sensory areas based on the intended action.

Dissociated Development of Speech and Limb Sensorimotor Learning in Stuttering: Speech Auditory-motor Learning is Impaired in Both Children and Adults Who Stutter.

Stuttering is a neurodevelopmental disorder of speech fluency. Various experimental paradigms have demonstrated that affected individuals show limitations in sensorimotor control and learning. However, controversy exists regarding two core aspects of this perspective. First, it has been claimed that sensorimotor learning limitations are detectable only in adults who stutter (after years of coping with the disorder) but not during childhood close to the onset of stuttering. Second, it remains unclear whether stuttering individuals' sensorimotor learning limitations affect only speech movements or also unrelated effector systems involved in nonspeech movements. We report data from separate experiments investigating speech auditory-motor learning (N = 60) and limb visuomotor learning (N = 84) in both children and adults who stutter versus matched nonstuttering individuals. Both children and adults who stutter showed statistically significant limitations in speech auditory-motor adaptation with formant-shifted feedback. This limitation was more profound in children than in adults and in younger children versus older children. Between-group differences in the adaptation of reach movements performed with rotated visual feedback were subtle but statistically significant for adults. In children, even the nonstuttering groups showed limited visuomotor adaptation just like their stuttering peers. We conclude that sensorimotor learning is impaired in individuals who stutter, and that the ability for speech auditory-motor learning-which was already adult-like in 3-6 year-old typically developing children-is severely compromised in young children near the onset of stuttering. Thus, motor learning limitations may play an important role in the fundamental mechanisms contributing to the onset of this speech disorder.

Motor memories in manipulation tasks are linked to contact goals between objects.

Skillful manipulation requires forming memories of object dynamics, linking applied force to motion. Although it has been assumed that such memories are linked to objects, a recent study showed that people can form separate memories when these are linked to different controlled points on an object (Heald JB, Ingram JN, Flanagan JR, Wolpert DM. Nat Hum Behav 2: 300-311, 2018). In that study, participants controlled the handle of a robotic device to move a virtual bar with circles (control points) on the left and right sides. Participants were instructed to move either the left or right control point to a target on the left or right, respectively, such that the required movement was constant. When these control points were paired with opposing force fields, adaptation was observed. In this previous study, both the controlled point and the target changed between contexts. To assess which of these factors is critical for learning, here, we used a similar paradigm but with a bar that automatically rotated as it was moved. In the first experiment, the bar rotated, such that the left and right control points moved to a common target. In the second experiment, the bar rotated such that a single control point moved to a target located on either the left or right. In both experiments, participants were able to learn opposing force fields applied in the two contexts. We conclude that separate memories of dynamics can be formed for different "contact goals," involving a unique combination of the controlled point on an object and the target location this point "contacts."NEW & NOTEWORTHY Skilled manipulation requires forming memories of object dynamics, previously assumed to be associated with entire objects. However, we recently demonstrated that people can form multiple motor memories when explicitly instructed to move different locations on an object to different targets. Here, we show that separate motor memories can be learned for different contact goals, which involve a unique combination of a control point and target.

Human decision making anticipates future performance in motor learning.

It is well-established that people can factor into account the distribution of their errors in motor performance so as to optimize reward. Here we asked whether, in the context of motor learning where errors decrease across trials, people take into account their future, improved performance so as to make optimal decisions to maximize reward. One group of participants performed a virtual throwing task in which, periodically, they were given the opportunity to select from a set of smaller targets of increasing value. A second group of participants performed a reaching task under a visuomotor rotation in which, after performing a initial set of trials, they selected a reward structure (ratio of points for target hits and misses) for different exploitation horizons (i.e., numbers of trials they might be asked to perform). Because movement errors decreased exponentially across trials in both learning tasks, optimal target selection (task 1) and optimal reward structure selection (task 2) required taking into account future performance. The results from both tasks indicate that people anticipate their future motor performance so as to make decisions that will improve their expected future reward.

Gaze behavior during visuomotor tracking with complex hand-cursor dynamics.

The ability to track a moving target with the hand has been extensively studied, but few studies have characterized gaze behavior during this task. Here we investigate gaze behavior when participants learn a new mapping between hand and cursor motion, such that the cursor represented the position of a virtual mass attached to the grasped handle via a virtual spring. Depending on the experimental condition, haptic feedback consistent with mass-spring dynamics could also be provided. For comparison a simple one-to-one hand-cursor mapping was also tested. We hypothesized that gaze would be drawn, at times, to the cursor in the mass-spring conditions, especially in the absence of haptic feedback. As expected hand tracking performance was less accurate under the spring mapping, but gaze behavior was virtually unaffected by the spring mapping, regardless of whether haptic feedback was provided. Specifically, relative gaze position between target and cursor, rate of saccades, and gain of smooth pursuit were similar under both mappings and both haptic feedback conditions. We conclude that even when participants are exposed to a challenging hand-cursor mapping, gaze is primarily concerned about ongoing target motion suggesting that peripheral vision is sufficient to monitor cursor position and to update hand movement control.

Separate motor memories are formed when controlling different implicitly specified locations on a tool.

Skillful manipulation requires forming and recalling memories of the dynamics of objects linking applied force to motion. It has been assumed that such memories are associated with entire objects. However, we often control different locations on an object, and these locations may be associated with different dynamics. We have previously demonstrated that multiple memories can be formed when participants are explicitly instructed to control different visual points marked on an object. A key question is whether this novel finding generalizes to more natural situations in which control points are implicitly defined by the task. To answer this question, we used objects with no explicit control points and tasks designed to encourage the use of distinct implicit control points. Participants moved a handle, attached to a robotic interface, to control the position of a rectangular object ("eraser") in the horizontal plane. Participants were required to move the eraser straight ahead to wipe away a column of dots ("dust"), located to either the left or right. We found that participants adapted to opposing dynamics when linked to the left and right dust locations, even though the movements required for these two contexts were the same. Control conditions showed this learning could not be accounted for by contextual cues or the fact that the task goal required moving in a straight line. These results suggest that people naturally control different locations on manipulated objects depending on the task context and that doing so affords the formation of separate motor memories. NEW & NOTEWORTHY Skilled manipulation requires forming motor memories of object dynamics, which have been assumed to be associated with entire objects. However, we recently demonstrated that people can form multiple memories when explicitly instructed to control different visual points on an object. In this article we show that this novel finding generalizes to more natural situations in which control points are implicitly defined by the task.