Subtle Visual Latency Can Profoundly Impair Implicit Sensorimotor Learning.

Short sub-100ms visual feedback latencies are common in many types of human-computer interactions yet are known to markedly reduce performance in a wide variety of motor tasks from simple pointing to operating surgical robotics. These latencies are also present in the computer-based experiments used to study the sensorimotor learning that underlies the acquisition of motor performance. Inspired by neurophysiological findings showing that cerebellar LTD and cortical LTP would both be disrupted by sub-100ms latencies, we hypothesized that implicit sensorimotor learning may be particularly sensitive to these short latencies. Remarkably, we find that improving latency by just 60ms, from 85 to 25ms in latency-optimized experiments, increases implicit learning by 50% and proportionally decreases explicit learning, resulting in a dramatic reorganization of sensorimotor memory. We go on to show that implicit sensorimotor learning is considerably more sensitive to latencies in the sub-100ms range than at higher latencies, in line with the latency-specific neural plasticity that has been observed. This suggests a clear benefit for latency reduction in computer-based training that involves implicit sensorimotor learning and that across-study differences in implicit motor learning might often be explained by disparities in feedback latency.

The cerebellum acts as the analog to the medial temporal lobe for sensorimotor memory.

The cerebellum is critical for sensorimotor learning. The specific contribution that it makes, however, remains unclear. Inspired by the classic finding that, for declarative memories, medial temporal lobe structures provide a gateway to the formation of long-term memory but are not required for short-term memory, we hypothesized that, for sensorimotor memories, the cerebellum may play an analogous role. Here we studied the sensorimotor learning of individuals with severe ataxia from cerebellar degeneration. We dissected the memories they formed during sensorimotor learning into a short-term temporally-volatile component, that decays rapidly with a time constant of just 15-20sec and thus cannot lead to long-term retention, and a longer-term temporally-persistent component that is stable for 60 sec or more and leads to long-term retention. Remarkably, we find that these individuals display dramatically reduced levels of temporally-persistent sensorimotor memory, despite spared and even elevated levels of temporally-volatile sensorimotor memory. In particular, we find both impairment that systematically increases with memory window duration over shorter memory windows (<12 sec) and near-complete impairment of memory maintenance over longer memory windows (>25 sec). This dissociation uncovers a new role for the cerebellum as a gateway for the formation of long-term but not short-term sensorimotor memories, mirroring the role of the medial temporal lobe for declarative memories. It thus reveals the existence of distinct neural substrates for short-term and long-term sensorimotor memory, and it explains both newly-identified trial-to-trial differences and long-standing study-to-study differences in the effects of cerebellar damage on sensorimotor learning ability. Significance Statement: A key discovery about the neural underpinnings of memory, made more than half a century ago, is that long-term, but not short-term, memory formation depends on neural structures in the brain's medial temporal lobe (MTL). However, this dichotomy holds only for declarative memories - memories for explicit facts such as names and dates - as long-term procedural memories - memories for implicit knowledge such as sensorimotor skills - are largely unaffected even with substantial MTL damage. Here we demonstrate that the formation of long-term, but not short-term, sensorimotor memory depends on a neural structure known as the cerebellum, and we show that this finding explains the variability previously reported in the extent to which cerebellar damage affects sensorimotor learning.

A double dissociation between savings and long-term memory in motor learning.

Memories are easier to relearn than learn from scratch. This advantage, known as savings, has been widely assumed to result from the reemergence of stable long-term memories. In fact, the presence of savings has often been used as a marker for whether a memory has been consolidated. However, recent findings have demonstrated that motor learning rates can be systematically controlled, providing a mechanistic alternative to the reemergence of a stable long-term memory. Moreover, recent work has reported conflicting results about whether implicit contributions to savings in motor learning are present, absent, or inverted, suggesting a limited understanding of the underlying mechanisms. To elucidate these mechanisms, we investigate the relationship between savings and long-term memory by experimentally dissecting the underlying memories based on short-term (60-s) temporal persistence. Components of motor memory that are temporally-persistent at 60 s might go on to contribute to stable, consolidated long-term memory, whereas temporally-volatile components that have already decayed away by 60 s cannot. Surprisingly, we find that temporally-volatile implicit learning leads to savings, whereas temporally-persistent learning does not, but that temporally-persistent learning leads to long-term memory at 24 h, whereas temporally-volatile learning does not. This double dissociation between the mechanisms for savings and long-term memory formation challenges widespread assumptions about the connection between savings and memory consolidation. Moreover, we find that temporally-persistent implicit learning not only fails to contribute to savings, but also that it produces an opposite, anti-savings effect, and that the interplay between this temporally-persistent anti-savings and temporally-volatile savings provides an explanation for several seemingly conflicting recent reports about whether implicit contributions to savings are present, absent, or inverted. Finally, the learning curves we observed for the acquisition of temporally-volatile and temporally-persistent implicit memories demonstrate the coexistence of implicit memories with distinct time courses, challenging the assertion that models of context-based learning and estimation should supplant models of adaptive processes with different learning rates. Together, these findings provide new insight into the mechanisms for savings and long-term memory formation.

Reinforcement Learning Is Impaired in the Sub-acute Post-stroke Period.

Background: Neurorehabilitation approaches are frequently predicated on motor learning principles. However, much is left to be understood of how different kinds of motor learning are affected by stroke causing hemiparesis. Here we asked if two kinds of motor learning often employed in rehabilitation, (1) reinforcement learning and (2) error-based adaptation, are altered at different times after stroke. Methods: In a cross-sectional design, we compared learning in two groups of patients with stroke, matched for their baseline motor execution deficit on the paretic side. The early group was tested within 3 months following stroke (N = 35) and the late group was tested more than 6 months after stroke (N = 30). Two types of task were studied: one based on reinforcement learning and the other on error-based learning. Results: We found that reinforcement learning was impaired in the early but not the late group, whereas error-based learning was unaffected compared to controls. These findings could not be attributed to differences in baseline execution, cognitive impairment, gender, age, or lesion volume and location. Conclusions: The presence of a specific impairment in reinforcement learning in the first 3 months after stroke has important implications for rehabilitation. It might be necessary to either increase the amount of reinforcement feedback given early or even delay onset of certain forms of rehabilitation training, e.g., like constraint-induced movement therapy, and instead emphasize others forms of motor learning in this early time period. A deeper understanding of stroke-related changes in motor learning capacity has the potential to facilitate the development of new, more precise treatment interventions.

Dissociation between abnormal motor synergies and impaired reaching dexterity after stroke.

Most patients with stroke experience motor deficits, usually referred to collectively as hemiparesis. Although hemiparesis is one of the most common and clinically recognizable motor abnormalities, it remains undercharacterized in terms of its behavioral subcomponents and their interactions. Hemiparesis comprises both negative and positive motor signs. Negative signs consist of weakness and loss of motor control (dexterity), whereas positive signs consist of spasticity, abnormal resting posture, and intrusive movement synergies (abnormal muscle co-activations during voluntary movement). How positive and negative signs interact, and whether a common mechanism generates them, remains poorly understood. Here, we used a planar, arm-supported reaching task to assess poststroke arm dexterity loss, which we compared with the Fugl-Meyer stroke scale; a measure primarily reflecting abnormal synergies. We examined 53 patients with hemiparesis after a first-time ischemic stroke. Reaching kinematics were markedly more impaired in patients with subacute (<3 mo) compared to chronic (>6 mo) stroke even for similar Fugl-Meyer scores. This suggests a dissociation between abnormal synergies (reflected in the Fugl-Meyer scale) and loss of dexterity, which in turn suggests different underlying mechanisms. Moreover, dynamometry suggested that Fugl-Meyer scores capture weakness as well as abnormal synergies, in line with these two deficits sharing a neural substrate. These findings have two important implications: First, clinical studies that test for efficacy of rehabilitation interventions should specify which component of hemiparesis they are targeting and how they propose to measure it. Metrics used widely for this purpose may not always be chosen appropriately. For example, as we show here, the Fugl-Meyer score may capture some hemiparesis components (abnormal synergies and weakness) but not others (loss of dexterity). Second, there may be an opportunity to design rehabilitation interventions to address specific subcomponents of hemiparesis.NEW & NOTEWORTHY Motor impairment is common after stroke and comprises reduced dexterity, weakness, and abnormal muscle synergies. Here we report that, when matched on an established synergy and weakness scale (Fugl-Meyer), patients with subacute stroke have worse reaching dexterity than chronic ones. This result suggests that the components of hemiparesis are dissociable and have separable mechanisms and, thus, may require distinct assessments and treatments.

Did We Get Sensorimotor Adaptation Wrong? Implicit Adaptation as Direct Policy Updating Rather than Forward-Model-Based Learning.

The human motor system can rapidly adapt its motor output in response to errors. The prevailing theory of this process posits that the motor system adapts an internal forward model that predicts the consequences of outgoing motor commands and uses this forward model to plan future movements. However, despite clear evidence that adaptive forward models exist and are used to help track the state of the body, there is no definitive evidence that such models are used in movement planning. An alternative to the forward-model-based theory of adaptation is that movements are generated based on a learned policy that is adjusted over time by movement errors directly ("direct policy learning"). This learning mechanism could act in parallel with, but independent of, any updates to a predictive forward model. Forward-model-based learning and direct policy learning generate very similar predictions about behavior in conventional adaptation paradigms. However, across three experiments with human participants (N = 47, 26 female), we show that these mechanisms can be dissociated based on the properties of implicit adaptation under mirror-reversed visual feedback. Although mirror reversal is an extreme perturbation, it still elicits implicit adaptation; however, this adaptation acts to amplify rather than to reduce errors. We show that the pattern of this adaptation over time and across targets is consistent with direct policy learning but not forward-model-based learning. Our findings suggest that the forward-model-based theory of adaptation needs to be re-examined and that direct policy learning provides a more plausible explanation of implicit adaptation.SIGNIFICANCE STATEMENT The ability of our brain to adapt movements in response to error is one of the most widely studied phenomena in motor learning. Yet, we still do not know the process by which errors eventually result in adaptation. It is known that the brain maintains and updates an internal forward model, which predicts the consequences of motor commands, and the prevailing theory of motor adaptation posits that this updated forward model is responsible for trial-by-trial adaptive changes. Here, we question this view and show instead that adaptation is better explained by a simpler process whereby motor output is directly adjusted by task errors. Our findings cast doubt on long-held beliefs about adaptation.

Postural control of arm and fingers through integration of movement commands.

Every movement ends in a period of stillness. Current models assume that commands that hold the limb at a target location do not depend on the commands that moved the limb to that location. Here, we report a surprising relationship between movement and posture in primates: on a within-trial basis, the commands that hold the arm and finger at a target location depend on the mathematical integration of the commands that moved the limb to that location. Following damage to the corticospinal tract, both the move and hold period commands become more variable. However, the hold period commands retain their dependence on the integral of the move period commands. Thus, our data suggest that the postural controller possesses a feedforward module that uses move commands to calculate a component of hold commands. This computation may arise within an unknown subcortical system that integrates cortical commands to stabilize limb posture.

Moving an arm requires the brain to send electrical signals to the arm's muscles, causing them to contract. Neuroscientists call these types of brain signals "move signals". The brain also sends so-called hold signals, which hold the arm still in a desired position. Part of the brain known as the primary motor cortex helps to calculate the move signals for the arm, but it was unclear how the brain produces the corresponding hold signals. Fortunately, the fact that the brain moves other things besides arms may help answer this question. Previous research has shown, for example, that a brain area called the "neural integrator" calculates the hold signals needed to hold the eye in a specific position. The neural integrator does this by using basic principles of physics, and details of the speed and duration of the eye's movements. Now, Albert et al. show a similar mechanism appears to control hold signals for arm movements. In one set of experiments, muscle activity was measured as monkeys moved their arms or fingers to different target positions. In other experiments, human volunteers held a robot arm, and Albert et al. measured the forces they produced while reaching and holding still. Both the human and monkey experiments revealed a relationship between move signals and hold signals. Like for eye movements, hold signals for the arm could be calculated from the move signals. In further experiments with stroke patients where the brain had been damaged, the move signals were found to be deteriorated, but the way hold signals were calculated stayed the same. This suggests that there is an unknown structure within the brain that calculates hold signals based on move signals. Investigating how the brain holds the arm still may help scientists understand why some neurological conditions like stroke or dystonia cause unwanted movements or unusual postures. This might also lead scientists to develop new ways to treat these conditions.

Motor Learning.

Motor learning encompasses a wide range of phenomena, ranging from relatively low-level mechanisms for maintaining calibration of our movements, to making high-level cognitive decisions about how to act in a novel situation. We survey the major existing approaches to characterizing motor learning at both the behavioral and neural level. In particular, we critically review two long-standing paradigms used in motor learning research-adaptation and sequence learning. We discuss the extent to which these paradigms can be considered models of motor skill acquisition, defined as the incremental improvement in our ability to rapidly select and then precisely execute appropriate actions, and conclude that they fall short of doing so. We then discuss two classes of emerging research paradigms-learning of arbitrary visuomotor mappings de novo and learning to execute movements with improved acuity-that more effectively address the acquisition of motor skill. Future work will be needed to determine the degree to which laboratory-based studies of skill, as described in this review, will relate to true expertise, which is likely dependent on the effects of practice on multiple cognitive processes that go beyond traditional sensorimotor neural architecture. © 2019 American Physiological Society. Compr Physiol 9:613-663, 2019.

Flexible Control of Safety Margins for Action Based on Environmental Variability.

To reduce the risk of slip, grip force (GF) control includes a safety margin above the force level ordinarily sufficient for the expected load force (LF) dynamics. The current view is that this safety margin is based on the expected LF dynamics, amounting to a static safety factor like that often used in engineering design. More efficient control could be achieved, however, if the motor system reduces the safety margin when LF variability is low and increases it when this variability is high. Here we show that this is indeed the case by demonstrating that the human motor system sizes the GF safety margin in proportion to an internal estimate of LF variability to maintain a fixed statistical confidence against slip. In contrast to current models of GF control that neglect the variability of LF dynamics, we demonstrate that GF is threefold more sensitive to the SD than the expected value of LF dynamics, in line with the maintenance of a 3-sigma confidence level. We then show that a computational model of GF control that includes a variability-driven safety margin predicts highly asymmetric GF adaptation between increases versus decreases in load. We find clear experimental evidence for this asymmetry and show that it explains previously reported differences in how rapidly GFs and manipulatory forces adapt. This model further predicts bizarre nonmonotonic shapes for GF learning curves, which are faithfully borne out in our experimental data. Our findings establish a new role for environmental variability in the control of action.

Environmental consistency determines the rate of motor adaptation.

BACKGROUND: The motor system has the remarkable ability not only to learn but also to learn how fast it should learn. However, the mechanisms behind this ability are not well understood. Previous studies have posited that the rate of adaptation in a given environment is determined by Bayesian sensorimotor integration based on the amount of variability in the state of the environment. However, experimental results have failed to support several predictions of this theory. RESULTS: We show that the rate at which the motor system adapts to changes in the environment is primarily determined not by the degree to which environmental change occurs but by the degree to which the changes that do occur persist from one movement to the next, i.e., the consistency of the environment. We demonstrate a striking double dissociation whereby feedback response strength is predicted by environmental variability rather than consistency, whereas adaptation rate is predicted by environmental consistency rather than variability. We proceed to elucidate the role of stimulus repetition in speeding up adaptation and find that repetition can greatly potentiate the effect of consistency, although unlike consistency, repetition alone does not increase adaptation rate. By leveraging this understanding, we demonstrate that the rate of motor adaptation can be modulated over a range that encompasses a 20-fold increase from lowest to highest. CONCLUSIONS: Understanding the mechanisms that determine the rate of motor adaptation could lead to the principled design of improved procedures for motor training and rehabilitation. Regimens designed to control environmental consistency and repetition during training might yield faster, more robust motor learning.

The motor system estimates uncertainty and higher order statistics for the control of grip forces.

Successful manipulation of an object requires exerting grip forces (GF) sufficient to prevent slippage. To prevent slip in more uncertain environments, GF would need to increase. Here we investigate the brain's ability to efficiently control grasp by producing GFs that correspond to confidence estimates of uncertain environments that are characterized by probability density functions of different variances and higher order moments. We found that GFs increased dramatically with the level of environmental uncertainty, and even when environmental uncertainty was held constant while higher order moments were varied, GFs changes in a way that was appropriate for kurtosis.