NSF DARE-transforming modeling in neurorehabilitation: a patient-in-the-loop framework.

In 2023, the National Science Foundation (NSF) and the National Institute of Health (NIH) brought together engineers, scientists, and clinicians by sponsoring a conference on computational modelling in neurorehabiilitation. To facilitate multidisciplinary collaborations and improve patient care, in this perspective piece we identify where and how computational modelling can support neurorehabilitation. To address the where, we developed a patient-in-the-loop framework that uses multiple and/or continual measurements to update diagnostic and treatment model parameters, treatment type, and treatment prescription, with the goal of maximizing clinically-relevant functional outcomes. This patient-in-the-loop framework has several key features: (i) it includes diagnostic and treatment models, (ii) it is clinically-grounded with the International Classification of Functioning, Disability and Health (ICF) and patient involvement, (iii) it uses multiple or continual data measurements over time, and (iv) it is applicable to a range of neurological and neurodevelopmental conditions. To address the how, we identify state-of-the-art and highlight promising avenues of future research across the realms of sensorimotor adaptation, neuroplasticity, musculoskeletal, and sensory & pain computational modelling. We also discuss both the importance of and how to perform model validation, as well as challenges to overcome when implementing computational models within a clinical setting. The patient-in-the-loop approach offers a unifying framework to guide multidisciplinary collaboration between computational and clinical stakeholders in the field of neurorehabilitation.

Punishment Leads to Greater Sensorimotor Learning But Less Movement Variability Compared to Reward.

When a musician practices a new song, hitting a correct note sounds pleasant while striking an incorrect note sounds unpleasant. Such reward and punishment feedback has been shown to differentially influence the ability to learn a new motor skill. Recent work has suggested that punishment leads to greater movement variability, which causes greater exploration and faster learning. To further test this idea, we collected 102 participants over two experiments. Unlike previous work, in Experiment 1 we found that punishment did not lead to faster learning compared to reward (n = 68), but did lead to a greater extent of learning. Surprisingly, we also found evidence to suggest that punishment led to less movement variability, which was related to the extent of learning. We then designed a second experiment that did not involve adaptation, allowing us to further isolate the influence of punishment feedback on movement variability. In Experiment 2, we again found that punishment led to significantly less movement variability compared to reward (n = 34). Collectively our results suggest that punishment feedback leads to less movement variability. Future work should investigate whether punishment feedback leads to a greater knowledge of movement variability and or increases the sensitivity of updating motor actions.

Reinforcement-based processes actively regulate motor exploration along redundant solution manifolds.

From a baby's babbling to a songbird practising a new tune, exploration is critical to motor learning. A hallmark of exploration is the emergence of random walk behaviour along solution manifolds, where successive motor actions are not independent but rather become serially dependent. Such exploratory random walk behaviour is ubiquitous across species' neural firing, gait patterns and reaching behaviour. The past work has suggested that exploratory random walk behaviour arises from an accumulation of movement variability and a lack of error-based corrections. Here, we test a fundamentally different idea-that reinforcement-based processes regulate random walk behaviour to promote continual motor exploration to maximize success. Across three human reaching experiments, we manipulated the size of both the visually displayed target and an unseen reward zone, as well as the probability of reinforcement feedback. Our empirical and modelling results parsimoniously support the notion that exploratory random walk behaviour emerges by utilizing knowledge of movement variability to update intended reach aim towards recently reinforced motor actions. This mechanism leads to active and continuous exploration of the solution manifold, currently thought by prominent theories to arise passively. The ability to continually explore muscle, joint and task redundant solution manifolds is beneficial while acting in uncertain environments, during motor development or when recovering from a neurological disorder to discover and learn new motor actions.

Validating the measurement of upper limb sensorimotor behavior utilizing a tablet in neurologically intact controls and individuals with chronic stroke.

BACKGROUND: Intact sensorimotor function of the upper extremity is essential for successfully performing activities of daily living. After a stroke, upper limb function is often compromised and requires rehabilitation. To develop appropriate rehabilitation interventions, sensitive and objective assessments are required. Current clinical measures often lack precision and technological devices (e.g. robotics) that are objective and sensitive to small changes in sensorimotor function are often unsuitable and impractical for performing home-based assessments. Here we developed a portable, tablet-based application capable of quantifying upper limb sensorimotor function after stroke. Our goal was to validate the developed application and accompanying data analysis against previously validated robotic measures of upper limb function in stroke. METHODS: Twenty individuals with stroke, twenty age-matched older controls, and twenty younger controls completed an eight-target Visually Guided Reaching (VGR) task using a Kinarm Robotic Exoskeleton and a Samsung Galaxy Tablet. Participants completed eighty trials of the VGR task on each device, where each trial consisted of making a reaching movement to one of eight pseudorandomly appearing targets. We calculated several outcome parameters capturing various aspects of sensorimotor behavior (e.g., Reaction Time, Initial Direction Error, Max Speed, and Movement Time) from each reaching movement, and our analyses compared metric consistency between devices. We used the previously validated Kinarm Standard Analysis (KSA) and a custom in-house analysis to calculate each outcome parameter. RESULTS: We observed strong correlations between the KSA and our custom analysis for all outcome parameters within each participant group, indicating our custom analysis accurately replicates the KSA. Minimal differences were observed for between-device comparisons (tablet vs. robot) in our outcome parameters. Additionally, we observed similar correlations for each device when comparing the Fugl-Meyer Assessment (FMA) scores of individuals with stroke to tablet-derived metrics, demonstrating that the tablet can capture clinically-based elements of upper limb impairment. CONCLUSIONS: Tablet devices can accurately assess upper limb sensorimotor function in neurologically intact individuals and individuals with stroke. Our findings validate the use of tablets as a cost-effective and efficient assessment tool for upper-limb function after stroke.

Visual accuracy dominates over haptic speed for state estimation of a partner during collaborative sensorimotor interactions.

We routinely have physical interactions with others, whether it be handing someone a glass of water or jointly moving a heavy object together. These sensorimotor interactions between humans typically rely on visual feedback and haptic feedback. Recent single-participant studies have highlighted that the unique noise and time delays of each sense must be considered to estimate the state, such as the position and velocity, of one's own movement. However, we know little about how visual feedback and haptic feedback are used to estimate the state of another person. Here, we tested how humans utilize visual feedback and haptic feedback to estimate the state of their partner during a collaborative sensorimotor task. Across two experiments, we show that visual feedback dominated haptic feedback during collaboration. Specifically, we found that visual feedback led to comparatively lower task-relevant movement variability, smoother collaborative movements, and faster trial completion times. We also developed an optimal feedback controller that considered the noise and time delays of both visual feedback and haptic feedback to estimate the state of a partner. This model was able to capture both lower task-relevant movement variability and smoother collaborative movements. Taken together, our empirical and modeling results support the idea that visual accuracy is more important than haptic speed to perform state estimation of a partner during collaboration.NEW & NOTEWORTHY Physical collaboration between two or more individuals involves both visual and haptic feedback. Here, we investigated how visual and haptic feedback is used to estimate the movements of a partner during a collaboration task. Our experimental and computational modeling results parsimoniously support the notion that greater visual accuracy is more important than faster yet noisier haptic feedback when estimating the state of a partner.

The nervous system tunes sensorimotor gains when reaching in variable mechanical environments.

Humans often move in the presence of mechanical disturbances that can vary in direction and amplitude throughout movement. These disturbances can jeopardize the outcomes of our actions, such as when drinking from a glass of water on a turbulent flight or carrying a cup of coffee while walking on a busy sidewalk. Here, we examine control strategies that allow the nervous system to maintain performance when reaching in the presence of mechanical disturbances that vary randomly throughout movement. Healthy participants altered their control strategies to make movements more robust against disturbances. The change in control was associated with faster reaching movements and increased responses to proprioceptive and visual feedback that were tuned to the variability of the disturbances. Our findings highlight that the nervous system exploits a continuum of control strategies to increase its responsiveness to sensory feedback when reaching in the presence of increasingly variable physical disturbances.

The sensorimotor system modulates muscular co-contraction relative to visuomotor feedback responses to regulate movement variability.

The naturally occurring variability in our movements often poses a significant challenge when attempting to produce precise and accurate actions, which is readily evident when playing a game of darts. Two differing, yet potentially complementary, control strategies that the sensorimotor system may use to regulate movement variability are impedance control and feedback control. Greater muscular co-contraction leads to greater impedance that acts to stabilize the hand, while visuomotor feedback responses can be used to rapidly correct for unexpected deviations when reaching toward a target. Here, we examined the independent roles and potential interplay of impedance control and visuomotor feedback control when regulating movement variability. Participants were instructed to perform a precise reaching task by moving a cursor through a narrow visual channel. We manipulated cursor feedback by visually amplifying movement variability and/or delaying the visual feedback of the cursor. We found that participants decreased movement variability by increasing muscular co-contraction, aligned with an impedance control strategy. Participants displayed visuomotor feedback responses during the task but, unexpectedly, there was no modulation between conditions. However, we did find a relationship between muscular co-contraction and visuomotor feedback responses, suggesting that participants modulated impedance control relative to feedback control. Taken together, our results highlight that the sensorimotor system modulates muscular co-contraction, relative to visuomotor feedback responses, to regulate movement variability and produce accurate actions.NEW & NOTEWORTHY The sensorimotor system has the constant challenge of dealing with the naturally occurring variability in our movements. Here, we investigated the potential roles of muscular co-contraction and visuomotor feedback responses to regulate movement variability. When we visually amplified movements, we found that the sensorimotor system primarily uses muscular co-contraction to regulate movement variability. Interestingly, we found that muscular co-contraction was modulated relative to inherent visuomotor feedback responses, suggesting an interplay between impedance and feedback control.

Failure induces task-irrelevant exploration during a stencil task.

During reward-based motor tasks, performance failure leads to an increase in movement variability along task-relevant dimensions. These increases in movement variability are indicative of exploratory behaviour in search of a better, more successful motor action. It is unclear whether failure also induces exploration along task-irrelevant dimensions that do not influence performance. In this study, we ask whether participants would explore the task-irrelevant dimension while they performed a stencil task. With a stylus, participants applied downward, normal force that influenced whether they received reward (task-relevant) as they simultaneously made erasing-like movement patterns along the tablet that did not influence performance (task-irrelevant). In this task, the movement pattern was analyzed as the distribution of movement directions within a movement. The results showed significant exploration of task-relevant force and task-irrelevant movement patterns. We conclude that failure can induce additional movement variability along a task-irrelevant dimension.

A response surface methodology to determine the optimal objective function weightings within a multi-objective optimization digital human model used to predict postures.

Multi-objective optimization digital human models permit users to predict postures that follow performance criteria, such as minimizing torques. Currently, it is unknown how to weight different objective functions to best predict postures. Objective one was to describe a response surface method to determine optimal objective function weightings to predict lift postures. Objective two was to evaluate the sensitivity of different error calculation methods. Our response surface approach has utility for determining optimal objective function weightings when using a digital human model to evaluate human-system interactions in early design stages. The approach was not dependent on variations in error calculation methods.

Accuracy and effort costs together lead to temporal asynchrony of multiple motor commands.

The timing of motor commands is critical for task performance. A well-known example is rapidly raising the arm while standing upright. Here, reaction forces from the arm movement to the body are countered by leg and trunk muscle activity starting before any sensory feedback from the perturbation and often before the onset of arm muscle activity. Despite decades of research on the patterns, modifiability, and neural basis of these "anticipatory postural adjustments," it remains unclear why asynchronous motor commands occur. Simple accuracy considerations appear unlikely since temporally advanced motor commands displace the body from its initial position. Effort is a credible and overlooked factor that has successfully explained coordination patterns of many behaviors including gait and reaching. We provide the first use of optimal control to address this question. Feedforward commands were applied to a body mass mechanically linked to a rapidly moving limb mass. We determined the feedforward actions with the lowest cost according to an explicit criterion, accuracy alone versus accuracy + effort. Accuracy costs alone led to synchronous activation of the body and limb controllers. Adding effort to the cost resulted in body commands preceding limb commands. This sequence takes advantage of the body's momentum in one direction to counter the limb's reaction force in the opposite direction, allowing a lower peak command and lower integral. With a combined accuracy + effort cost, temporal advancement was further impacted by various task goals and plant dynamics, replicating previous findings and suggesting further studies using optimal control principles.NEW & NOTEWORTHY An important goal in the fields of sensorimotor neuroscience and biomechanics is to explain the timing of different muscles during behavior. Here, we propose that energy and accuracy considerations underlie the asynchronous onset of postural and arm muscles during rapid movement. Our novel model-based framework replicates a broad range of observations across varying task demands and plant dynamics and offers a new perspective to study motor timing.

Exercising choice over feedback schedules during practice is not advantageous for motor learning.

The idea that there is a self-controlled learning advantage, where individuals demonstrate improved motor learning after exercising choice over an aspect of practice compared to no-choice groups, has different causal explanations according to the OPTIMAL theory or an information-processing perspective. Within OPTIMAL theory, giving learners choice is considered an autonomy-supportive manipulation that enhances expectations for success and intrinsic motivation. In the information-processing view, choice allows learners to engage in performance-dependent strategies that reduce uncertainty about task outcomes. To disentangle these potential explanations, we provided participants in choice and yoked groups with error or graded feedback (Experiment 1) and binary feedback (Experiment 2) while learning a novel motor task with spatial and timing goals. Across both experiments (N = 228 participants), we did not find any evidence to support a self-controlled learning advantage. Exercising choice during practice did not increase perceptions of autonomy, competence, or intrinsic motivation, nor did it lead to more accurate error estimation skills. Both error and graded feedback facilitated skill acquisition and learning, whereas no improvements from pre-test performance were found with binary feedback. Finally, the impact of graded and binary feedback on perceived competence highlights a potential dissociation of motivational and informational roles of feedback. Although our results regarding self-controlled practice conditions are difficult to reconcile with either the OPTIMAL theory or the information-processing perspective, they are consistent with a growing body of evidence that strongly suggests self-controlled conditions are not an effective approach to enhance motor performance and learning.

Exploring Optimal Objective Function Weightings to Predict Lifting Postures Under Unfatigued and Fatigued States.

OBJECTIVE: To explore whether the optimal objective function weightings change when using a digital human model (DHM) to predict origin and destination lifting postures under unfatigued and fatigued states. BACKGROUND: The ability to predict human postures can depend on state-based influences (e.g., fatigue). Altering objective function weightings within a predictive DHM could improve the ability to predict tasks specific lifting postures under unique fatigue states. METHOD: A multi-objective optimization-based DHM was used to predict origin and destination lifting postures for ten anthropometrically scaled avatars by using different objective functions weighting combinations. Predicted and measured postures were compared to determine the root mean squared error. A response surface methodology was used to identify the optimal objective function weightings, which was found by generating the posture that minimized error between measured and predicted lifting postures. The resultant weightings were compared to determine if the optimal objective function weightings changed for different lifting postures or fatigue states. RESULTS: Discomfort and total joint torque weightings were affected by posture (origin/destination) and fatigue state (unfatigued/fatigued); however, post-hoc differences between fatigue states and lifting postures were not sufficiently large to be detected. Weighting the discomfort objective function alone tended to predict postures that generalized well to both postures and fatigue states. CONCLUSION: Lift postures were optimal predicted using the minimization of discomfort objective function regardless of fatigue state. APPLICATION: Weighting the discomfort objective can predict unfatigued postures, but more research is needed to understand the optimal objective function weightings to predict postures during a fatigued state.

Humans utilize sensory evidence of others' intended action to make online decisions.

We often acquire sensory information from another person's actions to make decisions on how to move, such as when walking through a crowded hallway. Past interactive decision-making research has focused on cognitive tasks that did not allow for sensory information exchange between humans prior to a decision. Here, we test the idea that humans accumulate sensory evidence of another person's intended action to decide their own movement. In a competitive sensorimotor task, we show that humans exploit time to accumulate sensory evidence of another's intended action and utilize this information to decide how to move. We captured this continuous interactive decision-making behaviour with a drift-diffusion model. Surprisingly, aligned with a 'paralysis-by-analysis' phenomenon, we found that humans often waited too long to accumulate sensory evidence and failed to make a decision. Understanding how humans engage in interactive and online decision-making has broad implications that spans sociology, athletics, interactive technology, and economics.

Rapid Feedback Responses Parallel the Urgency of Voluntary Reaching Movements.

Optimal feedback control is a prominent theory used to interpret human motor behaviour. The theory posits that skilled actions emerge from control policies that link voluntary motor control (feedforward) with flexible feedback corrections (feedback control). It is clear the nervous system can generate flexible motor corrections (reflexes) when performing actions with different goals. We know little, however, about shared features of voluntary actions and feedback control in human movement. Here we reveal a link between the timing demands of voluntary actions and flexible responses to mechanical perturbations. In two experiments, 40 human participants (21 females) made reaching movements with different timing demands. We disturbed the arm with mechanical perturbations at movement onset (Experiment 1) and at locations ranging from movement onset to completion (Experiment 2). We used the resulting muscle responses and limb displacements as a proxy for the control policies that support voluntary reaching movements. We observed an increase in the sensitivity of elbow and shoulder muscle responses and a reduction in limb motion when the task imposed greater urgency to respond to the same perturbations. The results reveal a relationship between voluntary actions and feedback control as the limb was displaced less when moving faster in perturbation trials. Muscle responses scaled with changes in the displacement of the limb in perturbation trials within each timing condition. Across both experiments, human behaviour was captured by simulations based on stochastic optimal feedback control. Taken together, the results highlight flexible control that links sensory processing with features of human reaching movements.

Both fast and slow learning processes contribute to savings following sensorimotor adaptation.

Recent work suggests that the rate of learning in sensorimotor adaptation is likely not fixed, but rather can change based on previous experience. One example is savings, a commonly observed phenomenon whereby the relearning of a motor skill is faster than the initial learning. Sensorimotor adaptation is thought to be driven by sensory prediction errors, which are the result of a mismatch between predicted and actual sensory consequences. It has been proposed that during motor adaptation the generation of sensory prediction errors engages two processes (fast and slow) that differ in learning and retention rates. We tested the idea that a history of errors would influence both the fast and slow processes during savings. Participants were asked to perform the same force field adaptation task twice in succession. We found that adaptation to the force field a second time led to increases in estimated learning rates for both fast and slow processes. While it has been proposed that savings is explained by an increase in learning rate for the fast process, here we observed that the slow process also contributes to savings. Our work suggests that fast and slow adaptation processes are both responsive to a history of error and both contribute to savings. NEW & NOTEWORTHY We studied the underlying mechanisms of savings during motor adaptation. Using a two-state model to represent fast and slow processes that contribute to motor adaptation, we found that a history of error modulates performance in both processes. While previous research has attributed savings to only changes in the fast process, we demonstrated that an increase in both processes is needed to account for the measured behavioral data.

The gradient of the reinforcement landscape influences sensorimotor learning.

Consideration of previous successes and failures is essential to mastering a motor skill. Much of what we know about how humans and animals learn from such reinforcement feedback comes from experiments that involve sampling from a small number of discrete actions. Yet, it is less understood how we learn through reinforcement feedback when sampling from a continuous set of possible actions. Navigating a continuous set of possible actions likely requires using gradient information to maximize success. Here we addressed how humans adapt the aim of their hand when experiencing reinforcement feedback that was associated with a continuous set of possible actions. Specifically, we manipulated the change in the probability of reward given a change in motor action-the reinforcement gradient-to study its influence on learning. We found that participants learned faster when exposed to a steep gradient compared to a shallow gradient. Further, when initially positioned between a steep and a shallow gradient that rose in opposite directions, participants were more likely to ascend the steep gradient. We introduce a model that captures our results and several features of motor learning. Taken together, our work suggests that the sensorimotor system relies on temporally recent and spatially local gradient information to drive learning.

Neural signatures of reward and sensory error feedback processing in motor learning.

At least two distinct processes have been identified by which motor commands are adapted according to movement-related feedback: reward-based learning and sensory error-based learning. In sensory error-based learning, mappings between sensory targets and motor commands are recalibrated according to sensory error feedback. In reward-based learning, motor commands are associated with subjective value, such that successful actions are reinforced. We designed two tasks to isolate reward- and sensory error-based motor adaptation, and we used electroencephalography in humans to identify and dissociate the neural correlates of reward and sensory error feedback processing. We designed a visuomotor rotation task to isolate sensory error-based learning that was induced by altered visual feedback of hand position. In a reward learning task, we isolated reward-based learning induced by binary reward feedback that was decoupled from the visual target. A fronto-central event-related potential called the feedback-related negativity (FRN) was elicited specifically by binary reward feedback but not sensory error feedback. A more posterior component called the P300 was evoked by feedback in both tasks. In the visuomotor rotation task, P300 amplitude was increased by sensory error induced by perturbed visual feedback and was correlated with learning rate. In the reward learning task, P300 amplitude was increased by reward relative to nonreward and by surprise regardless of feedback valence. We propose that during motor adaptation the FRN specifically reflects a reward-based learning signal whereas the P300 reflects feedback processing that is related to adaptation more generally. NEW & NOTEWORTHY We studied the event-related potentials evoked by feedback stimuli during motor adaptation tasks that isolate reward- and sensory error-based learning mechanisms. We found that the feedback-related negativity was specifically elicited by binary reward feedback, whereas the P300 was observed in both tasks. These results reveal neural processes associated with different learning mechanisms and elucidate which classes of errors, from a computational standpoint, elicit the feedback-related negativity and P300.

Somatosensory perceptual training enhances motor learning by observing.

Action observation activates brain regions involved in sensory-motor control. Recent research has shown that action observation can also facilitate motor learning; observing a tutor undergoing motor learning results in functional plasticity within the motor system and gains in subsequent motor performance. However, the effects of observing motor learning extend beyond the motor domain. Converging evidence suggests that observation also results in somatosensory functional plasticity and somatosensory perceptual changes. This work has raised the possibility that the somatosensory system is also involved in motor learning that results from observation. Here we tested this hypothesis using a somatosensory perceptual training paradigm. If the somatosensory system is indeed involved in motor learning by observing, then improving subjects' somatosensory function before observation should enhance subsequent motor learning by observing. Subjects performed a proprioceptive discrimination task in which a robotic manipulandum moved the arm, and subjects made judgments about the position of their hand. Subjects in a Trained Learning group received trial-by-trial feedback to improve their proprioceptive perception. Subjects in an Untrained Learning group performed the same task without feedback. All subjects then observed a learning video showing a tutor adapting her reaches to a left force field. Subjects in the Trained Learning group, who had superior proprioceptive acuity before observation, benefited more from observing learning than subjects in the Untrained Learning group. Improving somatosensory function can therefore enhance subsequent observation-related gains in motor learning. This study provides further evidence in favor of the involvement of the somatosensory system in motor learning by observing. NEW & NOTEWORTHY We show that improving somatosensory performance before observation can improve the extent to which subjects learn from watching others. Somatosensory perceptual training may prime the sensory-motor system, thereby facilitating subsequent observational learning. The findings of this study suggest that the somatosensory system supports motor learning by observing. This finding may be useful if observation is incorporated as part of therapies for diseases affecting movement, such as stroke.

Dissociating error-based and reinforcement-based loss functions during sensorimotor learning.

It has been proposed that the sensorimotor system uses a loss (cost) function to evaluate potential movements in the presence of random noise. Here we test this idea in the context of both error-based and reinforcement-based learning. In a reaching task, we laterally shifted a cursor relative to true hand position using a skewed probability distribution. This skewed probability distribution had its mean and mode separated, allowing us to dissociate the optimal predictions of an error-based loss function (corresponding to the mean of the lateral shifts) and a reinforcement-based loss function (corresponding to the mode). We then examined how the sensorimotor system uses error feedback and reinforcement feedback, in isolation and combination, when deciding where to aim the hand during a reach. We found that participants compensated differently to the same skewed lateral shift distribution depending on the form of feedback they received. When provided with error feedback, participants compensated based on the mean of the skewed noise. When provided with reinforcement feedback, participants compensated based on the mode. Participants receiving both error and reinforcement feedback continued to compensate based on the mean while repeatedly missing the target, despite receiving auditory, visual and monetary reinforcement feedback that rewarded hitting the target. Our work shows that reinforcement-based and error-based learning are separable and can occur independently. Further, when error and reinforcement feedback are in conflict, the sensorimotor system heavily weights error feedback over reinforcement feedback.