NSF DARE-transforming modeling in neurorehabilitation: a patient-in-the-loop framework.

In 2023, the National Science Foundation (NSF) and the National Institute of Health (NIH) brought together engineers, scientists, and clinicians by sponsoring a conference on computational modelling in neurorehabiilitation. To facilitate multidisciplinary collaborations and improve patient care, in this perspective piece we identify where and how computational modelling can support neurorehabilitation. To address the where, we developed a patient-in-the-loop framework that uses multiple and/or continual measurements to update diagnostic and treatment model parameters, treatment type, and treatment prescription, with the goal of maximizing clinically-relevant functional outcomes. This patient-in-the-loop framework has several key features: (i) it includes diagnostic and treatment models, (ii) it is clinically-grounded with the International Classification of Functioning, Disability and Health (ICF) and patient involvement, (iii) it uses multiple or continual data measurements over time, and (iv) it is applicable to a range of neurological and neurodevelopmental conditions. To address the how, we identify state-of-the-art and highlight promising avenues of future research across the realms of sensorimotor adaptation, neuroplasticity, musculoskeletal, and sensory & pain computational modelling. We also discuss both the importance of and how to perform model validation, as well as challenges to overcome when implementing computational models within a clinical setting. The patient-in-the-loop approach offers a unifying framework to guide multidisciplinary collaboration between computational and clinical stakeholders in the field of neurorehabilitation.

Optimizing exoskeleton assistance to improve walking speed and energy economy for older adults.

BACKGROUND: Walking speed and energy economy tend to decline with age. Lower-limb exoskeletons have demonstrated potential to improve either measure, but primarily in studies conducted on healthy younger adults. Promising techniques like optimization of exoskeleton assistance have yet to be tested with older populations, while speed and energy consumption have yet to be simultaneously optimized for any population. METHODS: We investigated the effectiveness of human-in-the-loop optimization of ankle exoskeletons with older adults. Ten healthy adults > 65 years of age (5 females; mean age: 72 ± 3 yrs) participated in approximately 240 min of training and optimization with tethered ankle exoskeletons on a self-paced treadmill. Multi-objective human-in-the-loop optimization was used to identify assistive ankle plantarflexion torque patterns that simultaneously improved self-selected walking speed and metabolic rate. The effects of optimized exoskeleton assistance were evaluated in separate trials. RESULTS: Optimized exoskeleton assistance improved walking performance for older adults. Both objectives were simultaneously improved; self-selected walking speed increased by 8% (0.10 m/s; p = 0.001) and metabolic rate decreased by 19% (p = 0.007), resulting in a 25% decrease in energetic cost of transport (p = 8e-4) compared to walking with exoskeletons applying zero torque. Compared to younger participants in studies optimizing a single objective, our participants required lower exoskeleton torques, experienced smaller improvements in energy use, and required more time for motor adaptation. CONCLUSIONS: Our results confirm that exoskeleton assistance can improve walking performance for older adults and show that multiple objectives can be simultaneously addressed through human-in-the-loop optimization.

Optimized hip-knee-ankle exoskeleton assistance reduces the metabolic cost of walking with worn loads.

BACKGROUND: Load carriage is common in a wide range of professions, but prolonged load carriage is associated with increased fatigue and overuse injuries. Exoskeletons could improve the quality of life of these professionals by reducing metabolic cost to combat fatigue and reducing muscle activity to prevent injuries. Current exoskeletons have reduced the metabolic cost of loaded walking by up to 22% relative to walking in the device with no assistance when assisting one or two joints. Greater metabolic reductions may be possible with optimized assistance of the entire leg. METHODS: We used human-in the-loop optimization to optimize hip-knee-ankle exoskeleton assistance with no additional load, a light load (15% of body weight), and a heavy load (30% of body weight) for three participants. All loads were applied through a weight vest with an attached waist belt. We measured metabolic cost, exoskeleton assistance, kinematics, and muscle activity. We performed Friedman's tests to analyze trends across worn loads and paired t-tests to determine whether changes from the unassisted conditions to the assisted conditions were significant. RESULTS: Exoskeleton assistance reduced the metabolic cost of walking relative to walking in the device without assistance for all tested conditions. Exoskeleton assistance reduced the metabolic cost of walking by 48% with no load (p = 0.05), 41% with the light load (p = 0.01), and 43% with the heavy load (p = 0.04). The smaller metabolic reduction with the light load may be due to insufficient participant training or lack of optimizer convergence. The total applied positive power was similar for all tested conditions, and the positive knee power decreased slightly as load increased. Optimized torque timing parameters were consistent across participants and load conditions while optimized magnitude parameters varied. CONCLUSIONS: Whole-leg exoskeleton assistance can reduce the metabolic cost of walking while carrying a range of loads. The consistent optimized timing parameters across participants and conditions suggest that metabolic cost reductions are sensitive to torque timing. The variable torque magnitude parameters could imply that torque magnitude should be customized to the individual, or that there is a range of useful torque magnitudes. Future work should test whether applying the load to the exoskeleton rather than the person's torso results in larger benefits.

Optimized hip-knee-ankle exoskeleton assistance at a range of walking speeds.

BACKGROUND: Autonomous exoskeletons will need to be useful at a variety of walking speeds, but it is unclear how optimal hip-knee-ankle exoskeleton assistance should change with speed. Biological joint moments tend to increase with speed, and in some cases, optimized ankle exoskeleton torques follow a similar trend. Ideal hip-knee-ankle exoskeleton torque may also increase with speed. The purpose of this study was to characterize the relationship between walking speed, optimal hip-knee-ankle exoskeleton assistance, and the benefits to metabolic energy cost. METHODS: We optimized hip-knee-ankle exoskeleton assistance to reduce metabolic cost for three able-bodied participants walking at 1.0 m/s, 1.25 m/s and 1.5 m/s. We measured metabolic cost, muscle activity, exoskeleton assistance and kinematics. We performed Friedman's tests to analyze trends across walking speeds and paired t-tests to determine if changes from the unassisted conditions to the assisted conditions were significant. RESULTS: Exoskeleton assistance reduced the metabolic cost of walking compared to wearing the exoskeleton with no torque applied by 26%, 47% and 50% at 1.0, 1.25 and 1.5 m/s, respectively. For all three participants, optimized exoskeleton ankle torque was the smallest for slow walking, while hip and knee torque changed slightly with speed in ways that varied across participants. Total applied positive power increased with speed for all three participants, largely due to increased joint velocities, which consistently increased with speed. CONCLUSIONS: Exoskeleton assistance is effective at a range of speeds and is most effective at medium and fast walking speeds. Exoskeleton assistance was less effective for slow walking, which may explain the limited success in reducing metabolic cost for patient populations through exoskeleton assistance. Exoskeleton designers may have more success when targeting activities and groups with faster walking speeds. Speed-related changes in optimized exoskeleton assistance varied by participant, indicating either the benefit of participant-specific tuning or that a wide variety of torque profiles are similarly effective.

Deep reinforcement learning for modeling human locomotion control in neuromechanical simulation.

Modeling human motor control and predicting how humans will move in novel environments is a grand scientific challenge. Researchers in the fields of biomechanics and motor control have proposed and evaluated motor control models via neuromechanical simulations, which produce physically correct motions of a musculoskeletal model. Typically, researchers have developed control models that encode physiologically plausible motor control hypotheses and compared the resulting simulation behaviors to measurable human motion data. While such plausible control models were able to simulate and explain many basic locomotion behaviors (e.g. walking, running, and climbing stairs), modeling higher layer controls (e.g. processing environment cues, planning long-term motion strategies, and coordinating basic motor skills to navigate in dynamic and complex environments) remains a challenge. Recent advances in deep reinforcement learning lay a foundation for modeling these complex control processes and controlling a diverse repertoire of human movement; however, reinforcement learning has been rarely applied in neuromechanical simulation to model human control. In this paper, we review the current state of neuromechanical simulations, along with the fundamentals of reinforcement learning, as it applies to human locomotion. We also present a scientific competition and accompanying software platform, which we have organized to accelerate the use of reinforcement learning in neuromechanical simulations. This "Learn to Move" competition was an official competition at the NeurIPS conference from 2017 to 2019 and attracted over 1300 teams from around the world. Top teams adapted state-of-the-art deep reinforcement learning techniques and produced motions, such as quick turning and walk-to-stand transitions, that have not been demonstrated before in neuromechanical simulations without utilizing reference motion data. We close with a discussion of future opportunities at the intersection of human movement simulation and reinforcement learning and our plans to extend the Learn to Move competition to further facilitate interdisciplinary collaboration in modeling human motor control for biomechanics and rehabilitation research.

Optimizing Exoskeleton Assistance for Faster Self-Selected Walking.

Self-selected walking speed is an important aspect of mobility. Exoskeletons can increase walking speed, but the mechanisms behind these changes and the upper limits on performance are unknown. Human-in-the-loop optimization is a technique for identifying exoskeleton characteristics that maximize the benefits of assistance, which has been critical to achieving large improvements in energy economy. In this study, we used human-in-the-loop optimization to test whether large improvements in self-selected walking speed are possible through ankle exoskeleton assistance. Healthy participants (N =10) were instructed to walk at a comfortable speed on a self-paced treadmill while wearing tethered ankle exoskeletons. An algorithm sequentially applied different patterns of exoskeleton torque and estimated the speed-optimal pattern, which was then evaluated in separate trials. With torque optimized for speed, participants walked 42% faster than in normal shoes (1.83 ms-1 vs. 1.31 ms-1; Tukey HSD, p = 4 ×10-8 ), with speed increases ranging from 6% to 91%. Participants walked faster with speed-optimized torque than with torque optimized for energy consumption (1.55 ms-1) or torque chosen to induce slow walking (1.18 ms-1). Gait characteristics with speed-optimized torque were highly variable across participants, and changes in metabolic cost of transport ranged from a 31% decrease to a 78% increase, with a decrease of 2% on average. These results demonstrate that ankle exoskeletons can facilitate large increases in self-selected walking speed, which could benefit older adults and others with reduced walking speed.

Using force data to self-pace an instrumented treadmill and measure self-selected walking speed.

BACKGROUND: Self-selected speed is an important functional index of walking. A self-pacing controller that reliably matches walking speed without additional hardware can be useful for measuring self-selected speed in a treadmill-based laboratory. METHODS: We adapted a previously proposed self-pacing controller for force-instrumented treadmills and validated its use for measuring self-selected speeds. We first evaluated the controller's estimation of subject speed and position from the force-plates by comparing it to those from motion capture data. We then compared five tests of self-selected speed. Ten healthy adults completed a standard 10-meter walk test, a 150-meter walk test, a commonly used manual treadmill speed selection test, a two-minute self-paced treadmill test, and a 150-meter self-paced treadmill test. In each case, subjects were instructed to walk at or select their comfortable speed. We also assessed the time taken for a trial and a survey on comfort and ease of choosing a speed in all the tests. RESULTS: The self-pacing algorithm estimated subject speed and position accurately, with root mean square differences compared to motion capture of 0.023 m s -1 and 0.014 m, respectively. Self-selected speeds from both self-paced treadmill tests correlated well with those from the 10-meter walk test (R>0.93,p<1×10-13). Subjects walked slower on average in the self-paced treadmill tests (1.23±0.27 ms-1) than in the 10-meter walk test (1.32±0.18 ms-1) but the speed differences within subjects were consistent. These correlations and walking speeds are comparable to those from the manual treadmill speed selection test (R=0.89,p=3×10-11;1.18±0.24 ms-1). Comfort and ease of speed selection were similar in the self-paced tests and the manual speed selection test, but the self-paced tests required only about a third of the time to complete. Our results demonstrate that these self-paced treadmill tests can be a strong alternative to the commonly used manual treadmill speed selection test. CONCLUSIONS: The self-paced force-instrumented treadmill well adapts to subject walking speed and reliably measures self-selected walking speeds. We provide the self-pacing software to facilitate use by gait researchers and clinicians.

Predictive neuromechanical simulations indicate why walking performance declines with ageing.

KEY POINTS: Although the natural decline in walking performance with ageing affects the quality of life of a growing elderly population, its physiological origins remain unknown. By using predictive neuromechanical simulations of human walking with age-related neuro-musculo-skeletal changes, we find evidence that the loss of muscle strength and muscle contraction speed dominantly contribute to the reduced walking economy and speed. The findings imply that focusing on recovering these muscular changes may be the only effective way to improve performance in elderly walking. More generally, the work is of interest for investigating the physiological causes of altered gait due to age, injury and disorders. ABSTRACT: Healthy elderly people walk slower and energetically less efficiently than young adults. This decline in walking performance lowers the quality of life for a growing ageing population, and understanding its physiological origin is critical for devising interventions that can delay or revert it. However, the origin of the decline in walking performance remains unknown, as ageing produces a range of physiological changes whose individual effects on gait are difficult to separate in experiments with human subjects. Here we use a predictive neuromechanical model to separately address the effects of common age-related changes to the skeletal, muscular and nervous systems. We find in computer simulations of this model that the combined changes produce gait consistent with elderly walking and that mainly the loss of muscle strength and mass reduces energy efficiency. In addition, we find that the slower preferred walking speed of elderly people emerges in the simulations when adapting to muscle fatigue, again mainly caused by muscle-related changes. The results suggest that a focus on recovering these muscular changes may be the only effective way to improve performance in elderly walking.

Evaluation of a Neuromechanical Walking Control Model Using Disturbance Experiments.

Neuromechanical simulations have been used to study the spinal control of human locomotion which involves complex mechanical dynamics. So far, most neuromechanical simulation studies have focused on demonstrating the capability of a proposed control model in generating normal walking. As many of these models with competing control hypotheses can generate human-like normal walking behaviors, a more in-depth evaluation is required. Here, we conduct the more in-depth evaluation on a spinal-reflex-based control model using five representative gait disturbances, ranging from electrical stimulation to mechanical perturbation at individual leg joints and at the whole body. The immediate changes in muscle activations of the model are compared to those of humans across different gait phases and disturbance magnitudes. Remarkably similar response trends for the majority of investigated muscles and experimental conditions reinforce the plausibility of the reflex circuits of the model. However, the model's responses lack in amplitude for two experiments with whole body disturbances suggesting that in these cases the proposed reflex circuits need to be amplified by additional control structures such as location-specific cutaneous reflexes. A model that captures these selective amplifications would be able to explain both steady and reactive spinal control of human locomotion. Neuromechanical simulations that investigate hypothesized control models are complementary to gait experiments in better understanding the control of human locomotion.

A neural circuitry that emphasizes spinal feedback generates diverse behaviours of human locomotion.

KEY POINTS: It is often assumed that central pattern generators, which generate rhythmic patterns without rhythmic inputs, play a key role in the spinal control of human locomotion. We propose a neural control model in which the spinal control generates muscle stimulations mainly through integrated reflex pathways with no central pattern generator. Using a physics-based neuromuscular human model, we show that this control network is sufficient to compose steady and transitional 3-D locomotion behaviours, including walking and running, acceleration and deceleration, slope and stair negotiation, turning, and deliberate obstacle avoidance. The results suggest feedback integration to be functionally more important than central pattern generation in human locomotion across behaviours. In addition, the proposed control architecture may serve as a guide in the search for the neurophysiological origin and circuitry of spinal control in humans. ABSTRACT: Neural networks along the spinal cord contribute substantially to generating locomotion behaviours in humans and other legged animals. However, the neural circuitry involved in this spinal control remains unclear. We here propose a specific circuitry that emphasizes feedback integration over central pattern generation. The circuitry is based on neurophysiologically plausible muscle-reflex pathways that are organized in 10 spinal modules realizing limb functions essential to legged systems in stance and swing. These modules are combined with a supraspinal control layer that adjusts the desired foot placements and selects the leg that is to transition into swing control during double support. Using physics-based simulation, we test the proposed circuitry in a neuromuscular human model that includes neural transmission delays, musculotendon dynamics and compliant foot-ground contacts. We find that the control network is sufficient to compose steady and transitional 3-D locomotion behaviours including walking and running, acceleration and deceleration, slope and stair negotiation, turning, and deliberate obstacle avoidance. The results suggest feedback integration to be functionally more important than central pattern generation in human locomotion across behaviours. In addition, the proposed control architecture may serve as a guide in the search for the neurophysiological origin and circuitry of spinal control in humans.

Integration of an adaptive swing control into a neuromuscular human walking model.

Understanding the neuromuscular control underlying human locomotion has the potential to deliver practical controllers for humanoid and prosthetic robots. However, neurocontrollers developed in forward dynamic simulations are seldom applied as practical controllers due to their lack of robustness and adaptability. A key element for robust and adaptive locomotion is swing leg placement. Here we integrate a previously identified robust swing leg controller into a full neuromuscular human walking model and demonstrate that the integrated model has largely improved behaviors including walking on very rough terrain (±10 cm) and stair climbing (15 cm stairs). These initial results highlight the potential of the identified robust swing control. We plan to generalize it to a range of human locomotion behaviors critical in rehabilitation robotics.

Generalization of a muscle-reflex control model to 3D walking.

The neural controller that generates human locomotion can currently not be measured directly, and researchers often resort to forward dynamic simulations of the human neuromuscular system to propose and test different controller architectures. However, most of these models are restricted to locomotion in the sagittal plane, which limits the ability to study and compare proposed neural controls for 3D-related motions. Here we generalize a previously identified reflex control model for sagittal plane walking to 3D locomotion. The generalization includes additional degrees of freedom at the hips in the lateral plane, their actuation and control by hip abductor and adductor muscles, and 3D compliant ground contact dynamics. The resulting 3D model of human locomotion generates normal walking while producing human-like ground reaction forces and moments, indicating that the proposed neural controller based on muscle reflexes generalizes well to 3D locomotion.

The effect of foot compliance encoded in the windlass mechanism on the energetics of human walking.

The human foot, which is the part of the body that interacts with the environment during locomotion, consists of rich biomechanical design. One of the unique designs of human feet is the windlass mechanism. In a previous simulation study, we found that the windlass mechanism seems to improve the energy efficiency of walking. To better understand the origin of this efficiency, we here conduct both simulation and experimental studies exploring the influence of foot compliance, which is one of the functionalities that the windlass mechanism embeds, on the energetics of walking. The studies show that walking with compliant feet incurs more energetic costs than walking with stiff feet. The preliminary results suggest that the energy saved by introducing the windlass mechanism does not originate from the compliance it embeds. We speculate that the energy savings of the windlass mechanism are related more to its contribution to reducing the effective foot length in swing than to providing compliance in stance.