Multi Degree of Freedom Hybrid FES and Robotic Control of the Upper Limb.

Individuals who have suffered a spinal cord injury often require assistance to complete daily activities, and for individuals with tetraplegia, recovery of upper-limb function is among their top priorities. Hybrid functional electrical stimulation (FES) and exoskeleton systems have emerged as a potential solution to provide upper limb movement assistance. These systems leverage the user's own muscles via FES and provide additional movement support via an assistive exoskeleton. To date, these systems have focused on single joint movements, limiting their utility for the complex movements necessary for independence. In this paper, we extend our prior work on model predictive control (MPC) of hybrid FES-exo systems and present a multi degree of freedom (DOF) hybrid controller that uses the controller's cost function to achieve desired behavior. In studies with neurologically intact individuals, the hybrid controller is compared to an exoskeleton acting alone for movement assistance scenarios incorporating multiple degrees-of-freedom of the limb to explore the potential for exoskeleton power consumption reduction and impacts on tracking accuracy. Additionally, each scenario is explored in simulation using the models required to generate the MPC formulation. The two DOF hybrid controller implementation saw reductions in power consumption and satisfactory trajectory tracking in both the physical and simulated systems. In the four DOF implementation, the experimental results showed minor improvements for some joints of the upper limb. In simulation, we observed comparable performance as in the two DOF implementation.

Data-Driven Dynamic Motion Planning for Practical FES-Controlled Reaching Motions in Spinal Cord Injury.

Functional electrical stimulation (FES) is a promising technology for restoring reaching motions to individuals with upper-limb paralysis caused by a spinal cord injury (SCI). However, the limited muscle capabilities of an individual with SCI have made achieving FES-driven reaching difficult. We developed a novel trajectory optimization method that used experimentally measured muscle capability data to find feasible reaching trajectories. In a simulation based on a real-life individual with SCI, we compared our method to attempting to follow naive direct-to-target paths. We tested our trajectory planner with three control structures that are commonly used in applied FES: feedback, feedforward-feedback, and model predictive control. Overall, trajectory optimization improved the ability to reach targets and improved the accuracy for the feedforward-feedback and model predictive controllers ( ). The trajectory optimization method should be practically implemented to improve the FES-driven reaching performance.

Hybrid FES-exoskeleton control: Using MPC to distribute actuation for elbow and wrist movements.

Introduction: Individuals who have suffered a cervical spinal cord injury prioritize the recovery of upper limb function for completing activities of daily living. Hybrid FES-exoskeleton systems have the potential to assist this population by providing a portable, powered, and wearable device; however, realization of this combination of technologies has been challenging. In particular, it has been difficult to show generalizability across motions, and to define optimal distribution of actuation, given the complex nature of the combined dynamic system. Methods: In this paper, we present a hybrid controller using a model predictive control (MPC) formulation that combines the actuation of both an exoskeleton and an FES system. The MPC cost function is designed to distribute actuation on a single degree of freedom to favor FES control effort, reducing exoskeleton power consumption, while ensuring smooth movements along different trajectories. Our controller was tested with nine able-bodied participants using FES surface stimulation paired with an upper limb powered exoskeleton. The hybrid controller was compared to an exoskeleton alone controller, and we measured trajectory error and torque while moving the participant through two elbow flexion/extension trajectories, and separately through two wrist flexion/extension trajectories. Results: The MPC-based hybrid controller showed a reduction in sum of squared torques by an average of 48.7 and 57.9% on the elbow flexion/extension and wrist flexion/extension joints respectively, with only small differences in tracking accuracy compared to the exoskeleton alone. Discussion: To realize practical implementation of hybrid FES-exoskeleton systems, the control strategy requires translation to multi-DOF movements, achieving more consistent improvement across participants, and balancing control to more fully leverage the muscles' capabilities.

Proof-of-Concept: A Hands-Free Interface for Robot-Assisted Self-Feeding.

Eating and drinking is an essential part of every-day life. And yet, there are many people in the world today who rely on others to feed them. In this work, we present a prototype robot-assisted self-feeding system for individuals with movement disorders. The system is capable of perceiving, localizing, grasping, and delivering non-compliant food items to an individual. We trained an object recognition network to detect specific food items, and we compute the grasp pose for each item. Human input is obtained through an interface consisting of an eye-tracker and a display screen. The human selects options on the monitor with their eye and head movements and triggers responses with mouth movements. We performed a pilot study with four able-bodied participants and one participant with a spinal cord injury (SCI) to evaluate the performance of our prototype system. Participants selected food items with their eye movements, which were then delivered by the robot. We observed an average overall feeding success rate of 89.1% and an average overall task time of $31.4 \pm 2.4$ seconds per food item. The SCI participant gave scores of 90.0 and 8.3 on the System Usability Scale and NASA Task Load Index, respectively. We also conducted a custom, post-study interview to gather participant feedback to drive future design decisions. The quantitative results and qualitative user feedback demonstrate the feasibility of robot-assisted self-feeding and justify continued research into mealtime-related assistive devices.

Shared Control of Elbow Movements with Functional Electrical Stimulation and Exoskeleton Assistance.

Individuals who suffer from paralysis as a result of a spinal cord injury list restoration of arm and hand function as a top priority. FES helps restore movement using the user's own muscles, but does not produce accurate and repeatable movements necessary for many functional tasks. Robots can assist users in achieving accurate and repeatable movements, but often require bulky hardware to generate the necessary torques. We propose sharing torque requirements between a robot and FES to reduce robot torque output compared to a robot acting alone, yet maintain high accuracy. Cooperative PD and model predictive control algorithms were designed to share the control between these two torque sources. Corresponding PD and MPC algorithms that do not use FES were also designed. The control algorithms were tested with 10 able-bodied subjects. Torque and position tracking accuracy were compared when the system was commanded to follow a functional elbow flexion/extension trajectory. The robot torque required to achieve these movements was reduced for the shared control cases compared to the algorithms acting without FES. We observed a reduction in position accuracy with the MPC shared controller compared to the PD shared controller, while the MPC shared controller resulted in greater reductions in torque requirements. Both of these shared algorithms showed improvements over existing options, and can be used on any given trajectory, allowing for better transferability to functional tasks.

Controlling an effector with eye movements: The effect of entangled sensory and motor responsibilities.

Restoring arm and hand function has been indicated by individuals with tetraplegia as one of the most important factors for regaining independence. The overall goal of our research is to develop assistive technologies that allow individuals with tetraplegia to control functional reaching movements. This study served as an initial step toward our overall goal by assessing the feasibility of using eye movements to control the motion of an effector in an experimental environment. We aimed to understand how additional motor requirements placed on the eyes affected eye-hand coordination during functional reaching. We were particularly interested in how eye fixation error was affected when the sensory and motor functions of the eyes were entangled due to the additional motor responsibility. We recorded participants' eye and hand movements while they reached for targets on a monitor. We presented a cursor at the participant's point of gaze position which can be thought of as being similar to the control of an assistive robot arm. To measure eye fixation error, we used an offline filter to extract eye fixations from the raw eye movement data. We compared the fixations to the locations of the targets presented on the monitor. The results show that not only are humans able to use eye movements to direct the cursor to a desired location (1.04 ± 0.15 cm), but they can do so with error similar to that of the hand (0.84 ± 0.05 cm). In other words, despite the additional motor responsibility placed on the eyes during direct eye-movement control of an effector, the ability to coordinate functional reaching movements was unaffected. The outcomes of this study support the efficacy of using the eyes as a direct command input for controlling movement.

A review of methods for achieving upper limb movement following spinal cord injury through hybrid muscle stimulation and robotic assistance.

Individuals with tetraplegia, typically attributed to spinal cord injuries (SCI) at the cervical level, experience significant health care costs and loss of independence due to their limited reaching and grasping capabilities. Neuromuscular electrical stimulation (NMES) is a promising intervention to restore arm and hand function because it activates a person's own paralyzed muscles; however, NMES sometimes lacks the accuracy and repeatability necessary to position the limb for functional tasks, and repeated muscle stimulation can lead to fatigue. Robotic devices have the potential to restore function when used as assistive devices to supplement or replace limited or lost function of the upper limb following SCI. Unfortunately, most robotic solutions are bulky or require significant power to operate, limiting their applicability to restore functional independence in a home environment. Combining NMES and robotic support systems into a single hybrid neuroprosthesis is compelling, since the robotic device can supplement the action of the muscles and improve repeatability and accuracy. Research groups have begun to explore applications of movement assistance for individuals with spinal cord injury using these technologies in concert. In this review, we present the state of the art in hybrid NMES-orthotic systems for upper limb movement restoration following spinal cord injury, and suggest areas for emphasis necessary to move the field forward. Currently, NMES-robotic systems use either surface or implanted electrodes to stimulate muscles, with rigid robotic supports holding the limb against gravity, or providing assistance in reaching movements. Usability of such systems outside of the lab or clinic is limited due to the complexity of both the mechanical components, stimulation systems, and human-machine interfaces. Assessment of system and participant performance is not reported in a standardized way. Future directions should address wearability through improvements in component technologies and user interfaces. Further, increased integration of the control action between NMES and robotic subsystems to reanimate the limb should be pursued. Standardized reporting of system performance and expanded clinical assessments of these systems are also needed. All of these advancements are critical to facilitate translation from lab to home.

Predicting functional force production capabilities of upper extremity functional electrical stimulation neuroprostheses: a proof of concept study.

OBJECTIVE: This study's goal was to demonstrate person-specific predictions of the force production capabilities of a paralyzed arm when actuated with a functional electrical stimulation (FES) neuroprosthesis. These predictions allow us to determine, for each hand position in a person's workspace, if FES activated muscles can produce enough force to hold the arm against gravity and other passive forces, the amount of force the arm can potentially exert on external objects, and in which directions FES can move the arm. APPROACH: We computed force production predictions for a person with high tetraplegia and an FES neuroprosthesis used to activate muscles in her shoulder and arm. We developed Gaussian process regression models of the force produced at the end of the forearm when stimulating individual muscles at different wrist positions in the person's workspace. For any given wrist position, we predicted all possible forces a person can produce by any combination of individual muscles. Based on the force predictions, we determined if FES could produce force sufficient to overcome passive forces to hold a wrist position, the maximum force FES could produce in all directions, and the set of directions in which FES could move the arm. To estimate the error in our predictions, we then compared our force predictions based on single-muscle models to the actual forces produced when stimulating combinations of the person's muscles. MAIN RESULTS: Our models classified the person's ability to hold static arm positions correctly for 83% (Session #1) and 69% (Session #2) for 39 wrist positions over two sessions. We predicted this person's ability to produce force at the end of her arm with an RMS error of 5.5 N and the percent of directions for which FES could achieve motion with RMS error of 10%. The accuracy of these predictions is similar to that found in the literature for FES systems with fewer degrees of freedom and fewer muscles. SIGNIFICANCE: These person and device-specific predictions of functional capabilities of the arm allow neuroprosthesis developers to set achievable functional objectives for the systems they develop. These predictions can potentially serve as a screening tool for clinicians to use in planning neuroprosthetic interventions, greatly reducing the risk and uncertainty in such interventions.

Developing a Quasi-Static Controller for a Paralyzed Human Arm: A Simulation Study.

Individuals with paralyzed limbs due to spinal cord injuries lack the ability to perform the reaching motions necessary to every day life. Functional electrical stimulation (FES) is a promising technology for restoring reaching movements to these individuals by reanimating their paralyzed muscles. We have proposed using a quasi-static model-based control strategy to achieve reaching controlled by FES. This method uses a series of static positions to connect the starting wrist position to the goal. As a first step to implementing this controller, we have completed a simulated study using a MATLAB based dynamic model of the arm in order to determine the suitable parameters for the quasi-static controller. The selected distance between static positions in the path was 6 cm, and the amount of time between switching target positions was 1.3 s. The final controller can complete reaches of over 30 cm with a median accuracy of 6.8 cm.

Holding Static Arm Configurations With Functional Electrical Stimulation: A Case Study.

Functional electrical stimulation (FES) is a promising solution for restoring functional motion to individuals with paralysis, but the potential for achieving any desired full-arm reaching motion has not been realized. We present a combined feedforward-feedback controller capable of automatically calculating and applying the necessary muscle stimulations to hold the wrist of an individual with high tetraplegia in a desired static position. We used the controller to hold a complete arm configuration to maintain a series of static wrist positions. The average distance to the target wrist position, or accuracy, was 2.9 cm. The precision is defined as the radius of the 95% confidence ellipsoid for the final positions of a set of trials with the same muscle stimulations and starting position. The average precision was 3.7 cm. The control architecture used in this study to hold static positions has the potential to control arbitrary reaching motions.

Evaluating an open-loop functional electrical stimulation controller for holding the shoulder and elbow configuration of a paralyzed arm.

Functional electrical stimulation (FES) is a promising solution for restoring functional motion to individuals with paralysis, but the potential for achieving full-arm reaching motions with FES for various desired tasks has not been realized. We present an open-loop controller capable of calculating and applying the necessary muscle stimulations to hold the wrist of an individual with high tetraplegia at any desired position. We used the controller to hold the wrist at a series of static positions. The controller was capable of discriminating between different wrist positions. The average distance to the target wrist position, or accuracy, was 7.7 cm. The average radius of the 95% confidence ellipsoid for a set of trials with the same muscle stimulations, or precision, was 6.7 cm. Adding feedback or online model updates will likely improve the accuracy for tasks requiring finer control. The controller is a good first step to controlling full-arm motions with FES.

Semiparametric Identification of Human Arm Dynamics for Flexible Control of a Functional Electrical Stimulation Neuroprosthesis.

We present a method to identify the dynamics of a human arm controlled by an implanted functional electrical stimulation neuroprosthesis. The method uses Gaussian process regression to predict shoulder and elbow torques given the shoulder and elbow joint positions and velocities and the electrical stimulation inputs to muscles. We compare the accuracy of torque predictions of nonparametric, semiparametric, and parametric model types. The most accurate of the three model types is a semiparametric Gaussian process model that combines the flexibility of a black box function approximator with the generalization power of a parameterized model. The semiparametric model predicted torques during stimulation of multiple muscles with errors less than 20% of the total muscle torque and passive torque needed to drive the arm. The identified model allows us to define an arbitrary reaching trajectory and approximately determine the muscle stimulations required to drive the arm along that trajectory.

Multi-muscle FES force control of the human arm for arbitrary goals.

We present a method for controlling a neuroprosthesis for a paralyzed human arm using functional electrical stimulation (FES) and characterize the errors of the controller. The subject has surgically implanted electrodes for stimulating muscles in her shoulder and arm. Using input/output data, a model mapping muscle stimulations to isometric endpoint forces measured at the subject's hand was identified. We inverted the model of this redundant and coupled multiple-input multiple-output system by minimizing muscle activations and used this inverse for feedforward control. The magnitude of the total root mean square error over a grid in the volume of achievable isometric endpoint force targets was 11% of the total range of achievable forces. Major sources of error were random error due to trial-to-trial variability and model bias due to nonstationary system properties. Because the muscles working collectively are the actuators of the skeletal system, the quantification of errors in force control guides designs of motion controllers for multi-joint, multi-muscle FES systems that can achieve arbitrary goals.

Modeling open-loop stability of a human arm driven by a functional electrical stimulation neuroprosthesis.

Functional electrical stimulation (FES) can be used to restore movement control following paralysis. For complex multijoint systems, it is becoming increasingly apparent that closed-loop controllers are needed. Designing a closed-loop control system is easiest when the open-loop system is stable. In this study we developed a computational model to assess the open-loop stability of FES-control systems. We used the model to examine the open-loop stability of the human arm throughout its reachable workspace. For each simulated position of the hand we examined the stability of the arm, assuming that a minimal pattern of muscle activation was used to support the arm against gravity. Only muscles available to an existing FES user were considered. We found that with this reduced muscle set, the stability of the arm was severely compromised. We also demonstrated that muscle co-contraction can be an effective method to improve the stability for many postures.

Optimal sampling of recruitment curves for functional electrical stimulation control.

A major challenge in controlling multiple-input multiple output functional electrical stimulation systems is the large amount of time required to identify a workable system model due to the high dimensionality of the space of inputs. To address this challenge we are exploring optimal methods to sample the input space. In this paper we present two methods for optimally sampling isometric muscle force recruitment curves. One method maximizes the information about the recruitment curve parameters, and the second method minimizes the average variance of the predicted output force. We compared these methods to two previously-used methods in simulation. The simulation model was identified from recruitment data collected during experiments with a human subject with a high spinal cord injury. The optimal sampling methods on average produced estimates of the output force with less error than the two previously-used methods. The optimal sampling methods require fewer system identification experiments to identify models with similar output prediction accuracy.