A Novel Functional Electrical Stimulation-Induced Cycling Controller Using Reinforcement Learning to Optimize Online Muscle Activation Pattern.

This study introduces a novel controller based on a Reinforcement Learning (RL) algorithm for real-time adaptation of the stimulation pattern during FES-cycling. Core to our approach is the introduction of an RL agent that interacts with the cycling environment and learns through trial and error how to modulate the electrical charge applied to the stimulated muscle groups according to a predefined policy and while tracking a reference cadence. Instead of a static stimulation pattern to be modified by a control law, we hypothesized that a non-stationary baseline set of parameters would better adjust the amount of injected electrical charge to the time-varying characteristics of the musculature. Overground FES-assisted cycling sessions were performed by a subject with spinal cord injury (SCI AIS-A, T8). For tracking a predefined pedaling cadence, two closed-loop control laws were simultaneously used to modulate the pulse intensity of the stimulation channels responsible for evoking the muscle contractions. First, a Proportional-Integral (PI) controller was used to control the current amplitude of the stimulation channels over an initial parameter setting with predefined pulse amplitude, width and fixed frequency parameters. In parallel, an RL algorithm with a decayed-epsilon-greedy strategy was implemented to randomly explore nine different variations of pulse amplitude and width parameters over the same stimulation setting, aiming to adjust the injected electrical charge according to a predefined policy. The performance of this global control strategy was evaluated in two different RL settings and explored in two different cycling scenarios. The participant was able to pedal overground for distances over 3.5 km, and the results evidenced the RL agent learned to modify the stimulation pattern according to the predefined policy and was simultaneously able to track a predefined pedaling cadence. Despite the simplicity of our approach and the existence of more sophisticated RL algorithms, our method can be used to reduce the time needed to define stimulation patterns. Our results suggest interesting research possibilities to be explored in the future to improve cycling performance since more efficient stimulation cost dynamics can be explored and implemented for the agent to learn.

Development of a High-Power Capacity Open Source Electrical Stimulation System to Enhance Research into FES-Assisted Devices: Validation of FES Cycling.

Since the first Cybathlon 2016, when twelve teams competed in the FES bike race, we have witnessed a global effort towards the development of stimulation and control strategies to improve FES-assisted devices, particularly for cycling, as a means to practice a recreational physical activity. As a result, a set of technical notes and research paved the way for many other studies and the potential behind FES-assisted cycling has been consolidated. However, engineering research needs instrumented devices to support novel developments and enable precise assessment. Therefore, some researchers struggle to develop their own FES-assisted devices or find it challenging to implement their instrumentation using commercial devices, which often limits the implementation of advanced control strategies and the possibility to connect different types of sensor. In this regard, we hypothesize that it would be advantageous for some researchers in our community to enjoy access to an entire open-source FES platform that allows different control strategies to be implemented, offers greater adaptability and power capacity than commercial devices, and can be used to assist different functional activities in addition to cycling. Hence, it appears to be of interest to make our proprietary electrical stimulation system an open-source device and to prove its capabilities by addressing all the aspects necessary to implement a FES cycling system. The high-power capacity stimulation device is based on a constant current topology that allows the creation of biphasic electrical pulses with amplitude, width, and frequency up to 150 mA, 1000 µs, and 100 Hz, respectively. A mobile application (Android) was developed to set and modify the stimulation parameters of up to eight stimulation channels. A proportional-integral controller was implemented for cadence tracking with the aim to improve the overall cycling performance. A volunteer with complete paraplegia participated in the functional testing of the system. He was able to cycle indoors for 45 min, accomplish distances of more than 5 km using a passive cycling trainer, and pedal 2400 m overground in 32 min. The results evidenced the capacity of our FES cycling system to be employed as a cycling tool for individuals with spinal cord injury. The methodological strategies used to improve FES efficiency suggest the possibility of maximizing pedaling duration through more advanced control techniques.