Real-Time Wrist Motion Decoding with High Framerate Electrical Impedance Tomography (EIT).

Human wrist motion decoding with a biological-signal-based interface is a key technique in the upper-limb exoskeleton and prosthesis control. One critical issue in this field is achieving high recognition precision and fast time response while against external disturbances of sensor re-wearing. In this study, we proposed a high-framerate Electrical Impedance Tomography (EIT) system combined with an adaptive recognition algorithm for real-time wrist kinematics decoding. The high-framerate EIT system was developed by a parallel stimulation-measurement sequence, and the sampling rate was as high as 104 Hz. Compared to the most widely used myoelectric techniques, the EIT-based interface can provide extra deep muscular spatial information with similar surface electrodes. It greatly benefited the subsequent recognition algorithms, in which the key EIT regions indicating muscle morphology kept consistent after an arbitrary sensor re-donning. The designed adaptive algorithm achieved equally high performance with an automatic update of the classifier mean values with a fast self-operated calibration process. We validated the approach on 12 subjects with a 2-dimensional Fitts' law test. The wrist gestures and joint angles were mapped to the direction and speed of the cursor movement, respectively. The average throughputs (TPs) of Fitts' law tests were 1.0269 ± 0.0971 bits/s and 1.0095 ± 0.0931 bits/s without and with sensor re-donning, respectively, which were comparable to the TPs of sEMG-based studies. The results showed the promise of the EIT-based interface on real-time human motion intent recognition. Future endeavors are worth being paid in this direction for more complicated robotic tasks.

Stress prediction using micro-EMA and machine learning during COVID-19 social isolation.

Accurately predicting users' perceived stress is beneficial to aid early intervention and prevent both mental illness and physical disease during the COVID-19 pandemic. However, the existing perceived stress predicting system needs to collect a large amount of previous data for training but has a limited prediction range (i.e., next 1-2 days). Therefore, we propose a perceived stress prediction system based on the history data of micro-EMA for identifying risks 7 days earlier. Specifically, we first select and deliver an optimal set of micro-EMA questions to users every Monday, Wednesday, and Friday for reducing the burden. Then, we extract time-series features from the past micro-EMA responses and apply an Elastic net regularization model to discard redundant features. After that, selected features are fed to an ensemble prediction model for forecasting fine-grained perceived stress in the next 7 days. Experiment results show that our proposed prediction system can achieve around 4.26 (10.65% of the scale) mean absolute error for predicting the next 7 day's PSS scores, and higher than 81% accuracy for predicting the next 7 day's stress labels.

Continuous Multi-DoF Wrist Kinematics Estimation Based on a Human-Machine Interface With Electrical-Impedance-Tomography.

This study proposed a multiple degree-of-freedom (DoF) continuous wrist angle estimation approach based on an electrical impedance tomography (EIT) interface. The interface can inspect the spatial information of deep muscles with a soft elastic fabric sensing band, extending the measurement scope of the existing muscle-signal-based sensors. The designed estimation algorithm first extracted the mutual correlation of the EIT regions with a kernel function, and second used a regularization procedure to select the optimal coefficients. We evaluated the method with different features and regression models on 12 healthy subjects when they performed six basic wrist joint motions. The average root-mean-square error of the 3-DoF estimation task was 7.62°, and the average R 2 was 0.92. The results are comparable to state-of-the-art with sEMG signals in multi-DoF tasks. Future endeavors will be paid in this new direction to get more promising results.

Locomotion Mode Recognition with Inertial Signals for Hip Joint Exoskeleton.

Recognizing locomotion modes is a crucial step in controlling lower-limb exoskeletons/orthoses. Our study proposed a fuzzy-logic-based locomotion mode/transition recognition approach that uses the onrobot inertial sensors for a hip joint exoskeleton (active pelvic orthosis). The method outputs the recognition decisions at each extreme point of the hip joint angles purely relying on the integrated inertial sensors. Compared with the related studies, our approach enables calibrations and recognition without additional sensors on the feet. We validated the method by measuring four locomotion modes and eight locomotion transitions on three able-bodied subjects wearing an active pelvic orthosis (APO). The average recognition accuracy was 92.46% for intrasubject crossvalidation and 93.16% for intersubject crossvalidation. The average time delay during the transitions was 1897.9 ms (28.95% one gait cycle). The results were at the same level as the related studies. On the other side, the study is limited in the small sample size of the subjects, and the results are preliminary. Future efforts will be paid on more extensive evaluations in practical applications.

Identification of muscle morphology with noncontact capacitive sensing: Preliminary study.

Human-machine interface with muscle signals serves as an important role in the field of wearable robotics. To compensate for the limitations of the existing surface Electromyography (sEMG) based technologies, we previously proposed a noncontact capacitive sensing approach that could record the limb shape changes. The sensing approach frees the human skin from contacting to the metal electrodes, thus enabling the measurement of muscle signals by dressing the sensing front-ends outside of the clothes. We validated the capacitive sensing in human motion intent recognition tasks with the wearable robots and produced comparable results to existing studies. However, the biological significance of the capacitance signals is still unrevealed, which is an indispensable issue for robot intuitive control. In this study, we address the problems of identifying the relationships between the muscle morphological parameters and the capacitance signals. We constructed a measurement system that recorded the noncon-tact capacitive sensing signals and the muscle ultrasound (US) images simultaneously. With the designed device, five subjects were employed and the US images from the gastrocnemius muscle (GM) and the tibialis anterior (TA) muscle during level walking were sampled. We fitted the calculated muscle morphological parameters (the pinnation angles and the muscle fascicle length) and the capacitance signals of the same gait phases. The results demonstrated that at least one-channel capacitance signal strongly correlated to the muscle morphological parameters (R2 > 0.5, quadratic regression). The average R2s of the most correlated channels were up to 0.86 for pinnation angles and 0.83 for the muscle fascicle length changes. The interesting findings in this preliminary study suggest the biological physical significance of the capacitance signals during human locomotion. Future efforts are worth being paid in this new research direction for more promising results.

Locomotion Mode Recognition With Robotic Transtibial Prosthesis in Inter-Session and Inter-Day Applications.

Locomotion mode recognition across multiple sessions and days is an indispensable step towards the practical use of the robotic transtibial prosthesis. In this study, we proposed an adaptive recognition strategy to against the time-varying features of on-prosthesis mechanical signals in inter-session and inter-day recognition tasks. The strategy was designed with an automatic training algorithm which could update the classifiers with the data of the most recent completed gait cycles to seize the changes of the features brought by external disturbances. After implementation, we measured multiple experimental sessions on six transtibial amputees with intervals from a few hours (inter-session experiment) to several months (108 days at most in inter-day experiment). In each session, they performed five locomotion modes and eight locomotion transitions using the robotic prosthesis. Between each two experimental sessions, the subjects were required to doff the robotic prosthesis (including the socket). The proposed adaptive recognition algorithm significantly improved recognition accuracies in both experiments. In inter-session experiment, the proposed method increased the recognition accuracy from 89.3% to 92.8% than previous non-adaptive recognition method. In inter-day experiments, it increased the recognition accuracy from 60% to 88.8%. If taking three modes (level walking, stair ascending/descending) and four locomotion transitions into calculation, the recognizer produced an accuracy up to 96.6% (swing phase) for static mode and an accuracy of 96.9% for locomotion transitions on inter-day tasks without manual intervenes. Compared with state-of-the-art, our study extends the ability of the robotic transtibial prosthesis in locomotion mode recognition across multiple sessions and days. Future efforts are worth being paid in this direction to get more promising results.

A Pilot Study on Continuous Breaststroke Phase Recognition with Fast Training Based on Lower-Limb Inertial Signals.

In this study, we proposed a continuous stroke phase recognition method with lower-limb inertial signals. The aim of the method was to decrease the time needed and to relieve the burdensome manual configurations in the tasks of human underwater motion recognition. The method automatically segmented the data of a period of time into stroke cycles and three sub-phases (propulsion, glide and recovery). K-nearest neighbor algorithm (k-NN) was used as the classifier to train the segmented data and classify the new data on each sample interval. To validate the proposed recognition method, three elite swimmers were recruited. We also designed an wearable sensing system for human underwater motion sensing with inertial measurement units (IMUs). With only data of 5 stroke cycles for training, the recognizer produced accurate recognition results. The average precision across the phases and the subjects was 93.7% and the average recall was 92.6%. We also investigated the time difference of the key stroke events (stroke phase transitions) between the recognized decisions and the reference ones. The average time difference was 66.2 ms, which accounted for the 4.2% of a single stroke phase. The results of the pilot study proved the feasibility of the new method for human aquatic locomotion assistance tasks. Future efforts will be paid in this new direction for more promising results.

Identification of the relationships between noncontact capacitive sensing signals and continuous grasp forces: Preliminary study.

This study explores the relationships between noncontact capacitive sensing signals and continuous grasp forces. It is a crucial step towards the volitional control of robotic systems based on the noncontact sensing approach. We firstly designed a measurement system including the capacitive sensing front-ends, the grasp force sensor, the signal sampling circuits and the graphic user interface. The capacitive sensing front-end was specifically designed for human forearm signal sampling, which was worn outside of the clothes. After implementation of the system, we carried out experiments on five healthy subjects, and the sensing bands were customized with their arm shapes. The grasp force and the capacitance signals were record simultaneously when the subjects gradually increased the force according to instruction. Linear regression and quadratic regression were used to evaluate the regulated signals. For each subject, at least one channel of capacitance signals were linear correlated to the normalized grasp force with ${{R}^{2}}\ge 0.85$. We found there was inter-subject similarity on the capacitance-force relationships. Cross validation on grasp force estimation with capacitance signals were also carried out, and the average relative estimation error was about 18%. The results proved the feasibility of the noncontact capacitive sensing method for human joint force estimation.

Forearm Motion Recognition With Noncontact Capacitive Sensing.

This study presents a noncontact capacitive sensing method for forearm motion recognition. A method is proposed to record upper limb motion information from muscle contractions without contact with human skin, compensating for the limitations of existing sEMG-based methods. The sensing front-ends are designed based on human forearm shapes, and the forearm limb shape changes caused by muscle contractions will be represented by capacitance signals. After implementation of the capacitive sensing system, experiments on healthy subjects are conducted to evaluate the effectiveness. Nine motion patterns combined with 16 motion transitions are investigated on seven participants. We also designed an automatic data labeling method based on inertial signals from the measured hand, which greatly accelerated the training procedure. With the capacitive sensing system and the designed recognition algorithm, the method produced an average recognition of over 92%. Correct decisions could be made with approximately a 347-ms delay from the relaxed state to the time point of motion initiation. The confounding factors that affect the performances are also analyzed, including the sliding window length, the motion types and the external disturbances. We found the average accuracy increased to 98.7% when five motion patterns were recognized. The results of the study proved the feasibility and revealed the problems of the noncontact capacitive sensing approach on upper-limb motion sensing and recognition. Future efforts in this direction could be worthwhile for achieving more promising outcomes.

Gait Phase Estimation Based on Noncontact Capacitive Sensing and Adaptive Oscillators.

This paper presents a novel strategy aiming to acquire an accurate and walking-speed-adaptive estimation of the gait phase through noncontact capacitive sensing and adaptive oscillators (AOs). The capacitive sensing system is designed with two sensing cuffs that can measure the leg muscle shape changes during walking. The system can be dressed above the clothes and free human skin from contacting to electrodes. In order to track the capacitance signals, the gait phase estimator is designed based on the AO dynamic system due to its ability of synchronizing with quasi-periodic signals. After the implementation of the whole system, we first evaluated the offline estimation performance by experiments with 12 healthy subjects walking on a treadmill with changing speeds. The strategy achieved an accurate and consistent gait phase estimation with only one channel of capacitance signal. The average root-mean-square errors in one stride were 0.19 rad (3.0% of one gait cycle) for constant walking speeds and 0.31 rad (4.9% of one gait cycle) for speed transitions even after the subjects rewore the sensing cuffs. We then validated our strategy in a real-time gait phase estimation task with three subjects walking with changing speeds. Our study indicates that the strategy based on capacitive sensing and AOs is a promising alternative for the control of exoskeleton/orthosis.

Noncontact Capacitive Sensing-Based Locomotion Transition Recognition for Amputees With Robotic Transtibial Prostheses.

Recent advancement of robotic transtibial prostheses can restore human ankle dynamics in different terrains. Automatic locomotion transitions of the prosthesis guarantee the amputee's safety and smooth motion. In this paper, we present a noncontact capacitive sensing-based approach for recognizing locomotion transitions of amputees with robotic transtibial prostheses. The proposed sensing system is designed with flexible printed circuit boards which solves the walking instability brought by our previous system when using robotic prosthesis and improves the recognition performance. Six transtibial amputees were recruited and performed tasks of ten locomotion transitions with the robotic prosthesis that we recently constructed. The capacitive sensing system was integrated on the prosthesis and worked in combination with on-prosthesis mechanical sensors. With the cascaded classification method, the proposed system achieved 95.8% average recognition accuracy by support vector machine (SVM) classifier and 94.9% accuracy by quadratic discriminant analysis (QDA) classifier. It could accurately recognize the upcoming locomotion modes from the stance phase of the transition steps. In addition, we proved that adding capacitance signals could significantly reduce recognition errors of the robotic prosthesis in locomotion transition tasks. Our study suggests that the fusion of capacitive sensing system and mechanical sensors is a promising alternative for controlling the robotic transtibial prosthesis.

A locomotion intent prediction system based on multi-sensor fusion.

Locomotion intent prediction is essential for the control of powered lower-limb prostheses to realize smooth locomotion transitions. In this research, we develop a multi-sensor fusion based locomotion intent prediction system, which can recognize current locomotion mode and detect locomotion transitions in advance. Seven able-bodied subjects were recruited for this research. Signals from two foot pressure insoles and three inertial measurement units (one on the thigh, one on the shank and the other on the foot) are measured. A two-level recognition strategy is used for the recognition with linear discriminate classifier. Six kinds of locomotion modes and ten kinds of locomotion transitions are tested in this study. Recognition accuracy during steady locomotion periods (i.e., no locomotion transitions) is 99.71% ± 0.05% for seven able-bodied subjects. During locomotion transition periods, all the transitions are correctly detected and most of them can be detected before transiting to new locomotion modes. No significant deterioration in recognition performance is observed in the following five hours after the system is trained, and small number of experiment trials are required to train reliable classifiers.

A noncontact capacitive sensing system for recognizing locomotion modes of transtibial amputees.

This paper presents a noncontact capacitive sensing system (C-Sens) for locomotion mode recognition of transtibial amputees. C-Sens detects changes in physical distance between the residual limb and the prosthesis. The sensing front ends are built into the prosthetic socket without contacting the skin. This novel signal source improves the usability of locomotion mode recognition systems based on electromyography (EMG) signals and systems based on capacitance signals obtained from skin contact. To evaluate the performance of C-Sens, we carried out experiments among six transtibial amputees with varying levels of amputation when they engaged in six common locomotive activities. The capacitance signals were consistent and stereotypical for different locomotion modes. Importantly, we were able to obtain sufficiently informative signals even for amputees with severe muscle atrophy (i.e., amputees lacking of quality EMG from shank muscles for mode classification). With phase-dependent quadratic classifier and selected feature set, the proposed system was capable of making continuous judgments about locomotion modes with an average accuracy of 96.3% and 94.8% for swing phase and stance phase, respectively (Experiment 1). Furthermore, the system was able to achieve satisfactory recognition performance after the subjects redonned the socket (Experiment 2). We also validated that C-Sens was robust to load bearing changes when amputees carried 5-kg weights during activities (Experiment 3). These results suggest that noncontact capacitive sensing is capable of circumventing practical problems of EMG systems without sacrificing performance and it is, thus, promising for automatic recognition of human motion intent for controlling powered prostheses.

Non-contact capacitance sensing for continuous locomotion mode recognition: design specifications and experiments with an amputee.

Locomotion mode recognition plays an important role in the control of powered lower-limb prostheses. In this paper, we present a non-contact capacitance sensing system (C-Sens) to measure the interfacial signals between the residual limb and the prosthetic socket. The system includes sensing front-ends, a sensing circuit, a control circuit and foot pressure insoles. In the proposed system, the electrodes are fixed on the inner surface of the socket, which couple with the human body forming capacitors. The foot pressure insoles are built for detecting gait phases. The data sequence is controlled by the control circuit. To evaluate the capacitance sensing system, experiments with a transtibial amputee are carried out and seven kinds of locomotion modes are recorded. With the continuous phase dependent classification method and the quadratic discriminant analysis (QDA) classifier, the average recognition accuracies are 93.8% and 95.0% for the stance phase and the swing phase respectively. The results show the potential of the proposed system for the control of powered lower-limb prostheses.

Lower limb wearable capacitive sensing and its applications to recognizing human gaits.

In this paper, we present an approach to sense human body capacitance and apply it to recognize lower limb locomotion modes. The proposed wearable sensing system includes sensing bands, a signal processing circuit and a gait event detection module. Experiments on long-term working stability, adaptability to disturbance and locomotion mode recognition are carried out to validate the effectiveness of the proposed approach. Twelve able-bodied subjects are recruited, and eleven normal gait modes are investigated. With an event-dependent linear discriminant analysis classifier and feature selection procedure, four time-domain features are used for pattern recognition and satisfactory recognition accuracies (97:3% ± 0:5%, 97:0% ± 0:4%, 95:6% ± 0:9% and 97:0% ± 0:4% for four phases of one gait cycle respectively) are obtained. The accuracies are comparable with that from electromyography-based systems and inertial-based systems. The results validate the effectiveness of the proposed lower limb capacitive sensing approach in recognizing human normal gaits.

Locomotion mode classification using a wearable capacitive sensing system.

Locomotion mode classification is one of the most important aspects for the control of powered lower-limb prostheses. We propose a wearable capacitive sensing system for recognizing locomotion modes as an alternative solution to popular electromyography (EMG)-based systems, aiming to overcome drawbacks of the latter. Eight able-bodied subjects and five transtibial amputees were recruited for automatic classification of six common locomotion modes. The system measured ten channels of capacitance signals from the shank, the thigh, or both. With a phase-dependent linear discriminant analysis classifier and selected time-domain features, the system can achieve a satisfactory classification accuracy of 93.6% ±0.9% and 93.4% ±0.8% for able-bodied subjects and amputee subjects, respectively. The classification accuracy is comparable with that of EMG-based systems. More importantly, we verify that neuro-mechanical delay inherent in capacitive sensing does not affect the timeliness of classification decisions as the system, similar to EMG-based systems, can make multiple judgments during a gait cycle. Experimental results also indicate that capacitance signals from the thigh alone are sufficient for mode classification for both able-bodied and transtibial subjects. Our investigations demonstrate that capacitive sensing is a promising alternative to myoelectric sensing for real-time control of powered lower-limb prostheses.