Performance of multiple neural networks in predicting lower limb joint moments using wearable sensors.

Joint moment measurements represent an objective biomechemical parameter in joint health assessment. Inverse dynamics based on 3D motion capture data is the current 'gold standard' to estimate joint moments. Recently, machine learning combined with data measured by wearable technologies such electromyography (EMG), inertial measurement units (IMU), and electrogoniometers (GON) has been used to enable fast, easy, and low-cost measurements of joint moments. This study investigates the ability of various deep neural networks to predict lower limb joint moments merely from IMU sensors. The performance of five different deep neural networks (InceptionTimePlus, eXplainable convolutional neural network (XCM), XCMplus, Recurrent neural network (RNNplus), and Time Series Transformer (TSTPlus)) were tested to predict hip, knee, ankle, and subtalar moments using acceleration and gyroscope measurements of four IMU sensors at the trunk, thigh, shank, and foot. Multiple locomotion modes were considered including level-ground walking, treadmill walking, stair ascent, stair descent, ramp ascent, and ramp descent. We show that XCM can accurately predict lower limb joint moments using data of only four IMUs with RMSE of 0.046 ± 0.013 Nm/kg compared to 0.064 ± 0.003 Nm/kg on average for the other architectures. We found that hip, knee, and ankle joint moments predictions had a comparable RMSE with an average of 0.069 Nm/kg, while subtalar joint moments had the lowest RMSE of 0.033 Nm/kg. The real-time feedback that can be derived from the proposed method can be highly valuable for sports scientists and physiotherapists to gain insights into biomechanics, technique, and form to develop personalized training and rehabilitation programs.

A Novel Model to Generate Heterogeneous and Realistic Time-Series Data for Post-Stroke Rehabilitation Assessment.

The application of machine learning-based tele-rehabilitation faces the challenge of limited availability of data. To overcome this challenge, data augmentation techniques are commonly employed to generate synthetic data that reflect the configurations of real data. One such promising data augmentation technique is the Generative Adversarial Network (GAN). However, GANs have been found to suffer from mode collapse, a common issue where the generated data fails to capture all the relevant information from the original dataset. In this paper, we aim to address the problem of mode collapse in GAN-based data augmentation techniques for post-stroke assessment. We applied the GAN to generate synthetic data for two post-stroke rehabilitation datasets and observed that the original GAN suffered from mode collapse, as expected. To address this issue, we propose a Time Series Siamese GAN (TS-SGAN) that incorporates a Siamese network and an additional discriminator. Our analysis, using the longest common sub-sequence (LCSS), demonstrates that TS-SGAN generates data uniformly for all elements of two testing datasets, in contrast to the original GAN. To further evaluate the effectiveness of TS-SGAN, we encode the generated dataset into images using Gramian Angular Field and classify them using ResNet-18. Our results show that TS-SGAN achieves a significant accuracy increase of classification accuracy (35.2%-42.07%) for both selected datasets. This represents a substantial improvement over the original GAN.

Monocular 3D Human Pose Markerless Systems for Gait Assessment.

Gait analysis plays an important role in the fields of healthcare and sports sciences. Conventional gait analysis relies on costly equipment such as optical motion capture cameras and wearable sensors, some of which require trained assessors for data collection and processing. With the recent developments in computer vision and deep neural networks, using monocular RGB cameras for 3D human pose estimation has shown tremendous promise as a cost-effective and efficient solution for clinical gait analysis. In this paper, a markerless human pose technique is developed using motion captured by a consumer monocular camera (800 × 600 pixels and 30 FPS) for clinical gait analysis. The experimental results have shown that the proposed post-processing algorithm significantly improved the original human pose detection model (BlazePose)'s prediction performance compared to the gold-standard gait signals by 10.7% using the MoVi dataset. In addition, the predicted T2 score has an excellent correlation with ground truth (r = 0.99 and y = 0.94x + 0.01 regression line), which supports that our approach can be a potential alternative to the conventional marker-based solution to assist the clinical gait assessment.

Predicting the Internal Knee Abduction Impulse During Walking Using Deep Learning.

Knee joint moments are commonly calculated to provide an indirect measure of knee joint loads. A shortcoming of inverse dynamics approaches is that the process of collecting and processing human motion data can be time-consuming. This study aimed to benchmark five different deep learning methods in using walking segment kinematics for predicting internal knee abduction impulse during walking. Three-dimensional kinematic and kinetic data used for the present analyses came from a publicly available dataset on walking (participants n = 33). The outcome for prediction was the internal knee abduction impulse over the stance phase. Three-dimensional (3D) angular and linear displacement, velocity, and acceleration of the seven lower body segment's center of mass (COM), relative to a fixed global coordinate system were derived and formed the predictor space (126 time-series predictors). The total number of observations in the dataset was 6,737. The datasets were split into training (75%, n = 5,052) and testing (25%, n = 1685) datasets. Five deep learning models were benchmarked against inverse dynamics in quantifying knee abduction impulse. A baseline 2D convolutional network model achieved a mean absolute percentage error (MAPE) of 10.80%. Transfer learning with InceptionTime was the best performing model, achieving the best MAPE of 8.28%. Encoding the time-series as images then using a 2D convolutional model performed worse than the baseline model with a MAPE of 16.17%. Time-series based deep learning models were superior to an image-based method when predicting knee abduction moment impulse during walking. Future studies looking to develop wearable technologies will benefit from knowing the optimal network architecture, and the benefit of transfer learning for predicting joint moments.

A comprehensive evaluation of state-of-the-art time-series deep learning models for activity-recognition in post-stroke rehabilitation assessment.

The recent COVID-19 pandemic has further high-lighted the need for improving tele-rehabilitation systems. One of the common methods is to use wearable sensors for monitoring patients and intelligent algorithms for accurate and objective assessments. An important part of this work is to develop an efficient evaluation algorithm that provides a high-precision activity recognition rate. In this paper, we have investigated sixteen state-of-the-art time-series deep learning algorithms with four different architectures: eight convolutional neural networks configurations, six recurrent neural networks, a combination of the two and finally a wavelet-based neural network. Additionally, data from different sensors' combinations and placements as well as different pre-processing algorithms were explored to determine the optimal configuration for achieving the best performance. Our results show that the XceptionTime CNN architecture is the best performing algorithm with normalised data. Moreover, we found out that sensor placement is the most important attribute to improve the accuracy of the system, applying the algorithm on data from sensors placed on the waist achieved a maximum of 42% accuracy while the sensors placed on the hand achieved 84%. Consequently, compared to current results on the same dataset for different classification categories, this approach improved the existing state of the art accuracy from 79% to 84%, and from 80% to 90% respectively.

A novel gateway-based solution for remote elderly monitoring.

Internet of Things (IoT) technologies have been applied to various fields such as manufacturing, automobile industry and healthcare. IoT-based healthcare has a significant impact on real-time remote monitoring of patients' health and consequently improving treatments and reducing healthcare costs. In fact, IoT has made healthcare more reliable, efficient, and accessible. Two major drawbacks which IoT suffers from can be expressed as: first, thelimited battery capacityof thesensorsis quickly depleted due to the continuous stream of data; second, the dependence of the system on the cloud for computations and processing causes latency in data transmission which is not accepted in real-time monitoring applications. This research is conducted to develop a real-time, secure, and energy-efficient platform which provides a solution for reducing computation load on the cloud and diminishing data transmission delay. In the proposed platform, the sensors utilize a state-of-the-art power saving technique known as Compressive Sensing (CS). CS allows sensors to retrieve the sensed data using fewer measurements by sending a compressed signal. In this framework, the signal reconstruction and processing are computed locally on a Heterogeneous Multicore Platform (HMP) device to decrease the dependency on the cloud. In addition, a framework has been implemented to control the system, set different parameters, display the data as well as send live notifications to medical experts through the cloud in order to alert them of any eventual hazardous event or abnormality and allow quick interventions. Finally, a case study of the system is presented demonstrating the acquisition and monitoring of the data for a given subject in real-time. The obtained results reveal that the proposed solution reduces 15.4% of energy consumption in sensors, that makes this prototype a good candidate for IoT employment in healthcare.