JointTracker: Real-time inertial kinematic chain tracking with joint position estimation.

In-field human motion capture (HMC) is drawing increasing attention due to the multitude of application areas. Plenty of research is currently invested in camera-based (markerless) HMC, with the advantage of no infrastructure being required on the body, and additional context information being available from the surroundings. However, the inherent drawbacks of camera-based approaches are the limited field of view and occlusions. In contrast, inertial HMC (IHMC) does not suffer from occlusions, thus being a promising approach for capturing human motion outside the laboratory. However, one major challenge of such methods is the necessity of spatial registration. Typically, during a predefined calibration sequence, the orientation and location of each inertial sensor are registered with respect to the underlying skeleton model. This work contributes to calibration-free IHMC, as it proposes a recursive estimator for the simultaneous online estimation of all sensor poses and joint positions of a kinematic chain model like the human skeleton. The full derivation from an optimization objective is provided. The approach can directly be applied to a synchronized data stream from a body-mounted inertial sensor network. Successful evaluations are demonstrated on noisy simulated data from a three-link chain, real lower-body walking data from 25 young, healthy persons, and walking data captured from a humanoid robot. The estimated and derived quantities, global and relative sensor orientations, joint positions, and segment lengths can be exploited for human motion analysis and anthropometric measurements, as well as in the context of hybrid markerless visual-inertial HMC.

Automated detection and explainability of pathological gait patterns using a one-class support vector machine trained on inertial measurement unit based gait data.

BACKGROUND: Machine learning approaches for the classification of pathological gait based on kinematic data, e.g. derived from inertial sensors, are commonly used in terms of a multi-class classification problem. However, there is a lack of research regarding one-class classifiers that are independent of certain pathologies. Therefore, it was the aim of this work to design a one-class classifier based on healthy norm-data that provides not only a prediction probability but rather an explanation of the classification decision, increasing the acceptance of this machine learning approach. METHODS: The inertial sensor based gait kinematics of 25 healthy subjects was employed to train a one-class support vector machine. 25 healthy subjects, 20 patients after total hip arthroplasty and one transfemoral amputee served to validate the classifier. Prediction probabilities and feature importance scores were estimated for each subject. FINDINGS: The support vector machine predicted 100% of the patients as outliers from the healthy group. Three healthy subjects were predicted as outliers. The feature importance calculation revealed the hip in the sagittal plane as most relevant feature concerning the group after total hip arthroplasty. For the misclassified healthy subject with the lowest probability score the knee flexion and the pelvis obliquity were identified. INTERPRETATION: The support vector machine seems a useful tool to identify outliers from a healthy norm-group. The feature importance examination proved to provide valuable information on the musculoskeletal status of the subjects. In this combination, the present approach could be employed in various disciplines to identify abnormal gait and suggest subsequent training.

Towards an Inertial Sensor-Based Wearable Feedback System for Patients after Total Hip Arthroplasty: Validity and Applicability for Gait Classification with Gait Kinematics-Based Features.

Patients after total hip arthroplasty (THA) suffer from lingering musculoskeletal restrictions. Three-dimensional (3D) gait analysis in combination with machine-learning approaches is used to detect these impairments. In this work, features from the 3D gait kinematics, spatio temporal parameters (Set 1) and joint angles (Set 2), of an inertial sensor (IMU) system are proposed as an input for a support vector machine (SVM) model, to differentiate impaired and non-impaired gait. The features were divided into two subsets. The IMU-based features were validated against an optical motion capture (OMC) system by means of 20 patients after THA and a healthy control group of 24 subjects. Then the SVM model was trained on both subsets. The validation of the IMU system-based kinematic features revealed root mean squared errors in the joint kinematics from 0.24° to 1.25°. The validity of the spatio-temporal gait parameters (STP) revealed a similarly high accuracy. The SVM models based on IMU data showed an accuracy of 87.2% (Set 1) and 97.0% (Set 2). The current work presents valid IMU-based features, employed in an SVM model for the classification of the gait of patients after THA and a healthy control. The study reveals that the features of Set 2 are more significant concerning the classification problem. The present IMU system proves its potential to provide accurate features for the incorporation in a mobile gait-feedback system for patients after THA.

Validity of inertial sensor based 3D joint kinematics of static and dynamic sport and physiotherapy specific movements.

3D joint kinematics can provide important information about the quality of movements. Optical motion capture systems (OMC) are considered the gold standard in motion analysis. However, in recent years, inertial measurement units (IMU) have become a promising alternative. The aim of this study was to validate IMU-based 3D joint kinematics of the lower extremities during different movements. Twenty-eight healthy subjects participated in this study. They performed bilateral squats (SQ), single-leg squats (SLS) and countermovement jumps (CMJ). The IMU kinematics was calculated using a recently-described sensor-fusion algorithm. A marker based OMC system served as a reference. Only the technical error based on algorithm performance was considered, incorporating OMC data for the calibration, initialization, and a biomechanical model. To evaluate the validity of IMU-based 3D joint kinematics, root mean squared error (RMSE), range of motion error (ROME), Bland-Altman (BA) analysis as well as the coefficient of multiple correlation (CMC) were calculated. The evaluation was twofold. First, the IMU data was compared to OMC data based on marker clusters; and, second based on skin markers attached to anatomical landmarks. The first evaluation revealed means for RMSE and ROME for all joints and tasks below 3°. The more dynamic task, CMJ, revealed error measures approximately 1° higher than the remaining tasks. Mean CMC values ranged from 0.77 to 1 over all joint angles and all tasks. The second evaluation showed an increase in the RMSE of 2.28°- 2.58° on average for all joints and tasks. Hip flexion revealed the highest average RMSE in all tasks (4.87°- 8.27°). The present study revealed a valid IMU-based approach for the measurement of 3D joint kinematics in functional movements of varying demands. The high validity of the results encourages further development and the extension of the present approach into clinical settings.

Towards Inertial Sensor Based Mobile Gait Analysis: Event-Detection and Spatio-Temporal Parameters.

The aim of this study was to assess the validity and test-retest reliability of an inertial measurement unit (IMU) system for gait analysis. Twenty-four healthy subjects conducted a 6-min walking test and were instrumented with seven IMUs and retroreflective markers. A kinematic approach was used to estimate the initial and terminal contact events in real-time. Based on these events twelve spatio-temporal parameters (STP) were calculated. A marker based optical motion capture (OMC) system provided the reference. Event-detection rate was about 99%. Detection offset was below 0.017 s. Relative root mean square error (RMSE) ranged from 0.90% to 4.40% for most parameters. However, the parameters that require spatial information of both feet showed higher errors. Step length showed a relative RMSE of 6.69%. Step width and swing width revealed the highest relative RMSE (34.34% and 35.20%). Test-retest results ranged from 0.67 to 0.92, except for the step width (0.25). Summarizing, it appears that the parameters describing the lateral distance between the feet need further improvement. However, the results of the validity and reliability of the IMU system encourage its validation in clinical settings as well as further research.

Validity, Test-Retest Reliability and Long-Term Stability of Magnetometer Free Inertial Sensor Based 3D Joint Kinematics.

The present study investigates an algorithm for the calculation of 3D joint angles based on inertial measurement units (IMUs), omitting magnetometer data. Validity, test-retest reliability, and long-term stability are evaluated in reference to an optical motion capture (OMC) system. Twenty-eight healthy subjects performed a 6 min walk test. Three-dimensional joint kinematics of the lower extremity was recorded simultaneously by means of seven IMUs and an OptiTrack OMC system. To evaluate the performance, the root mean squared error (RMSE), mean range of motion error (ROME), coefficient of multiple correlations (CMC), Bland-Altman (BA) analysis, and intraclass correlation coefficient (ICC) were calculated. For all joints, the RMSE was lower than 2.40°, and the ROME was lower than 1.60°. The CMC revealed good to excellent waveform similarity. Reliability was moderate to excellent with ICC values of 0.52⁻0.99 for all joints. Error measures did not increase over time. When considering soft tissue artefacts, RMSE and ROME increased by an average of 2.2° ± 1.5° and 2.9° ± 1.7°. This study revealed an excellent correspondence of a magnetometer-free IMU system with an OMC system when excluding soft tissue artefacts.

Survey of Motion Tracking Methods Based on Inertial Sensors: A Focus on Upper Limb Human Motion.

Motion tracking based on commercial inertial measurements units (IMUs) has been widely studied in the latter years as it is a cost-effective enabling technology for those applications in which motion tracking based on optical technologies is unsuitable. This measurement method has a high impact in human performance assessment and human-robot interaction. IMU motion tracking systems are indeed self-contained and wearable, allowing for long-lasting tracking of the user motion in situated environments. After a survey on IMU-based human tracking, five techniques for motion reconstruction were selected and compared to reconstruct a human arm motion. IMU based estimation was matched against motion tracking based on the Vicon marker-based motion tracking system considered as ground truth. Results show that all but one of the selected models perform similarly (about 35 mm average position estimation error).

On Inertial Body Tracking in the Presence of Model Calibration Errors.

In inertial body tracking, the human body is commonly represented as a biomechanical model consisting of rigid segments with known lengths and connecting joints. The model state is then estimated via sensor fusion methods based on data from attached inertial measurement units (IMUs). This requires the relative poses of the IMUs w.r.t. the segments-the IMU-to-segment calibrations, subsequently called I2S calibrations-to be known. Since calibration methods based on static poses, movements and manual measurements are still the most widely used, potentially large human-induced calibration errors have to be expected. This work compares three newly developed/adapted extended Kalman filter (EKF) and optimization-based sensor fusion methods with an existing EKF-based method w.r.t. their segment orientation estimation accuracy in the presence of model calibration errors with and without using magnetometer information. While the existing EKF-based method uses a segment-centered kinematic chain biomechanical model and a constant angular acceleration motion model, the newly developed/adapted methods are all based on a free segments model, where each segment is represented with six degrees of freedom in the global frame. Moreover, these methods differ in the assumed motion model (constant angular acceleration, constant angular velocity, inertial data as control input), the state representation (segment-centered, IMU-centered) and the estimation method (EKF, sliding window optimization). In addition to the free segments representation, the optimization-based method also represents each IMU with six degrees of freedom in the global frame. In the evaluation on simulated and real data from a three segment model (an arm), the optimization-based method showed the smallest mean errors, standard deviations and maximum errors throughout all tests. It also showed the lowest dependency on magnetometer information and motion agility. Moreover, it was insensitive w.r.t. I2S position and segment length errors in the tested ranges. Errors in the I2S orientations were, however, linearly propagated into the estimated segment orientations. In the absence of magnetic disturbances, severe model calibration errors and fast motion changes, the newly developed IMU centered EKF-based method yielded comparable results with lower computational complexity.

Cognitive Learning, Monitoring and Assistance of Industrial Workflows Using Egocentric Sensor Networks.

Today, the workflows that are involved in industrial assembly and production activities are becoming increasingly complex. To efficiently and safely perform these workflows is demanding on the workers, in particular when it comes to infrequent or repetitive tasks. This burden on the workers can be eased by introducing smart assistance systems. This article presents a scalable concept and an integrated system demonstrator designed for this purpose. The basic idea is to learn workflows from observing multiple expert operators and then transfer the learnt workflow models to novice users. Being entirely learning-based, the proposed system can be applied to various tasks and domains. The above idea has been realized in a prototype, which combines components pushing the state of the art of hardware and software designed with interoperability in mind. The emphasis of this article is on the algorithms developed for the prototype: 1) fusion of inertial and visual sensor information from an on-body sensor network (BSN) to robustly track the user's pose in magnetically polluted environments; 2) learning-based computer vision algorithms to map the workspace, localize the sensor with respect to the workspace and capture objects, even as they are carried; 3) domain-independent and robust workflow recovery and monitoring algorithms based on spatiotemporal pairwise relations deduced from object and user movement with respect to the scene; and 4) context-sensitive augmented reality (AR) user feedback using a head-mounted display (HMD). A distinguishing key feature of the developed algorithms is that they all operate solely on data from the on-body sensor network and that no external instrumentation is needed. The feasibility of the chosen approach for the complete action-perception-feedback loop is demonstrated on three increasingly complex datasets representing manual industrial tasks. These limited size datasets indicate and highlight the potential of the chosen technology as a combined entity as well as point out limitations of the system.

Innovative system for real-time ergonomic feedback in industrial manufacturing.

This work presents a system that permits a real-time ergonomic assessment of manual tasks in an industrial environment. First, a biomechanical model of the upper body has been developed by using inertial sensors placed at different locations on the upper body. Based on this model, a computerized RULA ergonomic assessment was implemented to permit a global risk assessment of musculoskeletal disorders in real-time. Furthermore, local scores were calculated per segment, e.g. the neck region, and gave information on the local risks for musculoskeletal disorders. Visual information was fed back to the user by using a see-through head mounted display. Additional visual highlighting and auditory warnings were provided when some predefined thresholds were exceeded. In a user study (N = 12 participants) a group with the RULA feedback was compared to a control group. Results demonstrate that the real-time ergonomic feedback significantly decreased the outcome of both globally as well as locally hazardous RULA values that are associated with increased risk for musculoskeletal disorders. Task execution time did not differ between groups. The real-time ergonomic tool introduced in this study has the potential to considerably reduce the risk of musculoskeletal disorders in industrial settings. Implications for ergonomics in manufacturing and user feedback modalities are further discussed.