Time-Based and Path-Based Analysis of Upper-Limb Movements during Activities of Daily Living.

Patients after stroke need to re-learn functional movements required for independent living throughout the rehabilitation process. In the study, we used a wearable sensory system for monitoring the movement of the upper limbs while performing activities of daily living. We implemented time-based and path-based segmentation of movement trajectories and muscle activity to quantify the activities of the unaffected and the affected upper limbs. While time-based segmentation splits the trajectory in quants of equal duration, path-based segmentation isolates completed movements. We analyzed the hand movement path and forearm muscle activity and introduced a bimanual movement parameter, which enables differentiation between unimanual and bimanual activities. The approach was validated in a study that included a healthy subject and seven patients after stroke with different levels of disabilities. Path-based segmentation provides a more detailed and comprehensive evaluation of upper limb activities, while time-based segmentation is more suitable for real-time assessment and providing feedback to patients. Bimanual movement parameter effectively differentiates between different levels of upper limb involvement and is a clear indicator of the activity of the affected limb relative to the unaffected limb.

Modular Lidar System for Multiple Field-of-View Ranging.

This paper explores the possibility of distributing the fields of view (FOVs) of a centralized lidar cluster using fixed mirrors for future use in safety applications in robotics and elsewhere. A custom modular lidar system with time-over-threshold (TOT) walk error compensation was developed for the experiments. It comprises a control board that provides the processing power and adjustable voltage regulation, and multiple individually addressable analogue front end (AFE) boards that each contain a transmitter, a receiver, time-to-digital (TDC) converters for pulse width measurements on the bot Tx and Rx side, and adjustable reference voltage generators for both the Tx and Rx pulse detection threshold. The lidar system's performance with a target in the direct line of sight is compared to the configurations where the FOV is redirected with up to three mirrors in different configurations. The results show that the light path through the neighboring mirrors introduces a minor but noticeable measurement error on a portion of the measurement range.

Collaborative Robot Precision Task in Medical Microbiology Laboratory.

This study focuses on the feasibility of collaborative robot implementation in a medical microbiology laboratory by demonstrating fine tasks using kinesthetic teaching. Fine tasks require sub-millimetre positioning accuracy. Bacterial colony picking and identification was used as a case study. Colonies were picked from Petri dishes and identified using matrix-assisted laser desorption/ionization (MALDI) time-of-flight (TOF) mass spectrometry. We picked and identified 56 colonies (36 colonies of Gram-negative Acinetobacter baumannii and 20 colonies of Gram-positive Staphylococcus epidermidis). The overall identification error rate was around 11%, although it was significantly lower for Gram-positive bacteria (5%) than Gram-negative bacteria (13.9%). Based on the identification scores, it was concluded that the system works similarly well as a manual operator. It was determined that tasks were successfully demonstrated using kinesthetic teaching and generalized using dynamic movement primitives (DMP). Further improvement of the identification error rate is possible by choosing a different deposited sample treatment method (e.g., semi-extraction, wet deposition).

Distance Measurements in UWB-Radio Localization Systems Corrected with a Feedforward Neural Network Model.

An ultra-wideband (UWB) localization system is an alternative in a GPS-denied environment. However, a distance measurement with UWB modules using a two-way communication protocol induces an orientation-dependent error. Previous research studied this error by looking at parameters such as the received power and the channel response signal. In this paper, the neural network (NN) method for correcting the orientation-induced distance error without the need to calculate the signal strength, obtain the channel response or know any parameters of the antenna and the UWB modules is presented. The NN method utilizes only the measured distance and the tag orientation, and implements an NN model obtained by machine learning, using measurements at different distances and orientations of the two UWB modules. The verification of the experimental setup with 12 anchors and a tag shows that with the proposed NN method, 5 cm better root mean square error values (RMSEs) are obtained for the measured distance between the anchors and the tag compared to the calibration method that did not include orientation information. With the least-square estimator, 14 cm RMSE in 3D is obtained with the NN model corrected distances, with a 9 cm improvement compared to when raw distances are used. The method produces better results without the need to obtain the UWB module's diagnostics parameters that are required to calculate the received signal strength or channel response, and in this way maintain the minimum packet size for the ranging protocol.

Centre of Pressure Estimation during Walking Using Only Inertial-Measurement Units and End-To-End Statistical Modelling.

Estimation of the centre of pressure (COP) is an important part of the gait analysis, for example, when evaluating the functional capacity of individuals affected by motor impairment. Inertial measurement units (IMUs) and force sensors are commonly used to measure gait characteristic of healthy and impaired subjects. We present a methodology for estimating the COP solely from raw gyroscope, accelerometer, and magnetometer data from IMUs using statistical modelling. We demonstrate the viability of the method using an example of two models: a linear model and a non-linear Long-Short-Term Memory (LSTM) neural network model. Models were trained on the COP ground truth data measured using an instrumented treadmill and achieved the average intra-subject root mean square (RMS) error between estimated and ground truth COP of 12.3 mm and the average inter-subject RMS error of 23.7 mm which is comparable or better than similar studies so far. We show that the calibration procedure in the instrumented treadmill can be as short as a couple of minutes without the decrease in our model performance. We also show that the magnetic component of the recorded IMU signal, which is most sensitive to environmental changes, can be safely dropped without a significant decrease in model performance. Finally, we show that the number of IMUs can be reduced to five without deterioration in the model performance.

Pressure-Sensitive Insoles for Real-Time Gait-Related Applications.

Wearable robotic devices require sensors and algorithms that can recognize the user state in real-time, in order to provide synergistic action with the body. For devices intended for locomotion-related applications, shoe-embedded sensors are a common and convenient choice, potentially advantageous for performing gait assessment in real-world environments. In this work, we present the development of a pair of pressure-sensitive insoles based on optoelectronic sensors for the real-time estimation of temporal gait parameters. The new design makes use of a simplified sensor configuration that preserves the time accuracy of gait event detection relative to previous prototypes. The system has been assessed relatively to a commercial force plate recording the vertical component of the ground reaction force (vGRF) and the coordinate of the center of pressure along the so-called progression or antero-posterior plane (CoPAP) in ten healthy participants during ground-level walking at two speeds. The insoles showed overall median absolute errors (MAE) of 0.06 (0.02) s and 0.04 (0.02) s for heel-strike and toe-off recognition, respectively. Moreover, they enabled reasonably accurate estimations of the stance phase duration (2.02 (2.03) % error) and CoPAP profiles (Pearson correlation coefficient with force platform ρCoP = 0.96 (0.02)), whereas the correlation with vGRF measured by the force plate was lower than that obtained with the previous prototype (ρvGRF = 0.47 (0.20)). These results confirm the suitability of the insoles for online sensing purposes such as timely gait phase estimation and discrete event recognition.

Infant posture and movement analysis using a sensor-supported gym with toys.

Infant posture and motor pattern development are normally analyzed by clinical assessment scales. Lately, this approach is combined with the use of sensor-supported systems, such as optical, inertial, and electromagnetic measurement systems, as well as novel assessment devices, such as CareToy. CareToy is a modular device for assessment and rehabilitation of preterm infants, comprising pressure mattresses, inertial and magnetic measurement units, and sensorized toys. Since such integrated sensor system combination is new to the field of sensor-supported infant behavior assessment and rehabilitation, dedicated methods for data analysis were developed and presented. These comprise trunk rotation, arm movement, forearm orientation, and head movement analysis, along with toy play and trunk posture stability evaluation. Methods were tested on case study data, evaluating suitability of developed algorithms for infant posture and activity analysis, regardless of behavioral responses. Obtained results demonstrate suitability of the proposed methods for successful use in studies of different motor pattern subfields. This represents an important step on the course towards objective, accurate, sensor-supported infant motor development assessment. Graphical abstract Posture and movement assessment of infants using analysis of sensory data, obtained with a dedicated sensorized gym with toys.

The Challenges and Achievements of Experimental Implementation of an Active Transfemoral Prosthesis Based on Biological Quasi-Stiffness: The CYBERLEGs Beta-Prosthesis.

The CYBERLEGs Beta-Prosthesis is an active transfemoral prosthesis that can provide the full torque required for reproducing average level ground walking at both the knee and ankle in the sagittal plane. The prosthesis attempts to produce a natural level ground walking gait that approximates the joint torques and kinematics of a non-amputee while maintaining passively compliant joints, the stiffnesses of which were derived from biological quasi-stiffness measurements. The ankle of the prosthesis consists of a series elastic actuator with a parallel spring and the knee is composed of three different systems that must compliment each other to generate the correct joint behavior: a series elastic actuator, a lockable parallel spring and an energy transfer mechanism. Bench testing of this new prosthesis was completed and demonstrated that the device was able to create the expected torque-angle characteristics for a normal walker under ideal conditions. The experimental trials with four amputees walking on a treadmill to validate the behavior of the prosthesis proved that although the prosthesis could be controlled in a way that allowed all subjects to walk, the accurate timing and kinematic requirements of the output of the device limited the efficacy of using springs with quasi-static stiffnesses. Modification of the control and stiffness of the series springs could provide better performance in future work.

Using Inertial Measurement Units and Electromyography to Quantify Movement during Action Research Arm Test Execution.

In patients after stroke, ability of the upper limb is commonly assessed with standardised clinical tests that provide a complete upper limb assessment. This paper presents quantification of upper limb movement during the execution of Action research arm test (ARAT) using a wearable system of inertial measurement units (IMU) for kinematic quantification and electromyography (EMG) sensors for muscle activity analysis. The test was executed with each arm by a group of healthy subjects and a group of patients after stroke allocated into subgroups based on their clinical scores. Tasks were segmented into movement and manipulation phases. Each movement phase was quantified with a set of five parameters: movement time, movement smoothness, hand trajectory similarity, trunk stability, and muscle activity for grasping. Parameters vary between subject groups, between tasks, and between task phases. Statistically significant differences were observed between patient groups that obtained different clinical scores, between healthy subjects and patients, and between the unaffected and the affected arm unless the affected arm shows normal performance. Movement quantification enables differentiation between different subject groups within movement phases as well as for the complete task. Spearman's rank correlation coefficient shows strong correlations between patient's ARAT scores and movement time as well as movement smoothness. Weak to moderate correlations were observed for parameters that describe hand trajectory similarity and trunk stability. Muscle activity correlates well with grasping activity and the level of grasping force in all groups.

Compensation for Magnetic Disturbances in Motion Estimation to Provide Feedback to Wearable Robotic Systems.

The direction of the Earth's magnetic field is used as a reference vector to determine the heading in orientation estimation with wearable sensors. However, the magnetic field strength is weak and can be easily disturbed in the vicinity of ferromagnetic materials, which may result in inaccurate estimate of orientation. This paper presents a novel method for estimating and compensating for magnetic disturbances. The compensation algorithm is implemented within a kinematic-based extended Kalman filter and is based on an assessment of the magnetic disturbance and the change of orientation in each time step. The proposed algorithm was experimentally validated by measuring the orientation of a simple mechanical system with three degrees of freedom in an artificially disturbed magnetic field. The results of the experimental evaluation show that an Kalman filter algorithm that incorporates compensating for magnetic disturbances is capable of estimating the orientation with moderate error (the absolute median errors , ) when the Earth's magnetic field is disturbed by magnetic disturbance with a magnitude equal to twice the magnitude of the Earth's own magnetic field in different directions.

Whole Body Awareness for Controlling a Robotic Transfemoral Prosthesis.

Restoring locomotion functionality of transfemoral amputees is essential for early rehabilitation treatment and for preserving mobility and independence in daily life. Research in wearable robotics fostered the development of innovative active mechatronic lower-limb prostheses designed with the goal to reduce the cognitive and physical effort of lower-limb amputees in rehabilitation and daily life activities. To ensure benefits to the users, active mechatronic prostheses are expected to be aware of the user intention and properly interact in a closed human-in-the-loop paradigm. In the state of the art various cognitive interfaces have been proposed to online decode the user's intention. Electromyography in combination with mechanical sensing such as inertial or pressure sensors is a widely adopted solution for driving active mechatronic prostheses. In this framework, researchers also explored targeted muscles re-innervation for an objective-oriented surgical amputation promoting wider usability of active prostheses. However, information kept by the neural component of the cognitive interface deteriorates in a prolonged use scenario due to electrodes-related issues, thereby undermining the correct functionality of the active prosthesis. The objective of this work is to present a novel controller for an active transfemoral prosthesis based on whole body awareness relying on a wireless distributed non-invasive sensory apparatus acting as cognitive interface. A finite-state machine controller based on signals monitored from the wearable interface performs subject-independent intention detection of functional tasks such as ground level walking, stair ascent, and sit-to-stand maneuvres and their main sub-phases. Experimental activities carried out with four transfemoral amputees (among them one dysvascular) demonstrated high reliability of the controller capable of providing 100% accuracy rate in treadmill walking even for weak subjects and low walking speeds. The minimum success rate was of 94.8% in performing sit-to-stand tasks. All the participants showed high confidence in using the transfemoral active prosthesis even without training period thanks to intuitiveness of the whole body awareness controller.

Experimental Validation of Motor Primitive-Based Control for Leg Exoskeletons during Continuous Multi-Locomotion Tasks.

An emerging approach to design locomotion assistive devices deals with reproducing desirable biological principles of human locomotion. In this paper, we present a bio-inspired controller for locomotion assistive devices based on the concept of motor primitives. The weighted combination of artificial primitives results in a set of virtual muscle stimulations. These stimulations then activate a virtual musculoskeletal model producing reference assistive torque profiles for different locomotion tasks (i.e., walking, ascending stairs, and descending stairs). The paper reports the validation of the controller through a set of experiments conducted with healthy participants. The proposed controller was tested for the first time with a unilateral leg exoskeleton assisting hip, knee, and ankle joints by delivering a fraction of the computed reference torques. Importantly, subjects performed a track involving ground-level walking, ascending stairs, and descending stairs and several transitions between these tasks. These experiments highlighted the capability of the controller to provide relevant assistive torques and to effectively handle transitions between the tasks. Subjects displayed a natural interaction with the device. Moreover, they significantly decreased the time needed to complete the track when the assistance was provided, as compared to wearing the device with no assistance.

CareToy: Stimulation and Assessment of Preterm Infant's Activity Using a Novel Sensorized System.

Early intervention programs aim at improving cognitive and motor outcomes of preterm infants. Intensive custom-tailored training activities are usually accompanied by assessment procedures, which have shortcomings, such as subjectivity, complex setups, and need for structured environments. A novel sensorized system, called CareToy, was designed to provide stimulation in the form of goal-directed activity training scenarios and motor pattern assessment of main developmental milestones, such as rolling activity, grasping, and postural stability. A group of 28 differently skilled preterm infants were enrolled. Acquired measurement data were analysed with dedicated sensor data processing algorithms, along with clinical evaluation of motor ability. High correlation among technically determined parameters and Alberta Infant Motor Scale values was determined by Pearson correlation coefficients. Due to good accuracy and possibility of single motor skill subfield analysis, results confirm system suitability for motor ability assessment. Statistical analysis of inter-motor ability group and inter-training goal data comparisons demonstrate system's appropriateness for goal-directed activity stimulation. The proposed system has evident potential of being an important contribution to the field of infant motor development assessment, expanding accessibility of early intervention programs and affecting rehabilitation effectiveness of preterm infants.

Estimation of joint forces and moments for the in-run and take-off in ski jumping based on measurements with wearable inertial sensors.

This study uses inertial sensors to measure ski jumper kinematics and joint dynamics, which was until now only a part of simulation studies. For subsequent calculation of dynamics in the joints, a link-segment model was developed. The model relies on the recursive Newton-Euler inverse dynamics. This approach allowed the calculation of the ground reaction force at take-off. For the model validation, four ski jumpers from the National Nordic center performed a simulated jump in a laboratory environment on a force platform; in total, 20 jumps were recorded. The results fit well to the reference system, presenting small errors in the mean and standard deviation and small root-mean-square errors. The error is under 12% of the reference value. For field tests, six jumpers participated in the study; in total, 28 jumps were recorded. All of the measured forces and moments were within the range of prior simulated studies. The proposed system was able to indirectly provide the values of forces and moments in the joints of the ski-jumpers' body segments, as well as the ground reaction force during the in-run and take-off phases in comparison to the force platform installed on the table. Kinematics assessment and estimation of dynamics parameters can be applied to jumps from any ski jumping hill.

Fuzzy-logic-based hybrid locomotion mode classification for an active pelvis orthosis: Preliminary results.

In this paper, we present a fuzzy-logic-based hybrid locomotion mode classification method for an active pelvis orthosis. Locomotion information measured by the onboard hip joint angle sensors and the pressure insoles is used to classify five locomotion modes, including two static modes (sitting, standing still), and three dynamic modes (level-ground walking, ascending stairs, and descending stairs). The proposed method classifies these two kinds of modes first by monitoring the variation of the relative hip joint angle between the two legs within a specific period. Static states are then classified by the time-based absolute hip joint angle. As for dynamic modes, a fuzzy-logic based method is proposed for the classification. Preliminary experimental results with three able-bodied subjects achieve an off-line classification accuracy higher than 99.49%.

Sensory data fusion of pressure mattress and wireless inertial magnetic measurement units.

Head movement of infants is an important parameter for analysing infant motor patterns. Despite its importance, this field has received little sensory-based research in the past years. Therefore, we present a sensory-supported data fusion model for head movement analysis of infants in supine position. The sensory system comprises a pressure mattress and two wireless inertial magnetic measurement units, rendering precise, objective and non-intrusive information on pressure distribution and 3D trunk orientation, respectively. Algorithms first perform pressure data pre-processing and calculate image moments to acquire 2D trunk orientation. Afterwards, unscented Kalman filter is used for sensory data fusion. After additional data processing, head and trunk coordinates are calculated along with head displacement distance. The sensory system was tested on experimental measurements, performed in eight normally developing infants aged from 1 to 5 months. Results of several algorithm combinations were compared to referential video recordings in terms of head lifts. Combination of algorithms, incorporating head tracking and sensory data fusion provides completely accurate results in comparison to normative data. Statistical data analysis and referential optoelectronic measurements were performed to evaluate accuracy of the sensory fusion model. Suitability of the proposed sensory system for head movement analysis of infants in supine position was verified.

Toward real-time automated detection of turns during gait using wearable inertial measurement units.

Previous studies have presented algorithms for detection of turns during gait using wearable sensors, but those algorithms were not built for real-time use. This paper therefore investigates the optimal approach for real-time detection of planned turns during gait using wearable inertial measurement units. Several different sensor positions (head, back and legs) and three different detection criteria (orientation, angular velocity and both) are compared with regard to their ability to correctly detect turn onset. Furthermore, the different sensor positions are compared with regard to their ability to predict the turn direction and amplitude. The evaluation was performed on ten healthy subjects who performed left/right turns at three amplitudes (22, 45 and 90 degrees). Results showed that turn onset can be most accurately detected with sensors on the back and using a combination of orientation and angular velocity. The same setup also gives the best prediction of turn direction and amplitude. Preliminary measurements with a single amputee were also performed and highlighted important differences such as slower turning that need to be taken into account.

Infant trunk posture and arm movement assessment using pressure mattress, inertial and magnetic measurement units (IMUs).

BACKGROUND: Existing motor pattern assessment methods, such as digital cameras and optoelectronic systems, suffer from object obstruction and require complex setups. To overcome these drawbacks, this paper presents a novel approach for biomechanical evaluation of newborn motor skills development. Multi-sensor measurement system comprising pressure mattress and IMUs fixed on trunk and arms is proposed and used as alternative to existing methods. Observed advantages seem appealing for the focused field and in general. Combined use of pressure distribution data and kinematic information is important also for posture assessment, ulcer prevention, and non-invasive sleep pattern analysis of adults. METHODS: Arm kinematic parameters, such as root-mean-square acceleration, spectral arc length of hand velocity profile, including arm workspace surface area, and travelled hand path are obtained with the multi-sensor measurement system and compared to normative motion capture data for evaluation of adequacy. Two IMUs per arm, only one IMU on upper arm, and only one IMU on forearm sensor placement options are studied to assess influence of system configuration on method precision. Combination of pressure mattress and IMU fixed on the trunk is used to measure trunk position (obtained from mat), rotation (from IMUs) and associated movements on surface (from both). Measurement system is first validated on spontaneous arm and trunk movements of a dedicated baby doll having realistic anthropometric characteristics of newborns. Next, parameters of movements in a healthy infant are obtained with pressure mattress, along with trunk and forearm IMU sensors to verify appropriateness of method and parameters. RESULTS: Evaluation results confirm that full sensor set, comprising pressure mattress and two IMUs per arm is a reliable substitution to optoelectronic systems. Motor pattern parameter errors are under 10% and kinematic estimation error is in range of 2 cm. Although, use of only forearm IMU is not providing best possible kinematic precision, the simplicity of use and still acceptable accuracy are convincing for frequent practical use. Measurements demonstrated system high mobility and usability. CONCLUSIONS: Study results confirm adequacy of the proposed multi-sensor measurement system, indicating its enviable potential for accurate infant trunk posture and arm movement assessment.

Online phase detection using wearable sensors for walking with a robotic prosthesis.

This paper presents a gait phase detection algorithm for providing feedback in walking with a robotic prosthesis. The algorithm utilizes the output signals of a wearable wireless sensory system incorporating sensorized shoe insoles and inertial measurement units attached to body segments. The principle of detecting transitions between gait phases is based on heuristic threshold rules, dividing a steady-state walking stride into four phases. For the evaluation of the algorithm, experiments with three amputees, walking with the robotic prosthesis and wearable sensors, were performed. Results show a high rate of successful detection for all four phases (the average success rate across all subjects >90%). A comparison of the proposed method to an off-line trained algorithm using hidden Markov models reveals a similar performance achieved without the need for learning dataset acquisition and previous model training.

Automated detection of gait initiation and termination using wearable sensors.

This paper presents algorithms for detection of gait initiation and termination using wearable inertial measurement units and pressure-sensitive insoles. Body joint angles, joint angular velocities, ground reaction force and center of plantar pressure of each foot are obtained from these sensors and input into supervised machine learning algorithms. The proposed initiation detection method recognizes two events: gait onset (an anticipatory movement preceding foot lifting) and toe-off. The termination detection algorithm segments gait into steps, measures the signals over a buffer at the beginning of each step, and determines whether this measurement belongs to the final step. The approach is validated with 10 subjects at two gait speeds, using within-subject and subject-independent cross-validation. Results show that gait initiation can be detected timely and accurately, with few errors in the case of within-subject cross-validation and overall good performance in subject-independent cross-validation. Gait termination can be predicted in over 80% of trials well before the subject comes to a complete stop. Results also show that the two sensor types are equivalent in predicting gait initiation while inertial measurement units are generally superior in predicting gait termination. Potential use of the algorithms is foreseen primarily with assistive devices such as prostheses and exoskeletons.