Time-Based and Path-Based Analysis of Upper-Limb Movements during Activities of Daily Living.

Patients after stroke need to re-learn functional movements required for independent living throughout the rehabilitation process. In the study, we used a wearable sensory system for monitoring the movement of the upper limbs while performing activities of daily living. We implemented time-based and path-based segmentation of movement trajectories and muscle activity to quantify the activities of the unaffected and the affected upper limbs. While time-based segmentation splits the trajectory in quants of equal duration, path-based segmentation isolates completed movements. We analyzed the hand movement path and forearm muscle activity and introduced a bimanual movement parameter, which enables differentiation between unimanual and bimanual activities. The approach was validated in a study that included a healthy subject and seven patients after stroke with different levels of disabilities. Path-based segmentation provides a more detailed and comprehensive evaluation of upper limb activities, while time-based segmentation is more suitable for real-time assessment and providing feedback to patients. Bimanual movement parameter effectively differentiates between different levels of upper limb involvement and is a clear indicator of the activity of the affected limb relative to the unaffected limb.

Collaborative Robot Precision Task in Medical Microbiology Laboratory.

This study focuses on the feasibility of collaborative robot implementation in a medical microbiology laboratory by demonstrating fine tasks using kinesthetic teaching. Fine tasks require sub-millimetre positioning accuracy. Bacterial colony picking and identification was used as a case study. Colonies were picked from Petri dishes and identified using matrix-assisted laser desorption/ionization (MALDI) time-of-flight (TOF) mass spectrometry. We picked and identified 56 colonies (36 colonies of Gram-negative Acinetobacter baumannii and 20 colonies of Gram-positive Staphylococcus epidermidis). The overall identification error rate was around 11%, although it was significantly lower for Gram-positive bacteria (5%) than Gram-negative bacteria (13.9%). Based on the identification scores, it was concluded that the system works similarly well as a manual operator. It was determined that tasks were successfully demonstrated using kinesthetic teaching and generalized using dynamic movement primitives (DMP). Further improvement of the identification error rate is possible by choosing a different deposited sample treatment method (e.g., semi-extraction, wet deposition).

Infant posture and movement analysis using a sensor-supported gym with toys.

Infant posture and motor pattern development are normally analyzed by clinical assessment scales. Lately, this approach is combined with the use of sensor-supported systems, such as optical, inertial, and electromagnetic measurement systems, as well as novel assessment devices, such as CareToy. CareToy is a modular device for assessment and rehabilitation of preterm infants, comprising pressure mattresses, inertial and magnetic measurement units, and sensorized toys. Since such integrated sensor system combination is new to the field of sensor-supported infant behavior assessment and rehabilitation, dedicated methods for data analysis were developed and presented. These comprise trunk rotation, arm movement, forearm orientation, and head movement analysis, along with toy play and trunk posture stability evaluation. Methods were tested on case study data, evaluating suitability of developed algorithms for infant posture and activity analysis, regardless of behavioral responses. Obtained results demonstrate suitability of the proposed methods for successful use in studies of different motor pattern subfields. This represents an important step on the course towards objective, accurate, sensor-supported infant motor development assessment. Graphical abstract Posture and movement assessment of infants using analysis of sensory data, obtained with a dedicated sensorized gym with toys.

Using Inertial Measurement Units and Electromyography to Quantify Movement during Action Research Arm Test Execution.

In patients after stroke, ability of the upper limb is commonly assessed with standardised clinical tests that provide a complete upper limb assessment. This paper presents quantification of upper limb movement during the execution of Action research arm test (ARAT) using a wearable system of inertial measurement units (IMU) for kinematic quantification and electromyography (EMG) sensors for muscle activity analysis. The test was executed with each arm by a group of healthy subjects and a group of patients after stroke allocated into subgroups based on their clinical scores. Tasks were segmented into movement and manipulation phases. Each movement phase was quantified with a set of five parameters: movement time, movement smoothness, hand trajectory similarity, trunk stability, and muscle activity for grasping. Parameters vary between subject groups, between tasks, and between task phases. Statistically significant differences were observed between patient groups that obtained different clinical scores, between healthy subjects and patients, and between the unaffected and the affected arm unless the affected arm shows normal performance. Movement quantification enables differentiation between different subject groups within movement phases as well as for the complete task. Spearman's rank correlation coefficient shows strong correlations between patient's ARAT scores and movement time as well as movement smoothness. Weak to moderate correlations were observed for parameters that describe hand trajectory similarity and trunk stability. Muscle activity correlates well with grasping activity and the level of grasping force in all groups.

CareToy: Stimulation and Assessment of Preterm Infant's Activity Using a Novel Sensorized System.

Early intervention programs aim at improving cognitive and motor outcomes of preterm infants. Intensive custom-tailored training activities are usually accompanied by assessment procedures, which have shortcomings, such as subjectivity, complex setups, and need for structured environments. A novel sensorized system, called CareToy, was designed to provide stimulation in the form of goal-directed activity training scenarios and motor pattern assessment of main developmental milestones, such as rolling activity, grasping, and postural stability. A group of 28 differently skilled preterm infants were enrolled. Acquired measurement data were analysed with dedicated sensor data processing algorithms, along with clinical evaluation of motor ability. High correlation among technically determined parameters and Alberta Infant Motor Scale values was determined by Pearson correlation coefficients. Due to good accuracy and possibility of single motor skill subfield analysis, results confirm system suitability for motor ability assessment. Statistical analysis of inter-motor ability group and inter-training goal data comparisons demonstrate system's appropriateness for goal-directed activity stimulation. The proposed system has evident potential of being an important contribution to the field of infant motor development assessment, expanding accessibility of early intervention programs and affecting rehabilitation effectiveness of preterm infants.

An affordable, adaptable, and hybrid assistive device for upper-limb neurorehabilitation.

The paper presents a multisensory and multimodal device for neuromuscular rehabilitation of the upper limb, designed to enable enriched rehabilitation treatment in both clinical and home environments. Originating from an existing low-cost, variable-stiffness rehabilitation device, it expands its functionalities by integrating additional modules in order to augment application scenarios and applicable clinical techniques. The newly developed system focuses on the integration of a wearable neuromuscular electrical stimulation system, a virtual rehabilitation scenario, a low-cost unobtrusive sensory system and a patient model for adapting training task parameters. It also monitors the user behavior during each single session and its evolution throughout the entire training period. The result is a modular, integrated and affordable rehabilitation device, enabling a biomechanical, neurological, and physiological-based training of patients, including innovative features currently unavailable within off-the-shelf rehabilitation devices.

Sensory data fusion of pressure mattress and wireless inertial magnetic measurement units.

Head movement of infants is an important parameter for analysing infant motor patterns. Despite its importance, this field has received little sensory-based research in the past years. Therefore, we present a sensory-supported data fusion model for head movement analysis of infants in supine position. The sensory system comprises a pressure mattress and two wireless inertial magnetic measurement units, rendering precise, objective and non-intrusive information on pressure distribution and 3D trunk orientation, respectively. Algorithms first perform pressure data pre-processing and calculate image moments to acquire 2D trunk orientation. Afterwards, unscented Kalman filter is used for sensory data fusion. After additional data processing, head and trunk coordinates are calculated along with head displacement distance. The sensory system was tested on experimental measurements, performed in eight normally developing infants aged from 1 to 5 months. Results of several algorithm combinations were compared to referential video recordings in terms of head lifts. Combination of algorithms, incorporating head tracking and sensory data fusion provides completely accurate results in comparison to normative data. Statistical data analysis and referential optoelectronic measurements were performed to evaluate accuracy of the sensory fusion model. Suitability of the proposed sensory system for head movement analysis of infants in supine position was verified.

Infant trunk posture and arm movement assessment using pressure mattress, inertial and magnetic measurement units (IMUs).

BACKGROUND: Existing motor pattern assessment methods, such as digital cameras and optoelectronic systems, suffer from object obstruction and require complex setups. To overcome these drawbacks, this paper presents a novel approach for biomechanical evaluation of newborn motor skills development. Multi-sensor measurement system comprising pressure mattress and IMUs fixed on trunk and arms is proposed and used as alternative to existing methods. Observed advantages seem appealing for the focused field and in general. Combined use of pressure distribution data and kinematic information is important also for posture assessment, ulcer prevention, and non-invasive sleep pattern analysis of adults. METHODS: Arm kinematic parameters, such as root-mean-square acceleration, spectral arc length of hand velocity profile, including arm workspace surface area, and travelled hand path are obtained with the multi-sensor measurement system and compared to normative motion capture data for evaluation of adequacy. Two IMUs per arm, only one IMU on upper arm, and only one IMU on forearm sensor placement options are studied to assess influence of system configuration on method precision. Combination of pressure mattress and IMU fixed on the trunk is used to measure trunk position (obtained from mat), rotation (from IMUs) and associated movements on surface (from both). Measurement system is first validated on spontaneous arm and trunk movements of a dedicated baby doll having realistic anthropometric characteristics of newborns. Next, parameters of movements in a healthy infant are obtained with pressure mattress, along with trunk and forearm IMU sensors to verify appropriateness of method and parameters. RESULTS: Evaluation results confirm that full sensor set, comprising pressure mattress and two IMUs per arm is a reliable substitution to optoelectronic systems. Motor pattern parameter errors are under 10% and kinematic estimation error is in range of 2 cm. Although, use of only forearm IMU is not providing best possible kinematic precision, the simplicity of use and still acceptable accuracy are convincing for frequent practical use. Measurements demonstrated system high mobility and usability. CONCLUSIONS: Study results confirm adequacy of the proposed multi-sensor measurement system, indicating its enviable potential for accurate infant trunk posture and arm movement assessment.

Dynamic symmetrical pattern projection based laser triangulation sensor for precise surface position measurement of various material types.

This paper describes a custom, material-type-independent laser-triangulation-based measurement system that utilizes a high-quality ultraviolet laser beam. Laser structuring applications demand material surface alignment regarding the laser focus position, where fabrication conditions are optimal. Robust alignment of various material types was solved by introducing dynamic symmetrical pattern projection, and a "double curve fitting" centroid detection algorithm with subsurface scattering compensation. Experimental results have shown that the measurement system proves robust to laser intensity variation, with measurement bias lower than 50 µm and standard deviation lower than ±6.3 µm for all materials. The developed probe has been integrated into a PCB prototyping system for material referencing purposes.

Skill transfer from symmetric and asymmetric bimanual training using a robotic system to single limb performance.

BACKGROUND: Humans are capable of fast adaptation to new unknown dynamics that affect their movements. Such motor learning is also believed to be an important part of motor rehabilitation. Bimanual training can improve post-stroke rehabilitation outcome and is associated with interlimb coordination between both limbs. Some studies indicate partial transfer of skills among limbs of healthy individuals. Another aspect of bimanual training is the (a)symmetry of bimanual movements and how these affect motor learning and possibly post-stroke rehabilitation. METHODS: A novel bimanual 2-DOF robotic system was used for both bimanual and unimanual reaching movements. 35 young healthy adults participated in the study. They were divided into 5 test groups that performed movements under different conditions (bimanual or unimanual movements and symmetric or asymmetric bimanual arm loads). The subjects performed a simple tracking exercise with the bimanual system. The exercise was developed to stimulate motor learning by applying a velocity-dependent disturbance torque to the handlebar. Each subject performed 255 trials divided into three phases: baseline without disturbance torque, training phase with disturbance torque and evaluation phase with disturbance torque. RESULTS: Performance was assessed with the maximal values of rotation errors of the handlebar. After exposure to disturbance torque, the errors decreased for both unimanual and bimanual training. Errors in unimanual evaluation following the bimanual training phase were not significantly different from errors in unimanual evaluation following unimanual training. There was no difference in performance following symmetric or asymmetric training. Changing the arm force symmetry during bimanual movements from asymmetric to symmetric had little influence on performance. CONCLUSIONS: Subjects could adapt to an unknown disturbance torque that was changing the dynamics of the movements. The learning effect was present during both unimanual and bimanual training. Transfer of learned skills from bimanual training to unimanual movements was also observed, as bimanual training also improved single limb performance with the dominant arm. Changes of force symmetry did not have an effect on motor learning. As motor learning is believed to be an important mechanism of rehabilitation, our findings could be tested for future post-stroke rehabilitation systems.

Real-time closed-loop control of cognitive load in neurological patients during robot-assisted gait training.

Cognitively challenging training sessions during robot-assisted gait training after stroke were shown to be key requirements for the success of rehabilitation. Despite a broad variability of cognitive impairments amongst the stroke population, current rehabilitation environments do not adapt to the cognitive capabilities of the patient, as cognitive load cannot be objectively assessed in real-time. We provided healthy subjects and stroke patients with a virtual task during robot-assisted gait training, which allowed modulating cognitive load by adapting the difficulty level of the task. We quantified the cognitive load of stroke patients by using psychophysiological measurements and performance data. In open-loop experiments with healthy subjects and stroke patients, we obtained training data for a linear, adaptive classifier that estimated the current cognitive load of patients in real-time. We verified our classification results via questionnaires and obtained 88% correct classification in healthy subjects and 75% in patients. Using the pre-trained, adaptive classifier, we closed the cognitive control loop around healthy subjects and stroke patients by automatically adapting the difficulty level of the virtual task in real-time such that patients were neither cognitively overloaded nor under-challenged.

Psychophysiological measurements in a biocooperative feedback loop for upper extremity rehabilitation.

This paper examines the usefulness of psychophysiological measurements in a biocooperative feedback loop that adjusts the difficulty of an upper extremity rehabilitation task. Psychophysiological measurements (heart rate, skin conductance, respiration, and skin temperature) were used both by themselves and in combination with task performance and biomechanics. Data fusion was performed with discriminant analysis, and a special adaptive version was implemented that can gradually adapt to a subject. Both healthy subjects and hemiparetic patients participated in the study. The accuracy of the biocooperative controller was defined as the percentage of times it matched the subjects' preferences. The highest accuracy rate was obtained for task performance (approximately 82% for both healthy subjects and patients), with psychophysiological measurements yielding relatively low accuracy (approximately 60%). The adaptive approach increased accuracy of psychophysiological measurements to 76.4% for healthy subjects and 68.8% for patients. Combining psychophysiology with task performance yielded an accuracy rate of 84.7% for healthy subjects and 89.4% for patients. Results suggest that psychophysiological measurements are not reliable as a primary data source in motor rehabilitation, but can provide supplementary information. However, it is questionable whether the amount of additional information justifies the increased complexity of the system.

Challenges in biocooperative rehabilitation robotics.

Psychological states such as mood, motivation and engagement are known to be critical for the success of rehabilitation, and encouraging unmotivated stroke patients improves the likelihood of their eventual recovery. Psychological factors can be incorporated into the closed-loop control of biocooperative rehabilitation systems, augmenting the device with critical information about the patient state. However, in rehabilitation robotics, interpretation of psychophysiological measurements is made complex by the multi-task environment, the presence of strenuous physical activity and patient's damage to the central and autonomic nervous systems. The study examines these challenges and proposes possible solutions for implementation in biocooperative control of rehabilitation robots.

Psychophysiological responses to robot training in different recovery phases after stroke.

Psychophysiological responses have become a valuable tool in human-robot interaction since they provide an objective estimate of the user's psychological state. Unfortunately, their usefulness in rehabilitation robotics is uncertain since they are influenced by both physical activity and pathological conditions such as stroke. We performed psychophysiological measurements in subacute and chronic stroke patients as well as healthy controls during a reaching and grasping exercise task performed in a multimodal virtual environment. Furthermore, we evaluated the differences in kinematic and static parameters between the three groups of subjects. The results of the observed kinematic and static evaluation parameters showed significant differences when different assistive modes enabled the subject to focus on a particular function of the exercise, like reaching or grasping, or coordinated actions that combine reaching and grasping, reflecting the motor abilities of the individual. The analysis of psychophysiological responses suggests that both chronic and subacute stroke subjects have weaker psychophysiological responses than healthy subjects, though the responses of chronic patients have recovered somewhat. This certainly indicates that further studies are needed before psychophysiological responses can be used in clinical practice.

Task difficulty adjustment in biocooperative rehabilitation using psychophysiological responses.

This study presents a biocooperative feedback loop where the difficulty of an upper extremity rehabilitation task is adjusted based on four psychophysiological measurements: heart rate, skin conductance, respiration and skin temperature. They are used both by themselves and in combination with task performance and biomechanics. Different variants of linear discriminant analysis are used for data fusion, including a variant that can adjust the fusion rules online and thus gradually adapt to the subject. Both healthy subjects and hemiparetic patients participated in the study. The accuracy rate of the biocooperative controller was defined as the percentage of times it matched the subjects' preferences. Psychophysiological measurements yielded a relatively low accuracy rate by themselves (76.4% for healthy subjects and 68.2% for patients). Task performance, on the other hand, yielded an accuracy rate of approximately 82%. Combining task performance with psychophysiology increased the accuracy rate to 84.7% for healthy subjects and 89.4% for patients. Psychophysiology can thus provide additional information, but factors such as the increased cost and complexity of the system should also be taken into account.

River multimodal scenario for rehabilitation robotics.

This paper presents the novel "River" multimodal rehabilitation robotics scenario that includes video, audio and haptic modalities. Elements contributing to intrinsic motivation are carefully joined in the three modalities to increase motivation of the user. The user first needs to perform a motor action, then receives a cognitive challenge that is solved with adequate motor activity. Audio includes environmental sounds, music and spoken instructions or encouraging statements. Sounds and music were classified according to the arousal-valence space. The haptic modality can provide catching, grasping, tunnel or adaptive assistance, all depending on the user's needs. The scenario was evaluated in 16 stroke users, who responded to it favourably according to the Intrinsic Motivation Inventory questionnaire. Additionally, the river multimodal environment seems to elicit higher motivation than a simpler apple pick-and-place multimodal task.

Evaluation of upper extremity robot-assistances in subacute and chronic stroke subjects.

BACKGROUND: Robotic systems are becoming increasingly common in upper extremity stroke rehabilitation. Recent studies have already shown that the use of rehabilitation robots can improve recovery. This paper evaluates the effect of different modes of robot-assistances in a complex virtual environment on the subjects' ability to complete the task as well as on various haptic parameters arising from the human-robot interaction. METHODS: The MIMICS multimodal system that includes the haptic robot HapticMaster and a dynamic virtual environment is used. The goal of the task is to catch a ball that rolls down a sloped table and place it in a basket above the table. Our study examines the influence of catching assistance, pick-and-place movement assistance and grasping assistance on the catching efficiency, placing efficiency and on movement-dependent parameters: mean reaching forces, deviation error, mechanical work and correlation between the grasping force and the load force. RESULTS: The results with groups of subjects (23 subacute hemiparetic subjects, 10 chronic hemiparetic subjects and 23 control subjects) showed that the assistance raises the catching efficiency and pick-and-place efficiency. The pick-and-place movement assistance greatly limits the movements of the subject and results in decreased work toward the basket. The correlation between the load force and the grasping force exists in a certain phase of the movement. The results also showed that the stroke subjects without assistance and the control subjects performed similarly. CONCLUSIONS: The robot-assistances used in the study were found to be a possible way to raise the catching efficiency and efficiency of the pick-and-place movements in subacute and chronic subjects. The observed movement parameters showed that robot-assistances we used for our virtual task should be improved to maximize physical activity.

Psychophysiological responses to robotic rehabilitation tasks in stroke.

This paper presents the analysis of four psychophysiological responses in post-stroke upper extremity rehabilitation. The goal was to determine which psychophysiological responses would provide the most reliable information about subjects' psychological states during rehabilitation. Heart rate, skin conductance, respiration, and skin temperature were recorded in a stroke group and a control group during two difficulty levels of a pick-and-place task performed in a virtual environment using a haptic robot and during a cognitive task. Psychophysiological measurements were correlated with results of a self-report questionnaire. All four responses showed significant changes in response to the different tasks. Skin conductance differentiated between the two difficulty levels and was correlated with self-reported arousal in both stroke and control groups. Skin temperature differentiated between the two difficulty levels for the control group, but provided poor results for the stroke group. Heart rate and respiration increased during tasks, but their connection to psychological state was unclear. Results suggest that, of the four measured responses, skin conductance offers the most potential as a psychological state indicator, with other measures providing supplementary information. Psychophysiological measurements could thus be used in closed-loop biocooperative systems that would detect the user's psychological state and change the course of therapy accordingly.

ARMin: a robot for patient-cooperative arm therapy.

Task-oriented, repetitive and intensive arm training can enhance arm rehabilitation in patients with paralyzed upper extremities due to lesions of the central nervous system. There is evidence that the training duration is a key factor for the therapy progress. Robot-supported therapy can improve the rehabilitation allowing more intensive training. This paper presents the kinematics, the control and the therapy modes of the arm therapy robot ARMin. It is a haptic display with semi-exoskeleton kinematics with four active and two passive degrees of freedom. Equipped with position, force and torque sensors the device can deliver patient-cooperative arm therapy taking into account the activity of the patient and supporting him/her only as much as needed. The haptic display is combined with an audiovisual display that is used to present the movement and the movement task to the patient. It is assumed that the patient-cooperative therapy approach combined with a multimodal display can increase the patient's motivation and activity and, therefore, the therapeutic progress.

Unsupported standing with minimized ankle muscle fatigue.

In the past, limited unsupported standing has been restored in patients with thoracic spinal cord injury through open-loop functional electrical stimulation of paralyzed knee extensor muscles and the support of intact arm musculature. Here an optimal control system for paralyzed ankle muscles was designed that enables the subject to stand without hand support in a sagittal plane. The paraplegic subject was conceptualized as an underactuated double inverted pendulum structure with an active degree of freedom in the upper trunk and a passive degree of freedom in the paralyzed ankle joints. Control system design is based on the minimization of a cost function that estimates the effort of ankle joint muscles via observation of the ground reaction force position, relative to ankle joint axis. Furthermore, such a control system integrates voluntary upper trunk activity and artificial control of ankle joint muscles, resulting in a robust standing posture. Figures are shown for the initial simulation study, followed by disturbance tests on an intact volunteer and several laboratory trials with a paraplegic person. Benefits of the presented methodology are prolonged standing sessions and in the fact that the subject is able to maintain voluntary control over upper body orientation in space, enabling simple functional standing.