Real-time Tracking of Handheld Object from Color or Depth Images.

Object tracking during rehabilitation could help a therapist to evaluate a patient's movement and progress. Hence, we present an image-based method for real-time tracking of handheld objects due to its ease of use and availability of color or depth cameras. We use an efficient projective point correspondence method and generalize the use of precomputed spare viewpoint information to allow real-time tracking of a rigid object. The method runs at more than 30 fps on a CPU while achieving submillimeter accuracy on synthetic datasets and robust tracking on a semi-synthetic dataset.Clinical relevance Real-time, accurate, and robust tracking of an object using an image-based method is a promising tool for rehabilitation applications as it is practical for clinical settings.

Wireless Pressure Sensor Array Module for Sensorized Object.

To complement rehabilitation assessments that involve hand-object interaction with additional information on the grasping parameters, we sensorized an object with a pressure sensor array module that can generate a pressure distribution map. The module can be customized for cylindrical and cuboid objects with up to 1024 sensing elements and it supports the efficient transfer of data wirelessly at more than 30 Hz. Although the module uses inexpensive materials, it is sensitive to changes in pressure distribution. It can also depict the shape of various objects with reasonable details as shown in the small errors for object pose estimation and high accuracy scores for hand grasp classification. The module's modular design and wireless functionality help to simplify integration with existing objects to create a smart sensing surface.Clinical relevance The resulting pressure distribution map allows the therapist to analyze grasping parameters that cannot be determined from visual observations alone.

Autonomous modeling of repetitive movement for rehabilitation exercise monitoring.

BACKGROUND: Insightful feedback generation for daily home-based stroke rehabilitation is currently unavailable due to the inefficiency of exercise inspection done by therapists. We aim to produce a compact anomaly representation that allows a therapist to pay attention to only a few specific sections in a long exercise session record and boost their efficiency in feedback generation. METHODS: This study proposes a data-driven technique to model a repetitive exercise using unsupervised phase learning on an artificial neural network and statistical learning on principal component analysis (PCA). After a model is built on a set of normal healthy movements, the model can be used to extract a sequence of anomaly scores from a movement of the same prescription. RESULTS: The method not only works on a standard marker-based motion capture system but also performs well on a more compact and affordable motion capture system based-on Kinect V2 and wrist-worn inertial measurement units that can be used at home. An evaluation of four different exercises shows its potential in separating anomalous movements from normal ones with an average area under the curve (AUC) of 0.9872 even on the compact motion capture system. CONCLUSIONS: The proposed processing technique has the potential to help clinicians in providing high-quality feedback for telerehabilitation in a more scalable way.

Camera-based Hand Tracking using a Mirror-based Multi-view Setup.

Current clinical practice of measuring hand joint range of motion relies on a goniometer as it is inexpensive, portable, and easy to use, but it can only measure the static angle of a single joint at a time. To measure dynamic hand motion, a camera-based system that can perform markerless hand pose estimation is attractive, as the system is ubiquitous, low-cost, and non-contact. However, camera-based systems require line-of-sight, and tracking accuracy degrades when the joint is occluded from the camera view. Thus, we propose a multi-view setup using a readily available color camera from a single mobile phone, and plane mirrors to create multiple views of the hand. This setup eliminates the complexity of synchronizing multiple cameras and reduce the issue of occlusion. Experimental results show that the multi-view setup could help to reduce the error in measuring the flexion angle of finger joints. Dynamic hand pose estimation with object interaction is also demonstrated.

Hand and Object Segmentation from Depth Image using Fully Convolutional Network.

Semantic segmentation is an important step for hand and object tracking as subsequent tracking algorithms depend heavily on the accuracy of the segmented hand and object. However, current methods for hand and object segmentation are limited in the number of semantic labels, and lack of a large scale annotated dataset to train an end-to-end deep neural network for semantic segmentation. Thus, in this work, we present a framework for generating a publicly available synthetic dataset, that is targeted for upper limb rehabilitation involving hand-object interaction and uses it to train our proposed deep neural network. Experimental results show that even though the network is trained on synthetic depth images, it is able to achieve a mean intersection over union (mIoU) of 70.4% when tested on real depth images. Furthermore, the inference time of the proposed network takes around 6 ms on a GPU, thus making it suitable for real-time applications.

Automatic Inference of Rat's Hindlimb Trajectory to Synchronize with Forelimb Gait Through Phase.

Synchronous forelimb-hindlimb gait pattern is important to facilitate natural walking behavior of an injured rat with total transection. Since our ultimate research goal is to build a rehabilitation robotic system to simulate the natural walking pattern for spinalized rats, this research aims to address an immediate goal of automating the inference of the rat's hindlimb trajectory from its own forelimb movement. Our proposed method uses unsupervised learning to extract independent forelimb and hinblimb phases. From the phase information, a relationship between forelimb and hindlimb trajectory can then be calculated. Results show that the proposed method has the potential to be used in a rehabilitation robotic system.