Neuromorphic vision and tactile fusion for upper limb prosthesis control.

A major issue with upper limb prostheses is the disconnect between sensory information perceived by the user and the information perceived by the prosthesis. Advances in prosthetic technology introduced tactile information for monitoring grasping activity, but visual information, a vital component in the human sensory system, is still not fully utilized as a form of feedback to the prosthesis. For able-bodied individuals, many of the decisions for grasping or manipulating an object, such as hand orientation and aperture, are made based on visual information before contact with the object. We show that inclusion of neuromorphic visual information, combined with tactile feedback, improves the ability and efficiency of both able-bodied and amputee subjects to pick up and manipulate everyday objects. We discovered that combining both visual and tactile information in a real-time closed loop feedback strategy generally decreased the completion time of a task involving picking up and manipulating objects compared to using a single modality for feedback. While the full benefit of the combined feedback was partially obscured by experimental inaccuracies of the visual classification system, we demonstrate that this fusion of neuromorphic signals from visual and tactile sensors can provide valuable feedback to a prosthetic arm for enhancing real-time function and usability.

A Saccade Based Framework for Real-Time Motion Segmentation Using Event Based Vision Sensors.

Motion segmentation is a critical pre-processing step for autonomous robotic systems to facilitate tracking of moving objects in cluttered environments. Event based sensors are low power analog devices that represent a scene by means of asynchronous information updates of only the dynamic details at high temporal resolution and, hence, require significantly less calculations. However, motion segmentation using spatiotemporal data is a challenging task due to data asynchrony. Prior approaches for object tracking using neuromorphic sensors perform well while the sensor is static or a known model of the object to be followed is available. To address these limitations, in this paper we develop a technique for generalized motion segmentation based on spatial statistics across time frames. First, we create micromotion on the platform to facilitate the separation of static and dynamic elements of a scene, inspired by human saccadic eye movements. Second, we introduce the concept of spike-groups as a methodology to partition spatio-temporal event groups, which facilitates computation of scene statistics and characterize objects in it. Experimental results show that our algorithm is able to classify dynamic objects with a moving camera with maximum accuracy of 92%.

Stable force-myographic control of a prosthetic hand using incremental learning.

Force myography has been proposed as an appealing alternative to electromyography for control of upper limb prosthesis. A limitation of this technique is the non-stationary nature of the recorded force data. Force patterns vary under influence of various factors such as change in orientation and position of the prosthesis. We hereby propose an incremental learning method to overcome this limitation. We use an online sequential extreme learning machine where occasional updates allow continual adaptation to signal changes. The applicability and effectiveness of this approach is demonstrated for predicting the hand status from forearm muscle forces at various arm positions. The results show that incremental updates are indeed effective to maintain a stable level of performance, achieving an average classification accuracy of 98.75% for two subjects.