Robust whole-hand spatial manipulation via energy maps with caging, rolling, and sliding.

Humans regularly use all inner surfaces of the hand during manipulation, whereas traditional formulations for robots tend to use only the tips of their fingers, limiting overall dexterity. In this paper, we explore the use of the whole hand during spatial robotic dexterous within-hand manipulation. We present a novel four-fingered robotic hand called the Model B, which is designed and controlled using a straight-forward potential energy-based motion model that is based on the hand configuration and applied actuator torques. In this way the hand-object system is driven to a new desired configuration, often through sliding and rolling between the object and hand, and with the fingers "caging" the object to prevent ejection. This paper presents the first ever application of the energy model in three dimensions, which was used to compare the theoretical manipulability of popular robotic hands, which then inspired the design of the Model B. We experimentally validate the hand's performance with extensive benchtop experimentation with test objects and real world objects, as well as on a robotic arm, and demonstrate complex spatial caging manipulation on a variety of objects in all six object dimensions (three translation and three rotation) using all inner surfaces of the fingers and the palm.

Trajectory Control-An Effective Strategy for Controlling Multi-DOF Upper Limb Prosthetic Devices.

Despite great innovations in upper-extremity prosthetic hardware in recent decades, controlling a multiple joint upper limb prosthesis such as an elbow/wrist/hand system is still an open clinical challenge, in large part due to an insufficient number of control inputs available to users. While simultaneous control is in its early stages, the common control approach is sequential control, in which joints and grasps are driven one at a time. In this paper, we introduce and evaluate a concept we call trajectory control, that builds upon this approach, in which motions of the wrist, elbow, and shoulder DOFs (and subsets of them) are coupled into predefined sets of coordinated trajectories; to be selected by the user and driven with a single input variable. These trajectories were designed based on an earlier motion study of activities of daily life obtained from human demonstrations. We experimentally evaluate the efficacy of our approach through a human subjects study in which tasks are performed in a virtual environment. The results show that as device complexity increased (i.e. greater number of DOFs corresponding to more proximal amputations), participants were able to complete tasks faster with trajectory control while exhibiting similar levels of body compensation when compared to sequential and simultaneous control. Additionally, participants found trajectory control to be more intuitive and displayed more natural movement.

Complex manipulation with a simple robotic hand through contact breaking and caging.

Humans use all surfaces of the hand for contact-rich manipulation. Robot hands, in contrast, typically use only the fingertips, which can limit dexterity. In this work, we leveraged a potential energy-based whole-hand manipulation model, which does not depend on contact wrench modeling like traditional approaches, to design a robotic manipulator. Inspired by robotic caging grasps and the high levels of dexterity observed in human manipulation, a metric was developed and used in conjunction with the manipulation model to design a two-fingered dexterous hand, the Model W. This was accomplished by simulating all planar finger topologies composed of open kinematic chains of up to three serial revolute and prismatic joints, forming symmetric two-fingered hands, and evaluating their performance according to the metric. We present the best design, an unconventional robot hand capable of performing continuous object reorientation, as well as repeatedly alternating between power and pinch grasps-two contact-rich skills that have often eluded robotic hands-and we experimentally characterize the hand's manipulation capability. This hand realizes manipulation motions reminiscent of thumb-index finger manipulative movement in humans, and its topology provides the foundation for a general-purpose dexterous robot hand.

Manipulation for self-Identification, and self-Identification for better manipulation.

The process of modeling a series of hand-object parameters is crucial for precise and controllable robotic in-hand manipulation because it enables the mapping from the hand's actuation input to the object's motion to be obtained. Without assuming that most of these model parameters are known a priori or can be easily estimated by sensors, we focus on equipping robots with the ability to actively self-identify necessary model parameters using minimal sensing. Here, we derive algorithms, on the basis of the concept of virtual linkage-based representations (VLRs), to self-identify the underlying mechanics of hand-object systems via exploratory manipulation actions and probabilistic reasoning and, in turn, show that the self-identified VLR can enable the control of precise in-hand manipulation. To validate our framework, we instantiated the proposed system on a Yale Model O hand without joint encoders or tactile sensors. The passive adaptability of the underactuated hand greatly facilitates the self-identification process, because they naturally secure stable hand-object interactions during random exploration. Relying solely on an in-hand camera, our system can effectively self-identify the VLRs, even when some fingers are replaced with novel designs. In addition, we show in-hand manipulation applications of handwriting, marble maze playing, and cup stacking to demonstrate the effectiveness of the VLR in precise in-hand manipulation control.

Effect of Number of Digits on Human Precision Manipulation Workspaces.

Precision manipulation, or moving small objects held in the fingertips, is likely the most heavily utilized class of dexterous within-hand manipulation and adds greatly to the capabilities of the human hand. This article focuses on studying the effects of varying the number of digits used on the resulting manipulation abilities, in terms of translational workspaces and rotational ranges, by manipulating two circular objects, 50 mm and 80 mm in diameter. In general, as the number of digits in contact with the object increases, the results show a significant reduction in precision manipulation workspace range for four of the six translation and rotation directions and no significant change in the other two, suggesting that for these particular metrics, more fingers result in a reduction in performance. Furthermore, while two digits results in the largest workspaces for five of the six translation and rotation axes, the lack of ability to control rotation in the distal-proximal direction suggests that three digits may be more desirable for overall precision manipulation dexterity.

Dimensionality Reduction and Motion Clustering During Activities of Daily Living: Decoupling Hand Location and Orientation.

This article is the second in a two-part series analyzing human arm and hand motion during a wide range of unstructured tasks. In this work, we track the hand of healthy individuals as they perform a variety of activities of daily living (ADLs) in three ways decoupled from hand orientation: end-point locations of the hand trajectory, whole path trajectories of the hand, and straight-line paths generated using start and end points of the hand. These data are examined by a clustering procedure to reduce the wide range of hand use to a smaller representative set. Hand orientations are subsequently analyzed for the end-point location clustering results and subsets of orientations are identified in three reference frames: global, torso, and forearm. Data driven methods that are used include dynamic time warping (DTW), DTW barycenter averaging (DBA), and agglomerative hierarchical clustering with Ward's linkage. Analysis of the end-point locations, path trajectory, and straight-line path trajectory identified 5, 5, and 7 ADL task categories, respectively, while hand orientation analysis identified up to 4 subsets of orientations for each task location, discretized and classified to the facets of a rhombicuboctahedron. Together these provide insight into our hand usage in daily life and inform an implementation in prosthetic or robotic devices using sequential control.

Dimensionality Reduction and Motion Clustering During Activities of Daily Living: Three-, Four-, and Seven-Degree-of-Freedom Arm Movements.

This paper is the first in a two-part series analyzing human arm and hand motion during a wide range of unstructured tasks. The wide variety of motions performed by the human arm during daily tasks makes it desirable to find representative subsets to reduce the dimensionality of these movements for a variety of applications, including the design and control of robotic and prosthetic devices. This paper presents a novel method and the results of an extensive human subjects study to obtain representative arm joint angle trajectories that span naturalistic motions during Activities of Daily Living (ADLs). In particular, we seek to identify sets of useful motion trajectories of the upper limb that are functions of a single variable, allowing, for instance, an entire prosthetic or robotic arm to be controlled with a single input from a user, along with a means to select between motions for different tasks. Data driven approaches are used to discover clusters and representative motion averages for the wrist 3 degree of freedom (DOF), elbow-wrist 4 DOF, and full-arm 7 DOF motions. The proposed method makes use of well-known techniques such as dynamic time warping (DTW) to obtain a divergence measure between motion segments, Ward's distance criterion to build hierarchical trees, and functional principal component analysis (fPCA) to evaluate cluster variability. The emerging clusters associate various recorded motions into primarily hand start and end location for the full-arm system, motion direction for the wrist-only system, and an intermediate between the two qualities for the elbow-wrist system.

Towards Understanding Complex Human Dexterous Manipulation Strategies: Kinematics of Gaiting-based Object Rotations.

This paper presents a novel method for tracking gaiting-based (changing contacts, reciprocal, cyclical) withinhand manipulation strategies of a human hand. We present a kinematic model that relies on data collected from 6-DOF magnetic sensors attached to 7 external sites on the hand. The sensors are calibrated by three procedures-sensor-to-fingertip, constrained fingertip workspace limits, and flat hand configuration. Subjects rotated two cubes of different sizes around the 3 object-centric axes, while a synchronized camera recorded the object motion. Hand motions were segmented and then averaged using dynamic time warping (DTW) to yield a representative time-series motion primitive for the given task. The hand movements of two subjects during cube rotation tasks were reconstructed using a 22-degree of freedom (DOF) hand kinematic model. Based on a qualitative evaluation of the joint movements, intrasubject correlations of joint angles were found.

Using a Variable-Friction Robot Hand to Determine Proprioceptive Features for Object Classification During Within-Hand-Manipulation.

Interactions with an object during within-hand manipulation (WIHM) constitutes an assortment of gripping, sliding, and pivoting actions. In addition to manipulation benefits, the re-orientation and motion of the objects within-the-hand also provides a rich array of additional haptic information via the interactions to the sensory organs of the hand. In this article, we utilize variable friction (VF) robotic fingers to execute a rolling WIHM on a variety of objects, while recording 'proprioceptive' actuator data, which is then used for object classification (i.e., without tactile sensors). Rather than hand-picking a select group of features for this task, our approach begins with 66 general features, which are computed from actuator position and load profiles for each object-rolling manipulation, based on gradient changes. An Extra Trees classifier performs object classification while also ranking each feature's importance. Using only the six most-important 'Key Features' from the general set, a classification accuracy of 86% was achieved for distinguishing the six geometric objects included in our data set. Comparatively, when all 66 features are used, the accuracy is 89.8%.

Behavioral correlates of semi-zygodactyly in Ospreys (Pandion haliaetus) based on analysis of internet images.

Ospreys are renowned for their fishing abilities, which have largely been attributed to their specialized talon morphology and semi-zygodactyly-the ability to rotate the fourth toe to accompany the first toe in opposition of toes II and III. Anecdotal observations indicate that zygodactyly in Ospreys is associated with prey capture, although to our knowledge this has not been rigorously tested. As a first pass toward understanding the functional significance of semi-zygodactyly in Ospreys, we scoured the internet for images of Osprey feet in a variety of circumstances. From these we cross-tabulated the number of times each of three toe configurations (anisodactylous, zygodactylous, and an intermediate condition between these) was associated with different grasping scenarios (e.g., grasping prey or perched), contact conditions (e.g., fish, other objects, or substrate), object sizes (relative to foot size), and grasping behaviors (e.g., using one or both feet). Our analysis confirms an association between zygodactyly and grasping behavior; the odds that an osprey exhibited zygodactyly while grasping objects in flight were 5.7 times greater than whilst perched. Furthermore, the odds of zygodactyly during single-foot grasps were 4.1 times greater when pictured grasping fish compared to other objects. These results suggest a functional association between predatory behavior and zygodactyly and has implications for the selective role of predatory performance in the evolution of zygodactyly more generally.

Perching and resting-A paradigm for UAV maneuvering with modularized landing gears.

Perching helps small unmanned aerial vehicles (UAVs) extend their time of operation by saving battery power. However, most strategies for UAV perching require complex maneuvering and rely on specific structures, such as rough walls for attaching or tree branches for grasping. Many strategies to perching neglect the UAV's mission such that saving battery power interrupts the mission. We suggest enabling UAVs with the capability of making and stabilizing contacts with the environment, which will allow the UAV to consume less energy while retaining its altitude, in addition to the perching capability that has been proposed before. This new capability is termed "resting." For this, we propose a modularized and actuated landing gear framework that allows stabilizing the UAV on a wide range of different structures by perching and resting. Modularization allows our framework to adapt to specific structures for resting through rapid prototyping with additive manufacturing. Actuation allows switching between different modes of perching and resting during flight and additionally enables perching by grasping. Our results show that this framework can be used to perform UAV perching and resting on a set of common structures, such as street lights and edges or corners of buildings. We show that the design is effective in reducing power consumption, promotes increased pose stability, and preserves large vision ranges while perching or resting at heights. In addition, we discuss the potential applications facilitated by our design, as well as the potential issues to be addressed for deployment in practice.

Analyzing at-home prosthesis use in unilateral upper-limb amputees to inform treatment & device design.

New upper limb prosthetic devices are continuously being developed by a variety of industrial, academic, and hobbyist groups. Yet, little research has evaluated the long term use of currently available prostheses in daily life activities, beyond laboratory or survey studies. We seek to objectively measure how experienced unilateral upper limb prosthesis-users employ their prosthetic devices and unaffected limb for manipulation during everyday activities. In particular, our goal is to create a method for evaluating all types of amputee manipulation, including non-prehensile actions beyond conventional grasp functions, as well as to examine the relative use of both limbs in unilateral and bilateral cases. This study employs a head-mounted video camera to record participant's hands and arms as they complete unstructured domestic tasks within their own homes. A new 'Unilateral Prosthesis-User Manipulation Taxonomy' is presented based observations from 10 hours of recorded videos. The taxonomy addresses manipulation actions of the intact hand, prostheses, bilateral activities, and environmental feature-use (aiïordances). Our preliminary results involved tagging 23 minute segments of the full videos from 3 amputee participants using the taxonomy. This resulted in over 2,300 tag instances. Observations included that non-prehensile interactions outnumbered prehensile interactions in the affected limb for users with more distal amputation that allowed arm mobility.

Distance-based kinematics of the five-oblique-axis thumb model with intersecting axes at the metacarpophalangeal joint.

This paper proposes a novel and simple method to compute all possible solutions of the inverse kinematics problem of the five-oblique-axis thumb model with intersecting axes at the metacarpophalangeal joint. This thumb model is one of the suggested results by a magnetic-resonance-imaging-based study that, in contrast to those based on cadaver fingers or on the tracking of the surface of the fingers, takes into account muscle and ligament behaviors and avoids inaccuracies resulting from the movement of the skin with respect to the bones. The proposed distance-based inverse kinematics method eliminates the use of arbitrary reference frames as is usually required by standard approaches; this is relevant because the numerical conditioning of the resulting system of equations with such traditional approaches depends on the selected reference frames. Moreover, contrary to other parametrizations (e.g., Denavit-Hartenberg parameters), the suggested distance-based parameters for the thumb have a natural, human-understandable geometric meaning that makes them easier to be determined from any posture. These characteristics make the proposed approach of interest for those working in, for instance, measuring and modeling the movement of the human hand, developing rehabilitation devices such as orthoses and prostheses, or designing anthropomorphic robotic hands.

Design and Evaluation of Shape-Changing Haptic Interfaces for Pedestrian Navigation Assistance.

Shape-changing interfaces are a category of device capable of altering their form in order to facilitate communication of information. In this work, we present a shape-changing device that has been designed for navigation assistance. 'The Animotus' (previously, 'The Haptic Sandwich' ), resembles a cube with an articulated upper half that is able to rotate and extend (translate) relative to the bottom half, which is fixed in the user's grasp. This rotation and extension, generally felt via the user's fingers, is used to represent heading and proximity to navigational targets. The device is intended to provide an alternative to screen or audio based interfaces for visually impaired, hearing impaired, deafblind, and sighted pedestrians. The motivation and design of the haptic device is presented, followed by the results of a navigation experiment that aimed to determine the role of each device DOF, in terms of facilitating guidance. An additional device, 'The Haptic Taco', which modulated its volume in response to target proximity (negating directional feedback), was also compared. Results indicate that while the heading (rotational) DOF benefited motion efficiency, the proximity (translational) DOF benefited velocity. Combination of the two DOF improved overall performance. The volumetric Taco performed comparably to the Animotus' extension DOF.

Single-Grasp Object Classification and Feature Extraction with Simple Robot Hands and Tactile Sensors.

Classical robotic approaches to tactile object identification often involve rigid mechanical grippers, dense sensor arrays, and exploratory procedures (EPs). Though EPs are a natural method for humans to acquire object information, evidence also exists for meaningful tactile property inference from brief, non-exploratory motions (a 'haptic glance'). In this work, we implement tactile object identification and feature extraction techniques on data acquired during a single, unplanned grasp with a simple, underactuated robot hand equipped with inexpensive barometric pressure sensors. Our methodology utilizes two cooperating schemes based on an advanced machine learning technique (random forests) and parametric methods that estimate object properties. The available data is limited to actuator positions (one per two link finger) and force sensors values (eight per finger). The schemes are able to work both independently and collaboratively, depending on the task scenario. When collaborating, the results of each method contribute to the other, improving the overall result in a synergistic fashion. Unlike prior work, the proposed approach does not require object exploration, re-grasping, grasp-release, or force modulation and works for arbitrary object start positions and orientations. Due to these factors, the technique may be integrated into practical robotic grasping scenarios without adding time or manipulation overheads.

Investigation of a passive capstan based grasp enhancement feature in a voluntary-closing prosthetic terminal device.

Body-powered prosthetic terminal devices fall into two main categories: voluntary-closing devices, which require the user to exert a force to maintain a grasp, and voluntary opening devices, which generally utilize springs to close and maintain a force. As a result, voluntary-closing devices often have a locking feature that allows the user to relax and transport objects while maintaining a firm grip. In this paper, we examine a new type of capstan-based passive brake mechanism in a voluntary-closing prosthetic terminal device. Three different mechanisms were compared on the benchtop and with human subjects: the passive capstan grasp enhancement, a "pull-to-lock, pull-to-release" mechanism, and a manual cable locking mechanism. Standard tests of prosthetic device dexterity, including the Box and Blocks test and Southampton Hand Assessment Protocol, were performed with an instrumented prosthesis socket simulator with each device. While results are similar across the three mechanisms, the passive capstan mechanism does not require a physical user input to engage or disengage the lock, adding a benefit over the existing mechanisms.

Mechanical analysis of avian feet: multiarticular muscles in grasping and perching.

The grasping capability of birds' feet is a hallmark of their evolution, but the mechanics of avian foot function are not well understood. Two evolutionary trends that contribute to the mechanical complexity of the avian foot are the variation in the relative lengths of the phalanges and the subdivision and variation of the digital flexor musculature observed among taxa. We modelled the grasping behaviour of a simplified bird foot in response to the downward and upward forces imparted by carrying and perching tasks, respectively. Specifically, we compared the performance of various foot geometries performing these tasks when actuated by distally inserted flexors only, versus by both distally inserted and proximally inserted flexors. Our analysis demonstrates that most species possess relative phalanx lengths that are conducive to grasps actuated only by a single distally inserted tendon per digit. Furthermore, proximally inserted flexors are often required during perching, but the distally inserted flexors are sufficient when grasping and carrying objects. These results are reflected in differences in the relative development of proximally and distally inserted digital flexor musculature among 'perching' and 'grasping' taxa. Thus, our results shed light on the relative roles of variation in phalanx length and digit flexor muscle distribution in an integrative, mechanical context.

Biomechanical Effects of Stiffness in Parallel With the Knee Joint During Walking.

The human knee behaves similarly to a linear torsional spring during the stance phase of walking with a stiffness referred to as the knee quasi-stiffness. The spring-like behavior of the knee joint led us to hypothesize that we might partially replace the knee joint contribution during stance by utilizing an external spring acting in parallel with the knee joint. We investigated the validity of this hypothesis using a pair of experimental robotic knee exoskeletons that provided an external stiffness in parallel with the knee joints in the stance phase. We conducted a series of experiments involving walking with the exoskeletons with four levels of stiffness, including 0%, 33%, 66%, and 100% of the estimated human knee quasi-stiffness, and a pair of joint-less replicas. The results indicated that the ankle and hip joints tend to retain relatively invariant moment and angle patterns under the effects of the exoskeleton mass, articulation, and stiffness. The results also showed that the knee joint responds in a way such that the moment and quasi-stiffness of the knee complex (knee joint and exoskeleton) remains mostly invariant. A careful analysis of the knee moment profile indicated that the knee moment could fully adapt to the assistive moment; whereas, the knee quasi-stiffness fully adapts to values of the assistive stiffness only up to ∼80%. Above this value, we found biarticular consequences emerge at the hip joint.

Workspace Shape and Characteristics for Human Two- and Three-Fingered Precision Manipulation.

GOAL: To study precision manipulation, which involves repositioning an object in the fingertips and is used in everyday tasks such as writing and key insertion, and also for domain-specific tasks such as small scalpel cuts, using tweezers, and hand soldering. METHODS: In this study, the range of positions (workspace) through which 19 participants manipulated a 3.3-4.1 cm-diameter object are measured with a magnetic tracker. Each participant performed two conditions: a two-finger thumb-index finger condition and a three-finger thumb-index-middle finger condition. RESULTS: The observed workspaces, normalized to a 17.5 cm hand length, are small compared to free-finger trajectories; for the two-finger trials, 68% of points are within 1.05 cm of the centroid and 95% are within 2.31 cm, while the three-finger case shows a narrower distribution, with 68% of points within 0.94 cm of the centroid and 95% of points within 2.19 cm. The longest axis is a long thin arc in the proximal-palmar plane. Analysis of fingertip workspaces shows that the index fingertip workspace volume is the most linear predictor of object workspace (R(2) = 0.98). CONCLUSION: Precision manipulation workspace size and shape is shown, along with how the fingers are used during the manipulation. SIGNIFICANCE: The results have many applications, including normative data for rehabilitation, guidelines for ergonomic device design, and benchmarking prosthetic and robotic hands.

Estimating thumb-index finger precision grip and manipulation potential in extant and fossil primates.

Primates, and particularly humans, are characterized by superior manual dexterity compared with other mammals. However, drawing the biomechanical link between hand morphology/behaviour and functional capabilities in non-human primates and fossil taxa has been challenging. We present a kinematic model of thumb-index precision grip and manipulative movement based on bony hand morphology in a broad sample of extant primates and fossil hominins. The model reveals that both joint mobility and digit proportions (scaled to hand size) are critical for determining precision grip and manipulation potential, but that having either a long thumb or great joint mobility alone does not necessarily yield high precision manipulation. The results suggest even the oldest available fossil hominins may have shared comparable precision grip manipulation with modern humans. In particular, the predicted human-like precision manipulation of Australopithecus afarensis, approximately one million years before the first stone tools, supports controversial archaeological evidence of tool-use in this taxon.