Performance measures to benchmark the grasping, manipulation, and assembly of deformable objects typical to manufacturing applications.

The National Institute of Standards and Technology is developing performance tests and associated artifacts to benchmark research in the area of robotic assembly. Sets of components consistent with mechanical assemblies including screws, gears, electrical connectors, wires, and belts are configured for assembly or disassembly using a task board concept. Test protocols accompany the task boards and are designed to mimic low-volume, high-mixture assembly challenges typical to small and medium sized manufacturers. In addition to the typical rigid components found in assembled products, the task boards include many non-rigid component operations representative of wire harness and belt drive assemblies to support research in the area of grasping and manipulation of deformable objects, an area still considered to be an emerging research problem in robotics. A set of four primary task boards as well as competition task boards are presented as benchmarks along with scoring metrics and a method to compare robot system assembly times with human performance. Competitions are used to raise awareness to these benchmarks. Tools to progress and compare research are described along with emphasis placed on system competition-based solutions to grasp and manipulate deformable task board components.

Benchmarking Protocols for Evaluating Grasp Strength, Grasp Cycle Time, Finger Strength, and Finger Repeatability of Robot End-effectors.

This paper describes a set of metrics and supporting benchmarking protocols for determining the performance characteristics of robot end-effectors. In the short-term, these tools are proving useful as a common ground for assessing and comparing end-effectors. The long-term goal is a standard framework for providing technical specifications for robotic end-effectors to help pair technologies to application spaces. This paper presents a subset of the metrics - grasp strength, grasp cycle time, finger strength, and finger repeatability - with accompanying measurement techniques and supporting test artifacts. The application of these metrics and protocols is demonstrated using example implementations to characterize a variety of robot end-effectors, with example data sets and test designs provided for downloading.

Benchmarking Protocols for Evaluating Small Parts Robotic Assembly Systems.

This paper presents a set of performance metrics, test methods, and associated artifacts to help progress the development and deployment of robotic assembly systems. The designs for three task board artifacts that replicate small part insertion and fastening operations such as threading, snap fitting, and meshing with standard screws, nuts, washers, gears, electrical connectors, belt drives, and wiring are presented. To support the evaluation of robotic assembly and disassembly operations, benchmarking protocols and performance metrics are presented that leverage these task boards. Finally, robot competitions are discussed as use cases for these task boards.

Efficiently Improving and Quantifying Robot Accuracy In Situ.

The advancement of simulation-assisted robot programming, automation of high-tolerance assembly operations, and improvement of real-world performance engender a need for positionally accurate robots. Despite tight machining tolerances, good mechanical design, and careful assembly, robotic arms typically exhibit average Cartesian positioning errors of several millimeters. Fortunately, the vast majority of this error can be removed in software by proper calibration of the so-called "zero-offsets" of a robot's joints. This research developed an automated, inexpensive, highly portable, in situ calibration method that fine tunes these kinematic parameters, thereby, improving a robot's average positioning accuracy four-fold throughout its workspace. In particular, a prospective low-cost motion capture system and a benchmark laser tracker were used as reference sensors for robot calibration. Bayesian inference produced optimized zero-offset parameters alongside their uncertainty for data from both reference sensors. Relative and absolute accuracy metrics were proposed and applied for quantifying robot positioning accuracy. Uncertainty analysis of a validated, probabilistic robot model quantified the absolute positioning accuracy throughout its entire workspace. Altogether, three measures of accuracy conclusively revealed multi-fold improvement in the positioning accuracy of the robotic arm. Bayesian inference on motion capture data yielded zero-offsets and accuracy calculations comparable to those derived from laser tracker data, ultimately proving this method's viability towards robot calibration.

Multi-Robot Assembly Strategies and Metrics.

We present a survey of multi-robot assembly applications and methods and describe trends and general insights into the multi-robot assembly problem for industrial applications. We focus on fixtureless assembly strategies featuring two or more robotic systems. Such robotic systems include industrial robot arms, dexterous robotic hands, and autonomous mobile platforms, such as automated guided vehicles. In this survey, we identify the types of assemblies that are enabled by utilizing multiple robots, the algorithms that synchronize the motions of the robots to complete the assembly operations, and the metrics used to assess the quality and performance of the assemblies.

Comparative Peg-in-Hole Testing of a Force-Based Manipulation Controlled Robotic Hand.

Force-based manipulation control strategies are evolving as a primary mechanism in robotics for performing the fine manipulation tasks typical within manufacturing assembly. The ability to systematically compare robotic system performance and quantify true advancement in fine manipulation is of utmost importance. Accordingly, the objectives of this paper are threefold: 1) creation of a peg-in-hole test method with associated performance metrics and a systematic data analysis strategy for performance benchmarking, 2) first demonstration of a recently developed manipulation controller piloting a robotic hand and its paired task-level logic for completing the peg-in-hole test, and 3) exemplifying the performance benchmarking technique by comparing two approaches for robotic insertions-the previously mentioned compliant hand, stiff arm system, and a stiff gripper, compliant arm system. Analyses reveal that the unconventional hand system can perform at and sometimes above the level of the gripper system in the developed peg-in-hole scenario. Moreover, the hand's active control of the peg's full Cartesian pose reduces positional error sensitivity and minimizes exerted insertion forces, highlighting the strategy's potential for fine manipulation tasks. Results indicate that robotic arms equipped with highly articulated and sensorized robotic hands can provide a truly realizable solution path for performing peg-in-hole tasks.