A GPU-Accelerated Model-Based Tracker for Untethered Submillimeter Grippers.

Miniaturized grippers that possess an untethered structure are suitable for a wide range of tasks, ranging from micromanipulation and microassembly to minimally invasive surgical interventions. In order to robustly perform such tasks, it is critical to properly estimate their overall configuration. Previous studies on tracking and control of miniaturized agents estimated mainly their 2D pixel position, mostly using cameras and optical images as a feedback modality. This paper presents a novel solution to the problem of estimating and tracking the 3D position, orientation and configuration of the tips of submillimeter grippers from marker-less visual observations. We consider this as an optimization problem, which is solved using a variant of the Particle Swarm Optimization algorithm. The proposed approach has been implemented in a Graphics Processing Unit (GPU) which allows a user to track the submillimeter agents online. The proposed approach has been evaluated on several image sequences obtained from a camera and on B-mode ultrasound images obtained from an ultrasound probe. The sequences show the grippers moving, rotating, opening/closing and grasping biological material. Qualitative results obtained using both hydrogel (soft) and metallic (hard) grippers with different shapes and sizes ranging from 750 microns to 4 mm (tip to tip), demonstrate the capability of the proposed method to track the agent in all the video sequences. Quantitative results obtained by processing synthetic data reveal a tracking position error of 25 ± 7 µm and orientation error of 1.7 ± 1.3 degrees. We believe that the proposed technique can be applied to different stimuli responsive miniaturized agents, allowing the user to estimate the full configuration of complex agents from visual marker-less observations.

Design, characterization and control of thermally-responsive and magnetically-actuated micro-grippers at the air-water interface.

The design and control of untethered microrobotic agents has drawn a lot of attention in recent years. This technology truly possesses the potential to revolutionize the field of minimally invasive surgery and microassembly. However, miniaturization and reliable actuation of micro-fabricated grippers are still challenging at sub-millimeter scale. In this study, we design, manufacture, characterize, and control four similarly-structured semi-rigid thermoresponsive micro-grippers. Furthermore, we develop a closed loop-control algorithm to demonstrate and compare the performance of the said grippers when moving in hard-to-reach and unpredictable environments. Finally, we analyze the grasping characteristics of three of the presented designs. Overall, not only does the study demonstrate motion control in unstructured dynamic environments-at velocities up to 3.4, 2.9, 3.3, and 1 body-lengths/s with 980, 750, 250, and 100 µm-sized grippers, respectively-but it also aims to provide quantitative data and considerations to help a targeted design of magnetically-controlled thin micro-grippers.

Autonomous planning and control of soft untethered grippers in unstructured environments.

The use of small, maneuverable, untethered and reconfigurable robots could provide numerous advantages in various micromanipulation tasks. Examples include microassembly, pick-and-place of fragile micro-objects for lab-on-a-chip applications, assisted hatching for in-vitro fertilization and minimally invasive surgery. This study assesses the potential of soft untethered magnetic grippers as alternatives or complements to conventional tethered or rigid micromanipulators. We demonstrate closed-loop control of untethered grippers and automated pick-and-place of biological material on porcine tissue in an unstructured environment. We also demonstrate the ability of the soft grippers to recognize and sort non-biological micro-scale objects. The fully autonomous nature of the experiments is made possible by the integration of planning and decision-making algorithms, as well as by closed-loop temperature and electromagnetic motion control. The grippers are capable of completing pick-and-place tasks of biological material at an average velocity of 1.8 ±0.71 mm/s and a drop-off error of 0.62 ±0.22 mm. Color-sensitive sorting of three micro-scale objects is completed at a velocity of 1.21 ±0.68 mm/s and a drop-off error of 0.85 ±0.41 mm. Our findings suggest that improved autonomous untethered grippers could augment the capabilities of current soft-robotic instruments especially in advancedtasks involving manipulation.

Magnetic Motion Control and Planning of Untethered Soft Grippers using Ultrasound Image Feedback.

Soft miniaturized untethered grippers can be used to manipulate and transport biological material in unstructured and tortuous environments. Previous studies on control of soft miniaturized grippers employed cameras and optical images as a feedback modality. However, the use of cameras might be unsuitable for localizing miniaturized agents that navigate within the human body. In this paper, we demonstrate the wireless magnetic motion control and planning of soft untethered grippers using feedback extracted from B-mode ultrasound images. Results show that our system employing ultrasound images can be used to control the miniaturized grippers with an average tracking error of 0.4±0.13 mm without payload and 0.36±0.05 mm when the agent performs a transportation task with a payload. The proposed ultrasound feedback magnetic control system demonstrates the ability to control miniaturized grippers in situations where visual feedback cannot be provided via cameras.

Stimuli-Responsive Soft Untethered Grippers for Drug Delivery and Robotic Surgery.

Untethered microtools that can be precisely navigated into deep in vivo locations are important for clinical procedures pertinent to minimally invasive surgery and targeted drug delivery. In this mini-review, untethered soft grippers are discussed, with an emphasis on a class of autonomous stimuli-responsive gripping soft tools that can be used to excise tissues and release drugs in a controlled manner. The grippers are composed of polymers and hydrogels and are thus compliant to soft tissues. They can be navigated using magnetic fields and controlled by robotic path-planning strategies to carry out tasks like pick-and-place of microspheres and biological materials either with user assistance, or in a fully autonomous manner. It is envisioned that the use of these untethered soft grippers will translate from laboratory experiments to clinical scenarios and the challenges that need to be overcome to make this transition are discussed.

Human-robot formation control via visual and vibrotactile haptic feedback.

In this paper we present a new visuo-haptic interaction mechanism for human-robot formation control. The formation setup consists of a human leader and multiple follower robots. The mobile robots are equipped only with RGB-D cameras, and they should maintain a desired distance and orientation to the leader at all times. Mechanical limitations common to all the robots limit the possible trajectories that the human can take. In this regard, vibrotactile feedback provided by a haptic bracelet guides the human along trajectories that are feasible for the team by warning her/him when the formation constraints are being violated. Psychophysical tests on the bracelet together with real-world experiments conducted with a team of Pioneer robots show the effectiveness of the proposed visuo-haptic paradigm for the coordination of mixed human-robot teams.