Learning to cooperate for low-Reynolds-number swimming: a model problem for gait coordination.

Biological microswimmers can coordinate their motions to exploit their fluid environment-and each other-to achieve global advantages in their locomotory performance. These cooperative locomotion require delicate adjustments of both individual swimming gaits and spatial arrangements of the swimmers. Here we probe the emergence of such cooperative behaviors among artificial microswimmers endowed with artificial intelligence. We present the first use of a deep reinforcement learning approach to empower the cooperative locomotion of a pair of reconfigurable microswimmers. The AI-advised cooperative policy comprises two stages: an approach stage where the swimmers get in close proximity to fully exploit hydrodynamic interactions, followed a synchronization stage where the swimmers synchronize their locomotory gaits to maximize their overall net propulsion. The synchronized motions allow the swimmer pair to move together coherently with an enhanced locomotion performance unattainable by a single swimmer alone. Our work constitutes a first step toward uncovering intriguing cooperative behaviors of smart artificial microswimmers, demonstrating the vast potential of reinforcement learning towards intelligent autonomous manipulations of multiple microswimmers for their future biomedical and environmental applications.

A 3D-Printed Self-Learning Three-Linked-Sphere Robot for Autonomous Confined-Space Navigation.

Reinforcement learning control methods can impart robots with the ability to discover effective behavior, reducing their modeling and sensing requirements, and enabling their ability to adapt to environmental changes. However, it remains challenging for a robot to achieve navigation in confined and dynamic environments, which are characteristic of a broad range of biomedical applications, such as endoscopy with ingestible electronics. Herein, a compact, 3D-printed three-linked-sphere robot synergistically integrated with a reinforcement learning algorithm that can perform adaptable, autonomous crawling in a confined channel is demonstrated. The scalable robot consists of three equally sized spheres that are linearly coupled, in which the extension and contraction in specific sequences dictate its navigation. The ability to achieve bidirectional locomotion across frictional surfaces in open and confined spaces without prior knowledge of the environment is also demonstrated. The synergistic integration of a highly scalable robotic apparatus and the model-free reinforcement learning control strategy can enable autonomous navigation in a broad range of dynamic and confined environments. This capability can enable sensing, imaging, and surgical processes in previously inaccessible confined environments in the human body.