Collaborative hunting in artificial agents with deep reinforcement learning.

Collaborative hunting, in which predators play different and complementary roles to capture prey, has been traditionally believed to be an advanced hunting strategy requiring large brains that involve high-level cognition. However, recent findings that collaborative hunting has also been documented in smaller-brained vertebrates have placed this previous belief under strain. Here, using computational multi-agent simulations based on deep reinforcement learning, we demonstrate that decisions underlying collaborative hunts do not necessarily rely on sophisticated cognitive processes. We found that apparently elaborate coordination can be achieved through a relatively simple decision process of mapping between states and actions related to distance-dependent internal representations formed by prior experience. Furthermore, we confirmed that this decision rule of predators is robust against unknown prey controlled by humans. Our computational ecological results emphasize that collaborative hunting can emerge in various intra- and inter-specific interactions in nature, and provide insights into the evolution of sociality.

From wolves to ants, many animals are known to be able to hunt as a team. This strategy may yield several advantages: going after bigger preys together, for example, can often result in individuals spending less energy and accessing larger food portions than when hunting alone. However, it remains unclear whether this behavior relies on complex cognitive processes, such as the ability for an animal to represent and anticipate the actions of its teammates. It is often thought that 'collaborative hunting' may require such skills, as this form of group hunting involves animals taking on distinct, tightly coordinated roles ­ as opposed to simply engaging in the same actions simultaneously. To better understand whether high-level cognitive skills are required for collaborative hunting, Tsutsui et al. used a type of artificial intelligence known as deep reinforcement learning. This allowed them to develop a computational model in which a small number of 'agents' had the opportunity to 'learn' whether and how to work together to catch a 'prey' under various conditions. To do so, the agents were only equipped with the ability to link distinct stimuli together, such as an event and a reward; this is similar to associative learning, a cognitive process which is widespread amongst animal species. The model showed that the challenge of capturing the prey when hunting alone, and the reward of sharing food after a successful hunt drove the agents to learn how to work together, with previous experiences shaping decisions made during subsequent hunts. Importantly, the predators started to exhibit the ability to take on distinct, complementary roles reminiscent of those observed during collaborative hunting, such as one agent chasing the prey while another ambushes it. Overall, the work by Tsutsui et al. challenges the traditional view that only organisms equipped with high-level cognitive processes can show refined collaborative approaches to hunting, opening the possibility that these behaviors may be more widespread than originally thought ­ including between animals of different species.

Flexible prediction of opponent motion with internal representation in interception behavior.

Skilled interception behavior often relies on accurate predictions of external objects because of a large delay in our sensorimotor systems. To deal with the sensorimotor delay, the brain predicts future states of the target based on the current state available, but it is still debated whether internal representations acquired from prior experience are used as well. Here we estimated the predictive manner by analyzing the response behavior of a pursuer to a sudden directional change of the evasive target, providing strong evidence that prediction of target motion by the pursuer was incompatible with a linear extrapolation based solely on the current state of the target. Moreover, using neural network models, we validated that nonlinear extrapolation as estimated was computationally feasible and useful even against unknown opponents. These results support the use of internal representations in predicting target motion, suggesting the usefulness and versatility of predicting external object motion through internal representations.

Human Navigational Strategy for Intercepting an Erratically Moving Target in Chase and Escape Interactions.

Pursuit and interception of moving targets are fundamental skills of many animal species. Although previous studies in human interception behaviors have proposed several navigational strategies for intercepting a moving target, it is still unknown which navigational strategy humans use in chase-and-escape interactions. In the present experimental study, by using two one-on-one tasks as seen in ball sports, we showed that human interception behaviors were statistically consistent with a time-optimal model. Our results provide the insight about the navigational strategy for intercepting a moving target in chase-and-escape interactions, which may be common across species.

Spatiotemporal characteristics of an attacker's strategy to pass a defender effectively in a computer-based one-on-one task.

For modern humans, chase-and-escape behaviors are fundamental skills in many sports. A critical factor related to the success or failure of chase-and-escape is the visuomotor delay. Recent studies on sensorimotor decision making have shown that humans can incorporate their own visuomotor delay into their decisions. However, the relationship between the decision of an attacker and the visuomotor delay of a defender is still unknown. Here, we conducted a one-on-one chase-and-escape task for humans and investigated the characteristics of the direction changes of the attacker and the responses of the defender. Our results showed that the direction change of the attacker has two characteristics: uniformity of spatial distribution and bimodality of temporal distribution. In addition, we showed that the response of the defender did not depend on the position but it was delayed to the direction change of the attacker with a short interval. These results suggest that the characteristics of direction change of an attacker increased unpredictability, and it could be useful for preventing the predictive response of the defender and to receive the benefit of an extra response delay of tens of milliseconds, respectively.

Underlying structure in the dynamics of chase and escape interactions.

Chase and escape behaviors are important skills in many sports. Previous studies have described the behaviors of the attacker (escaper) and defender (chaser) by focusing on their positional relationship and have presented several key parameters that affect the outcome (successful attack or defense). However, it remains unclear how each individual agent moves, and how the outcome is determined in this type of interaction. To address these questions, we constructed a chase and escape task in a virtual space that allowed us to manipulate agents' kinematic parameters. We identified the basic strategies of each agent and their robustness to changes in their parameters. Moreover, we identified the determinants of the outcome and a geometrical explanation of their importance. Our results revealed the underlying structure of a simplified human chase and escape interaction and provided the insight that, although each agent apparently moves freely, their strategies in two-agent interactions are in fact rather constrained.