How a Minimal Learning Agent can Infer the Existence of Unobserved Variables in a Complex Environment.

According to a mainstream position in contemporary cognitive science and philosophy, the use of abstract compositional concepts is amongst the most characteristic indicators of meaningful deliberative thought in an organism or agent. In this article, we show how the ability to develop and utilise abstract conceptual structures can be achieved by a particular kind of learning agent. More specifically, we provide and motivate a concrete operational definition of what it means for these agents to be in possession of abstract concepts, before presenting an explicit example of a minimal architecture that supports this capability. We then proceed to demonstrate how the existence of abstract conceptual structures can be operationally useful in the process of employing previously acquired knowledge in the face of new experiences, thereby vindicating the natural conjecture that the cognitive functions of abstraction and generalisation are closely related.

Honeybee communication during collective defence is shaped by predation.

BACKGROUND: Social insect colonies routinely face large vertebrate predators, against which they need to mount a collective defence. To do so, honeybees use an alarm pheromone that recruits nearby bees into mass stinging of the perceived threat. This alarm pheromone is carried directly on the stinger; hence, its concentration builds up during the course of the attack. We investigate how bees react to different alarm pheromone concentrations and how this evolved response pattern leads to better coordination at the group level. RESULTS: We first present a dose-response curve to the alarm pheromone, obtained experimentally. This data reveals two phases in the bees' response: initially, bees become more likely to sting as the alarm pheromone concentration increases, but aggressiveness drops back when very high concentrations are reached. Second, we apply Projective Simulation to model each bee as an artificial learning agent that relies on the pheromone concentration to decide whether to sting or not. Individuals are rewarded based on the collective performance, thus emulating natural selection in these complex societies. By also modelling predators in a detailed way, we are able to identify the main selection pressures that shaped the response pattern observed experimentally. In particular, the likelihood to sting in the absence of alarm pheromone (starting point of the dose-response curve) is inversely related to the rate of false alarms, such that bees in environments with low predator density are less likely to waste efforts responding to irrelevant stimuli. This is compensated for by a steep increase in aggressiveness when the alarm pheromone concentration starts rising. The later decay in aggressiveness may be explained as a curbing mechanism preventing worker loss. CONCLUSIONS: Our work provides a detailed understanding of alarm pheromone responses in honeybees and sheds light on the selection pressures that brought them about. In addition, it establishes our approach as a powerful tool to explore how selection based on a collective outcome shapes individual responses, which remains a challenging issue in the field of evolutionary biology.

Development of swarm behavior in artificial learning agents that adapt to different foraging environments.

Collective behavior, and swarm formation in particular, has been studied from several perspectives within a large variety of fields, ranging from biology to physics. In this work, we apply Projective Simulation to model each individual as an artificial learning agent that interacts with its neighbors and surroundings in order to make decisions and learn from them. Within a reinforcement learning framework, we discuss one-dimensional learning scenarios where agents need to get to food resources to be rewarded. We observe how different types of collective motion emerge depending on the distance the agents need to travel to reach the resources. For instance, strongly aligned swarms emerge when the food source is placed far away from the region where agents are situated initially. In addition, we study the properties of the individual trajectories that occur within the different types of emergent collective dynamics. Agents trained to find distant resources exhibit individual trajectories that are in most cases best fit by composite correlated random walks with features that resemble Lévy walks. This composite motion emerges from the collective behavior developed under the specific foraging selection pressures. On the other hand, agents trained to reach nearby resources predominantly exhibit Brownian trajectories.

Modelling collective motion based on the principle of agency: General framework and the case of marching locusts.

Collective phenomena are studied in a range of contexts-from controlling locust plagues to efficiently evacuating stadiums-but the central question remains: how can a large number of independent individuals form a seemingly perfectly coordinated whole? Previous attempts to answer this question have reduced the individuals to featureless particles, assumed particular interactions between them and studied the resulting collective dynamics. While this approach has provided useful insights, it cannot guarantee that the assumed individual-level behaviour is accurate, and, moreover, does not address its origin-that is, the question of why individuals would respond in one way or another. We propose a new approach to studying collective behaviour, based on the concept of learning agents: individuals endowed with explicitly modelled sensory capabilities, an internal mechanism for deciding how to respond to the sensory input and rules for modifying these responses based on past experience. This detailed modelling of individuals favours a more natural choice of parameters than in typical swarm models, which minimises the risk of spurious dependences or overfitting. Most notably, learning agents need not be programmed with particular responses, but can instead develop these autonomously, allowing for models with fewer implicit assumptions. We illustrate these points with the example of marching locusts, showing how learning agents can account for the phenomenon of density-dependent alignment. Our results suggest that learning agent-based models are a powerful tool for studying a broader class of problems involving collective behaviour and animal agency in general.

Quantum-coherent mixtures of causal relations.

Understanding the causal influences that hold among parts of a system is critical both to explaining that system's natural behaviour and to controlling it through targeted interventions. In a quantum world, understanding causal relations is equally important, but the set of possibilities is far richer. The two basic ways in which a pair of time-ordered quantum systems may be causally related are by a cause-effect mechanism or by a common-cause acting on both. Here we show a coherent mixture of these two possibilities. We realize this nonclassical causal relation in a quantum optics experiment and derive a set of criteria for witnessing the coherence based on a quantum version of Berkson's effect, whereby two independent causes can become correlated on observation of their common effect. The interplay of causality and quantum theory lies at the heart of challenging foundational puzzles, including Bell's theorem and the search for quantum gravity.