Flexible learning in complex worlds.

Cognitive flexibility can enhance the ability to adjust to changing environments. Here, we use learning simulations to investigate the possible advantages of flexible learning in volatile (changing) environments. We compare two established learning mechanisms, one with constant learning rates and one with rates that adjust to volatility. We study an ecologically relevant case of volatility, based on observations of developing cleaner fish Labroides dimidiatus that experience a transition from a simpler to a more complex foraging environment. There are other similar transitions in nature, such as migrating to a new and different habitat. We also examine two traditional approaches to volatile environments in experimental psychology and behavioral ecology: reversal learning, and learning set formation (consisting of a sequence of different discrimination tasks). These provide experimental measures of cognitive flexibility. Concerning transitions to a complex world, we show that both constant and flexible learning rates perform well, losing only a small proportion of available rewards in the period after a transition, but flexible rates perform better than constant rates. For reversal learning, flexible rates improve the performance with each successive reversal because of increasing learning rates, but this does not happen for constant rates. For learning set formation, we find no improvement in performance with successive shifts to new stimuli to discriminate for either flexible or constant learning rates. Flexible learning rates might thus explain increasing performance in reversal learning but not in learning set formation, and this can shed light on the nature of cognitive flexibility in a given system.

Reinforcement Learning Theory Reveals the Cognitive Requirements for Solving the Cleaner Fish Market Task.

Learning is an adaptation that allows individuals to respond to environmental stimuli in ways that improve their reproductive outcomes. The degree of sophistication in learning mechanisms potentially explains variation in behavioral responses. Here, we present a model of learning that is inspired by documented intra- and interspecific variation in the performance of a simultaneous two-choice task, the biological market task. The task presents a problem that cleaner fish often face in nature: choosing between two client types, one that is willing to wait for inspection and one that may leave if ignored. The cleaner's choice hence influences the future availability of clients (i.e., it influences food availability). We show that learning the preference that maximizes food intake requires subjects to represent in their memory different combinations of pairs of client types rather than just individual client types. In addition, subjects need to account for future consequences of actions, either by estimating expected long-term reward or by experiencing a client leaving as a penalty (negative reward). Finally, learning is influenced by the absolute and relative abundance of client types. Thus, cognitive mechanisms and ecological conditions jointly explain intra- and interspecific variation in the ability to learn the adaptive response.

Queen-worker conflict can drive the evolution of social polymorphism and split sex ratios in facultatively eusocial life cycles.

Hamilton's idea that haplodiploidy favors the evolution of altruism-the haplodiploidy hypothesis-relies on the relatedness asymmetry between the sexes caused by the sex-specific ploidies. Theoretical work on the consequences of relatedness asymmetries has significantly improved our understanding of sex allocation and intracolony conflicts, but the importance of haplodiploidy for the evolution of altruism came to be seen as minor. However, recently it was shown that haplodiploidy can strongly favor the evolution of eusociality, provided additional "preadaptations" are also present, such as the production of multiple broods per season and maternal ability to bias offspring sex ratios. These results were obtained assuming no influence of workers on the sex ratio, even though worker control of the sex ratio is known to occur. Here, we model the evolution of sex-specific fratricide as a mechanism of worker control over the sex ratio. We show that fratricide can facilitate the initial evolution of helping. However, fratricide can also hamper the evolution of unconditional help. Instead, social polymorphism evolves a mixture of helping and dispersing offspring. Finally, we show that the co-evolution of sex-allocation strategies of workers (fratricide) and queens leads to a split production of the sexes, with some colonies specializing in males and others in females. Thus, the model predicts that fratricide spawns a diversity of co-existing life cycles that strongly vary in degree of sociality and sex ratios.

A unified model of Hymenopteran preadaptations that trigger the evolutionary transition to eusociality.

Explaining the origin of eusociality, with strict division of labour between workers and reproductives, remains one of evolutionary biology's greatest challenges. Specific combinations of genetic, behavioural and demographic traits in Hymenoptera are thought to explain their relatively high frequency of eusociality, but quantitative models integrating such preadaptations are lacking. Here we use mathematical models to show that the joint evolution of helping behaviour and maternal sex ratio adjustment can synergistically trigger both a behavioural change from solitary to eusocial breeding, and a demographic change from a life cycle with two reproductive broods to a life cycle in which an unmated cohort of female workers precedes a final generation of dispersing reproductives. Specific suits of preadaptations are particularly favourable to the evolution of eusociality: lifetime monogamy, bivoltinism with male generation overlap, hibernation of mated females and haplodiploidy with maternal sex ratio adjustment. The joint effects of these preadaptations may explain the abundance of eusociality in the Hymenoptera and its virtual absence in other haplodiploid lineages.

Negotiation and appeasement can be more effective drivers of sociality than kin selection.

Two alternative frameworks explain the evolution of cooperation in the face of conflicting interests. Conflicts can be alleviated by kinship, the alignment of interests by virtue of shared genes, or by negotiation strategies, allowing mutually beneficial trading of services or commodities. Although negotiation often occurs in kin-structured populations, the interplay of kin- and negotiation-based mechanisms in the evolution of cooperation remains an unresolved issue. Inspired by the biology of a cooperatively breeding fish, we developed an individual-based simulation model to study the evolution of negotiation-based cooperation in relation to different levels of genetic relatedness. We show that the evolution of negotiation strategies leads to an equilibrium where subordinates appease dominants by conditional cooperation, resulting in high levels of help and low levels of aggression. This negotiation-based equilibrium can be reached both in the absence of relatedness and in a kin-structured population. However, when relatedness is high, evolution often ends up in an alternative equilibrium where subordinates help their kin unconditionally. The level of help at this kin-selected equilibrium is considerably lower than at the negotiation-based equilibrium, and it corresponds to a level reached when responsiveness is prevented from evolving in the simulations. A mathematical invasion analysis reveals that, quite generally, the alignment of payoffs due to the relatedness of interaction partners tends to impede selection for harsh but effective punishment of defectors. Hence kin structure will often hamper rather than facilitate the evolution of productive cooperation.