A general framework for modelling trade-offs in adaptive behaviour.

An animal's behaviour can influence many variables, such as its energy reserves, its risk of injury or mortality, and its rate of reproduction. To identify the optimal action in a given situation, these various effects can be compared in the common currency of reproductive value. While this idea has been widely used to study trade-offs between pairs of variables, e.g. between energy gain versus survival, here we present a unified framework that makes explicit how these various trade-offs fit together. This unification covers a wide range of biological phenomena, highlighting similarities in their logical structure and helping to identify knowledge gaps. To fill one such gap, we present a new model of foraging under the risk of predation and damage accumulation. We conclude by discussing the use and limitations of state-dependent optimisation theory in predicting biological observations.

A critical review of risk-sensitive foraging.

Foraging is risk sensitive if choices depend on the variability of returns from the options as well as their mean return. Risk-sensitive foraging is important in behavioural ecology, psychology and neurophysiology. It has been explained both in terms of mechanisms and in terms of evolutionary advantage. We provide a critical review, evaluating both mechanistic and evolutionary accounts. Some derivations of risk sensitivity from mechanistic models based on psychophysics are not convincing because they depend on an inappropriate use of Jensen's inequality. Attempts have been made to link risk sensitivity to the ecology of a species, but again these are not convincing. The field of risk-sensitive foraging has provided a focus for theoretical and empirical work and has yielded important insights, but we lack a simple and empirically defendable general account of it in either mechanistic or evolutionary terms. However, empirical analysis of choice sequences under theoretically motivated experimental designs and environmental settings appears a promising avenue for mapping the scope and relative merits of existing theories. Simply put, the devil is in the sequence.

Behavioural specialization and learning in social networks.

Interactions in social groups can promote behavioural specialization. One way this can happen is when individuals engage in activities with two behavioural options and learn which option to choose. We analyse interactions in groups where individuals learn from playing games with two actions and negatively frequency-dependent payoffs, such as producer-scrounger, caller-satellite, or hawk-dove games. Group members are placed in social networks, characterized by the group size and the number of neighbours to interact with, ranging from just a few neighbours to interactions between all group members. The networks we analyse include ring lattices and the much-studied small-world networks. By implementing two basic reinforcement-learning approaches, action-value learning and actor-critic learning, in different games, we find that individuals often show behavioural specialization. Specialization develops more rapidly when there are few neighbours in a network and when learning rates are high. There can be learned specialization also with many neighbours, but we show that, for action-value learning, behavioural consistency over time is higher with a smaller number of neighbours. We conclude that frequency-dependent competition for resources is a main driver of specialization. We discuss our theoretical results in relation to experimental and field observations of behavioural specialization in social situations.

Biological adaptation in light of the Lewontin-Williams (a)symmetry.

Neo-Darwinism characterizes biological adaptation as a one-sided process, in which organisms adapt to their environment but not vice versa. This asymmetric relationship-here called Williams' asymmetry-is called into question by Niche Construction Theory, which emphasizes that organisms and their environments often mutually affect each other. Here, we clarify that Williams' asymmetry is specifically concerned with (quasi)-directed modifications toward phenotypes that increase individual fitness. This directedness-which drives the adaptive fit between organism and environment-entails far more than the mere presence of cause-effect relationships. We argue that difficulties with invoking fitness as the guiding principle of adaptive evolution are resolved with an appropriate definition of fitness and that objections against Williams' asymmetry reflect confusions about the nature of biological adaptation.

A critical evaluation of the index of patch quality.

The inverse optimality approach can allow us to learn about an animal's environment by assuming their behaviour is optimal. This approach has been applied to animals diving underwater for food to produce the index of patch quality (IPQ), which aims to provide a proxy for prey abundance or quality in a foraging patch based on the animal's diving behaviour. The IPQ has been used in several empirical studies but has never been evaluated theoretically. Here, we discuss the strengths and weaknesses of the IPQ approach from a theoretical angle and review the empirical evidence supporting its use. We highlight several potential issues, in particular with the gain function-the function describing the energetic gain of an animal during a dive-used to calculate the IPQ. We investigate an alternative gain function which is appropriate in some cases, provide a new model based on this function, and discuss differences between the IPQ model and ours. We also find that there is little supporting empirical evidence justifying the general use of the IPQ and suggest future empirical validation methods which could help strengthen the case for the IPQ. Our findings have implications for the field of diving ecology and habitat assessment.

How to Model Optimal Group Size in Social Carnivores.

AbstractModels of optimal group size need to identify the currency that correctly captures the fitness consequences of foraging. Although daily intake or daily net energy gain per animal are widely used as currencies, they are not ideal. They predict that all available time should be spent hunting and do not reflect performance during a hunt. We argue that the net rate while hunting is a better currency. Using an example based on the African wild dog, we illustrate the difference between maximizing daily net energy and net rate. Using the same example, we show that if foraging is limited by constraints on energy expenditure, then the optimal group size can be different from the size that maximizes the net rate while hunting. The direction of the effect depends on whether costs increase or decrease with group size. Furthermore, the proportion of time spent resting can be predicted. We suggest two novel approaches for future models: to consider the optimal hunting group size given a fixed group size and to investigate how the presence of dependent young may affect foraging behavior. We hope this will lead to meaningful conclusions on the role cooperative hunting has played in the evolution of sociality in social carnivores.

Learning, exploitation and bias in games.

We focus on learning during development in a group of individuals that play a competitive game with each other. The game has two actions and there is negative frequency dependence. We define the distribution of actions by group members to be an equilibrium configuration if no individual can improve its payoff by unilaterally changing its action. We show that at this equilibrium, one action is preferred in the sense that those taking the preferred action have a higher payoff than those taking the other, more prosocial, action. We explore the consequences of a simple 'unbiased' reinforcement learning rule during development, showing that groups reach an approximate equilibrium distribution, so that some achieve a higher payoff than others. Because there is learning, an individual's behaviour can influence the future behaviour of others. We show that, as a consequence, there is the potential for an individual to exploit others by influencing them to be the ones to take the non-preferred action. Using an evolutionary simulation, we show that population members can avoid being exploited by over-valuing rewards obtained from the preferred option during learning, an example of a bias that is 'rational'.

Matching Behaviours and Rewards.

Matching describes how behaviour is related to rewards. The matching law holds when the ratio of an individual's behaviours equals the ratio of the rewards obtained. From its origins in the study of pigeons working for food in the laboratory, the law has been applied to a range of species, both in the laboratory and outside it (e.g., human sporting decisions). Probability matching occurs when the probability of a behaviour equals the probability of being rewarded. Input matching predicts the distribution of individuals across habitats. We evaluate the rationality of the matching law and probability matching, expose the logic of matching in real-world cases, review how recent neuroscience findings relate to matching, and suggest future research directions.

Gains v. losses, or context dependence generated by confusion?

Tversky and Kahneman introduced the term framing for the finding that people give different answers to the same question depending on the way it is posed. One form of framing involves presenting the same outcome as either a gain or a loss. An experiment on starlings by Marsh and Kacelnik suggests that this form of framing occurs in non-humans. We argue that the experimental result demonstrates framing in the general sense of context dependence but does not provide compelling evidence of framing in terms of gains and losses. A version of scalar utility theory which is extended to include the possibility of memory errors accounts for the data and suggests future lines of research.

The overall discount rate.

Discounting refers to the way in which the value of an outcome depends on the delay until it is obtained. If an organism's discount function is known, then its rate of discounting at any delay can be found. If the function is not known, the normalised area under an estimate of the discount function has been used as a measure that summarises the strength of discounting over a range of delays. We propose a new measure of the strength of discounting: the overall discount rate W, which is the drop in value from the start to the end of the value curve divided by the area under this curve. We show that our measure has various advantages over the normalised area, namely it can be linked to the instantaneous rate of discounting and it respects the special nature of exponential discounting. It does not give a unique value to each curve, but we prove that this is incompatible with the requirement that similar discount functions be assigned similar values, and so this is not a defect of the measure.

Mechanistic models must link the field and the lab.

In the theory outlined in the target article, an animal forages continuously, making sequential decisions in a world where the amount of food and its uncertainty are fixed, but delays are variable. These assumptions contrast with the risk-sensitive foraging theory and create a problem for comparing the predictions of this model with many laboratory experiments that do not make these assumptions.

Optimal gut size of small birds and its dependence on environmental and physiological parameters.

Most optimal foraging models assume that the foraging behaviour of small birds depends on a single state variable, their energy reserves in the form of stored fat. Here, we include a second state variable-the contents of the bird's gut-to investigate how a bird should optimise its gut size to minimise its long-term mortality, depending on the availability of food, the size of meal and the bird's digestive constraints. Our results show that (1) the current level of fat is never less important than gut contents in determining the bird's survival; (2) there exists a unique optimal gut size, which is determined by a trade-off between the energetic gains and costs of maintaining a large digestive system; (3) the optimal gut size increases as the bird's digestive cycle becomes slower, allowing the bird to store undigested food; (4) the critical environmental factor for determining the optimal gut size is the mass of food found in a successful foraging effort ("meal size"). We find that when the environment is harsh, it is optimal for the bird to maintain a gut that is larger than the size of a meal. However, the optimal size of the gut in rich environments exactly matches the meal size (i.e. the mass of food that the optimal gut can carry is exactly the mass of food that can be obtained in a successful foraging attempt).

Trust your gut: using physiological states as a source of information is almost as effective as optimal Bayesian learning.

Approaches to understanding adaptive behaviour often assume that animals have perfect information about environmental conditions or are capable of sophisticated learning. If such learning abilities are costly, however, natural selection will favour simpler mechanisms for controlling behaviour when faced with uncertain conditions. Here, we show that, in a foraging context, a strategy based only on current energy reserves often performs almost as well as a Bayesian learning strategy that integrates all previous experiences to form an optimal estimate of environmental conditions. We find that Bayesian learning gives a strong advantage only if fluctuations in the food supply are very strong and reasonably frequent. The performance of both the Bayesian and the reserve-based strategy are more robust to inaccurate knowledge of the temporal pattern of environmental conditions than a strategy that has perfect knowledge about current conditions. Studies assuming Bayesian learning are often accused of being unrealistic; our results suggest that animals can achieve a similar level of performance to Bayesians using much simpler mechanisms based on their physiological state. More broadly, our work suggests that the ability to use internal states as a source of information about recent environmental conditions will have weakened selection for sophisticated learning and decision-making systems.

Adaptive and non-adaptive models of depression: A comparison using register data on antidepressant medication during divorce.

Divorce is associated with an increased probability of a depressive episode, but the causation of events remains unclear. Adaptive models of depression propose that depression is a social strategy in part, whereas non-adaptive models tend to propose a diathesis-stress mechanism. We compare an adaptive evolutionary model of depression to three alternative non-adaptive models with respect to their ability to explain the temporal pattern of depression around the time of divorce. Register-based data (304,112 individuals drawn from a random sample of 11% of Finnish people) on antidepressant purchases is used as a proxy for depression. This proxy affords an unprecedented temporal resolution (a 3-monthly prevalence estimates over 10 years) without any bias from non-compliance, and it can be linked with underlying episodes via a statistical model. The evolutionary-adaptation model (all time periods with risk of divorce are depressogenic) was the best quantitative description of the data. The non-adaptive stress-relief model (period before divorce is depressogenic and period afterwards is not) provided the second best quantitative description of the data. The peak-stress model (periods before and after divorce can be depressogenic) fit the data less well, and the stress-induction model (period following divorce is depressogenic and the preceding period is not) did not fit the data at all. The evolutionary model was the most detailed mechanistic description of the divorce-depression link among the models, and the best fit in terms of predicted curvature; thus, it offers most rigorous hypotheses for further study. The stress-relief model also fit very well and was the best model in a sensitivity analysis, encouraging development of more mechanistic models for that hypothesis.

Scalar utility theory and proportional processing: What does it actually imply?

Scalar Utility Theory (SUT) is a model used to predict animal and human choice behaviour in the context of reward amount, delay to reward, and variability in these quantities (risk preferences). This article reviews and extends SUT, deriving novel predictions. We show that, contrary to what has been implied in the literature, (1) SUT can predict both risk averse and risk prone behaviour for both reward amounts and delays to reward depending on experimental parameters, (2) SUT implies violations of several concepts of rational behaviour (e.g. it violates strong stochastic transitivity and its equivalents, and leads to probability matching) and (3) SUT can predict, but does not always predict, a linear relationship between risk sensitivity in choices and coefficient of variation in the decision-making experiment. SUT derives from Scalar Expectancy Theory which models uncertainty in behavioural timing using a normal distribution. We show that the above conclusions also hold for other distributions, such as the inverse Gaussian distribution derived from drift-diffusion models. A straightforward way to test the key assumptions of SUT is suggested and possible extensions, future prospects and mechanistic underpinnings are discussed.

Fatness and fitness: exposing the logic of evolutionary explanations for obesity.

To explore the logic of evolutionary explanations of obesity we modelled food consumption in an animal that minimizes mortality (starvation plus predation) by switching between activities that differ in energy gain and predation. We show that if switching does not incur extra predation risk, the animal should have a single threshold level of reserves above which it performs the safe activity and below which it performs the dangerous activity. The value of the threshold is determined by the environmental conditions, implying that animals should have variable 'set points'. Selection pressure to prevent energy stores exceeding the optimal level is usually weak, suggesting that immediate rewards might easily overcome the controls against becoming overweight. The risk of starvation can have a strong influence on the strategy even when starvation is extremely uncommon, so the incidence of mortality during famine in human history may be unimportant for explanations for obesity. If there is an extra risk of switching between activities, the animal should have two distinct thresholds: one to initiate weight gain and one to initiate weight loss. Contrary to the dual intervention point model, these thresholds will be inter-dependent, such that altering the predation risk alters the location of both thresholds; a result that undermines the evolutionary basis of the drifty genes hypothesis. Our work implies that understanding the causes of obesity can benefit from a better understanding of how evolution shapes the mechanisms that control body weight.

Costs of Foraging Predispose Animals to Obesity-Related Mortality when Food Is Constantly Abundant.

Obesity is an important medical problem affecting humans and animals in the developed world, but the evolutionary origins of the behaviours that cause obesity are poorly understood. The potential role of occasional gluts of food in determining fat-storage strategies for avoiding mortality have been overlooked, even though animals experienced such conditions in the recent evolutionary past and may follow the same strategies in the modern environment. Humans, domestic, and captive animals in the developed world are exposed to a surplus of calorie-rich food, conditions characterised as 'constant-glut'. Here, we use a mathematical model to demonstrate that obesity-related mortality from poor health in a constant-glut environment should equal the average mortality rate in the 'pre-modern' environment when predation risk was more closely linked with foraging. It should therefore not be surprising that animals exposed to abundant food often over-eat to the point of ill-health. Our work suggests that individuals tend to defend a given excessive level of reserves because this level was adaptive when gluts were short-lived. The model predicts that mortality rate in constant-glut conditions can increase as the assumed health cost of being overweight decreases, meaning that any adaptation that reduced such health costs would have counter-intuitively led to an increase in mortality in the modern environment. Taken together, these results imply that efforts to reduce the incidence of obesity that are focussed on altering individual behaviour are likely to be ineffective because modern, constant-glut conditions trigger previously adaptive behavioural responses.

Adaptive learning can result in a failure to profit from good conditions: implications for understanding depression.

BACKGROUND AND OBJECTIVES: Depression is a major medical problem diagnosed in an increasing proportion of people and for which commonly prescribed psychoactive drugs are frequently ineffective. Development of treatment options may be facilitated by an evolutionary perspective; several adaptive reasons for proneness to depression have been proposed. A common feature of many explanations is that depressive behaviour is a way to avoid costly effort where benefits are small and/or unlikely. However, this viewpoint fails to explain why low mood persists when the situation improves. We investigate whether a behavioural rule that is adapted to a stochastically changing world can cause inactivity which appears similar to the effect of depression, in that it persists after the situation has improved. METHODOLOGY: We develop an adaptive learning model in which an individual has repeated choices of whether to invest costly effort that may result in a net benefit. Investing effort also provides information about the current conditions and rates of change of the conditions. RESULTS: An individual following the optimal behavioural strategy may sometimes remain inactive when conditions are favourable (i.e. when it would be better to invest effort) when it is poorly informed about the current environmental state. Initially benign conditions can predispose an individual to inactivity after a relatively brief period of negative experiences. CONCLUSIONS AND IMPLICATIONS: Our approach suggests that the antecedent factors causing depressed behaviour could go much further back in an individual s history than is currently appreciated. The insights from our approach have implications for the ongoing debate about best treatment options for patients with depressive symptoms.

Risk attitudes in a changing environment: An evolutionary model of the fourfold pattern of risk preferences.

A striking feature of human decision making is the fourfold pattern of risk attitudes, involving risk-averse behavior in situations of unlikely losses and likely gains, but risk-seeking behavior in response to likely losses and unlikely gains. Current theories to explain this pattern assume particular psychological processes to reproduce empirical observations, but do not address whether it is adaptive for the decision maker to respond to risk in this way. Here, drawing on insights from behavioral ecology, we build an evolutionary model of risk-sensitive behavior, to investigate whether particular types of environmental conditions could favor a fourfold pattern of risk attitudes. We consider an individual foraging in a changing environment, where energy is needed to prevent starvation and build up reserves for reproduction. The outcome, in terms of reproductive value (a rigorous measure of evolutionary success), of a one-off choice between a risky and a safe gain, or between a risky and a safe loss, determines the risk-sensitive behavior we should expect to see in this environment. Our results show that the fourfold pattern of risk attitudes may be adaptive in an environment in which conditions vary stochastically but are autocorrelated in time. In such an environment the current options provide information about the likely environmental conditions in the future, which affect the optimal pattern of risk sensitivity. Our model predicts that risk preferences should be both path dependent and affected by the decision maker's current state.

The starvation-predation trade-off shapes the strategic use of protein for energy during fasting.

The primary function of lipid storage by animals is as an energy source for surviving periods without food. However, muscle and organ protein can be metabolised for energy, and empirical studies have shown that the onset of protein metabolism begins before the exhaustion of lipid reserves. Since protein tissues are important for reasons other than resisting starvation, the adaptive basis for this early onset is unclear. Here, we report the results of a model of the optimal proportion of energy to obtain from protein catabolism during a period without food of unpredictable duration. We assume either that the animal aims only to maximise the duration of survival or that it also has to take account of its future reproductive success given its state when the food supply recommences. In the latter case we find impressive quantitative agreement with observations on lean and obese penguins and rats. Analysis shows that this agreement breaks down if predation risk is insignificant, protein in the form of muscle is ineffective against predation, or there is no benefit to conserving lipid (e.g. for reproduction). This result implies that animals have not evolved to maximise their starvation resistance because doing so would leave them vulnerable when an interruption ends. Our model allows us to make several specific predictions concerning the relationship between the ecological pressures on animals and their starvation survival strategies.