Emergent Resource Exchange and Tolerated Theft Behavior Using Multiagent Reinforcement Learning.

For decades, the evolution of cooperation has piqued interest in numerous academic disciplines, such as game theory, economics, biology, and computer science. In this work, we demonstrate the emergence of a novel and effective resource exchange protocol formed by dropping and picking up resources in a foraging environment. This form of cooperation is made possible by the introduction of a campfire, which adds an extended period of congregation and downtime for agents to explore otherwise unlikely interactions. We find that the agents learn to avoid getting cheated by their exchange partners, but not always from a third party. We also observe the emergence of behavior analogous to tolerated theft, despite the lack of any punishment, combat, or larceny mechanism in the environment.

A unicellular walker controlled by a microtubule-based finite-state machine.

Cells are complex biochemical systems whose behaviors emerge from interactions among myriad molecular components. Computation is often invoked as a general framework for navigating this cellular complexity. However, it is unclear how cells might embody computational processes such that the theories of computation, including finite-state machine models, could be productively applied. Here, we demonstrate finite-state-machine-like processing embodied in cells using the walking behavior of Euplotes eurystomus, a ciliate that walks across surfaces using fourteen motile appendages (cirri). We found that cellular walking entails regulated transitions among a discrete set of gait states. The set of observed transitions decomposes into a small group of high-probability, temporally irreversible transitions and a large group of low-probability, time-symmetric transitions, thus revealing stereotypy in the sequential patterns of state transitions. Simulations and experiments suggest that the sequential logic of the gait is functionally important. Taken together, these findings implicate a finite-state-machine-like process. Cirri are connected by microtubule bundles (fibers), and we found that the dynamics of cirri involved in different state transitions are associated with the structure of the fiber system. Perturbative experiments revealed that the fibers mediate gait coordination, suggesting a mechanical basis of gait control.

Nanofluid Boiling on Micro/Nano-engineered Surfaces.

This work characterizes the impact of boiling aqueous nanofluids on engineered surfaces designed for boiling enhancement with pure water. Although micro/nano-engineered surfaces have been shown to enhance boiling, these achievements are typically demonstrated using deionized water or other purified fluids. In parallel, particulate-laden fluids, also known as nanofluids, have been shown to enhance boiling as well. In this study, we investigate a variety of engineered surfaces and the boiling degradation due to the addition of SiO2 particles at a fixed concentration of 0.2% by volume but varying sizes from 7 nm to 10 µm. Although the addition of SiO2 particles is shown to moderately improve critical heat flux (CHF) on all the surfaces considered, the heat transfer coefficient (HTC) is seen to deteriorate with the addition of particles of any size. The bare copper surface and a nanostructured surface show particle size-dependent degradation of the HTC due to clogging. Bi-conductive surfaces also show a degradation of the HTC, but it was shown to be independent of the particle size. This work has shown specific and unique degradation mechanisms for each of the surfaces considered including the reduction of nucleation sites and thermal insulation. Additionally, the surfaces tested in this work exhibited a partial-CHF condition occurring with the addition of particles.

Evolving Complexity in Cooperative and Competitive Noisy Prediction Games.

We examine the effect of cooperative and competitive interactions on the evolution of complex strategies in a prediction game. We extend previous work to the domain of noisy games, defining a new organism and mutation model, and an accompanying novel complexity metric. We find that a mix of cooperation and competition is the most effective in driving complexity growth, confirming prior results. We also compare our complexity metric with simpler metrics such as raw strategy size, and demonstrate the effectiveness of our metric in distinguishing true complexity from mere genetic bloat.

Evolving Complexity in Prediction Games.

To study open-ended coevolution, we define a complexity metric over interacting finite state machines playing formal language prediction games, and study the dynamics of populations under competitive and cooperative interactions. In the past purely competitive and purely cooperative interactions have been studied extensively, but neither can successfully and continuously drive an arms race. We present quantitative results using this complexity metric and analyze the causes of varying rates of complexity growth across different types of interactions. We find that while both purely competitive and purely cooperative coevolution are able to drive complexity growth above the rate of genetic drift, mixed systems with both competitive and cooperative interactions achieve significantly higher evolved complexity.

Escalation of Memory Length in Finite Populations.

The escalation of complexity is a commonly cited benefit of coevolutionary systems, but computational simulations generally fail to demonstrate this capacity to a satisfactory degree. We draw on a macroevolutionary theory of escalation to develop a set of criteria for coevolutionary systems to exhibit escalation of strategic complexity. By expanding on a previously developed model of the evolution of memory length for cooperative strategies by Kristian Lindgren, we resolve previously observed limitations on the escalation of memory length by extending operators of evolutionary variation. We present long-term coevolutionary simulations showing that larger population sizes tend to support greater escalation of complexity than smaller ones do. Additionally, we investigate the sensitivity of escalation during transitions of complexity. The Lindgren model has often been used to argue that the escalation of competitive coevolution has intrinsic limitations. Our simulations show that coevolutionary arms races can continue to escalate in computational simulations given sufficient population sizes.

Increasing Boiling Heat Transfer using Low Conductivity Materials.

We report the counterintuitive mechanism of increasing boiling heat transfer by incorporating low-conductivity materials at the interface between the surface and fluid. By embedding an array of non-conductive lines into a high-conductivity substrate, in-plane variations in the local surface temperature are created. During boiling the surface temperature varies spatially across the substrate, alternating between high and low values, and promotes the organization of distinct liquid and vapor flows. By systematically tuning the peak-to-peak wavelength of this spatial temperature variation, a resonance-like effect is seen at a value equal to the capillary length of the fluid. Replacing ~18% of the surface with a non-conductive epoxy results in a greater than 5x increase in heat transfer rate at a given superheat temperature. This drastic and counterintuitive increase is shown to be due to optimized bubble dynamics, where ordered pathways allow for efficient removal of vapor and the return of replenishing liquid. The use of engineered thermal gradients represents a potentially disruptive approach to create high-efficiency and high-heat-flux boiling surfaces which are naturally insensitive to fouling and degradation as compared to other approaches.

Cracking the egg: virtual embryogenesis of real robots.

All multicellular living beings are created from a single cell. A developmental process, called embryogenesis, takes this first fertilized cell down a complex path of reproduction, migration, and specialization into a complex organism adapted to its environment. In most cases, the first steps of the embryogenesis take place in a protected environment such as in an egg or in utero. Starting from this observation, we propose a new approach to the generation of real robots, strongly inspired by living systems. Our robots are composed of tens of specialized cells, grown from a single cell using a bio-inspired virtual developmental process. Virtual cells, controlled by gene regulatory networks, divide, migrate, and specialize to produce the robot's body plan (morphology), and then the robot is manually built from this plan. Because the robot is as easy to assemble as Lego, the building process could be easily automated.

Imaging analysis of clock neurons reveals light buffers the wake-promoting effect of dopamine.

How animals maintain proper amounts of sleep yet remain flexible to changes in environmental conditions remains unknown. We found that environmental light suppressed the wake-promoting effects of dopamine in fly brains. The ten large lateral-ventral neurons (l-LNvs), a subset of clock neurons, are wake-promoting and respond to dopamine, octopamine and light. Behavioral and imaging analyses suggested that dopamine is a stronger arousal signal than octopamine. Notably, light exposure not only suppressed l-LNv responses, but also synchronized responses of neighboring l-LNvs. This regulation occurred by distinct mechanisms: light-mediated suppression of octopamine responses was regulated by the circadian clock, whereas light regulation of dopamine responses occurred by upregulation of inhibitory dopamine receptors. Plasticity therefore alters the relative importance of diverse cues on the basis of the environmental mix of stimuli. The regulatory mechanisms described here may contribute to the control of sleep stability while still allowing behavioral flexibility.

Evolutionary dynamics of finite populations in games with polymorphic fitness equilibria.

The hawk-dove (HD) game, as defined by Maynard Smith [1982. Evolution and the Theory of Games. Cambridge University Press, Cambridge], allows for a polymorphic fitness equilibrium (PFE) to exist between its two pure strategies; this polymorphism is the attractor of the standard replicator dynamics [Taylor, P.D., Jonker, L., 1978. Evolutionarily stable strategies and game dynamics. Math. Biosci. 40, 145-156; Hofbauer, J., Sigmund, K., 1998. Evolutionary Games and Population Dynamics. Cambridge University Press, Cambridge] operating on an infinite population of pure-strategists. Here, we consider stochastic replicator dynamics, operating on a finite population of pure-strategists playing games similar to HD; in particular, we examine the transient behavior of the system, before it enters an absorbing state due to sampling error. Though stochastic replication prevents the population from fixing onto the PFE, selection always favors the under-represented strategy. Thus, we may naively expect that the mean population state (of the pre-absorption transient) will correspond to the PFE. The empirical results of Fogel et al. [1997. On the instability of evolutionary stable states. BioSystems 44, 135-152] show that the mean population state, in fact, deviates from the PFE with statistical significance. We provide theoretical results that explain their observations. We show that such deviation away from the PFE occurs when the selection pressures that surround the fitness-equilibrium point are asymmetric. Further, we analyze a Markov model to prove that a finite population will generate a distribution over population states that equilibrates selection-pressure asymmetry; the mean of this distribution is generally not the fitness-equilibrium state.

Modular interdependency in complex dynamical systems.

Herbert A. Simon's characterization of modularity in dynamical systems describes subsystems as having dynamics that are approximately independent of those of other subsystems (in the short term). This fits with the general intuition that modules must, by definition, be approximately independent. In the evolution of complex systems, such modularity may enable subsystems to be modified and adapted independently of other subsystems, whereas in a nonmodular system, modifications to one part of the system may result in deleterious side effects elsewhere in the system. But this notion of modularity and its effect on evolvability is not well quantified and is rather simplistic. In particular, modularity need not imply that intermodule dependences are weak or unimportant. In dynamical systems this is acknowledged by Simon's suggestion that, in the long term, the dynamical behaviors of subsystems do interact with one another, albeit in an "aggregate" manner--but this kind of intermodule interaction is omitted in models of modularity for evolvability. In this brief discussion we seek to unify notions of modularity in dynamical systems with notions of how modularity affects evolvability. This leads to a quantifiable measure of modularity and a different understanding of its effect on evolvability.

Ideal evaluation from coevolution.

In many problems of interest, performance can be evaluated using tests, such as examples in concept learning, test points in function approximation, and opponents in game-playing. Evaluation on all tests is often infeasible. Identification of an accurate evaluation or fitness function is a difficult problem in itself, and approximations are likely to introduce human biases into the search process. Coevolution evolves the set of tests used for evaluation, but has so far often led to inaccurate evaluation. We show that for any set of learners, a Complete Evaluation Set can be determined that provides ideal evaluation as specified by Evolutionary Multi-Objective Optimization. This provides a principled approach to evaluation in coevolution, and thereby brings automatic ideal evaluation within reach. The Complete Evaluation Set is of manageable size, and progress towards it can be accurately measured. Based on this observation, an algorithm named DELPHI is developed. The algorithm is tested on problems likely to permit progress on only a subset of the underlying objectives. Where all comparison methods result in overspecialization, the proposed method and a variant achieve sustained progress in all underlying objectives. These findings demonstrate that ideal evaluation may be approximated by practical algorithms, and that accurate evaluation for test-based problems is possible even when the underlying objectives of a problem are unknown.

A computational model of symbiotic composition in evolutionary transitions.

Several of the major transitions in evolutionary history, such as the symbiogenic origin of eukaryotes from prokaryotes, share the feature that existing entities became the components of composite entities at a higher-level of organization. This composition of pre-adapted extant entities into a new whole is a fundamentally different source of variation from the gradual accumulation of small random variations, and it has some interesting consequences for issues of evolvability. Intuitively, the pre-adaptation of sets of features in reproductively independent specialists suggests a form of 'divide and conquer' decomposition of the adaptive domain. Moreover, the compositions resulting from one level may become the components for compositions at the next level, thus scaling-up the variation mechanism. In this paper, we explore and develop these concepts using a simple abstract model of symbiotic composition to examine its impact on evolvability. To exemplify the adaptive capacity of the composition model, we employ a scale-invariant fitness landscape exhibiting significant ruggedness at all scales. Whilst innovation by mutation and by conventional evolutionary algorithms becomes increasingly more difficult as evolution continues in this landscape, innovation by composition is not impeded as it discovers and assembles component entities through successive hierarchical levels.

On the origin of modular variation.

We study the dynamics of modularization in a minimal substrate. A module is a functional unit relatively separable from its surrounding structure. Although it is known that modularity is useful both for robustness and for evolvability (Wagner 1996), there is no quantitative model describing how such modularity might originally emerge. Here we suggest, using simple computer simulations, that modularity arises spontaneously in evolutionary systems in response to variation, and that the amount of modular separation is logarithmically proportional to the rate of variation. Consequently, we predict that modular architectures would appear in correlation with high environmental change rates. Because this quantitative model does not require any special substrate to occur, it may also shed light on the origin of modular variation in nature. This observed relationship also indicates that modular design is a generic phenomenon that might be applicable to other fields, such as engineering: Engineering design methods based on evolutionary simulation would benefit from evolving to variable, rather than stationary, fitness criteria, as a weak and problem-independent method for inducing modularity.

Creating high-level components with a generative representation for body-brain evolution.

One of the main limitations of scalability in body-brain evolution systems is the representation chosen for encoding creatures. This paper defines a class of representations called generative representations, which are identified by their ability to reuse elements of the genotype in the translation to the phenotype. This paper presents an example of a generative representation for the concurrent evolution of the morphology and neural controller of simulated robots, and also introduces GENRE, an evolutionary system for evolving designs using this representation. Applying GENRE to the task of evolving robots for locomotion and comparing it against a non-generative (direct) representation shows that the generative representation system rapidly produces robots with significantly greater fitness. Analyzing these results shows that the generative representation system achieves better performance by capturing useful bias from the design space and by allowing viable large scale mutations in the phenotype. Generative representations thereby enable the encapsulation, coordination, and reuse of assemblies of parts.