Expected Policy Gradient for Network Aggregative Markov Games in Continuous Space.

In this article, we investigate the Nash-seeking problem of a set of agents, playing an infinite network aggregative Markov game. In particular, we focus on a noncooperative framework where each agent selfishly aims at maximizing its long-term average reward without having explicit information on the model of the environment dynamics and its own reward function. The main contribution of this article is to develop a continuous multiagent reinforcement learning (MARL) algorithm for the Nash-seeking problem in infinite dynamic games with convergence guarantee. To this end, we propose an actor-critic MARL algorithm based on expected policy gradient (EPG) with two general function approximators to estimate the value function and the Nash policy of the agents. We consider continuous state and action spaces and adopt a newly proposed EPG to alleviate the variance of the gradient approximation. Based on such formulation and under some conventional assumptions (e.g., using linear function approximators), we prove that the policies of the agents converge to the unique Nash equilibrium (NE) of the game. Furthermore, an estimation error analysis is conducted to investigate the effects of the error arising from function approximation. As a case study, the framework is applied on a cloud radio access network (C-RAN) by modeling the remote radio heads (RRHs) as the agents and the congestion of baseband units (BBUs) as the dynamics of the environment.

Inactivation of tumor suppressor genes and cancer therapy: An evolutionary game theory approach.

Inactivation of alleles in tumor suppressor genes (TSG) is one of the important issues resulting in evolution of cancerous cells. In this paper, the evolution of healthy, one and two missed allele cells is modeled using the concept of evolutionary game theory and replicator dynamics. The proposed model also takes into account the interaction rates of the cells as designing parameters of the system. Different combinations of the equilibrium points of the parameterized nonlinear system is studied and categorized into some cases. In each case, the interaction rates' values are suggested in a way that the equilibrium points of the replicator dynamics are located on an appropriate region of the state space. Based on the suggested interaction rates, it is proved that the system doesn't have any undesirable interior equilibrium point as well. Therefore, the system will converge to the desirable region, where there is a scanty level of cancerous cells. In addition, the proposed conditions for interaction rates guarantee that, when a trajectory of the system reaches the boundaries, then it will stay there forever which is a desirable property since the equilibrium points have been already located on the boundaries, appropriately. The simulation results show the effectiveness of the suggestions in the elimination of the cancerous cells in different scenarios.

Effects of selection and mutation on epidemiology of X-linked genetic diseases.

The epidemiology of X-linked recessive diseases, a class of genetic disorders, is modeled with a discrete-time, structured, non linear mathematical system. The model accounts for both de novo mutations (i.e., affected sibling born to unaffected parents) and selection (i.e., distinct fitness rates depending on individual's health conditions). Assuming that the population is constant over generations and relying on Lyapunov theory we found the domain of attraction of model's equilibrium point and studied the convergence properties of the degenerate equilibrium where only affected individuals survive. Examples of applications of the proposed model to two among the most common X-linked recessive diseases (namely the red and green color blindness and the Hemophilia A) are described.

Selection and mutation in X-linked recessive diseases epidemiological model.

To describe the epidemiology of X-linked recessive diseases we developed a discrete time, structured, non linear mathematical model. The model allows for de novo mutations (i.e. affected sibling born to unaffected parents) and selection (i.e., distinct fitness rates depending on individual's health conditions). Applying Lyapunov direct method we found the domain of attraction of model's equilibrium point and studied the convergence properties of the degenerate equilibrium where only affected individuals survive.