Coordinateurs: E. Altman A. Jean Marie Maestro, INRIA, France

1. Bordeaux, Oct 2009 Coordinateurs: E. Altman A. Jean Marie Maestro, INRIA, France

2. Overview of the talk • Scientific Background • Participants • Meetings • Forms of collaboration • Publications • Other outcome

3. Background: What is Popeye? • Large complex systems involving interactions among one or more populations. • Population = large set of individuals, may be modeled as individual agents, or as a continuum of non-atomic agents. • Different disciplines: computer science and network engineering, mathematics, economics, biology. • Aim: develop new theoretical tools as well as at their applications to dynamic and spatial aspects of populations. Focus on applications in biology and networking.

4. Objectives • Understand the dynamics of complex communication networks and the design of novel methods for controlling them. • Most of the theoretical tools in population dynamics and control as well as the experience in applying them to complex systems in biology have not been exploited in applications in computer sciecne. • Create new theoretical tools in the areas of (i) population dynamics and its control, (ii) spatial aspects of populations, (iii) competition between populations. • Create new collaborations and enhance existing ones between mathematicians, computer scientists and biologists

5. PARTICIPANTS

6. Seminars and Workshops • Kickoff Meeting INRA, Avignon, Dec 2007 • Popeye Seminar LIA, Avignon, March 28, 2008 • Workshop POPEYE IMAG Grenoble, 21-22 May 2008 • Workshop Game Theory with applications to Networking and Computer Science 22-23 May 2008; In conjuction with POPEYE workshop • Popeye Meeting on game theory 10 February 2009 • Popeye Meeting on population models and game theory 9 April 2009 • Meeting planned in Dec 2009 in conjunction with BIONETICS (Avignon).

7. Research Topics • Branching Processes • Spatial population models and routing in adhoc networks • Models for competition between populations: evolutionary games, dynamic conjectural variations equilibria

8. Collaborations • Maestro-LIA: intensive collaboration on Evolutionary Games (EG) • Maestro-INRA-Tosca-Mescal : epidemic routing and its control. Application: Delay Tolerant Networks (DTNs). Joint Internship (PhD student) 3 months • Maestro - Nice Polytech: population modelling methods for ad hoc networks • ANR (MODECOL) Maestro-INRA-Mere

9. 1. Branching Processes. Background: • 1870: concern among aristocratic families that the surnames were becoming extinct.. Galton posed the question of computing the extinction probability. • Watson came to the wrong conclusion that all families sooner or later die out. • French statistician I. J. Bienayme (1845) obtained the correct answer but did not publish a formal proof. Unknown till it was rediscovered in 1962 by Heyde and Seneta. • Same problem formulated independently by A. K. Erlang.. The full proof given first by Steffensen 1933. • Recent studies: Immigration processes, Multitype branching

10. Branching Processes Summary of results • Previous results rquire independence assumptions between generations. We developed tools that allow dependence. • Applications to probability of successful meassage delivery in Ad-Hoc networks, and to queueing models. • Branching in continuous state space • Novel definitions of branching processes (max,+) algebra

11. Publications: branching processes N. Champagnat and A. Lambert. Adaptive dynamics in logistic branching population. Stochastic Models in Biological Sciences, Banach Center Publ. 80: 235-244 (2008). E. Altman , "Semi-linear stochastic difference equations", Discrete Event Dynamic Systems , 19:115-136, 2008. D. Fiems and E .Altman, Markov-modulated stochastic recursive equations with applications to delay-tolerant networks , INRIA RR 6872, Bionetics, Dec 2009. D. Fiems and E. Altman, Applying branching processes to delay-tolerant networks, to appear in the proceedings of BIONETICS, Avignon, Dec. 2009. N. Champagnat and S. Méléard, Polymorphic evolution sequence and evolutionary branching. Preprint (2008), in revision for publication in Probability Theory and Related Fields. N. Champagnat, "A study of evolutionary branching in a logistically regulated population", 27th European Meeting of Statisticians, Université Paul-Sabatier, Toulouse, du 20 au 24 juillet 2009.

12. 2. massively dense ad-hoc networks • Research direction initiated by Philippe Jacquet. Had tried to create a large ARC on this theme • Altman, Silva, Bernhard"The Mathematics of routing in Massively Dense Ad-Hoc Networks", 7th International Conference on Ad-Hoc Networks and Wireless, Sept, 2008, Sophia Antipolis, France. • Silva, Bernhard, Altman"Numerical solutions of Continuum Equilibria for Routing in Dense Ad-hoc Networks", Inter-Perf : Workshop on Interdisciplinary Systems Approach in Performance Evaluation and Design of Computer and Communication Systems, Athens, Oct 2008. • Altman, Silva, Bernhard, Debbah “Continuum Equilibria for Routing in Dense Static Ad-hoc Networks”, Computer Networks, 2009 Previous methodologies (last 10 years):Geometric Optics, Electrostatics. Our contribution:Formulated theh problem as a game with a continuum of players and of strategies. Inspired by models in road traffic Joint work between Maestro (INRIA) Nice Polytech, Supelec

13. Impact • 62 papers cited in a survey in Computer Networks (2008) by Toumpis. He writes: “However, it is regrettable that the connection was not made until very recently, in [16], and until thenthe research activities of the wireless networking community had been totally independent of the previous results of road traffic engineers”. [16] Our first paper (conference version of our Computer Networks paper).

14. 3. Evolutionary games in Biology and Engineering • BIOLOGY CONTEXT: Central tool defined by Meynard Smith (1972) for explaining and predicting dynamics of large competing populations with many limited local interactions. • TELECOM CONTEXT: Competition between protocols, technologies. Can be used to design and regulate evolution

15. Classical Framework • Large population • Several strategies (behavior of individuals). Call all those who use a strategy a subpopulation • Competition between the strategies through a very large number of interactions each involving a small number of individuals • Typical framework of pairwize interactions

16. Ex 1: Hawk and Dove Game • Large population of animals. Occasionally two animal find themselves in competition on the same piece of food. An animal can adopt an aggressive behavior (Hawk) or a peaceful one (Dove). D-D: peaceful, equal-sharing of the food. fitness of 0.5 to each player. H-D or D-H: 0 fitness to D and 1 for H that gets all the food no fight

17. HD Game • H-H: fight in which with equal chances to obtain the food but also to be wounded. Then the fitness of each player is 0.5-d, -d is the expected loss of fitness due to being injured.

18. Ex 2: Competition between protocols • There are various flow control protocols to regulate traffic in the Internet. • Huge number of file transfers every second • Interactions occur between limited number of connections that use the same bottleneck link • The average speed of transfer, the delay etc depend on the versions of the protocol involved in the interaction

19. How to predict evolution? • 1st Approach: Analyse which type performs better when interacting with each other • 2nd Approach: Imagine a world with only one type of protocole, and check which world is better Evolujtionary Game (EG) approach show: The evolution is a function of both

20. Guidelines for upgrade • Upgrading a protocol occurs at a time scale of 3 years • (when purchasing a new computer). The delay may cause instabilities. • We proposed guidelines for upgrades so as to avoid instability

21. Ex 3: Wireless communications • Cellular network contains many mobiles and many cells. One base station per cell • At each time an individual wishes to send a packet it may interact with other (small ) number of mobiles in the same cell • A mobile can transmit with high or low power. Higher power is costly • Two or more simultaneous transmissions collide. A packet is successfully transmitted at power p if it is the only one transmitted at power p or higher

22. New theoretical results on EG • Adaptation to the case of more players involved in local interactions. • Possibly random number of players • Consider non reciprocal interactions.

23. Publications: Evolutionary Games In Red: joint work between Maestro (INRIA) and LIA (Avignon) • H. Tembine, E. Altman , R. El-Azouzi and Y. Hayel, , "Evolutionary games with random number of interacting players applied to access control", , WIOPT, Berlin, April 2, 2008 • E. Altman, R. El-Azouzi, Y. Hayel and H. Tembine,"Evolutionary power control games in wireless networks", Networking, Singapore, 2008. • E. Altman, R. El-Azouzi, Y. Hayel and H. Tembine, "An Evolutionary Game approach for the design of congestion cont protocols in wireless networks", Physicomnet workshop, Berlin, April 4, 2008. • P. Coucheney et C. Touati. Replicator Dynamics Based Adaptive Algorithm for Heterogeneous Wireless Systems. Proceedings of the 13th International Symposium on Dynamic Games and Applications, (2008). • P. Bernhard, ESS, population games, replicator dynamics: dynamics and games if not dynamic games, Keynote talk in the 13th Symposium on Dynamic Games and Applications Annals of Dynamic Games, 2009. • P. Coucheney, C. Touati et B. Gaujal. «Fair and Efficient User-Network Association Algorithm for Multi-Technology Wireless Networks» Proc. of the 28th IEEE INFOCOM, 2009. • E. Altman, R. El-Azouzi, Y. Hayel and H. Tembine, The Evolution of Transport Protocols: An Evolutionary Game Perspective", Computer Networks, 53(10), 2009, 1751-1759. • J. Hofbauer, S. Sorin and Y. Viossat, Time average replicator and best reply dynamics, Mathematics of Operations Research, 34, 263-269 2009. • H. Tembine, E. Altman, R. El-Azouzi and Y. Hayel, "Evolutionary Games in Wireless Networks", IEEE Transactions on Systems, Man, and Cybernetics: Part B, special issue on Game Theory. 2009.

24. 4. Markov Decision EG: Individual States • Different behavior may be a result of different inherent characteristics – individual states • Example: weather conditions, age, • The individual state can be random • Description through a Markov chain • Local interactions with players chosen at random; their state is unknown

25. Indiv. states in HD Game • The decisions H or D determine whether a fight will occur • There is also a true identity -- Strong or Weak We call this the individual STATE • If there is a fight then the states determine the outcome. • Note: the decision H/D are taken without knowing the state of the other.

26. Individual States in Networks • Flow control protocol: large end to end delay slows the protocol and decreases its throughput • Wireless: - the power received may depend on the radio channel conditions - the transmitted power may depend on the battery

27. Markov Decision Evol Game • Each player has a controlled Markov chain (MDP) • A player has finite or infinite life time. It has several interactions each time with another randomly selected player • Each local interaction results in an immediate fitness that depends on the actions and states of the players involved • The states and actions determine also the probability distribution of the next state

28. Ex 1: Hawk and Dove game • A bird that looses becomes weaker (less energy) • A weaker bird has less chances to win a fight, or may not even be able to fight • A very weak birdr dies • State: Energy level • Would a weaker bird be more or less aggressive? • Here the transitions are determined by states and actions of both birds. Alternatively: a bird that fights becomes weaker (wounded). A very wounded bird dies.

29. Ex 2: Battery dependent power control • Transmitting at higher power empties faster the battery • A battery with little energy left is not able to support transmissions at high power • The state: remaining energy in the battery • The transitions do not depend on other mobiles

30. Ex 3: channel dependent power control • Assume expected average power constraints for each mobile • The decision to transmit at a given power may depend on the channel state • Seems “degenerate”: the mobile does not control the transitions • Restriction: discrete power set; if a power level is chosen then the next power cannot differ by more than one unit. • This creates non-trivial transitions. The state = (Channel state, current power level)

31. Markov Decision Evolutionary Games In Red: joint work between Maestro (INRIA) and LIA (Avignon) • E. Altman, Y. Hayel, H. Tembine, R. El-Azouzi, "Markov decision Evolutionay Games with Time Average Expected Fitness Criterion", Valuetools, Athens, October, 2008. • E. Altman and Y. Hayel, "A Stochastic Evolutionary Game Approach to Energy Management in a Distributed Aloha Network", IEEE INFOCOM, April 2008. • E. Altman and Y. Hayel, "Stochastic Evolutionary Games", Proceedings of the 13th Symposium on Dynamic Games and Applications, 30th June-3rd July, 2008. • P. Wiecek, E. Altman and Y. Hayel, "An Anonymous Sequential Game Approach for Battery State Dependent Power Control", NET-COOP, Eurandom, the Netherlands, Nov 2009. • H. Tembine, Y. Le Boudec, R. El-Azouzi, E. Altman"Mean Field Asymptotics of Markov Decision Evolutionary Games and Teams", Gamenets, May 2009, Istanbul, Turkey. • Y. Hayel, H. Tembine, E. Altman and R. El-Azouzi"A Markov Decision Evolutionary Game for Individual Energy Management", Annals of the International Society of Dynamic Games, 2009

32. Post Doc: Evolution by learningUse of Stochastic Approximation Tools • Sabir, El-Azouzi, Kavitha, Hayel and Bouyakhf, "Stochastic Learning Solution for Constrained Nash Equilibrium Throughput in Non Saturated Wireless Collision Channels" GameCom 2009 . (LIA) • Ramanath, Altman, Kumar, Kavitha, Thomas, "Fair assignment of base stations in cellular networks", 22nd World Wireless Research Forum (WWRF) Conference, May 5-7, 2009, Paris, France. (INRIA) • V. Kavitha, E. Altman, R. El-Azouzi, R. Sundareshan, Opportunistic scheduling in cellular systems in the presence of non-cooperative mobiles. IEEE CDC 2009, Beijing, China. Joint work Maestro-LIA

33. Other publications • H. Kameda, E. Altman, C. Touati and A. Legrand, "Nash Equilibrium Based Fairness", GameNets International Conference on Game Theory for Networks, May 2009, Bogazici University, Istanbul, Turkey. Joint work Maestro-Mescal • P. Bernhard, "Nonzero-sum dynamic games in the management of biological systems", Third International Conference on Game Theory and Management, St Petersburg, Russia, 2009. • N. Champagnat, Large deviations for singular and degenerate diffusion models in adaptive evolution.To appear in Markov Processes and Related Fields (2009). • N. Champagnat, R. Ferrière and S. Méléard. From individual stochastic processes to macroscopic models in adaptive evolution. Stochastic Models, Suppl. 1 of Vol. 24 No. 4, pp 2-44, 2008.