Time Scales in Evolutionary Dynamics

1. Time Scales in Evolutionary Dynamics Angel Sánchez Grupo Interdisciplinar de Sistemas Complejos (GISC) Departamento de Matemáticas – Universidad Carlos III de Madrid Instituto de Biocomputación y Física de Sistemas Complejos (BIFI) Universidad de Zaragoza with Carlos P. Roca and José A. Cuesta

2. Cooperation: the basis of human societies Anomaly in the animal world: • Occurs between genetically unrelated individuals

3. Cooperation: the basis of human societies Anomaly in the animal world: • Shows high division of labor

4. Cooperation: the basis of human societies Anomaly in the animal world: • Valid for large scale organizations… …as well as hunter-gatherer groups

5. Cooperation: the basis of human societies Some animals form complex societies… …but their individuals are genetically related

6. Altruism: key to cooperation Altruism: fitness-reducing act that benefits others Pure altruism is ruled out by natural selection acting on individualsá la Darwin

7. How did altruism arise? He who was ready to sacrifice his life (…), rather than betray his comrades, would often leave no offspring to inherit his noble nature… Therefore,it seems scarcely possible(…)that the number of men gifted with such virtues(…) would be increased by natural selection, that is, by the survival of the fittest. Charles Darwin (Descent of Man, 1871)

8. Altruism is an evolutionary puzzle

9. Group selection? Cultural evolution? A man who was not impelled by any deep, instinctive feeling, to sacrifice his life for the good of others, yet was roused to such actions by a sense of glory, would by his example excite the same wish for glory in other men, and would strengthen by exercise the noble feeling of admiration.He might thus do far more good to his tribethan by begetting offspringswith a tendency to inherit his own high character. Charles Darwin (Descent of Man, 1871)

10. Answers to the puzzle… • Kin cooperation(Hamilton, 1964) common to animals and humans alike • Reciprocal altruism in repeated interactions(Trivers, 1973; Axelrod & Hamilton, 1981) primates, specially humans • Indirect reciprocity (reputation gain)(Nowak & Sigmund, 1998) primates, specially humans None true altruism: individual benefits in the long run

11. … but only partial! • Strong reciprocity (Gintis, 2000; Fehr, Fischbacher & Gächter, 2002) typically human (primates?) • altruistic rewarding:predisposition to reward others for cooperative behavior • altruistic punishment:propensity to impose sanctions on non-cooperators Strong reciprocators bear the cost of altruistic acts even if they gain no benefits Hammerstein (ed.), Genetic and cultural evolution of cooperation (Dahlem Workshop Report 90, MIT, 2003)

12. One of the 25 problems for the XXI century: E. Pennisi, Science309, 93 (2005) “Others with a more mathematical bent are applying evolutionary game theory, a modeling approach developed for economics, to quantify cooperation and predict behavioral outcomes under different circumstances.”

13. Evolution • There are populations of reproducing individuals • Reproduction includes mutation • Some individuals reproduce faster than other (fitness). This results in selection Game theory • Formal way to analyze interactions between agents who behave strategically (mathematics of decision making in conflict situations) • Usual to assume players are “rational” • Widely applied to the study of economics, warfare, politics, animal behaviour, sociology, business, ecology and evolutionary biology

14. Evolutionary Game Theory Successful strategies spread by natural selection Payoff = fitness • Everyone starts with a random strategy • Everyone inpopulationplays game against everyone else • Population is infinite • Payoffs are added up • Total payoff determines the number of offspring:Selection • Offspring inherit approximately the strategy of their parents:Mutation John Maynard Smith 1972 (J.B.S. Haldane, R. A. Fisher, W. Hamilton, G. Price)

15. Quasispecies equation replicator-mutator Price equation Replicator-mutator equation Price equation Lotka-Volterra equation Game dynamical equation replicator Price equation Adaptive dynamics Equations for evolutionary dynamics

16. Case study on strong reciprocity and altruistic behavior: Ultimatum Games, altruism and individual selection

17. OK NO u M-u The Ultimatum Game (Güth, Schmittberger & Schwarze, 1982) experimenter M euros proposer M-u 0 0 u responder

18. Experimental results Extraordinary amount of data Camerer, Behavioral Game Theory(Princeton University Press, 2003) “At this point, we should declare a moratorium on creating ultimatum game data and shift attention towards new games and new theories.” Henrich et al. (eds.), Foundations of Human Sociality : Economic Experiments and Ethnographic Evidence from Fifteen Small-Scale Societies(Oxford University Press, 2004)

19. What would you offer?

20. Rational responder’s optimal strategy: accept anything Rational proposer’s optimal strategy: offer minimum Experimental results • Proposers offer substantial amounts (50% is a typical modal offer) • Responders reject offers below 25% with high probability • Universal behavior throughout the world • Large degree of variability of offers among societies (26 - 58%)

21. ...... Nplayers M monetary units (M=100) ti , oi: thresholds (minimum share player i accepts / offers) fi : fitness (accumulated capital) playeri A.S. & J. A. Cuesta, J. Theor. Biol. 235, 233 (2005) Model

22. op tr op tr < ≥ Game event ...... Nplayers responder proposer tr fr op fp +op +M-op

23. Reproduction event (after s games) ...... Nplayers new player minimum fitness maximum fitness t, omax fmax t, omin fmin t’, o’max fmax (prob.=1/3) mutation: t’, o’max=t, omax± 1

24. Slow evolution (large s) N =1000, 109 games, s = 105, ti= oi=1 initial condition accept offer

25. Fast evolution (small s) N =1000, 106 games, s =1, uniform initial condition accept offer

26. Adaptive dynamics (“mean-field”) results Results for small s (fast selection) differ qualitatively Implications in behavioral economics and evolutionary ideas on human behavior!

27. Selection/reproduction interplay in simpler settings: Equilibrium selection in 2x2 games

28. P. A. P. Moran, The statistical processes of evolutionary theory (Clarendon, 1962) Moran Process Select one, proportional to fitness Substitute a randomly chosen individual Game event 2x2 game Choose s pairs of agents to play the game between reproduction events Reset fitness after reproduction C. P. Roca, J. A. Cuesta, A. Sánchez, Phys. Rev. Lett.97, 158701 (2006)

29. Fixation probability Probability to reach state N when starting from state i =1 1-x1 x1 Absorbing states

30. Fixation probability Probability to reach state N when starting from state n

31. Fixation probability Probability to reach state N when starting from state n

32. Fixation probability Probability to reach state N when starting from state n Number of games s enters through transition probabilities

33. Fixation probability Probability to reach state N when starting from state n Fitness:possible game sequences times corresponding payoffs per population

34. Example 1: Harmony game Payoff matrix: Unique Nash equilibrium in pure strategies: (C,C) (C,C) is the only reasonable behavior anyway

35. Example 1: Harmony game s infinite (round-robin, “mean-field”)

36. Example 1: Harmony game s = 1 (reproduction following every game)

37. Example 1: Harmony game Consequences Round-robin: cooperators are selected One game only: defectors are selected! Result holds for any population size In general: for any s, numerical evaluation of exact expressions

38. Example 1: Harmony game Numerical evaluation of exact expressions

39. Example 2: Stag-hunt game Payoff matrix: Two Nash equilibria in pure strategies: (C,C), (D,D) Equilibrium selection depends on initial condition

40. Example 2: Stag-hunt game Numerical evaluation of exact expressions

41. Example 3: Snowdrift game Payoff matrix: One mixed equilibrium Replicator dynamics goes always to mixed equilibrium Moran dynamics does not allow for mixed equilibria

42. Example 3: Snowdrift game Numerical evaluation of exact expressions

43. Example 3: Snowdrift game Numerical evaluation of exact expressions s = 5 s = 100

44. Example 4: Prisoner’s dilemma Payoff matrix: Unique Nash equilibrium in pure strategies: (C,C) Paradigm of the emergence of cooperation problem

45. Example : Prisoner’s dilemma Numerical evaluation of exact expressions

46. Results are robust Increasing system size does not changes basins of attrractions, only sharpens the transitions Small s is like an effective small population, because inviduals that do not play do not get fitness Introduce background of fitness: add fb to all payoffs

47. Background of fitness: Stag-hunt game Numerical evaluation of exact expressions fb = 0.1 fb = 1

48. Conclusions • In general, evolutionary game theory studies a limit situation: s infinite! (every player plays every other one before selection) • Number of games per player may be finite, even Poisson distributed • Fluctuations may keep players with smaller ‘mean-field’ fitness alive • Changes to equilibrium selection are non trivial and crucial New perspective on evolutionary game theory:more general dynamics, dictated by the specific application (change focus from equilibrium selection problems)

49. Time Scales in Evolutionary Dynamics A. Sánchez & J. A. Cuesta, J. Theor. Biol. 235, 233 (2005) A. Sánchez, J. A. Cuesta & C. P. Roca, in “Modeling Cooperative Behavior in the Social Sciences”, eds. P. Garrido, J. Marro & M. A. Muñoz, 142–148. AIP Proceedings Series (2005). C. P. Roca, J. A. Cuesta, A. Sánchez, arXiv:q-bio/0512045 (submitted to European Physical Journal Special Topics) C. P. Roca, J. A. Cuesta, A. Sánchez, Phys. Rev. Lett.97, 158701 (2006)