Introduction to Game Theory and its Applications in Computer Networks

1. Introduction to Game Theory and its Applications in Computer Networks John C.S. Lui Dept. of Computer Science & Engineering The Chinese University of Hong Kong Daniel R. Figueiredo School of Computer and Communication Sciences Swiss Federal Institute of Technology – Lausanne (EPFL) ACM SIGMETRICS / IFIP Performance June 2006

2. Tutorial Organization • Two parts of 90 minutes • 15 minutes coffee break in between • First part: introduction to game theory • definitions, important results, (simple) examples • divided in two 45 minutes sessions (Daniel + John) • Second part: game theory and networking • game-theoretic formulation of networking problems • 1st 45 minute session (Daniel) • routing games and congestion control games • 2nd 45 minute session (John) • overlay games and wireless games

3. 2 2 What is Game Theory About? • Analysis of situations where conflict of interests are present • Game of Chicken • driver who steers away looses • What should drivers do? • Goal is to prescribe how conflicts can be resolved

4. Applications of Game Theory • Theory developed mainly by mathematicians and economists • contributions from biologists • Widely applied in many disciplines • from economics to philosophy, including computer science (Systems, Theory and AI) • goal is often to understand some phenomena • “Recently” applied to computer networks • Nagle, RFC 970, 1985 • “datagram networks as a multi-player game” • paper in first volume of IEEE/ACM ToN (1993) • wider interest starting around 2000

5. Limitations of Game Theory • No unified solution to general conflict resolution • Real-world conflicts are complex • models can at best capture important aspects • Players are (usually) considered rational • determine what is best for them given that others are doing the same • No unique prescription • not clear what players should do • But it can provide intuitions, suggestions and partial prescriptions • best mathematical tool we currently have

6. What is a Game? • A Game consists of • at least two players • a set of strategies for each player • a preference relation over possible outcomes • Player is general entity • individual, company, nation, protocol, animal, etc • Strategies • actions which a player chooses to follow • Outcome • determined by mutual choice of strategies • Preference relation • modeled as utility (payoff) over set of outcomes

7. Classification of Games • Many, many types of games • three major categories • Non-Cooperative (Competitive) Games • individualized play, no bindings among players • Repeated and Evolutionary Games • dynamic scenario • Cooperative Games • play as a group, possible bindings

8. Matrix Game (Normal form) • Representation of a game Strategy set for Player 2 • Simultaneous play • players analyze the game and write their strategy on a paper • Combination of strategies determines payoff Strategy set for Player 1 Player 2 Player 1 Payoff to Player 1 Payoff to Player 2

9. More Formal Game Definition • Normal form (strategic) game • a finite set N of players • a set strategies for each player • payoff function for each player • where is the set of strategies chosen by all players • A is the set of all possible outcomes • is a set of strategies chosen by players • defines an outcome

10. Two-person Zero-sum Games • One of the first games studied • most well understood type of game • Players interest are strictly opposed • what one player gains the other loses • game matrix has single entry (gain to player 1) • Intuitive solution concept • players maximize gains • unique solution

11. Analyzing the Game • Player 1 maximizes matrix entry, while player 2 minimizes Player 2 Player 1 Strictly dominated strategy (dominated by C) Strictly dominated strategy (dominated by B)

12. Dominance • Strategy S strictly dominates a strategy T if every possible outcome when S is chosen is better than the corresponding outcome when T is chosen • Dominance Principle • rational players never choose strictly dominated strategies • Idea: Solve the game by eliminating strictly dominated strategies! • iterated removal

13. Solving the Game • Iterated removal of strictly dominated strategies Player 2 Player 1 • Player 1 cannot remove any strategy (neither T or B dominates the other) • Player 2 can remove strategy R (dominated by M) • Player 1 can remove strategy T (dominated by B) • Player 2 can remove strategy L (dominated by M) • Solution: P1 -> B, P2 -> M • payoff of 2

14. Solving the Game • Removal of strictly dominates strategies does not always work • Consider the game Player 2 Player 1 • Neither player has dominated strategies • Requires another solution concept

15. Analyzing the Game Player 2 Player 1 Outcome (C, B) seems “stable” • saddle point of game

16. Saddle Points • An outcome is a saddle point if it is both less than or equal to any value in its row and greater than or equal to any value in its column • Saddle Point Principle • Players should choose outcomes that are saddle points of the game • Value of the game • value of saddle point outcome if it exists

17. Why Play Saddle Points? Player 2 • If player 1 believes player 2 will play B • player 1 should play best response to B (which is C) • If player 2 believes player 1 will play C • player 2 should play best response to C (which is B) Player 1

18. Powerful arguments to play saddle point! Why Play Saddle Points? Player 2 • Why should player 1 believe player 2 will play B? • playing B guarantees player 2 loses at most v (which is 2) • Why should player 2 believe player 1 will play C? • playing C guarantees player 1 wins at least v (which is 2) Player 1

19. Solving the Game (min-max algorithm) Player 2 • choose minimum entry in each row • choose the maximum among these • this is maximin value Player 1 • choose maximum entry in each column • choose the minimum among these • this is the minimax value • if minimax == maximin, then this is the saddle point of game

20. Multiple Saddle Points • In general, game can have multiple saddle points Player 2 Player 1 • Same payoff in every saddle point • unique value of the game • Strategies are interchangeable • Example: strategies (A, B) and (C, C) are saddle points then (A, C) and (C, B) are also saddle points

21. Games With no Saddle Points • What should players do? • resort to randomness to select strategies Player 2 Player 1

22. Mixed Strategies • Each player associates a probability distribution over its set of strategies • players decide on which prob. distribution to use • Payoffs are computed as expectations Player 1 Payoff to P1 when playing A = 1/3(4) + 2/3(0) = 4/3 Payoff to P1 when playing B = 1/3(-5) + 2/3(3) = 1/3 • How should players choose prob. distribution?

23. Mixed Strategies • Idea: use a prob. distribution that cannot be exploited by other player • payoff should be equal independent of the choice of strategy of other player • guarantees minimum gain (maximum loss) • How should Player 2 play? Player 1 Payoff to P1 when playing A = x(4) + (1-x)(0) = 4x Payoff to P1 when playing B = x(-5) + (1-x)(3) = 3 – 8x 4x = 3 – 8x, thus x = 1/4

24. Mixed Strategies • Player 2 mixed strategy • 1/4 C , 3/4 D • maximizes its loss independent of P1 choices • Player 1 has same reasoning Player 2 Player 1 Payoff to P2 when playing C = x(-4) + (1-x)(5) = 5 - 9x Payoff to P2 when playing D = x(0) + (1-x)(-3) = -3 + 3x 5 – 9x = -3 + 3x, thus x = 2/3 Payoff to P2 = -1

25. Minimax Theorem • Every two-person zero-sum game has a solution in mixed (and sometimes pure) strategies • solution payoff is the value of the game • maximin = v = minimax • v is unique • multiple equilibrium in pure strategies possible • but fully interchangeable • Proved by John von Neumann in 1928! • birth of game theory…

26. Two-person Non-zero Sum Games • Players are not strictly opposed • payoff sum is non-zero Player 2 Player 1 • Situations where interest is not directly opposed • players could cooperate

27. What is the Solution? • Ideas of zero-sum game: saddle points • mixed strategies equilibrium • no pure strategy eq. • pure strategy equilibrium Player 2 Player 2 Player 1 Player 1

28. Multiple Solution Problem • Games can have multiple equilibria • not equivalent: payoff is different • not interchangeable: playing an equilibrium strategy does not lead to equilibrium Player 2 Player 1 equilibria

29. The Good News: Nash’s Theorem • Every two person game has at least one equilibrium in either pure or mixed strategies • Proved by Nash in 1950 using fixed point theorem • generalized to N person game • did not “invent” this equilibrium concept • Def: An outcome o* of a game is a NEP (Nash equilibrium point) if no player can unilaterally change its strategy and increase its payoff • Cor: any saddle point is also a NEP

30. The Prisoner’s Dilemma • One of the most studied and used games • proposed in 1950s • Two suspects arrested for joint crime • each suspect when interrogated separately, has option to confess or remain silent Suspect 2 payoff is years in jail (smaller is better) Suspect 1 better outcome single NEP

31. Pareto Optimal • Prisoner’s dilemma: individual rationality Suspect 2 Pareto Optimal Suspect 1 • Another type of solution: group rationality • Pareto optimal • Def: outcome o* is Pareto Optimal if no other outcome is better for all players

32. 2 2 Game of Chicken Revisited • Game of Chicken (aka. Hawk-Dove Game) • driver who swerves looses Driver 2 Drivers want to do opposite of one another Driver 1 Will prior communication help?

33. How much should firm i produce? Example: Cournot Model of Duopoly • Several firms produce exactly same product • : quantity produced by firm • Cost to firm i to produce quantity • Market clearing price (price paid by consumers) • where • Revenue of firm i

34. Example: Cournot Model of Duopoly • Consider two firms: • Simple production cost • no fixed cost, only marginal cost with constant c • Simple market (fixed demand a) • where • Revenue of firm • Firms choose quantities simultaneously • Assume c < a

35. Example: Cournot Model of Duopoly • Two player game: Firm 1 and Firm 2 • Strategy space • production quantity • since if , • What is the NEP? • To find NEP, firm 1 solves • To find NEP, firm 2 solves value chosen by firm 2 value chosen by firm 1

36. Example: Cournot Model of Duopoly • Solution to maximization problem • first order condition is necessary and sufficient and • Best response functions • best strategy for player 1, given choice for player 2 • At NEP, strategies are best response to one another • need to solve pair of equations • using substitution… and

37. Competition can be good! Example: Cournot Model of Duopoly • NEP is given by • Total amount produced at NEP: • Price paid by consumers at NEP: • Consider a monopoly (no firm 2, ) less quantity produced • Equilibrium is given by • Total amount produced: • Price paid by consumers: higher price

38. Example: Cournot Model of Duopoly • Graphical approach: best response functions • Plot best response for firm 1 • Plot best response for firm 2 NEP: strategies are mutual best responses • all intersections are NEPs

39. Game Trees (Extensive form) • Sequential play • players take turns in making choices • previous choices can be available to players • Game represented as a tree • each non-leaf node represents a decision point for some player • edges represent available choices • Can be converted to matrix game (Normal form) • “plan of action” must be chosen before hand

40. Game Trees Example Player 1 R L • Strategy set for Player 1: {L, R} Player 2 Player 2 Payoff to Player 1 R L R L Payoff to Player 2 -2, 1 3, 1 0, -1 1, 2 • Strategy for Player 2: __, __ what to do when P1 plays R what to do when P1 plays L • Strategy set for Player 2: {LL, LR, RL, RR}

41. More Formal Extensive Game Definition • An extensive form game • a finite set N of players • a finite height game tree • payoff function for each player • where s is a leaf node of game tree • Game tree: set of nodes and edges • each non-leaf node represents a decision point for some player • edges represent available choices (possibly infinite) • Perfect information • all players have full knowledge of game history

42. Microsoft java .net Mozilla Mozilla java .net java .net 0, 0 2, 2 1, 0 3, 1 Game Tree Example • Microsoft and Mozilla are deciding on adopting new browser technology (.net or java) • Microsoft moves first, then Mozilla makes its move • Non-zero sum game • what are the NEP?

43. java .net java .net java .net 0, 0 2, 2 1, 0 3, 1 Converting to Matrix Game • Every game in extensive form can be converted into normal form • exponential growth in number of strategies Mozilla Microsoft

44. java .net java .net java .net 0, 0 2, 2 1, 0 3, 1 NEP and Incredible Threats Mozilla NEP Microsoft incredible threat • Play “java no matter what” is not credible for Mozilla • if Microsoft plays .net then .net is better for Mozilla than java

45. java .net java .net java .net java .net 2, 2 3, 1 0, 0 2, 2 1, 0 3, 1 Solving the Game (backward induction) • Starting from terminal nodes • move up game tree making best choice Best strategy for Mozilla: .net, java (follow Microsoft) Equilibrium outcome Best strategy for Microsoft: .net • Single NEP Microsoft -> .net, Mozilla -> .net, java

46. Backward Induction on Game Trees • Kuhn’s Thr: Backward induction always leads to saddle point (on games with perfect information) • game value at equilibrium is unique (for zero-sum games) • In general, multiple NEPs are possible after backward induction • cases with no strict preference over payoffs • Effective mechanism to remove “bad” NEP • incredible threats

47. Mozilla java .net Microsoft Microsoft java .net java .net 0, 0 2, 2 0, 1 1, 3 Leaders and Followers • What happens if Mozilla is moves first? Mozilla: java Microsoft: .net, java • NEP after backward induction: • Outcome is better for Mozilla, worst for Microsoft • incredible threat becomes credible! • 1st mover advantage • but can also be a disadvantage…

48. Microsoft java .net Mozilla Mozilla java .net java .net 0, 0 2, 2 1, 0 3, 1 The Subgame Concept • Def: a subgame is any subtree of the original game that also defines a proper game • includes all descendents of non-leaf root node • 3 subtrees • full tree, left tree, right tree

49. Subgame Perfect Nash Equilibrium • Def: a NEP is subgame perfect if its restriction to every subgame is also a NEP of the subgame • Thr: every extensive form game has at least one subgame perferct Nash equilibrium • Kuhn’s theorem, based on backward induction • Set of NEP that survive backward induction • in games with perfect information

50. Microsoft java .net Mozilla Mozilla java .net java .net 0, 0 2, 2 1, 0 3, 1 Subgame Perfect Nash Equilibrium • (N, NN) is not a NEP when restricted to the subgame starting at J • (J, JJ) is not a NEP when restricted to the subgame starting at N • (N, NJ) is a subgame perfect Nash equilibrium J N Mozilla Subgame Perfect NEP MS Not subgame Perfect NEP