1 / 28

Punishment and Forgiveness in Repeated Games

Punishment and Forgiveness in Repeated Games. A review of present values. Calculating sums. In a repeated game, with probability d of continuation after each round, the probability that the game is still going at round k is d k-1 Calculate expected winnings if you receive

kohana
Download Presentation

Punishment and Forgiveness in Repeated Games

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Punishment and Forgivenessin Repeated Games

  2. A review of present values

  3. Calculating sums • In a repeated game, with probability d of continuation after each round, the probability that the game is still going at round k is dk-1 • Calculate expected winnings if you receive a fixed amount, R so long as the game continues. S = R+dR+d2R+ d3R+ d4R + ….+ S = R(1+d +d2 + d3 + d4 + ….+ ) • What is this infinite sum?

  4. Adding forever • The series 1+d +d2 + d3 + d4 + ….+ Is known as a geometric series. When |d|<1, this series converges. That is, to say, as n approaches infinity, the sum Tn =1+d +d2 + d3 + d4 + ….+ dnis bounded and has a limiting value S. Let S= 1+d +d2 + d3 + d4 + ….+ Then d × S= d +d2 + d3 + d4 + ….+ And S-(d × S)= 1. So S×(1-d)=1 S=1/(1-d).

  5. A generous friend offers to give you $10 and roll a fair die. If it comes up 6, she stops. If she rolls something other than 6, she will give you another $10 and roll again. She will continue to roll the die until she rolls a 6. Each time she rolls the die, she gives you another $10. What is the probability that she rolls the die at least 3 times? • 1/6×5/6×5/6 • 1/6×1/6×5/6 • 3×5/6 • 5/6×5/6×5/6 • 1/6×1/6×1/6

  6. What is the expected amount of money that you will get from your generous friend’s offer • $60 • $360 • $10 • $244 • $40

  7. Why so? • On the first roll, she will give you $10. With probability 5/6, she will do a second roll and give you another $10. With probability (5/6)2 she will not get a 6 on her first two rolls and give you a third $10. With probability (5/6)3she will not get a 6 on her first 3 rolls and you will win another $10, and so on…. • So your expected winnings are $10(1+5/6+(5/6)2 +…+(5/6)n +… )=$10×(1/(1-5/6) =$10×6=$60.

  8. Punishment and reward in repeated play. • Suppose we play Prisoners’s dilemma repeatedly with no known last round, but where there may be a next round with probability p<1. • We consider possibilities for rewards for “good behavior” and punishment for “bad behavior”.

  9. Prisoners’ Dilemma(the stage game) Player 2 Cooperate Defect P LAyER 1 Cooperate Defect T > R > P > S “Temptation” “Reward” “Punishment” “Sucker”

  10. Extreme Punishment:(the Grim Trigger Strategy) • In prisoner’s dilemma Grim Trigger is the following strategy. • Play cooperate on first play. So long as opponent plays cooperate, you cooperate. • But if cooperator ever defects, you defect on all remaining plays. • Grim trigger never forgives.

  11. When is there a symmetric SPNE where all play Grim Trigger? • Suppose that the other player is playing Grim Trigger. • If you play Grim Trigger as well, then you will cooperate as long as the game continues and and you will receive a payoff of R. If the probability that the game will continue after any round Is d, the expected payoff from playing Grim Trigger if the other guy is playing Grim Trigger is R×(1+d +d2 + d3 + d4 + ….+ )=R/(1-d)

  12. What if you defect against Grim Trigger • If you defect and the other guy is playing Grim Trigger, you will get a payoff of T>R the first time that you defect. But after this, the other guy will always play defect. The best you can do, then is to always defect as well. • Your expected payoff from defecting is therefore T+ P(d +d2 + d3 + d4 + ….+ ) =T+Pd/1-d

  13. Cooperate vs Defect • If other guy is playing Grim trigger and nobody has yet defected, your expected payoff from playing cooperate is R/(1-d) • If other guy is playing Grim trigger and nobody has yet defected, your expected payoff from playing defect is T+Pd/(1-d) • Cooperate is R/(1-d) better for you if R/(1-d)>T+Pd/(1-d) which implies d>(T-R)/(T-P) • Example If T=10, R=5, P=2, then condition is d>5/8. • If d is too small, it pays to “take the money and run”

  14. Other equilbria? • Grim trigger is a SPNE if d is large enough. • Are there other SPNEs? • Yes, for example both play Always Defect is an equilibrium. • If other guy is playing Always Defect, what is your best response in any subgame? • Another is Play Defect the first 10 rounds, then play Grim Trigger.

  15. Forgiveness • Does the grim trigger strategy have to be so unrelenting? • In the real world, why might it not be a good idea to have an unforgiving punishment? • What if you get a noisy signal about other player’s action? • What if other player made a one-time mistake? • This question is much wrestled with in religion and in politics.

  16. Tit for Tat-a more forgiving strategy • What is both players play the following strategy in infinitely repeated P.D? • Cooperate on the first round. Then on any round do what the other guy did on the previous round. • Suppose other guy plays tit for tat. • If I play tit for tat too, what will happen?

  17. Payoffs • If you play tit for tat when other guy is playing tit for tat, you get expected payoff of R(1+d +d2 + d3 + d4 + ….+ )=R/(1-d) • Suppose instead that you choose to play “Always defect” when other guy is tit for tat. • You will get T+ P(d +d2 + d3 + d4 + ….+ ) =T+Pd/1-d Same comparison as with Grim Trigger. Tit for tat is a better response to tit for tat than always defect if d>(T-R)/(T-P)

  18. Another try • Sucker punch him and then get him to forgive you. • If other guy is playing tit for tat and you play D on first round, then C ever after, you will get payoff of T on first round, S on second round, and then R for ever. Expected payoff is T+ Sd+d2R(1+d +d2 + d3 + d4 + ….+ )=T+ Sd+d2R/(1-d).

  19. Which is better? • Tit for tat and Cheat and ask forgiveness give same payoff from round 3 on. • Cheat and ask for forgiveness gives T in round 1 and S in round 2. • Tit for tat give R in all rounds. • So tit for tat is better if R+dR>T+dS, which means d(R-S)>T-R or d>(T-R)(R-S) If T=10, R=6, and S=1, this would mean if d>4/5. But if T=10, R=5, and S=1, this would be the case only if d>5/4, which can’t happen. In this case, tit for tat could not be a Nash equilibrium.

  20. A FISHERMAN’S CATCH AND PAYOFF Number of Number of Own Own Boats Other People’s PAYOFF Boats 1 2 25 1 3 20 1 4 15 2 2 45 2 3 35 2 4 20

  21. What is the Nash equilibrium for the stage game for the three fishermen? • All send one boat. • All send two boats. • There is more than one Nash equilibrium for the stage game. • There are no pure strategy Nash equilibria, but there is a mixed strategy Nash equilibrium for the stage game. • There are no pure or mixed strategy Nash equilibria for the stage game.

  22. Can efficiency be sustained by the Grim Trigger? • Suppose that the other two fishermen are playing the grim trigger strategy of sending one boat until somebody sends two boats and if anybody ever sends two boats, you send two boats ever after. • If you and the others play the grim trigger strategy, you will always send 1 boat and so will they.

  23. If others are playing grim trigger strategy, would you want to? • If you play grim trigger, you will always send 1 boat. Your payoff will be 25 in every period. Assume that a fisherman discounts later profits at rate d. Value of this stream is then 25(1+d+d2+d3 +…)=25(1/1-d) • If instead you send 2 boats, you will get payoff of 45 the first time, but only 20 thereafter. • Value of this stream is 45+ 20(d+d2+d3 +…) • Grim trigger is bigger if • 20<5 (d+d2+d3 +…) • This means 20<5d/(1-d) which implies d>4/5

  24. Problem 7 The stage game: • Payoff to player 1 is V1(x1,x2)=5+x1-2x2 • Payoff to player 2 is V2(x1,x2)=5+x2-2x1 • Strategy set for each player is the interval [1,4] What is a Nash equilibrium for the stage game?

  25. What is a Nash equilibrium for the stage game? • Both players choose 4 • Both players choose 3 • Both players choose 2 • Both players choose 1 • There is no pure strategy Nash equilibrium.

  26. Part b (i) • If the strategy set is X={2,3}, when is there a subgame perfect Nash equilibrium in which both players always play 2 so long as nobody has ever played anything else. • Compare payoff v(2,2) forever with payoff v(3,2) in first period, then v(3,3) ever after. • That is, compare 3 forever with 4 in the first period and then 2 forever.

  27. Part b(ii) X=[1,4] • When is there a subgame perfect equilibrium where everybody does y so long as nobody has ever done anything differently and everybody does z>y if anyone ever does anything other than y? • First of all, it must be that z=4. Because actions after a violation must be Nash for stage game. • When is it true that getting V(y,y) forever is better than getting V(4,y) in the first period and then V(4,4) forever.

  28. Comparison V(y,y) forever is worth V(y,y)/(1-d)=(5-y)/(1-d) V(4,y) and then V(4,4) forever is worth 9-y+1d+1d2+…=9-y+d/1-d) Works out that V(y,y)>V(4,y) if d(8-y)>4

More Related