Intrinsic Robustness of the Price of Anarchy

1. Intrinsic Robustness of the Price of Anarchy Tim Roughgarden Stanford University

2. The Mathematical Model • a directed graph G = (V,E) • k source-destination pairs (s1 ,t1), …, (sk ,tk) • a rate (amount) ri of traffic from si to ti • for each edge e, a cost function ce(•) • assumed nonnegative, continuous, nondecreasing Example: (k,r=1) c(x)=x Flow = ½ s1 t1 c(x)=1 Flow = ½

3. s t Routings of Traffic Traffic and Flows: • fP = amount of traffic routed on si-ti path P • flow vector f routing of traffic Selfish routing: what are the equilibria?

4. Nash Flows Some assumptions: • agents small relative to network (nonatomic game) • want to minimize cost of their path Def: A flow is at Nash equilibrium (or is a Nash flow) if all flow is routed on min-cost paths [given current edge congestion] Example: Flow = 1 Flow = .5 x x s t s t 1 1 Flow = .5 Flow = 0

5. History + Generalizations • model, defn of Nash flows by [Wardrop 52] • Nash flows exist, are (essentially) unique • due to [Beckmann et al. 56] • general nonatomic games: [Schmeidler 73] • congestion game (payoffs fn of # of players) • defined for atomic games by [Rosenthal 73] • previous focus: Nash eq in pure strategies exist • potential game (equilibria as optima) • defined by [Monderer/Shapley 96]

6. The Cost of a Flow Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate) x ½ s t ½ 1 Cost = ½•½ +½•1 = ¾

7. s t The Cost of a Flow Def: the cost C(f) of flow f = sum of all costs incurred by traffic (avg cost × traffic rate) Formally: ifcP(f) = sum of costs of edges of P (w.r.t. the flow f), then: C(f) = P fP •cP(f) x ½ s t ½ 1 Cost = ½•½ +½•1 = ¾

8. Inefficiency of Nash Flows Note: Nash flows do not minimize the cost • observed informally by [Pigou 1920] • Cost of Nash flow = 1•1 + 0•1 = 1 • Cost of optimal (min-cost) flow = ½•½ +½•1 = ¾ • Price of anarchy := Nash/OPT ratio = 4/3 x 1 ½ s t 1 0 ½

9. ½ ½ x 1 s t ½ ½ x 1 Braess’s Paradox Initial Network: cost = 1.5

10. ½ ½ x 1 s t ½ ½ x 1 Braess’s Paradox Initial Network: Augmented Network: ½ ½ x 1 0 s t ½ ½ x 1 cost = 1.5 Now what?

11. ½ ½ x 1 s t ½ ½ x 1 Braess’s Paradox Initial Network: Augmented Network: x 1 0 s t x 1 cost = 1.5 cost = 2

12. ½ ½ x 1 s t ½ ½ x 1 Braess’s Paradox Initial Network: Augmented Network: All traffic incurs more cost![Braess 68] • see also [Cohen/Horowitz 91], [Roughgarden 01] x 1 0 s t x 1 cost = 1.5 cost = 2

13. xd 1 1-Є s t 1 0 Є The Bad News Bad Example: (r = 1, d large) Nash flow has cost 1, min cost0  Nash flow can cost arbitrarily more than the optimal (min-cost) flow • even if cost functions are polynomials

14. Linear Cost Functions First:focus on special case. Def:linear cost fn is of form ce(x)=aex+be Theorem:[Roughgarden/Tardos 00] for every network with linear cost fns: ≤ 4/3 × i.e., price of anarchy ≤ 4/3 in the linear case. cost of Nash flow cost of opt flow

15. x 1 ½ s t 1 ½ 0 Sources of Inefficiency Corollary of previous Theorem: • For linear cost fns, worst Nash/OPT ratio is realized in a two-link network! • simple explanation for worst inefficiency • confronted w/two routes, selfish users overcongest one of them • Cost of Nash = 1 • Cost of OPT = ¾

16. Simple Worst-Case Networks Theorem:[Roughgarden 02]fix any class of cost fns, and the worst Nash/OPT ratio occurs in a two-node, two-link network. • under mild assumptions • inefficiency of Nash flows always has simple explanation; simple networks are worst examples

17. Simple Worst-Case Networks Theorem:[Roughgarden 02]fix any class of cost fns, and the worst Nash/OPT ratio occurs in a two-node, two-link network. • under mild assumptions • inefficiency of Nash flows always has simple explanation; simple networks are worst examples Proof Idea: Nash flows minimize potential function • potential function “close” to total cost function

18. Computing the Price of Anarchy Application: worst-case examples simple  worst-case ratio is easy to calculate Example: polynomials with degree ≤ d, nonnegative coeffs  POA ≈ d/log d • quartic functions: worst-case POA ≈ 2 • 10% extra "capacity": worst-case POA ≈ 2

19. But Are We at Equilibrium? Since 2002:price of anarchy (i.e., worst Nash/OPT ratio) analyzed in many models. Critique: Usual interpretation of a POA bound presumes players reach equilibrium. .

20. But Are We at Equilibrium? Since 2002:price of anarchy (i.e., worst Nash/OPT ratio) analyzed in many models. Critique: Usual interpretation of a POA bound presumes players reach equilibrium. Soln #1: Justify via convergence theorems. Soln #2: [taken here] Prove bounds for much bigger sets than just Nash equilibria.

21. Weaker Equilibrium Concepts no regret correlated eq mixed Nash pure Nash

22. Weaker Equilibrium Concepts no regret correlated eq mixed Nash pure Nash best- response dynamics

23. Main Result (Informal) Informal Theorem:[Roughgarden 09]under “surprisingly general” conditions, a bound on the price of anarchy (for pure Nash) extends automatically to all 5 bigger sets. Example Application:selfish routing games (nonatomic or atomic) with cost functions in an arbitrary fixed set.

24. The Setup • n players, each picks a strategy si • player i incurs a cost Ci(s) Important Assumption: objective function is cost(s) := i Ci(s) Next: generic template for upper bounding price of anarchy of pure Nash equilibria. • notation: s = a Nash eq; s* = an optimal

25. An Upper Bound Template Suppose we have: cost(s) = i Ci(s) [defn of cost] ≤ i Ci(s*i,s-i) [s a Nash eq]

26. An Upper Bound Template Suppose we have: cost(s) = i Ci(s) [defn of cost] ≤ i Ci(s*i,s-i) [s a Nash eq] ≤ λ●cost(s*) + μ●cost(s) [(*)] Then: POA (of pure Nash eq) ≤ λ/(1-μ).

27. An Upper Bound Template Suppose we have: cost(s) = i Ci(s) [defn of cost] ≤ i Ci(s*i,s-i) [s a Nash eq] ≤ λ●cost(s*) + μ●cost(s) [(*)] Then: POA (of pure Nash eq) ≤ λ/(1-μ). Definition: A game is (λ,μ)-smooth if (*) holds for every pair s,s* outcomes. • not only when s is a pure Nash eq!

28. Main Result #1 Examples: selfish routing, linear cost fns. • every nonatomic game is (1,1/4)-smooth • every atomic game is (5/3,1/3)-smooth

29. Main Result #1 Examples: selfish routing, linear cost fns. • every nonatomic game is (1,1/4)-smooth • every atomic game is (5/3,1/3)-smooth Theorem 1: in a (λ,μ)-smooth game, expected cost of each outcomes in the 5 sets above is at most λ/(1-μ). • such a POA bound “automatically” far more general

30. Illustration So: in every (λ,μ)-smooth game with a sum objective, inefficiency of outcomes in the 5 sets looks like: worst pure Nash worst correlated equilibium worst mixed Nash worst no regret sequence optimal outcome 1 λ/(1-μ)

31. Main Result #2 Theorem 2 (informal): in sufficiently rich classes of games, smoothness arguments suffice for a tight worst-case bound (even for pure Nash equilibria). pure Nash correlated equilibium no regret sequence optimal outcome mixed Nash 1 λ/(1-μ) for tightest choice of λ,μ

32. Special Case of Result #1 Definition: a sequence s1,s2,...,sTof outcomes is no-regret if: • for each player i, each fixed action qi: • average cost player i incurs over sequence no worse than playing action qievery time • simple hedging strategies can be used by players to enforce this (for suff large T) Result: in a (λ,μ)-smooth game, average cost of every no-regret sequence at most λ/(1-μ) cost of optimal outcome.

33. Take-Home Points • guarantees on equilibrium quality possible in interesting problem domains • the most common way of proving such bounds automatically yields a much more robust guarantee • and this technique often gives tight bounds Future research agenda: broader understanding of performance guarantees for adaptive systems.