Algorithmic game theory because a game is nice when it s not too long
This presentation is the property of its rightful owner.
Sponsored Links
1 / 61

Algorithmic Game Theory (because a game is nice when it’s not too long!) PowerPoint PPT Presentation


  • 60 Views
  • Uploaded on
  • Presentation posted in: General

Algorithmic Game Theory (because a game is nice when it’s not too long!). Guido Proietti Dipartimento di Informatica, Università degli Studi dell’Aquila & Istituto di Analisi dei Sistemi ed Informatica – CNR Roma. Roadmap. Nash Equilibria (NE) Does a NE always exist?

Download Presentation

Algorithmic Game Theory (because a game is nice when it’s not too long!)

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Algorithmic game theory because a game is nice when it s not too long

Algorithmic Game Theory(because a game is nice when it’s not too long!)

Guido Proietti

Dipartimento di Informatica, Università degli Studi dell’Aquila

&

Istituto di Analisi dei Sistemi ed Informatica – CNR Roma


Roadmap

Roadmap

  • Nash Equilibria (NE)

    • Does a NE always exist?

    • Can a NE be feasibly computed, once it exists?

    • Which is the “quality” of a NE?

    • How long does it take to converge to a NE?

  • Algorithmic Mechanism Design

    • Which social goals can be (efficiently) implemented in a non-cooperative selfish distributed system?

    • VCG-mechanisms and one-parameter mechanisms

    • Mechanism design for some basic network design problems


Algorithmic game theory because a game is nice when it s not too long

FIRST PART:

Nash equilibria


Two research traditions

Two Research Traditions

  • Theory of Algorithms: computational issues

    • What can be feasibly computed?

    • How much does it take to compute a solution?

    • Which is the quality of a computed solution?

    • Centralized or distributed computational models

  • Game Theory: interaction between self-interested individuals

    • What is the outcome of the interaction?

    • Which social goals are compatible with selfishness?


Different assumptions

Different Assumptions

  • Theory of Algorithms (in DCMs):

    • Processors are obedient, faulty, or adversarial

    • Large systems, limited comp. resources

  • Game Theory:

    • Players are strategic(selfish)

    • Small systems, unlimited comp. resources


The internet world

The Internet World

  • Agents often autonomous (users)

    • Users have their own individual goals

    • Network components owned by providers

  • Often involve “Internet” scales

    • Massive systems

    • Limited communication/computational resources

       Both strategic and computational issues!


Fundamental questions

Fundamental Questions

  • What are the computational aspects of a game?

  • What does it mean to design an algorithm for a strategic distributed system?

Algorihmic

Game Theory

Theory of Algorithms

Game Theory

=

+


Game theory

Game Theory

  • Given a game, predict the outcome by analyzing the individual behavior of the players (agents)

  • Game:

    • N players

    • Rules of encounter: Who should act when, and what are the possible actions

    • Outcomes of the game


Normal form games

Normal Form Games

  • N rational and non-cooperative players

  • Si =Strategy set of player i

  • The strategy combination (s1, s2, …, sN) gives payoff (p1, p2, …, pN) to the N players

  • All the above information is known to all the players and it is common knowledge

  • Simultaneous move: each player i chooses a strategy siSi(nobody can observe others’ move)


Equilibrium

Equilibrium

  • An equilibrium s*= (s1*, s2*, …, sN*) is a strategy combination consisting of a best strategy for each of the N players in the game

  • What is a best strategy? depends on the game…informally, it is a strategy that a players selects in trying to maximize his individual payoff, knowing that other players are also doing the same


Dominant strategy equilibrium prisoner s dilemma

Dominant Strategy Equilibrium: Prisoner’s Dilemma

Strategy

Set

Payoffs

Strategy Set


Prisoner i s decision

Prisoner I’s decision

  • Prisoner I’s decision:

    • If II chooses Don’t Implicate then it is best to Implicate

    • If II chooses Implicate then it is best to Implicate

    • It is best to Implicate for I, regardless of what II does: Dominant Strategy


Prisoner ii s decision

Prisoner II’s decision

  • Prisoner II’s decision:

    • If I chooses Don’t Implicatethen it is best to Implicate

    • If I chooses Implicatethen it is best to Implicate

    • It is best to Implicate for II, regardless of what I does: Dominant Strategy


Hence

Hence…

  • It is best for both to implicate regardless of what the other one does

  • Implicate is a Dominant Strategy for both

  • (Implicate, Implicate) becomes the Dominant Strategy Equilibrium

  • Note: If they might collude, then it’s beneficial for both to Not Implicate, but it’s not an equilibrium as both have incentive to deviate


Dominant strategy equilibrium

Dominant Strategy Equilibrium

  • Dominant Strategy Equilibrium: is a strategy combination s*= (s1*, s2*, …, sN*), such that si* is a dominant strategy for each i, namely, for each s= (s1, s2, …, si , …, sN):

    pi(s1, s2, …, si*, …, sN) ≥ pi(s1, s2, …, si, …, sN)

  • Dominant Strategy is the best response to any strategy of other players

  • It is good for agent as it needs not to deliberate about other agents’ strategies

  • Not all games have a dominant strategy equilibrium


A beautiful mind nash equilibrium

A Beautiful Mind: Nash Equilibrium

  • Nash Equilibrium: is a strategy combination

    s*= (s1*, s2*, …, sN*) such that for each i, si* is a best response to (s1*, …,si-1*,si+1*,…, sN*), namely, for any possible alternative strategy si

    pi(s*) ≥ pi(s1*, s2*, …, si, …, sN*)

  • Note: We are playing simultaneous games, and so nobody knows a priori the choice of other agents


Nash equilibrium

Nash Equilibrium

  • In a NE no agent can unilaterally deviate from its strategy given others’ strategies as fixed

  • There may be no, one or many NE, depending on the game

  • Agent has to deliberate about the strategies of the other agents

  • If the game is played repeatedly and players converge to a solution, then it has to be a NE

  • Dominant Strategy Equilibrium  Nash Equilibrium (but the converse is not true)


Nash equilibrium the battle of the sexes coordination game

Nash Equilibrium: The Battle of the Sexes (coordination game)

  • (Stadium,Stadium) is a NE: Best responses to each other

  • (Cinema, Cinema) is a NE: Best responses to each other

     but they are not Dominant Strategy Equilibria … are we really sure they will eventually go out together????


A conflictual game head or tail

A conflictual game: Head or Tail

  • Player I (row) prefers to do what Player II does, while Player II prefer to do the opposite of what Player I does!

     In any configuration, one of the players prefers to change his strategy, and so on and so forth…thus, there are no NE!


Three big computational issues

Three big computational issues

  • Finding a NE, once it does exist

  • Establishing the quality of a NE, as compared to a cooperative system, i.e., a system in which agents can cooperate (recall the Prisoner’s Dilemma)

  • In a repeated game, establishing whether and in how many steps the system will eventually converge to a NE (recall the Battle of the Sex)


On the existence of a ne

On the existence of a NE

Theorem (Nash, 1951): Any game with a finite set of players and finite set of strategies has a NE of mixed strategies.

  • Mixed strategies: each player independently selects his strategy by using a probability distribution over his set of possible strategies

  • Head or Tail game: if each player sets p(Head)=p(Tail)=1/2, then the expected payoff of each player is 0, and this is a NE, since no player can improve on this by choosing a different randomization!


On the computability of a ne

On the computability of a NE

  • But how do we select this probability distribution?

    It looks like a problem in the continuous…

    …but it’s not, actually! It can be shown that such a distribution can be found by checking for all the (exponentially large) possible combinations for each player of the underlying pure strategies!

  • And why should we be interested on that?

    Because “If your laptop cannot find a NE, then the market probably cannot either”


Is finding a ne np hard

Is finding a NE NP-hard?

  • W.l.o.g., we restrict ourself to 2-player games: The problem can be solved by a simplex-like technique called the Lemke–Howson algorithm, which however is exponential in the worst case

  • Reminder: a problem P is NP-hard if one can reduce any NP-complete problem P’ to it:

    • “yes”-instance of P’ → “yes”-instance of P

    • “no”-instance of P’→ “no”-instance of P

  • But each instance of 2-NASH is a “yes”-instance! (since every game has a NE)

     if 2-NASH is NP-hard then NP = coNP (hard to believe!)


The complexity class ppad

The complexity class PPAD

  • Definition (Papadimitriou, 1994): roughly speaking, PPAD (Polynomial Parity Argument – Directed case) is the class of all problems whose solution space can be set up as the set of all sinks in a suitable directed graph (generated by the input instance), having an exponential number of vertices in the size of the input, though.

  • Remark: It could very well be that PPAD=PNP…

    …but several PPAD-complete problems are resisting for decades to poly-time attacks (e.g., finding Brouwer fixed points)


2 nash is ppad complete

2-NASH is PPAD-complete!

  • 3D-BROUWER is PPAD-complete (Papadimitriou, JCSS’94)

  • 4-NASH is PPAD-complete (Daskalakis, Goldberg, and Papadimitriou, STOC’06)

  • 3-NASH is PPAD-complete (Daskalakis & Papadimitriou, ECCC’05, Chen & Deng, ECCC’05)

  • 2-NASH is PPAD-complete !!!(Chen & Deng, FOCS’06)


On the quality of a ne

On the quality of a NE

  • How inefficient is a NE in comparison to an idealized situation in which the players would strive to collaborate selflessly with the common goal of maximazing the social welfare?

  • Recall: in the Prisoner’s Dilemma game, the DSE  NE means a total of 10 years in jail for the players. However, if they would not implicate reciprocally, then they would stay a total of only 2 years in jail!


The price of anarchy

The price of anarchy

  • Definition (Koutsopias & Papadimitriou, 1999): Given a game G and a social-choice minimization (resp., maximization) function f (i.e., the sum of all players’ payoffs), let S be the set of NE, and let OPT be the outcome of G optimizing f. Then, the Price of Anarchy (PoA) of G w.r.t. f is:

  • Example: in the PD game, G(f)=-10/-2=5


Algorithmic game theory because a game is nice when it s not too long

A case study: selfish routing on Internet

  • Internet components are made up of heterogeneous nodes and links, and the network architecture is open-based and dynamic

  • Internet users behave selfishly: they generate traffic, and their only goal is to download/upload data as fast as possible!

  • But the more a link is used, the more is slower, and there is no central authority “optimizing” the data flow…

  • So, why does Internet eventually work is such a jungle???


Algorithmic game theory because a game is nice when it s not too long

Example: Pigou’s game (network congestion game)

Latency depends on the congestion (x is the fraction of flow using the edge)

s

t

Latency is fixed

  • What is the NE of this game? Trivial: all the fraction of flow tends to travel on the upper edge  the cost of the flow is 1·1+0·1 =1

  • What is the PoA of this NE? The optimal solution is the minimum of f(x)=x·x +(1-x)·1  f ’(x)=2x-1  OPT=1/2  f(OPT)=1/2·1/2+(1-1/2)·1=0.75  G(f) = 1/0.75 = 4/3


Flows and ne

Flows and NE

  • Assume now we are given a directed graph G = (V,E) and a set of source–sink pairs si,ti  V between which selfish users want to push a certain amount of flow. Then, a flow is at Nash equilibrium (or is a Nash flow) if no agent can improve its latency by changing its path

  • Theorem (Beckmann et al., 1956): If edge latency functions are continuous and non-decreasing, and users control an infinitesimal amount of flow, then the Nash flow exists and is unique.


Flows and price of anarchy

Flows and Price of Anarchy

  • Theorem 1:In a network with linear latency functions, the cost of a Nash flow is at most 4/3 times that of the minimum-latency flow.

  • Theorem 2:In a network with general latency functions, the cost of a Nash flow is at most n/2 times that of the minimum-latency flow.

    (Roughgarden & Tardos, JACM’02)


A bad example for non linear latencies

A bad example for non-linear latencies

Assume i>>1

xi

1-

1

s

t

1

0

A Nash flow (of cost 1) is arbitrarily more expensive than the optimal flow (of cost close to 0)


Convergence towards a ne in pure strategies games

Convergence towards a NE(in pure strategies games)

  • Ok, we know that selfish routing is not so bad at its NE, but are we really sure this point of equilibrium will be eventually reached?

  • Convergence Time: number of moves made by the players to reach a NE from an initial arbitrary state

  • Question:Is the convergence time (polynomially) bounded in the number of players?


The potential function method

The potential function method

  • Roughly speaking, a potential function for a game is a real-valued function, defined on the set of possible outcomes of the game, such that the equilibria of the game are precisely the local optima of the potential function

  • Potential games: broad class of games admitting a potential function

  • Theorem: In any finite potential game, best response dynamics always converge to a NE of pure strategies.


Potential function and congestion games

Potential function and congestion games

  • How many steps are needed to reach a NE? It depends on the combinatorial structure of the players' strategy space

  • Definition(Matroid Congestion Games): A congestion gameG is called MCG if:

    • The strategy space of every player is the basis of a matroid over the set of congested resources (recall that the size of this strategy space corresponds to the rank of the player's matroid);

    • The rank of the game, r(G), is defined to be the maximum matroid rank over all players.


Convergence in congestion games

Convergence in congestion games

  • Theorem (Achermann et al., FOCS'06): In a MCG G with n players and m resources, all best response improvement sequences have length O(n2m r(G)).

  • Example of a MCG: Load balancing

  • Instead, in general, a network congestion game is not a MCG

  • Moreover, it is possible to show that there exist instances for which the convergence time is exponential (unless finding a local optimum in any Polynomial Local Search (PLS class) problem can be done in polynomial time, against the common belief)

    Still, Internet works quite fine!


Algorithmic game theory because a game is nice when it s not too long

And now…


Algorithmic game theory because a game is nice when it s not too long

SECOND PART:

Algorithmic Mechanism Design

(or, the art of convincing a capitalist to behave like a socialist )


Mechanism design the goal

Mechanism Design: the goal

  • Given:

    • System comprising of self-interested, rational agents

    • Set of system-wide goals

  • Mechanism Design

    • Does there exist a mechanism that can implement the goals?

  • Implementation of the goals depends on the individual behavior of the agents


Mechanism design a picture

Mechanism Design: a picture

Private “types”

Reported types

Mechanism

t1

r1

p1

Agent 1

Output

tn

rn

pn

Agent n

Payments

Each agent reports strategicallyto maximize its well-being…

…in response to a payment which is a function of the output!


Overview of the results

Overview of the results

  • Algorithms  Mechanisms

  • Centralized  Decentralized (Non-cooperative)

    • Network design problems


Mechanism design

Mechanism Design

  • Games induced by mechanisms are different from games in standard form:

    • Players hold independent private values

    • The payoff matrix is a function of these types

       each player doesn’t really know about the other players’ payoffs, but only about its one!

       Games of Incomplete information

       Dominant Strategy Equilibrium is used


Mechanism design problem type of an agent

Mechanism Design Problem: Type of an Agent

  • N agents, and each agent has some private information called thetype, tiTi, and performs a strategic action

  • We restrict ourself to direct revelation mechanisms, in which the action is reporting a typeriTi(with possibly ri  ti)

    • Example: Auction Game

      • Each agent knows its cost for doing a job, but not the others’ one; the type of the agent is its cost

      • Ti= [0, +]: The agent’s cost may be any positive amount of money

      • ti= 80: Minimum amount of money the agent i is willing to be paid

      • ri= 85: Exact amount of money the agent i bids to the system for doing the job (not known to other agents)


Mechanism design problem output specification

Mechanism Design Problem: Output Specification

  • F is the set of feasible outputs

  • Output Specification: For a given reported-type configuration r=(r1, r2, …, rN), it specifies a valid outcome x(r)F which should optimize an objective function f(t) (the so-called social choice function)

    • Example: Auction Game

      • F : Different winners of the auction

      • f(t): mini (t1, t2, …, tN) (the lowest true cost)

      • x(r): allocate to the bidder with lowest reported cost


Mechanism design problem valuation and utility

Mechanism Design Problem: Valuation and Utility

  • If x is the outcome, then the valuation that agent i makes of x is given by a real valued function: vi(ti,x)

    • Auction Game: If agent i wins the auction then its valuation is equal to its actual cost for doing the job, otherwise it is 0

  • If pi is the payment given to the agent, then the utility of the outcome x is:

    ui(ti,x) = pi - vi(ti,x)

    • Auction Game: If agent’s cost for the job is 90, and it gets the contract for 100 (i.e., it is paid 100), then its utility is 10


The mechanism

The Mechanism

A mechanismis a pairM=<x=g(r), p(x)> specifying:

  • An algorithm g(r) which computes the outcome x as a function of the reported typesr

  • A paymentscheme p as a function of the output x


Strategy proof mechanisms

Strategy-Proof Mechanisms

  • If truth telling is the dominant strategy in a mechanism then it is called Strategy-Proof

    • Agents report their true types instead of strategically manipulating it

  • Utilitarian Problems: A problem is utilitarian if its objective function is such that f(t) =i vi(ti,x)

    • The Auction game is utilitarian


Vickrey clarke groves vcg mechanisms

Vickrey-Clarke-Groves (VCG) Mechanisms

  • A VCG-mechanism is (the only) strategy-proof mechanism for utilitarianproblems:

    • Algorithm:

      g(r)= arg maxxFi vi(ri,x)

    • Payment function:

      pi (x)= hi(r-i) -j≠i vj(rj,x)

      wherehi(r-i) is an arbitrary function of the types of other players

  • What about non-utilitarian problems? We will see…


Vcg mechanisms are strategy proof

VCG-Mechanisms are Strategy-Proof

  • Proof (Intuitive sketch):

    • Payment given to agent i

      pi (x)=hi(r-i)-j≠i vj(rj,x)

      and both the terms above are independent of the type, strategy and valuation of agent i

    • So it is best for agent i to report its true value. Strategic behavior does not lead to a beneficial outcome.


Clarke mechanisms

Clarke Mechanisms

  • This is a special VCG-mechanism (known as Clarke mechanism) in which

    hi(r-i)=j≠i vj(rj,x(r-i))

  • pi =j≠i vj(rj,x(r-i)) -j≠i vj(rj,x)

  • In Clarke mechanisms, agents’ utility are always non-negative


Clarke mechanism for the vickrey auction

Clarke mechanism for the Vickrey auction

  • The winner is paid pi = j≠i vj(rj,x(r-i)) -j≠i vj(rj,x) =

    = j≠i vj(rj,x(r-i))  second lowest offer

  • Let us convince ourself it is strategy-proof by case analysis. For a player i, let T=minj≠i {rj}:

    • ti<T: then, if ri<ti, he still wins, but he keeps on to be paid T, while if ri>ti, he may still win (again being paid T), but he may also lose (if ri>T), by getting a null utility;

    • ti>T: then, if ri>ti, he keeps on not to win, while if ri<ti, he may win, but he will be paid T<ti, by getting a negative utility.

  • Remark: the difference between the second lowest offer and the lowest offer is unbounded (frugality issue)


Vcg clarke mechanisms pros and cons

VCG (Clarke)-Mechanisms: ProsandCons

  • The goal, i.e., the optimization of the social-choice function, is achieved with certainty

  • The payments may be sub-optimal


Vcg mechanisms algorithmic issues

VCG-mechanisms: algorithmic issues

  • System has to calculate N+1 functions: once with all agents (for g(r)) and once for every agent (for the associated payments)

     What is the time complexity of the mechanism?

  • What happens if the problem is non-utilitarian?

  • What happens if computing the optimal solution is NP-hard?


Vcg mechanisms and graph problems

VCG-mechanisms and graph problems

  • Following the Internet model, we assume that each agent owns a single edge of a graph G=(V,E), and establishes the cost for using it

     The agent’s type is the true weight of the edge

  • Classic optimization problems on G become mechanism design optimization problems!

  • Many basic network design problems have been faced: minimum spanning tree, shortest paths, minimum Steiner tree, and many others

  • Let’s see what happens for shortest paths…


Vcg mechanism for the shortest path problem

VCG-mechanism for the shortest path problem

  • Input: a selfish-edge undirected graph G (2-edge-connected), a source node s and a destination node t;

  • SCF: a true shortest path in G between s and t.

  • VCG-mechanism: The problem is indeed utilitarian!

    • g(r): given G and the reported edge weights r=(r1,…,rm), compute PG(s,t)

    • pe: For any edge e  PG(s,t), set pe=dG-e(s,t)-dG(s,t)+re

       For any e  PG(s,t), we have to compute PG-e(s,t), namely the replacement shortest path in G-e =(V,E\{e})) between s and t


Replacement shortest path

Replacement shortest path

s

PG(s,t)

3

2

PG-e(s,t)

4

5

6

5

e

2

10

12

5

t


A very efficient solution

A very efficient solution

  • Brute force:  ePG(s,t), apply Dijkstra to find PG-e(s,t)  time complexity: O(mn + n2 logn) time

  • In late 80’s, Malik et al. solved in O(m+n log n) time the following vitality problem: given a shortest path PG(s,t), which is the most vital edge of it, i.e., the edge whose removal causes the longest replacement path between s and t?

  • Their approach consisted of computing all the replacement paths between s and t…

    …but this is exactly what we are looking for in our VCG-mechanism!


Beyond vcg mechanisms

Beyond VCG-mechanisms

  • One-parametermechanisms (Archer & Tardos, SODA’01): provide strategy-proof mechanisms whenever the type of an agent is just a single parameter, as soon as;

    • the underlying algorithm is monotone(i.e., it keeps on to (not) use an agent as soon as he decreases (increases) its reported type);

    • agents are paid w.r.t. their threshold value.

  • One-parameter mechanisms are used in some classic non-utilitarian problems (e.g., the Shortest-Paths Tree problem)

  • They are also used for several NP-hard problems, for which the fact that the underlying algorithm is sub-optimal prevents the use of a VCG-mechanism (e.g., the Steiner Tree problem)


Some of our results

Some (of our) results


Conclusions

Conclusions

  • AGT is a rapidly evolving discipline

  • Many big questions are left open…

    …so, why not to try? (OR methods are sought!)

  • Suggested readings:

    • Algorithmic Game Theory, Edited by Noam Nisan, Tim Roughgarden, Eva Tardos, and Vijay V. Vazirani, Cambridge University Press.

    • Algorithmic Mechanism Design for Network Optimization Problems, Luciano Gualà, PhD Thesis, Università degli Studi dell’Aquila, 2007.

    • Web pages by Éva Tardos, Christos Papadimitriou, Tim Roughgarden, and then follow the links therein


Algorithmic game theory because a game is nice when it s not too long

Game over!


  • Login