Bayesian Games - PowerPoint PPT Presentation

Slide1 l.jpg
Download
1 / 40

  • 287 Views
  • Uploaded on
  • Presentation posted in: Sports / Games

Bayesian Games. Bayesian Games. Definitions and Static Bayesian Games Example of solving a static Bayesian game Sender-Receiver Bayesian Games Equilibrium concepts An example of Bayesian games in Computer Science: Ad Hoc Networks: Modelling cooperation. What is a Bayesian game?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

Bayesian Games

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Slide1 l.jpg

Bayesian Games

© Petteri Nurmi 2003


Bayesian games l.jpg

Bayesian Games

  • Definitions and Static Bayesian Games

  • Example of solving a static Bayesian game

  • Sender-Receiver Bayesian Games

  • Equilibrium concepts

  • An example of Bayesian games in Computer Science: Ad Hoc Networks: Modelling cooperation

© Petteri Nurmi 2003


Slide3 l.jpg

What is a Bayesian game?

  • A strategic form game with incomplete information

    • What is incomplete information?

      • Some players don’t know the payoff of the others

      • Incomplete information is not imperfect information

      • Imperfect information = Players don’t observe the actions of others correctly

  • Classic reference:

  • Harsanyi J., 1967-1968,

  • Games with incomplete information played by Bayesian players

  • Management Science 14: 159-182; 320-334; 486-502

© Petteri Nurmi 2003


Harsanyi s model l.jpg

Harsanyi’s Model

  • The game is transformed into a game of imperfect information

    • A prior move by nature

    • Nature’s move determines player’s ”type”

    • The equilibrium of this game is the Bayes-Nash equilibrium

  • John C. Harsanyi (29.5. 1920 – 2000)

  • Nobel prize in 1994 together with

  • John F. Nash Jr.

  • Reinhard Selten

  • "for their pioneering analysis of equilibria in the theory of non-cooperative games"

© Petteri Nurmi 2003


Mathematical intermezzo l.jpg

Mathematical Intermezzo

  • A Bayesian Game consists of the following components

    • A (finite) set of players N = {1,…, n}

    • An action set for each player Ai ; A = χi ∈ N Ai

    • A type set Θi ; Θi = χi ∈ N Θ i

    • A probability function pi: Θ i Δ (Θ -i)

    • A payoff function: ui: A x Θ  ℝ

© Petteri Nurmi 2003


Slide6 l.jpg

Type?

  • In Harsanyi model nature selected player’s type  What is a type?

    • Any private information (or is not common knowledge) that is relevant to the players’ decision making

    • Such as?

      • Player’s payoff function

      • Player’s beliefs about other players’ payoff functions

      • Beliefs about what players believe his beliefs are

      • … and so on

© Petteri Nurmi 2003


Type continued l.jpg

Type continued

  • Let us denote Θi player i’s type

  • Type Θi is observed by player i only

  • p(Θi | Θ-i ) denotes player i’s conditional probability about his opponents types given his type

    Θ-i = (Θ1 , …, Θi-1, Θi+1 , …, ΘI )

  • We assume that the marginal pi (Θi) is strictly positive.

© Petteri Nurmi 2003


Strategy l.jpg

Strategy

  • A pure-strategy space Si represents choices of actions

  • σ(Θi) is the strategy player i chooses when his type is Θi

  • Mathematically defined:

    • Strategy is a mapping from set of types to the set of Actions

      • σ(Θi) = Ai; σi : Θi Ai∀ i ∈ N (pure)

      • σ(Ai | Θi) = α; σi : Θi Δ(Ai)∀ i ∈ N (mixed)

        • α represents the (conditional) probability that player i chooses action i when the type is Θi

© Petteri Nurmi 2003


Equilibrium l.jpg

Equilibrium

  • Bayesian equilibrium = Bayes-Nash equilibrium

    • Player i maximizes her expected utility conditional on type Θi

      si (Θi) ∈ arg max  p(Θi | Θ-i) ui (s i’, s-i (Θ-i), (Θi, Θ-i))

      si’ ∈ SiΘ-i

© Petteri Nurmi 2003


Example l.jpg

Enter

Don’t

0, -1

2, 0

1.5, -1

3.5, 0

B

2, 1

3, 0

2, 1

3, 0

DB

LOW

Example

  • Consider the following game

    • Player I decides whether to build a new factory

    • Simultaneously player II decides whether to enter or not

    • Player I’s decision depends on her building cost that is unknown to player II

Enter

Don’t

B

DB

HIGH

© Petteri Nurmi 2003


Example cont l.jpg

Example cont.

  • This can be seen as a Bayesian game

    • Set of players N = { I, II }

    • Action sets A1 = { B, DB} ; A2 = {Enter, Don’t }

    • Type sets Θ1 = { HIGH, LOW } ; Θ2 = { X }

  • Player II has a singleton set as the type space so we can ignore it.

  • Let cl be low cost and ch high cost types

© Petteri Nurmi 2003


Example cont12 l.jpg

Enter

Don’t

Enter

Don’t

0, -1

2, 0

1.5, -1

3.5, 0

B

B

2, 1

3, 0

DB

2, 1

3, 0

DB

HIGH

LOW

Example cont.

  • A strategy for player I is an action for EACH of it types

  • For the high-cost type of player I we have a dominant strategy don’t build

© Petteri Nurmi 2003


Example cont13 l.jpg

Enter

Don’t

1.5, -1

3.5, 0

B

2, 1

3, 0

DB

LOW

Example cont.

  • The best-response for player I low-cost type depends on player II’s strategy

    u1(B; y; cl) = 1.5y + 3.5(1-y) = 3,5 – 2y

    u1(DB; y; ch) = 2y + 3(1 – y) = 3 – y

  • Player I’s low-cost type

    prefers building IF y ≤ ½

© Petteri Nurmi 2003


Example cont14 l.jpg

Example cont.

  • For player II we must first consider the possibility that the cost is actually low 

    u2(E;x) = p + (1-p) [-2x + 2 (1 –x)] = 2 – 4(1 – p)x

    x ≤ 1 / 2(1 – p) (:=w)

  • Now we need to compare the best-response correspondences, for player II the correspondence is

    {1}x < w

    y*(x) = [0, 1]x = w

    {0}x > w

© Petteri Nurmi 2003


Example cont15 l.jpg

Example cont.

  • In a similar way we get the correspondence mapping for player I.

  • Now the Bayesian equilibrium is the intersection of the correspondence functions (with a fixed value for p1)

© Petteri Nurmi 2003


Slide16 l.jpg

© Petteri Nurmi 2003


Sender receiver games l.jpg

Sender-Receiver Games

  • Game has two players a sender and a receiver

  • Sender sends a signal to receiver who then chooses an appropriate action

  • Player I has private information about his type

  • Player II has only one type, which is considered common knowledge

  • “A move by nature”

© Petteri Nurmi 2003


Strategies in s r games l.jpg

Strategies in S-R games

  • A pure strategy for the Sender is a one-to-one correspondence mapping m: Θ M

  • Let σ(m | Θ) be the probability that a type Θ-sender sends message m  mixed-strategy for sender

  • For receiver let p(a | m) be a mixed-strategy (choose action a if message = m)

  • On-the-path messages

    ℳ+(Θ) = {m: ∃ θ ∈ Θσ(m | Θ) > 0 } = supp σ(Θ)

© Petteri Nurmi 2003


Payoffs in s r games l.jpg

Payoffs in S-R games

  • Sender’s (expected) payoff: a∈Aρ(a|m)u(m,a,θ)

  • For the receiver?

    • Must consider every type and every message

    • E(v(a,σ)) = m∈Mθ∈Θρ(θ) σ(m| θ)u(m,a,θ)

© Petteri Nurmi 2003


Bayes rule in action l.jpg

Bayes’ rule in action

  • For any on-the-path message m, the receiver’s posterior belief that player I is of type θ, is pB(θ |m)

    pB(θ | m) = p(θ)σ(m|θ)

    θ’∈Θp(θ’)σ(m| θ)

  • POSTERIOR rule for updating PRIOR beliefs!

© Petteri Nurmi 2003


Bayes equilibrium in s r games l.jpg

Bayes Equilibrium in S-R games

  • In a Sender-Receiver game the Bayesian equilibrium is a triple (σ, ρ, θ) ∈ℳθxAMx(Δ(θ))M satisfying the following conditions:

    • For all types θ ∈ Θ

      supp σ(Θ) ⊂M´(ρ, Θ)

    • For all on-path-messages: ∀ m∈ M+(σ)

      supp ρ(m) ⊂A´(ρ´, Θ)

    • The conditional posterior belief system is consistent with Bayes’ rule whenever possible

© Petteri Nurmi 2003


Perfect bayesian equilibrium l.jpg

Perfect Bayesian Equilibrium

  • In a Sender-Receiver game the perfect Bayesian equilibrium is a triple (σ, ρ, θ) ∈ℳθxAMx(Δ(θ))M satisfying the following conditions:

    • For all types θ ∈ Θ

      supp σ(Θ) ⊂M´(ρ, Θ)

    • For all messages: ∀ m∈ M(σ)

      supp ρ(m) ⊂A´(ρ´, Θ)

    • The conditional posterior belief system is consistent with Bayes’ rule whenever possible

© Petteri Nurmi 2003


Other types of equilibrium l.jpg

Other types of Equilibrium

  • Perfect Bayesian equilibrium in Multi-Stage Games

    • Posterior beliefs are independent, and all types of player i have the same beliefs

    • Bayes rule to update beliefs (history information?)

    • “no signalling what you don’t know”

    • Posterior beliefs need to be consistent with a common joint distribution

  • Extensive-Form games: Sequential Equilibrium

    • See Fudenberg, D., and J.Tirole Game Theory 1991 The MIT Press p. 337-341

  • Trembling-Hand Perfect Equilibrium (p. 351-356)

  • Proper equilibrium (p.356-359)

© Petteri Nurmi 2003


Bayesian games in cs l.jpg

Bayesian games in CS

  • Ad Hoc Networks

  • Auctions

  • Social learning

  • The web search game

  • Voting

© Petteri Nurmi 2003


Ad hoc networks l.jpg

Ad hoc networks

  • MANET = Mobile Ad Hoc Networks

    • A set of mobile hosts, each with a transceiver

    • No base stations; no fixed network infrastructure

    • Multi-hop communication

    • Routing and packet forwarding takes place in a dynamical network topology

  • Game Theory and MANET?

    • Routing mechanisms for “selfish cooperation”

© Petteri Nurmi 2003


Cooperation in manet l.jpg

Cooperation in MANET

  • Reference: Modelling cooperation in Mobile Ad Hoc Networks: A formal description of Selfishness; Urpi A., Bonuccelli M., and Giordano S.

  • Modelling Ad Hoc Networks with Bayesian games

    • The nodes are the players

    • Nodes have to periodically select whether to forward or not

    • Nodes have incomplete information about the total traffic in the network

    • Nodes have local information about their neighbourhood

© Petteri Nurmi 2003


Slide27 l.jpg

cont.

  • Important issues for each node

    • Energy consumption

    • The packets are forwarded by someone

  • “A shared medium”

    • Packets are send to every node that is within the transmission range

  • Prior to choosing its next action, a node has an opportunity to analyze its neighbours past behaviour

  • Node most decide to whom to send packets and to whom to discard packets.

© Petteri Nurmi 2003


The model l.jpg

The Model

  • Time is discrete and divided into timeslots t1,…, tn

  • Node i has the following information in the beginning of frame tk

    • Ni(tk): Set of neighbours, assumed to be fixed during a single frame

    • Bi(tk): The remaining energy units (in the battery)

    • Tij(tk): The traffic node i generated as a source and has to send to node j during frame k. (for each node j in node i’s neighbourhood)

    • Fij(tk-1): The number of packets that j forwarded for i during the previous frame

    • Rij(tk-1): The number of packets i received from j during the previous frame

    • Ȓij(tk-1): The number of packets i received from j during the previous frame as a final destination

© Petteri Nurmi 2003


The forwarding game l.jpg

The forwarding game

  • Nodes are the players

  • Player i’s type is its energy class e(i) = α, where 0 ≤ α≤ 1

  • Player i as an action sets Sij(tk) i.e. the number of packets she will send to node j, and Fij(tk) the number of packets received from j during the previous frame she will forward to her

  • Player i’s payoff is:

    αe(i) Wi(tk) + (1 – αe(i)) Gi(tk)

    • Where

      • Wi(tk) is a measure of the energy spent succesfully

      • Gi(tk) is a the ratio of sent packets over packets that player i wanted to send.

© Petteri Nurmi 2003


The forwarding game cont l.jpg

The forwarding game cont.

  • Player i has prior belief for every player j in its neighbourhood, what its energy class is.

  • A node tries to maximize its payoff function

     SELFISHNESS

  • We need to analyze the game as a repeated (dynamic) game and provide a utility function that makes it profitable to player i to cooperate

© Petteri Nurmi 2003


Problems l.jpg

Problems

  • How to get the forwarding information?

  • Badly defined utility function and/or policy leads to self destruction

  • The usage of time slots

    • There is no synchronization!

  • Too simple decision space?

    • Possible other constaints.

© Petteri Nurmi 2003


Problems cont l.jpg

Problems cont.

  • Malicious and selfish users?

    • Need a stronger policy

      • Punishing vs. Encouraging

        • Punished better suitable because

          • How to reward agents? (better throughput in a network with no authority?)

          • Punishing more suitable to both malicious and selfish users, encouraging/rewarding suitable only for encouraging cooperation

  • Theorem: Cooperation can be enforced in a mobile Ad Hoc network, provided that enough members agree on it and that no node has to forward more traffic than it generates.

© Petteri Nurmi 2003


Additional references l.jpg

Additional References

  • Cooperation in wireless ad hoc networks. Srinivasan V., Nuggehalli P., Chiasserini C-F, and Ramesh R. R., In Proceedings of IEEE Infocom 2003 http://citeseer.nj.nec.com/568937.html

  • Game Theoretic analysis of security in mobile ad hoc networks. Michiardi P., and Molva R. Technical Report RR-02-070, Institut Eurecom 2002.

© Petteri Nurmi 2003


Slide34 l.jpg

Summary

© Petteri Nurmi 2003


Types of bayesian games l.jpg

Types of Bayesian games

  • Static Bayesian games

  • Dynamic Bayesian games

    • Sender-Receiver Games

    • Extensive Form Games

    • Multi-Stage Games

  • Equilibrium concepts

    • Bayesian Equilibrium = Bayes-Nash Equilibrium

    • Bayes Equilibrium (in dynamic games)

    • Perfect Bayes Equilibrium

© Petteri Nurmi 2003


Applications of static bayesiang l.jpg

Applications of static BayesianG

  • Packet forwarding in Ad Hoc networks

  • Voting mechanisms

  • Auction mechanisms

  • = MULTI-AGENT SYSTEMS

  • Requires:

    • Simultaneous competition

    • Multiple agents with incomplete information

    • Can also be non-simultaneous competition if the agents/players don’t know each others’ decisions (but have same beliefs that affect their decision-making).

© Petteri Nurmi 2003


Applications of dyn bayesiang l.jpg

Applications of Dyn. BayesianG.

  • Many economic applications

  • Design model for network protocols?

  • Design model for multiprocessor architectures?

  • Bayesian games are a suitable tool for modelling situations where there is interaction between two or more agents and the prior information is incomplete.

© Petteri Nurmi 2003


References l.jpg

References

  • Fudenberg, D., and J.Tirole Game Theory 1991 The MIT Press

  • Kockesen L., Bayesian Games, http://www.columbia.edu/~lk290/ugbayes.pdf

  • Ratliff J., Static Games of Incomplete Information

  • Myatt D. P., Who Am I Playing? Incomplete Information and Bayesian Games,

    http://malroy.econ.ox.ac.uk/dpm/MPhilGameTheory/IncompleteStrategic.pdf

  • Urpi A., Bonuccelli M., Giordano S., Modelling cooperation in mobile ad hoc networks: a formal description of selfishness

© Petteri Nurmi 2003


Additional material l.jpg

Additional Material

  • Eyster E., and M.Rabin Cursed Equilibrium 2000

  • Jackson M., Kalai E., Social Learning in Recurring Games

  • Khoussainov R., and N. Kushmerick Playing the Web Search Game

  • Tenneholtz M., Robust Decision-Making in Multi-Agent Systems

© Petteri Nurmi 2003


The end l.jpg

The End

Bayesian Games

by Petteri Nurmi

http://www.cs.helsinki.fi/u/ptnurmi/papers.html

© Petteri Nurmi 2003


  • Login