1 / 18

Bayesian and non-Bayesian Learning in Games

Bayesian and non-Bayesian Learning in Games. Ehud Lehrer. Tel Aviv University, School of Mathematical Sciences. Including joint works with: Ehud Kalai, Rann Smorodinsky, Eioln Solan. Learning in Games. Informal definition of learning : a decentralized process that

wetherbee
Download Presentation

Bayesian and non-Bayesian Learning in Games

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Bayesian and non-Bayesian Learning in Games Ehud Lehrer Tel Aviv University, School of Mathematical Sciences Including joint works with: Ehud Kalai, Rann Smorodinsky, Eioln Solan.

  2. Learning in Games Informal definition of learning: a decentralized process that converges (in some sense) to (some) equilibrium. • Bayesian (rational) learning: Players do not start in equilibrium, but • they have some initial belief about other players’ strategies • they are rational: they maximize their payoffs • they take into account future payoffs • Convergence in REPEATED GAME • Non-Bayesian learning: Players • don’t have any initial belief about other players’ strategies • don’t maximize their payoffs • don’t take into account future payoffs • Convergence (of the empirical frequency) to an equilibrium of the ONE-SHOT GAME

  3. Bayesian vs. non-Bayesian Bayesian learning: Players do not start in equilibrium, but they start with a “grain” of idea about what other players do. Nature of results: players eventually play something close to an equilibrium of the repeated game. Non-Bayesian learning: Players have no idea about other players’ actions. They don’t care to maximize payoffs. Nature of results: the statistics of past actions looks like an equilibrium of the one-shot game.

  4. Important tools • Bayesian learning: merging of two probability measures along a a filtration (an increasing sequence of - fields) Non-Bayesian learning: approachability Both were initiated by Blackwell (the first with Dubins)

  5. A set F is excludable by player 2 if there is a strategy  s.t. Repeated Games with Vector Payoffs • I = finite set of actions of player 1. • J = finite set of actions of player 2. • M = (mi,j) = a payoff matrix. Entries are vectors in Rd. A set F is approachable by player 1 if there is a strategy  s.t. There are sets which are neither approachable nor excludable.

  6. Approachability • Applications (a sample): • No-regret (Hannan) • Repeated games with incomplete information (Aumann-Maschler) • Learning (Foster-Vohra, Hart-Mas Colell) • Manipulation of calibration tests (Foster-Vohra, Lehrer, • Smorodinsky-Sandroni-Vohra) • Generating generalized normal-number (Lehrer)

  7. Characterization of Approachable Sets H(p0) x y the line xy F the hyperplane perpendicular to xy that passes through y mp,q = i,j pi mi,j qj H(p) = { mp,q , q (I) } • A closed set F Rd is a B-set if for every xFthere is y F that satisfies: • y is a closest point in F to x. • The hyperplane perpendicular to the line xy that passes through y separates between x and H(p), for some p  (I).

  8. Characterization of Approachable Sets Theorem [Blackwell, 1956]: every B-set F is approachable. The approaching strategy plays at each stage n the mixed action p such that H(p) and x are separated by the hyperplane connecting xand a closest point to x in F. With this strategy: Theorem [Blackwell, 1956]: every convex set is either approachable or excludable. Theorem [Hou, 1971; Spinat, 2002]: every minimal (w.r.t. set inclusion) approachable set is a B-set. Or: A set is approachable if and only if it contains a B-set.

  9. Bounded Computational Capacity A strategy is k-bounded-recall if it depends only on the last k pairs of actions (and it does not depend on previously played actions). • A (non-deterministic) automaton is given by: • A finite state space. • A probability distribution over states, according to which the initial state is chosen. • A set of inputs (say, the set I× J of action pairs). • A set of outputs (say, I , the set of player 1’s actions). • A rule that assigns to each state a probability distribution over outputs. • A transition rule that assigns to every state and every input a probability distribution over the next state.

  10. Approachability and Bounded Capacity A set F is approachable with bounded-recall strategies by player 1 if for every >0, the set B(F, ) := { y : d(y, F)  } is approachable by some bounded-recall strategy. A set F is excludable against bounded-recall strategies by player 2 if player 2 has a strategy  such that • Theorem (w/ Eilon Solan): The following statements are equivalent. • The set F is approachable with bounded-recall strategies. • The set F is approachable with automata. • The set Fcontains a convex approachable set. • The set F is not excludable against bounded-recall strategies. 4 points to note

  11. Main Theorem • Theorem: The following statements are equivalent for closed sets. • The set F is approachable with bounded-recall strategies. • The set F is approachable with automata. • The set F contains a convex approachable set. • The set F is not excludable against bounded-recall strategies. • A set is approachable with automata if and only if it is approachable by bounded-recall strategies. 2. A complete characterization of sets that are approachable with bounded-recall strategies. 3. A set which is not approachable with bounded-recall strategies, is excludable against all bounded-recall strategies. 4. We do not know whether the same holds for automata.

  12. Example On board Good news: in applications target sets are convex ( a point or a whole -- positive or negative -- orthant).

  13. Approachability in Hilbert space • I = finite set of actions of player 1. • J = finite set of actions of player 2. • M = (mi,j) = a payoff matrix. Entries are points in HS • (random variables). • All may change with the stage n. A set F is approachable by player 1 if there is a strategy  s.t. Advantage: allows for infinitely many constraints Theorem: Suppose that at stage n, the average payoff is and y is a closest point in F to . If the hyperplane perpendicular to the line that passes through y separates between and H(p), for some p  (I), then F is approachable.

  14. Approachability and law of large numbers are uncorrelated r.v.’s with . is the dot product. F is At any stage n, .

  15. F The game: each players has only one action. The payoff at stage n is . Thus, F is approachable. This is the strong law of large numbers. (When the payoffs are not uniformly bounded, there is an additional boundedness condition.)

  16. Problem: Approachability in norm spaces.

  17. Activeness function H is (even over a finite probability space). At stage n the characteristic function indicates which coordinates are active and which are not. The average payoff at stage n is Applications: 1. repeated games with incomplete information – different games are active on different times 2. construction of normal numbers 3. manipulability of many calibration tests 4. general no-regret theorem (against many replacing schemes) 5. convergence to correlated eq. along many sequences

  18. Activeness function – cont. Theorem: suppose that F is convex. Let be the closest point in F to the average payoff at time n, . If the hyperplane perpendicular to the line that passes through separates between and H(p), for some p  (I), then F is approachable.

More Related