Create Presentation
Download Presentation

Download Presentation
## COLOR TEST COLOR TEST COLOR TEST COLOR TEST

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Dueling Algorithms**Nicole Immorlica, Northwestern University with A. TaumanKalai, B. Lucier, A. Moitra, A. Postlewaite, and M. Tennenholtz**Social Contexts**Normal-form games: Players choose strategies to maximize expected von Neumann-Morgenstern utility. Social context games [AKT’08]: Players choose strategies to achieve particular social status among peers.**Social Contexts**Ranking games [BFHS’08]: Players choose strategies to achieve particular payoff rank among peers.**Two-Player Ranking Games**Bob G Alice and Bob play game: Alice 1 Alice beats Bob in G ½ Alice ties Bob in G RG payoff of Alice: 0 Alice loses to Bob in G**Implicit Representations**Succinct games [FIKU’08]: Payoff matrix represented by boolean circuit. NE hard to solve or approximate. Blotto games [B’21, GW’50, R’06, H’08]: Distribute armies to battlefields.**Implicit Representations**Optimization duels [this work]: Underlying game is optimization problem. Goal is to optimize better than opponent.**Ranking Duel**A search engine is an algorithm that inputs • set Ω = {1, 2, …, n} of items • probabilities p1 + … + pn = 1 of each and outputs a permutation π of Ω. Monopolist objective: minimize Ei~p[π(i)].**Ranking Duel**Competitive objective: Let the expected score of a ranking π versus a ranking π’ be Pri~p[ π(i) < π’(i) ] + (½) Pri~p[ π(i) = π’(i) ]. Then objective is to output a π that maximizes expected score given algorithm of opponent.**Optimizing a Search Engine**? User searches for object drawn according to known probability dist.**Greedy is optimal.**0.19 0.16 0.27 0.07 0.22 0.09 Search: pretty shape 1. (27%) 2. (22%) 3. (19%) 4. (16%) 5. (09%) 6. (07%)**Choosing a Search Engine**Search for “pretty shape”. See which search engine ranks my favorite shape higher. Thereafter, use that one.**0.19**0.16 0.27 0.07 0.22 0.09 Search: Search: pretty shape pretty shape 6. 1. (27%) (27%) 2. 1. (22%) (22%) 2. 3. (19%) (19%) 4. 3. (16%) (16%) 4. 5. (09%) (09%) 6. 5. (07%) (07%)**Questions**Can we efficiently compute an equilibrium of a ranking duel? How poorly does greedy perform in a competitive setting? What consequences does the duel have for the searcher?**Optimization Problems as Duels**Ranking Binary Search Routing Finish ? ? ? ? ? ? ? Start Hiring Compression Parking**Duel Framework**Finite feasible set X of strategies. Prob. distribution p over states of nature Ω. Objective cost c: Ω × X R. Monopolist: choose x to minimize Eω~p[cω(x)].**Duel Framework**1 if cω(x) < cω(x’) v(x,x’) = Eω~p 0 if cω(x) > cω(x’) ½ if cω(x) = cω(x’) Players select strategies x, x’ from X. Nature selects state ωfrom Ωaccording to p. Payoffs v(x,x’), (1-v(x,x’)) are realized.**Results: Computation**An LP-based technique to compute exact equilibria, A low-regret learning technique to compute approximate equilibria, … and a demonstration of these techniques in our sample settings**Computing Exact Equilibria**Formulate game as bilinear duel: • Efficiently map strategies to points X in Rn. • Define constraints describing K=convex-hull(X). • Define payoff matrix M that computes values. • Maps points in K back to strategies in original setting.**Bilinear Duels**If feasible strategies X are points in Rn, and payoff v(x, x’) is xtMx’ for some M in Rnxn, then maxv,x v s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X)) Exponential, but equivalent poly-sized LP.**Ranking Duel**Formulate game as bilinear duel: • Efficiently map strategies to points X in Rn. X = set of permutation matrices (entry xij indicates item i placed in position j) • Define constraints describing K=convex-hull(X). K = set of doubly stochastic matrices (entry yij = prob. item i placed in position j)**Ranking Duel**Formulate game as bilinear duel: • Design “rounding alg.” that maps points in K back to strategies in original setting. Birkhoff–von Neumann Theorem: Can efficiently construct permutation basis for doubly stochastic matrix (e.g., via matching).**Ranking Duel**Formulate game as bilinear duel: • Define payoff matrix M that computes values. Ep,y,y’[v(x,x’)] = ∑i p(i) ( ½ Pry,y’ [xi= x’i] + Pry,y’ [xi> x’i]) = ∑i p(i) (∑iyij ( ½ y’ij + ∑k>j y’ik)) which is bilinear in y,y’ and so can be written ytMy’.**Ranking Duel**Result: Can reduce computation time to poly(n) versus poly(n!) with standard LP approach. Technique also applies to hiring duel and binary search duel.**Compression Duel**data (each with prob. p(.)) Goal: smaller compression (i.e., lower depth in tree).**Classical Algorithm**Huffman coding: Repeatedly pair nodes with lowest probability.**Compression Duel**Formulate game as bilinear duel: • Efficiently map strategies to points X in Rn. X = subset of zero-one matrices* (entry xij indicates item i placed at depth j) • Define constraints describing K=convex-hull(X). K = subset of row-stochastic matrices* (entry yij = prob. item i placed at depth j) * Must correspond to depth profile of some binary tree!**Compression Duel**Formulate game as bilinear duel: • Define payoff matrix M that computes values. Ep,y,y’[v(x,x’)] = ∑i p(i) (∑iyij ( ½ y’ij + ∑k>j y’ik)) which is bilinear in y,y’ and so can be written ytMy’.**Compression Duel**Bilinear Form: maxv,x v s.t. xtMx’ ≥ v for all x’ in X x is in K (=convex-hull(X)) Problems: 1. How to round points in K back to a random binary tree with right depth profile? 2. How to succinctly express constraints describing K?**Approximate Minimax**Defn. For any ε > 0, an approximate minimaxstrategy guarantees payoff not worse than best possible value minus ε. Defn. For any ε > 0, an approximate best response has payoff not worse than payoff of best response minus ε.**Best-Response Oracle**Idea. Use approximate best-response oracle to get approximate minimax strategies. 1. Low-regret learning: if x1,…,xT and x’1,…,x’T have low regret, then ave. is approx minimax. 2. Follow expected leader: on round t+1, play best-response to x1,…,xt to get low-regret.**Compression Best-Response**Multiple-choice Knapsack: Given lists of items with values and weights, pick one from each list with max total value and total weight at most one.**Compression Best-Response**Depth: 1 2 3 4**Compression Best-Response**For j from 1..n, list of depth j: v( ) = Pr[win at depth j | x’ ] w( ) = 2-j … Kraft inequality (each with prob. p(.)) x’ in K**Other Duels**• Hiring duel: constraints defining Euclidean subspace correspond to hiring probabilities. • Binary search duel: similar to hiring duel, but constraints defining Euclidean subspace more complex (must correspond to search trees). • Racing duel: seems computationally hard, even though single-player problem easy.**Conclusion**• Every optimization problem has a duel. • Classic solutions (and all deterministic algorithms) can usually be badly beaten. • Duel can be easier or harder to solve, and can lead to inefficiencies. OPEN QUESTION: effect of duel on the solution to the optimization problem?