Mechanism Design for Computationally Bounded Agents

1. Mechanism Design for Computationally Bounded Agents Kate Larson Carnegie Mellon University Thesis Proposal April 8, 2002

2. Introduction • Fundamental problem in building open distributed systems: • There are users and agents acting in their own self interest. • Designer wishes to guarantee optimal system wide solutions while being unable to enforce particular behavior on users.

3. Introduction • Techniques from game theory and mechanism design have been very useful: • Routing in networks • Resource allocation in operating systems • Auction design in electronic markets • Bidding agents for electronic auctions • ….

4. Introduction • However, perfect rationality is often assumed! • But in real systems… Hard problems Cost Time constraints

5. Package Delivery and Vehicle Routing Chicago Chicago to Home Toronto to Home The delivery route can be computed. Chicago to Toronto Home 2 Depot 3 4 Home to Chicago 1 5 Toronto

6. Package Delivery and Vehicle Routing Chicago Chicago to Home Chicago to Toronto Home 4 2 Depot 1 Home to Chicago 5 3 Toronto to Home The new package can be easily fit into the delivery route. The cost can be kept low, and thus the bid also. Toronto

7. Computing of Valuations • Agents may need to compute their valuations for (bundles of) goods • Vehicle routing problem in transportation exchanges • Manufacturing scheduling problem in procurement • Value of a set of items =value of solution with items - value of solution without • May involve solving multiple NP-complete problems • Infeasible to solve optimally

8. Incentives • Game theory handles incentives • System designer specifies the mechanism(auction rules) • Each agent picks its own strategy • Computational issues have to be handled also! • Computing as part of an agent’s strategy • Interaction between incentives and computing • Equilibrium for rational agents is not always one for computationally bounded agents!

9. My Approach Resource Bounded Reasoning from AI Game Theory and Mechanism Design Normative model of bounded rationality. Mechanism design for computationally bounded agents.

10. Thesis Statement • It is possible to incorporate agents’ computational actions into a game theoretic model. • Agents’ computational restrictions have a strong impact on the choice of strategies. • It is possible to design mechanisms that take into account agents’ computational restrictions.

11. Structure Model of bounded rationality Game theoretic formulation Impact on existing mechanisms Mechanisms for bounded agents Experiments

12. Outline • A model of bounded rationality • A normative model of deliberation control for computationally bounded agents • Game theory for computationally bounded agents • The impact of agents’ computation • Mechanism design for computationally bounded agents • Experiments • Schedule

13. Restricted Computational Resources • Agents have limited computational resources • Deadlines • All parcels must be delivered by Friday evening! • Battery on my laptop only lasts 3 hours • Cost associated with computation and deliberation • Sending jobs to rendering farms

14. Anytime Algorithms • Anytime algorithms can be used to approximate valuations • Anytime algorithms can return a solution at any time • Solution improves over time Allows a trade off between computing time and solution quality • Examples • Iterative refinement algorithms: Local search, simulated annealing • Search algorithms: Depth first search, branch and bound

15. Performance Profiles • Performance profiles describe how computation changes the solution • Characterize the quality of an algorithm’s output as a function of computing time • Performance profile representation is important • Earlier methods were not normative: They did not capture all possible ways an agent can control its deliberation

16. Variance introduced by different problem instances Solution quality Computing time Performance Profiles Deterministic performance profile Solution quality Optimum Computing time [Horvitz 87, 89, Dean & Boddy 89]

17. Conditioning on solution quality so far [Hansen & Zilberstein 96] Ignores conditioning on the path Table-based Representation of Uncertainty in Performance Profiles [Zilberstein & Russell 91, 96] Solution quality Computing time

18. 5 P(B|A) B 4 4 10 P(1) A P(2) 0 7 C P(C|A) Solution Quality P(3) 15 2 5 20 Value Node Random Node Performance Profile Tree [Larson & Sandholm 2001a] Stores value of solution so far Captures uncertainty from random algorithms, unknown algorithms

19. Properties of Performance Profile Trees • Normative - allows conditioning on everything including • Path of solution quality • Path of other solution features • Problem instance features • Can model uncertainty from • Randomized algorithms • Lack of knowledge about what algorithms other agents use • Problem instances

20. Outline • A model of bounded rationality • Game theory for computationally bounded agents • Roles of computation • Strategic computing • Deliberation equilibrium • Effects of agents’ computation • Mechanism design for computationally bounded agents • Experiments • Schedule

21. Improving: Agents compute to improve valuations Computation changes the valuation Example: Solving TSP’s to find cost of best route Refining: Agents compute to refine valuations Computation changes agents’ beliefs about valuations Example: Researching if an art work is a forgery Roles of Computing

22. Roles of Computation • For computationally bounded agents, computing has several strategic roles: • Improves or refines the solution to the agent’s own problem • Reduces uncertainty as to what future computing steps will bring • Improves the agent’s knowledge about others’ valuations • Improves the agent’s knowledge about what problems others may have computed on

23. Strategic Computing • Good estimates of other agents’ valuations can allow an agent to tailor its strategy to achieve higher utility • Strong strategic computing:Agent uses some of its deliberation resources to compute on others’ problems • Weak strategic computing:Agent uses information from others’ performance profiles

24. Incorporate computation actions into agents’ strategies. Analyze games for deliberation equilibria Strategic Computing • How an agent should allocate its computation can depend on • how others allocate their computation • what actions they all plan on taking

25. Deliberation Equilibria • A (Nash, dominant, etc.) deliberation equilibrium for computationally bounded agents is a (Nash, dominant, etc.) equilibrium where: • The agents’ computing strategies form a (Nash, dominant, etc) equilibrium, and • The agents’ noncomputing strategies form a (Nash, dominant, etc) equilibrium.

26. Outline • A model of bounded rationality • Game theory for computationally bounded agents • Effects of agents’ computation • One-to-One Negotiation (Bargaining) • One-to-Many Negotiation (Auctions) • Many-to-Many Negotiation (Exchanges) • Mechanism design for computationally bounded agents • Experiments • Schedule

27. B A A B Bargaining • Bargaining A B A B A B

28. Take-it-or-leave-it: One agent makes an offer to the other who can accept or reject. Determined under what settings agents have dominant strategies. Determined under what settings agents will not split their computation. Designed algorithms for determining offline, agents optimal online strategies Bargaining[Larson and Sandholm, 2001a]

29. Bargaining[Larson and Sandholm, 2002] • Alternating Offers Model: Agents take turns making proposals and counter proposals • Despite richer strategy space, alternating offers model is similar to take-it-or-leave it model • In (Bayes Nash) deliberation equilibrium, agents behave as though they were in take-it-or-leave-it setting

30. Future Bargaining Work • So far have only worked in settings where computation is free but limited • What happens if computation is costly? Conjecture: Costly computation will not change the results.

31. yes yes yes yes no yes no yes yes yes Auctions and Strategic Computing Counter-speculation by rational agents ? Auction mechanism Strong strategic computing Limited computation Costly computation Single item for sale First price sealed-bid yes Dutch(1st price descending) yes Vickrey(2nd price sealed bid) no English (1st price ascending) no Multiple items for sale Generalized Vickrey On which agent, bundle pair to allocate next computation step ? no [Larson and Sandholm 2001b, Larson and Sandholm 2001c]

32. Exchanges • Exchanges are a generalization of auctions where there are potentially multiple buyers and multiple sellers • Does strategic computing occur in exchanges? • Does the type of restriction on agents’ computational capabilities determine the occurrence of strategic computation? Conjecture: Strategic computation can occur with either type of restriction (costly or limited computation).

33. Outline • A model of bounded rationality • Game theory for computationally bounded agents • Effects of agents’ computation • Mechanism design for computationally bounded agents • Experiments • Schedule

34. Mechanism Design • So far have focused on standard mechanisms and determining their strengths and limitations if agents have computational restriction. • Change focus: • Mechanisms for computationally restricted agents. • Measures for mechanisms.

35. Issues for Mechanism Design • There are several issues that must be addressed for computationally restricted agents • What should agents report to the mechanism? • Valuations once computed? • Valuations agents expect to obtain? • Results of partial computation? • What is the role of the mechanism? • To specify allocations only? • To specify allocations and computation policies?

36. Issues for Mechanism Design • Does there exist a mechanism that is: • Strategy proof, • Efficient, • Individually rational, and • Limits wastage of computational resources.

37. Measuring Mechanisms: Miscomputing Ratio • How does allowing agents free choice of computing policies effect the outcome, o? • Miscomputing Ratio: R=SW(o*)/SW(o(NE)) SW(o*)=Social welfare if a global controller enforces computing policies so as to maximize social welfare SW(o(NE))=Social welfare of worst Nash Eq.

38. Results and Future Work • Results (Vickrey auctions): • If agents have limited or costly computation then the miscomputing ratio can be unbounded. • The choice of appropriate cost functions can motivate bidders to choose strategies that maximize social welfare. • Future Work: • Constraining deadlines or cost functions • Using performance profile trees to predict R • Constraining performance profile trees • Comparing mechanisms using miscomputing ratio, R

39. Outline • A model of bounded rationality • Game theory for computationally bounded agents • Effects of agents’ computation • Mechanism design for computationally bounded agents • Experiments • Schedule

40. Experiments • Goals: • To demonstrate that the performance profile tree is a feasible model. • To demonstrate that the performance profile tree model is general. • To determine if strategic computing is an artifice of analysis or something that occurs in the “real world”. • Tools for controlling computation and visualization of deliberation and negotiation

41. Status completed Model of bounded rationality partially future work Game theoretic formulation Impact on existing mechanisms Mechanisms for bounded agents Experiments

42. Schedule Sp02 Sm02 F02 Sp03 Sm03 F03

43. Related Publications • Larson and Sandholm, 2001a, Bargaining with Limited Computation:Deliberation Equilibrium, Artificial Intelligence, 132(2), pp.183-217 • Larson and Sandholm 2001b, Computationally Limited Agents in Auctions, Workshop on Agent based approaches to B2B, Autonomous Agents 2001 • Larson and Sandholm, 2001c, Costly Valuation Computation in Auction, TARK VIII • Larson and Sandholm, 2002, An Alternating Offers Bargaining Model for Computationally Limited Agents, AAMAS 2002 • Larson and Sandholm, 2002, Miscomputing Ratio: The Social Cost of Selfish Computing, draft.

44. Other Publications • Larson and Sandholm, 2000, Anytime Coalition Generation: An Average Case Study, Journal of Experimental and Theoretical Artificial Intelligence, 12(2000),23-42. • Sandholm, Larson, Andersson, Shehory, and Tohme, 1999, Anytime Coalition Structure Generation with Worst Case Guarantees, Artificial Intelligence, (111)1-2, pp209-238.

45. Future Auction Work: Reverse Auctions • Are they different from standard auctions if agents have bounded computational resources? • Maybe or maybe not! • Reverse auctions differ from auctions in: • Complexity of winner determination • Communication complexity • At first look, it does not appear as though bidding strategies are different.

46. Related Work • Parkes 2001, Iterated combinatorial auctions so that both the computational burden of agents and mechanism are reduced • Persico 2000, Information acquisition in auctions • Bergemann and Pesendorfer 2001, Model where the auctioneer controls what information agents have available to them • Bergemann and Välimäki 2001, Costly information acquisition, but no clear model of how information is obtained

47. Related Work • Nisan and Ronen 2000: Algorithmic mechanism design. What happens if the mechanism is computationally limited? Investigates ways of introducing approximation algorithms to compute outcomes without loosing incentive compatibility. • Kfir-Dahav, Monderer, Tennenholtz 2000

48. Experiments • To demonstrate that the performance profile tree is a feasible model: • Multi-item auction where agents submit bids where the value is determined by the solution to a TSP problem • Prebundling to simplify the winner determination problem and to restrict the number of optimization problems agents are faced with • Focus on having agents use performance profile trees, determining how to make the performance profile tree model feasible for actual problems.

49. Experiments • To demonstrate that the performance profile tree is general: • Single item auction where agents valuations of the item is based on schedules they compute. • Use off the shelf schedulers (Micro-Boss, or a stochastic scheduler from Steve Smith’s group) to show that the techniques I propose are general

50. Experiments • To determine if strategic computation is an artifice of analysis or something that occurs in the “real world”: • Multi-item reverse auction using data from actual delivery companies • Agents must solve multiagent vehicle routing problems • How does one tailor performance profile trees to actual data without losing their analytical power? • Does strategic computing occur?