Experiments

184 Views

Download Presentation
## Experiments

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -

**Experiments**on combinatorial auctions Only some of the techniques of CABOB deployed in these results**Experimental setup**• Comparison between CABOB and CPLEX 7.0 • CPLEX 7.0 was the fastest earlier algorithm for the problem • System: 933 MHz Pentium III, 512 MB RAM, Linux 2.2 • 100 instances for each data point • All distributions produced distinct bids**Experiments: Random Distribution**• Random: Choose a random number of items without • replacement. Pick price from [0,1]. [Sandholm IJCAI-99] • CABOB does well primarily because of column dominance test. • CPLEX is able to solve these without searching 47% of the time.**Experiments: Weighted Random Distribution**• Weighted Random: Choose a random number of items without • replacement. Pick the price from between 0 and the number of • items in the bid. [Sandholm IJCAI-99] • CPLEX only searches 5% of the time. • CABOB searches 88% of the time.**Experiments: CATS Distributions**• CATS: Combinatorial Auction Test Suite [Leyton-Brown,Pearson, Shoham, 2000] • Random bid distributions • Modeled after realistic bidder preferences • Distributions have many parameters; we varied # bids and used default parameters for rest**Experiments: CATS PATHS**• CATS PATHS • Simulates bids on paths in a 2-D space. • Examples: truck routes, bandwidth allocation • Neither algorithm searched. CABOB’s simple preprocessing • techniques are faster**Experiments: CATS MATCHING**• CATS MATCHING • Simulates bid where complementarity is based on a temporal aspect of items • Example: airport take-off and landing slots • CPLEX never searched, CABOB rarely searched**Experiment: Uniform Distribution**• Each bid has same number of items • Prices chosen at random from [0,1] • Cases with few items per bid were hardest for earlier algorithms [Sandholm IJCAI-99, Fujishima et al IJCAI-99]**Experiments: Bounded Distribution…**• Bounded: Choose number of items between a lower and upper bound. Pick price between 0 and number of items in bid • Similar to Uniform distribution [Sandholm IJCAI-99], but more realistic • CABOB performs better here mainly because of the complete bid graph test**Experiments: Components Distribution**• A number of independent components from the uniform distribution where each bid has same #items • CABOB’s decomposition techniques capitalize on this structure, CPLEX does not**Anytime Performance**• Feasible solution found quickly • Solution improves rapidly over time • Optimal algorithms might be the best approximation algorithms too !**Generalizations of combinatorial auctions**• Free disposal • Substitutability • Multiple units of each item • Combinatorial exchanges (= many-to-many auctions) • Reservation prices • On items • On combinations • With substitutability • Combinatorial reverse auctions • Combinations of these generalizations**Generalization: substitutability [Sandholm IJCAI-99]**• What if agent 1 bids • $7 for {1,2} • $4 for {1} • $5 for {2} ? • Bids joined with XOR • Allows bidders to express general preferences • Groves-Clarke pricing mechanism can be applied to make truthful bidding a dominant strategy • Worst case: Need to bid on all 2#items-1 combinations • OR-of-XORs bids maintain full expressiveness & are more concise • E.g. (B2XOR B3) OR (B1XOR B3XOR B4) OR ... • Our algorithm applies (simply more edges in bid graph )**Winner determination in combinatorial auction**generalizations Tuomas Sandholm Subhash Suri Andrew Gilpin David Levine Carnegie Mellon University University of California CombineNet Inc. Computer Science Department Santa Barbara Pittsburgh, PA Dept of Computer Science**New generalizations of combinatorial auctions**• No free disposal (sellers cannot keep items, buyers cannot take extras) • Single- or multi-unit • Combinatorial reverse auctions • Single- or multi-unit • Combinatorial exchanges (= many-to-many auctions) • Single- or multi-unit**Combinatorial reverse auction**• Example: procurement in supply chains • Auctioneer wants to buy a set of items (has to get all) • Can take extras if there is free disposal • Sellers place bids on how cheaply they are willing to sell bundles of items • Thrm. Winner determination is NP-complete even in single-unit case with free disposal • Thrm. Single unit case with free disposal is approximable • k = 1 + log m (m = largest number of items that any bid contains) • Greedy algorithm: Keep choosing bid with lowest price / #items**No free disposal**• Free disposal: seller can keep items, buyers can take extras • Free disposal has been assumed in the combinatorial auction literature so far • In practice, freeness of disposal can vary across items & bidders • Without free disposal, the set of feasible solutions is same for combinatorial auctions & reverse auctions • Thrm. Even finding a feasible solution is NP-complete**Combinatorial exchange**• Example bid: (buy 20 tons of water, sell 10 cubic meters of hydrogen, sell 5 cubic meters of oxygen, ask $500) • Example application: manufacturing where a participant bids for inputs & outputs of a production plan simultaneously • Label bids as winning or losing so as to maximize (revealed) surplus: sum of amounts paid by bidders minus sum of amounts paid to bidders • On each item, sell quantity buy quantity • Equality if there is no free disposal • Thrm. NP-complete even in the single-unit case • Thrm. Inapproximable even in the single-unit case • Could also maximize trading volume • Thrm. Without free disposal, even finding a feasible solution is NP-complete (even in the single-unit case)**Experiments on generalizations**• 933 MHz Pentium III, 512M RAM • CPLEX 7.0 • Each plot point is mean over 50 instances • Significantly slower to find optimal solution than to prove infeasibility => we plot times on feasible instances • With free disposal, all instances are feasible • On distributions where CPLEX finds optimum with no search, it also tends to prove infeasibility with no search • On distributions where CPLEX needs search to find optimum, it also tends to need search to prove infeasibility**Lack of free disposal makes problem much harder**Complexity is polynomial in bids (even in worst case) Reverse auctions with free disposal seldom require search on these distributions Auctions require more search & more often (as inapproximability suggests) Single unit auctions & reverse auctions**Single unit auctions & reverse auctions…**• On decay distribution, even with free disposal, reverse auctions take longer than auctions (unlike approximability would suggest)**Multi-unit auctions & reverse auctions**• Decay-decay: Number of units for each item chosen with decay probability .99 • For each bid • Number of items chosen with decay probability a1 • For each item, #units chosen with decay probability a2 • All instances were easy. E.g., at a1 = .6, a2 = .9, LP solved • 74% of reverse auctions with free disposal • 52% of auctions with free disposal • 50% of auctions without free disposal • 22% of reverse auctions without free disposal • Hardest setting (a1 = .8, a2 = .8):**Multi-unit auctions & reverse auctions…**• CATS multipaths: • Almost all reverse auctions (with/without free disposal) & auctions without free disposal were infeasible • CPLEX could not scale to 2,000 bids on auctions with free disposal**Exchanges**• Exchange decay-decay distribution: For each bid • Number of items chosen with decay probability a1 • For each item, #units chosen with decay probability a2 • Sign is negated w.p. .5 • Price is random number between 0 and 1, multiplied by total #units (negative half the time) • Single unit case comes from a2 = 0 • Single-unit of each item • Scales well • Free disposal case slightly harder**Multi-unit exchanges**• CPLEX 7.1 scales very poorly: #bids/#items = 10 (a1 = .8, a2 = .8)**Multi-unit exchanges**• 50 items, 500 bids (a1 = .8, a2 = .8) • CPLEX 7.1 never finished • Without free disposal, did not even find a feasible solution • CPLEX 7.1 had very poor anytime performance:**Conclusions**• Generalizations of combinatorial auctions • No free disposal • Reverse auctions • Exchanges • Single- and multi-unit settings • Theoretical results • All these generalizations are NP-complete • With free disposal • Auction and exchanges are inapproximable • Reverse auctions are approximable • Even finding a feasible solution is NP-complete if XORs are allowed • Without free disposal, even finding a feasible solution is NP-complete • Experimental results • Search does well on auctions & at times even better on reverse auctions • Search does well on single-unit exchanges, poorly on multi-unit exchanges • Better algorithms needed • Lack of free disposal makes the problem much harder**Hot off the press[Kothari, Suri & Sandholm 2002]**• Q: How many bids have to be accepted fractionally (in worst case) so as to obtain maximum surplus in a multi-item multi-unit combinatorial exchange / combinatorial auction? • Trivial answer: #bids • A: #items (this is independent of #units) • Q: How many bids have to be accepted fractionally (in worst case) so as to maximize liquidity in a multi-item multi-unit combinatorial exchange? • Trivial answer: #bids • A: #items + 1 (this is independent of #units) • Q: How complex is it to find such a solution? • A: Polynomial time = fast