Challenging Problems and New Directions in Automated Negotiation

1. Challenging Problems and New Directions in Automated Negotiation Ben Kwang-Mong Sim Editor, IEEE Transactions on Systems, Man & Cybernetics Guest-Editor-in-Chief, IEEE Systems Journal, Official Journal of The IEEE Systems Council (formed by 15 IEEE Societies) Director, Multiagent Systems Lab. http://infcom.gist.ac.kr/~kmsim/MAS

2. What is Negotiation? • a process by which a group of agents communicate with one another to try and come to a mutually acceptable agreement on some matter • a form of decision-making with two or more actively involved agents who cannot make decisions independently (or achieve their goals unilaterally), and therefore must make concessions to achieve a compromise

3. Outline • One-to-one (bilateral) • Complete Information - Optimal strategy • Incomplete Information - Learning (GA + BL) • Many-to-many (multilateral) • Market-driven Negotiation • Relaxed-criteria Negotiation • From e-commerce negotiation to Grid commerce negotiation • Complex Negotiation • Concurrent negotiation in multiple markets • Establishment and decommitment of contracts • Coordination • Grid resource co-allocation

4. Buyer Agent Seller Agent The Problem of Bilateral Negotiation 0S, S: S’s deadline IPS : S’s initial price RPS : S’s reserve price 0B, B: B’s deadline IPB : B’s initial price RPB : B’s reserve price price price 0<lB<1 lB=1 lB>1 t t B S

5. Bilateral Negotiation • Negotiation with complete information [Sim et al. 2007] • Negotiation with incomplete information [Sim et al. 2007] • when one agent has an estimate of its opponent’s RP& deadline through Bayesian Learning (BL) and uses GA to search for an appropriate proposal taking into consideration its opponent’s estimated RP • when both agents have an estimate of each other’s RP & deadline through BL and use GA to search for an appropriate proposal taking into consideration an estimation of their opponent’s RP • Both agents do not have an estimate of each other’s RP • Reference [Sim et al. 2007] K. M. Sim, Y. Guo and B. Shi. Agents that Negotiate Optimally and Rapidly. Proceedings of the IEEE Congress on Evolutionary Computation, 2007, Singapore, pp. 1007-1014.

6. Bilateral Negotiation with Complete Information • Theorem 1.[Sim et al. 2007]Agent B achieves maximal utility when it adopts the strategy • Theorem 2. [Sim et al. 2007]Agent S obtains the maximal utility when it adopts the strategy

7. Bilateral Negotiation with Complete Information S’s concession making: S will propose RPS at S: price minimal possible agreement-price for B = RPs Optimal strategy for B satisfies t S B B’s concession making:

8. Bilateral Negotiation with Complete Information

9. Bilateral Negotiation with Complete Information • Theorem 1. [Sim et al. 2007]Agent B achieves maximal utility when it adopts the strategy • Theorem 2. [Sim et al. 2007]Agent S obtains the maximal utility when it adopts the strategy

10. Bilateral Negotiation with Complete Information Non-optimal strategy price Agreement price >RPs t S B

11. Bilateral Negotiation with Complete Information Non-optimal strategy price No agreement t S B

12. Bilateral Negotiation with incomplete information • BLGAN containing two procedures [Sim et. Al 2007] : • BL-Procedure • GA-Procedure • BLGAN Overview : In each negotiation round, after receiving the opponent’s proposal, the agent will decide whether to accept the proposal. • If it agrees, a successful deal is made. • Else, it generates the next proposal as follows: (1) In BL-Procedure, a Bayesian updating method is adopted to learnopponent's RP & a procedure is used to estimate opponent’s deadline; (2) Use the estimated opponent's RP and deadline, compute a new  (3) Use  to generate possible proposal Probl; (4) Compensate for error in estimating opponent's RP using a genetic algorithm (GA) to search for a better proposal within a dynamic search space confined to an area around Probl . (5) A possibly better proposal is revised and sent to the opponent.

13. Experimental Results: Both agents learn

14. Experimental Results: Only S learns

15. Multi-lateral Negotiation e-market Buyer Agent Seller Seller Agent Buyer Contract Seller Agent Buyer Agent Seller Buyer Seller Agent Buyer Agent Seller Buyer

16. current Narrowing differences kt next kt+1 D t How much to concede? Market-Driven Negotiation Making adjustable amountsof concession Time, Opportunity, Competition

17. The Role of Time • Bilateral negotiation - only time is considered. • kt+1 = T(t,, )kt T(t,, ) = 1-[t/] • t : current round, : deadline, • [0, ] : stategy Attitudes toward time “Sit-and-wait” k l= Conservative Difference l>1 l=1 k0 k Conciliatory k` 0<l<1 t Linear t t` t0  Negotiation round

18. k1 Outside Options, Conflict & Differences Probability of reaching consensus v BS1 v S1B B S1 v S2B v BS2 k2 S2 B v BSn v SnB kn Sn B kt+1 = O(n, v BSi, {v SiB})kt O(n, v BSi, {v SiB})1, concede less to narrow difference O(n, v BSi, {v SiB})0, concede more to narrow difference

19. chance of not generating the highest utility for its trading parties nTrading parties m-1 competitors Each trading party i m-1 competitors Rivalry & Competition Rivalry inherent in many-to-many negotiation Utility maximizing – most likely to reach agreement, if proposal ranked highest Probabilityof being ranked highest by some trading party: Buyer-seller ratio • kt+1 = C(m,n)kt • C(m,n)1, less competition  concede less • C(m,n)0, more competition  concede more chance of not generating the highest utility for its trading party

20. kt Adjustable Amount of Concession current Narrowing differences Difference next kt+1 D t time kt+1= f[O(n,v BSi,{v SiB}),C(m,n), T(t,, )] kt

21. Empirical Results Conservative Conciliatory Linear

22. Relaxed-Criteria Negotiation • Conventional negotiation: a process by which a group of agents communicate with one another to try and come to a mutually acceptable agreement on some matter • Relaxed-criteria negotiation [Sim 2004]:a process by which a group of agents communicate with one another to try and come to a roughly acceptable agreement on some matter Reference [Sim 2004] K. M. Sim and S.Y. Wang. Flexible Negotiation Agent with Relaxed Decision Rules. IEEE Transactionson Systems, Man and Cybernetics, Part B, Vol. 34, No. 3., pp. 1602- 1608, Jun., 2004.

23. Rubinstein’s Alternating Offers Protocol • Agents negotiate by making proposals in alternate rounds • If no agreement is reached, negotiation proceeds to another round. • Negotiation between two agents terminates: • (i) when an agreement is reached or • (ii) with a conflict when one of the two agents’ deadline is reached. • An agreement is reached when one agent proposes a deal that matches (or exceeds) what another agent asks for.

24. Rubinstein’s Alternating Offers Protocol agreement Counter-propose accept Start negotiation Propose Counter-propose deadline reached deadline reached accept conflict agreement conflict

25. Sim’s Relaxed-Criteria Protocol • Agents negotiate by making proposals in alternate rounds • If no agreement is reached, negotiation proceeds to another round. • Negotiation between two agents terminates: • (i) when an agreement is reached or • (ii) with a conflict when one of the two agents’ deadline is reached. • Reaching Agreements: • R1: An agreement is reached when one agent proposes a deal that matches (or exceeds) what another agent asks for. • R2: An agreement is reached if the offer is sufficiently close (albeit, it does not totally match the agent’s bargaining terms).

26. Relaxing Criteria • How close is acceptable? • Remaining time (deadline fast approaching?) • Degree of competition • Relative distances between the proposal of an agent and all the proposals of its opponents ? ?

27. best proposal very close relativeto other proposals Relaxing Criteria

28. Fuzzy Decision Controller (FDC) • Vague concepts : • fast approaching deadlines • strong competition • proposals are prettyclose • very urgent to acquire product FDC η : degree of competition : time pressure (remaining time) : Relative distances between the proposal of an agent and all the proposals of its opponents

29. A high value of indicates that an agent faces more competition A high value of indicates that an agent’s deadline is fast approaching A high value of indicates that the best proposal from its opponent is very close to an agent’s proposal relative to all other proposals from all other opponents.

30. Empirical Results

31. Grid Commercialization, Utility Computing &Negotiation • Utility computing (http://www.sun.com/service/sungrid/index.jsp) • a business model whereby computer resources are provided on an on-demand and pay-per-use basis • seeks to maximize efficient use of computing resources and minimize user costs • Grid (http://www.gridcomputing.com/gridfaq.html) • cyberinfrastructure for coupling & pooling distributed resources, which can be used for realizing utility computing. • Negotiation (our work) • establishment & management of contracts • allocation of resources to meet competing demands from multiple consumers

32. From Relaxed-criteria e-Negotiation to Relaxed-criteria G-negotiation • Applying Sim’s Relaxed-Criteria Negotiation to Grid Commerce • In a Grid market, consumers (applications) & resource providers negotiate to establish contracts for resource utilization. • Relaxed-Criteria Negotiation enhances success rates in acquiring resources & perhaps acquire resources more rapidly

33. ? ? Relaxation Criteria • How close is acceptable? • amount of relaxation (η) • Consumers • Recent statistics in failing/succeeding in acquiring resources (fst) • Demand for computing resources (dft) • Providers • Amount of resources currently being used (ult) • Recent resource requests from consumers (rft)

34. Relaxation Criteria • Vague concepts : • high demand for computing resources • large amount of resources currently being used • proposals are pretty close fst dft FDC-C η ult rft FDC-P η

35. If I was quite successful in acquiring resources and I have very low resource demand Then I would agree only if the difference is very small

36. If much of my resources are being occupied and I receive a lot of requests Then I would be very unlikely to relax my bargaining terms

37. Empirical Results

38. Complex Negotiation in multiple markets market 3 market 1 Buyer-1 Seller-3-1 Seller-1-1 Seller-3-2 Seller-1-2 Buyer-2 Seller-3-k Seller-1-i Seller-2-1 market 2 Seller-2-2 Buyer-3 Buyer-4 Seller-2-j

40. Grid Resource Co-allocation Grid Resource Co-allocation problem: to allocate to an application multiple resources belonging to possibly different administrative domains simultaneously before a deadline.

41. Use concurrent negotiation for Grid resource co-allocation? • Negotiation • obtain commitments or contracts from a resource owner to provide a service/resource • a means for different parties to resolve differences and conflicting goals • players in Grid marketplace to optimize their return-on-investment/cost • Concurrent Negotiation • Multiple negotiations to acquire multiple resources • Coordination is needed in Resource Co-allocation

42. Co-allocation  Concurrent Negotiation

43. Concurrent G-Negotiation Mechanism Contracting Coordination

44. Concurrent G-Negotiation MechanismCommitment Managers(CM) • Each one-to-many negotiation has a commitment managerCMi • Each CMi • manages multiple concurrent bi-lateral negotiation threads • manages both commitment and de-commitment of (intermediate) contracts for resources • de-commitment • if a consumer cannot acquire all its required resources before deadline, it can release those resources acquired, so that providers can re-assign them to other consumers • allows an agent to reach an intermediate deal & continue to search for a better deal one-to-manynegotiation for Resource R1

45. Concurrent G-Negotiation MechanismCoordinator Module • Each CMisends the predictedchange in utility in next negotiation round for resource Ri to the coordinator. • Using the information supplied by each CMi, coordinator decides when to terminate the entire concurrent negotiation process. change in utility in next round

46. Contracting Algorithm • Step 1: Estimate reneging probability • Step 2: Compute expected utility of provider’s proposal. • Step 3: Determine if provider’s proposal is acceptable taking into account penalty payment • Step 4: Request for contract. • Step 5: If receive confirmation then accept contract else revise proposal by making concession.

47. Commitment Management StrategyStep 1: Reneging Probability General Idea Reneging prob. of a provider associated with its price proposal relative to others Resource Provider 1 Commitment Manager i Resource Provider 2 Resource Provider ni • the price proposals of providers 1, 2, …,ni

48. Step 1: Reneging Probability • Case 1: Very high prob. that resource provider reneges from deal • From consumer’s perspective, if proposal of Resource Provider j is too far below the average • can be easily chosen by other consumers. • Case 2: When distance of resource provider j’s proposal is below average, but not too far below average (e.g., WITHIN one standard deviation), it is believed that provider jis not likely to renege on deal • if provider j reneges, it needs to pay penalty • since it is close to the average, it is unlikely to obtain a better utility by breaking a deal Case 3: Resource provider j’s proposal is above average • relatively good deal unlikely to renege

49. Commitment Management StrategiesStep 2: Expected Utility Consumer’s expected utility from proposal of Resource Provider jat current round t : Does not renege Provider reneges • where is the consumer’s utility function for resource i. • utility function • preference ordering for negotiation outcome • higher number =>outcome more preferred

50. Commitment Management Strategies Step 3: Acceptable Proposals A provider’s proposal is acceptable to the consumer at current round t if it generates an expected utility that is equal to or higher than the utility generated from the consumer’s counter-proposal Case 1: No previous commitment: