Reaching Agreements: Negotiation

1. Reaching Agreements: Negotiation

2. Typical Competition Mechanisms • Auction:allocate goods or tasks to agents through market. Need a richer technique for reaching agreements • Negotiation:reach agreements through interaction. • Argumentation:resolve confliction through debates.

3. Negotiation Mechanism Negotiation is the process of reaching agreements on matters of common interest. It usually proceeds in a series of rounds, with every agent making a proposal at every round. • Issues in negotiation process: • Negotiation Space: Allpossible deals that agents can make, i.e., the set of candidate deals. • Negotiation Protocol: – A rule that determines the process of a negotiation: how and when a proposal can be made, when a deal has been struck, when the negotiation should be terminated, and so. • Negotiation Strategy: When and what proposals should be made.

4. Protocol • Means kinds of deals that can be made • Means sequence of offers and counter-offers • Protocol is like rules of chess game, whereas strategy is way in which player decides which move to make

5. Game Theory • Computers make concrete the notion of strategy which is central to game playing

6. Mechanisms Design • Mechanism design is the design of protocols for governing multi-agent interactions. • Desirable properties of mechanisms are: • Convergence/guaranteed success • Maximising global welfare: sum of agent benefits are maximized • Pareto efficiency • Individual rationality • Stability: no agent should have incentive to deviate from strategy • Simplicity: low computational demands, little communication • Distribution: no central decision maker • Symmetry: not wwant agents to play different roles

7. Attributes not universally accepted • Sometimes be tradeoffs – efficiency and stability and sometimes in conflict with each other

8. Protocols • What is an elevator protocol? • Direction you face • How close to another do you stand • Are you allowed to talk to a stranger? • Where you stand (not in front of buttons, near back) • What to do if person is running for elevator

9. Negotiation Protocol • Who begins • Take turns • Build off previous offers • Obligations • Privacy • Legal proposals you can make as a result of negotiation history

10. Proposal Counter Proposal Agenti concedes Agenti Agentj Negotiation Process 1 • Negotiation usually proceeds in a series of rounds, with every agent making a proposal at every round. • Communication during negotiation:

11. Point of Acceptance/ aggreement Proposals by Aj Proposals by Ai Negotiation Process 2 • Another way of looking at the negotiation process is:

12. Typical Negotiation Problems Task-Oriented Domains(TOD): Domains in which an agent's activity can be defined in terms of a set of tasks that it has to achieve. The target of a negotiation is to minimize the cost of completing the tasks. State Oriented Domains(SOD): Domains where each agent is concerned with moving the world from an initial state into one of a set of goal states. The target of a negotiation is to achieve a common goal. Main attribute: actions have side effects (positive/negative) Worth Oriented Domains(WOD): Domains where agents assign a worth to each potential state, which captures its desirability for the agent. The target of a negotiation is to maximize mutual worth.

13. Single issue negotiation • Like money • Symmetric (what is more for you is less for me, both benefit equally if roles reversed) • If you get more money, I get less • If you travel less than I do, I would benefit by switching routes with you

14. Multiple Issue negotiation • Could be hundreds of issues (cost, delivery date, size, quality) • Some may be inter-related (as size goes down, cost goes down, quality goes up?) • Not clear what a true concession is (larger may be cheaper, but harder to store or spoils before can be used) • May not even be clear what is up for negotiation (I didn’t realize not having any test was an option)

15. How many agents are involved? • One to one • One to many (auction is an example of one seller and many buyers) • Many to many (could be divided into buyers and sellers, or all could be equal) • n(n-1)/2 number of pairs

16. Negotiation Domains:Task-oriented • ”Domains in which an agent’s activity can be defined in terms of a set of tasks that it has to achieve”, (Rosenschein & Zlotkin, 1994) • An agent can carry out the tasks without interference from other agents • All resources are available to the agent • Tasks redistributed for the benefit of all agents

17. Formalization of TOD A Task Oriented Domain(TOD) is a triple <T, Ag, c> where: • T is a finite set of all possible tasks; • Ag={A1, A2,…, An} is a list of participant agents; • c:(T)R+defines cost of executing each subset of tasks. • Assumptions on cost function: • c() = 0. • The cost of a subset of tasks does not depend on who carries out them. (Idealized situation) • Cost function is monotonic, which means that more tasks, more cost. (It can’t cost less to take on more tasks.) • T1  T2 implies c(T1)  c(T2)

18. Redistribution of Tasks Given a TOD <T, {A1,A2}, c>, • An encounter (instance) within the TODis an ordered list (T1, T2) such that for all k, Tk  T. This is an original allocation of tasks that they might want to reallocate. • A pure deal on an encounteris the redistribution of tasks among agents: (D1, D2), such that D1 D2= T1 T2 Specifically, (T1, T2) is called the conflict deal. • For each deal=(D1, D2), the cost of such a deal to agent k is Costk()=c(Dk)

19. Examples of TOD • Parcel Delivery: Several couriers have to deliver sets of parcels to different cities. The target of negotiation is to reallocate deliveries so that the cost of travel to each courier is minimal. • Database Queries: Several agents have access to a common database, and each has to carry out a set of queries. The target of negotiation is to arrange queries so as to maximize efficiency of database operations (Join, Projection, Union, Intersection, …) .

20. Possible Deals Consider an encounter from the Parcel Delivery Domain. Suppose we have two agents. Both agents have parcels to deliver to city aand only agent 2 has parcels to deliver to city b. There are nine distinct pure deals in this encounter: • ({a}, {b}) • ({b}, {a}) • ({a,b}, ) • (, {a,b}) • ({a}, {a,b}) ({b}, {a,b}) ({a,b}, {a}) ({a,b}, {b}) ({a,b}, {a,b}) the conflict deal

21. Utility Function for Agents Given an encounter (T1, T2), the utility function for each agent is defined as follow: Utilityk()=c(Tk)-Costk() where • =(D1, D2) is a deal; • c(Tk) is the stand-alone cost to agent k (the cost of achieving its goal with no help) • Costk() is the cost of its part of the deal. Note that the utility of the conflict deal is always 0.

22. Parcel Delivery Domain (cont) Distribution Point Cost function: c()=0 c({a})=1 c({b})=1 c({a,b)}=3 1 1 city a city b Utility for agent 1: Utility1({a}, {b}) = 0 Utility1({b}, {a}) = 0 Utility1({a, b}, ) = -2 Utility1(, {a, b}) = 1 … Utility for agent 2: Utility2({a}, {b}) = 2 Utility2({b}, {a}) = 2 Utility2({a, b}, ) = 3 Utility2(, {a, b}) = 0 …

23. Dominant Deals • Deal  dominates deal ' if  is better for at least one agent and not worse for the other, i.e., •  is at least as good for every agent as ': k{1,2},Utilityk() Utilityk(') •  is better for some agent than ': k{1,2},Utilityk()>Utilityk(') • Deal  weakly dominates deal ' if at least the first condition holds. Any reasonable agent would prefer (or go along with)  over ' if  dominates or weakly dominates '.

24. Negotiation Set: Space of Negotiation • A deal  is called individual rational if  weaklydominates the conflict deal. (no worse than what you have already) • A deal  is called Pareto optimalif there does not exit another deal ' that dominates . (best deal for x without disadvantaging y) • The set of all deals that are individual rational and Pareto optimal is called the negotiation set(NS).

25. Utility Function for Agents Utility2({a}, {b}) =2 Utility2 ({b}, {a})=2 Utility2 ({a,b}, )=3 Utility2 (, {a,b})=0 Utility2 ({a}, {a,b})=0 Utility2 ({b}, {a,b})=0 Utility2 ({a,b}, {a})=2 Utility2 ({a,b}, {b})=2 Utility2 ({a,b}, {a,b})=0 • Utility1({a}, {b}) =0 • Utility1({b}, {a})=0 • Utility1({a,b}, )=-2 • Utility1(, {a,b})=1 • Utility1({a}, {a,b})=0 • Utility1({b}, {a,b})=0 • Utility1({a,b}, {a})=-2 • Utility1({a,b}, {b})=-2 • Utility1({a,b}, {a,b})=-2

26. Individual Deals • ({a}, {b}) • ({b}, {a}) • ({a,b}, ) • (, {a,b}) • ({a}, {a,b}) • ({b}, {a,b}) • ({a,b}, {a}) • ({a,b}, {b}) • ({a,b}, {a,b}) ({a}, {b}) ({b}, {a}) (, {a,b}) ({a}, {a,b}) ({b}, {a,b}) individual rational

27. Pareto Optimal Deals • ({a}, {b}) • ({b}, {a}) • ({a,b}, ) • (, {a,b}) • ({a}, {a,b}) • ({b}, {a,b}) • ({a,b}, {a}) • ({a,b}, {b}) • ({a,b}, {a,b}) ({a}, {b}) ({b}, {a}) ({a,b}, ) (, {a,b}) Pareto Optimal

28. Negotiation Set Individual Rational Deals ({a}, {b}) ({b}, {a}) (, {a,b}) ({a}, {a,b}) ({b}, {a,b}) Pareto Optimal Deals ({a}, {b}) ({b}, {a}) ({a,b}, ) (, {a,b}) Negotiation Set ({a}, {b}) ({b}, {a}) (, {a,b})

29. Negotiation Set illustrated • Create a scatter plot of the utility for i over the utility for j • Only those where both is positive are individually rational (for both) (origin is conflict deal) • Which are pareto optimal? Utility for i Utility for j

30. Negotiation Set in Task-oriented Domains Utility for agent i Negotiation set: (pareto optimal+ Individual rational) B A C Utility of conflict Deal for agent i The circle delimits the space of all possible deals E Conflict deal D Utility for agent j Utility of conflict Deal for agent j

31. The Monotonic Concession Protocol Rules of this protocol are as follows. . . • Negotiation proceeds in rounds. • On round 1, agents simultaneously propose a deal from the negotiation set. (can re-propose same one) • Agreement is reached if one agent finds that the deal proposed by the other is at least as good or better than its proposal. • If no agreement is reached, then negotiation proceeds to another round of simultaneous proposals. • An agent is not allowed to offer the other agent less (in term of utility ) than it did in the previous round. It can either stand still or make a concession. Assumes we know what the other agent values. • If neither agent makes a concession in some round, then negotiation terminates, with the conflict deal.

32. Condition to Consent an Agreement If one of the agents finds that the deal proposed by the other is at least as good or better than the proposal it made. Utility1(2) Utility1(1) and Utility2(1) Utility2(2)

33. The Monotonic Concession Protocol • Advantages: • Symmetrically distributed (no agent plays a special role) • Ensures convergence • It will not go on indefinitely • Disadvantages: • Agents can run into conflicts • Inefficient – no quarantee that an agreement will be reached quickly

34. Negotiation Strategy Given the negotiation space and the Monotonic Concession Protocol, a strategy of negotiation is an answer to the following questions: • What should an agent’s first proposal be? • On any given round, who should concede? • If an agent concedes, then how much should it concede?

35. The Zeuthen Strategy Q: What should my first proposal be? A: the best deal for you among all possible deals in the negotiation set. (Is a way of telling others what you value.) agent 2's best deal Agent 1's best deal

36. The Zeuthen Strategy Q:Do I need to make a concession in this round? A: If you are not willing to risk a conflict, you should make a concession. How much am I willing to risk a conflict? How much am I willing to risk a conflict? Agent 1's best deal agent 2's best deal

37. Willingness to Risk Conflict Suppose you have conceded a lot. Then: – You have lost your expected utility (closer to zero). – In case conflict occurs, you are not much worse off. – You are more willingto risk conflict. An agent will be more willing to risk conflict if the difference in utility between your loss in making an concession and your loss in taking a conflict deal with respect to your current offer.

38. Risk Evaluation • You have to calculate? • How much you will lose if you make a concession and accept your opponent's offer? • How much you will lose if you stand still which causes a conflict? utility agent i loses by conceding and accepting agent j's offer riski= utility agent 1 loses by not conceding and causing a conflict Utilityi (i )-Utilityi (j ) = Utilityi (i ) where i and i are the current offer of agent i and j, respectively. Thus, riski is willingness to risk conflict (1 is perfectly willing to risk)

39. Maximum loss from conflict How much am I willing to riskaconflict? Maximum loss from concession Conflict deal Ai best deal Aj best deal The Risk Factor One way to think about which agent should concede is to consider how much each has to loose by running into conflict at that point.

40. The Zeuthen Strategy Q:If I concedes, then how much should I concede? A: Just enough to change the balance of risk. (Otherwise, it will just be your turn to concede again at the next round)

41. About MCP and Zeuthen Strategies • Advantages: • Simple and reflects the way human negotiations work. • Stability – in Nash equilibrium – if one agent is using the strategy, then the other can do no better than using it him/herself. • Disadvantages: • Computationally expensive – players need to compute the entire negotiation set. • Communication burden – negotiation process may involve several steps.

42. Parcel Delivery Domain: recall, agent 1 delivered to a, agent 2 delivered to both a and b Negotiation Set ({a}, {b}) ({b}, {a}) (, {a,b}) Utility of agent 1 Utility1({a}, {b}) = 0 Utility1({b}, {a}) = 0 Utility1(, {a,b})=1 Utility of agent 2 Utility2({a}, {b}) =2 Utility2({b}, {a}) = 2 Utility2(, {a,b})=0 First offer (, {a,b}) ({a}, {b}) Risk of conflict 1 1 Agent 1 Agent 2 Can they reach an agreement? Who will concede?

43. Conflict Deal He should concede. He should concede. Agent 1's best deal agent 2's best deal

44. Parcel Delivery Domain: Example 2 Distribution Point Conflict Deal: ({a,b,c,d}, {a,b,c,d}) 7 7 1 1 1 a c d Negotiation Set: ({a,b,c,d}, ) ({a,b,c), {d}) ({a,b}, {c,d}) ({a}, {b,c,d}) (, {a,b,c,d}) b Cost function: c()=0 c({a})=c({d})=7 c({b})=c({c})=c({a,b})=c({c,d})=8 c({b,c})=c({a,b,c})=c({b,c,d})=9 c({a,d})=c({a,b,d})=c({a,c,d})=c({a,b,c,d})=10

45. Parcel Delivery Domain: Example 2 agent 1 agent 2 5 4 2 1 3

46. Nash Equilibrium • The Zeuthen strategy is in Nash equilibrium under the assumption that one agent is using the strategy the other can do no better than use it himself. • Generally Nash equilibrium is not applicable in negotiation setting because it requires both sides utility function. • It is of particular interest to the designer of automated agents. It does away with any need for secrecy on the part of the programmer. • An agent’s strategy can be publicly known, and no other agent designer can exploit the information by choosing a different strategy. In fact, it is desirable that the strategy be known, to avoid inadvertent conflicts.

47. Task Oriented Domain • Non-conflicting jobs • Negotiation : Redistribute tasks to everyone’s mutual benefit • Example - Postmen domain

48. State Oriented Domain • Goals are acceptable final states (superset of TOD) • Have side effects - agent doing one action might hinder or help another agent. Example, on(white,gray) has side effect of clear(black). • Negotiation : develop joint plans and schedules for the agents, to help and not hinder other agents • Example – Slotted blocks world -blocks cannot go anywhere on table – only in slots (restricted resource)

49. Joint plan is used to mean “what they both do” not “what they do together” – just the joining of plans. There is no joint goal! • The actions taken by agent k in the joint plan are called k’s role and is written as Jk • C(J)k is the cost of k’s role in joint plan J. • In TOD, you cannot do another’s task or get in their way. • In TOD, coordinated plans are never worse, as you can just do your original task. • With SOD, you may get in each other’s way • Don’t accept partially completed plans.

50. Assumptions of SOD • Agents will maximize expected utility (will prefer 51% chance of getting $100 than a sure $50) • Agent cannot commit himself (as part of current negotiation) to behavior in future negotiation. • Interagent comparison of utility: common utility units • Symmetric abilities (all can perform tasks, and cost is same regardless of agent performing) • Binding commitments • No explicit utility transfer (no “money” that can be used to compensate one agent for a disadvantageous agreement)