Authors: S. Fatima, S. Kraus, & M. Wooldridge

1. NEW BOOK Authors: S. Fatima, S. Kraus, & M. Wooldridge Date of publication:Nov 2014 Publisher: Cambridge University Press ISBN: 9781107002548 Book webpage: http://tinyurl.com/PanBook

2. Negotiation Negotiation is the process of interaction through which parties with conflicting goals and preferences try to reach a mutually acceptable agreement • Conflicts typically arise when • some scarce resources must be allocated between two or more parties • each party is self-interested • all parties can gain from reaching a mutually acceptable agreement • The conflicting parties themselves must resolve the conflicts Chapter 1

3. Automated negotiation Automated negotiation is negotiation conducted by software agents on behalf of humans Key issues in the design of negotiating agents • computational feasibility • agreement desirability Chapter 1

4. Parameters of negotiation • A set of negotiators • One-to-one • One-to-many • Many-to-many • A negotiation set/ domain • A negotiation protocol • A set of players’ strategies Chapter 1

5. The structure of negotiation • Existence of a social conflict • Negotiation participants • Gathering and analyzing opponent information • Protocol selection • Strategy selection • Learning • Dynamic strategy selection • Impasse resolution • Renegotiation Chapter 1

6. Desiderata for automated negotiation • Economic efficiency • Individual rationality • Simplicity • Distribution • Fairness • Timeliness • Uniqueness of agreement Chapter 1

7. Advantages and disadvantages of automated negotiation • Advantages • Better quality outcomes • Reduced costs • Elimination of cultural issues • Disadvantages • Lack of trust in automated negotiation • Legal and contractual considerations • The preference elicitation bottleneck • Ethical considerations • Putting people out of work Chapter 1

8. Multiagent systems and Game theory • Multiagent systems: A study of strategic interactions between self-interested computational entities • Game theory: A study of strategic decision making in systems comprised of self-interested entities • Cooperative game theory • Noncooperative game theory Chapter 2

9. One shot bargaining • Two players must agree on how to split a unit sized pie • Rules of the game: • Each player proposes a fraction of the pie he wants • If the sum of the offers does not exceed the size of the pie then the pie is split as per the offers • Otherwise neither player gets any share and the game ends What is the Nash equilibrium? Chapter 2

10. “All female employees earning over $50,000 a year.” “All female employees with more than three children.” 2 1 Task oriented domains (TODs) Agents who have tasks to carry out may benefit from reorganizing the distribution of tasks among themselves Example: Database domain Chapter 4

11. 1 2 Postmen Domain Post Office TOD a / / c b / / f / e d

12. Task oriented domains (TODs) A TOD is a triple (P, Tsk, c) where • N = {1, 2} is the set of negotiation participants (may, in general contain more than 2 agents) • Tsk is the set of possible tasks • c: 2Tsk is a function that defines the cost of executing each subset of tasks • Encounter: A collection of tasks (T1, T2) where Ti the set of tasks initially allocated to agent i N • Deal: A pair (D1, D2) such that D 1 D2 = T 1 T2 where Di is the set of tasks allocated to agent i after negotiation Chapter 4

13. Task oriented domains contd. • Conflict: This means each agent must perform his originally allocated task Ti • The utility of a deal to agent i is the difference in the costs of executing Ti and Di Ui((D1, D2)) = Ui((T1, T2)) - Ui((D1, D2)) • Each agent negotiates so as to reach a deal that maximizes his individual utility Chapter 4

14. Task oriented domains contd. Negotiation set: The set of all individually rational and Pareto optimal deals Chapter 4

15. Limitations of TODs Two limiting assumptions: • It is assumed that there is no limit on the availability of resources • It is assumed that there are no side effects Chapter 4

16. Side effects: Examples Example 1: Blocks world On(White, Grey) --------------------->Clear(Black) Side effect Example 2 A couple who must use a shared car to go to their respective places of work Chapter 4

17. State oriented domains (SODs) Agents who want to transform the environment to a goal state must coordinate their actions into a coherent joint plan A SOD is a tuple (N, St, Pln, c) where • N = {1, 2} is the set of negotiation participants (may, in general contain more than 2 agents) • St is the set of possible environment states • Pln is the set of all possible joint plans (a joint plan moves the environment from one state to another) • c: Pln 2 is a function that defines the cost of executing each joint plan (c(x)i is the cost to agent i of the joint plan x) Chapter 4

18. State oriented domains contd. • An encounter is a tuple (st0, 1 , 2) where • st0 is the initial state • i is the set of goals states that satisfy agent i • A deal is a joint plan that yields a goal state that satisfies both agents • Each agent i has a utility function • Ui : PlnSt   • Each agent then negotiates so as to reach a deal that maximizes his individual utility Chapter 4

19. Partially achievable goals • In a SOD, the goals are binary • In some domains, it is possible for goals to be partially achieved • Worth oriented domains model interactions where quality/ cost tradeoffs can be made when achieving goals Chapter 4

20. Worth oriented domains (WODs) Each agent associates a worth with each possible environment state hole agents tile B A 2 2 5 Tileworld example 5 2 obstacle 4 3 2 Chapter 4

21. Worth oriented domains (WODs) • Encounter: A tuple (st0, W) where • st0 is the initial state • W: St  N  is a worth function • Deal: A joint plan that yields a goal state satisfying both agents • Each agent then negotiates so as to reach a deal that maximizes the worth of its resulting state Chapter 4

22. Task-Oriented Domains: 1 • A TOD is a tuple <T, A, c>, where T is the set of tasks, A is the set of agents, and c(X) is a monotonic function for the cost of executing the set of tasks X • Examples • Deliveries: c(X) = length of minimal path that visits X • Postmen: c(X) = length of minimal path plus return • Databases: c(X) = minimal number of needed DB ops Service-Oriented Computing: Semantics, Processes, Agents - Munindar Singh and Michael Huhns

23. Task-Oriented Domains: 2 • A deal is a redistribution of tasks • Utility of deal d for agent k isUk (d) = c(Tk) - c(dk) • The conflict deal, D, is no deal • A deal d is individual rational if d>D • Deal d dominates d’ if d is better for at least one agent and not worse for the rest • Deal d is Pareto Optimal if there is no other deal d’ such that d’ is better for one player and not worth to the other. • The set of all deals that are individual rational and Pareto optimal is the negotiation set, NS Service-Oriented Computing: Semantics, Processes, Agents - Munindar Singh and Michael Huhns

24. Monotonic Concession Protocol • Each agent proposes a deal • If one agent matches or exceeds what the other demands, the negotiation ends • Else, the agents propose the same or more (concede) • If no agent concedes, the negotiation ends with the conflict dealThis protocol is simple, symmetric, distributed, and guaranteed to end in a finite number of steps in any TOD. What strategy should an agent adopt?

25. Zeuthen Strategy Offer deal that is best among all deals in NS • Calculate risks of self and opponentR1=(utility A1 loses by accepting A2’s offer) (utility A1 loses by causing a conflict) • If risk is smaller than opponent, offer minimal sufficient concession (a sufficient concession makes opponent’s risk less than yours); else offer original deal • If both use this strategy, they will agree on deal that maximizes the product of their utilities (Pareto optimal) • The strategy is not stable (when both should concede on last step, but it’s sufficient for only one to concede, then one can benefit by dropping strategy) Service-Oriented Computing: Semantics, Processes, Agents - Munindar Singh and Michael Huhns

26. P.O. A1 (hidden) A1 A2 Deception-Free Protocols • Zeuthen strategy requires full knowledge of • Tasks • Protocol • Strategies • Commitments • Hidden tasks • Phantom tasks • Decoy tasks Service-Oriented Computing: Semantics, Processes, Agents - Munindar Singh and Michael Huhns

27. Mixed deal • Element of probability – Agents will perform (DA,DB) with probability p or (DA,DB) with probability 1-p • Costi([(DA,DB):p]) = pCost(Di) + (1-p)Cost(Dj) • Utilityi([d:p]) = Cost(Li) – Costi([d:p]) • All or nothing deal – 0<=p<=1 such that • mixed deal m = [({LA,LB}, f ):p] e NS • P(m) = maxdeNSp(d)

28. Incomplete Information • G and w – common knowledge • i knows Li, not Lj : j!=I • Solution • Exchange missing information • Penalty for lie • Possible lies • False information • Hiding letters • Phantom letters • Not carry out a commitment

29. Hidden letters • Utility of A • Expected(on telling the truth) = 4 • Pure deal – [(d,f):1/2] = 6 • Mixed deal - [(d,f):3/8] = 33/4

30. Phantom letters • Utility of A • Expected(on telling the truth) = 3 • Pure deal – [(d,f):1/2] = 4 • Mixed deal – possibility of being caught (all or nothing deal)

31. Alternating Offer Bargaining Game Two players are to divide a sum of money (S) is a finite number (N) of alternating offers. Player 1 (‘Buyer’) goes first; Player 2 (‘Seller’) can either accept or counter offer, and so on. The game continues until an offer is accepted or N is reached. If no offer is accepted, the players each get zero. A.Rubinstein, 1982

32. Alternating Offer Bargaining Game 1 (a,S-a) 2 (b,S-b) 1 (c,S-c) (0,0) S = $5.00 N = 3

33. Alternating Offer Bargaining Game 1 (a,S-a) 2 (b,S-b) 1 (4.99, 0.01) (0,0) S = $5.00 N = 3

34. Alternating Offer Bargaining Game 1 (4.99,0.01) 2 (b,S-b) 1 (4.99,0.01) (0,0) S = $5.00 N = 3 SPNE = (4.99,0.01) The game reduces to an Ultimatum Game

35. Shrinking Pie Game Now consider what happens if the sum to be divided decreases with each round of the game (e.g., transaction costs, risk aversion, impatience). Let S = Sum of money to be divided N = Number of rounds d = Discount parameter

36. Shrinking Pie Game 1 (3.74,1.26)2 (1.25, 1.25) 1 (1.24,0.01) (0,0) 1 S = $5.00 N = 3 d = 0.5

37. Shrinking Pie Game 1 (3.13,1.87)2 (0.64,1.86) 1 (0.63,0.62) 2 (0.01, 0.61) (0,0) 1 S = $5.00 N = 4 d = 0.5

38. Shrinking Pie Game for d = ½ N = 1 (4.99, 0.01) 2 (2.50, 2.50) 3 (3.74, 1.26) 4 (3.13, 1.87) 5 (3.43, 1.57) … … This series converges to (S/(1+d), S – S/(1+d)) = (3.33, 1.67) This pair {S/(1+ d),S-S/(1+ d)} are the payoffs of the unique SPNE. P2 5 1.67 0 2 4 5 3 1 0 3.33 5 P1

39. Shrinking Pie Game Optimal Offer (O*) expressed as a share of the total sum to be divided = [S-S/(1+d)]/S O* = d/(1+d) SPNE = {1- d /(1+ d), d /(1+ d)} Thus both d=1 and d=0 are special cases of Rubinstein’s model: When d=1 (no bargaining costs), O* = 1/2 When d=0, game collapses to the ultimatum version and O* = 0 (+e)

40. Alternating offers protocols • Game of sequential moves and discrete time periods • Two variants: • Infinite horizon • Finite horizon • Strategies: The set of strategies for player i N is Si= {Offer (x, 1-x) | 0 ≤ x ≤ 1}  {Accept, Reject} Chapter 5

41. Alternating offers: Infinite horizon • Assumptions: • Disagreement is the worst outcome • Each player seeks to maximize his utility t  1 t  t + 1 Does the receiving agent accept? The current proposer offers (x, 1 x) No What is the equilibrium? Yes The current proposal is implemented and the game ends Chapter 5

42. Equilibrium strategies • One round: Ultimatum game • The first mover gets the entire surplus and the other agent gets nothing • Two rounds: • The last mover gets the entire surplus and the other agent gets nothing • n > 2 rounds: • Same outcome as that for two rounds • Infinite horizon • Any pair (x, 1-x) is a Nash equilibrium agreement reached instantly Chapter 5

43. Impatient players and finite horizon • Time is valuable • Player i ’s utility from receiving a share of x at time t is ui (x, t) = xit-1 (0 < i≤ 1 is the i’s discount factor) • Deadline: Negotiation must end within n time periods. Chapter 5

45. What is the difference between Diplomacy and Chess? Sarit Kraus

46. Diplomacy’s Rules • Each player represents one of seven European powers: England, Germany, Russia, Turkey, Austria-Hungary, Italy and France.

47. Diplomacy’s Map

49. Diplomacy’s Rules (Cont.) • Winner: The power that gains control over the majority of the board. • Beginning: 1901; two seasons a year. • A season: consists of a negotiations stage and a move stage. • Moves: All players secretly write the orders for all of their units simultaneously. • Negotiations: Coalitions and agreements among the players reached in the negotiations stage significantly affect the course of the game. The rules of the game do not bind a player to anything she says. Deciding who to trust as situations arise is part of the game.

50. If you support my attack on Vienna I will support your attack on Rumania F I know that Italy is going to attack Trieste R G E Negotiations in Diplomacy If you will not help me I will attack you Don’t trust Germany