Agent Technology for e-Commerce

Agent Technology for e-Commerce Chapter 9: Negotiation II Maria Fasli http://cswww.essex.ac.uk/staff/mfasli/ATe-Commerce.htm

Agreement zone £ Seller’s surplus Buyer’s surplus Seller’s valuation: wants to receive ps or more Buyer’s valuation: wants to pay pb or less Agreement price p* ps pb Buyer wants to decrease p* Seller wants to increase p* Bargaining A bargaining situation: two or more agents have a common interest and could reach a mutually beneficial agreement, but have a conflict of interest about which one to reach

Bargaining power The bargaining power of the participating agents in a bargaining situation is determined by a number of factors • Impatience • Risk of breakdown • Outside options • Inside options • Commitment tactics • Asymmetric information

Axiomatic bargaining • Axiomatic bargaining theory assumes no equilibrium • Axiomatic models of bargaining yield solutions that satisfy a set of desired properties – the axioms of the bargaining solution Example • Two agents A and B need to divide a cake of size  • The set of possible agreements that they can reach is: ={(oA,oB):0oA  and oB= -oA • The agents’ utilities are: UA (oA)= uA and UB (oB)= uA • If the agents fail to reach a deal, then a default solution is implemented and they gain utility (dA, dB)

Nash Bargaining Solution • The Nash Bargaining Solution (NBS) of this bargaining situation is the allocation of utilities (uA, uB) which solves: o*=max(uA- dA) (uB- dB) subject to (uA dA) and (uBdB) The NBS is the only bargaining solution that satisfies the following: • Pareto efficiency • Symmetry • Invariance • Independence of irrelevant alternatives

Returning to the example: uA=oA and uB= oB = (1- oA ) • The NBS is the sharing rule that maximizes the Nash product: (oA- dA) (oB- dB) • The NBS is at: uA=[+dA-dB]/2 and uB=[+dB-dA]/2 uA=dA +[-dA-dB]/2 and uB=dB +[-dB-dA]/2 • As a result the two agents split the difference: the agents first agree to take a part of the cake equal to their di and then they split the remaining cake equally between themselves

Strategic bargaining • In strategic models of bargaining, the bargaining solution emerges as the equilibrium of a sequential game in which the parties take turns in making offers and counteroffers • Two agents A and B bargain about the partition of a cake  • Offers are made at discrete points in time • An offer is a number 0 and   • At each moment in time each agent makes an offer to the other; if the other accepts, the game ends, otherwise the game continues with the other agent now making an offer

The bargaining process is not frictionless: agents are impatient and they would rather agree on the same deal today rather than tomorrow. This is expressed as a discount factor =exp(-ri) • If the agents reach a deal at time point t then agent i’s payoff is oiexp(-rit) • The bargaining situation can be depicted as a sequential game with subgames in extensive form

Subgame 1 A offer oA B B Subgame 2 reject offer oB accept A A Subgame 3 reject [oA , (1-oA)] offer oA accept B B reject [A(1- oB), BoB] accept … [AA oA, B B(1-oA)]

The basic alternating offers game has a subgame perfect Nash equilibrium: • Agent A gets (1-B)/(1- A B) • Agent B gets 1 minus (1-B)/(1- A B) The unique subgame perfect Nash equilibrium satisfies two properties • No delay: whenever an agent has to make an offer, the equilibrium offer is accepted by the other agent • Stationarity: in equilibrium, a player makes the same offer whenever it has to make an offer

The following strategies define the unique subgame perfect equilibrium Player A always offers and always accepts an offer Player B always offers and always accepts an offer

The Strategic Negotiation Protocol Based on Rubinstein’s protocol of alternating offers • N agents A={a1,….,an} need to agree on a given issue • They can take actions at certain times T={0,1,..} • In each period tT of the negotiation if an agreement hasn’t been reached, the agent whose turn is to make an offer at time t will suggest a possible solution • Each of the other agents responds by accepting (Yes), refusing (No), or opting out of the negotiation (Opt)

If all the agents choose Yes then the negotiation ends and the solution/offer is implemented • If at least one of the agents opts out, then the negotiation ends and a default solution is implemented • If no agent has opted out, but at least one has refused the offer, the negotiation proceeds to cycle t+1 and the next agent makes a counteroffer • An agent that responds to an offer is not aware of the other agents’ responses in the current negotiation period

Assumptions: • Rationality • Agents avoid opting out • Agreements are honoured • No long-term commitments • Common knowledge. Assumptions 1-4 are common knowledge

Utility functions • An agent has a utility function over all possible outcomes o • The time and resources spent on the negotiation process affect this utility Types of utility functions: • Fixed losses/gains per time unit: ui(o,t)=ui(o,0)+tci • Time constant discount rate: ui (o,t)= it ·ui(o,0) where 0<it<1. Every agent i has a fixed discount rate it

Models with a financial system with an interest rate r: • Finite-horizon models with fixed losses per time unit: ui(o,t) = ui(o,0)(1-t/k)-tc for tk (applicable when it is known in advance that the outcome is valid for k periods)

Applications of the SNP The SNP is useful in situations where: • Agents do not agree on any entity-oracle who may provide a centralized solution • The system is dynamic and therefore a predefined solution cannot be imposed • A centralized solution may cause a performance bottleneck • There is incomplete information and no entity-oracle has all the relevant information Applications: data and ask allocation, negotiation over pollution issues, hostage negotiation

Negotiation in different domains Two broad categories: • Task-oriented domains • Worth-oriented domains

Negotiation in task-oriented domains • Task-oriented domains (TOD): an agent’s activity can be defined in terms of a set of tasks, where a task is a nondivisible job • Example • A has to post letters and return a few books to the library • B has to post a package and visit the library to borrow this month’s National Geographic • Both agents could benefit if they could reach an agreement

Task-oriented domains A task-oriented domain can be formalized as a tuple T,A,c: • T is a finite set of tasks • A is the set of agents and any agent is capable of carrying out any combination of tasks • c is the cost function which takes as parameters the set of tasks; c(T’) is independent of which agent carries the tasks in list T’

An encounter within a TOD is an ordered list of tasks T1,…,Tn such that Ti is the list of tasks allocated to agent ai • A deal = D1,D2 is an allocation of tasks T1T2 • The cost of a deal to agent ai will be denoted costi() and the agent’s utility is: ui()=c(Ti)- costi() • If the agents fail to agree on a deal, a default conflict deal  is implemented and ui()=0 • A Pareto efficient allocation or deal cannot be improved upon by any of the agents without making any other agent worse off

Monotonic concession protocol The negotiation proceeds in rounds: • In round 1, both agents propose a deal from the negotiation set simultaneously • An agreement is reached and the protocol terminates when one of the agents finds that the deal proposed by the other is at least as good or better than its own proposal • If no agreement is reached, the negotiation proceeds to the next round

In round t+1, both agents make proposals: • A new proposal can be the previously made proposal by the agent (the agent stands still), or • A new proposal which gives the other agent more utility than the proposal made in round t (the agent concedes) • If none of the agents make a concession, the protocol terminates with the conflict deal

A’s best deal B’s best deal Conflict deal Maximal loss from concession Maximal loss from conflict deal

The Zeuthian strategy Three aspects: • What should an agent’s first proposal be? The best deal for that agent • Who should concede on any given round? The agent that has more to loose if the conflict deal is imposed • If an agent concedes, how much should it concede? As much as it is required so that the balance of risk is changed between the agent and its opponent

Measuring the degree of willingness to risk • Suppose A has conceded a lot already, then the deal is very close to the conflict deal and A does not have much to loose • The extent to which an agent is more willing to risk conflict is: • As dwriski,t increases, the agent has less to lose if a conflict occurs and as a result will not be willing to concede • The agent with the lowest dwriski,t should concede

Features of the Zeuthian strategy • The agents will not run into conflict, i.e. the outcome reached is going to be Pareto efficient • Not in Nash equilibrium, a self-interested agent knowing that the opponent is using the Zeuthian strategy can try and exploit this • Extended Zeuthian strategy: who concedes in case both agents have the same dwriski,t is decided on the flip of a fair coin • This is now a game where the players play with mixed strategies, so there is at least on mixed strategies Nash equilibrium • But there is some positive probability that the conflict deal will be reached. So although the extended Zeuthian strategy is stable, it may yield an inefficient outcome • Not computational and communication efficient

Deception in TODs • Agents have to declare their tasks, and may do so insincerely • An agent can declare phantom or decoy tasks in an attempt to influence the outcome of the negotiation process. • If an agent can produce a phantom task on demand then this is called a decoy • Phantom tasks that cannot be easily produced make deception detection easier • An agent can also hide tasks

Worth Oriented Domains • Agents are interested in bringing about states that have the greatest value • Agents’ goals can be achieved through joint plans

Worth-oriented domains can be formalized as a tuple S, A, J, c: • S is the set of all possible states • A is the set of agents • J is the set of all possible joint plans • c is the cost function which represents the cost of a joint plan to an agent ai • j:s1|→s2 denotes that the execution of plan j is s1 leads to s2 • If the agent were alone in the world, then its utility from bringing the world to its own ‘stand-alone optimal’ using its own plan is:

It may be impossible for each of the agents to perform single-agent plans to bring the world to a desirable state • Agents in WODs can reach a compromise by negotiating not only over what parts of their goals will be achieved, but also over the means • State-oriented domains: the worth value is associated only with the achievement of an agent’s full goal

Coalitions • A coalition is a set of agents that agree to cooperate in order to achieve a common objective • The incentives for creating/joining a coalition can be: • Monetary: reduction of cost or increased profit • Risk reduction (or allowing someone else to assume risk) • Increase in market size or share

Coalition formation Coalition formation can be studied in the context of characteristic function games (CFG): • A set N of agents in which each subset is called a coalition • The value of a coalition S is given by a characteristic function vS • CS: the coalition structure is the set of all coalitions such that every agent belongs to one • The solution of a game with side payments is a coalition configuration which consists of a partition S of N, the coalition structure CS, and an n-dimensional payoff vector

Coalition formation in CFG games involves two activities: • Coalition structure generation • Division of the value of the generated coalition structure among all agents The two activities are intertwined

Coalition structure generation The formation of an optimal, maximum welfare coalition structure is trivial when the coalition values are: • Super-additive: there is at least one optimal coalition structure, the grand coalition • Sub-additive: the optimal coalition structure is the one in which every agent acts on their own When games are neither sub-additive or super-additive some coalitions are best off merging whereas others are not

The objective is to maximize the social welfare of the agents by finding an optimal coalition structure CS*: where V(CS) is the value of a coalition structure:

The number of coalition structures CS is exponential in the number of coalitions S, the agent must search among O(nn) coalition structures to find the optimal one • The number of coalitions is • Not all coalition structures can be enumerated unless n is small • Can the agents approximate the optimal coalition structure? • Can they search through a subset LM such that:

Coalition structures for four agents • The lowest two levels of the ordering (j=1 and j=2) the agents have seen all the possible coalition structures • The agents must at least inspect 2n-1 different coalition structures in order to determine a worse-case bound • If more time for computation is available more coalition structures can be inspected

Division of payoffs Payoff division is important as it affects the stability of the coalition Many coalition formation algorithms rely on game theory concepts The Core • The strongest solution concept; it may be empty • Agents may switch indefinitely between coalitions • The Core may contain multiple solutions – the agents need to agree on one: the nucleolus • Calculating the Core is an NP-hard problem

The Shapley value: • Agent i is a dummy if vSi-vS=vi for every coalition S that does not include i • Agents i and j are interchangeable if for all S with either i or j, vS remains the same if i is replaced by j We require a set of payoffs that satisfy: • Symmetry: if i and j are interchangeable then pi=pj • Dummies: if i is a dummy, then pi=v{i} • Additivity: for two games v and w, pi in v+w is equal to pi in v plus pi in w

The Shapley value satisfies these conditions and sets the payoffs to • It always exists and is unique • Pareto efficient • It guarantees that individual agents and the grand coalition have an incentive to stay with the coalition structure • No guarantee that all subgroups of agents are better off in the coalition structure than by splitting out into a coalition of their own

Customer coalitions • Suppose you want to buy a PC, you can do so at retail price • If nine of your friends are interested in the same type of PC, you can join forces and ask retailers to make you a better offer as this is a bulk purchase • What the discount is depends on the number of PCs • The vendor has an incentive to lower the price, as otherwise the sale will be lost

Supplier incentive to sell wholesale Utility to sell wholesale: The utility of selling n items retail: The utility of selling n items wholesale: Up to some number nretail, the supplier does not have an incentive to sell wholesale as marketing costs are identical

Customer incentive to buy wholesale A customer’s utility: ucustomer = vitem – pitem – cstorage • Maximum utility range: MUR(nmin,nmax) – utility is high while the management or storage costs remain low • If nwholesaleMUR then the customer can purchase the items at wholesale price • But the customer needs to be given incentives to buy larger quantities, i.e, the supplier needs to lower the price

In practice, individual consumers very rarely require large enough quantities so that they can purchase at wholesale prices • But by forming coalitions, consumers can increase the quantity purchased so as to be charged wholesale prices • The utility of the coalition is now MURcoalition = MURi • If nwholesaleMURcoalition then the coalition can make a wholesale purchase

Coalition protocols The general stages involved in a coalition protocol are: • Negotiation: The coalition leader/representative negotiates with suppliers • Coalition formation: The initiator/leader invites potential members to join the coalition; possible admission constraints • Leader election/voting: The members may elect a leader. Not all protocols have this stage • Payment collection: The coalition leader/representative collects payments and pays supplier. • Execution/distribution: The transaction is executed; the goods arrive and they are distributed to the members of the coalition

Issues in coalition protocols • Coalition stability • Distribution of utility and costs • Trust • Negotiation stage • Payment collection stage • Distribution stage • Distribution of risk • Risk of transaction failure • Risk of coalition failure • Price uncertainty

Coalition protocols • Assume a coalition leader (L), a set of suppliers S={s1,s2,…,sk} and a set of potential coalition members M={m1,m2,…,mn} • Based on the order in which the negotiation and coalition formation stages take place there are two types: • Post-Negotiation • Pre-Negotiation

Post-negotiation protocol • LCS: L advertisesthe creation of a coalition with certain parameters (deadline, maximum number etc.) • Each miM considers whether to join the coalition and sends necessary message mi L: “Join the Coalition” • At the expiration of the coalition deadline/size limit, the leader enters thenegotiation with the suppliers si S using its private protocol/strategyand decides on a deal • L collects money from group members, and arranges forthe shipping and distribution of goods

Issues • Trust in the coalition leader is required • Shills can startcoalitions Trust can be established in a number of ways • Leaderscan be elected • A trusted third party can be appointed to conduct the negotiations • The coalition leader could be compelled to open everystep of the negotiation to the scrutiny of group members • Members can vote on the suppliers’ bids – but time-consuming

Agent Technology for e-Commerce