distributed rational decision making n.
Skip this Video
Download Presentation
Distributed Rational Decision Making

Loading in 2 Seconds...

play fullscreen
1 / 56

Distributed Rational Decision Making - PowerPoint PPT Presentation

  • Uploaded on

Distributed Rational Decision Making. Author : Tuomas W. Sandholm Speakers : Praveen Guddeti (1---5) Tibor Moldovan (6---9) CSE 976, April 15, 2002. Outline. Introduction Evaluation criteria Non-cooperative interaction protocols Voting Auctions Bargaining

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Distributed Rational Decision Making' - enid

Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
distributed rational decision making

Distributed Rational Decision Making

Author: Tuomas W. Sandholm

Speakers: Praveen Guddeti (1---5)

Tibor Moldovan (6---9)

CSE 976, April 15, 2002

  • Introduction
  • Evaluation criteria
  • Non-cooperative interaction protocols
    • Voting
    • Auctions
    • Bargaining
    • General equilibrium market mechanisms
    • Contract nets
    • Coalition formation
  • Conclusions
  • Automated negotiation systems with self-interested agents are becoming increasing important.

1. Technology push.

2. Application pull.

  • Paper deals with protocols designed using a non-cooperative, strategic perspective.
evaluation criteria
Evaluation Criteria
  • Social welfare
  • Pareto efficiency
  • Individual rationality
  • Stability
  • Computational efficiency
  • Distribution and communication efficiency
evaluation criteria 1 social welfare
Evaluation Criteria1. Social Welfare
  • It is the sum of all agents’ payoffs or utilities in a given solution.
  • Requires inter-agent utility comparisons.
evaluation criteria 2 pareto efficiency
Evaluation Criteria2. Pareto Efficiency
  • A solution x is Pareto efficient if there is no other solution x’ such that
    • at least one agent is better off in x’ than in x, and
    • no agent is worse off in x’ than in x.
  • Does not require inter-agent utility comparisons.
  • Social welfare maximizing solutions are a subset of Pareto efficient ones.
evaluation criteria 3 individual rationality
Evaluation Criteria3. Individual Rationality
  • Participation in a negotiation is individually rational to an agent only if it is profitable.
  • A mechanism is individually rational if participation is individually rational for all agents.
  • Only individually rational mechanisms are viable.
evaluation criteria 4 stability
Evaluation Criteria4. Stability
  • The protocol mechanisms should motivate each agent to behave in the desired manner.
  • Protocol mechanisms may have dominant strategies. This means that an agent is best off by using a specific strategy no matter what strategies the other agents use.
  • Nash equilibrium: Each agent chooses a strategy that is a best response to the other agents’ strategies.
nash equilibrium formal definition
Nash Equilibrium:Formal definition
  • The strategy profile S*A={ S*1, S*2,…, S*|A|} among agents A is in Nash equilibrium if for each agent i , S*i is the agent’s best strategy given that the other agents choose strategies {S*1, S*2,…, S*i-1, S*i+1,… S*|A|}.
  • In some games no Nash equilibrium exists.
  • Some games have multiple Nash equilibrium.
nash equilibrium comments
Nash Equilibrium:Comments
  • Even if Nash equilibrium exists and is unique, there are limitations regarding what the Nash equilibrium guarantees.
  • In sequential games it only guarantees stability in the beginning of the game.
  • Subgame perfect Nash equilibrium.
  • Nash equilibrium is often too weak because subgroups of agents can deviate in a coordinated manner.
  • Sometimes efficiency and stability goals conflict.
evaluation criteria 5 computational efficiency
Evaluation Criteria5. Computational Efficiency
  • The protocol mechanisms when used by agents should need as little computation as possible.
  • Trade off between:
    • the cost of the computation needed for the protocol mechanisms and
    • the solution quality.
evaluation criteria 6 distribution and communication efficiency
Evaluation Criteria6. Distribution and Communication Efficiency
  • Distributed protocols should be preferred in order to avoid a single point of failure and a performance bottleneck – among other reasons.
  • Minimize the amount of communication required to get to a desired global solution.
  • These two goals can conflict.
non cooperative interaction protocols
Non-cooperative interaction protocols
  • Voting
  • Auctions
  • Bargaining
  • General equilibrium market mechanisms
  • Contract nets
  • Coalition formation
non cooperative interaction protocols 1 voting
Non-cooperative Interaction Protocols1. Voting
  • All agents give input to a mechanism.
  • Outcome chosen by the mechanism is solution for all agents.
  • Outcome is enforced.
  • Voters.
    • Truthful voters.
    • Strategic (Insincere) voters.
voters truthful voters 1
VotersTruthful Voters (1)
  • Each agent i  A has an asymmetric and transitive strict preference relations  i on O.
  • Social choice rule.
    • input the agents’ preference relations ( 1,…, |A|).
    • output the social preferences denoted by a relation *.
truthful voters 2 properties of a social choice rule
Truthful Voters (2)Properties of a social choice rule
  • * should exist for all possible inputs.
  • * should be defined for every pair o, o’ O.
  • * should be asymmetric and transitive over O.
  • The outcome should be Pareto efficient.
  • The scheme should be independent of irrelevant alternatives.
  • No agent should be a dictator.
truthful voters 3
Truthful Voters (3)
  • Arrow’s Impossibility Theorem: No social rule satisfies all of these six conditions.
  • Relax the first property.
  • Relax the third property.
    • Plurality protocol.
    • Binary protocol.
    • Borda protocol.
truthful voters 4 plurality protocol
Truthful Voters (4)Plurality Protocol
  • Majority voting protocol.
  • All alternatives are compared simultaneously.
  • The one having the highest number of votes wins.
  • Irrelevant alternative can split the majority.
truthful voters 5 binary protocol 1
Truthful Voters (5)Binary Protocol (1)
  • Pair wise voting with the winner staying to challenge remaining alternatives.
  • Irrelevant alternatives can change outcomes.
  • Agenda i.e. order of the pairings can change the outcomes.
truthful voters 6 binary protocol 2
Truthful Voters (6)Binary Protocol (2)

35 % of agents have preferences cdba

33 % of agents have preferences acdb

32 % of agents have preferences bacd

truthful voters 7 borda protocol 1
Truthful Voters (7)Borda Protocol (1)
  • Assign an alternative |O| points whenever it is the highest in some agent’s preference list, |O| -1 when it is second and so on.
  • Sum the counts of all alternatives.
  • Alternative with highest count is the winner.
  • Irrelevant alternatives lead to paradoxical results.
borda protocol 2
Borda Protocol (2)

Agent Preferences

1 a  b  c  d

2 b  c  d  a

3 c  d  a  b

4 a  b  c  d

5 b  c  d  a

6 c  d  a  b

7 a  b  c  d

Borda count c wins with 20, b has 19, a has 18, d loses with 13

Borda count a wins with 15, b has 14,loses with 13

with d removed

voters strategic insincere voters 1
VotersStrategic (Insincere) Voters (1)
  • Revelation principle: Suppose some protocol implements social choice function f(.) in Nash (or dominant strategy) equilibrium, then f(.) is implementable in Nash (or dominant strategy) equilibrium via a single-step protocol where the agents reveal their types truthfully.
voters strategic insincere voters 2
VotersStrategic (Insincere) Voters (2)
  • Gibbard-Satterthwaite impossibility theorem: Let each agent’s type i, consist of a preference order i on O. Let there be no restrictions on i, i.e. each agent may rank the outcomes O in any order. Let |O|  3. Now, if the social choice function f(.) is truthfully implementable in a dominant strategy equilibrium, then f(.) is dictatorial, i.e. there is some agent i who gets (one of) his most preferred outcomes chosen no matter what types the others reveal.
voters strategic insincere voters 3
VotersStrategic (Insincere) Voters (3)
  • Circumventing the GSIT:
    • Restricted preferences.
    • Groves-Clarke Tax Mechanism.
  • Groves-Clarke Tax Mechanism:
    • o = (g ,1,… |A|).
    • i is the amount agent i receives.
    • g encodes the other features of the outcome.
strategic insincere voters 4 groves clarke tax mechanism 1
Strategic (Insincere) Voters (4)Groves-Clarke Tax Mechanism(1)
  • Quasilinear preferences: ui(o) = vi(g) + i.
  • Net benefit: vi(g) = vi gross(g) – P / |A|.
  • Every agent iA reveals his valuation vi(g) for every possible g.
  • The social choice is g* =arg maxg i vi(g).
  • Every agent is levied a tax: tax i =  ji vj (g*) -  ji vj (arg maxg  ki vk (g)).
strategic insincere voters 5 groves clarke tax mechanism 2
Strategic (Insincere) Voters (5)Groves-Clarke Tax Mechanism(2)
  • Size of an agent’s tax is exactly how much his vote lowers the other’s utility.
  • Quasilinearity:
    • No agent should care how others divide payoffs among themselves.
    • An agent’s valuation vi gross(g) should not depend on the amount of money that the agent will have.
voters strategic insincere voters 6
VotersStrategic (Insincere) Voters (6)
  • If each agent has quasilinear preferences, then each agent’s dominant strategy is to reveal his true preferences.
  • Agents need not waste effort in counter speculating each others’ preference declarations.
  • Participation is individually rational.
voters strategic insincere voters 7
VotersStrategic (Insincere) Voters (7)
  • Problems of Groves-Clarke Tax Mechanism:
    • Does not maintain budget balance.
    • Not coalition proof.
    • Intractable.
  • Other ways to circumvent the GSIT:
    • Choosing a dictator randomly.
    • Make the computation of an untruthful revelation prohibitively costly.
non cooperative interaction protocols 2 auctions
Non-cooperative Interaction Protocols2.Auctions
  • Unlike voting where the outcome binds all agents, in auctions the outcome is usually a deal between two agents.
  • In voting the protocol designer wants to enhance the social good, while in auctions, the auctioneer wants to maximize his own profit.
  • Classical setting.
  • Contracting setting.
auctions 2
Auctions (2)
  • Auction settings.
  • Auctions protocols.
  • Efficiency of the resulting allocation.
  • Revenue equivalence and non-equivalence.
  • Bidder collusion.
  • Lying auctioneer.
  • Bidders lying in non-private-value auctions.
  • Undesirable private information revelation.
  • Roles of computation in auctions.
1 auction settings
1. Auction settings
  • Three qualitatively different auctions depending on how an agent’s value of the item is formed:
    • Private value.
    • Common value.
    • Correlated value.
2 auctions protocols
2. Auctions protocols
  • English (first-price open-cry) auction.
  • First-price sealed-bid auction.
  • Dutch (descending) auction.
  • Vickrey (second-price sealed-bid) auction.
    • Allocation of computation resources in OS, allocation of bandwidth in computer networks, computationally control building heating.
    • Has not been widely adopted in auctions among humans.
3 efficiency of the resulting allocation
3. Efficiency of the resulting allocation.
  • In isolated private value or common value auctions, each one of the four auction protocols allocates the auctioned item Pareto efficiently to the bidder who values it the most.
  • All four protocols are Pareto efficient in the allocation.
  • The dominant strategies (Vickrey and English) are more efficient.
4 revenue equivalence and non equivalence
4. Revenue equivalence and non-equivalence.
  • Revenue equivalence: All of the four auction protocols produce the same expected revenue to the auctioneer in private value auctions where the values are independently distributed and bidders are risk-neutral.
  • Among risk averse bidders, the Dutch and the first-price sealed-bid protocols give higher expected revenue to the auctioneer.
  • A risk averse auctioneer achieves higher expected utility via the Vickrey or English protocols.
revenue equivalence and non equivalence 2
Revenue equivalence and non-equivalence. (2)
  • In non-private value auctions, both the English and Vickrey protocols produce greater expected revenue to the auctioneer than the first-price sealed-bid auction or Dutch auction.
  • In non-private value auctions with at least three bidders, the English auction leads to higher revenue than the Vickrey auction.
5 bidder collusion
5. Bidder collusion.
  • The English auction and the Vickrey auction actually self-enforce some of the most likely collusion agreements.
  • First-price sealed-bid and the Dutch auctions are preferred for deterring collusion.
  • For collusion to take place in Vickrey, first-price sealed-bid or Dutch auctions the bidders have to identify each other before placing the bids.
6 lying auctioneer
6. Lying Auctioneer.
  • In Vickrey auction the auctioneer may lie about the value of the second highest bidder.
  • In the English auction the auctioneer can use shills that bid in the auction in order to make the real bidders increase their valuations of the item.
  • The auctioneer may bid himself to guarantee that the item will not be sold below a certain price.
7 bidders lying in non private value auctions
7. Bidders lying in non-private-value auctions.
  • Winner’s curse: If an agent bids its valuations and wins the auction, it will know that its valuation was too high because the other agents bid less.
  • Agents should bid less than their valuations.
  • This is the best strategy in Vickrey auctions.
  • Vickrey fails to induce truthful bidding in most auction settings.
8 undesirable private information revelation
8. Undesirable private information revelation.
  • In Vickrey auctions the agents often bid truthfully. This leads to the bidders revealing their true valuations.
  • This information is sensitive and the bidders would prefer not to reveal it.
  • Another reason why the Vickrey auction protocol is not widely used among humans.
9 roles of computation in auctions
9. Roles of computation in auctions.
  • Two issues arise from computation in auctions:
  • Computationally complex look ahead that arises when auctioning interrelated items one at a time.
  • Implications of costly local marginal cost (valuation) computation or information gathering in a single-shot auction.
interrelated auctions
Interrelated auctions.
  • Look ahead:
    • Without look ahead the allocation may be inefficient.
    • With look ahead the agents will not bid their true per-item cost.
    • Computation cost may be prohibitively great.
  • Allow agents to backtrack from commitments by paying penalties.
single shot auctions
Single-shot auctions
  • Incentive to counter speculate: In a single-shot private value Vickrey auction with uncertainty about an agent’s own valuations, a risk neutral agent’s best action can depend on the other agents. It follows that is is worth counter speculating.
non cooperative interaction protocols 3 bargaining
Non-cooperative Interaction Protocols3. Bargaining
  • Real world settings usually consist of a finite number of competing agents, so neither monopoly,nor monopsony nor perfect competition assumptions strictly apply.
  • Bargaining theory fits in this gap.
  • Bargaining theory:
        • Axiomatic
        • Strategic
bargaining theory axiomatic
Bargaining Theory Axiomatic
  • Does not use the idea of equilibrium.
  • Desirable properties for a solution, called axioms of the bargaining solution, are postulated.
  • Then the solution that satisfies these axioms are sought.
  • Nash bargaining solution.
axiomatic bargaining theory nash bargaining solution 1
Axiomatic Bargaining Theory Nash Bargaining Solution (1)
  • Nash analyzed a 2-agent setting where the agents have to decide on an outcome o  O, and the fallback outcome ofallback occurs if no agreement is reached.
  • There is a utility function

ui: O  R for each agent i  [1,2].

  • It is assumed that the set of feasible utility vectors { (u1 (o), u2 (o)) | o  O} is convex.
axiomatic bargaining theory nash bargaining solution 2
Axiomatic Bargaining TheoryNash Bargaining Solution (2)
  • Axioms for the Nash bargaining solution u* = (u1(o*), u2(o*)) are:

1. Invariance.

2. Anonymity (symmetry).

3. Independence of irrelevant alternatives.

4. Pareto efficiency.

axiomatic bargaining theory nash bargaining solution 3
Axiomatic Bargaining TheoryNash Bargaining Solution (3)
  • The unique solution that satisfies these four axioms is:

o* = arg maxo[u1(o)–u1(ofallback)][u2(o)–u2 (ofallback)]

  • Other bargaining solutions also exist.
bargaining theory strategic 1
Bargaining TheoryStrategic (1)
  • Bargaining situation is modeled as a game.
  • Solution is based on an analysis of which of the players’ strategies are in equilibrium.
  • Solution is not unique.
  • Explains the behavior of rational utility maximizing agents better than axiomatic approaches.
  • Usually analyses sequential bargaining.
bargaining theory strategic 2
Bargaining TheoryStrategic (2)
  • Finite number of offers with no time discount.
  • Finite number of offers with time discount.
  • Infinite number of offers with no time discount.
  • Infinite number of offers with time discount.
strategic bargaining theory rubinstein bargaining solution
Strategic Bargaining TheoryRubinstein Bargaining Solution
  • In a discounted infinite round setting, the subgame perfect Nash equilibrium outcome is unique. Agent 1 gets (1- 2) / (1- 12), where 1 is 1’s discount factor, and 2 is 2’s. Agent 2 gets one minus this. Agreement is reached in the first round.
  • The proof gives a way to solve for subgame perfect Nash equilibrium payoffs.
strategic bargaining theory fixed bargaining cost per negotiation round
Strategic Bargaining TheoryFixed Bargaining Cost per negotiation round
  • If the agents have symmetric bargaining costs, the solution concept is powerless.
  • If 1’s bargaining cost c1 is even slightly smaller than 2’s cost c2, then 1 gets the entire dollar.
  • If 1’s bargaining cost is greater than 2’s, then 1 receives a payoff that equals the second agent’s bargaining cost, and agent 2 receives the rest.
strategic bargaining theory recent extensions kraus et al
Strategic Bargaining TheoryRecent extensions [Kraus et al.]
  • Sequential bargaining with outside options.
  • Sequential bargaining where one agent gains and one loses over time.
  • Negotiation over time when agents do not know each other’ types.
bargaining computation 1
BargainingComputation (1)
  • Assume perfect rationality.
  • The space of deals is assumed to be fully comprehended by the agents.
  • The value of each potential contract known.
  • Focus of future work:
    • make the cost of search explicit and.
    • consider its trade-off with bargaining gains.
bargaining computation 2
BargainingComputation (2)
  • There are two searches occurring in bargaining:
    • Intra-agent deliberative search: an agent locally generates alternatives, evaluates them, counter speculates, does look ahead etc.
    • Inter-agent committal search: the agents make agreements with each other regarding the solution.
  • Evaluation criteria
  • Non-cooperative interaction protocols
    • Voting
    • Auctions
    • Bargaining
    • General equilibrium market mechanisms
    • Contract nets
    • Coalition formation
  • Conclusions