Distributed rational decision making
This presentation is the property of its rightful owner.
Sponsored Links
1 / 56

Distributed Rational Decision Making PowerPoint PPT Presentation


  • 57 Views
  • Uploaded on
  • Presentation posted in: General

Distributed Rational Decision Making. Author : Tuomas W. Sandholm Speakers : Praveen Guddeti (1---5) Tibor Moldovan (6---9) CSE 976, April 15, 2002. Outline. Introduction Evaluation criteria Non-cooperative interaction protocols Voting Auctions Bargaining

Download Presentation

Distributed Rational Decision Making

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Distributed rational decision making

Distributed Rational Decision Making

Author: Tuomas W. Sandholm

Speakers: Praveen Guddeti (1---5)

Tibor Moldovan (6---9)

CSE 976, April 15, 2002


Outline

Outline

  • Introduction

  • Evaluation criteria

  • Non-cooperative interaction protocols

    • Voting

    • Auctions

    • Bargaining

    • General equilibrium market mechanisms

    • Contract nets

    • Coalition formation

  • Conclusions


Introduction

Introduction

  • Automated negotiation systems with self-interested agents are becoming increasing important.

    1. Technology push.

    2. Application pull.

  • Paper deals with protocols designed using a non-cooperative, strategic perspective.


Evaluation criteria

Evaluation Criteria

  • Social welfare

  • Pareto efficiency

  • Individual rationality

  • Stability

  • Computational efficiency

  • Distribution and communication efficiency


Evaluation criteria 1 social welfare

Evaluation Criteria1. Social Welfare

  • It is the sum of all agents’ payoffs or utilities in a given solution.

  • Requires inter-agent utility comparisons.


Evaluation criteria 2 pareto efficiency

Evaluation Criteria2. Pareto Efficiency

  • A solution x is Pareto efficient if there is no other solution x’ such that

    • at least one agent is better off in x’ than in x, and

    • no agent is worse off in x’ than in x.

  • Does not require inter-agent utility comparisons.

  • Social welfare maximizing solutions are a subset of Pareto efficient ones.


Evaluation criteria 3 individual rationality

Evaluation Criteria3. Individual Rationality

  • Participation in a negotiation is individually rational to an agent only if it is profitable.

  • A mechanism is individually rational if participation is individually rational for all agents.

  • Only individually rational mechanisms are viable.


Evaluation criteria 4 stability

Evaluation Criteria4. Stability

  • The protocol mechanisms should motivate each agent to behave in the desired manner.

  • Protocol mechanisms may have dominant strategies. This means that an agent is best off by using a specific strategy no matter what strategies the other agents use.

  • Nash equilibrium: Each agent chooses a strategy that is a best response to the other agents’ strategies.


Nash equilibrium formal definition

Nash Equilibrium:Formal definition

  • The strategy profile S*A={ S*1, S*2,…, S*|A|} among agents A is in Nash equilibrium if for each agent i , S*i is the agent’s best strategy given that the other agents choose strategies {S*1, S*2,…, S*i-1, S*i+1,… S*|A|}.

  • In some games no Nash equilibrium exists.

  • Some games have multiple Nash equilibrium.


Nash equilibrium comments

Nash Equilibrium:Comments

  • Even if Nash equilibrium exists and is unique, there are limitations regarding what the Nash equilibrium guarantees.

  • In sequential games it only guarantees stability in the beginning of the game.

  • Subgame perfect Nash equilibrium.

  • Nash equilibrium is often too weak because subgroups of agents can deviate in a coordinated manner.

  • Sometimes efficiency and stability goals conflict.


Evaluation criteria 5 computational efficiency

Evaluation Criteria5. Computational Efficiency

  • The protocol mechanisms when used by agents should need as little computation as possible.

  • Trade off between:

    • the cost of the computation needed for the protocol mechanisms and

    • the solution quality.


Evaluation criteria 6 distribution and communication efficiency

Evaluation Criteria6. Distribution and Communication Efficiency

  • Distributed protocols should be preferred in order to avoid a single point of failure and a performance bottleneck – among other reasons.

  • Minimize the amount of communication required to get to a desired global solution.

  • These two goals can conflict.


Non cooperative interaction protocols

Non-cooperative interaction protocols

  • Voting

  • Auctions

  • Bargaining

  • General equilibrium market mechanisms

  • Contract nets

  • Coalition formation


Non cooperative interaction protocols 1 voting

Non-cooperative Interaction Protocols1. Voting

  • All agents give input to a mechanism.

  • Outcome chosen by the mechanism is solution for all agents.

  • Outcome is enforced.

  • Voters.

    • Truthful voters.

    • Strategic (Insincere) voters.


Voters truthful voters 1

VotersTruthful Voters (1)

  • Each agent i  A has an asymmetric and transitive strict preference relations  i on O.

  • Social choice rule.

    • input the agents’ preference relations ( 1,…, |A|).

    • output the social preferences denoted by a relation *.


Truthful voters 2 properties of a social choice rule

Truthful Voters (2)Properties of a social choice rule

  • * should exist for all possible inputs.

  • * should be defined for every pair o, o’ O.

  • * should be asymmetric and transitive over O.

  • The outcome should be Pareto efficient.

  • The scheme should be independent of irrelevant alternatives.

  • No agent should be a dictator.


Truthful voters 3

Truthful Voters (3)

  • Arrow’s Impossibility Theorem: No social rule satisfies all of these six conditions.

  • Relax the first property.

  • Relax the third property.

    • Plurality protocol.

    • Binary protocol.

    • Borda protocol.


Truthful voters 4 plurality protocol

Truthful Voters (4)Plurality Protocol

  • Majority voting protocol.

  • All alternatives are compared simultaneously.

  • The one having the highest number of votes wins.

  • Irrelevant alternative can split the majority.


Truthful voters 5 binary protocol 1

Truthful Voters (5)Binary Protocol (1)

  • Pair wise voting with the winner staying to challenge remaining alternatives.

  • Irrelevant alternatives can change outcomes.

  • Agenda i.e. order of the pairings can change the outcomes.


Truthful voters 6 binary protocol 2

Truthful Voters (6)Binary Protocol (2)

35 % of agents have preferences cdba

33 % of agents have preferences acdb

32 % of agents have preferences bacd


Truthful voters 7 borda protocol 1

Truthful Voters (7)Borda Protocol (1)

  • Assign an alternative |O| points whenever it is the highest in some agent’s preference list, |O| -1 when it is second and so on.

  • Sum the counts of all alternatives.

  • Alternative with highest count is the winner.

  • Irrelevant alternatives lead to paradoxical results.


Borda protocol 2

Borda Protocol (2)

Agent Preferences

1 a  b  c  d

2 b  c  d  a

3 c  d  a  b

4 a  b  c  d

5 b  c  d  a

6 c  d  a  b

7 a  b  c  d

Borda count c wins with 20, b has 19, a has 18, d loses with 13

Borda count a wins with 15, b has 14,loses with 13

with d removed


Voters strategic insincere voters 1

VotersStrategic (Insincere) Voters (1)

  • Revelation principle: Suppose some protocol implements social choice function f(.) in Nash (or dominant strategy) equilibrium, then f(.) is implementable in Nash (or dominant strategy) equilibrium via a single-step protocol where the agents reveal their types truthfully.


Voters strategic insincere voters 2

VotersStrategic (Insincere) Voters (2)

  • Gibbard-Satterthwaite impossibility theorem: Let each agent’s type i, consist of a preference order i on O. Let there be no restrictions on i, i.e. each agent may rank the outcomes O in any order. Let |O|  3. Now, if the social choice function f(.) is truthfully implementable in a dominant strategy equilibrium, then f(.) is dictatorial, i.e. there is some agent i who gets (one of) his most preferred outcomes chosen no matter what types the others reveal.


Voters strategic insincere voters 3

VotersStrategic (Insincere) Voters (3)

  • Circumventing the GSIT:

    • Restricted preferences.

    • Groves-Clarke Tax Mechanism.

  • Groves-Clarke Tax Mechanism:

    • o = (g ,1,… |A|).

    • i is the amount agent i receives.

    • g encodes the other features of the outcome.


Strategic insincere voters 4 groves clarke tax mechanism 1

Strategic (Insincere) Voters (4)Groves-Clarke Tax Mechanism(1)

  • Quasilinear preferences: ui(o) = vi(g) + i.

  • Net benefit: vi(g) = vi gross(g) – P / |A|.

  • Every agent iA reveals his valuation vi(g) for every possible g.

  • The social choice is g* =arg maxg i vi(g).

  • Every agent is levied a tax: tax i =  ji vj (g*) -  ji vj (arg maxg  ki vk (g)).


Strategic insincere voters 5 groves clarke tax mechanism 2

Strategic (Insincere) Voters (5)Groves-Clarke Tax Mechanism(2)

  • Size of an agent’s tax is exactly how much his vote lowers the other’s utility.

  • Quasilinearity:

    • No agent should care how others divide payoffs among themselves.

    • An agent’s valuation vi gross(g) should not depend on the amount of money that the agent will have.


Voters strategic insincere voters 6

VotersStrategic (Insincere) Voters (6)

  • If each agent has quasilinear preferences, then each agent’s dominant strategy is to reveal his true preferences.

  • Agents need not waste effort in counter speculating each others’ preference declarations.

  • Participation is individually rational.


Voters strategic insincere voters 7

VotersStrategic (Insincere) Voters (7)

  • Problems of Groves-Clarke Tax Mechanism:

    • Does not maintain budget balance.

    • Not coalition proof.

    • Intractable.

  • Other ways to circumvent the GSIT:

    • Choosing a dictator randomly.

    • Make the computation of an untruthful revelation prohibitively costly.


Non cooperative interaction protocols 2 auctions

Non-cooperative Interaction Protocols2.Auctions

  • Unlike voting where the outcome binds all agents, in auctions the outcome is usually a deal between two agents.

  • In voting the protocol designer wants to enhance the social good, while in auctions, the auctioneer wants to maximize his own profit.

  • Classical setting.

  • Contracting setting.


Auctions 2

Auctions (2)

  • Auction settings.

  • Auctions protocols.

  • Efficiency of the resulting allocation.

  • Revenue equivalence and non-equivalence.

  • Bidder collusion.

  • Lying auctioneer.

  • Bidders lying in non-private-value auctions.

  • Undesirable private information revelation.

  • Roles of computation in auctions.


1 auction settings

1. Auction settings

  • Three qualitatively different auctions depending on how an agent’s value of the item is formed:

    • Private value.

    • Common value.

    • Correlated value.


2 auctions protocols

2. Auctions protocols

  • English (first-price open-cry) auction.

  • First-price sealed-bid auction.

  • Dutch (descending) auction.

  • Vickrey (second-price sealed-bid) auction.

    • Allocation of computation resources in OS, allocation of bandwidth in computer networks, computationally control building heating.

    • Has not been widely adopted in auctions among humans.


3 efficiency of the resulting allocation

3. Efficiency of the resulting allocation.

  • In isolated private value or common value auctions, each one of the four auction protocols allocates the auctioned item Pareto efficiently to the bidder who values it the most.

  • All four protocols are Pareto efficient in the allocation.

  • The dominant strategies (Vickrey and English) are more efficient.


4 revenue equivalence and non equivalence

4. Revenue equivalence and non-equivalence.

  • Revenue equivalence: All of the four auction protocols produce the same expected revenue to the auctioneer in private value auctions where the values are independently distributed and bidders are risk-neutral.

  • Among risk averse bidders, the Dutch and the first-price sealed-bid protocols give higher expected revenue to the auctioneer.

  • A risk averse auctioneer achieves higher expected utility via the Vickrey or English protocols.


Revenue equivalence and non equivalence 2

Revenue equivalence and non-equivalence. (2)

  • In non-private value auctions, both the English and Vickrey protocols produce greater expected revenue to the auctioneer than the first-price sealed-bid auction or Dutch auction.

  • In non-private value auctions with at least three bidders, the English auction leads to higher revenue than the Vickrey auction.


5 bidder collusion

5. Bidder collusion.

  • The English auction and the Vickrey auction actually self-enforce some of the most likely collusion agreements.

  • First-price sealed-bid and the Dutch auctions are preferred for deterring collusion.

  • For collusion to take place in Vickrey, first-price sealed-bid or Dutch auctions the bidders have to identify each other before placing the bids.


6 lying auctioneer

6. Lying Auctioneer.

  • In Vickrey auction the auctioneer may lie about the value of the second highest bidder.

  • In the English auction the auctioneer can use shills that bid in the auction in order to make the real bidders increase their valuations of the item.

  • The auctioneer may bid himself to guarantee that the item will not be sold below a certain price.


7 bidders lying in non private value auctions

7. Bidders lying in non-private-value auctions.

  • Winner’s curse: If an agent bids its valuations and wins the auction, it will know that its valuation was too high because the other agents bid less.

  • Agents should bid less than their valuations.

  • This is the best strategy in Vickrey auctions.

  • Vickrey fails to induce truthful bidding in most auction settings.


8 undesirable private information revelation

8. Undesirable private information revelation.

  • In Vickrey auctions the agents often bid truthfully. This leads to the bidders revealing their true valuations.

  • This information is sensitive and the bidders would prefer not to reveal it.

  • Another reason why the Vickrey auction protocol is not widely used among humans.


9 roles of computation in auctions

9. Roles of computation in auctions.

  • Two issues arise from computation in auctions:

  • Computationally complex look ahead that arises when auctioning interrelated items one at a time.

  • Implications of costly local marginal cost (valuation) computation or information gathering in a single-shot auction.


Interrelated auctions

Interrelated auctions.

  • Look ahead:

    • Without look ahead the allocation may be inefficient.

    • With look ahead the agents will not bid their true per-item cost.

    • Computation cost may be prohibitively great.

  • Allow agents to backtrack from commitments by paying penalties.


Single shot auctions

Single-shot auctions

  • Incentive to counter speculate: In a single-shot private value Vickrey auction with uncertainty about an agent’s own valuations, a risk neutral agent’s best action can depend on the other agents. It follows that is is worth counter speculating.


Non cooperative interaction protocols 3 bargaining

Non-cooperative Interaction Protocols3. Bargaining

  • Real world settings usually consist of a finite number of competing agents, so neither monopoly,nor monopsony nor perfect competition assumptions strictly apply.

  • Bargaining theory fits in this gap.

  • Bargaining theory:

    • Axiomatic

    • Strategic


Bargaining theory axiomatic

Bargaining Theory Axiomatic

  • Does not use the idea of equilibrium.

  • Desirable properties for a solution, called axioms of the bargaining solution, are postulated.

  • Then the solution that satisfies these axioms are sought.

  • Nash bargaining solution.


Axiomatic bargaining theory nash bargaining solution 1

Axiomatic Bargaining Theory Nash Bargaining Solution (1)

  • Nash analyzed a 2-agent setting where the agents have to decide on an outcome o  O, and the fallback outcome ofallback occurs if no agreement is reached.

  • There is a utility function

    ui: O  R for each agent i  [1,2].

  • It is assumed that the set of feasible utility vectors { (u1 (o), u2 (o)) | o  O} is convex.


Axiomatic bargaining theory nash bargaining solution 2

Axiomatic Bargaining TheoryNash Bargaining Solution (2)

  • Axioms for the Nash bargaining solution u* = (u1(o*), u2(o*)) are:

    1. Invariance.

    2. Anonymity (symmetry).

    3. Independence of irrelevant alternatives.

    4. Pareto efficiency.


Axiomatic bargaining theory nash bargaining solution 3

Axiomatic Bargaining TheoryNash Bargaining Solution (3)

  • The unique solution that satisfies these four axioms is:

    o* = arg maxo[u1(o)–u1(ofallback)][u2(o)–u2 (ofallback)]

  • Other bargaining solutions also exist.


Bargaining theory strategic 1

Bargaining TheoryStrategic (1)

  • Bargaining situation is modeled as a game.

  • Solution is based on an analysis of which of the players’ strategies are in equilibrium.

  • Solution is not unique.

  • Explains the behavior of rational utility maximizing agents better than axiomatic approaches.

  • Usually analyses sequential bargaining.


Bargaining theory strategic 2

Bargaining TheoryStrategic (2)

  • Finite number of offers with no time discount.

  • Finite number of offers with time discount.

  • Infinite number of offers with no time discount.

  • Infinite number of offers with time discount.


Strategic bargaining theory rubinstein bargaining solution

Strategic Bargaining TheoryRubinstein Bargaining Solution

  • In a discounted infinite round setting, the subgame perfect Nash equilibrium outcome is unique. Agent 1 gets (1- 2) / (1- 12), where 1 is 1’s discount factor, and 2 is 2’s. Agent 2 gets one minus this. Agreement is reached in the first round.

  • The proof gives a way to solve for subgame perfect Nash equilibrium payoffs.


Strategic bargaining theory fixed bargaining cost per negotiation round

Strategic Bargaining TheoryFixed Bargaining Cost per negotiation round

  • If the agents have symmetric bargaining costs, the solution concept is powerless.

  • If 1’s bargaining cost c1 is even slightly smaller than 2’s cost c2, then 1 gets the entire dollar.

  • If 1’s bargaining cost is greater than 2’s, then 1 receives a payoff that equals the second agent’s bargaining cost, and agent 2 receives the rest.


Strategic bargaining theory recent extensions kraus et al

Strategic Bargaining TheoryRecent extensions [Kraus et al.]

  • Sequential bargaining with outside options.

  • Sequential bargaining where one agent gains and one loses over time.

  • Negotiation over time when agents do not know each other’ types.


Bargaining computation 1

BargainingComputation (1)

  • Assume perfect rationality.

  • The space of deals is assumed to be fully comprehended by the agents.

  • The value of each potential contract known.

  • Focus of future work:

    • make the cost of search explicit and.

    • consider its trade-off with bargaining gains.


Bargaining computation 2

BargainingComputation (2)

  • There are two searches occurring in bargaining:

    • Intra-agent deliberative search: an agent locally generates alternatives, evaluates them, counter speculates, does look ahead etc.

    • Inter-agent committal search: the agents make agreements with each other regarding the solution.


Distributed rational decision making

  • Introduction

  • Evaluation criteria

  • Non-cooperative interaction protocols

    • Voting

    • Auctions

    • Bargaining

    • General equilibrium market mechanisms

    • Contract nets

    • Coalition formation

  • Conclusions


  • Login