Loading in 5 sec....

Oblivious AQM and Nash EquilibriaPowerPoint Presentation

Oblivious AQM and Nash Equilibria

- 217 Views
- Uploaded on

Download Presentation
## PowerPoint Slideshow about 'Oblivious AQM and Nash Equilibria' - Roberta

**An Image/Link below is provided (as is) to download presentation**

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

### Oblivious AQM and Nash Equilibria

### Existence

### Efficiency

### Achievability

### Summary and Future Work

### Thank You !

### Appendixes

D. Dutta, A. Goel and J. Heidemann

USC/ISI USC USC/ISI

IEEE INFOCOM 2003 - The 22nd Annual Joint Conference of the IEEE Computer and Communications Societies

Presented By Sharon Mendel

Game Theory in Networks Seminar 25/01/2006

Active Queue Management ??

- A congestion control protocol (e.g. TCP) operates at the end-points and uses the drops or marks received from the Active Queue Management policies (e.g. Drop-tail, RED) at routers as feedback signals to adaptively modify the sending rate in order to maximize its own goodput.

Oblivious AQM & Nash Equilibria

Transport Control Protocol

- TCP is the dominating transport layer protocol in the Internet and accounts for over 90% of the total traffic.
- The TCP Protocol is well defined, robust and congestion-reactive (thus stable).
- The end-to-end congestion control mechanisms of TCP have been a critical factor in the robustness of the Internet.
- It is widely believed that if all users deployed TCP, networks will rarely see congestion collapses and the overall utilization of the network will be high.

Oblivious AQM & Nash Equilibria

Today’s Internet

- There are indications that the amount of non-congestion-reactive traffic is on the rise.
- Most of this misbehaving traffic does not use TCP.
- e.g. Real media, network games, other real time multimedia applications.

- The unresponsive behavior can result in both unfairness and congestion collapse for the Internet.
- The network itself must now participate in controlling its own resource utilization.

* Some of the previous slides:

“Promoting the Use of End-to-End Congestion Control in the Internet”,

S. Floyd and K. Fall, IEEE/ACM Transactions on Networking Vol. 7 1999.

Oblivious AQM & Nash Equilibria

The Paper’s Motivation

- TCP (and in fact, any transport protocol) does not guarantee good performance in the face of aggressive, greedy users (who are willing to violate the protocol to obtain better performance).
- Protocol Equilibrium – A protocol which leads to an efficient utilization and a somewhat fair distribution of network resources (like TCP does), and also ensure that no user can obtain better performance by deviating from the protocol.
- If protocol equilibrium is achievable, then it would be a useful tool in designing robust networks.

Oblivious AQM & Nash Equilibria

Oblivious AQM

- Oblivious (stateless) AQM scheme – a router strategy that does not differentiate between packets belonging to different flows.
- Stateful schemes – e.g. Fair-Queuing.
- Stateful Schemes offer good performance, but oblivious schemes are easier to implement.

Oblivious AQM & Nash Equilibria

Popular AQM Schemes (1)

- Congestion avoidance is achieved through packet dropping.
- Drop-Tail – Buffers as many packets as it can and drops the ones it can't buffer:
- Distributes buffer space unfairly among traffic flows.
- Can lead to global synchronization as all TCP connections "hold back" simultaneously, hence networks become under-utilized.

DT

Oblivious AQM & Nash Equilibria

Popular AQM Schemes (2)

- RED – Random Early Dedication - Monitors the average queue size and drops packets based on statistical probabilities:
- If the buffer is almost empty, all incoming packets are accepted; As the queue grows, the probability for dropping an incoming packet grows; When the buffer is full, the probability has reached 1 and all incoming packets are dropped.
- Considered more fair than tail drop - The more a host transmits, the more likely it is that its packets are dropped.
- Prevents global synchronization and achieves lower average buffer occupancies.

RED

Oblivious AQM & Nash Equilibria

Oblivious AQM and Nash Equilibria

- The paper studies the existence and quality of Nash equilibria imposed by oblivious AQM schemes on selfish agents:
- Existence
- Efficiency
- Achievability

Oblivious AQM & Nash Equilibria

Content

- Introduction
- The Model
- Existence
- Efficiency
- Achievability
- Summary and Future Work

Oblivious AQM & Nash Equilibria

The Internet Game

- Players – The end-to-end selfish traffic agents.
- Each player has a strategy which is to control the average rate he tries to push through the network.
- Users’ Performance Metric – goodput.

- Rules – set by the AQM policies (AQM schemes in routers).
- Nash Equilibrium – No selfish agent has any incentive to unilaterally deviate from it’s current state.
- Oblivious AQM scheme leads to a protocol equilibrium only if it imposes a Nash equilibrium on the selfish users.
- Paper’s focus– AQM schemes that guarantee bounded average buffer occupancy regardless of the total arrival rate.

Oblivious AQM & Nash Equilibria

The Markovian Internet Game

- The agents generate Poisson traffic.
- Does not accurately model Internet traffic, yet a reasonable first step.
- Each user controls its own offered load = the average Poisson traffic rate.

- The system is modeled as a M/M/1/K queue:
- Average service time - Without loss of generality assumed to be unity.
- Buffers capacity - K=B.

- No assumptions are made on the selfish protocol (i.e. TCP, AIMD etc’).

Oblivious AQM & Nash Equilibria

The M/M/1/K Internet Game

- n – Number of users – players.
- λi – The Poisson average arrival rate of player i.
- Ui – Utility function of player i.
- μi – Goodput , Ui = μi .
- p – AQM router drop probability due an average aggregated load (offered load) of λ and an average service time of unity.
- A symmetric Nash equilibrium - Ensures that every agent has the same goodput at equilibrium.
- Unless mentioned otherwise, quantities such as the rates, goodput and throughput are averages (Poisson traffic sources).

Oblivious AQM & Nash Equilibria

RED

The Model

Nash Equilibrium Conditions (1)- No agent can increase their goodput, at Nash equilibrium, by either increasing or decreasing their throughput:
- A symmetric Nash equilibrium:
- Oblivious AQM scheme, hence functions of router state (drop probability, queue length) are independent in i:

Utility Function in Nash Equilibrium :

Nash Equilibrium Condition :Nash Equilibrium Satisfying Condition –Necessary and sufficient

Oblivious AQM & Nash Equilibria

Nash Equilibrium Conditions (2)

- λNash , μNash,pNash - Theaggregate throughput (offered load), goodput, drop probability respectively.
- The Nash equilibrium imposed by an AQM scheme isefficient if the goodput of any selfish agent isbounded below when the throughput (offered load) of the same agent isboundedabove:

Nash Equilibrium Efficiency Condition :

c1 c2some constants

pNash is bounded

Oblivious AQM & Nash Equilibria

Are there oblivious AQM schemes that impose Nash equilibria on selfish users?

Oblivious AQM & Nash Equilibria

Drop-Tail Queuing

- p - Drop Probability = Probability to find a full system.
- From Queuing Theory:
- Theorem 1: There is NO Nash Equilibrium for selfish agents and routes implementing Drop-Tail queuing.
- Proof: applying the condition for Nash equilibrium:

QED.

Oblivious AQM & Nash Equilibria

If lq <minth

0

p =

If minth lq maxth

1

Otherwisw

Existence

Random Early Detection- Approximated steady state RED model:
- From Queuing Theory, at steady state:
- RED Router, at steady state:
Steady State:

“Faster network simulation with scenario pre-filtering” Tech. Rep., USC/ISI Tech Report 550, November 2001.

lq – Queue length

p – Drop Probability

λ – Aggregated Offered Load

lq maxth

Oblivious AQM & Nash Equilibria

RED and Nash Equilibria

- Theorem 2: RED Does NOT impose a Nash equilibrium on uncontrolled selfish agents.
- Proof: applying the condition for Nash equilibrium:
- Summary:
- RED punishes all flows with the same drop probability.
- The nature of the drop function is considerably gentle.
- Misbehaving flows can push more traffic and get less hurt (marginally).
- There is no incentive for any source to stop pushing packets.

QED.

RED is oblivious:

Nash Equlibria does not exist.

Oblivious AQM & Nash Equilibria

If lvq <minth

p =

If minth lvq maxth

Otherwise

1

Existence

Virtual Load RED- Virtual Load RED model:
- Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if
- Proof:
- Throughput at Nash equilibrium is independent of minth:

lvq – M/M/1 Queue length when facing the same load :

Oblivious AQM & Nash Equilibria

If an Oblivious AQM scheme can impose a Nash equilibria, is that equilibria efficient, in terms of achieving high goodput and low drop probability?

Oblivious AQM & Nash Equilibria

Normalized Data

Goodput

minth = 0

maxth =50

n – 1,..,50

Number Of Flows

Efficiency

Example - Efficiency of VLRED- Proof: Applying Nashequilibrium satisfying andefficiency conditions:

Theorem 4: VLRED is not efficientimposing a Nash Equilibriumon selfish agents

α, β - some constants

nμ = θ(lvq2)

nμ/α

lvq2

The goodput falls to zero asymptotically.

QED.

Oblivious AQM & Nash Equilibria

Goodput

Normalized Data

Drop Probability

Number Of Flows

Efficiency

Efficient Nash AQM scheme- Assume – Totaldesirable offered loadat Nash equilibrium:
- Oblivious mechanism can ensure an efficient Nash equilibria under selfish behavior of users.

ENAQM drop probability is bounded

Oblivious AQM & Nash Equilibria

How easy is it for players (users) to reach the equilibrium point? orHow can we ensure that agents actually reach the Nash equilibrium state?

Oblivious AQM & Nash Equilibria

Ensuring a Nash equilibrium by an Oblivious AQM

- λ*i – Offered load at equilibria when the number of agents is i.
- Δi = λ*i -λ*i-1 = iααsome constant
- p = ƒ(λ*i) non decreasing and convex.
- Applying Nash equilibrium satisfying and efficiency conditions:
- From convexity of p*i:
- From efficiency:
- The equilibrium imposed by any oblivious AQM strategy is (very) sensitive to the number of agents, thus making it impractical to deploy in the Internet.
- Agents need the help of the router to reach the equliibria.

c1, c2 - some constants

The sensitivity coefficient falls faster than the inverse quadric.

c - constant

Oblivious AQM & Nash Equilibria

Overview

Now what? – Future Work

Some further work

Oblivious AQM & Nash Equilibria

Overview

- Introduction – Today’s Internet.
- The proposed model – Markovian (M/M/1/K) Game
- Existence –
- Drop tail and RED cannot impose a Nash equilibra.
- VLRED imposes a Nash equilibra.But the equilibrium points do not have a very high utilization.

- Efficiency - ENAQM imposes an efficient Nash equilibra.
- Achievability - Equilibrium points in oblivious AQM strategies are very sensitive to the change in the number of users.
- It may be hard to deploy oblivious schemes that do have Nash equilibria without the explicit help of a protocol.

Oblivious AQM & Nash Equilibria

Now What ? - Future Work

- VLRED – Explore why the Nash equilibria do not result in good network utilization.
- Conjecture – VLRED’s drop function becomes very harsh as we reach equilibria.

- Study gentler versions of VLRED and determine whether such modification can still impose Nash equilibria.
- Design protocols which lead to efficient network operation, such that no user has any incentive to unilaterally deviate from the protocol – can it be done ? – The Protocol Equilibrium Question.

Oblivious AQM & Nash Equilibria

Some Further work (1)

- “Towards Protocol Equilibrium with Oblivious Routers” D. Dutta, A. Goel and J. Heidemann, IEEE INFOCOM 2004.
- “…In this paper, we show that if routers used EWMA to measure the aggregate rate, then the best strategy for a selfish agent to minimize its losses is to arrive at a constant rate. Even though the protocol space is arbitrary, our scheme ensures that the best greedy strategy is simple, i.e. send with CBR. Then, we show how we can use the results of an earlier paper to enforce simple and efficient protocol equilibria on selfish traffic agents…”

Oblivious AQM & Nash Equilibria

Some Further work(2)

- “Pricing Differentiated Services A Game -Theoretic Approach”, E. Altman, D. Barman, R. El Azouzi, D. Ros, B. Tuffin, (accepted for publication in Computer Networks, 2005).
- “…The goal of this paper is to study pricing of differentiated services and its impact on the choice of service priority at equilibrium. We consider both TCP connections as well as non controlled (real time) connections…. We first study the performance of the system as a function of the connections’ parameters and their choice of service classes. We then study the decision problem of how to choose the service classes. We model the problem as a noncooperative game. We establish conditions for an equilibrium to exist and to be uniquely defined. We further provide conditions for convergence to equilibrium from non equilibria initial states. We finally study the pricing problem of how to choose prices so that the resulting equilibrium would maximize the network benefit…”
- “…The paper (“Oblivious AQM and Nash Equilibria”) restricted itself to symmetric users and symmetric equilibria and the pricing issue was not considered. In this framework, with a common RED buffer, it was shown that an equilibrium does not exist. An equilibrium was obtained and characterized for an alternative buffer management that was proposed, called VLRED. We note that in contrast to (Oblivious AQM and Nash Equilibria”), since we also include in the utility of CBR traffic a penalty for losses we do obtain an equilibrium when using RED…”

Oblivious AQM & Nash Equilibria

Oblivious AQM & Nash Equilibria

Stateful Schemes – Fair Queuing

From Queuing Theory

Nash Equilibrium Conditions

Approximated Steady State RED Model

VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof

Efficient Nash AQM Drop Probability

Oblivious AQM & Nash Equilibria

Stateful Schemes – Fair Queuing - Nagle’s Algorithm

- Gateways maintain separate queues for packets from each individual source.
- The queues are serviced in a round-robin manner.
- Nagle’s algorithm, by changing the way packets from different sources interact, does not reward, nor leave others vulnerable to, anti-social behavior.

“Analysis and Simulation of a Fair Queuing Algorithm”,

A.Demers, S. Keshav and S.J. Shenker, ACM SIGCOMM, 1989.

Oblivious AQM & Nash Equilibria

λ

λ

λ

λ

0

1

K-1

K

2

μ

μ

μ

μ

μ

From Queuing Theory- M/M/1/K Queuing system - Poisson arrivals, Exponentially distributed service times, one server and finite capacity buffer:
- PASTA - Poisson Arrivals See Time Averages – For a queuing system, when the arrival process is Poisson and independent of the service process:The probability that an arriving customer finds i customers in the system Equals The probability that the system is at state i.
- πi – Probability that the system is in state i, Using birth-death model:
- Block Probability -

Oblivious AQM & Nash Equilibria

Nash Equilibrium Condition

- Substituting 2,3 in 5 we obtain:

Nash Equilibrium Satisfying Condition :

Oblivious AQM & Nash Equilibria

Approximated RED Transfer Function

Drop prob.

Standard RED Transfer Function

Drop prob.

Normalized Offered Load

Approximated RED Buffer Occupancy

Avg. Buffer Occupancy

Avg. Buffer Occupancy

Normalized Offered Load

Existence - Appendix

Approximated Steady State RED Model“Faster network simulation with scenario pre-filtering”

Tech. Rep., USC/ISI Tech Report 550, November 2001.

Example:

minth = 10 maxth =20

n – 1,..,50 Pmax = 0.3

Oblivious AQM & Nash Equilibria

If lvq <minth

p =

If minth lvq maxth

Otherwise

1

Existence - Appendix

VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (1)- Virtual Load RED model:
- Proof of Theorem 3: VLRED imposes a Nash Equilibrium on selfish agents if
- Assume the drop probability can be written as a continues function for all lvq<maxth:
- Nash equilibrium condition:

lvq – M/M/1 Queue length when facing the same load :

Oblivious AQM & Nash Equilibria

VLRED Model, Theorem 3 - Nash Equilibrium Existence - Proof (2)

- Substituting and simplifying we obtain:
- This is true if (by Substituting n=1)

Oblivious AQM & Nash Equilibria

Efficient Nash AQM Drop Probability

- Total desirable offered load at Nash equilibrium:
- Substituting y2=1-λ and then integrating:
- k is determined so when there is one user, the drop probability is zero as long as his offered load is less than unity
- The offered load at Nash equilibria is bounded drop probability at equilibria is bounded (proved by substitution).

k - arbitrary constant to be determined

Oblivious AQM & Nash Equilibria

Download Presentation

Connecting to Server..