Network simulation and testing
This presentation is the property of its rightful owner.
Sponsored Links
1 / 66

Network Simulation and Testing PowerPoint PPT Presentation


  • 46 Views
  • Uploaded on
  • Presentation posted in: General

Network Simulation and Testing. Polly Huang EE NTU http://cc.ee.ntu.edu.tw/~phuang [email protected] Traffic Papers. V. Paxson, and S. Floyd, Wide-Area Traffic: The Failure of Poisson Modeling. IEEE/ACM Transactions on Networking, Vol. 3 No. 3, pp. 226-244, June 1995

Download Presentation

Network Simulation and Testing

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Network simulation and testing

Network Simulation and Testing

Polly Huang

EE NTU

http://cc.ee.ntu.edu.tw/~phuang

[email protected]


Traffic papers

Traffic Papers

  • V. Paxson, and S. Floyd, Wide-Area Traffic: The Failure of Poisson Modeling. IEEE/ACM Transactions on Networking, Vol. 3 No. 3, pp. 226-244, June 1995

  • W. E. Leland, M. S. Taqqu, W. Willinger, and D. V. Wilson, On the Self-Similar Nature of Ethernet Traffic. IEEE/ACM Transactions on Networking, Vol. 2, No. 1, pp. 1-15, Feb. 1995

  • M. E. Crovella and A. Bestavros, Self-Similarity in World Wide Web Traffic: Evidence and Possible Causes. IEEE/ACM Transactions on Networking, Vol 5, No. 6, pp. 835-846, December 1997

  • Anja Feldmann; Anna C. Gilbert; Polly Huang; Walter Willinger, Dynamics of IP traffic: A study of the role of variability and the impact of control. In the Proceeding of SIGCOMM '99, Cambridge, Massachusetts, September 1999


Paper selection

Paper Selection


Identifying internet traffic

Identifying Internet Traffic

Failure of Poisson

Self-similar Traffic

Practical Model


The problem

The Problem

  • What is the traffic workload like?

  • Call/packet arrival rate as a process

  • What kind of process is it?

  • Very old problem and a lot of work


Because

Because

  • Traces are available

  • Researchers care about

    • The validness of their assumption

    • The network traffic being independent Poisson

  • Operation people care a lot about

    • The amount of buffer/bandwidth to provision for their networks

    • The profit comes from satisfying customers with minimum infrastructure cost


Telephone network

Telephone Network

  • Assumptions

    • Poisson call arrivals

    • Exponential call duration

  • Wonderful Property

    • Poisson mixing with Poisson is still Poisson

    • Average rate well-characterize a call

  • The whole queueing theory


Data network

Data Network?

  • Wide-Area Traffic: The Failure of Poisson Modeling

  • V. Paxson, and S. Floyd

  • IEEE/ACM Transactions on Networking, Vol. 3 No. 3, pp. 226-244, June 1995


A study of the wide area traffic

A Study of the Wide-Area Traffic

  • Two units of examination

    • Connections vs. packets

  • A sizeable number of traces

    • ~4M connections, ~26M packets

    • Different location and different time

  • Inter-arrival processes

    • TCP connections

    • Telnet packets

    • FTPDATA connections

  • Going self-similar


Unit of observation

Unit of Observation

  • Telephone network

    • Circuit-switched

    • The unit is circuit, i.e., a call

    • People picking up the phone and talk for a while

  • Data network

    • Packet-switched

    • The unit is packet

    • Another unit is connection, comparable to call

    • People starting up an FTP connection and send data for a while


Packet connection

Packet  Connection

  • Hosts send/receive packets over a channel at the transport layer

    • Reliable: TCP

    • Non-reliable: UDP

  • Packets from various channels multiplex at the the network layer

    • IP Routers switched on the packets


Inter arrival process a little exercise

?

?

Inter-Arrival Process: A Little Exercise

Beginning

SYN

ACK&SYN

ACK&Segment 1

FIN

ACK&FIN

ACK

End

Beginning


Tcp connection arrival poisson

TCP Connection Arrival Poisson?

Depends


Application dependent

Application Dependent

  • TELNET

    • Users typing ‘telnet cc.ee.ntu.edu.tw’

  • FTP

    • User typing ‘ftp cc.ee.ntu.edu.tw’

  • FTPDATA burst

    • User typing ‘mget net-simtest-*.ppt’

  • FTPDATA

    • Each individual TCP transfer

  • NNTP & SMTP

    • Machine initiated and/or timer-driven


Independent and poisson

Independent and Poisson?


Quick summary

Quick Summary

  • TELNET and FTP

    • Independent and Poisson

    • Both the 1-hour and 10-min scales

  • FTPDATA bursts and SMTP

    • At the 10-min interval

    • Not ‘terribly far’ from Poisson

    • SMTP inter-arrival is not independent

  • FTPDATA, NNTP

    • Clearly not Poisson


Before one can explain

Before One Can Explain

  • Human-initiated process

    • Independent and Poisson

  • Non-human-initiated process

    • Well, who knows


Explanations i

Explanations I

  • TELNET and FTP

    • User initiated

    • Users typing ‘telnet cc.ee.ntu.edu.tw’

    • User typing ‘ftp cc.ee.ntu.edu.tw’

  • FTPDATA bursts

    • User typing ‘mget net-simtest-*.ppt’

    • Actually, taking the closely-spaced connections… (<= 4 sec)

  • FTPDATA

    • TCP connections


Explanations ii

Explanations II

  • NNTP

    • Flooding to propagate network news

    • Arrival of news trigger another

    • Periodical and implementation/configuration dependent

  • SMTP

    • Mailing list

    • Timer effects from the DNS queries


Telnet packets poisson

TELNET Packets Poisson?

No, heavy-tailed!


Show in 4 ways

Show in 4 Ways

  • Distribution of packet inter-arrival time

    • Exponential processes ramp up significantly slower

  • Packet arrival pattern in seconds and 10 seconds

    • Exponential processes are smoother at the 10sec scale

  • Variance-time plot

    • Change of variance to time scale

    • Var of exponential processes decays quickly

  • Packet arrival rate process in seconds

    • By the sole visual effect

    • Exponential processes are less spiky


Full telnet model

Full TELNET model?

Poisson connection arrival

Heavy-tailed packet arrival within a connection


Ftpdata

FTPDATA

  • Connection arrival is not Poisson

    • Clustered in bursts

  • Burst sizes in bytes is quite heavy-tailed

    • A 0.5% of bursts contribute to 50% of the traffic volume


Ok we know it s not poisson but what

OK. We know it’s not Poisson. But what?


Going self similar

Going Self-Similar

  • Well, since other evidences suggest so

  • And it’s the next good thing

  • Go straight into producing self-similar traffic


Producing self similar traffic

Producing Self-Similar Traffic

  • ON/OFF sources

    • Fix ON period rate

    • ON/OFF period length heavy-tailed

  • M/G/

    • Customer arrival being Poisson

    • Service time being heavy-tailed with infinite variance

  • Authors’ own model

    • Pseudo-self-similar

    • Not long-range dependent though


Performance implication

Performance Implication

  • Low-priority traffic starvation

    • Shall the high-priority traffic being long-range dependent (bursty)

  • Admission control based on recent traffic failing

    • ‘Congestions haven’t happened for a long while’ does not mean it won’t happen now


The real message

The Real Message

Poisson is no longer sufficient!


Identifying internet traffic1

Identifying Internet Traffic

Failure of Poisson

Self-similar Traffic

Practical Model


Self similar what

Self-Similar What?

  • On the Self-Similar Nature of Ethernet Traffic

  • Will E. Leland; Murad S. Taqqu; Walter Willinger; Daniel V. Wilson

  • IEEE/ACM Transactions on Networking, Vol. 2, No. 1, pp. 1-15, Feb. 1995


This one easier

This One Easier

  • Self-similarity in World Wide Web Traffic: Evidence and Possible Causes

  • Mark E. Crovella; Azer Bestavros

  • IEEE/ACM Transactions on Networking, Vol 5, No. 6, pp. 835-846, December 1997


Self similar process

Self-Similar Process

Serpgask Triangles


Definition

Definition

  • X: a stationary time series

  • X(m): the m-aggregates

    • Summing the time series over non-overlapping blocks of m

  • X is H-self-similar if

    • X (m) has the same distribution for all positive m


Same distribution

Same Distribution?

  • Same autocorrelation function

    • r(k) = E[(Xt - )(Xt+k - )]/2

  • r(k) ~ k-

    • k  

    • 0 <  < 1


Significance of k

Significance of k-

  • Long-range dependence

    • Just another way of characterizing the same thing

  • Power-law decay

    • Slower than exponential decay

    • Therefore traffic does not smooth up

  •  < 1

    • r(k) does not converge

    • Sum of r(k) infinite, I.e., variance infinite


Just fyi

Just FYI

  • The Hurst parameter: 1- /2


Tests for self similarity

Tests for Self-Similarity

  • Variance-time plot

    • A line with slope - > -1

  • R/S plot

    • Rescaled range grows as the number points included

    • A line with slope H an the log-log scale

  • Periodogram

    • Power spectrum to frequency

    • A line with slope  - 1 at the log-log scale

  • Whittle estimator

    • Confidence to a form

    • FGN or Fractional ARIMA


Pareto review

Pareto Review

  • Exponential

    • f(x) = ce-cx

  • Heavy-tailed

    • F(x) ~ x-c, 0 < c < 2

    • Hyperbolic

  • Pareto

    • f(x) = ckc x-c-1

    • F(x) = 1- (k/x)c

    • A line at the log-log scale of F(x) plot


In addition to the theory

In Addition to the Theory

  • A HUGE volume of Ethernet traces

  • Show consistency of being self-similar in all sorts of tests

  • Implication to traffic engineering

  • A bombshell!


Why self similar

Why Self-Similar?

  • Theory suggests

    • Fix rate ON/OFF process

    • Heavy-tailed length

  • Looking into the length

    • The ON time: transmission time

    • The OFF time: silent time


Physical cause

Physical Cause

  • Heavy-tailed transmission time

    • Heavy-tailed file sizes

    • Magic of the nature

    • E.g., book size in library


Identifying internet traffic2

Identifying Internet Traffic

Failure of Poisson

Self-similar Traffic

Practical Model


So enough math just tell me what to do

So, enough Math. Just tell me what to do!

It depends!


Cutting to the chase

Cutting to the Chase

  • The structural model

    • user level: Poisson arrival and heavy-tailed duration

    • network level: TCP closed-loop feedback control and ack clocking

    • Variability: delay and congestion

  • Let simulators track the complex behavior


Why not fgn

Why not FGN?

  • IP Traffic Dynamics: The Role of Variability and Control

  • Anja Feldmann; Anna C. Gilbert; Polly Huang; Walter Willinger

  • In the Proceeding of SIGCOMM '99, Cambridge, Massachusetts, September 1999


Remember wavelet analysis

Remember Wavelet Analysis?

  • FFT

    • Frequency decomposition

    • fj, Fourier coefficient

    • Amount of the signal in frequency j

  • WT: wavelet transform

    • Frequency (scale) and time decomposition

    • dj,k, wavelet coefficient

    • Amount of the signal in frequency j, time k


Self similarity

Self-similarity

  • Energy function

    • Ej = Σ(dj,k)2/Nj

    • Weighted average of the signal strength at scale j

  • Self-similar process

    • Ej = 2j(2H-1) C <- the magic!!

    • log2 Ej = (2H-1)j + log2C

    • linear relationship between log2 Ej andj


Shape of self similarity

?? RTT

‘Shape’ of Self-Similarity

Self-similar


Wavelet example

0 0 0 0 2 2 2 2

0 0 0 0 0 0 0 0

0 0 4 4

0 0 0 0

0 8

0 0

8

8

Wavelet Example

1

0

-1

00 00 00 00 11 11 11 11

s1

s2

s3

s4

d1

d2

d3

d4


Adding periodicity

10 00 00 00 10 00 00 00

s1

s2

s3

s4

d1

d2

d3

d4

1 0 0 0 1 0 0 0

1 0 0 0 1 0 0 0

1 0 1 0

1 0 1 0

1 1

1 1

2

0

Adding Periodicity

  • packets arrive periodically, 1 pkt/23 msec

  • coefficients cancel out at scale 4


Visualization

Adding Periodicity

Visualization

J=4


Shape of self similarity1

Yes RTT!

’Shape' of self-similarity

Self-similar


Large scale

Large Scale

  • Heavy-tailed connection duration


Medium scale

Medium Scale

  • TCP close-loop control

RTT


Tcp flow control

TCP Flow Control

Time

RTT

RTT

RTT

source

sink


Variability

Variability

  • Delay and congestion (bandwidth & load)

Simulation

Measurement


Internet traffic is weird

Internet Traffic is Weird!

  • Different properties at different time scales

    • Large scales: self-similarity

    • Medium scale: periodicity

    • Small scale: ??? (possibly multifractal)


New queuing theory

New Queuing Theory?

  • For chaotic Internet traffic

  • Only pen and paper


Network simulation and testing

NO!

  • Probably not in the near future

  • Confirmed by the experts


A few reasons

A Few Reasons

  • Not exactly self-similar (FGN - big no no)

  • ’Shape' of self-similarity changes with the network conditions

  • Don't know what self-similar processes add up to (mathematically intractable)

  • Don’t know what those strange small-scale behavior is exactly


Therefore

Therefore

  • The structural model

    • User level: Poisson arrival and heavy-tailed duration

    • Network level: TCP closed-loop feedback control and ack clocking

    • Variability: delay and congestion

  • Let simulators track the complex behavior


Questions

Questions?


On the review forms

On the Review Forms

  • Novelty

    • New idea

  • Clarity

    • The problem

  • Reality (practicality)

    • Evaluation

  • Importance, significance, relevance

    • How much impact?

    • Would things change?


Ok for beginners

OK for Beginners

  • Clarity

    • Easiest

    • Judging the writing

  • Evaluation

    • Easy

    • Judging the experiments and technical content


Challenging for the advanced

Challenging for the Advanced

  • Novelty

    • Hard

    • Need to follow/read enough papers in the area

  • Importance

    • Hardest

    • Need to have breadth and know enough development in the area


Show your freebsd installation

Show your FreeBSD installation!


  • Login