Scheduling Your
This presentation is the property of its rightful owner.
Sponsored Links
1 / 35

Mor Harchol-Balter Carnegie Mellon PowerPoint PPT Presentation


  • 48 Views
  • Uploaded on
  • Presentation posted in: General

Scheduling Your Network Connections. Mor Harchol-Balter Carnegie Mellon. with Mukesh Agrawal. with Bianca Schroeder. with Nikhil Bansal. FCFS. jobs. jobs. PS. SRPT. jobs. Q: Which minimizes mean response time?. “size” = service requirement. load r < 1.

Download Presentation

Mor Harchol-Balter Carnegie Mellon

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Mor harchol balter carnegie mellon

Scheduling Your

Network Connections

Mor Harchol-Balter

Carnegie Mellon

with Mukesh

Agrawal

with Bianca

Schroeder

with Nikhil

Bansal


Mor harchol balter carnegie mellon

FCFS

jobs

jobs

PS

SRPT

jobs

Q: Which minimizes

mean response time?

“size” = service requirement

load r < 1


Mor harchol balter carnegie mellon

Q: Which best represents

scheduling in web servers ?

FCFS

jobs

“size” = service requirement

load r < 1

jobs

PS

SRPT

jobs


Mor harchol balter carnegie mellon

How about using SRPT in Web servers as opposed

to the traditional processor-sharing (PS) type

scheduling ?


Mor harchol balter carnegie mellon

Immediate Objections

1) Can’t assume known job size

Many servers receive mostly static web requests.“GET FILE”For static web requests, know file size

Approx. know service requirement of request.

2) But the big jobs will starve ...


Mor harchol balter carnegie mellon

THEORY

IMPLEMENT

Outline of Talk

1) “Analysis of SRPT Scheduling: Investigating Unfairness”

with Nikhil Bansal

2) “Implementation of SRPT Scheduling in Web Servers”

with Nikhil Bansal and Bianca Schroeder and Mukesh Agrawal


Mor harchol balter carnegie mellon

THEORY

SRPT has a long history ...

1966 Schrage & Miller derive M/G/1/SRPT response time:

1968 Schrage proves optimality

1979 Pechinkin & Solovyev & Yashkov generalize

1990 Schassberger derives distribution on queue length

BUT WHAT DOES IT ALL MEAN?


Mor harchol balter carnegie mellon

THEORY

SRPT has a long history (cont.)

1990 - 97 7-year long study at Univ. of Aachen under Schreiber

SRPT WINS BIG ON MEAN!

1998, 1999 Slowdown for SRPT under adversary:

Rajmohan, Gehrke, Muthukrishnan, Rajaraman, Shaheen,

Bender, Chakrabarti, etc.

SRPT STARVES BIG JOBS!

Various o.s. books: Silberschatz, Stallings, Tannenbaum:

Warn about starvation of big jobs ...

Kleinrock’s Conservation Law:

“Preferential treatment given to one class of customers

is afforded at the expense of other customers.”


Mor harchol balter carnegie mellon

I

SRPT

THEORY

Counter-intuitive!

Our Analytical Results (M/G/1):

All-Can-Win Theorem:

Under workloads with

heavy-tailed (HT) property,

ALL jobs, including the very

biggest, prefer SRPT to PS,

provided load not too close to 1.

Almost-All-Win-Big Theorem:

Under workloads with

HT property, 99% of all jobs

perform orders of magnitude

better under SRPT.

PS

SRPT

Many more such results in Sigmetrics talk.


Mor harchol balter carnegie mellon

THEORY

Our Analytical Results (M/G/1):

Moderate-Load Theorem:

If load < .5, for every job size distribution,

ALL jobs prefer SRPT to PS.

Bounding-the-damage Theorem:

For any load, for every job size distribution, for every size x,

æ

ö

r

ç

÷

<

+

E

[

T

(

x

)]

1

E

[

T

(

x

)]

ç

÷

SRPT

PS

2(1

-

r

)

è

ø


Mor harchol balter carnegie mellon

What’s the Heavy-Tail

property?

Defn: heavy-tailed distribution:

-

a

>

<

a

<

Pr

{

X

x

}

~

x

,

0

2

Many real-world workloads well-modeled by truncated HT distribution.

Key property: HT Property:

“Largest 1% of jobs comprise half the load.”


Mor harchol balter carnegie mellon

IMPLEMENT

From theory to practice:

What does SRPT mean within aWeb server?

  • Many devices:Where to do the scheduling?

  • No longer one job at a time.


Mor harchol balter carnegie mellon

IMPLEMENT

Previous work on

implementation:

1998 Almeida, Dabu, Manikutty, Cao:

Prioritizing HTTP requests at Web servers.

“Nice” the low-priority process.

1999 Crovella, Frangioso, Harchol-Balter:

SRPT scheduling of requests.

User-level scheduling of reads and writes.

What is the problem with both of these?


Mor harchol balter carnegie mellon

Web

Server

Network/O.S. insides of

traditional Web server

IMPLEMENT

Socket 1

Client1

Network

Card

Socket 2

Client2

BOTTLENECK

Client3

Socket 3

Sockets take turns

draining --- FAIR = PS.


Mor harchol balter carnegie mellon

IMPLEMENT

Web

Server

Network/O.S. insides of

our improved Web server

Socket 1

Client1

S

Network

Card

1st

Socket 2

Client2

2nd

M

BOTTLENECK

3rd

Client3

Socket 3

L

priority

queues.

Socket corresponding to file

with smallest remaining data

gets to feed first.


Mor harchol balter carnegie mellon

IMPLEMENT

Experimental Setup

1

1

APACHE

WEB

SERVER

CLIENTS

2

2

3

3

Direct

Connection

10Mb/s

200

200

Linux 0.S.

Linux 0.S.

  • Implement SRPT-based scheduling:

  • 1) Modifications to Linux O.S.: 6 priority Levels

  • 2) Modifications to Apache Web server

  • 3) Priority algorithm design.


Mor harchol balter carnegie mellon

IMPLEMENT

Experimental Setup

1

1

APACHE

WEB

SERVER

2

2

CLIENTS

3

3

200

200

Linux 0.S.

Linux 0.S.

  • Comparison of SRPT implementation vs. FAIR scheduling standard.

  • Comparison done under both APACHE and FLASH web servers.

  • Comparisons done under trace-based workload and under

  • Web workload generator.

  • Under trace-based workload:

  • -- Number requests made: 1,000,000

  • -- Size of file requested: 41B -- 2 MB

  • -- Distribution of file sizes requested has HT property.

  • Experiment under range of loads.


Mor harchol balter carnegie mellon

IMPLEMENT

Preliminary Comments to

Results:

1

1

APACHE

WEB

SERVER

2

2

CLIENTS

3

3

200

200

Linux 0.S.

Linux 0.S.

  • Measured job throughput, byte throughput, and bandwidth

  • utilization were same under SRPT and FAIR scheduling.

  • Same set of requests complete.

  • No additional CPU overhead under SRPT scheduling.

  • CPU utilization always 1% - 5%.

  • Network was bottleneck in all experiments.


Mor harchol balter carnegie mellon

Results: Mean Response Time

Mean Response Time (ms)

FAIR

SRPT

Load


Mor harchol balter carnegie mellon

Results: Mean Slowdown

FAIR

Mean Slowdown

SRPT

Load


Mor harchol balter carnegie mellon

Results: Mean Response Time vs. Size

Load = 0.8

Mean Response Time (ms)

FAIR

SRPT

Requested File Size (bytes)


Mor harchol balter carnegie mellon

Mean Response Time vs.

Size Percentile

Load =0.8

FAIR

Mean Response time (ms)

SRPT

Percentile of Request Size


Mor harchol balter carnegie mellon

Summary so far ...

  • SRPT scheduling yields significant improvements in Mean Response Time at the server.

  • Negligible starvation.

  • No CPU overhead.

  • No drop in throughput.


Mor harchol balter carnegie mellon

More questions …

  • This study involved a LAN.

  • Are the effects ofSRPT in a WANas strong?

  • So far we’ve only experimented with load < 1.

  • What happens under SRPT vs. FAIR when the

  • server runs under transient overload?

  • -> new analysis

  • -> implementation study


Mor harchol balter carnegie mellon

WAN setting

Question:

Will improvement of SRPT over FAIR scheduling

appear greater in a LAN setting or a WAN setting?


Mor harchol balter carnegie mellon

WAN setting

Answer:

Normally LAN setting shows more improvement.

But for very high load, WAN setting shows more

improvement. Why?

Load 0.7

Load 0.9


Mor harchol balter carnegie mellon

Zzzzzzz

zzz...

Overload

Person

under

overload


Mor harchol balter carnegie mellon

Web server under overload

When reach SYN-queue limit,

server drops all connection

requests.

SYN-queue

Clients

Server

SYN-queue

ACK-queue

Apache-processes


Mor harchol balter carnegie mellon

FAIR vs. SRPT

Question: What will happen under FAIR vs. SRPT

wrt number of connections at server and

wrt response times?

(a) Given persistent overload

(b) Given transient overload

r>1

r>1

r>1

r>1

r<1

r<1


Mor harchol balter carnegie mellon

Persistent Overload:

Time until SYN-queue limit is hit

r=1.5

FAIR

FAIR+ (tuned version)

SRPT


Mor harchol balter carnegie mellon

Persistent Overload:

Buildup in number of connections

+

FAIR+

SRPT


Mor harchol balter carnegie mellon

Transient Overload:

Buildup of Connections at Server

FAIR+

SRPT


Mor harchol balter carnegie mellon

Transient Overload:

Mean response time (in msec)

FAIR+

SRPT


Mor harchol balter carnegie mellon

Transient overload:

Response time as function of job size

FAIR+

SRPT

small jobs

win big!

big jobs

aren’t hurt!

WHY?


Mor harchol balter carnegie mellon

Conclusion

  • SRPT scheduling yields significant improvements in Mean Response Time at the server under

    LAN, WAN, under high load, and under overload.

  • Negligible or zero unfairness. Often better

    for all requests.

  • All results corroborated via implementation and analysis.


  • Login