Cataclysm policing extreme overloads in internet applications
Download
1 / 26

Cataclysm: Policing Extreme Overloads in Internet Applications - PowerPoint PPT Presentation


  • 108 Views
  • Uploaded on

Cataclysm: Policing Extreme Overloads in Internet Applications. Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts. Motivation. Internet applications used in a variety of domains Online banking, online brokerage, online music store, e-commerce

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Cataclysm: Policing Extreme Overloads in Internet Applications' - sherry


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Cataclysm policing extreme overloads in internet applications

Cataclysm: Policing Extreme Overloads in Internet Applications

Bhuvan Urgaonkar and Prashant Shenoy

University of Massachusetts


Motivation
Motivation Applications

  • Internet applications used in a variety of domains

    • Online banking, online brokerage, online music store, e-commerce

  • Internet usage continues to grow rapidly

    • Broadband deployment is accelerating

  • Outages of Internet applications more common

“Site not responding”

“connection timed out”


Internet application outages
Internet Application Outages Applications

Holiday Shopping Season 2000:

Down for 30 minutes

Periodic outages over 4 days

Average download time ~ 260 sec

9/11: site inaccessible for brief periods

Cause: Too many users leading to overload


Internet data centers
Internet Data Centers Applications

  • Internet applications run on data centers

    • Server farms

      • Provide computational and storage resources

  • Applications share data center resources

  • Problem: How can the platform handle extreme overloads seen by applications?


Handling extreme overloads
Handling Extreme Overloads Applications

  • Existing work is based on three approaches

    • Request policing [Kanodia00, Li00, Verma03, Welsh03, …]

    • Dynamic capacity provisioning [Chase01, Ranjan04]

    • Degrade performance of admitted requests [Abdelzaher99]

  • Shortcomings of existing work:

    • Does not attempt to integrate these three approaches

    • Does not address scalability of the policer!

      • The policer itself may become the bottleneck during overloads


Our contribution cataclysm
Our Contribution: Cataclysm Applications

  • Comprehensive approach

    • Novel policer that can scale during overloads

    • Dynamic provisioning for both application and policer

    • SLA-based performance adaptation

  • Implementation and evaluation on a Linux cluster

  • Focus of this talk: design of the policer


Talk outline
Talk Outline Applications

  • Motivation

  • Internet data center model

  • Request policing

  • Cataclysm Server Platform

  • Experimental results

  • Summary


Data center model
Data Center Model Applications

Retail

Web site

streaming

  • Dedicated hosting: each application runs on a subset of servers in the data center

    • Subsets are mutually exclusive: no server sharing

    • Data center hosts multiple applications

  • Free server pool: unused servers


Internet application model
Internet Application Model Applications

load

balancing

sentry

  • Internet applications replicated on multiple servers

    • E.g., clustered HTTP

  • Each application employs a sentry

    • Load balancing and request policing

  • One or more request classes

  • Service-level agreement

    • Specifies certain guaranteed request admission rate per class

    • Specifies allowed degradation in response time with arrival rate

requests

dropped

requests

http


Talk outline1
Talk Outline Applications

  • Motivation

  • Internet data center model

  • Request policing

  • Cataclysm Server Platform

  • Experimental results

  • Summary


Policer design goals
Policer: Design Goals Applications

  • Class-based differentiation

    • Each class should sustain its guaranteed admission rate

  • Revenue maximization

    • Challenging due to online nature of the problem

      • An admitted request may cause a more important request arriving later to be dropped

    • Approach: Preferential admission to higher class requests

  • Scalability

    • The policer should remain operational even under extremely high arrival rates


Overview of policer design
Overview of Policer Design Applications

Admission

control

dgold

  • Cataclysm policer has three components

    • Request classifier and per-class leaky buckets

    • Class-specific queues

    • Admission control

Class gold

admitted

dsilver

Class silver

Classifier

dropped

dbronze

Class bronze

Leaky buckets

Class-specific queues


Class based differentiation
Class-based Differentiation Applications

Admission

control

dgold

Class gold

admitted

dsilver

Class silver

Classifier

dropped

dbronze

Class bronze

Leaky buckets

Class-specific queues

  • Each incoming request undergoes classification

  • Per-class leaky buckets used to ensure that rates guaranteed in SLA are admitted


Revenue maximization
Revenue Maximization Applications

Admission

control

dgold

Class gold

admitted

dsilver

Class silver

Classifier

dropped

dbronze

Class bronze

Leaky buckets

Class-specific queues

  • Idea: Add different delays in processing of requests of different classes

  • More important requests processed more frequently

  • Methodology to compute delay values in online manner

    • Bounds probability of a request denying admission to a more important request


Admission control
Admission Control Applications

Admission

control

dgold

Class gold

admitted

dsilver

Class silver

Classifier

dropped

dbronze

Class bronze

Leaky buckets

Class-specific queues

  • Goal: Ensure that an admitted request meets its response time target

    • Measurement-based admission control algorithm

    • Use information about current load on servers and estimated size of new request to make decision


Scalability of admission control
Scalability of Admission Control Applications

  • Idea #1: Reduce the per-request admission control cost

  • Admission control on every request may be expensive

    • Bursty arrivals during overloads => batches get formed

    • Delays for class-based differentiation => batches get formed

    • Admission control test that operates on batches instead of requests

  • Idea #2: Sacrifice accuracy for computational overhead

  • When batch-based processing becomes prohibitive

    • Threshold-based scheme

      • E.g., Admit all Gold requests, drop all Silver and Bronze requests

      • Thresholds chosen based on observed arrival rates and service times

      • Extremely efficient

      • Wrong threshold => bad response times or fewer requests admitted


Scaling even further
Scaling Even Further … Applications

  • Protocol processing overheads will saturate sentry resources at extremely high arrival rates

    • Indiscriminate dropping of requests will occur

      • Important requests may be turned away without even undergoing the admission control test

      • Loss in revenue!

    • Sentry should still be able to process each arriving request!

  • Idea: Dynamic capacity provisioning for sentry

    • Pull in an additional sentry if CPU utilization of existing sentries exceeds a threshold (e.g., 90%)

    • Round-robin DNS to load balance among sentries


Talk outline2
Talk Outline Applications

  • Motivation

  • Internet data center model

  • Request policing

  • Cataclysm Server Platform

  • Experimental results

  • Summary


Cataclysm server platform
Cataclysm Server Platform Applications

  • Prototype data center

    • 20 Pentium servers

    • Gigabit switches

    • Linux-based platform

  • Sentry implemented in Layer-7 switch

    • Linux module ktcpvs

  • Replicated Web server applications using Apache

    • Dynamic content using PHP


Class based differentiation1

Fraction admitted Applications

1

0.8

Fraction admitted

Gold

0.6

Silver

0.4

Bronze

0.2

0

0

100

200

300

400

500

Time (sec)

Class-based Differentiation

  • Three classes of requests: Gold, Silver, Bronze

  • Policer successful in providing preferential admission to important requests


Threshold based higher scalability
Threshold-based: Higher Scalability Applications

  • Threshold-based processing allows the policer to handle upto 4 times higher arrival rate

    • Single sentry can handle about 19000 req/s


Threshold based loss of accuracy
Threshold-based: Loss of Accuracy Applications

  • Higher scalability comes at a loss in accuracy of admission control

  • Occasional violations of response time targets


Sentry provisioning

Arrical rate Applications

50000

40000

30000

Total arrival

Arrival rate (req/s)

Arival at sentry 1

20000

10000

0

0

100

200

300

400

500

600

Time (sec)

Sentry Provisioning


Summary
Summary Applications

  • Cataclysm: a comprehensive overload management technique consisting of

    • Request policing

    • Dynamic capacity provisioning

    • SLA-based performance adaptation

  • Cataclysm achieves the following

    • Class-based differentiation

    • Revenue maximization

    • Ability to scale to extreme overloads

  • More information: http://lass.cs.umass.edu




ad