Cataclysm policing extreme overloads in internet applications
This presentation is the property of its rightful owner.
Sponsored Links
1 / 26

Cataclysm: Policing Extreme Overloads in Internet Applications PowerPoint PPT Presentation


  • 90 Views
  • Uploaded on
  • Presentation posted in: General

Cataclysm: Policing Extreme Overloads in Internet Applications. Bhuvan Urgaonkar and Prashant Shenoy University of Massachusetts. Motivation. Internet applications used in a variety of domains Online banking, online brokerage, online music store, e-commerce

Download Presentation

Cataclysm: Policing Extreme Overloads in Internet Applications

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Cataclysm policing extreme overloads in internet applications

Cataclysm: Policing Extreme Overloads in Internet Applications

Bhuvan Urgaonkar and Prashant Shenoy

University of Massachusetts


Motivation

Motivation

  • Internet applications used in a variety of domains

    • Online banking, online brokerage, online music store, e-commerce

  • Internet usage continues to grow rapidly

    • Broadband deployment is accelerating

  • Outages of Internet applications more common

“Site not responding”

“connection timed out”


Internet application outages

Internet Application Outages

Holiday Shopping Season 2000:

Down for 30 minutes

Periodic outages over 4 days

Average download time ~ 260 sec

9/11: site inaccessible for brief periods

Cause: Too many users leading to overload


Internet data centers

Internet Data Centers

  • Internet applications run on data centers

    • Server farms

      • Provide computational and storage resources

  • Applications share data center resources

  • Problem: How can the platform handle extreme overloads seen by applications?


Handling extreme overloads

Handling Extreme Overloads

  • Existing work is based on three approaches

    • Request policing [Kanodia00, Li00, Verma03, Welsh03, …]

    • Dynamic capacity provisioning [Chase01, Ranjan04]

    • Degrade performance of admitted requests [Abdelzaher99]

  • Shortcomings of existing work:

    • Does not attempt to integrate these three approaches

    • Does not address scalability of the policer!

      • The policer itself may become the bottleneck during overloads


Our contribution cataclysm

Our Contribution: Cataclysm

  • Comprehensive approach

    • Novel policer that can scale during overloads

    • Dynamic provisioning for both application and policer

    • SLA-based performance adaptation

  • Implementation and evaluation on a Linux cluster

  • Focus of this talk: design of the policer


Talk outline

Talk Outline

  • Motivation

  • Internet data center model

  • Request policing

  • Cataclysm Server Platform

  • Experimental results

  • Summary


Data center model

Data Center Model

Retail

Web site

streaming

  • Dedicated hosting: each application runs on a subset of servers in the data center

    • Subsets are mutually exclusive: no server sharing

    • Data center hosts multiple applications

  • Free server pool: unused servers


Internet application model

Internet Application Model

load

balancing

sentry

  • Internet applications replicated on multiple servers

    • E.g., clustered HTTP

  • Each application employs a sentry

    • Load balancing and request policing

  • One or more request classes

  • Service-level agreement

    • Specifies certain guaranteed request admission rate per class

    • Specifies allowed degradation in response time with arrival rate

requests

dropped

requests

http


Talk outline1

Talk Outline

  • Motivation

  • Internet data center model

  • Request policing

  • Cataclysm Server Platform

  • Experimental results

  • Summary


Policer design goals

Policer: Design Goals

  • Class-based differentiation

    • Each class should sustain its guaranteed admission rate

  • Revenue maximization

    • Challenging due to online nature of the problem

      • An admitted request may cause a more important request arriving later to be dropped

    • Approach: Preferential admission to higher class requests

  • Scalability

    • The policer should remain operational even under extremely high arrival rates


Overview of policer design

Overview of Policer Design

Admission

control

dgold

  • Cataclysm policer has three components

    • Request classifier and per-class leaky buckets

    • Class-specific queues

    • Admission control

Class gold

admitted

dsilver

Class silver

Classifier

dropped

dbronze

Class bronze

Leaky buckets

Class-specific queues


Class based differentiation

Class-based Differentiation

Admission

control

dgold

Class gold

admitted

dsilver

Class silver

Classifier

dropped

dbronze

Class bronze

Leaky buckets

Class-specific queues

  • Each incoming request undergoes classification

  • Per-class leaky buckets used to ensure that rates guaranteed in SLA are admitted


Revenue maximization

Revenue Maximization

Admission

control

dgold

Class gold

admitted

dsilver

Class silver

Classifier

dropped

dbronze

Class bronze

Leaky buckets

Class-specific queues

  • Idea: Add different delays in processing of requests of different classes

  • More important requests processed more frequently

  • Methodology to compute delay values in online manner

    • Bounds probability of a request denying admission to a more important request


Admission control

Admission Control

Admission

control

dgold

Class gold

admitted

dsilver

Class silver

Classifier

dropped

dbronze

Class bronze

Leaky buckets

Class-specific queues

  • Goal: Ensure that an admitted request meets its response time target

    • Measurement-based admission control algorithm

    • Use information about current load on servers and estimated size of new request to make decision


Scalability of admission control

Scalability of Admission Control

  • Idea #1: Reduce the per-request admission control cost

  • Admission control on every request may be expensive

    • Bursty arrivals during overloads => batches get formed

    • Delays for class-based differentiation => batches get formed

    • Admission control test that operates on batches instead of requests

  • Idea #2: Sacrifice accuracy for computational overhead

  • When batch-based processing becomes prohibitive

    • Threshold-based scheme

      • E.g., Admit all Gold requests, drop all Silver and Bronze requests

      • Thresholds chosen based on observed arrival rates and service times

      • Extremely efficient

      • Wrong threshold => bad response times or fewer requests admitted


Scaling even further

Scaling Even Further …

  • Protocol processing overheads will saturate sentry resources at extremely high arrival rates

    • Indiscriminate dropping of requests will occur

      • Important requests may be turned away without even undergoing the admission control test

      • Loss in revenue!

    • Sentry should still be able to process each arriving request!

  • Idea: Dynamic capacity provisioning for sentry

    • Pull in an additional sentry if CPU utilization of existing sentries exceeds a threshold (e.g., 90%)

    • Round-robin DNS to load balance among sentries


Talk outline2

Talk Outline

  • Motivation

  • Internet data center model

  • Request policing

  • Cataclysm Server Platform

  • Experimental results

  • Summary


Cataclysm server platform

Cataclysm Server Platform

  • Prototype data center

    • 20 Pentium servers

    • Gigabit switches

    • Linux-based platform

  • Sentry implemented in Layer-7 switch

    • Linux module ktcpvs

  • Replicated Web server applications using Apache

    • Dynamic content using PHP


Class based differentiation1

Fraction admitted

1

0.8

Fraction admitted

Gold

0.6

Silver

0.4

Bronze

0.2

0

0

100

200

300

400

500

Time (sec)

Class-based Differentiation

  • Three classes of requests: Gold, Silver, Bronze

  • Policer successful in providing preferential admission to important requests


Threshold based higher scalability

Threshold-based: Higher Scalability

  • Threshold-based processing allows the policer to handle upto 4 times higher arrival rate

    • Single sentry can handle about 19000 req/s


Threshold based loss of accuracy

Threshold-based: Loss of Accuracy

  • Higher scalability comes at a loss in accuracy of admission control

  • Occasional violations of response time targets


Sentry provisioning

Arrical rate

50000

40000

30000

Total arrival

Arrival rate (req/s)

Arival at sentry 1

20000

10000

0

0

100

200

300

400

500

600

Time (sec)

Sentry Provisioning


Summary

Summary

  • Cataclysm: a comprehensive overload management technique consisting of

    • Request policing

    • Dynamic capacity provisioning

    • SLA-based performance adaptation

  • Cataclysm achieves the following

    • Class-based differentiation

    • Revenue maximization

    • Ability to scale to extreme overloads

  • More information: http://lass.cs.umass.edu


Policing and provisioning

Policing and Provisioning


Policing and provisioning1

Policing and Provisioning


  • Login