Experiences with coralcdn a five year operational view
This presentation is the property of its rightful owner.
Sponsored Links
1 / 49

Experiences with CoralCDN A Five-Year Operational View PowerPoint PPT Presentation


  • 31 Views
  • Uploaded on
  • Presentation posted in: General

Experiences with CoralCDN A Five-Year Operational View. Michael J. Freedman Princeton University www.coralcdn.org. A Cooperative, Self-Organizing CDN. Client. Resolver. 2. 5. 1. CoralCDN HTTP Proxy. CoralCDN DNS Server. CoralCDN HTTP Proxy. CoralCDN DNS Server. Coral index node.

Download Presentation

Experiences with CoralCDN A Five-Year Operational View

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Experiences with coralcdn a five year operational view

Experiences with CoralCDNA Five-Year Operational View

Michael J. Freedman

Princeton University

www.coralcdn.org


A cooperative self organizing cdn

A Cooperative, Self-Organizing CDN

Client

Resolver

2

5

1

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

Coral index node

Coral index node

3

6

CoralCDN

4

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

Coral index node

Coral index node

Goal: To make desired content widely available

regardless of publisher’s own resources, by

organizing and utilizing any cooperative resources


Http example com path

http://example.com/path

http://example.com.nyud.net/path


Adopted by clients servers third parties

Adopted by:ClientsServersThird-parties


Many of you have used coralcdn

Many of you have used CoralCDN


Many of you have used coralcdn1

Many of you have used CoralCDN


Many of you have used coralcdn2

Many of you have used CoralCDN


Many of you have used coralcdn3

Many of you have used CoralCDN


Many of you have used coralcdn4

Many of you have used CoralCDN


Daily request volume

Daily Request Volume

2M clients – 2 TB content – 20K origin domains

From 300-400 PlanetLab servers


Based on peer to peer dht

Based on peer-to-peer DHT

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

Coral index node

Coral index node

CoralCDN

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

Coral index node

Coral index node

1.Distributed sloppy hash table, or DSHT

2.Democratizing content publication with Coral


Based on peer to peer dht1

Based on peer-to-peer DHT

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

Coral index node

Coral index node

CoralCDN

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

Coral index node

Coral index node

Weakened consistency + algorithms that prevent tree saturation during lookup

Decentralized clustering for locality and hierarchical lookup

Cooperative HTTP / DNS that leverages locality


Experiences with coralcdn a five year operational view

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

Coral index node

Coral index node

CoralCDN

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

Coral index node

Coral index node

Interactions with the External Environment

Virtualization Layer

Clients

Origin Domains


Experiences with coralcdn a five year operational view

  • Experiences

    • Naming

    • Fault Tolerance

    • Resource management

  • Revisit CoralCDN’s design


Naming

Naming

Flexible, open API

x

Mismatch with domain-based access control policies


Coralcdn s platform as a service api

CoralCDN’s Platform-as-a-Service API

Rewrite rules in origin webservers

RewriteEngine on

RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx

RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$

RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net%{REQUEST_URI} [R,L]


Elastic provisioning

Elastic Provisioning

CoralCDN’s Platform-as-a-Service API

Rewrite rules in origin webservers

RewriteEngine on

RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx

RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$

RewriteCond %{HTTP_REFERER} slashdot\.org [NC]

RewriteCond %{HTTP_REFERER} digg\.com [NC,OR]

RewriteCond %{HTTP_REFERER} blogspot\.com [NC,OR]

RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net%{REQUEST_URI} [R,L]

  • Sites integrate with load/bandwidth monitoring


Naming conflation

Naming Conflation

http://domain/path

.service1

.service2

x

x

Location to retrieve content

Human-readable name for administrative entity

Security policies to govern objects’ interactions


Domain based security policies

Domain-based Security Policies

Web Page

evil.com

target.com

Cookies

Document Object Model


Domain based security policies1

Domain-based Security Policies

Web Page

evil.com

.nyud.net

target.com

.nyud.net

Cookies

Document Object Model

Defaults violate least privilege


Fault tolerance failure decoupling

Fault Tolerance: Failure Decoupling

  • Internal failures:

  • DHT nodes

  • DNS servers, HTTP proxies

  • Management service

x

  • External failures:

  • Interactions with origin sites

  • - Node failures

  • - DHT / DHS / HTTP level

  • - Management layer

  • Couldn't decouple IPs from host


Happens

happens!

Origin Status

CoralCDN Reaction

Cache negative results

Serve stale content

Use whole-file overwrites

  • Unresponsive

  • Returns error code

  • Reply truncated


Happens1

happens!

Origin Status

CoralCDN Reaction

Cache negative results

Serve stale content

Use whole-file overwrites

  • Unresponsive

  • Returns error code

  • Reply truncated

Maintain status quo unless improvements are possible


What is failure

What is “failure”?

Return values should have fail-safe defaults


Resource management

Resource Management

Control over bandwidth consumption

x

Control and visibility into environment’s resources

  • - Node failures

  • - DHT / DHS / HTTP level

  • - Management layer

  • Couldn't decouple IPs from host


Some timeline

Some timeline…

Mar 2004

CoralCDN

released on

PlanetLab


Some timeline1

Some timeline…

Mar 2004

CoralCDN

released on

PlanetLab

Aug 2004

Slashdotted


Some timeline2

Some timeline…

Mar 2004

CoralCDN

released on

PlanetLab

Aug 2004

Slashdotted

Dec 2004

Asian

Tsunami

PlanetLab traffic jumps

Site threatens to yank PL

PL admin kills slice

Slice restored next day

Initiates discussion of resource limits for slices


Demand supply enter fair sharing algorithms

Demand >> Supply:Enter Fair-Sharing Algorithms

?

Σi di ≤ S

Avg MB per hour (di)

Domains with heaviest consumption


Demand supply enter fair sharing algorithms1

Demand >> Supply:Enter Fair-Sharing Algorithms

find max λ, s.t.

Σimin (λ, di) ≤ S

λ

Avg MB per hour (di)

Domains with heaviest consumption


Demand supply enter fair sharing algorithms2

Demand >> Supply:Enter Fair-Sharing Algorithms

find max λ, s.t.

Σimin (λ, di) ≤ S

λ

Domains with heaviest consumption


Admission control under fair sharing

Admission Control under Fair-Sharing

~10 kB imgs

3.3% rejected

~5 MB videos

89% rejected

Demand > 10 TB Supply ≤ 2 TB


Some timeline3

Some timeline…

Mar 2004

CoralCDN

released on

PlanetLab

Aug 2004

Slashdotted

Dec 2004

Asian

Tsunami

Mar 2006

PL deploys

bandwidth

throttling

PlanetLab traffic jumps

Site threatens to yank PL

PL admin kills slice

Slice restored next day

Initiates discussion of resource limits for slices


Resource management us vs them

Resource Management: Us vs. Them

Application Hammer

Platform Hammer

Track all network traffic

If total > 80% daily rate, BW shaping in kernel

  • Track HTTP traffic

  • If site > fair share rate, reject via HTTP 403

  • If total > peak rate, close server socket


Resource management us vs them1

Resource Management: Us vs. Them

Application Hammer

Track all network traffic

If total > 80% daily rate, BW shaping in kernel

Platform Hammer

  • Track HTTP traffic

  • If site > fair share rate, reject via HTTP 403

  • If total > peak rate, close server socket

Result: HTTP traffic is 1/2 - 2/3 of all traffic

Lower layers should expose greater

visibility and control over resources


Experiences with coralcdn a five year operational view

  • Experiences

    • Naming

    • Fault Tolerance

    • Resource management

  • Revisit CoralCDN’s design


Usage scenarios

Usage Scenarios

Resurrecting old content

Accessing unpopular content


Usage scenarios1

Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content


Usage scenarios2

Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content


Usage scenarios3

Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content


Usage scenarios4

Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content

Surviving flash crowds to content


Usage scenarios5

Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content

Surviving flash crowds to content


Usage scenarios6

Usage Scenarios

5 second epochs

10 minute epochs

Resurrecting old content

Accessing unpopular content

Serving long-term popular content

Surviving flash crowds to content


Conclusions

Conclusions?

Most requested content is long-term popular and already cached locally

“Flash” crowds occur, but on order of minutes


Conclusions1

Conclusions?

  • Focus on long-term popular

  • Little / no HTTP cooperation

  • Global discovery (e.g., DNS)

  • Focus on flash crowds

  • Regional coop. as default

  • Global coop. as failover

Most requested content is long-term popular and already cached locally

“Flash” crowds occur, but on order of minutes


Reconfiguring coralcdn s design

Reconfiguring CoralCDN’s design

Latency 90% Origin Load 5% Failover to global 0.5%

Leverage Coral hierarchy for lookup


Reconfiguring coralcdn s design1

Reconfiguring CoralCDN’s design

Σimin (λ, di) < S

λ

heavily weight history in ewma

Leverage Coral hierarchy for lookup

During admission control, bias against long-term use


Conclusions2

Conclusions

  • Experiences

    • Naming

    • Fault Tolerance

    • Resource management

  • Revisit CoralCDN’s design

    • Current design unnecessary for deployment / most use

    • Easy changes to promote flash-crowd mitigation


Can we reach internet scale

Can we reach Internet scale?

www.firecoral.net

Initial beta-release

of browser-based P2P web cache


  • Login