Experiences with coralcdn a five year operational view
Download
1 / 49

Experiences with CoralCDN A Five-Year Operational View - PowerPoint PPT Presentation


  • 48 Views
  • Uploaded on

Experiences with CoralCDN A Five-Year Operational View. Michael J. Freedman Princeton University www.coralcdn.org. A Cooperative, Self-Organizing CDN. Client. Resolver. 2. 5. 1. CoralCDN HTTP Proxy. CoralCDN DNS Server. CoralCDN HTTP Proxy. CoralCDN DNS Server. Coral index node.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Experiences with CoralCDN A Five-Year Operational View' - yves


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Experiences with coralcdn a five year operational view

Experiences with CoralCDNA Five-Year Operational View

Michael J. Freedman

Princeton University

www.coralcdn.org


A cooperative self organizing cdn
A Cooperative, Self-Organizing CDN

Client

Resolver

2

5

1

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

Coral index node

Coral index node

3

6

CoralCDN

4

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

Coral index node

Coral index node

Goal: To make desired content widely available

regardless of publisher’s own resources, by

organizing and utilizing any cooperative resources


Http example com path

http://example.com/path

http://example.com.nyud.net/path


Adopted by clients servers third parties

Adopted by: Clients Servers Third-parties







Daily request volume
Daily Request Volume

2M clients – 2 TB content – 20K origin domains

From 300-400 PlanetLab servers


Based on peer to peer dht
Based on peer-to-peer DHT

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

Coral index node

Coral index node

CoralCDN

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

Coral index node

Coral index node

1.Distributed sloppy hash table, or DSHT

2.Democratizing content publication with Coral


Based on peer to peer dht1
Based on peer-to-peer DHT

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

Coral index node

Coral index node

CoralCDN

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

Coral index node

Coral index node

Weakened consistency + algorithms that prevent tree saturation during lookup

Decentralized clustering for locality and hierarchical lookup

Cooperative HTTP / DNS that leverages locality


CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

Coral index node

Coral index node

CoralCDN

CoralCDN

HTTP Proxy

CoralCDN

DNS Server

CoralCDN

HTTP Proxy

Coral index node

Coral index node

Interactions with the External Environment

Virtualization Layer

Clients

Origin Domains


  • Experiences

    • Naming

    • Fault Tolerance

    • Resource management

  • Revisit CoralCDN’s design


Naming
Naming

Flexible, open API

x

Mismatch with domain-based access control policies


Coralcdn s platform as a service api
CoralCDN’s Platform-as-a-Service API

Rewrite rules in origin webservers

RewriteEngine on

RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx

RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$

RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net% {REQUEST_URI} [R,L]


Elastic provisioning
Elastic Provisioning

CoralCDN’s Platform-as-a-Service API

Rewrite rules in origin webservers

RewriteEngine on

RewriteCond %{HTTP_USER_AGENT} !^CoralWebPrx

RewriteCond %{QUERY_STRING} !(^|&)coral-no-serve$

RewriteCond %{HTTP_REFERER} slashdot\.org [NC]

RewriteCond %{HTTP_REFERER} digg\.com [NC,OR]

RewriteCond %{HTTP_REFERER} blogspot\.com [NC,OR]

RewriteRule ^(.*)$ http://%{HTTP_HOST}.nyud.net% {REQUEST_URI} [R,L]

  • Sites integrate with load/bandwidth monitoring


Naming conflation
Naming Conflation

http://domain/path

.service1

.service2

x

x

Location to retrieve content

Human-readable name for administrative entity

Security policies to govern objects’ interactions


Domain based security policies
Domain-based Security Policies

Web Page

evil.com

target.com

Cookies

Document Object Model


Domain based security policies1
Domain-based Security Policies

Web Page

evil.com

.nyud.net

target.com

.nyud.net

Cookies

Document Object Model

Defaults violate least privilege


Fault tolerance failure decoupling
Fault Tolerance: Failure Decoupling

  • Internal failures:

  • DHT nodes

  • DNS servers, HTTP proxies

  • Management service

x

  • External failures:

  • Interactions with origin sites

  • - Node failures

  • - DHT / DHS / HTTP level

  • - Management layer

  • Couldn't decouple IPs from host


Happens
happens!

Origin Status

CoralCDN Reaction

Cache negative results

Serve stale content

Use whole-file overwrites

  • Unresponsive

  • Returns error code

  • Reply truncated


Happens1
happens!

Origin Status

CoralCDN Reaction

Cache negative results

Serve stale content

Use whole-file overwrites

  • Unresponsive

  • Returns error code

  • Reply truncated

Maintain status quo unless improvements are possible


What is failure
What is “failure”?

Return values should have fail-safe defaults


Resource management
Resource Management

Control over bandwidth consumption

x

Control and visibility into environment’s resources

  • - Node failures

  • - DHT / DHS / HTTP level

  • - Management layer

  • Couldn't decouple IPs from host


Some timeline
Some timeline…

Mar 2004

CoralCDN

released on

PlanetLab


Some timeline1
Some timeline…

Mar 2004

CoralCDN

released on

PlanetLab

Aug 2004

Slashdotted


Some timeline2
Some timeline…

Mar 2004

CoralCDN

released on

PlanetLab

Aug 2004

Slashdotted

Dec 2004

Asian

Tsunami

PlanetLab traffic jumps

Site threatens to yank PL

PL admin kills slice

Slice restored next day

Initiates discussion of resource limits for slices


Demand supply enter fair sharing algorithms
Demand >> Supply:Enter Fair-Sharing Algorithms

?

Σi di ≤ S

Avg MB per hour (di)

Domains with heaviest consumption


Demand supply enter fair sharing algorithms1
Demand >> Supply:Enter Fair-Sharing Algorithms

find max λ, s.t.

Σimin (λ, di) ≤ S

λ

Avg MB per hour (di)

Domains with heaviest consumption


Demand supply enter fair sharing algorithms2
Demand >> Supply:Enter Fair-Sharing Algorithms

find max λ, s.t.

Σimin (λ, di) ≤ S

λ

Domains with heaviest consumption


Admission control under fair sharing
Admission Control under Fair-Sharing

~10 kB imgs

3.3% rejected

~5 MB videos

89% rejected

Demand > 10 TB Supply ≤ 2 TB


Some timeline3
Some timeline…

Mar 2004

CoralCDN

released on

PlanetLab

Aug 2004

Slashdotted

Dec 2004

Asian

Tsunami

Mar 2006

PL deploys

bandwidth

throttling

PlanetLab traffic jumps

Site threatens to yank PL

PL admin kills slice

Slice restored next day

Initiates discussion of resource limits for slices


Resource management us vs them
Resource Management: Us vs. Them

Application Hammer

Platform Hammer

Track all network traffic

If total > 80% daily rate, BW shaping in kernel

  • Track HTTP traffic

  • If site > fair share rate, reject via HTTP 403

  • If total > peak rate, close server socket


Resource management us vs them1
Resource Management: Us vs. Them

Application Hammer

Track all network traffic

If total > 80% daily rate, BW shaping in kernel

Platform Hammer

  • Track HTTP traffic

  • If site > fair share rate, reject via HTTP 403

  • If total > peak rate, close server socket

Result: HTTP traffic is 1/2 - 2/3 of all traffic

Lower layers should expose greater

visibility and control over resources


  • Experiences

    • Naming

    • Fault Tolerance

    • Resource management

  • Revisit CoralCDN’s design


Usage scenarios
Usage Scenarios

Resurrecting old content

Accessing unpopular content


Usage scenarios1
Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content


Usage scenarios2
Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content


Usage scenarios3
Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content


Usage scenarios4
Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content

Surviving flash crowds to content


Usage scenarios5
Usage Scenarios

Resurrecting old content

Accessing unpopular content

Serving long-term popular content

Surviving flash crowds to content


Usage scenarios6
Usage Scenarios

5 second epochs

10 minute epochs

Resurrecting old content

Accessing unpopular content

Serving long-term popular content

Surviving flash crowds to content


Conclusions
Conclusions?

Most requested content is long-term popular and already cached locally

“Flash” crowds occur, but on order of minutes


Conclusions1
Conclusions?

  • Focus on long-term popular

  • Little / no HTTP cooperation

  • Global discovery (e.g., DNS)

  • Focus on flash crowds

  • Regional coop. as default

  • Global coop. as failover

Most requested content is long-term popular and already cached locally

“Flash” crowds occur, but on order of minutes


Reconfiguring coralcdn s design
Reconfiguring CoralCDN’s design

Latency 90% Origin Load 5% Failover to global 0.5%

Leverage Coral hierarchy for lookup


Reconfiguring coralcdn s design1
Reconfiguring CoralCDN’s design

Σimin (λ, di) < S

λ

heavily weight history in ewma

Leverage Coral hierarchy for lookup

During admission control, bias against long-term use


Conclusions2
Conclusions

  • Experiences

    • Naming

    • Fault Tolerance

    • Resource management

  • Revisit CoralCDN’s design

    • Current design unnecessary for deployment / most use

    • Easy changes to promote flash-crowd mitigation


Can we reach internet scale
Can we reach Internet scale?

www.firecoral.net

Initial beta-release

of browser-based P2P web cache


ad