skipnet a scalable overlay network with practical locality properties l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
SkipNet: A Scalable Overlay Network with Practical Locality Properties PowerPoint Presentation
Download Presentation
SkipNet: A Scalable Overlay Network with Practical Locality Properties

Loading in 2 Seconds...

play fullscreen
1 / 36

SkipNet: A Scalable Overlay Network with Practical Locality Properties - PowerPoint PPT Presentation


  • 200 Views
  • Uploaded on

SkipNet: A Scalable Overlay Network with Practical Locality Properties. Nick Harvey, Mike Jones, Stefan Saroiu , Marvin Theimer, Alec Wolman Microsoft Research University of Washington. Overlay Networks. Overlays have achieved several goals: Scalable and decentralized infrastructure

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'SkipNet: A Scalable Overlay Network with Practical Locality Properties' - Jims


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
skipnet a scalable overlay network with practical locality properties

SkipNet: A Scalable Overlay Network with Practical Locality Properties

Nick Harvey, Mike Jones,

Stefan Saroiu, Marvin Theimer, Alec Wolman

Microsoft Research

University of Washington

overlay networks
Overlay Networks
  • Overlays have achieved several goals:
    • Scalable and decentralized infrastructure
    • Uniform and random load and data distribution
  • But, at the price of data controllability
    • Data may be stored far from its users
    • Data may be stored outside its domain
    • Local accesses leave local organization
  • Basic trade-off: data controllability vs. data uniformity
  • SkipNet:
    • Traditional overlay functionality
    • Provides an abstraction to control this trade-off:
      • Constrained load balancing (CLB)
talk outline
Talk Outline
  • Practical data locality requirements
  • Basic SkipNet design
  • SkipNet locality properties
  • Performance evaluation
  • Conclusions
talk outline4
Talk Outline
  • Practical data locality requirements
  • Basic SkipNet design
  • SkipNet locality properties
  • Performance evaluation
  • Conclusions
key locality properties and abstraction
Key Locality Properties and Abstraction
  • In practice, two properties are important:
    • Content Locality – ability to explicitly place data
      • Placement on a single node or on a set of nodes
    • Path Locality – ability to guarantee that local traffic remains local
  • One abstraction is important – CLB:
    • SkipNet abstraction to control the trade-off
    • Multiple DHT scopes within one single overlay
practical requirements
Practical Requirements
  • Data Controllability:
    • Organizations want control over their own data
    • Even if local data is globally available
  • Manageability:
    • Data control allows for data administration, provisioning and manageability
    • Data center/cluster = constrained set of nodes
    • CLB ensures load balance across data center/cluster
practical requirements cont d
Practical Requirements (cont’d)
  • Security:
    • Content and path locality are key building blocks for dealing with certain external attacks
  • Data availability
    • Local data survives network partitions
  • Performance
    • Data can be stored near clients that use it
talk outline8
Talk Outline
  • Practical data locality requirements
  • Basic SkipNet design
  • SkipNet locality properties
  • Performance evaluation
  • Conclusions
skipnet
SkipNet
  • Key property: two address spaces
    • Name ID space: nodes are sorted by their names (e.g. DNS names)
    • Numeric ID space: nodes are randomly distributed
  • Combining both spaces achieves
    • Content + Path locality
    • Other uses could emerge: range queries [AS ’03]
  • Scalable peer-to-peer overlay network
    • O(log N) routing performance in both spaces
    • O(log N) routing state per node
skipnet ring

X

Z

T

A

D

V

O

M

SkipNet Ring
  • Pointers at level h skip over 2h nodes
  • Nodes are ordered by names
skipnet ring11

X

Z

T

A

D

V

O

M

SkipNet Ring
  • Pointers at level h skip over 2h nodes
  • Nodes are ordered by names
skipnet ring12

S

Z

H

A

E

M

G

F

SkipNet Ring
  • Pointers at level h skip over 2h nodes
  • Nodes are ordered by names
skipnet global view

Ring

100

Ring

101

Ring

110

Ring

111

Ring

000

Ring

001

Ring

010

Ring

011

O

M

D

A

T

Z

V

X

M

O

D

Ring 01

Ring 00

Ring 10

Ring 11

A

T

V

Z

X

M

O

D

Ring 1

Ring 0

T

A

Z

V

X

SkipNet Global View

L = 3

L = 2

L = 1

M

D

O

Root Ring

Level: L = 0

T

A

Z

V

X

skipnet global view14

Ring

100

Ring

101

Ring

110

Ring

111

Ring

000

Ring

001

Ring

010

Ring

011

M

O

D

A

T

Z

V

X

M

O

D

Ring 01

Ring 00

Ring 10

Ring 11

A

T

V

Z

X

M

O

D

Ring 1

Ring 0

T

A

Z

V

X

SkipNet Global View

L = 3

L = 2

L = 1

M

D

O

Root Ring

Level: L = 0

T

A

Z

V

X

two address spaces
Two Address Spaces
  • SkipNet can route efficiently in both address spaces:
    • Name ID space (e.g. DNS names)
    • Numeric ID space
routing by name id

Node A’s

Routing

Table

Routing by Name ID

Ring

100

Ring

101

Ring

110

Ring

111

Ring

000

Ring

001

Ring

010

Ring

011

M

O

D

L = 3

T

A

V

X

Z

M

D

Ring 01

Ring 00

Ring 10

Ring 11

A

T

L = 2

O

V

Z

X

M

O

D

Ring 1

Ring 0

T

L = 1

A

Z

V

X

M

D

O

Root Ring

T

A

Level: L = 0

Z

V

X

  • Example: route from A to V
  • Simple Rule: Forward the message to node that is closest to dest, without going too far.
routing by name id17
Routing by Name ID

Ring

100

Ring

101

Ring

110

Ring

111

Ring

000

Ring

001

Ring

010

Ring

011

M

O

D

L = 3

T

A

V

X

Z

M

D

Ring 01

Ring 00

Ring 10

Ring 11

A

T

L = 2

O

V

Z

X

M

O

D

Ring 1

Ring 0

T

L = 1

A

Z

V

X

M

D

O

Root Ring

T

A

Level: L = 0

Z

V

X

  • Example: route from A to V
  • Simple Rule: Forward the message to node that is closest to dest, without going too far.
routing by name id18

Node T’s

Routing

Table

Routing by Name ID

Ring

100

Ring

101

Ring

110

Ring

111

Ring

000

Ring

001

Ring

010

Ring

011

M

O

D

L = 3

T

A

V

X

Z

M

D

Ring 01

Ring 00

Ring 10

Ring 11

A

T

L = 2

O

V

Z

X

M

O

D

Ring 1

Ring 0

T

L = 1

A

Z

V

X

M

D

O

Root Ring

T

A

Level: L = 0

Z

V

X

  • Example: route from A to V
  • Simple Rule: Forward the message to node that is closest to dest, without going too far.
routing by name id19

Node T’s

Routing

Table

Routing by Name ID

Ring

100

Ring

101

Ring

110

Ring

111

Ring

000

Ring

001

Ring

010

Ring

011

M

O

D

L = 3

T

A

V

X

Z

M

D

Ring 01

Ring 00

Ring 10

Ring 11

A

T

L = 2

O

V

Z

X

M

O

D

Ring 1

Ring 0

T

L = 1

A

Z

V

X

M

D

O

Root Ring

T

A

Level: L = 0

Z

V

X

  • Example: route from A to V
  • Simple Rule: Forward the message to node that is closest to dest, without going too far.
routing by name id20

Node T’s

Routing

Table

Routing by Name ID

Ring

100

Ring

101

Ring

110

Ring

111

Ring

000

Ring

001

Ring

010

Ring

011

M

O

D

L = 3

T

A

V

X

Z

M

D

Ring 01

Ring 00

Ring 10

Ring 11

A

T

L = 2

O

V

Z

X

M

O

D

Ring 1

Ring 0

T

L = 1

A

Z

V

X

M

D

O

Root Ring

T

A

Level: L = 0

Z

V

X

  • Example: route from A to V
  • Simple Rule: Forward the message to node that is closest to dest, without going too far.
routing by name id21
Routing by Name ID

Ring

100

Ring

101

Ring

110

Ring

111

Ring

000

Ring

001

Ring

010

Ring

011

M

O

D

L = 3

T

A

V

X

Z

M

D

Ring 01

Ring 00

Ring 10

Ring 11

A

T

L = 2

O

V

Z

X

M

O

D

Ring 1

Ring 0

T

L = 1

A

Z

V

X

M

D

O

Root Ring

T

A

Level: L = 0

Z

V

X

  • Example: route from A to V
  • Simple Rule: Forward the message to node that is closest to dest, without going too far.
routing by numeric id
Routing by Numeric ID
  • Provides the basic DHT primitive
  • To store file “Foo.c”
    • Hash(“Foo.c”)  a random numeric ID
    • Find highest ring matching that numeric ID
    • Store file on node in that ring
  • Log N routing efficiency
dht example

Foo.c

DHT Example

Ring

100

Ring

101

Ring

110

Ring

111

Ring

000

Ring

001

Ring

010

Ring

011

  • Store file “Foo.c” from node A
    • Hash(“Foo.c”) = 101…
  • Route from A to V in numeric space

M

O

D

L = 3

T

A

V

Z

X

M

D

Ring 01

Ring 00

Ring 10

Ring 11

A

T

L = 2

O

V

Z

X

M

O

D

Ring 1

Ring 0

T

L = 1

A

Z

V

X

M

D

O

Root Ring

T

A

Level: L = 0

Z

V

X

talk outline24
Talk Outline
  • Practical data locality requirements
  • Basic SkipNet design
  • SkipNet locality properties
  • Performance evaluation
  • Conclusions
constrained load balancing clb

Numeric Routing

Name Routing

Constrained Load Balancing (CLB)
  • Multiple DHTs with differing scopes using a single SkipNet structure
  • A result of the ability to route in both address spaces
  • Divide data object names into 2 partsusing the ‘!’ special character

CLB DomainCLB Suffix

microsoft.com!skipnet.html

clb example

skipnet.

html

CLB Example
  • To read file “com.microsoft!skipnet.html”
    • Route by name ID to “com.microsoft”
    • Route by numeric ID to Hash(“skipnet.html”)within the “com.microsoft” constraint

com.microsoft

com.sun

gov.irs

edu.ucb

skipnet path locality

com.microsoft.research

SkipNet Path Locality
  • Organizations correspond to contiguous SkipNet segments
    • Internal routing by NameID remains internal
  • Nodes have left / right pointers

com.microsoft

com.sun

gov.irs

edu.ucb

fault tolerance
Fault Tolerance
  • Many failures occur along organizational boundaries:
    • Gateway/firewall failure, BGP misconfig, physical network cut, …
  • SkipNet handles organizational disconnect gracefully
    • Results in two well-connected, partitioned SkipNets
    • Efficient remerging algorithms
  • Node independent failures
    • Same resiliency as systems such as Chord and Pastry
    • Similar approach to repair (Leaf Set)
primary security benefit weakness
Primary Security Benefit & Weakness
  • SkipNet + name access control mechanism:
    • Content locality ensures that content stays within organization
    • Path locality prevents:
      • malicious forwarders
      • analysis of internal traffic
      • external tampering
  • Easier to target organizations:
    • Someone creates one million nodes with name prefixes microsofa.com and microsort.com
    • Most traffic to/from Microsoft will go through a microsofa / microsort intermediate node
talk outline30
Talk Outline
  • Practical data locality requirements
  • Basic SkipNet design
  • SkipNet locality properties
  • Performance evaluation
  • Conclusions
methodology
Methodology
  • Packet-level, event-driven simulator:
    • SkipNet implementation
      • Basic SkipNet
      • Full SkipNet = Basic SkipNet + network proximity
    • Pastry and Chord implementation
  • Uses Mercator and GT-ITM network topologies
  • Experimentally evaluated:
    • Name ID routing performance
    • Tolerance to organizational disconnect
methodology32
Methodology
  • Packet-level, event-driven simulator:
    • SkipNet implementation
      • Basic SkipNet
      • Full SkipNet = Basic SkipNet + network proximity
    • Pastry and Chord implementation
  • Uses Mercator and GT-ITM network topologies
  • Experimentally evaluated:
    • Name ID routing performance
    • Tolerance to organizational disconnect
    • Numeric ID routing performance
    • Effectiveness of network proximity optimizations
    • Effectiveness of CLB routing optimizations
routing by name id performance
Routing by Name ID Performance

Benefits come at no extra cost

surviving organizational disconnect
Surviving Organizational Disconnect

Disconnected Org Size = 15% of all nodes

conclusions
Conclusions
  • SkipNet:
    • Traditional overlay functionality
    • Explicit control of data placement
      • Constrained load balancing
  • Content + Path Locality are basic ingredients to:
    • Data controllability
    • Manageability
    • Security
    • Data availability
    • Performance
questions
Questions?

http://research.microsoft.com/sn/Herald/