non transitive connectivity and dhts n.
Download
Skip this Video
Download Presentation
Non-Transitive Connectivity and DHTs

Loading in 2 Seconds...

play fullscreen
1 / 25

Non-Transitive Connectivity and DHTs - PowerPoint PPT Presentation


  • 73 Views
  • Uploaded on

Non-Transitive Connectivity and DHTs. Mike Freedman Karthik Lakshminarayanan Sean Rhea Ion Stoica WORLDS 2005. Distributed Hash Tables…. k. System assigns keys to nodes All nodes agree on assignment Chord assigns keys as integers modulo 2 160 Assigns keys via successor relationship

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Non-Transitive Connectivity and DHTs' - kevyn-maxwell


Download Now An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
non transitive connectivity and dhts

Non-Transitive Connectivity and DHTs

Mike Freedman

Karthik Lakshminarayanan

Sean Rhea

Ion Stoica

WORLDS 2005

distributed hash tables
Distributed Hash Tables…

k

  • System assigns keys to nodes
  • All nodes agree on assignment
  • Chord assigns keys as integers modulo 2160
  • Assigns keys via successor relationship
  • Each node must know predecessor

R

distributed hash tables1
Distributed Hash Tables…

k

  • Used to store and retrieve (key, value) pairs
  • Any node can discover key’s successor, yet without full knowledge of network
    • Implies some form of routing

R

distributed hash tables2
Distributed Hash Tables…
  • All have implicit assumption: full connectivity
distributed hash tables3
Distributed Hash Tables…
  • All have implicit assumption: full connectivity
  • Non-transitive connectivity (NTC)not uncommon

B ↔ C , C ↔ A , A ↔ B

  • A thinks C is its successor!

X

A

B

C

k

does non transitivity exist
Does non-transitivity exist?
  • Gerding/Stribling PlanetLab study
    • 9% of all node triples exhibit NTC
    • Attributed high extent to Internet-2
  • Yet NTC is also transient
    • One 3 hour PlanetLab all-pair-pings trace
    • 2.9% have persistent NTC
    • 2.3% have intermittent NTC
    • 1.3% fail only for a single 15-minute snapshot
  • Level3 ↔ Cogent, but Level3 ↔ X ↔ Cogent
  • NTC motivates RON, Detour, and SOSR!
our contributions
Our contributions
  • We have built and run Bamboo (OpenDHT), Chord (i3), Kademlia (Coral) for > 1 year
  • Vanilla DHT algorithms break under NTC
  • Identify four main algorithmic problems and present our solutions
our goals
Our goals
  • Short-term
    • Inform other developers about NTC solutions
    • Important: DHTs are being widely deployed in Overnet, Morpheus, and BitTorrent
  • Long-term
    • Encourage new designs to directly handle NTC
    • (This topic is far from solved)
dhts 101 routing

A

B

k

R

S

DHTs 101: Routing
  • Key space defines an identifier distance
  • Routing ideally proceeds by halving distance to destination per overlay hop

Iterative

dhts 101 routing1

A

B

k

R

S

A

B

k

R

S

DHTs 101: Routing

Iterative

Recursive

dhts 101 routing tables
DHTs 101: Routing tables
  • successors / leaf set: ensure correctness
  • fingers / routing table: efficient routing
    • O ( log (n) ) hops, generally

k

R

problems we identify
Problems we identify
  • Invisible nodes
  • Routing loops
  • Broken return paths
  • Inconsistent roots
ntc problem fundamental
NTC problem fundamental?

S

A

B

C

R

Traditional routing

ntc problem fundamental1
NTC problem fundamental?

S

A

B

C

R

  • DHTs implement greedy routing for scalability
  • Sender might not use path, even though exists: finds local minima when id-distance routing

Greedy routing

Traditional routing

problems we identify1
Problems we identify
  • Invisible nodes
  • Routing loops
  • Broken return paths
  • Inconsistent roots

(First discuss how problems apply to iterative routing, then consider recursive routing.)

iterative routing invisible nodes
Iterative routing: Invisible nodes

B

C

k

A

R

X

S

  • Invisible nodes cause lookup to halt
iterative routing invisible nodes1
Iterative routing: Invisible nodes

X

B

C

D

k

A

R

X

S

  • Invisible nodes cause lookup to halt
  • Enable lookup to continue
    • Tighter timeouts via network coordinates
    • Lookup RPCs in parallel
    • Unreachable node cache
routing table pollution
Routing table pollution

B

C

k

A

R

S

  • Many proposals for maintaining routing tables
    • E.g., replace nodes with larger RTT
  • Must first prevent routing table pollution
    • Only add new nodes upon contacting directly
    • Do not immediately remove nodes from hearsay
inconsistent roots
Inconsistent roots
  • Nodes do not agree where key is assigned: inconsistent views of root
    • Can be caused by membership changes
    • Also due to non-transitive connectivity
      • May persist indefinitely

k

R

S’

?

X

S

R’

inconsistent roots1
Inconsistent roots
  • No solution when network partitions
  • If non-transitivity is limited:
    • Consensus among leaf set?
      • [Etna, Rosebud]
      • Expensive in messages and bandwidth
    • Link-state routing among leaf set?
      • [Pastry 1.4.1]
  • Can use application-level solutions!
inconsistent roots2
Inconsistent roots
  • Root replicates (key,value) among leaf set
    • Leafs periodically synchronize
    • Get gathers results from multiple leafs
    • [OpenDHT, DHash]
  • Not applicable when require fast update (i3)

k

R

X

S

R’

M

N

recursive routing
Recursive routing
  • Invisible nodes
    • Must also prevent routing table pollution
    • Easier to achieve accurate timeouts
    • Harder to perform concurrent RPCs
  • Inconsistent Roots
    • Similar solutions
  • (Routing Loops)
  • One new problem…
broken return paths
Broken return paths
  • Direct path back from R to S fails
    • Source-route reverse path
    • Use single intermediate hop
      • RON, Detour, SOSR…

k

R

X

S

T

summary
Summary
  • Non-transitive connectivity exists
    • DHTs must deal with it
  • Discovered problems the “hard way”
    • OpenDHT / Bamboo, i3 / Chord, Coral / Kademlia
    • Presented our “from the trenches” fixes
  • NTC should be considered during design phase
thanks

Thanks…

Watch Our Real, Large Distributed Systems…

coralcdn.org

opendht.org

i3.cs.berkeley.edu