distributed hash tables n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Distributed Hash Tables PowerPoint Presentation
Download Presentation
Distributed Hash Tables

Loading in 2 Seconds...

play fullscreen
1 / 46

Distributed Hash Tables - PowerPoint PPT Presentation


  • 149 Views
  • Uploaded on

Distributed Hash Tables. CS653, Fall 2010. Implementing insert/retrieve: distributed hash table (DHT). Hash table data structure that maps “keys” to “values” essential building block in software systems Distributed Hash Table (DHT) similar, but spread across the Internet DHT Interface:

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Distributed Hash Tables' - kaili


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
distributed hash tables

Distributed Hash Tables

CS653, Fall 2010

implementing insert retrieve distributed hash table dht
Implementing insert/retrieve: distributed hash table (DHT)
  • Hash table
    • data structure that maps “keys” to “values”
    • essential building block in software systems
  • Distributed Hash Table (DHT) similar, but spread across the Internet
  • DHT Interface:
    • insert(key, value)
    • lookup(key)
  • DHT: overlay (on top of internet) of nodes
    • each DHT node supports single operation:
    • given input key, route messages toward node holding key
dht in action

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

DHT in action
dht in action1

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

DHT in action
dht in action2

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

DHT in action

Operation: take key as input; route messages to

node holding key

dht in action insert

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

insert(K1,V1)

K V

DHT in action: insert()

Operation: take key as input; route messages to

node holding key

dht in action insert1

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

DHT in action: insert()

insert(K1,V1)

Operation: take key as input; route messages to

node holding key

dht in action insert2

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

DHT in action: insert()

(K1,V1)

Operation: take key as input; route messages to

node holding key

dht in action lookup

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

K V

DHT in action: lookup()

lookup (K1)

Operation: take key as input; route messages to

node holding key

dht design goals
DHT Design Goals
  • An “overlay” network with:
    • flexible mapping of keys to physical nodes
    • small network diameter
    • small degree
    • local routing decisions
  • A “storage” or “memory” mechanism with
    • best-effort persistence (soft state)
example dht chord
Example DHT: Chord
  • m-bit identifier space:2m ids
  • Identifiers ordered on identifier ring (Chord ring) modulo 2m
  • Each key, k, maps to node on ring
    • # key values may be > # nodes
    • key k maps to node on ring with next largest id: successor (k)

Example: 10 node Chord with m=6 storing 5 keys.

basic lookup
Basic lookup

Lookup:

  • node i forwards to node i+1, until node holding k is reached
  • memory; O(1), each node need only know one neighbor
  • routing time: O(N)
accelerating lookups
Accelerating Lookups
  • Lookups accelerated by maintaining additional routing information
  • Each node maintains a routing table with (at most) m entries (where N=2m) called finger table
  • ithentry in table at node n contains identity of first node, s, that succeeds n by at least 2i-1 on chord
  • s = successor(n + 2i-1) (all arithmetic mod 2)
routing by halving distance
Routing by halving distance
  • Node uses finger tables to halve the ID space distance to destination in each step
churn
Churn
  • Churn: nodes join, leave system
    • stored keys must be move to (on leave) or moved from (join) neighboring nodes
    • finger tables must be updated
    • too much churn: too much overhead
  • Joins: Node n joins by asking any node m for n’s successor
  • Voluntary leaves
    • Transfer keys to successor, notify predecessor
  • Failure handling: maintain r backup successors
    • r tuned to the mean-time-to-failure of nodes
dhts discussion
DHTs discussion
  • What systems can you build using DHTs?
  • How to reduce stretch? [Plaxton97]
  • How to support range requests or partial matches between request and key?
  • What real applications use DHTs today?
    • Why or why not?
  • Pros and cons of a centralized index?
indirection1
Indirection

Indirection: rather than reference an entity directly, reference it (“indirectly) via another entity, which in turn can or will access the original entity

"Every problem in computer

science can be solved by

adding another level of

indirection"

-- Butler Lampson

x

A

B

multicast one sender to many receivers
Multicast: act of sending datagram to multiple receivers with single “transmit” operation

Question: how to achieve multicast

Multicast: one sender to many receivers

Network multicast

  • router actively participate in multicast, making copies of packets as needed and forwarding towards multicast receivers

Multicast

routers (red) duplicate and

forward multicast datagrams

internet multicast service model
Internet Multicast Service Model

128.59.16.12

multicast group concept: use of indirection

  • host addresses IP datagram to multicast group
  • routers forward multicast datagrams to hosts that have “joined” that multicast group

128.119.40.186

multicast

group

226.17.30.197

128.34.108.63

128.34.108.60

multicast via indirection why
Multicast via Indirection: why?
  • Naming and forwarding in IP tailored for point-to-point communication
  • Indirection
    • Provides flexible naming
    • Decouples sender from receivers (and their joins and leaves)
how do you contact a mobile friend
How do you contact a mobilefriend?

Mobility and Indirection

  • search all phone books?
  • call her parents?
  • expect her to let you know where he/she is?

I wonder where Alice moved to?

Consider friend frequently changing addresses, how do you find her?

mobility and indirection
Mobility and indirection:
  • mobile node moves from network to network
  • correspondents want to send packets to mobile node
  • two approaches:
    • indirect routing: communication from correspondent to mobile goes through home agent, then forwarded to remote
    • direct routing: correspondent gets foreign address of mobile, sends directly to mobile
mobility vocabulary
Mobility: Vocabulary

home network: permanent “home” of mobile

(e.g., 128.119.40/24)

home agent: entity that will perform mobility functions on behalf of mobile, when mobile is remote

wide area network

permanent address: address in home network, can always be used to reach mobile

e.g., 128.119.40.186

mobility more vocabulary
Mobility: more vocabulary

visited network: network in which mobile currently resides (e.g., 79.129.13/24)

permanent address: remains constant (e.g., 128.119.40.186)

care-of-address: address in visited network.

(e.g., 79,129.13.2)

wide area network

foreign agent: entity in visited network that performs mobility functions on behalf of mobile.

correspondent: wants to communicate with mobile

mobility registration

mobile contacts foreign agent on entering visited network

foreign agent contacts home agent home: “this mobile is resident in my network”

1

2

Mobility: registration

visited network

home network

End result:

  • foreign agent knows about mobile
  • home agent knows location of mobile

wide area network

mobility via indirect routing

foreign agent receives packets, forwards to mobile

home agent intercepts packets, forwards to foreign agent

correspondent addresses packets using home address of mobile

mobile replies directly to correspondent

3

2

4

1

Mobility via Indirect Routing

visited

network

home

network

wide area network

indirect routing comments
Indirect Routing: comments
  • mobile uses two addresses:
    • permanent address: used by correspondent (hence mobile location is transparent to correspondent)
    • care-of-address: used by home agent to forward datagrams to mobile
  • foreign agent functions may be done by mobile itself
  • triangle routing: correspondent-home-network-mobile
    • inefficient when

correspondent, mobile

are in same network

indirect routing moving between networks
Indirect Routing: moving between networks
  • suppose mobile user moves to another network
    • registers with new foreign agent
    • new foreign agent registers with home agent
    • home agent updates care-of-address for mobile
    • packets continue to be forwarded to mobile (but with new care-of-address)
  • mobility, changing foreign networks transparent: ongoing connections can be maintained!
mobility via direct routing

foreign agent receives packets, forwards to mobile

mobile replies directly to correspondent

4

2

5

1

3

Mobility via Direct Routing

correspondent forwards to foreign agent

visited

network

home

network

wide area network

correspondent requests, receives foreign address of mobile

mobility via direct routing comments
Mobility via Direct Routing: comments
  • overcomes triangle routing problem
  • non-transparent to correspondent: correspondent must get care-of-address from home agent
    • what happens if mobile changes networks?
mobile ip
Mobile IP
  • RFC 3220
  • Has many features we’ve seen:
    • home agents, foreign agents, foreign-agent registration, care-of-addresses, encapsulation (packet-within-a-packet)
  • Three components to standard:
    • agent discovery
    • registration with home agent
    • indirect routing of datagrams
mobility via indirection why indirection
Mobility via indirection: why indirection?
  • Transparency to correspondent
  • “Mostly” transparent to mobile (except mobile must register with foreign agent)
    • transparent to routers, rest of infrastructure
    • practical concern: if egress filtering is in place in foreign networks (since source IP address of mobile is its home address): spoofing?
an internet indirection infrastructure
An Internet Indirection Infrastructure

Motivation:

  • Today’s Internet is built around point-to-point communication abstraction:
    • send packet “p” from host “A” to host “B”
    • one sender, one receiver, at fixed and well-known locations
  • … not appropriate for applications that require other communications primitives:
    • multicast (one to many)
    • mobility (one to anywhere)
    • anycast (one to any)
  • We’ve seen indirection used to provide these services
    • idea: make indirection a “first-class object”
internet indirection infrastructure i3

send(R, data)

send(ID, data)

trigger

ID

R

Internet Indirection Infrastructure (i3)
  • Change communication abstraction: instead of point-to-point, exchange packets by name
    • each packet has an identifier ID
    • to receive packet with identifier ID, receiver R stores trigger (ID, R) in network
    • triggers stored in network overlay nodes

Sender

Receiver (R)

service model
Service Model

packet p: (ID,data)

trigger t: (ID, addr)

  • API
    • sendPacket(p);
    • insertTrigger(t);
    • removeTrigger(t); // optional
  • Best-effort service model (like IP)
  • Triggers periodically refreshed by end-hosts
    • Q: what is this approach called?
  • Reliability, congestion control, flow-control implemented at end hosts, and trigger-storing overlay nodes
discussion
Discussion
  • Trigger is similar to routing table entry
  • Essentially: application layer publish-subscribe infrastructure
  • Unlike IP, end hosts control triggers, i.e., end hosts responsible for setting and maintaining “routing tables”
  • Provide support for
    • mobility
    • multicast
    • anycast
    • composable services
mobility

send(R1, data)

send(ID,data)

Mobility
  • Receiver updates its trigger as it moves from one subnet to another
    • mobility transparent to sender
    • location privacy

Receiver

(R1)

Sender

trigger

ID

R1

mobility1

send(ID,data)

send(R2, data)

(new) trigger

Mobility
  • Receiver updates its trigger as it moves from one subnet to another
    • mobility transparent to sender
    • location privacy

Sender

ID

R2

Receiver

(R2)

multicast
Multicast
  • Unifies multicast and unicast abstractions
    • multicast: receivers insert triggers with same ID
  • Application naturally moves between multicast and unicast, as needed
    • “impossible” in current IP model

send(R1, data)

send(ID,data)

ID

R1

Receiver (R1)

Sender

ID

R2

send(R2, data)

Receiver (R2)

anycast cont d
Anycast (cont’d)
  • Route to any one in set of receivers
  • Receiver i in anycast group inserts same ID, with anycast qualifications
  • Route to receiver with best match between a and si

send(R1,data)

Receiver (R1)

ID|s1

R1

send(ID|a,data)

ID|s2

R2

Sender

Receiver (R2)

ID|s3

R3

Receiver (R3)

composable services

send((ID_MPEG/JPEG,ID), data)

send(R, data)

send(ID, data)

Composable Services
  • Use stack of IDs to encode successive operations to be performed on data (e.g., transcoding)
  • Don’t need to configure path between services

S_MPEG/JPEG

Receiver R

(JPEG)

Sender

(MPEG)

ID

R

S_MPEG/JPEG

ID_MPEG/JPEG

composable services cont d

send(R, data)

send(ID, data)

send((ID_MPEG/JPEG, R), data)

Composable Services (cont’d)
  • Both receivers and senders can specify operations to be performed on data

S_MPEG/JPEG

Receiver R

(JPEG)

Sender

(MPEG)

S_MPEG/JPEG

ID_MPEG/JPEG

ID

(ID_MPEG/JPEG, R)

heterogeneous multicast

send(R1, data)

send(R2, data)

send(ID, data)

ID

R2

send((ID_MPEG/JPEG, R1), data)

Heterogeneous Multicast
  • Both receivers and senders can specify operations to be performed on data

S_MPEG/JPEG

Receiver R1

(JPEG)

Sender

(MPEG)

S_MPEG/JPEG

ID_MPEG/JPEG

ID

(ID_MPEG/JPEG, R1)

Receiver R2

(JPEG)

discussion of i3
Discussion of I3
  • How would receiver signal ACK to sender? what is needed?
  • Does many-to-one fit well in this paradigm?
  • Security, snooping, information gathering: what are the issues?
  • In-network storage to handle disconnection?
indirection summary
Indirection: Summary

We’ve seen indirection used in many ways:

  • multicast
  • mobility
  • Internet indirection

Uses of indirection:

  • sender does not need to know receiver id - do not want sender to know intermediary identities
  • elegant
  • transparency of indirection is important
  • performance: is it more efficient?
  • security: important open issue for I3