Distributed hash tables
Sponsored Links
This presentation is the property of its rightful owner.
1 / 26

Distributed Hash Tables PowerPoint PPT Presentation


  • 63 Views
  • Uploaded on
  • Presentation posted in: General

Distributed Hash Tables. David Tam Patrick Pang. Presentation Outline. What is DHT (Distributed Hash Table)? Why DHTs? Applications How lookup works? Alternatives to DHTs Performance – Routing Performance – Load Balancing Security – Routing Attack Security – Inconsistent Behaviour

Download Presentation

Distributed Hash Tables

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Distributed Hash Tables

David Tam

Patrick Pang


Presentation Outline

  • What is DHT (Distributed Hash Table)?

  • Why DHTs?

  • Applications

  • How lookup works?

  • Alternatives to DHTs

  • Performance – Routing

  • Performance – Load Balancing

  • Security – Routing Attack

  • Security – Inconsistent Behaviour

  • Comparison to Other Facilities

  • Current Research Projects

  • Conclusion


….

node

node

node

What is DHT?

Distributed application

data

get (key)

put(key, data)

Distributed hash table

  • DHT provides the information look up service for P2P applications.

  • Nodes uniformly distributed across key space

  • Nodes form an overlay network

  • Nodes maintain list of neighbours in routing table

  • Decoupled from physical network topology

(Figure adopted from Frans Kaashoek)


Why DHTs?

  • Why Middleware?

  • Simplifies the development for large-scale distributed Apps

  • Better security and robustness

  • Simple API

  • Why Do We Need DHTs?

  • Simplifies the development for large-scale distributed Apps

  • Better security and robustness

  • Simple API

  • Exploits P2P resources


Applications

  • Anything that requires a hash table

  • Databases, FSes, storage, archival

  • Web serving, caching

  • Content distribution

  • Query & indexing

  • Naming systems

  • Communication primitives

  • Chat services

  • Application-layer multi-casting

  • Event notification services

  • Publish/subscribe systems ?


How lookup works?

Example: Chord [Stoica et. al.]

0

1

15

Finger Table for Node 2

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works?

Example: Chord

0

1

15

Finger Table for Node 10

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works?

Example: Chord

0

1

15

Finger Table for Node 10

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works?

Example: Chord

0

1

15

Finger Table for Node 14

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works?

Example: Chord

0

1

15

Finger Table for Node 14

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works?

Example: Chord

0

1

15

2

14

3

Now Node 2 can retrive information for key 0 from Node 1.

4

12

5

11

10

6

7

9

8


N1

N2

N4

N3

Start

Target

N10

N6

N8

N7

N9

Server

N1

Client

Client

N2

Internet

N4

N3

Start

Target

DB

N10

Client

N6

N8

Client

N7

N9

Server

Alternatives to DHTs

  • Distributed file system

  • Centralized lookup

  • P2P flooding queries

(Figures adopted from Frans Kaashoek)


Performance -- Lookup

  • Purpose -- to locate a target node

  • Each step, try to get closer to locating target node

    • Ask a closer neighbour

  • Performance & scalability tied directly to lookup algorithm

  • 2 Aspects to Performance

  • Path latency

  • Lookup path length (# hops)

  • 2 Aspects to Scalability

  • size of routing table – O(log N)

  • lookup path length – O(log N)

  • 3 Techniques

  • proximity lookup

  • proximity neighbour selection

  • geographic layout


Performance -- Load Balancing

  • Issues

  • Hot-spots

    • Content

    • Lookup

  • Heterogeneous nodes & paths

  • System flux

  • Solution

  • Replication is the key

    • Also good for fault-tolerance

  • Cache lookup answers backwards along path


Security – Incorrect Lookup (1)

  • When asked for the “next hop”, give a wrong answer

0

Finger Table for Node 2

1

15

2

14

3

13

4

12

5

11

Node 2 to Node 10: Please tell me how to reach key 0 ….

10

6

7

9

8


Security – Incorrect Lookup (2)

  • When asked for the “next hop”, give a wrong answer

0

Finger Table for Node 10

1

15

2

14

3

13

4

12

5

11

Node 2 to Node 10: Please tell me how to reach key 0 ….

Node 10 answers: ask Node 14

10

6

7

9

8


Security – Incorrect Lookup (3)

  • When asked for the “next hop”, give a wrong answer

0

Finger Table for Node 14

1

15

2

14

3

13

4

12

5

11

Node 2 to Node 14: Please tell me how to reach key 0 ….

Node 14 answers: ask Node 10

10

6

7

9

8


Security – Incorrect Lookup (4)

  • Solution [Sit and Morris]:

    • “Define verifiable system invariant”

    • “Allow the querier to observe lookup progress”

  • Our idea how this can be implemented:

    • Concretely, using an integral monotonically decreasing quantity to implement the idea of “progress”.

    • The concept of “monotonically decreasing quantity” has been used in program construction guaranteeing total correctness. [Parnas]


Security – Inconsistent Behaviour

  • Inconsistent Behaviour, i.e., lie intelligibly

  • Sybil attack [Kaashoek]

Solution 1: public key solution


Security – Inconsistent Behaviour

  • Inconsistent Behaviour, i.e., lie intelligibly

  • Sybil attack [Kaashoek]

Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem:

How to find out the traitors among the Generals? [Lamport]


Security – Inconsistent Behaviour

Commander

attack

attack

“he said ‘retreat’”

Lieutenant 1

Lieutenant 2

  • Inconsistent Behaviour, i.e., lie intelligibly

  • Sybil attack [Kaashoek]

Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem:

How to find out the traitors among the Generals? [Lamport]


Security – Inconsistent Behaviour

Commander

attack

retreat

“he said ‘retreat’”

Lieutenant 1

Lieutenant 2

  • Inconsistent Behaviour, i.e., lie intelligibly

  • Sybil attack [Kaashoek]

Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem:

How to find out the traitors among the Generals? [Lamport]


Comparison to Other Facilities


Research Projects

Iris – security & fault-tolerance – US Gov’t

Chord – circular key space

Pastry – circular key space

Tapestry – hypercube space

CAN – n-dimensional key space

Kelips – n-dimensional key space

DDS-- middleware platform for internet service construction

-- cluster-based

-- incremental scalability


Summary

  • Good middleware platform

  • Exploits P2P networks

  • An exciting new research area


References

  • Lamport, Leslie et. al. The Byzantine Generals Problem

  • Sit, Emil, Morris, Robert. Security Considerations for Peer-to-Peer Distributed Hash Tables

  • Kaashoek, Frans. Distributed Hash Tables – Building large-sacle, robust distributed applications

  • Stoica, Ion et. al. Chord: A scalable peer-to-peer lookup service for Internet applications

  • Parnas, D. L. Connecting Theory to Practice: Software Engineering Programme


  • Login