Distributed hash tables
This presentation is the property of its rightful owner.
Sponsored Links
1 / 26

Distributed Hash Tables PowerPoint PPT Presentation


  • 56 Views
  • Uploaded on
  • Presentation posted in: General

Distributed Hash Tables. David Tam Patrick Pang. Presentation Outline. What is DHT (Distributed Hash Table)? Why DHTs? Applications How lookup works? Alternatives to DHTs Performance – Routing Performance – Load Balancing Security – Routing Attack Security – Inconsistent Behaviour

Download Presentation

Distributed Hash Tables

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Distributed hash tables

Distributed Hash Tables

David Tam

Patrick Pang


Presentation outline

Presentation Outline

  • What is DHT (Distributed Hash Table)?

  • Why DHTs?

  • Applications

  • How lookup works?

  • Alternatives to DHTs

  • Performance – Routing

  • Performance – Load Balancing

  • Security – Routing Attack

  • Security – Inconsistent Behaviour

  • Comparison to Other Facilities

  • Current Research Projects

  • Conclusion


What is dht

….

node

node

node

What is DHT?

Distributed application

data

get (key)

put(key, data)

Distributed hash table

  • DHT provides the information look up service for P2P applications.

  • Nodes uniformly distributed across key space

  • Nodes form an overlay network

  • Nodes maintain list of neighbours in routing table

  • Decoupled from physical network topology

(Figure adopted from Frans Kaashoek)


Why dhts

Why DHTs?

  • Why Middleware?

  • Simplifies the development for large-scale distributed Apps

  • Better security and robustness

  • Simple API

  • Why Do We Need DHTs?

  • Simplifies the development for large-scale distributed Apps

  • Better security and robustness

  • Simple API

  • Exploits P2P resources


Applications

Applications

  • Anything that requires a hash table

  • Databases, FSes, storage, archival

  • Web serving, caching

  • Content distribution

  • Query & indexing

  • Naming systems

  • Communication primitives

  • Chat services

  • Application-layer multi-casting

  • Event notification services

  • Publish/subscribe systems ?


How lookup works

How lookup works?

Example: Chord [Stoica et. al.]

0

1

15

Finger Table for Node 2

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works1

How lookup works?

Example: Chord

0

1

15

Finger Table for Node 10

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works2

How lookup works?

Example: Chord

0

1

15

Finger Table for Node 10

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works3

How lookup works?

Example: Chord

0

1

15

Finger Table for Node 14

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works4

How lookup works?

Example: Chord

0

1

15

Finger Table for Node 14

2

14

3

13

4

12

5

11

10

6

7

9

8


How lookup works5

How lookup works?

Example: Chord

0

1

15

2

14

3

Now Node 2 can retrive information for key 0 from Node 1.

4

12

5

11

10

6

7

9

8


Alternatives to dhts

N1

N2

N4

N3

Start

Target

N10

N6

N8

N7

N9

Server

N1

Client

Client

N2

Internet

N4

N3

Start

Target

DB

N10

Client

N6

N8

Client

N7

N9

Server

Alternatives to DHTs

  • Distributed file system

  • Centralized lookup

  • P2P flooding queries

(Figures adopted from Frans Kaashoek)


Performance lookup

Performance -- Lookup

  • Purpose -- to locate a target node

  • Each step, try to get closer to locating target node

    • Ask a closer neighbour

  • Performance & scalability tied directly to lookup algorithm

  • 2 Aspects to Performance

  • Path latency

  • Lookup path length (# hops)

  • 2 Aspects to Scalability

  • size of routing table – O(log N)

  • lookup path length – O(log N)

  • 3 Techniques

  • proximity lookup

  • proximity neighbour selection

  • geographic layout


Performance load balancing

Performance -- Load Balancing

  • Issues

  • Hot-spots

    • Content

    • Lookup

  • Heterogeneous nodes & paths

  • System flux

  • Solution

  • Replication is the key

    • Also good for fault-tolerance

  • Cache lookup answers backwards along path


Security incorrect lookup 1

Security – Incorrect Lookup (1)

  • When asked for the “next hop”, give a wrong answer

0

Finger Table for Node 2

1

15

2

14

3

13

4

12

5

11

Node 2 to Node 10: Please tell me how to reach key 0 ….

10

6

7

9

8


Security incorrect lookup 2

Security – Incorrect Lookup (2)

  • When asked for the “next hop”, give a wrong answer

0

Finger Table for Node 10

1

15

2

14

3

13

4

12

5

11

Node 2 to Node 10: Please tell me how to reach key 0 ….

Node 10 answers: ask Node 14

10

6

7

9

8


Security incorrect lookup 3

Security – Incorrect Lookup (3)

  • When asked for the “next hop”, give a wrong answer

0

Finger Table for Node 14

1

15

2

14

3

13

4

12

5

11

Node 2 to Node 14: Please tell me how to reach key 0 ….

Node 14 answers: ask Node 10

10

6

7

9

8


Security incorrect lookup 4

Security – Incorrect Lookup (4)

  • Solution [Sit and Morris]:

    • “Define verifiable system invariant”

    • “Allow the querier to observe lookup progress”

  • Our idea how this can be implemented:

    • Concretely, using an integral monotonically decreasing quantity to implement the idea of “progress”.

    • The concept of “monotonically decreasing quantity” has been used in program construction guaranteeing total correctness. [Parnas]


Security inconsistent behaviour

Security – Inconsistent Behaviour

  • Inconsistent Behaviour, i.e., lie intelligibly

  • Sybil attack [Kaashoek]

Solution 1: public key solution


Security inconsistent behaviour1

Security – Inconsistent Behaviour

  • Inconsistent Behaviour, i.e., lie intelligibly

  • Sybil attack [Kaashoek]

Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem:

How to find out the traitors among the Generals? [Lamport]


Security inconsistent behaviour2

Security – Inconsistent Behaviour

Commander

attack

attack

“he said ‘retreat’”

Lieutenant 1

Lieutenant 2

  • Inconsistent Behaviour, i.e., lie intelligibly

  • Sybil attack [Kaashoek]

Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem:

How to find out the traitors among the Generals? [Lamport]


Security inconsistent behaviour3

Security – Inconsistent Behaviour

Commander

attack

retreat

“he said ‘retreat’”

Lieutenant 1

Lieutenant 2

  • Inconsistent Behaviour, i.e., lie intelligibly

  • Sybil attack [Kaashoek]

Solution 1: public key solution

Solution 2: Byzantine Protocol

Byzantine Generals Problem:

How to find out the traitors among the Generals? [Lamport]


Comparison to other facilities

Comparison to Other Facilities


Research projects

Research Projects

Iris – security & fault-tolerance – US Gov’t

Chord – circular key space

Pastry – circular key space

Tapestry – hypercube space

CAN – n-dimensional key space

Kelips – n-dimensional key space

DDS-- middleware platform for internet service construction

-- cluster-based

-- incremental scalability


Summary

Summary

  • Good middleware platform

  • Exploits P2P networks

  • An exciting new research area


References

References

  • Lamport, Leslie et. al. The Byzantine Generals Problem

  • Sit, Emil, Morris, Robert. Security Considerations for Peer-to-Peer Distributed Hash Tables

  • Kaashoek, Frans. Distributed Hash Tables – Building large-sacle, robust distributed applications

  • Stoica, Ion et. al. Chord: A scalable peer-to-peer lookup service for Internet applications

  • Parnas, D. L. Connecting Theory to Practice: Software Engineering Programme


  • Login