Ins twine a scalable peer to peer architecture for intentional resource discovery
This presentation is the property of its rightful owner.
Sponsored Links
1 / 30

INS/Twine : A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery PowerPoint PPT Presentation


  • 65 Views
  • Uploaded on
  • Presentation posted in: General

INS/Twine : A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery. Magdalena Balazinska , Hari Balakrishnan, and David Karger MIT – Laboratory for Computer Science http://nms.lcs.mit.edu/. Problem Description. Abundant ubiquitous computation and communication

Download Presentation

INS/Twine : A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Ins twine a scalable peer to peer architecture for intentional resource discovery

INS/Twine : A ScalablePeer-to-Peer Architecture for Intentional Resource Discovery

Magdalena Balazinska, Hari Balakrishnan, and David Karger

MIT – Laboratory for Computer Science

http://nms.lcs.mit.edu/


Problem description

Problem Description

  • Abundant ubiquitous computation and communication

  • Increasingly large computing environments

  • Dynamic environments

  • Many possible “cool” applications

  • Locate resources using intentional descriptions


Ins overview

INR

INR

INR

INR

INR

INR

INS Overview

Client describes attributes of required resources

Self-configuring

resolver network

Resources advertise their capabilities

INR: Intentional Name Resolver


Resource discovery goals

Resource Discovery Goals

  • Allow client applications to locate services and devices

  • Handle sophisticated resource descriptions

  • Handle dynamism in the operating environment

  • Scale to large numbers of resources


Existing solutions for scalability

Sensor

Proxy

Sensor

Proxy

Sensor

Proxy

Existing Solutions for Scalability

Partitioning

Cameras

Bldg 3

Resolver

Resolver

?

Floors 1-3

Floors 4-6

Bldg 1

Bldg 2

Resolver

Resolver

Resolver

Resolver


Existing solutions for scalability1

Sensor

Proxy

Sensor

Proxy

Sensor

Proxy

Existing Solutions for Scalability

Hierarchies

Resolver

Resolver

Resolver

Resolver

Resolver

Resolver

Resolver


Ins twine contributions

INS/Twine Contributions

  • Collaborating peer resolvers: no content or location constraints on queries

  • Scalability and load distribution via hash-based partitioning of resource descriptions among resolvers

  • Semi-structured resource descriptions with arbitrary attribute-set

  • Network dynamism

  • Designed for an environment where all resources are equally important to users


Ins twine approach overview

Sensor

Proxy

INS/Twine Approach Overview

resource = camera

resource = motion sensor

Resolver

Resolver

subject = traffic

Resolver

Resolver

Resolver

Resolver

Resolver

subject = traffic

resource = motion sensor

subject = traffic

resource = camera

subject = traffic


Ins twine approach overview1

INS/Twine Approach Overview

  • A resource advertises its descriptions and network location to any resolver

  • The description is converted into a canonical form: attribute-value tree (AVTree)

  • Using the content of the advertised description, the resolver determines which resolvers should know about the resource

  • The resolver forwards the description to these resolvers for storage

  • Similarly for queries


Architecture of one resolver

Resolver

Res adv.

Strand

Mapper

a1

v1

a2

h : 0110

1001

0000

v2

Strand

Key

Router

0110

1001

0000

0110

1001

0000

Best(01101001000)

K nodes are chosen

Key

Distributed Hash Table

Architecture of One Resolver

h = hash(a1v1-a2v2)


Splitting descriptions into strands

resource

subject

root

camera

traffic

subject

resource

resource

resource

camera

traffic

camera

camera

manufacturer

model

manufacturer

model

ACompany

AModel

resource

resource

camera

camera

model

manufacturer

AModel

ACompany

Splitting Descriptions into Strands

Resource description: AVTrees

Six strands

  • Each strand is then hashed into a 128 bit value which determines the nodes that will store the resource information.

  • All queries, even short stranded queries require asking only one resolver!


Distributed hash table chord

Distributed Hash Table: Chord

N5

  • Nodes and keys have 160-bit ID’s

  • Chord maps ID’s to “successor”

  • Successor: Node with next highest ID

N10

N110

N20

N99

Circular

ID Space

N32

Stores key-values

for keys 21..32

N80

N60

Keys 33..60


Basic lookup

Basic Lookup

N120

N10

“Where is key 80?”

N105

Successor pointer

N32

“N90 has K80”

N90

K80

N60


Finger table allows log n time lookups

“Finger table” allows log(N)-time lookups

½

¼

K = log(n) immediate

Successors for robustness

Stabilization methods for concurrency

1/8

1/16

1/32

1/64

1/128

finger[i] points to

successor (n + 2i)

log(n) fingers in all

N80


Back to example

Sensor

Proxy

Back to Example

resource = camera

resource = motion sensor

Resolver

Resolver

subject = traffic

Resolver

Resolver

Resolver

Resolver

Resolver

subject = traffic

resource = motion sensor

subject = traffic

resource = camera

subject = traffic


Properties of ins twine

Properties of INS/Twine

  • For a resource description with a attributes, t at the top-level :

    • Number of strands is : s = 2a – t

  • For R resources, S strands, K replication level, and N resolvers :

    • Storage requirement at each resolver : (RSK)/N

  • Advertisement:

    • SK resolvers contacted (+ O(log N) for key routing)

  • Query:

    • K resolvers contacted (+ O(log N) for key routing)

    • 100% success rate for less than K failures

    • Failure rate decreases exponentially with K


State management

State Management

  • Resources join, move, leave and fail

  • Resolvers join and fail

  • How to maintain consistency while achieving fault tolerance?

    • Hard state

    • Soft state

    • Hybrid solution implemented in INS/Twine


State management1

INR

INR

INR

INR

INR

INR

INR

INR

Resource

State Management

D

D

D

d

INR: Intentional Name Resolver


State management2

INR

INR

INR

INR

INR

INR

INR

INR

Resource

State Management

Remove

Remove

Remove

INR: Intentional Name Resolver


State management3

INR

INR

INR

INR

INR

INR

INR

INR

Resource

State Management

D

D

D

d

INR: Intentional Name Resolver


State management4

INR

INR

INR

INR

INR

INR

INR

INR

Resource

State Management

Expire

Expire

Expire

INR: Intentional Name Resolver


Evaluation data distribution

Evaluation: Data Distribution

Data distribution rather even.

Each resolvers holds a small fraction of resource descriptions


Evaluation query resolution

Evaluation: Query Resolution

Even distribution of queries among resolvers


Conclusion

Conclusion

  • Intentional resource discovery

  • Scalable peer-to-peer network of resolvers

  • Hash-based mapping of resource descriptions to resolvers

  • Dynamic and even distribution of resource information and queries

  • Handles dynamism of resources and resolvers

http://nms.lcs.mit.edu/projects/twine/


Appendix

Appendix


Ins overview1

INS Overview

INR: Intentional Name Resolver


Describing resources

Describing Resources

  • INS name-specifier

  • XML

  • AVTrees


Problems using concatenation

Problems using concatenation

  • If numerous resources share the same prefix, some nodes may receive significantly more load than others

  • Fully solving short stranded queries requires the colaboration of a linearly growing number of resolvers (with respect to network size)

  • 1) and 2) are contradictory requirements!


Distributed hash table chord1

Distributed Hash Table: Chord

A distributed hash-table is used to map keys onto resolvers efficiently:

From: Chord: A Peer-to-Peer Lookup Service for Internet Applications

Ion Stoica, Robert Morris, David Karger, Frans Kaashoek, Hari Balakrishnan Proc. ACM SIGCOMM Conf., San Diego, CA, September 2001.


Problems using prefixes

Problems using prefixes

  • More insertions for each resource. Small factor since we expect descriptions to be rather short

  • Very popular prefixes may overload certain nodes : many advertisements and queries => the prefix should then become unusable

    • Nodes stop storing resources for that prefix

    • Nodes answer queries for the prefix specifying that they provide a partial answer due to the vague nature of the query


  • Login