1 / 47

15-829A/18-849B/95-811A/19-729A Internet-Scale Sensor Systems: Design and Policy

15-829A/18-849B/95-811A/19-729A Internet-Scale Sensor Systems: Design and Policy. Lecture 9 – Lookup Algorithms (DNS & DHTs). Overview. Lookup Overview Hierarchical Lookup: DNS Routed Lookups Chord CAN. The Lookup Problem. N 2. N 1. N 3. Key=“title” Value=MP3 data…. Internet. ?.

knguyen
Download Presentation

15-829A/18-849B/95-811A/19-729A Internet-Scale Sensor Systems: Design and Policy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. 15-829A/18-849B/95-811A/19-729AInternet-Scale Sensor Systems: Design and Policy Lecture 9 – Lookup Algorithms (DNS & DHTs)

  2. Overview • Lookup Overview • Hierarchical Lookup: DNS • Routed Lookups • Chord • CAN Lecture #9

  3. The Lookup Problem N2 N1 N3 Key=“title” Value=MP3 data… Internet ? Client Publisher Lookup(“title”) N4 N6 N5 • Problem in DNS, P2P, routing, etc. • Sensor networks: how to find sensor readings? Lecture #9

  4. Centralized Lookup (Napster) N2 N1 SetLoc(“whale”, N4) N3 Client DB N4 sensor@ Lookup(“whale”) Key=“whale” Value=image data… N8 N9 N7 N6 Simple, but O(N) state and a single point of failure Lecture #9

  5. Hierarchical Queries (DNS) N2 N1 Lookup(“whale”) N3 Client Root N4 Publisher@ Key=“whale” Value=image data… N6 N8 N7 N9 Scalable (log n), but could be brittle Lecture #9

  6. Flooded Queries (Gnutella, /etc/hosts) N2 N1 Lookup(“whale”) N3 Client N4 Publisher@ Key=“whale” Value=image data… N6 N8 N7 N9 Robust, but worst case O(N) messages per lookup Lecture #9

  7. Routed Queries (DHTs) N2 N1 N3 Client N4 Lookup(“whale”) Publisher Key=“whale” Value=image data… N6 N8 N7 N9 Scalable (log n), but limited search styles Lecture #9

  8. Overview • Lookup Overview • Hierarchical Lookup: DNS • Routed Lookups • Chord • CAN Lecture #9

  9. Obvious Solutions (1) Objective: provide name to address translations Why not centralize DNS? • Single point of failure • Traffic volume • Distant centralized database • Single point of update • Doesn’t scale! Lecture #9

  10. Obvious Solutions (2) Why not use /etc/hosts? • Original Name to Address Mapping • Flat namespace • /etc/hosts • SRI kept main copy • Downloaded regularly • Count of hosts was increasing: machine per domain  machine per user • Many more downloads • Many more updates Lecture #9

  11. Domain Name System Goals • Basically building a wide area distributed database • Scalability • Decentralized maintenance • Robustness • Global scope • Names mean the same thing everywhere • Don’t need • Atomicity • Strong consistency Lecture #9

  12. FOR IN class: Type=A name is hostname value is IP address Type=NS name is domain (e.g. foo.com) value is name of authoritative name server for this domain RR format: (class, name, value, type, ttl) DNS Records • DB contains tuples called resource records (RRs) • Classes = Internet (IN), Chaosnet (CH), etc. • Each class defines value associated with type • Type=CNAME • name is an alias name for some “canonical” (the real) name • value is canonical name • Type=MX • value is hostname of mailserver associated with name Lecture #9

  13. DNS Design: Hierarchy Definitions • Each node in hierarchy stores a list of names that end with same suffix • Suffix = path up tree • E.g., given this tree, where would following be stored: • Fred.com • Fred.edu • Fred.cmu.edu • Fred.cmcl.cs.cmu.edu • Fred.cs.mit.edu root org com uk net edu mit gwu ucb cmu bu cs ece cmcl Lecture #9

  14. DNS Design: Zone Definitions • Zone = contiguous section of name space • E.g., Complete tree, single node or subtree • A zone has an associated set of name servers root org ca com uk net edu mit gwu ucb cmu bu Subtree cs ece cmcl Single node Complete Tree Lecture #9

  15. DNS Design: Cont. • Zones are created by convincing owner node to create/delegate a subzone • Records within zone stored multiple redundant name servers • Primary/master name server updated manually • Secondary/redundant servers updated by zone transfer of name space • Zone transfer is a bulk transfer of the “configuration” of a DNS server – uses TCP to ensure reliability • Example: • CS.CMU.EDU created by CMU.EDU administrators Lecture #9

  16. Servers/Resolvers • Each host has a resolver • Typically a library that applications can link to • Local name servers hand-configured (e.g. /etc/resolv.conf) • Name servers • Either responsible for some zone or… • Local servers • Do lookup of distant host names for local hosts • Typically answer queries about local zone Lecture #9

  17. Responsible for “root” zone Approx. dozen root name servers worldwide Currently {a-m}.root-servers.net Local name servers contact root servers when they cannot resolve a name Configured with well-known root servers DNS: Root Name Servers Lecture #9

  18. Recursive query: Server goes out and searches for more info (recursive) Only returns final answer or “not found” Iterative query: Server responds with as much as it knows (iterative) “I don’t know this name, but ask this server” Workload impact on choice? Local server typically does recursive Root/distant server does iterative local name server dns.eurecom.fr intermediate name server dns.umass.edu Lookup Methods root name server 2 iterated query 3 4 7 5 6 authoritative name server dns.cs.umass.edu 1 8 requesting host surf.eurecom.fr gaia.cs.umass.edu Lecture #9

  19. www.cs.cmu.edu NS ns1.cmu.edu NS ns1.cs.cmu.edu A www=IPaddr Typical Resolution root & edu DNS server www.cs.cmu.edu ns1.cmu.edu DNS server Local DNS server Client ns1.cs.cmu.edu DNS server Lecture #9

  20. Workload and Caching • What workload do you expect for different servers/names? • Why might this be a problem? How can we solve this problem? • DNS responses are cached • Quick response for repeated translations • Other queries may reuse some parts of lookup • NS records for domains • DNS negative queries are cached • Don’t have to repeat past mistakes • E.g. misspellings, search strings in resolv.conf • Cached data periodically times out • Lifetime (TTL) of data controlled by owner of data • TTL passed with every record Lecture #9

  21. Prefetching • Name servers can add additional data to any response • Typically used for prefetching • CNAME/MX/NS typically point to another host name • Responses include address of host referred to in “additional section” Lecture #9

  22. www.cs.cmu.edu NS ns1.cmu.edu NS ns1.cs.cmu.edu A www=IPaddr Typical Resolution root & edu DNS server www.cs.cmu.edu ns1.cmu.edu DNS server Local DNS server Client ns1.cs.cmu.edu DNS server Lecture #9

  23. Subsequent Lookup Example root & edu DNS server ftp.cs.cmu.edu cmu.edu DNS server Local DNS server Client ftp.cs.cmu.edu cs.cmu.edu DNS server ftp=IPaddr Lecture #9

  24. Reliability • DNS servers are replicated • Name service available if ≥ one replica is up • Queries can be load balanced between replicas • UDP used for queries • Need reliability  must implement this on top of UDP! • Why not just use TCP? • Try alternate servers on timeout • Exponential backoff when retrying same server • Same identifier for all queries • Don’t care which server responds • Is it still a brittle design? Lecture #9

  25. Overview • Lookup Overview • Hierarchical Lookup: DNS • Routed Lookups • Chord • CAN Lecture #9

  26. Routing: Structured Approaches • Goal: make sure that an item (file) identified is always found in a reasonable # of steps • Abstraction: a distributed hash-table (DHT) data structure • insert(id, item); • item = query(id); • Note: item can be anything: a data object, document, file, pointer to a file… • Proposals • CAN (ICIR/Berkeley) • Chord (MIT/Berkeley) • Pastry (Rice) • Tapestry (Berkeley) Lecture #9

  27. Routing: Chord • Associate to each node and item a unique id in an uni-dimensional space • Properties • Routing table size O(log(N)) , where N is the total number of nodes • Guarantees that a file is found in O(log(N)) steps Lecture #9

  28. Aside: Consistent Hashing [Karger 97] Key 5 K5 Node 105 N105 K20 Circular 7-bit ID space N32 N90 K80 A key is stored at its successor: node with next higher ID Lecture #9

  29. Routing: Chord Basic Lookup N120 N10 “Where is key 80?” N105 N32 “N90 has K80” N90 K80 N60 Lecture #9

  30. Routing: Finger table - Faster Lookups ½ ¼ 1/8 1/16 1/32 1/64 1/128 N80 Lecture #9

  31. Routing: Chord Summary • Assume identifier space is 0…2m • Each node maintains • Finger table • Entry i in the finger table of n is the first node that succeeds or equals n + 2i • Predecessor node • An item identified by id is stored on the successor node of id Lecture #9

  32. Routing: Chord Example • Assume an identifier space 0..8 • Node n1:(1) joinsall entries in its finger table are initialized to itself Succ. Table 0 i id+2i succ 0 2 1 1 3 1 2 5 1 1 7 6 2 5 3 4 Lecture #9

  33. Routing: Chord Example • Node n2:(3) joins Succ. Table 0 i id+2i succ 0 2 2 1 3 1 2 5 1 1 7 6 2 Succ. Table i id+2i succ 0 3 1 1 4 1 2 6 1 5 3 4 Lecture #9

  34. Routing: Chord Example • Nodes n3:(0), n4:(6) join Succ. Table i id+2i succ 0 1 1 1 2 2 2 4 0 0 Succ. Table 1 7 i id+2i succ 0 2 2 1 3 6 2 5 6 Succ. Table i id+2i succ 0 7 0 1 0 0 2 2 2 6 2 Succ. Table i id+2i succ 0 3 6 1 4 6 2 6 6 5 3 4 Lecture #9

  35. Routing: Chord Examples Succ. Table • Nodes: n1:(1), n2(3), n3(0), n4(6) • Items: f1:(7), f2:(2) Items 7 i id+2i succ 0 1 1 1 2 2 2 4 0 0 Succ. Table Items 1 1 i id+2i succ 0 2 2 1 3 6 2 5 6 7 6 2 Succ. Table i id+2i succ 0 7 0 1 0 0 2 2 2 Succ. Table i id+2i succ 0 3 6 1 4 6 2 6 6 5 3 4 Lecture #9

  36. Routing: Query Succ. Table • Upon receiving a query for item id, a node • Check whether stores the item locally • If not, forwards the query to the largest node in its successor table that does not exceed id Items 7 i id+2i succ 0 1 1 1 2 2 2 4 0 0 Succ. Table Items 1 1 i id+2i succ 0 2 2 1 3 6 2 5 6 7 query(7) 6 2 Succ. Table i id+2i succ 0 7 0 1 0 0 2 2 2 Succ. Table i id+2i succ 0 3 6 1 4 6 2 6 6 5 3 4 Lecture #9

  37. Overview • Lookup Overview • Hierarchical Lookup: DNS • Routed Lookups • Chord • CAN Lecture #9

  38. CAN • Associate to each node and item a unique id in an d-dimensional space • Virtual Cartesian coordinate space • Entire space is partitioned amongst all the nodes • Every node “owns” a zone in the overall space • Abstraction • Can store data at “points” in the space • Can route from one “point” to another • Point = node that owns the enclosing zone • Properties • Routing table size O(d) • Guarantees that a file is found in at most d*n1/d steps, where n is the total number of nodes Lecture #9

  39. CAN E.g.: Two Dimensional Space • Space divided between nodes • All nodes cover the entire space • Each node covers either a square or a rectangular area of ratios 1:2 or 2:1 • Example: • Assume space size (8 x 8) • Node n1:(1, 2) first node that joins  cover the entire space 7 6 5 4 3 n1 2 1 0 0 2 3 4 6 7 5 1 Lecture #9

  40. CAN E.g.: Two Dimensional Space • Node n2:(4, 2) joins  space is divided between n1 and n2 7 6 5 4 3 n2 n1 2 1 0 0 2 3 4 6 7 5 1 Lecture #9

  41. CAN E.g.: Two Dimensional Space • Node n2:(4, 2) joins  space is divided between n1 and n2 7 6 n3 5 4 3 n2 n1 2 1 0 0 2 3 4 6 7 5 1 Lecture #9

  42. CAN E.g.: Two Dimensional Space • Nodes n4:(5, 5) and n5:(6,6) join 7 6 n5 n4 n3 5 4 3 n2 n1 2 1 0 0 2 3 4 6 7 5 1 Lecture #9

  43. CAN E.g.: Two Dimensional Space • Nodes: n1:(1, 2); n2:(4,2); n3:(3, 5); n4:(5,5);n5:(6,6) • Items: f1:(2,3); f2:(5,1); f3:(2,1); f4:(7,5); 7 6 n5 n4 n3 f4 5 4 f1 3 n2 n1 2 f3 1 f2 0 0 2 3 4 6 7 5 1 Lecture #9

  44. CAN E.g.: Two Dimensional Space • Each item is stored by the node who owns its mapping in the space 7 6 n5 n4 n3 f4 5 4 f1 3 n2 n1 2 f3 1 f2 0 0 2 3 4 6 7 5 1 Lecture #9

  45. CAN: Query Example • Each node knows its neighbors in the d-space • Forward query to the neighbor that is closest to the query id • Example: assume n1 queries f4 7 6 n5 n4 n3 f4 5 4 f1 3 n2 n1 2 f3 1 f2 0 0 2 3 4 6 7 5 1 Lecture #9

  46. Routing: Concerns • Each hop in a routing-based lookup can be expensive • No correlation between neighbors and their location • A query can repeatedly jump from Europe to North America, though both the initiator and the node that store the item are in Europe! • Solutions: Tapestry takes care of this implicitly; CAN and Chord maintain multiple copies for each entry in their routing tables and choose the closest in terms of network distance • What type of lookups? • Only exact match!  no beatles.* Lecture #9

  47. Summary • The key challenge of building wide area P2P systems is a scalable and robust location service • Solutions covered in this lecture • Naptser: centralized location service • Gnutella: broadcast-based decentralized location service • Freenet: intelligent-routing decentralized solution • DHTs: structured intelligent-routing decentralized solution Lecture #9

More Related