Incrementally Improving Lookup Latency in Distributed Hash Table Systems

1. Incrementally Improving Lookup Latency in Distributed Hash Table Systems Hui Zhang1, Ashish Goel2, Ramesh Govindan1 1University of Southern California 2Stanford University

2. Outline • Latency stretch problem in Distributed Hash Table (DHT) systems, with Chord as an example • Two “latency stretch” theorems • Lookup-Parasitic Random Sampling (LPRS) • Simulation & Internet measurement results • Conclusion & future work Sigmetrics'03

3. DHT systems • A new class of peer-to-peer routing infrastructures • CAN, Chord, Pastry, Tapestry, etc. • Support a hash table-like functionality on Internet-like scale • a global key space: each data item is a key in the space, and each node is responsible for a portion of the key space. • given a key, map it onto a node. • Our research results apply to frugal DHT systems. • The search space for the key decreases by a constant factor after each lookup hop. • Examples: Chord, Pastry, Tapestry. Sigmetrics'03

4. 0 Data 0 256 224 32 256 64 192 96 160 120 128 Chord – key space Network node A Chord network with 8 nodes and 8-bit key space Sigmetrics'03

5. Data 32 [2,4) [4,8) [16,32) [128,256) [1,2) [8,16) [32,64) [64,128) 64 96 128 192 160 224 Range 8 Range 2 Range 1 Range 3 Range 5 Range 6 Range 4 Range 7 Chord – routing table setup Network node Pointer 0 255 In node i’s routing table: One entry is created to point to to the first node in its jth ranges [i+2j-1, i+2j), 1  j  m. A Chord network with N(=8) nodes and m(=8)-bit key space Sigmetrics'03

6. Data 120 32 96 64 160 192 224 128 U.S.A China 64 0 96 128 Latency stretch in Chord Network node Overlay routing physical link 0 255 A Chord network with N(=8) nodes and m(=8)-bit key space Sigmetrics'03

7. Latency stretch [Ratnasamy et al. 2001] latency for each lookup on the overlay topology • = average latency on the underlying topology • In Chord, (logN) hops per lookup in average • (logN) stretch in original Chord. • Could Chord do better, e.g., O(1) stretch, without much change? Sigmetrics'03

8. Our contributions • Theory • Latency expansion characteristic of the underlying network topology decides latency optimization in frugal DHT systems. • Exponential latency expansion: bad news. • Power-law latency expansion: good news. • System • Lookup-Parasitic Random Sample (LPRS), an incremental latency optimization technique. • Achieve O(1) stretch under power-law latency topologies. • Internet measurement. • The Internet router-level topology resembles power-law latency expansion. Sigmetrics'03

9. Latency expansion • Let Nu(x) denote the number of nodes in the network G that are within latency x of node u. • - power-law latency expansion:Nu(x) grows (i.e. ``expands'‘) proportionally to xd, for all nodes u. • Examples: ring (d=1), mesh (d=2). • - exponential latency expansion:Nu(x) grows proportionally to x for some constant  > 1. • Examples: random graphs. Sigmetrics'03

10. Chord node Physical link Router The worse-case scenario: equal-distance “Latency-stretch” theorem - I • “Bad news” Theorem • If the underlying topology G is drawn from a family of graphs with exponential latency expansion, then the expected latency of Chord is (L•logN), where L is the expected latency between pairs of nodes in G. Sigmetrics'03

11. “Latency-stretch” theorem - II • “Good news” Theorem • If • (1) the underlying topology G is drawn from a family of graphs with d-power-law latency expansion, and • (2) for each node u in the Chord network, it samples (log N)d nodes in each range with uniform randomness and keeps the pointer to the nearest node for future routing, • then the expected latency of a request is O(L), where L is the expected latency between pairs of nodes in G. Sigmetrics'03

12. Two remaining questions • How does each node efficiently achieve (log N)d samples from each range? • Do real networks have power-law latency expansion characteristic? Sigmetrics'03

13. Node 1 node t is in its range-x (< d) Node 0 node t is in its range-d Node 2 node t is in its range-y (< x) Node t For a routing request with (log N) hops, final node t will be a random node in (log N) different ranges Uniform sampling in terms of ranges Node x: the node at hop x Node 0: the request initiator Node t: the request terminator routing path Sigmetrics'03

14. Lookup-Parasitic Random Sampling 1. Recursive lookup. 2. Each intermediate hop appends its IP address to the lookup message. 3. When the lookup reaches its target, the target informs each listed hop of its identity. 4. Each intermediate hop then sends one (or a small number) of pings to get a reasonable estimate of the latency to the target, and update its routing table accordingly. Sigmetrics'03

15. LPRS-Chord: convergence time Convergence Time Sigmetrics'03

16. LPRS-Chord: topology with power-law expansion Ring Stretch (at time 2logN) Sigmetrics'03

17. on the end-user level on the router level What’s the latency expansion characteristic of Internet? Sigmetrics'03

18. Internet router-level topology: latency measurement • Approximate link latency by geographical latency • - assign geo-locations to nodes using Geotrack[Padmanabhan2001]. • A large router-level topology dataset • - 320,735 nodes, mapped to 603 distinct cities all over the world. • - 92,824 node pairs are sampled to tractably compute the latency expansion of this large topology. Sigmetrics'03

19. Internet router-level topology: latency expansion latency expansion Sigmetrics'03

20. LPRS-Chord on router-level topology Stretch on the router-level subgraphs (at time 2logN) Sigmetrics'03

21. Conclusion • LPRS has significant practical applicability as a general latency reduction technique for frugal DHT systems. • Future work • - Studying the interaction of LPRS scheme with the dynamics of P2P systems. Sigmetrics'03

22. Thank you! Sigmetrics'03

23. Backup slides Sigmetrics'03

24. 2m-2 2m-1 A simple random sampling solution Network node Pointer Distance measurement 2m-1 0 A Chord network with m-bit key space Sigmetrics'03

25. 2m-2 2m-1 A simple random sampling solution Network node Pointer Distance measurement 2m-1 0 A Chord network with m-bit key space Sigmetrics'03

26. Term definition (II) • Range • - for a given node in a Chord overlay with ID j, its i-th rangeRi(j) is the interval [j+2i-1, j+2i) on the key space, where 1  i  m. • Frugal routing • 1. after each hop, the search space for the target reduces by a constant factor, and • 2. If w is an intermediate node in the route, v is the destination, and v  Ri(w), then the node after w in the route depends only on w and i. Sigmetrics'03

27. LPRS-Chord: simulation methodology Phase 1. N nodes join the network one-by-one. Phase 2. each node on average inserts four documents into the network. Phase 3. each node generates, on average 3logNdata requests one-by-one. - LPRS actions are enabled only in Phase 3 - Performance measurement begins at Phase 3 Sigmetrics'03

28. Comparison of 5 sampling strategies: definitions • Consider a lookup that is initiated by node x0, then forwarded to node x1, x2, ..., and finally reaches the request terminator, node xn: • 1. Node xi samples node xn, 0  i < n • 2. Node xn samples nodes x0, …, xn-1 • 3. Node xi samples node xi-1, 1  i  n • 4. Node x0 samples nodes xn • 5. Node xi samples node x0, 0 < i  n Sigmetrics'03

29. Comparison of 5 sampling strategies: simulation result Sigmetrics'03

30. Zipf-ian document popularity Sigmetrics'03

31. Impact of skewed request distributions Sigmetrics'03