An Introduction to Peer-to-Peer networks

1. An Introduction to Peer-to-Peer networks Presentation for CSE620:Advanced networking Anh Le Nov. 4

2. Outline • Overview of P2P • Classification of P2P • Unstructured P2P systems • Napster (Centralized) • Gnutella (Distributed) • Kazaa/Fasttrack (Super-node) • Structured P2P systems (DHTs): • Chord • YAPPERS (hybrid) • Conclusions Anh Le + Tuong Nguyen

3. What is P2P systems? • Clay Shirkey: • P2P refers to applications that take advantage of resources (storage, cycles, content, human presence) available at the edges of the internet • The “litmus test:” • Does it allow for variable connectivity and temporary network addresses? • Does it give the nodes at the edges of the network significant autonomy? • P2P Working Group (A Standardization Effort): P2P computing is: • The sharing of computer resources and services by directexchange between systems. • Peer-to-peer computing takes advantage of existing computing power and networking connectivity, allowing economical clients to leverage their collective power to benefit the entire enterprise. Anh Le + Tuong Nguyen

4. What is P2P systems? • Multiple sites (at edge) • Distributed resources • Sites are autonomous (different owners) • Sites are both clients and servers (“servent”) • Sites have equal functionality Anh Le + Tuong Nguyen

5. P2P benefits • Efficient use of resources • Scalability: • Consumers of resources also donate resources • Aggregate resources grow naturally with utilization • Reliability • Replicas • Geographic distribution • No single point of failure • Ease of administration • Nodes self organize • No need to deploy servers to satisfy demand • Built-in fault tolerance, replication, and load balancing Anh Le + Tuong Nguyen

6. Napster • was used primarily for file sharing • NOT a pure P2P network=> hybrid system • Ways of action: • Client sends server the query, server ask everyone and responds to client • Client gets list of clients from server • All Clients send ID’s of the data they hold to the server and when client asks for data, server responds with specific addresses • peer downloads directly from other peer(s) Anh Le + Tuong Nguyen

7. Napster • Further services: • Chat program, instant messaging service, tracking program,… • Centralized system • Single point of failure => limited fault tolerance • Limited scalability (server farms with load balancing) • Query is fast and upper bound for duration can be given Anh Le + Tuong Nguyen

8. Napster 5 4 6 central DB 3 3. Download Request 2. Response 1. Query 4. File 1 2 Peer Anh Le + Tuong Nguyen

9. Gnutella • pure peer-to-peer • very simple protocol • no routing "intelligence" • Constrained broadcast • Life-time of packets limited by TTL (typically set to 7) • Packets have unique ids to detect loops Anh Le + Tuong Nguyen

10. Gnutella - PING/PONG 3 6 Ping 1 Ping 1 Pong 3 Pong 6 Pong 6,7,8 Pong 6,7,8 Ping 1 7 Pong 3,4,5 Pong 5 5 1 2 Pong 7 Ping 1 Ping 1 Ping 1 Pong 2 Known Hosts: 2 Pong 8 Pong 4 8 Ping 1 3,4,5 6,7,8 Query/Response analogous 4 Anh Le + Tuong Nguyen

11. Free riding • File sharing networks rely on users sharing data • Two types of free riding • Downloading but not sharing any data • Not sharing any interesting data • On Gnutella • 15% of users contribute 94% of content • 63% of users never responded to a query • Didn’t have “interesting” data Anh Le + Tuong Nguyen

12. Gnutella:summary • Hit rates are high • High fault tolerance • Adopts well and dynamically to changing peer populations • High network traffic • No estimates on duration of queries • No probability for successful queries • Topology is unknown => algorithm cannot exploit it • Free riding is a problem • A significant portion of Gnutella peers are free riders • Free riders are distributed evenly across domains • Often hosts share files nobody is interested in Anh Le + Tuong Nguyen

13. Gnutella discussion • Search types: • Any possible string comparison • Scalability • Search very poor with respect to number of messages • Probably search time O(logn) due to small world property • Updates excellent: nothing to do • Routing information: low cost • Robustness • High, since many paths are explored • Autonomy: • Storage: no restriction, peers store the keys of their files • Routing: peers are target of all kind of requests • Global knowledge • None required Anh Le + Tuong Nguyen

14. iMesh, Kazaa • Hybrid of centralized Napster and decentralized Gnutella • Super-peers act as local search hubs • Each super-peer is similar to a Napster server for a small portion of the network • Super-peers are automatically chosen by the system based on their capacities (storage, bandwidth, etc.) and availability (connection time) • Users upload their list of files to a super-peer • Super-peers periodically exchange file lists • Queries are sent to a super-peer for files of interest Anh Le + Tuong Nguyen

15. Structured Overlay Networks / DHTs Chord, Pastry, Tapestry, CAN, Kademlia, P-Grid, Viceroy Set of Nodes Keys of Nodes Common Identifier Space Hashing ConnectThe nodes Smartly Keys of Values Keys of Values Hashing … Node IdentifierValue Identifier Anh Le + Tuong Nguyen

16. node A node B node C node D →Node D : lookup(9) • Each node has a routing table • Pointers to some other nodes • Typically, a constant or a logarithmic number of pointers The Principle Of Distributed Hash Tables • A dynamic distribution of a hash table onto a set of cooperating nodes • Basic service: lookup operation • Key resolution from any node Anh Le + Tuong Nguyen

17. DHT Desirable Properties • Keys mapped evenly to all nodes in the network • Each node maintains information about only a few other nodes • Messages can be routed to a node efficiently • Node arrival/departures only affect a few nodes Anh Le + Tuong Nguyen

18. Chord [MIT] • consistent hashing (SHA-1) assigns each node and object an m-bit ID • IDs are ordered in an ID circle ranging from 0 – (2m-1). • New nodes assume slots in ID circle according to their ID • Key k is assigned to first node whose ID ≥ k •  successor(k) Anh Le + Tuong Nguyen

19. identifier node 6 X key 0 1 7 6 2 5 3 4 2 Consistent Hashing- Successor Nodes 1 successor(1) = 1 identifier circle successor(6) = 0 6 2 successor(2) = 3 Anh Le + Tuong Nguyen

20. Consistent Hashing – Join and Departure • When a node n joins the network, certain keyspreviously assigned to n’s successor now become assigned ton. • When node n leaves the network, all of its assigned keys arereassigned to n’s successor. Anh Le + Tuong Nguyen

21. 0 1 7 6 2 5 3 4 Consistent Hashing – Node Join keys 5 7 keys 1 keys keys 2 Anh Le + Tuong Nguyen

22. 0 1 7 6 2 5 3 4 Consistent Hashing – Node Dep. keys 7 keys 1 keys 6 keys 2 Anh Le + Tuong Nguyen

23. Scalable Key Location – Finger Tables • To acceleratelookups, Chord maintains additional routing information. • Thisadditional information is not essential for correctness, which isachieved as long as each node knows its correct successor. • Each node n’ maintains a routing table with up tomentries (which is in fact the number of bits in identifiers), called finger table. • The ith entry in the table at node n contains theidentity of the first node s that succeeds n by at least 2i-1 on the identifier circle. • s = successor(n+2i-1). • s is called the ith finger of node n, denoted by n.finger(i) Anh Le + Tuong Nguyen

24. 0 1 7 6 2 5 3 4 Scalable Key Location – Finger Tables finger table keys start succ. 6 For. 1 2 4 1 3 0 0+20 0+21 0+22 finger table keys For. start succ. 1 1+20 1+21 1+22 2 3 5 3 3 0 finger table keys For. start succ. 2 4 5 7 0 0 0 3+20 3+21 3+22 Anh Le + Tuong Nguyen

25. Chord key location • Lookup in finger table the furthest node that precedes key • -> O(log n) hops Anh Le + Tuong Nguyen

26. Node Joins and Stabilizations • The most important thing is the successor pointer. • If the successor pointer is ensured to be up to date, which is sufficient to guarantee correctness of lookups, then finger table can always be verified. • Each node runs a “stabilization” protocol periodically in the background to update successor pointer and finger table. Anh Le + Tuong Nguyen

27. Node Joins and Stabilizations • “Stabilization” protocol contains 6functions: • create() • join() • stabilize() • notify() • fix_fingers() • check_predecessor() • When node n first starts, it calls n.join(n’), where n’ is any known Chord node. • The join() function asks n’ to find the immediate successorof n. Anh Le + Tuong Nguyen

28. Node Joins – stabilize() • Each time node n runs stabilize(), it asks its successorfor the it’s predecessor p, and decides whether pshould be n’s successor instead. • stabilize() notifies noden’s successor of n’s existence, giving the successor the chanceto change its predecessor to n. • The successor does this only if itknows of no closer predecessor than n. Anh Le + Tuong Nguyen

29. nil Node Joins – Join and Stabilization • n joins • predecessor = nil • n acquires ns as successor via some n’ • n runs stabilize • n notifies ns being the new predecessor • ns acquires n as its predecessor • np runs stabilize • np asks ns for its predecessor (now n) • np acquires n as its successor • np notifies n • n will acquire np as its predecessor • all predecessor and successor pointers are now correct • fingers still need to be fixed, but old fingers will still work ns pred(ns) = n n succ(np) = ns pred(ns) = np succ(np) = n np Anh Le + Tuong Nguyen

30. Node Failures • Key step in failure recovery is maintaining correct successor pointers • To help achieve this, each node maintains a successor-list of its r nearest successors on the ring • If node n notices that its successor has failed, it replaces it with the first live entry in the list • Successorlists are stabilized as follows: • node n reconciles its list withits successor s by copying s’s successor list, removing its lastentry, and prepending s to it. • If node n notices that its successorhas failed, it replaces it with the first live entry in its successorlist and reconciles its successor list with its new successor. Anh Le + Tuong Nguyen

31. Handling failures: redundancy • Each node knows IP addresses of next r nodes. • Each key is replicated at next r nodes Anh Le + Tuong Nguyen

32. Chord – simulation result [Stoica et al. Sigcomm2001] Anh Le + Tuong Nguyen

33. Chord – “failure” experiment The fraction of lookups that fail as a function of the fraction of nodes that fail. [Stoica et al. Sigcomm2001] Anh Le + Tuong Nguyen

34. Chord discussion • Search types • Only equality, exact keys need to be known • Scalability • Search O(logn) • Update requires search, thus O(logn) • Construction: O(log^2 n) if a new node joins • Robustness • Replication might be used by storing replicas at successor nodes • Autonomy • Storage and routing: none • Global knowledge • Mapping of IP addresses and data keys to key common key space Anh Le + Tuong Nguyen

35. YAPPERS: a P2P lookup service over arbitrary topology • Motivation: • Gnutella-style Systems • work on arbitrary topology, flood for query • Robust but inefficient • Support for partial query, good for popular resources • DHT-based Systems • Efficient lookup but expensive maintenance • By nature, no support for partial query • Solution: Hybrid System • Operate on arbitrary topology • Provide DHT-like search efficiency Anh Le + Tuong Nguyen

36. Design Goals • Impose no constraints on topology • No underlying structure for the overlay network • Optimize for partial lookups for popular keys • Observation: Many users are satisfied with partial lookup • Contact only nodes that can contribute to the search results • no blind flooding • Minimize the effect of topology changes • Maintenance overhead is independent of system size Anh Le + Tuong Nguyen

37. Basic Idea: • Keyspace is partitioned into a small number of buckets. Each bucket corresponds to a color. • Each node is assigned a color. • # of buckets = # of colors • Each node sends the <key, value> pairs to the node with the same color as the key within its Immediate Neighborhood. • IN(N): All nodes within h hops from Node N. Anh Le + Tuong Nguyen

38. Partition Nodes Given any overlay, first partition nodes into buckets (colors) based on hash of IP Anh Le + Tuong Nguyen

39. X Y Partition Nodes (2) Around each node, there is at least one node of each color May require backup color assignments Anh Le + Tuong Nguyen

40. register yellow content at a yellow node Register Content Partition content space into buckets (colors) and register pointer at “nearby” nodes. Nodes around Z form a small hash table! Z register red content locally Anh Le + Tuong Nguyen

41. Searching Content Start at a “nearby” colored node, search other nodes of the same color. W X Y U V Z Anh Le + Tuong Nguyen

42. Searching Content (2) A smaller overlay for each color and use Gnutella-style flood Fan-out = degree of nodes in the smaller overlay Anh Le + Tuong Nguyen

43. More… • When node X is inserting <key, value> • Multiple nodes in IN(X) have the same color? • No node in IN(X) has the same color as key k? • Solution: • P1: randomly select one • P2: Backup scheme: Node with next color • Primary color (unique) & Secondary color (zero or more) • Problems coming with this solution: • No longer consistent and stable • The effect is isolated within the Immediate neighborhood Anh Le + Tuong Nguyen

44. Extended Neighborhood • IN(A): Immediate Neighborhood • F(A): Frontier of Node A • All nodes that are directly connected to IN(A), but not in IN(A) • EN(A): Extended Neighborhood • The union of IN(v) where v is in F(A) • Actually EN(A) includes all nodes within 2h + 1 hops • Each node needs to maintain these three set of nodes for query. Anh Le + Tuong Nguyen

45. The network state information for node A (h = 2) Anh Le + Tuong Nguyen

46. Searching with Extended Neighborhood • Node A wants to look up a key k of color C(k), it picks a node B with C(k) in IN(A) • If multiple nodes, randomly pick one • If none, pick the backup node • B, using its EN(B), sends the request to all nodes which are in color C(k). • The other nodes do the same thing as B. • Duplicate Message problem: • Each node caches the unique query identifier. Anh Le + Tuong Nguyen

47. More on Extended Neighborhood • All <key, value> pairs are stored among IN(X). (h hops from node X) • Why each node needs to keep an EN(X)? • Advantage: • The forwarding node is chosen based on local knowledge • Completeness: a query (C(k)) message can reach all nodes in C(k) without touching any nodes in other colors (Not including backup node) Anh Le + Tuong Nguyen

48. Maintaining Topology • Edge Deletion: X-Y • Deletion message needs to be propagated to all nodes that have X and Y in their EN set • Necessary Adjustment: • Change IN, F, EN sets • Move <key, value> pairs if X/Y is in IN(A) • Edge Insertion: • Insertion message needs to include the neighbor info • So other nodes can update their IN and EN sets Anh Le + Tuong Nguyen

49. Maintaining Topology • Node Departure: • a node X with w edges is leaving • Just like w edge deletion • Neighbors of X initiates the propagation • Node Arrival: X joins the network • Ask its new neighbors for their current topology view • Build its own extended neighborhood • Insert w edges. Anh Le + Tuong Nguyen

50. Problems with basic design • Fringe node: • Those low connectivity node allocates a large number of secondary colors to its high-connectivity neighbors. • Large fan-out: • The forwarding fan-out degree at A is proportional to the size of F(A) • This is desirable for partial lookup, but not good for full lookup Anh Le + Tuong Nguyen