1 / 43

Large-scale adaptive systems

Large-scale adaptive systems. Lecture 3: Gossip-based algorithms. Dr. Stefan Dulman s.o.dulman@tudelft.nl. Review previous lecture. Adaptation Mechanisms Introduction Feedback mechanisms Example: Schools of fish, flocks of birds Stigmergy Example: Economic based mechanisms Autopoiesis

edolie
Download Presentation

Large-scale adaptive systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Large-scale adaptive systems Lecture 3: Gossip-based algorithms Dr. Stefan Dulman s.o.dulman@tudelft.nl

  2. Review previous lecture • Adaptation Mechanisms • Introduction • Feedback mechanisms • Example: Schools of fish, flocks of birds • Stigmergy • Example: Economic based mechanisms • Autopoiesis • Reinforcement learning • Example: trust-based system Large-scale adaptive systems, 2010

  3. Related work • This lecture is based on the following material: • Maarten van Steen: Gossiping in Large-Scale Distributed Systems http://pds.twi.tudelft.nl/~epema/asci-a9-2010.html • OzalpBabaoglu: Complex Systemshttp://www.cs.unibo.it/~babaoglu/courses/cas/ • Giovanna diMarzoSerugendo: Adaptive Systemshttp://www.dcs.bbk.ac.uk/~dimarzo/courses/as.html Large-scale adaptive systems, 2010

  4. Gossiping - Introduction • Large-scale systems around us: • Computer networks of all sorts • Supply chains (supply networks) • Peer-2-Peer data sharing systems • Sensor-actuation systems • Mobile ad-hoc networks • Size of systems increases fast • Scalability of algorithms becomes an issue • Distributed system – centralized control? • Even apparently static networks show mobility • Possible solution: IBM’s “autonomic computing” Large-scale adaptive systems, 2010

  5. Autonomic computing • Large-scale systems requirements • Self configuring, self optimizing, self healing, self protecting… • Umbrella concept: “Self-*” • System characteristics: • Large number of entities interacting anyhow in simple ways • Decentralized – all “peers” are equal • Shows emergent behavior • Even if central control is applied! • Power grids, telephone systems, supply chains • Parasitic emergence is usually unforeseen (“cascade failures”) • Difficult to control (if feasible at all) Large-scale adaptive systems, 2010

  6. Basic idea • Make interactions work for you rather than against you • Make use of the “parasitic behavior” – learn how to build on top of it • Simple actions based on local information • Simple interactions with neighbors • Use the power of randomization and large numbers Large-scale adaptive systems, 2010

  7. Gossip-based interaction • Gossip: basic mechanism for spreading and aggregating information • Communication takes place locally • Nodes exchange information with neighbors • Robust, fast, scales well • Exchanged data can be anything • Exchanged data, references to nodes, actual programs, etc. • There is no centralized control or management Large-scale adaptive systems, 2010

  8. Gossiping as a mechanism Large-scale adaptive systems, 2010

  9. Gossip-based applications • Examples • Raw information dissemination • Data aggregation • Topology construction for overlay networks • Semantic clustering of nodes • Realizing storage facilities in ad hoc networks • Distributed signal processing • Not everything can be accomplished via gossiping! Large-scale adaptive systems, 2010

  10. But… • Some observations • Emergent phenomena often encountered • Global properties emerging unexpectedly from local interactions • Theory is still lacking: models are difficult to validate • Adaptive random geometric graph theory • Many practical issues still to resolve • Adaptiveness (too many design-time parameters) • Security (attacking a gossip-based system is easy) • Competitive alternative single-point solutions Large-scale adaptive systems, 2010

  11. Example – gossiping (multicasting) P. Eugster: Epidemic Information Dissemination in Distributed Systems (IEEE Computer, 2004)

  12. Gossip (push) – fully connected network Round 1 Round 2 Round 3 Round 4 Round 5 Round 6 Round 7 Round 8 200x200 nodes Large-scale adaptive systems, 2010

  13. Gossip (push-pull) – mesh network Round 1 Round 10 Round 20 Round 30 Round 40 Round 50 Round 60 Round 70 100x100 nodes, transmission range = 10 units Large-scale adaptive systems, 2010

  14. Lecture 3: Overview • Introduction • Basics of gossiping algorithms • Applications • Firefly synchronization • Cooperation in selfish environments • Random sampling • Formation creation • Summary Large-scale adaptive systems, 2010

  15. Gossiping mechanism • Terminology • Anti-entropy: each node chooses another node and exchanges information about difference in states • Both nodes have access to identical information -> identical states • Gossiping: a node informs other nodes about its own state (infects the other nodes) and may stop • Notations • Object O has on node S the value val(O,S) and timestamp T(O,S) !!! We will use “gossiping” terminology for both mechanisms !!! Large-scale adaptive systems, 2010

  16. Anti-entropy • Node S exchanges information with node S’ • Push: T(O,S’)<T(O,S) -> val(O,S’) <- val(O,S) • Pull: T(O,S’)>T(O,S) -> val(O,S) <- val(O,S’) • Push-pull: S and S’ exchange their updates • Convergence speed: • O(logn) cycles update speed for fully connected network, random unicast model • 40000 nodes – 8 cycles to computethe average value Round 1 Round 8 Large-scale adaptive systems, 2010

  17. Anti-entropy analysis • Setup: a source propagates updates in a fully connected network • Pi = probability that a node has not received update after i-th cycle • Two cases: • Pull: • the node was not updated during the i-th cycle and should contact another node during next cycle • Push: • the node did not update during i-thround and no node chooses it for the next round Large-scale adaptive systems, 2010

  18. Gossiping mechanism • Basic mechanism • Node S (containing the update) contacts node S’ • If node S’ knows the update • Node S stops with probability p=1/k (S is effectively removed) • Otherwise • Node S contacts another random node S’’ Large-scale adaptive systems, 2010

  19. Gossiping – mathematical model S – nodes not updated I – active nodes (updated) R – passive nodes (updated) Large-scale adaptive systems, 2010

  20. Gossiping – mathematical model !!! Gossiping is not perfect !!! Large-scale adaptive systems, 2010

  21. Convergence speed • Demonstration sketch: • Consider at each step in time values xi • Compute the standard deviation of xi at each time round (si) • Show that si decreases exponentially with time Fully connected networks, anti-entropy Large-scale adaptive systems, 2010

  22. PushSum algorithm • Simple algorithm of computing average values • Each node starts with a (random) value & a weight • Nodes exchange pieces of the value with neighbors • In time, all values converge to a common mean value • PushSum is not strictly an “anti-entropy” algorithm! Node i: 1: {(mr,t-1,ωr,t-1)} – received pairs in round t-1 2: mi,t = Σr(mr,t-1) ωi,t = Σr (ωr,t-1) 3: choose random neighbor j 4: send to i and j: ( ½ mi,t, ½ ωi,t) 5: estimate in round t is: mi,t/ωi,t D. Kempe et al.: Gossip-Based Computation of Aggregate Information (FOCS 2003)

  23. PushSum example Initial values mi+ mj ⅔(mi+ mj) ⅓(mi+ mj) mj mi = = ⅔ (ωi+ωj) ⅓ (ωi+ωj) 2 ωi ωj Final values: mi+ mj = = 2 i j 3 -> ⅓(mi+ mj) 6 -> ⅔ (mi+ mj) 0.67 → ⅓ (ωi+ωj) 1.33 → ⅔ (ωi+ωj)

  24. Alternative approach – PushSum alg. • Introduce notion of diffusion speed • Speed with which the initial value diffuses through the network • Introduce the protocol PushVector • Identical to PushSum only that it uses binary weights for links • Prove diffusion speed obeys O(logn) law • Prove constant mass property • Prove geometric convergence in values D. Kempe et al.: Gossip-Based Computation of Aggregate Information (FOCS 2003)

  25. PushSum & network dynamics • Convergence speed is influenced by: • Network diameter – reduced the convergence speed • Node mobility – mobility increases convergence speed • Number of random neighbors selected • Unicast – Multicast – Broadcast Anand D. Sarwate: The Impact of Mobility on Gossip Algorithms, IEEE Transactions on Information Theory

  26. Influence of failures • Under ideal conditions (no failures): Σimi,t-1 = Σimi,t – mass conservation Σiωi,t-1 = Σiωi,t – weight conservation • In real environment: • Node failures: mass loss • Communication failures: mass loss • Churn: severe mass loss • Conclusion: • Average value decreases with time in real world conditions MárkJelasity: Gossip-based aggregation in large dynamic networks. ACM Trans. Computer Systems, 2005

  27. Extension of PushSum • Computing averages: • mi,0 = “random”, ωi,0 = 1 • mi,∞ = sumi(mi,0)/n • Computing sums: • mi,0 = “random”, ωi,0 = 0, ω0,0 = 1 • mi,∞ = sumi(mi,0) • Counting the nodes in the network: • mi,0 = 1, ωi,0 = 0 • mi,∞ = sumi(mi,0) = n • Main properties hold for all variations: • The algorithms are guaranteed to converge • The convergence speed is the same in all cases Large-scale adaptive systems, 2010

  28. Lecture 3: Overview • Introduction • Basics of gossiping algorithms • Applications • Firefly synchronization • Cooperation in selfish environments • Random sampling • Formation creation • Summary Large-scale adaptive systems, 2010

  29. Firefly synchronization • Several astonishing synchronization mechanisms in nature • Heart pacemaker cells • Chirping crickets • Clapping of an audience at a concert • Flashing of fireflies • A common phenomena underlying everything: coupled oscillators • Each actor is an independent “oscillator” • Oscillators couple through environment • Mechanical vibrations, air pressure, visual clues, olfactory signals • They influence each other, causing continuous minor adjustments, leading to synchrony “SYNC: The Emerging Science of Spontaneous Order” – Steven Strogatz

  30. Fireflies • Certain species of male fireflies are known to synchronize their flashes despite: • Small connectivity (each firefly sees a limited number of neighbors) • Communication is not instantaneous • Independent local “oscillators” with random initial periods • Engineering approach • Style of interaction: anti-entropy, push mechanism • Local state S: current phase of the local oscillator and period • Small set of random neighbors • Update function: reset local oscillator based on phase of arriving flash Large-scale adaptive systems, 2010

  31. The Ermentrout model • Modify the local oscillator period based on when the flash arrives: • If “too late” (φ<1/2) then “slow down” (increase period delta) • If “too early” (φ>1/2) then “speed up” (decrease period delta) Node i Node j Δφ Time !!!NetLogo animation !!! Large-scale adaptive systems, 2010

  32. Lecture 3: Overview • Introduction • Basic gossip algorithm • Applications • Firefly synchronization • Cooperation in selfish environments • Random sampling • Formation creation • Summary Large-scale adaptive systems, 2010

  33. Cooperation in selfish environments • Example: P2P systems are usually open systems • Possible free riders • High level of free riding can degrade performance • Gossip-based algorithms can be used to sustain high levels of cooperation despite selfish nodes • Based on “copy” and “rewire” operations • Engineering the system • Pull interaction, local state: current utility, strategy and neighborhood within an interaction network • Select single random sample • Update function: copy strategy and neighborhood if the peer is achieving better utility Large-scale adaptive systems, 2010

  34. Prisoner’s dilemma • Prisoner’s dilemma in SLAC • Nodes play PD with neighbors chosen randomly in the interaction network • Only pure strategies (always C or always D) • Strategy mutation: flip current strategy • Utility: average payoff achieved D. Hales: SLACER: Randomness to Cooperation in Peer-to-Peer Networks. StoDiS 2005

  35. Lecture 3: Overview • Introduction • Basic gossip algorithm • Applications • Firefly synchronization • Cooperation in selfish environments • Random sampling • Formation creation • Summary Large-scale adaptive systems, 2010

  36. Random sampling • Problem: extract random samples from a set • Example: • The set: {1,2,3,1,2,3,2,1,2,3,2,1,2,3,2,1,2,3,3,3} • Solutions: • Pick randomly i in the range 1-20 and pick the ith element • Etc… • Solution needs to generate random samples according to the distribution of the set: • Element frequencies {1,2,3}→ {0.25,0.4,0.35} Large-scale adaptive systems, 2010

  37. Distributed random sampling • Set is distributed over n nodes • Problem: pick random samples from the set without gathering all the data at a central point {2,3,2,1} {2,3} {3,2,1} {1} {1,2} {1,2,3} {3,3} Original set: { 1,2,3, 1, 2,3,2, 1,2, 3,2,1, 2,3,2,1, 2,3, 3,3 } {2,3,2} Large-scale adaptive systems, 2010

  38. Push-Random algorithm • Protocol uses short messages • Each node holds only one element • Messages contain one element + one weight Node i: 1: {(qr,t-1,ωr,t-1)} – received pairs in round t-1 2: ωi,t = Σr (ωr,t-1) qi,t = picked at random from {qr,t-1} with prob. wr,t-1/ωi,t 3: choose shares αi,j,t for each neighbor j 4: send to each neighbor j: (qi,t, αi,j,t·ωi,t) 5: random sample in round t is: qi,t Large-scale adaptive systems, 2010

  39. Lecture 3: Overview • Introduction • Basic gossip algorithm • Applications • Firefly synchronization • Cooperation in selfish environments • Random sampling • Formation creation • Summary Large-scale adaptive systems, 2010

  40. Formation creation • Ubiquitous computing • Large number of mobile “smart” physical objects with identification, sensing and computing power • Rescue teams with mobile devices • Automobiles in vehicular networks • Swarms of robots • Satellites in earth orbit • Adhoc wireless communication networks based on physical proximity and contact • Broadcast and multicast model rather than unicast Large-scale adaptive systems, 2010

  41. Formation creation • Dynamic collection of agents that can move in any direction • Each agent has an unique ID and can determine the relative position of any other agent • Agents have access to a peer sampling service • Devise a protocol such that mobile agents self organize intro pre-specified global formations without central control Large-scale adaptive systems, 2010

  42. Gossip framework instantiation • Engineering a solution: • Pull interaction, local state S: current physical position and motion vector • Select k random samples from populations • Update function: compute motion vector based on positions of most and least preferred neighbor Mark Jelasity: T-Man: Gossip-based Overlay Topology Management. Engineering Self Organizing Systems

  43. Summary • Introduction • Basic gossip algorithm • Applications • Firefly synchronization • Cooperation in selfish environments • Random sampling • Formation creation • Summary Large-scale adaptive systems, 2010

More Related