1 / 55

Dynamics of large networks

Jure Leskovec Machine Learning Department Carnegie Mellon University. Dynamics of large networks. Thesis defense, September 3 2008. Web: Rich data. Today: L arge on-line systems have detailed records of human activity On-line communities:

Download Presentation

Dynamics of large networks

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Jure Leskovec Machine Learning Department Carnegie Mellon University Dynamics of large networks Thesis defense, September 3 2008

  2. Web: Rich data • Today: Large on-line systems have detailed records of human activity • On-line communities: • Facebook (64 million users, billion dollar business) • MySpace (300 million users) • Communication: • Instant Messenger (~1 billion users) • News and Social media: • Blogging (250 million blogs world-wide, presidential candidates run blogs) • On-line worlds: • World of Warcraft (internal economy 1 billion USD) • Second Life (GDP of 700 million USD in ‘07) Can study phenomena and behaviors at scales that before were never possible

  3. Rich data: Networks c) Social networks b) Internet (AS) a) World wide web d) Communication e) Citations f) Biological networks

  4. Networks: What do we know? • We know lots about the network structure: • Properties: Scale free [Barabasi ’99], Clustering [Watts-Strogatz ‘98], Navigation [Adamic-Adar ’03, LibenNowell ’05],Bipartite cores [Kumar et al. ’99], Network motifs [Milo et al. ‘02], Communities [Newman ‘99], Conductance [Mihail-Papadimitriou-Saberi ‘06], Hubs and authorities [Page et al. ’98, Kleinberg ‘99] • Models: Preferential attachment [Barabasi ’99], Small-world [Watts-Strogatz ‘98], Copying model [Kleinberg el al. ’99], Heuristically optimized tradeoffs [Fabrikant et al. ‘02],Congestion [Mihail et al. ‘03], Searchability [Kleinberg ‘00],Bowtie [Broder et al. ‘00], Transit-stub [Zegura ‘97], Jellyfish [Tauro et al. ‘01] We know much less about processes and dynamics of networks

  5. This thesis: Network dynamics • Network evolution • How network structure changes as the network grows and evolves? • Diffusion and cascading behavior • How do rumors and diseases spread over networks? • Large data • Observe phenomena that is “invisible” at smaller scales

  6. Data size matters • We need massive network data for the patterns to emerge • MSN Messenger network[WWW ’08] (the largest social network ever analyzed) • 240M people, 255B messages, 4.5 TB data • Product recommendations [EC ‘06] • 4M people, 16M recommendations • Blogosphere[in progress] • 164M posts, 127M links

  7. This thesis: The structure

  8. This thesis: The structure

  9. Background: Network models • Empirical findings on real graphs led to new network models • Such models make assumptions/predictions about other network properties • What about network evolution? Model Explains log prob. Power-law degree distribution Preferential attachment log degree

  10. [w/ Kleinberg-Faloutsos, KDD ’05] Q1) Network evolution Internet • Networks are denser over time • Densification Power Law: a … densification exponent (1 ≤ a ≤ 2) • What is the relation between the number of nodes and the edges over time? • Prior work assumes: constant average degree over time E(t) a=1.2 N(t) Citations E(t) a=1.6 N(t)

  11. [w/ Kleinberg-Faloutsos, KDD ’05] Q1) Network evolution Internet • Prior models and intuition say that the network diameter slowly grows (like log N, log log N) • Diameter shrinks over time • as the network grows the distances between the nodes slowly decrease diameter size of the graph Citations diameter time

  12. [w/ Backstrom-Kumar-Tomkins, KDD ’08] Q2) Modeling edge attachment • We directly observe atomic events of network evolution(and not only network snapshots) and so on for millions… We can model evolution at finest scale • Test individual edge attachment • Directly observe events leading to network properties • Compare network models by likelihood (and not by just summary network statistics)

  13. [w/ Backstrom-Kumar-Tomkins, KDD ’08] Setting: Edge-by-edge evolution • Network datasets • Full temporal information from the first edge onwards • LinkedIn (N=7m, E=30m), Flickr (N=600k, E=3m), Delicious (N=200k, E=430k), Answers (N=600k, E=2m) • We model 3 processes governing the evolution • P1) Node arrival: node enters the network • P2) Edge initiation: node wakes up, initiates an edge, goes to sleep • P3) Edge destination: where to attach a new edge • Are edges more likely to attach to high degree nodes? • Are edges more likely to attach to nodes that are close?

  14. [w/ Backstrom-Kumar-Tomkins, KDD ’08] Edge attachment degree bias • Are edges more likely to connect to higher degree nodes? PA Gnp Flickr First direct proof of preferential attachment!

  15. [w/ Backstrom-Kumar-Tomkins, KDD ’08] But, edges also attach locally • Just before the edge (u,w) is placed how many hops is between u and w? w Fraction of triad closing edges u PA v Gnp Flickr Real edges are local. Most of them close triangles!

  16. [w/ Backstrom-Kumar-Tomkins, KDD ’08] How to best close a triangle? • New triad-closing edge (u,w) appears next • We model this as: • uchooses neighbor v • v chooses neighbor w • Connect (u,w) • We consider 25 triad closing strategies • and compute their log-likelihood • Triad closing is best explained by • choosing a node based on the number of common friends and time since last activity • (just choosing random neighbor also works well) w u v

  17. Q3) Generating realistic graphs Problem: generate a realistic looking synthetic network • Why synthetic graphs? • Anomaly detection, Simulations, Predictions, Null-model, Sharing privacy sensitive graphs, … • Q:Which network properties do we care about? • Q:What is a good model and how do we fit it? Given a real network Generate a synthetic network Compare graph properties, e.g., degree distribution

  18. [w/ Chakrabarti-Kleinberg-Faloutsos, PKDD ’05] Q3) The model: Kronecker graphs Edge probability Edge probability • We prove Kronecker graphs mimic real graphs: • Power-law degree distribution, Densification, Shrinking/stabilizing diameter, Spectral properties pij (3x3) (9x9) (27x27) Initiator Given a real graph. How to estimate the initiator G1? Kronecker product of graph adjacency matrices

  19. [w/ Faloutsos, ICML ’07] Q5) Kronecker graphs: Estimation • Maximum likelihood estimation • Naïve estimation takes O(N!N2): • N!for different node labelings: • Our solution: Metropolis sampling: N! (big) const • N2 for traversing graph adjacency matrix • Our solution: Kronecker product (E << N2):N2E • Do stochastic gradient descent P( | ) Kronecker arg max We estimate the model in O(E)

  20. [w/ Faloutsos, ICML ’07] Estimation: Epinions (N=76k, E=510k) • We search the space of ~101,000,000 permutations • Fitting takes 2 hours • Real and Kronecker are very close Degree distribution Path lengths “Network” values probability # reachable pairs network value node degree rank number of hops

  21. Thesis: The structure

  22. Thesis: The structure

  23. Part 2: Diffusion and Cascades • Behavior that cascades from node to node like an epidemic • News, opinions, rumors • Word-of-mouth in marketing • Infectious diseases • As activations spread through the network they leave a trace – a cascade We observe cascading behavior in large networks Cascade (propagation graph) Network

  24. [w/ Adamic-Huberman, EC ’06] 10% credit 10% off Setting 1: Viral marketing • People send and receive product recommendations, purchase products • Data:Large online retailer:4 million people, 16 million recommendations, 500k products

  25. [w/ Glance-Hurst et al., SDM ’07] Setting 2: Blogosphere • Bloggers write posts and refer (link) to other posts and the information propagates • Data:10.5 million posts, 16 million links

  26. [w/ Kleinberg-Singh, PAKDD ’06] Q4) What do cascades look like? • Are they stars? Chains? Trees? • Information cascades (blogosphere): propagation • Viral marketing (DVD recommendations): (ordered by frequency) • Viral marketing cascades are more social: • Collisions (no summarizers) • Richer non-tree structures

  27. Q5) Human adoption curves • Prob. of adoption depends on the number of friends who have adopted [Bass ‘69, Granovetter ’78] • What is the shape? • Distinction has consequences for models and algorithms To find the answer we need lots of data Prob. of adoption Prob. of adoption k = number of friends adopting k = number of friends adopting Diminishing returns? Critical mass?

  28. [w/ Adamic-Huberman, EC ’06] Q5) Adoption curve: Validation DVD recommendations (8.2 million observations) Probability of purchasing # recommendations received Adoption curve follows the diminishing returns. Can we exploit this? Later similar findings were made for group membership [Backstrom-Huttenlocher-Kleinberg ‘06], and probability of communication [Kossinets-Watts ’06]

  29. Q6) Cascade & outbreak detection • Blogs – information epidemics • Which are the influential/infectious blogs? • Viral marketing • Who are the trendsetters? • Influential people? • Disease spreading • Where to place monitoring stations to detect epidemics?

  30. [w/ Krause-Guestrin et al., KDD ’07] Q6) The problem: Detecting cascades How to quickly detect epidemics as they spread? c1 c3 c2

  31. [w/ Krause-Guestrin et al., KDD ’07] Two parts to the problem • Cost: • Cost of monitoring is node dependent • Reward: • Minimize the number of affected nodes: • If A are the monitored nodes, let R(A) denote the number of nodes we save We also consider other rewards: • Minimize time to detection • Maximize number of detected outbreaks A R(A) ( )

  32. Optimization problem • Given: • Graph G(V,E), budget M • Data on how cascades C1, …, Ci,…,CKspread over time • Select a set of nodes A maximizing the reward subject to cost(A) ≤ M • Solving the problem exactly is NP-hard • Max-cover [Khuller et al. ’99] Reward for detecting cascade i

  33. [w/ Krause-Guestrin et al., KDD ’07] Solution: CELF Algorithm • We develop CELF(cost-effective lazy forward-selection) algorithm: • Two independent runs of a modified greedy • Solution set A’: ignore cost, greedily optimize reward • Solution set A’’: greedily optimize reward/cost ratio • Pick best of the two: arg max(R(A’), R(A’’)) • Theorem: If R is submodular then CELF is near optimal • CELF achieves ½(1-1/e) factor approximation

  34. [w/ Krause-Guestrin et al., KDD ’07] Problem structure: Submodularity New monitored node: • Theorem: Reward function R is submodular(diminishing returns, think of it as “concavity”) S1 S1 S’ S’ R(A  {u}) – R(A) ≥ R(B  {u}) – R(B) S2 S3 Adding S’helps a lot Adding S’helps very little Gain of adding a node to a small set Gain of adding a node to a large set S2 S4 Placement A={S1, S2} Placement B={S1, S2, S3, S4} A B

  35. Blogs: Information epidemics • Question: Which blogs should one read to catch big stories? • Idea: Each blog covers part of the blogosphere • Each dot is a blog • Proximity is based on the number of common cascades

  36. Blogs: Information epidemics • Which blogs should one read to catch big stories? For more info see our website: www.blogcascade.org CELF Reward (higher is better) In-links (used by Technorati) Out-links # posts Random Number of selected blogs (sensors)

  37. CELF: Scalability Exhaustive search Greedy Run time (seconds) (lower is better) CELF Number of selected blogs (sensors) CELF runs 700x faster than simple greedy algorithm

  38. [w/ Krause et al., J. of Water Resource Planning] Same problem: Water Network • Given: • a real city water distribution network • data on how contaminants spread over time • Place sensors (to save lives) • Problem posed by the US Environmental Protection Agency c1 S S c2

  39. [w/ Krause et al., J. of Water Resource Planning] Water network: Results CELF • Our approach performed best at the Battle of Water Sensor Networks competition Degree Random Population saved (higher is better) Population Flow Number of placed sensors

  40. Thesis: The structure

  41. Thesis: The structure

  42. 3 case studies on large data Benefits from working with large data: • Q7) Can test hypothesis at planetary scale • 6 degrees of separation • Q8) Observe phenomena previously invisible • Network community structure • Q9) Making global predictions from local network structure • Web search

  43. [w/ Horvitz, WWW ’08] Q7) Planetary look on a small-world • Small-world experiment [Milgram ‘67] • People send letters from Nebraska to Boston • How many steps does it take? • Messenger social network – largest network analyzed • 240M people, 255B messages, 4.5TB data MSN Messenger network Milgram’s small world experiment Avg. is 6.2. Thus, 6 degrees of separation Avg. is 6.6! Wikipedia calls it 7 degrees of separation (i.e., hops + 1)

  44. [w/ Dasgupta-Lang-Mahoney, WWW ’08] Q8) Network community structure S • How community like is a set of nodes? • Need a natural intuitive measure • Conductance: S’ • Φ(S) = # edges cut / # edges inside • Plot: Score of best cut of volume k=|S|

  45. Example: Small network Collaborations between scientists (N=397, E=914) log Φ(k) Cluster size, log k

  46. [w/ Dasgupta-Lang-Mahoney, WWW ’08] Example: Large network Collaboration network (N=4,158, E=13,422) log Φ(k) Cluster size, log k

  47. Q8) Suggested network structure Denser and denser core of the network 100 Network structure: Core-periphery (jellyfish, octopus) Expander like core contains 60% of the nodes and 80% edges • Good cuts exist at small scales • Communities blend into the core as they grow • Consequences: There is a scale to a cluster (community) size Good cuts exist only to size of 100 nodes

  48. [w/ Dumais-Horvitz, WWW ’07] Q9) Web Projections • User types in a query to a search engine • Search engine returns results: Result returned by the search engine Hyperlinks Non-search results connecting the graph Is this a good set of search results?

  49. [w/ Dumais-Horvitz, WWW ’07] Q9) Web Projections: Results • We can predict search result quality with 80% accuracyjust from the connection patterns between the results Good? Poor? Predict “Good” Predict “Poor”

  50. Thesis: The structure

More Related