CS 4700 / CS 5700 Network Fundamentals

1. Lecture 17: Network Modeling (Not Everyone has a Datacenter) CS 4700 / CS 5700Network Fundamentals

2. Wide-Area Network Research • Most research now focused on large-scale systems • Challenges: testing and evaluation • How to perform wide-area tests in a repeatable, reliable manner • ModelNet, Emulab • Challenge: understanding/capturing Internet topologies • Graph characterization: dK-series

3. Outline ModelNet dK

4. A Case for Network Emulation • Need a way to test large-scale Internet services • Peer-to-peer, overlay networks, novel protocols • Testing in the real world • PlanetLab… • Results not reproducible or predictable • Difficult to deploy and administer research software • Simulation tools • Allows control over test environment • May miss important system interactions • Emulation • Emulators subject application traffic to end-to-end bandwidth constraints, latency, and loss rate of user specified topology • Previous implementations not scalable

5. ModelNet A scalable, cluster-based, comprehensive network emulation environment

6. Design • User run configurable number of instances of application on Edge Nodes within cluster • Each instance is a Virtual Edge Node (VN) • Each VN has a unique IP address • Edge nodes route traffic through cluster of Core Routers • Equipped with large memories, modified FreeBSD kernels • Core routers route traffic through emulated links or “pipes” • Each pipe has own packet queue and queuing discipline

7. ModelNet Phases • Create • Generates a network topology as a graph • From Internet traces, BGP dumps, synthetic topology generators, etc. • Annotate graph with loss rates, failure distributions… • Distillation • Transforms GMLgraph into pipe topology • Assignment • Maps pipe topology to core nodes, distributing emulation load across core nodes • Finding ideal mapping is NP-complete • ModelNet uses greedy k-clusters assignment • For k core nodes, randomly select k nodes in distilled topology. Greedily select links from connected component in round robin

8. ModelNet Phases • Binding • Multiplex multiple VNs to each physical edge nodes • Bind each physical edge node to a core router • Generate shortest path routes between all VNs and install in core routing tables • Run • Executes target application code on edge nodes

9. Inside the Core • Route traffic through emulated “pipes” • Each route is an ordered list of pipes • Packets move through pipes by reference • Routing table requires O(n2) space • Packet Scheduling • When packet arrives, put at tail of first pipe in its route. • Scheduler stores heap of pipes sorted by earliest deadline - exit time for first packet in its queue • Once every clock tick • Traverse pipes in heap for packets that are ready to exit • Move packets to tail of next pipe or schedule for delivery • Calculate new deadlines • Multi-core Configuration • Next pipe in route may be on different machine • If so, core node tunnels packet descriptor to next node

10. Scalability Issues • Traffic traversing core is limited by cluster’s physical internal bandwidth • ModelNet must buffer up to full bandwidth-delay product of target network. • 250 MB of packet buffer space to carry flows at aggregate bandwidth of 10 GB/s with 200 ms roundtrip latency. • Assumes perfect routing protocol

11. Baseline Accuracy • Want to insure that under load, packets are subject to correct end-to-end delays • Used kernel logging to track ModelNet performance and accuracy • Results show that by running ModelNet scheduler at highest kernel priority • Packets are delivered within 1ms of target end-to-end value • Accuracy is maintained up to 100% CPU usage

12. Scalability • Additional Cores • Adding core routers allows ModelNet to deliver higher throughput • Communication between core routers introduces overhead. Higher cross-core communication results in less throughput benefit • VN Multiplexing • Higher degrees of multiplexing enable larger network emulation • Inaccuracies introduced due to context switching, scheduling, resource contention, etc

13. Accuracy vs. Scalability Reduce overhead by deviating from target network requirements Changes should minimally impact application behavior Ideally, system reports degree and nature of emulation inaccuracy

14. Scalability via Distillation • Pure hop-by-hop emulation • Distilled topology is isomorphic to target network • High per packet overhead • End-to-end distillation • Remove all interior network nodes • Collapse each path into single pipe • Latency = sum of latencies along path • Reliability = product of link reliabilities along path • Low per packet overhead • Does not emulate link contention along path

15. Time Dilation on Modelnet • The challenge • Need to emulate networks with more resources • E.g. fast CPU (20Ghz), large b/w networks (TB/s) • But only commodity machines available • Solution • Modelnet + time dilation via virtual machines • Run application inside single VMs • Slow down time inside VM • Result: everything looks faster/bigger/fatter • More CPU cycles/time, packets/time, disk I/O /time

16. How It’s Done • Must isolate VM from outside measures of time • Time based on shared data structure provided by VMM • Scale data structure by a Time Dilation Factor (TDF) • Also scale hardware timer by TDF • How do we scale only some resources? • Slow the others backdown!! • Example: speed upnetwork by TDF=10 • B/w increases by 10, but delay dec by 10So inc delay by 10 Node Mgr Local Admin VM1 VM2 VMn … Virtual Machine Monitor (VMM)

17. ModelNet Summary • ModelNet, antithesis of PlanetLab • Testing of unmodified applications • Reproducible results • Experimentation using broad range of network topologies and characteristics • Large scale experiments (thousands of nodes and gigabits of cross traffic) • Can scale to emulate non-existent resource levels • But what if you want real deployment on-demand? • Emulab / NetBed

18. Emulab / NetBed • A shared configuration on-demand testbed • What if you don’t have your own cluster • What if you need to test specific environments/HW? • What if you need this in 5 mins? • Emulab / NetBed • Hardware: 328 PCs, high speed Gb Cisco switches • Software: OS-loader and manager via web interface • Wipe all disks, load OS-images, configure routers in <2 mins • Reboot and give ssh access

19. Emulab Web Interface

20. Outline ModelNet dK

21. Importance of Network Topology • Access to real-world network topologies is vital for research • New routing and other protocol design, development, testing, etc. • Analysis: performance of a routing algorithm strongly depends on topology • Generation: empirical estimation of scalability • Network robustness, resilience under attack, worm spreading, etc.

22. Network Topology Research DynamicTopologies Static Topologies

23. Trade Secrets • Unfortunately, large scale network topologies are often proprietary • Think about BGP • ISPs want to hide their internal topology • Real datasets are rare • Small scale • Out of date • Static (i.e. not dynamic)

24. Towards Synthetic Topologies • Question: can we use graph models to capture real network topologies? • Fit a model to a real topology • Use a generator to produce synthetic topologies that are similar, but not identical to the real topology • Benefits • Privacy – synthetic graphs are not proprietary • Randomization – produce an infinite number of stochastic snapshots • Scalable – generator can produce similar topologies of any size

25. Important Topology Metrics Degree distribution Clustering Assortativity Distance distribution Betweennessdistribution Problems • No way to reproduce most of the important metrics • No guarantee there will not be any other/new metric found important

26. The Approach Look at inter-dependencies among topology characteristics See if by reproducing most basic, simple, characteristics, we can also reproduce all other characteristics, including practically important Try to find the characteristic(s) that define all others Key Observation: Graphs are structures of connections between nodes

27. Definition of dK-distributions dK-distributions are degree correlations within simple connected graphs of size d • For example • 1K distributioncorrelations between node degree distribution • 2K distributioncorrelations on joint node degree distribution • 3K distributioncorrelations on clustering coefficient

28. An Example of dK • xKis distribution of subgraphs with particular degrees • dK-1 describes node degree distribution • dK-2 describes joint node degree distribution • dK-3 captures clustering coefficient dk-0: average degree=2 dk-1: P(1)=1, P(2)=2, P(3)=1 dk-2: P(1,3)=1, P(2,2)=1, P(2,3)=2 dk-3: P(1,3,2)=2, P(2,2,3)=1

29. Nice properties ofdK-series Constructability: we can construct graphs having properties Pd (dK-graphs) Inclusion: if a graph has property Pd, then it also has all properties Pi, with i < d (dK-graphs are also iK-graphs) Convergence: the set of graphs having property Pn consists only of one element, G itself (dK-graphs converge to G) Guarantees that all (even not yet defined!) graph metrics can be captured by sufficiently high d

30. Inclusion and dK-randomness Given G 2K 2K-random 1K-random nK 1K 0K-random 0K

31. How Do We Generate Graphs? • A number of different approaches • Stochastic • Pseudograph • Matching • Rewriting • Some are extensible to d=3, others are not • New research proposed d=2.5, to make generation tractible

32. Stochastic approach • Classical (Erdos-Renyi) random graphs are 0K-random graph in the stochastic approach • Easily generalizable for any d: • Reproduce the expected value of the dK-distributions by connecting random d-plets of nodes with (conditional) probabilities extracted from G • Best for theory • Worst in practice

33. Pseudograph approach Reproduces dK-distributions exactly Constructs not necessarily connected pseudographs Extended for d = 2 Failed to generalize for d > 2: d-sized subgraphs start overlap over edges at d = 3

34. Pseudograph details k2 k1 k1 k3 k1 k2 k1 k4 1K • dissolve graph into a random soup of nodes • crystallize it back 2K • dissolve graph into a random soup of edges • crystallize it back k1 k1-ends

35. dK-Randomizing Rewiring • Can generate random graphs from original • Generalizes to any d • But cannot generate desired graph from dK-distributions

36. Algorithms • All algorithms deliver consistent results for d = 0 • All algorithms, except stochastic(!), deliver consistent results for d = 1 and d = 2 • Both rewiring algorithms deliver consistent results for d = 3 • Eventual choice • Use pseudograph to construct 1K graphs • Use targeted rewriting to build higher d graphs

37. Skitter Scalar Metrics

38. HOT Scalar Metrics

39. HOT 0K HOT 0K True HOT Graph

40. HOT 1K HOT 1K True HOT Graph

41. HOT 2K HOT 2K True HOT Graph

42. HOT 3K HOT 3K True HOT Graph