Scalability and Accuracy in a Large-Scale Network Emulator

1. Scalability and Accuracy in a Large-Scale Network Emulator Amin Vahdat, Ken Yocum, Kevin Walsh, Priya Mahadevan, Dejan Kostic, Jeff Chase, and David Becker Presented by Stacy Patterson

2. Outline • Motivation • ModelNet Design • Evaluation • Conclusion

3. Motivation • Need a way to test large-scale Internet services • Peer-to-peer, overlay networks, wide area replication • Testing in the real world • Results not reproducible or predictable • Difficult to deploy and administer research software • Simulation tools • Allows control over test environment • May miss important system interactions • Emulation • Network emulators can subject application traffic to end-to-end bandwidth constraints, latency, and loss rate of user specified topology • Previous implementations not scalable

4. ModelNet • A scalable, cluster-based, comprehensive network emulation environment

5. Design • User run configurable number of instances of application on Edge Nodes within dedicated server cluster • Each instance is a Virtual Edge Node (VN) • Each VN has a unique IP address • Edge nodes route traffic through cluster of Core Routers • Equipped with large memories and modified FreeBSD kernels • Can emulate configured target network traffic • Core routers route traffic through emulated links or “pipes” each with its own packet queue and queuing discipline

6. ModelNet Phases • (1) Create • Generates a network topology in GML - Graph with vertices: clients, stubs, transits and edges: network links • Can be generated from Internet traces, BGP dumps, synthetic topology generators, etc. • Users can annotate graph with packet loss rates, failure distributions, etc • (2) Distillation • Transforms GMLgraph into pipe topology

7. ModelNet Phases • (3) Assignment • Maps pipe topology to core nodes, distributing emulation load across core nodes • Finding ideal mapping is NP-complete problem. Depends on routing, link properties and traffic load. • ModelNet uses greedy k-clusters assignment • For k core nodes, randomly select k nodes in distilled topology. Greedily select links from connected component in round robin.

8. ModelNet Phases • (4) Binding • Multiplex multiple VNs to each physical edge nodes • Bind each physical edge node to a core router • Generate shortest path routes between all VNs and install in core routing tables • (5) Run • Executes target application code on edge nodes

9. Inside the Core • Route traffic through emulated “pipes” • Each route is an ordered list of pipes • Packets move through pipes by reference • Routing table requires O(n2) space • Packet Scheduling • When packet arrives, put at tail of first pipe in its route. • Scheduler stores heap of pipes sorted by earliest deadline - exit time for first packet in its queue • Once every clock tick • Traverse pipes in heap for packets that are ready to exit • Move packets to tail of next pipe or schedule for delivery • Calculate new deadlines • Multi-core Configuration • Next pipe in route may be on different machine • If so, core node tunnels packet descriptor to next node

10. Scalability Issues • Traffic traversing core is limited by cluster’s physical internal bandwidth • ModelNet must buffer up to full bandwidth-delay product of target network. • 250 MB of packet buffer space to carry flows at aggregate bandwidth of 10 GB/s with 200 ms roundtrip latency. • Assumes perfect routing protocol

11. Evaluation • Core routers - 1.4 Ghz Pentium III, 1 GB memory • Connected using 1 GB switch • Edge nodes - 1 Ghz Pentium III 256 MB memory • Connected using 100 MB/s

12. Baseline Accuracy • Want to insure that under load, packets are subject to correct end-to-end delays • Used kernel logging to track ModelNet performance and accuracy • Results show that by running ModelNet scheduler at highest kernel priority • Packets are delivered within 1ms of target end-to-end value • Accuracy is maintained up to 100% CPU usage

13. Capacity • Quantify capacity of ModelNet as function of load and number of emulated hops • Tested 1-5 edge nodes • Each edge node hosts up to 24 netperf senders and 24 netperf receivers • Topology connects each sender to a receiver

14. Capacity

15. Scalability • Additional Cores • Adding core routers allows ModelNet to deliver higher throughput • Communication between core routers introduces overhead. Higher cross-core communication results in less throughput benefit • VN Multiplexing • Higher degrees of multiplexing enable larger network emulation • Inaccuracies introduced due to context switching, scheduling, resource contention, etc

16. Accuracy vs. Scalability • Reduce overhead by deviating from target network requirements • Changes should minimally impact application behavior • Ideally, system reports degree and nature of emulation inaccuracy

17. Distillation • Pure hop-by-hop emulation • Distilled topology is isomorphic to target network • High per packet overhead • End-to-end distillation • Remove all interior nodes in network • Collapse each path into single pipe • Latency = sum of latencies along path • Reliability = product of link reliabilities along path • Low per packet overhead • Does not emulate link contention along path

18. Distillation - continued • Walk-In • Preserve the first walk-in links from edges • Interior links replaced with full mesh • Does not model contention in interior • Walk-out • Extension to walk-in to support interior link contention • Preserves set of links in interior • Collapses paths between walk-out and walk-in sets

19. Evaluating Distillation • Ring topology with 20 routers interconnected at 20 MB/s each • Each router as 20 VNs • Routers partitioned into generator and reciever sets • 419 pipes shared between 400 VNs • End-to-end distallation contains 79,800 pipes • Last mile distillation preserves 400 edge links • Test distribution of bandwidth between nodes

20. Evaluating Distillation

21. Changing Network Statistics • ModelNet allows users to modify pipe parameters while emulation is in progress • User can change bandwidth, delay and loss rate of set of links • Also support for modeling node and link failures

22. Case Studies • Able to evaluate a 10,000 node network of unmodified Gnutella clients • 100 edge nodes with 100 VNs each • Extensions support emulation of ad hoc wireless networks • Broadcast communication and node mobility • CFS • Able to reproduce CFS implementation running on RON testbed • ModelNet results closely match CFS/RON in all cases

23. Case Studies • Replicated Web Services • Need to investigate replica placement and routing policies under realistic wide-area conditions • Study effects of replication on client latencies using 2.5 minute trace of requests to www.ibm.com • Adaptive Overlays • ACDC, an adaptive overlay system that tries to build routes that deliver better cost, delay or both. • 600 nodes in topology, 120 of them in overlay network • Test the behavior of the system to increasing delays between links • Results very similar to experiment performed under ns2

24. Conclusion • ModelNet provides an emulation environment that allows • Testing of unmodified applications • Reproducible results • Experimentation using broad range of network topologies and characteristics • Large scale experiments (thousands of nodes and gigabits of cross traffic)