Network Management Game

1. Network Management Game EnginArslan, Murat Yuksel, Mehmet H. Gunes LANMAN 2011 North Carolina

2. Outline • Motivation • Related Work • Network Management Game (NMG) Framework • User Experiments & Results • Conclusion

3. Motivation • High demand for multimedia applications (VoIP, IPTV, teleconferencing, Youtube) • ISPs have to meet customer demand Service Level Agreement (SLA) • Network management and automated configuration of large-scale networks is a crucial issue for ISPs • ISPs generally trust experienced administrators to manage network and for better Traffic Engineering

4. Training Network Administrators • Network administrator training is a long-term process • Exposing inexperienced administrators to the network is too risky • Current practice to train is apprenticeship Can we train the network administrators using a game-like environment rather than months of years of apprenticeship?

5. Related Work • Training by virtualized game-like environment • Pilot training • Investor training • Commander training • Compeauet al. : End-user training and learning Deborah Compeau, Lorne Olfman, MaungSei, and Jane Webster. 1995. End-user training and learning. Commun. ACM 38, 7 • Chatham et al. : Games for training Ralph E. Chatham. 2007. Games for training. Commun. ACM 50, 7 (July 2007), 36-43. • Network administrator programs:Cisco Certification

6. Change link weight Framework Network Configuration Display traffic 1 3 7 2 6 Graphical User Interface Traffic traces Simulation Engine (NS-2) 4 5 Calculate new routes Block diagram of Network Management Game (NMG) components.

7. Network Simulator (NS-2) NS-2 System Configuration Output • No real time interactivity Run simulation See the results • Necessitates adequate level of TCL scripting • Not designed for training purpose

8. Simulator-GUI Interaction • Concurrencyis challenging • Run the simulation engine for a time period then animate in GUI before the engine continues • Slowdown animator – chose this approach • GUI-Engine interaction is achieved via TCP port • Animator opens a socket to send simulation traces • GUI opens a socket to send commands Sample Message: $ns $n1 $n2 2 set weight of link between n1 and n2 to 2

9. NMG Screenshot

10. User Goal • Increase Overall Throughputby manipulating link weights within a given time period B 1Mbps 1Mb/s 1Mb/s E A C D 3Mb/s 3Mbps 3Mb/s 4Mb/s

11. User Goal VIDEO

12. User Experiments We conducted 2 user experiments • Training without Mastery • No specific skills targeted • No success level obligated • Training with Mastery • Two skills are targeted to train • Success level obligated Introduction| Related Work | NMG Framework |User Experiments| Conclusion

13. Training without Mastery • 5 training scenarios • For every scenario, user has fixed 3-5 minutes to maximize overall throughput • 8 users attended • Took around 45 minutes for each user • User performance evaluated for failure and no failure cases

14. User Experiment Failure scenarios No failure scenarios Tutorial 6 7 1 2 3 4 5 • 6’ 7’ Before Training Training After Training

15. No Failure Case P-test value :0.0002 Before Training After Training 16% increase

16. Failure Case Users outperform heuristic solutions P-test value: 0.27 After Training Before Training 2.2% increase

17. Training with Mastery • Two skills are targeted • High bandwidth path selection • Decoupling of flows • 7 training scenarios 7 levels • Success level is obligated to advance next level • 5 users attended • Took 2-3 hours on average per user Introduction| Related Work | NMG Framework |User Experiments| Conclusion

18. Training with Mastery Tutorial 8 1 2 3 4 5 • 6 • 7 • 8’ Before Training Training After Training Introduction| Related Work | NMG Framework |User Experiments| Conclusion

19. Results of Training with Mastery P-test value: 0.00001 Introduction| Related Work | NMG Framework |User Experiments| Conclusion

20. Conclusion • Performance of a person in network management can be improved via our tool • 16% improvement  first user experiment • 13%- 21% improvement second user experiment • People outperform heuristic algorithms in case of dynamism in network • Targeting skills and designing specific scenarios for skills lead better training • Success level of second user training Introduction| Related Work | NMG Framework | User Experiments|Conclusion

21. Future Work • Extend for large scale networks • Extend quantity and quality of test cases • Using different metrics in addition to throughput such as delay or loss • Improve for investment based simulations (what-if scenario) • Simulate multiple link failure (disastrous scenario)

22. Thank you! For offline questions: enginars@buffalo.edu

23. Related Work • Ye et al. :Large-scale network parameter configuration using an on-line simulation framework Tao Ye, Hema T. Kaur, ShivkumarKalyanaraman, and Murat Yuksel. 2008. Large-scale network parameter configuration using an on-line simulation framework. IEEE/ACM Trans. Netw • Gonen et al. :Trans-Algorithmic search for automated network management and configuration B. Gonen, etal. Probabilistic Trans-Algorithmic search for automated network management and configuration. In IEEE International Workshop on Management of Emerging Networks and Services (IEEE MENS 2010 • Wang et al. :IGP weight setting in multimedia ip networks R. D. D. Wang, G. Li, “Igp weight setting in multimedia ipnetworks,”inIEEE Infocom Mini’07, 2007.