The Effects of Loss and Latency on User Performance in Unreal Tournament 2003

1. The Effects of Loss and Latency on User Performance in Unreal Tournament 2003 Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, Mark Claypool Computer Science Department Worcester Polytechnic Institute Worcester, MA, USA http://www.cs.wpi.edu/~claypool/papers/ut2003/

2. General Motivation • In 2003, despite economic downturn, games only entertainment industry to grow [“Essential Facts”, ESA - 2004] • For frequent game players, 43% play online [ESA - 2004] • First-Person Shooters a top selling genre • 11.5% [“Essential Facts”, ESA, 2003] • Unreal Tournament 2003 • Popular (1700 servers, 4400 players) [Gamespy – Oct 2003] • Award winning [“Best of Show,” E3 expo] • Studied less than Quake-derived FPS NetGames'04, Portland, OG, USA

3. UT2003 http://www.youtube.com/watch?v=j5yQQTuit4k NetGames'04, Portland, OG, USA

4. Specific Motivation – The Network and Network Games • Network games can be demanding in terms of network requirements • Capacity? (Not usually) • Latency? (Sometimes) • What parts of a FPS game … Moving? Shooting? • Quantitative effect? • Loss? (Unknown) • Retransmission will increase latency • Knowing game constraints useful for • Building better network games • Building better networks to support games (QoS) • Our goal •  Quantitative effects of loss and latency on the performance of UT2003 NetGames'04, Portland, OG, USA

5. Outline • Introduction  • Experiments  • Analysis • Conclusions NetGames'04, Portland, OG, USA

6. Methodology • Characterize fundamental user interaction components of FPS games • Design maps that exercise each type of interaction • Construct test environment for measuring the effects of latency and loss • Conduct user studies • Analyze results NetGames'04, Portland, OG, USA

7. FPS User Interaction Components • Movement • Simple (Moving in a straight line) • Complex (Running, jumping, turning…) • Shooting Precision Example Weapons High Shock Rifle, Link Gun, Lightning Gun Medium Assault Rifle, Mini-Gun, Bio Rifle Low Flak Cannon, Rocket Launcher • Movement and Shooting • Design maps for each component NetGames'04, Portland, OG, USA

8. Controlling Loss and Latency • Ethereal for traces • UT2003 is Client-Server, uses UDP, Server updates every 50 ms • NIST Net for Router  Control network characteristics to each client • Control loss [0%, 6% ] • Control latency [0 ms, 400 ms] NetGames'04, Portland, OG, USA

9. Outline • Introduction  • Experiments  • Analysis • Application Level • Network Level • User Level • Conclusions NetGames'04, Portland, OG, USA

10. Complex Movement • Flux2 map with pre-defined obstacle course • Movement not affected by loss or latency NetGames'04, Portland, OG, USA

11. Precision Shooting • CTF-Face3 map with lightning gun versus dodging human opponent • Precision shooting not affected by loss • Precision shooting is affected by latency NetGames'04, Portland, OG, USA

12. Restricted Deathmatch • Training Day map with only medium precision weapons versus bot • Movement and shooting not affected by loss • Movement and shooting affected by latency • Less significantly than shooting alone NetGames'04, Portland, OG, USA

13. Full Deathmatch • Training Day map with full precision weapons versus Bot • Movement and shooting not affected by loss • Movement and shooting barely affected by latency NetGames'04, Portland, OG, USA

14. Outline • Introduction  • Experiments  • Analysis • Application Level  • Network Level  • User Level • Conclusions NetGames'04, Portland, OG, USA

15. Network Turbulence 5 • No visible effects of loss or latency • Holds for packet sizes, inter packet times NetGames'04, Portland, OG, USA

16. Outline • Introduction  • Experiments  • Analysis • Application Level  • Network Level  • User Level  • Conclusions NetGames'04, Portland, OG, USA

17. User-Level Analysis • 75 ms latency, users noticed sluggishness • Felt play was worse • True even for unrestricted tests • 100 ms latency always noticeable • Play less enjoyable • Frustrating for precise shooting • 0-3% loss rarely even noticed • 3%+ loss sometimes noticed, but only because gun effects not always displayed NetGames'04, Portland, OG, USA

18. Conclusions • FPS games not affected by typical Internet packet loss • Loss up to 3% and not even noticed • TCP fairness at higher bitrates? • FPS games can be affected by latency • Perceived impact as low as 75 ms • Quantitative impact at 100 ms • At the network level: • Small packets and low bitrates • Turbulence consistent over all network conditions NetGames'04, Portland, OG, USA

19. Future Work? NetGames'04, Portland, OG, USA

20. Future Work • Effects of variance in latency (jitter) • User adaptation strategy • Build model for FPS interactions so can apply to different FPS games (i.e. BF1942) NetGames'04, Portland, OG, USA

21. The Effects of Loss and Latency on User Performance in Unreal Tournament 2003 Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, Mark Claypool Computer Science Department Worcester Polytechnic Institute Worcester, MA, USA http://www.cs.wpi.edu/~claypool/papers/ut2003/