1 / 40

From Airplanes to Elephants (The Pac-10 meets Internet2)

Learn about the use of video in coaching, the video exchange process in college sports, and the challenges faced in transferring large video files between schools. Discover how Internet2 has revolutionized file exchange in the Pac-10 conference.

eddings
Download Presentation

From Airplanes to Elephants (The Pac-10 meets Internet2)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. From Airplanes to Elephants(The Pac-10 meets Internet2) Chris Thomas Sr. Network Engineer UCLA Office of Information Technology chris.thomas@ucla.edu April 25, 2006

  2. Introduction (“Bring it on!”)

  3. Use of Video in Coaching • Essential to any modern sports program • It’s not your father’s athletic department • Football is the largest sports video user today • Other sports interested • Multi-purpose (officiating, recruiting, coaching) • No In-person scouting (NCAA) • Schools exchange video

  4. Video Exchange Process • Each school tapes own football games • Broadcast quality video equipment • Captured to disk as DV-25 (25 Mbit/sec uncompressed video stream) • Typical game is 18 gigabyte video file • Loaded into video editing system • Manually index start & end of each play

  5. (Example Video)

  6. Process (cont) • Early each Sunday • Put all season’s games (on disks) into a box • Go to airport (may not be close), send box to next opponent, wait until his box arrives • Return to office, import and mark plays, distribute to coaches • Coaches expect video by Sunday evening • Repeat next weekend with next opponent • Obvious opportunity (FTP)

  7. The Challenge • Exchange multiple 18 GB files between ten schools every weekend (FTP) • Windows-based systems (and users) • File transfer is only one step in a process • Time-constrained • Dependable (predictable) • People interested in solution rather than networks

  8. I2 File Exchange Pilot Results • Began with Fall 2005 football season in Pac-10 • Five participants in pilot • UCLA, USC, Stanford, Washington, Norte Dame • Used BBFTP (freeware) • http://doc.in2p3.fr/bbftp/ • UNIX server with UNIX & Windows clients • Selected on basis of performance on simulated network • Central server model (run by UCLA) • Alternative is peer to peer

  9. Central Server vs. Peer-to-Peer • BBFTP server runs on UNIX only • LINUX sender faster than Windows sender • Server activity is mostly sending • Upload a game only once, not once/week • Talk to same server every week • Expertise on one end of every connection • Get full benefit of your link speed

  10. Results (cont) Two schools with 1Gb/s local connections, three with 100 Mb/s local connections: Uploads:         65 games Downloads:      160 games Total gigabytes transferred:    4100 GB Largest file transferred:              62.7 GB

  11. How long is typical transfer? • 100 Mb/s – 22-26 min per game • Gigabit – 6-9 min per game • 10 Gigabit – 15 seconds!

  12. Results (cont) • Initial glitch in BBFTP client (Cygwin) • After correction (4th week), essentially 100% success • Pleasant surprise (no drops, no garbles) • All 10 conference schools are passionate about participating next season • “Saved me an incredible amount of time” • “Coaches got video 12 hours earlier” • “I got to spend Sunday mornings with my kids”

  13. Results (cont) • Mostly, very large file transfer over I2 works well • 4 TB on schedule, without incident • Long term stability? • Elimination of courier costs makes participation by other sports feasible • Saving $10K/yr offsets a good part of I2 membership fee

  14. “What works? What doesn’t?” • My comments are meant as observations andnot as criticisms • Project impossible without Internet2 • What is acceptable use of Internet2? • “Academic Use”? • Not fully appreciated by all implementers

  15. WW? WD? (cont) • High-performance networking expertise not always readily available to end-user • “Big numbers are at your end.” • We need to build local expertise, and make sure the right people find it • Biggest load your campus has seen • “Network police” (exception monitoring) • What is message to our user communities? • You WILL debug campus network problems

  16. Why doesn’t FTP go fast? • Network speeds (slowest link governs) • Speed of light (fast but not infinite) • Network congestion (other traffic) • PC hardware & software

  17. WW? WD? (cont) • Abilene is a fine backbone • But you don’t connect directly (most cases) to Abilene • Slowest regional network links are 100 times slower than Abilene and fastest regional networks • 10 Gb down to OC3 (155 Mb/s) • Slow (OC3) links are often heavily congested • 0.5% busy on 10G is 50% busy on 100M

  18. WW? WD? (cont) • Effective BW of Internet2 is overstated for non-aggregate flows • Two of ten Pac-10 schools are OC3 limited • Reason is almost always cost • Internet2 was formed to make HP networking between universities universal • We’re not there yet • Firewalls & other campus bottlenecks

  19. WW? WD? • Firewalls • Hacker legacy  Performance-crippled networks • Significant problem now – imposes limits on high-speed networking in many places • People installing firewalls often have no idea of performance impact • Some inappropriate forces (grant applications) • Valid protocols broken: PMTU, ECN, et al • Firewall-free VLANs for applications requiring high-performance networking is a possible solution

  20. “Speed of Light” Issue

  21. TCP Window Size • “Ping-pong” • Due to speed of light, 15 serves/second • Need to have a lot of balls in flight at once (500+) • Controlled by “window_size” connection parameter • Windows default wsize 1000x too small (UNIX is “only” 100x too small) • Hard cap on maximum pps / transfer speed • Independent of link speed • Set “window scaling” in registry to ON (RFC1323) to permit large window( > 64KB) • Up to software how much window it uses

  22. Key Registry Settings

  23. Receive Windows for 1 Gbps 1 MB 64KB limit is 32 miles 2 MB 3 MB 4 MB 5 MB Dykstra, SC2004

  24. 622Mbps 100Mbps 45Mbps Maximum TCP/IP Data RateWith 64KB window Dykstra, SC2004

  25. Congestion Control • Original TCP lacked ability to deal effectively with congestion (other traffic, speed mismatch) • Need to find receiver’s maximum speed • TCP Reno, congestion window • Turns out TCP Reno’s fix is overkill • Only some versions of UNIX have optimum congestion control today • Big throughput advantage with LINUX sender • CW is invisible (Web100 kernel patch) • Selective ACK matters

  26. PC Hardware Matters for FTP! • At 100 Mb/sec, typical recent PCs can keep up with network • At gigabit speeds, most PCs can’t go fast enough and become the bottleneck • Use iperf to measure • http://dast.nlanr.net/Projects/Iperf/ • Warning: Doesn’t include effect of disk speed • Warning: Make sure to use large windowsize • Warning: There are “better” tools

  27. PC Hardware (cont.) • Disk speed • Most likely Gb bottleneck. Single disks go about 55 MB/sec. A Gb network is 125 MB/sec (2.5x). • Older disks can be 20 – 30 MB/sec • http://www.simplisoftware.com/Public/index.php?request=HdTach • (Some) disk arrays are fast • Watch RAID-5 write speeds – can be 10% of read speed • TIP: pair of new WD Raptor II in RAID-0 stripe will do Gb

  28. PC Hardware (cont.) • Motherboard / Bus • Look for PCI Express bus & slots • PCI is just too slow • Affects both NIC and disk speed • CPU • 2.5 Ghz good (mono-core P4) • Memory • Usually not an issue (512 MB)

  29. PC Hardware (cont.) • Network Card (NIC) • Large onboard buffers (e.g., 96KB) • TCP offload • For future, jumbo frames (9000 bytes) • Highly recommend: SysKonnect SK-9S21 / SK-9E21 (about $100) • Even if you have a Gb port on the m’board, adding an SK card may help (2x)

  30. PC Operating System • Anti-virus software typically has a major impact (~50%) on disk write speed • Dual copy • Temporarily disable or configure to bypass exchange directory. • Test performance under real conditions • Real networks have delay, packet duplication & reordering, and even packet loss • Build and use a (cheap) network simulator

  31. PC Operating System (cont.) • High-speed TCP & FTP development is ongoing • Done by Computer Science departments on UNIX • Done at physics institutes on LINUX machines (10G) • LINUX is actively involved in high-performance TCP • Windows XP TCP Reno is > 5 years behind current (2.6.12+) LINUX in the critical area of congestion control. • Can make a major difference on busy links (sender side matters)

  32. FTP application • Use of TCP window • Can it use large (~5MB) window? • Client-side specified? • Multiple streams • Conventional wisdom doesn’t seem to hold; Optimum varies with link quality; 4-5 good • Warning: XP SP2 limits half-open connections to 10; others wait (event log ID 4226) • Encryption • UID/PW = Yes!! Data = No

  33. Jumbo Frames • Ethernet requires small pieces for network transmission • 1500 bytes -- Called MTU Size (Maximum Transmission Unit) or Frame • Hasn’t changed in 25 years (speeds have!) • Larger pieces, called Jumbo Frames, are more efficient (faster) • Internet2 Jumbo size is 9000 bytes • Traditionally, throughput scales with MTU (6x) • 2x – 2.5x is more typical

  34. WW? WD? • Jumbo frames are still a “future objective” in most cases • Every switch and router in path must support jumbo frame size • Abilene and up-to-date regional networks support jumbo frames today • Campus networks mostly don’t

  35. Network Simulator • Not $$$ Smartbits™ • Linux PC • Any basic PC with two network interface cards • NISTNet (Linux 2.4) • http://snad.ncsl.nist.gov/itg/nistnet/ • Netem (Linux 2.6) • http://linux-net.osdl.org/index.php/Netem

  36. Simulator

  37. WW? WD? • Ubiquitous 100 isn’t here yet • Lack thereof is impacting deployment of new apps with high-end requirements • File exchange testing with ~25 schools • 10 Mb/s connection seems standard for Athletics to campus backbone • With 100M/1G, 30-40Mb/s throughput is typical

  38. WW? WD? • Recurring issue: “Transfer is slow. Is bottleneck the PC or the network?” • I’d like to see I2 maintain a “connectivity database” • I’d like to see every I2 member running: • “Public” iperf server • “Public” pathrate / pathload server • http://www.pathrate.org/ • My current solution: ship known PC around to each client

  39. WW? WD? • Know your own network • What file transfer rates are realistic from your campus to another? • Whom do users get referred to when they call NOC with performance issue? • Make sure your (potential) users have access to campus expertise • “Be Galileo, not Aristotle”

  40. Thank you!Questions?

More Related