Proposed future direction for CHEETAH

1. Proposed future direction for CHEETAH • Outline • What's our goal for the network: • eScience network or large-scale GP network? • Book-Ahead (BA) or Immediate-Request (IR)? • CHEETAH network evolution to support IR • Software modules that need to be implemented • Applications • CHEETAH/DRAGON/HOPI interconnectivity Malathi Veeraraghavan University of Virginia August 9, 2006

2. eScience or GP? BA or IR?

3. Reasons for choosing IR • Metcalfe's value statement • SCALE, SCALE, SCALE • IR seems more natural in data world • Bandwidth not as scarce as airline seats that only BA can be supported; so IR seems like a must-have • Cisco/Juniper/Sycamore control-plane implementation + community's standardization effort is "wasted" to a large extent with BA • Shouldn't at least one testbed to study whether and, if so, how, IR bandwidth-management software built into switches can be useful

4. Co-management of bandwidthon IR and BA basis • But, BA and IR cannot coexist without bandwidth partitioning • BA allows for high-BW, long-duration calls • IR calls will suffer a high call blocking rate if supported through BA scheduler

5. UVa Via Vortex/HOPI CUNY Via Nysernet/HOPI OC-192 GaTech CHEETAH wide-area network Raleigh PoP (MCNC) SN16000 via NCREN NCSU GbE/ 10GbE card OC192 card Control card End hosts OC-192 (via NLR/SLR/NCREN) ORNL PoP Atlanta PoP (SOX/SLR) SN16000 SN16000 GbE/ 10GbE card Control card OC192 card GbE/ 10GbE card OC192 card Control card End hosts End hosts ORNL

6. What CHEETAH s/w enables now. VT UVa duke Cville, VA mvstu6 UGa GbE circuit CHEETAH UNC wukong zelda2 Gatech Atlanta, GA Releigh, NC NCSU zelda4 ORNL, TN UT Once zelda2's GbE card is LOCKED in a circuit headed to mvstu6, it cannot communicate with wukong or zelda4 via CHEETAH X. Fang R. Gisiger

7. Network evolution to support IR • Current CHEETAH network only supports 10 circuits per OC192 link • IR mode inefficient if m, the link capacity in channels, is small (i.e., 10) • Recoup OC1-crossconnect capability of the SN16Ks from its current 1Gbps use • Has three advantages • supports higher m; better for IR • GMPLS standards based signaling (GbE-SONET signaling is Sycamore proprietary) • Call setup delay: 166ms for two-hop instead of 1.5sec!

8. Network evolution options • Four options: • NICs with VLAN support PLUS VLSR for SN16K with control-plane support for VLANs • 15454 with VLSR • IP router with VLSR • Ethernet switch with VLSR

9. NICs with VLAN support • Useful for web caching application since a cache may need to talk simultaneously to two other caches • In general useful for end-to-end circuits (whether router-to-router or host-to-host) • BUT, since the SN16K GMPLS software does not support VLAN mapping to multiples of OC1s, this solution is insufficient; hence the next slide

10. SN16K/VLSR at each PoP • Can write a VLSR for SN16K to support GMPLS control of VLAN-OCxx circuits and then use VLAN-capable NICs • Use hdb commands for flow-based Ethernet service, i.e., to map VLANs to OCxx circuits, but hdb software is unsupported • Beats the reason for our choice of SN16Ks (GA support of GMPLS control plane)

11. 15454/VLSR at each PoP • Make the 15454 serve as the intermediary between Ethernet NICs in hosts and SONET based SN16Ks at cheetah PoPs • Useful for end-to-end circuits (whether router-to-router or host-to-host), e.g., webcaching. • Two problems: • Needs a heterogeneous signaling procedure, Intserv based Path message and VLAN IDs as labels to a SONET based Path message and SKULM labels • VLAN ID continuity requirement • Cannot support partial-path circuits

12. IP router/VLSR at each PoP • Use OCxx SONET interfaces to connect IP router to SN16K • Connect web caches to router • Have router initiate pure SONET circuit setup • Use PBR or just ordinary routing table update to map flows to different OCxx circuits; support multiple circuits from one web cache

13. IP router/VLSR at each PoP • Can support end-to-end circuits • web caching • CDN servers • router-to-router apps: • administrator initiated increases • SNMP log monitor initiated increases • video apps at 10-15Mbps - map to one OC1 • Has the potential to support PPCs (partial-path circuits) • Place router with VLSR in enterprises at edge of GbE cheetah access link

14. Ethernet switch/VLSR at each PoP • Does not help with the problems noted in today's Gb/s circuit use of the SN16K • long call setup delays: 1.5sec • non-standard solution • high per-circuit BW • Using an Ethernet switch/VLSR at an enterprise (e.g. CUNY) requires all VLANs sharing 1Gbps CHEETAH access link to be switched to the same exit SN16K. • Even worse, m=1 if whole 1Gb/s link used for a circuit

15. Software modules required • CVLSR for IP router • Web caching & video apps • CTCP code to support multiple simultaneous flows

16. Applications • In order of preference: • web caching • video apps at 10-15Mbps - map to one OC1 • router-to-router apps: • administrator initiated increases • SNMP log monitor initiated increases • storage • Contact: Paul Sheldon, Vanderbilt, has NSF MRI to install data depots (600 TB) • CDN servers • IEEE, ACM, ComSoc web sites

17. Equipment purchase • IP routers with channelized SONET cards with GA GMPLS UNI implementation at each CHEETAH PoP • IP routers with GbE blades at enterprises • IP router in NLR McLean (fourth CHEETAH PoP) • unless MAX/Dragon router is available - then buy GbE and OC192 blades for that router • If NC 454 transponders are unavailable, purchase transponders for DC-Raleigh NLR link - since HOPI doesn't have this • Colocation costs at NLR McLean and NLR Raleigh

18. DRAGON • How many WDM channels per link? • m has to be small due to transponder costs • makes it hard to support IR • hence the interest in BA • GMPLS in Movaz switches useful only for provisioning but not BW sharing • Heterogeneous circuit setup not advisable in IR mode: • should provision fat pipes based on aggregate usage observation, not per small-BW VLAN • Does each DRAGON PoP have an Ethernet VLAN switch with VLSR?

19. Interest in using CHEETAH as part of the Southern leg for HOPI • HOPI is public-domain "commercial" • Further HOPI uses VLAN switches - small granularity! • CAC even w/o enforcement is fine if CTCP is the only conduit at end hosts - can enforce there

20. HOPI and web caching • Seems like a good match • Rick's black cloud experiment - same as web caching • Also Rick wants the "hybrid" part to be exercised - here's how with IP and VCs/circuits

21. What am I asking Rick? • Web caches at PoPs • Permission to run router VLSR software on Abilene routers that will need to update routing tables! • Will of course demo this first thoroughly on cheetah network

22. What am I asking Jerry? • CVLSR on HOPI switches - hop-by-hop IR mode • IR support - GMPLS peer model • call setup Path message between router VLSRs (in different domains) will carry final dest. IP address • Router VLSR within cheetah (HOPI) domain finds exit router for that domain and issues SONET circuit Path message with that exit router as destination • Web caches at DRAGON PoPs

23. Interconnect CHEETAH, DRAGON HOPI • Through IP routers of Cheetah & Abilene • With our IP router/VLSR combo, setup router-to-route SONET OC1 circuits via cheetah and router-to-router VLAN virtual circuits through DRAGON/HOPI. At routers, do PBR mapping for flows or just update routing tables for destination IP addresses (data-plane • This means packets go back to IP layer between two networks

24. Figure • HOPI + CHEETAH with routers in between

25. Web caching example • Example: web cache in CA to web-cache at ORNL • VLAN from Sunnyvale Abilene router through five HOPI nodes to McLean router • McLean router to ORNL router through SONET nodes