Detailed plan - UVA

1. Detailed plan - UVA • Dynamic circuit setup/release • Equipment to purchase • What we plan to implement • Wide-area circuits • High-throughput transport protocol item • Support for delay-controlled transport protocol • Support for router disconnect Malathi Veeraraghavan mv@cs.virginia.edu

2. Already purchased • Two 64-bit 100Mhz bus PCs with GbE • One Cisco 15454 unit • 10/100 Mbps Ethernet card – 12 ports • 1 OC3 card – 4 ports • STS-1 level crossconnect card • Control card with release 4.0 software • Augment 15454 with • OC48 card • GbE card

3. Routing decision PC Routing decision Eth. Sw. PC FTP FTP Signaling Signaling NIC I NIC I TCP TCP Sig ST NIC II NIC II ST Ethernet Ctrl XC OC3 A simple Item I demo • FTP sessions starts on TCP connection through NIC1 • Signaling module uses TL1 to set up the circuit • Large file transfer through NIC II using ST transport protocol • Routing decision based on file size • if large enough, attempt circuit setup

4. Software architecture • End host software modules • Develop each module separately and perform “unit” tests • System integration & testing File transfer app. Remote visualization app. High-throughput transport protocol Middleware Database: name, gateway, authentication Delay-controlled transport protocol Routing decision Signaling

5. For a wide-area test • To test this software on a wide-area basis, we will need additional software • Multiple 15454s • need routing database • need inventory database • Obtain software from Canarie or Starlight teams and demo centralized solution

6. Value limited • In implementing centralized solution • Others are already testing this solution • Not scalable • Utilization and hence cost implications • Conclusion: Explore distributed solution

7. Equipment to purchase • One more PC with GbE card, 64-bit PCI 133Mhz bus and high-end disks • Two Cisco 15454s with GbE and OC48 cards • Three SONET crossconnects with UNI-N/NNI software • Unfortunate that 15454 only implements UNI-C, and not UNI-N or NNI • This means to set up a crossconnection through the 15454, TL1 is the only choice. • Hence to explore distributed solution, we need crossconnects with UNI-N/NNI

8. PC 15454 XC Signaling software – option 1 • Provision GbE/EoS circuits through 15454s using TL1 commands a priori • Signaling software at end host generates UNI-C RSVP-TE Path messages • Leverages routing and inventory data located at XCs PC Ethernet switch PC 15454 15454 SONET XC with UNI-N/NNI SONET XC with UNI-N/NNI

9. TL1 TL1 Trigger UNI-C message Database: map dest. IP addr. to 15454 address Signaling software – option 2 • Mixed information on Cisco’s plans to implement UNI-N/NNI – best date heard: first half of 2005 • Late for our Yr. 1 demo PC Ethernet switch PC 15454 15454 SONET XC with UNI-N/NNI SONET XC with UNI-N/NNI

10. For SONET XC purchase • To get competitive bids from equipment vendors, team up with: • Guy Almes, Internet 2 • Bill St. Arnaud, CA*net 4 • Tom West, NLR • Bill Wing, ESnet

11. Wide-area circuits • Choices widening • Internet2 placing a 15454 in MANLAN facility • 10Gbps Lambda between NY and Amsterdam planned to be moved from Abilene router to MANLAN TDM node • CUNY connected by NYSERnet dark fiber to MANLAN site and CA*4net site • Allows us to place one PC with our software at CUNY and ship another software module with/without PC to a prof. in Chicago or Amsterdam to test our software long-distance

12. Wide-area circuits • Two options: • Ship equipment (PCs, 15454s, XCs) to organizations already connected by high-speed circuits and test our software • Pay for high-speed circuits to interconnect our four institutions • Two tests: • Application, middleware, transport • Dynamic circuit setup/release

13. Documentation plans • Overall architecture – Feb. 28, 03 • Detailed specifications • Signaling module – March 31, 03 • Routing decision module – April 30, 03 • Database – March 31, 03

14. Detailed plan - UVA • Dynamic circuit setup/release • Equipment to purchase • What we plan to implement • Wide-area circuits • High-throughput transport protocol item • Support for delay-controlled transport protocol • Support for router disconnect Malathi Veeraraghavan mv@cs.virginia.edu

15. Lessons learned from demos • Losses not only due to link errors • Losses occur when receive buffer overflows • With rate control based flow control, set circuit and sending rate to the receive rate • Nevertheless, if other tasks are scheduled on the receive PC, the receive rate is not a guaranteed steady rate

16. Follow-up steps: Step 1 • Investigate receive buffer overflow losses • if window based flow control is used, why is the sender not just shutdown when the receive buffer is full? • if timed correctly, should have idle circuit rather than losses • use tcpdump to analyze

17. Step 2 • Work with application-level transport implementation to achieve the desired set of features for memory-to-memory transfers • e.g., retransmissions on circuit unless needed at the end, etc.

18. Step 3 • Disk-to-disk: • Stay with application-level transport impl. • But use zero-copy patch of Linux • it removes need for copy from kernel memory to application memory • sendfile system call • is there a corresponding recvfile call? • Achieve max. rate possible with IDE disks or high-end disks • 135MB/sec with dual channels > 1Gbps

19. Step 4 • Test kernel-level transport implementations • Net100 implementation of TCP can be tuned to run with no congestion control • How about NAKs and rate-based flow control? • Retransmissions at the end on TCP/IP path

20. O/S bypass implementation • This requires implementation of ST on processor/FPGA on Ethernet card • If ST is run on top of Ethernet, then it is not an OS bypass implementation • Ethernet driver moves payload from NIC to kernel memory • This is a step beyond kernel-level impl! • Leave task for a DOE proposal with LANL for 10GbE

21. Equipment for transport protocol implementations • Already purchased: Two 64-bit 100Mhz bus PCs with GbE • Augment with • High-end disks • Purchase one more PC with high-end disks

22. Detailed plan - UVA • Dynamic circuit setup/release • Equipment to purchase • What we plan to implement • Wide-area circuits • High-throughput transport protocol item • Support for delay-controlled transport protocol • Support for router disconnect Malathi Veeraraghavan mv@cs.virginia.edu

23. Plans for measurements • Purchase a high-end workstation with large disk space to ship to NCSU for data collection • Remote visualization application • To determine if a low-speed (10 or 100Mbps) bidirectional circuit can be established and left open even during think times • Establish high-speed unidirectional circuit for large downloads on-demand

24. Detailed plan - UVA • Dynamic circuit setup/release • Equipment to purchase • What we plan to implement • Wide-area circuits • High-throughput transport protocol item • Support for delay-controlled transport protocol • Support for router disconnect Malathi Veeraraghavan mv@cs.virginia.edu

25. Good news • Just heard that we won an Internet2/Cisco grant of two Cisco 12000 series routers • Loan for one year • Renewable • Will be shipped soon after Thanskiving

26. Plans for personnel • Senior graduate student: Xuan Zheng • Offered RA to a new PhD student starting from the Spring semester • Will select two more MS/Phd students from CS students admitted in Fall 2003 • Advertise post-doc position • Created web site