1 / 19

FABRIC WP1.2 Broadband Data Path: Protocols and Processor Interface

Explore Broadband Protocols report for real-time e-VLBI applications, investigating TCP delay, UDP-based VLBI system, and congestion control protocols like DCCP. Learn about current and upcoming developments in data processing interfaces and protocols.

dianetodd
Download Presentation

FABRIC WP1.2 Broadband Data Path: Protocols and Processor Interface

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. FABRIC WP1.2 Broadband Data Path:Protocols and Processor Interface Bonn 20/09/07 Ralph Spencer The University of Manchester

  2. Contents: • Outline • WP1.2.1 Broadband Protocols • WP1.2.2 Broadband data processing interface Fabric: WP1.2 Broadband data path

  3. Outline • WP1.2.1 Protocols • Investigation of suitable protocols for real time e-VLBI in EVN context • 1 FTE funded from EXPReS, RA: Stephen Kershaw • Contributed work over last year funded by ESLEA project • Strategic document May 2006 • Protocols performance report (interim) June 2007 • WP1.2.2 Broadband Data Processor interface • Interface to e-MERLIN correlator • 4 Gbps input (from Onsala) • 4 x 1 Gbps output to JIVE (SA1, EXPReS) • 2 FTE (EXPReS+FABRIC), Johnathan Hargreaves (since Dec 2006) • Using iBOBs: Xilinx vertex 2 FPGAs • e-MERLIN station boards: Xilinx IVs. Fabric: WP1.2 Broadband data path

  4. WP1.2.1 Protocols: What’s in the report? • TCP_delay – constant bit rate data transfer over TCP • Reaction to lost packets – data delayed • Can catch up, needs large data buffers and provided link bandwidth adequate • Impractical, needs alternative protocol • VLBI_UDP • UDP based transfer system using ring buffers • Allows selective packet dropping • Implementation on PCs works, tests with correlator • Implemented on MkVAs – code diversion (JIVE/JBO) • Both work at 512 Mbps. • 1 Gbps tests….. • DCCP • Datagram congestion control • In Linux kernel, uses selectable congestion control algorithm (CCID) • Needs suitable CCID for e-VLBI • Further work needed if to be sued in eVLBI Fabric: WP1.2 Broadband data path

  5. WP1.2.1 Protocols: What’s now/next? • Work on TCP-delay completed - Stephen • VLBI_UDP ideas incorporated into Haro’s/Arpad’s code – 512 Mbps successful on Mk5A’s • Bottleneck on VLBI_UDP identified: selective packet dropping implemented (can run 1024 Mbps VLBI over 1 GE) -Simon • Work on multi-destination protocols initiated - Stephen • VSI-E implemented, trans-Atlantic tests underway -Tony • 10 Gbps tests undertaken on GEANT2 research network - Rich • Tests to Onsala being planned Fabric: WP1.2 Broadband data path

  6. 4 Gbit flows over GÉANT2 • Set up 4 Gigabit Lightpath Between GÉANT2 PoPs • Collaboration with DANTE • GÉANT2 Testbed London – Prague – London • And London-Amsterdam-Frankfurt-Prague-Paris-London • PCs in the DANTE London PoP with 10 Gigabit NICs • VLBI Tests: • UDP Performance • Throughput, jitter, packet loss, 1-way delay, stability • Continuous (days) Data Flows – VLBI_UDP and udpmon • Multi-Gigabit TCP performance with current kernels • Multi-Gigabit CBR over TCP/IP • Experience for FPGA Ethernet packet systems • DANTE Interests: • Multi-Gigabit TCP performance • The effect of (Alcatel 1678 MCC 10GE port) buffer size on bursty TCP using BW limited Lightpaths • 10 Gigabit London –New York Alcatel-Ciena Interoperability Fabric: WP1.2 Broadband data path

  7. The GÉANT2 Testbed • 10 Gigabit SDH backbone • Alcatel 1678 MCCs • GE and 10GE client interfaces • Node location: • London • Amsterdam • Paris • Prague • Frankfurt • Can do lightpath routingso make paths of different RTT • Locate the PCs in London Fabric: WP1.2 Broadband data path

  8. 4 Gbps on GÉANT: UDP Throughput • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • rx-usecs=25 Coalescence ON • MTU 9000 bytes • Max throughput 4.199 Gbit/s • Sending host, 3 CPUs idle • For <8 µs packets, 1 CPU is >90% in kernel modeinc ~10% soft int • Receiving host3 CPUs idle • For <8 µs packets, 1 CPU is ~37% in kernel modeinc ~9% soft int Fabric: WP1.2 Broadband data path

  9. 4 Gig Flows on GÉANT: UDP Flow Stability • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R Fibre • Coalescence OFF • MTU 9000 bytes • Packet spacing 18 us • Trials send 10 M packets • Ran for 26 Hours • Throughput very stable3.9795 Gbit/s • Occasional trials have packet loss ~40 in 10M - investigating • Our thanks go to all our collaborators • DANTE really provided “Bandwidth on Demand” • A record 6 hours ! including • Driving to the PoP • Installing the PCs • Provisioning the Light-path Fabric: WP1.2 Broadband data path

  10. Alcatel Buffer size: Method • Slope gives buffer size • ~57 kBytes • Classic Bottleneck • 10 Gbit/s input 4 Gbit/s output • Use udpmon to send a stream of spaced UDP packets • Measure packet number of first lost frame as function of w packet spacing Fabric: WP1.2 Broadband data path

  11. WP1.2.2 Processor Interface • University of Berkeley iBOB design (Dan Wertheimer) • 10 tested iBOBs delivered to JBO in June 2007 • Firmware being developed - Jonathan • Priority: 10 GE data transfer through CX4 connector • iBOB connects via VSI-H to EVLA/e-MERLIN station board • Prototype station board tested at Penticton- new version will be produced • Delivery of SBs to JBO expected after end of year • Fringe tests will need correlator cards – some time in 2008? Fabric: WP1.2 Broadband data path

  12. Station Board VSI VSI to ZDOK iBOB 0 Switch Switch VLBI Mk V b receivers CX4 1Gbps VSI VSI to ZDOK Station Board VSI VSI to ZDOK iBOB 0 CX4 1Gbps VSI VSI to ZDOK Station Board VSI VSI to ZDOK iBOB 0 CX4 1Gbps VSI VSI to ZDOK Station Board VSI VSI to ZDOK iBOB 0 CX4 1Gbps VSI VSI to ZDOK JBO JIVE eMERLIN CORRELATOR JBO Onsala Station Board VSI iBOB 0 CX4 4Gbps Switch Switch CX4 4Gbps iBOB 0 ADC Or fibre if > 15m VSI Connection to e-MERLIN Fabric: WP1.2 Broadband data path

  13. IBOB under test Fabric: WP1.2 Broadband data path

  14. iBOB Configured as network testing device Network PC Or Switch CX4 10Gbps up to 15m Optional second CX4 JTAG RS232 10/100 Ethernet Local PC Download FPGA firmware over JTAG Local Monitoring over RS232 Removed when firmware is stable Remote PC Remote login to network PC to run tests from JBO, Manchester or elsewhere iBOB Test Configuration Fabric: WP1.2 Broadband data path

  15. iBOB test set up Fabric: WP1.2 Broadband data path

  16. Simulink Design for Generating Bursts of UDP Packets Fabric: WP1.2 Broadband data path

  17. UDP Throughput vs. Packet Spacing • PC • Kernel 2.6.20-web100_pktd-plus • Myricom 10G-PCIE-8A-R CX4 • rx-usecs=25 Coalescence ON • MTU 9000 bytes • UDP Packets • Max throughput 9.4 Gbit/s • iBoB • Packet 8234Data: 8192+ Header: 42 • 100 MHz clock • Max rate 6.6 Gbit/s • See 6.44Gbit/s Fabric: WP1.2 Broadband data path

  18. Current status • Using Network PC to test 10Gbps capability of iBOB • Can ARP, PING and send and receive UDP packets using software running on the iBOB’s PowerPC. • 10 Gbps packets sent using FPGA hardware Next few weeks: • UDP network tests • DevelopVSI-E control protocols using Linux Next 6 months • iBOB to iBOB transmission over a network using a modified RTP packet header. Algorithms to buffer and re-order late packets in the receiver need to be developed and tested. • Develop algorithms on a Xilinx development board • to remove the e-Merlin delay model, • remove the n x 10kHz offset, • filter a 128MHz band into VLBI compatible sub-bands. • Implement on the Virtex 4 SX35 chips on the station board. Fabric: WP1.2 Broadband data path

  19. Questions? Monty Midnight Maroon Nov 2006 • Contact information: ralph.spencer@manchester.ac.uk • EXPReS is made possible through the support of the European Commission (DG-INFSO), Sixth Framework Programme, Contract #026642 Fabric: WP1.2 Broadband data path

More Related