1 / 17

Panda Collaboration Meeting: FPGA-based Compute Nodes for High-Speed Data Acquisition

This article discusses the motivation and features of FPGA-based compute nodes designed for high-speed data acquisition in the Panda Collaboration. Topics covered include the ATCA platform, fabric and base interface, IPMI system, remote upgrade capabilities, and Ethernet performance. The text language is English.

josefinea
Download Presentation

Panda Collaboration Meeting: FPGA-based Compute Nodes for High-Speed Data Acquisition

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Panda Collaboration Meeting FPGA-based compute nodes • Motivation • Compute Node Features • ATCA • Fabric Interface • Base Interface • IPMI system: IPMB i2c bus • Remote Upgrade • Ethernet Performance • Summary / Outlook

  2. Panda Collaboration Meeting Motivation • Interaction rates of the order of 10MHz typical event sizes 4 to 8 kB.  raw data rates 40GB/s - 200 GB/s • high flexibility and selectivity • continuously sampling data acquisition • Event selection in programmable processing units • Connections via high speed networks • (1 Gbps)

  3. Panda Collaboration Meeting Main Features Universal highly configurable and scalable hardware platform for multiple applications • Large Compute Power • 4 Virtex-4 FPGAs with local DRAM • Total Bandwidth ~ 26Gbps • 13x RocketIO to backplane • 4x Gbit Ethernet • 4x Optical Links • 2 Embedded PowerPC in each FPGA • Slow control • Firmware update • ATCA compliant

  4. Panda Collaboration Meeting ATCA • Advance Telecomm Computer Architecture • PICMG standard • 19” Rack: 14 slots – 8U • Air flow cooling -- fans • Rough Mechanics, • Shelf management • Shelf Manager • IPMI Bus (i2c) • IPMI Controller on each board • Power ~ 200W/slot • Full mesh backplane • High Speed differential lines

  5. Panda Collaboration Meeting ATCA (2) Full mesh fabric interface. Main data transport channel Dual Star – Base interface Management Communications ZONE 1: Power and IPMI Bus

  6. Panda Collaboration Meeting Each node is connected to each other High Speed differential lines (up to 10gbps) 13 RocketIO links 1 link to any other node 1.25 Gbps Total bandwidth ~ 16Gps Data Transport Full Mesh Fabric Interface

  7. Panda Collaboration Meeting Dual Star Base Interface • Basic channel for configuration • 100Mb Ethernet • Firmware updates • Normal CPUs as hubs

  8. Panda Collaboration Meeting Shelf Manager • ATCA shelf have an intelligent manager • Collects status information of the crate • Power consumption • Cooling capacity • Power Available • Negotiates power usage of every board

  9. Panda Collaboration Meeting Shelf Manager (2) • Communication IPMI protocol • Hardware layer: i2C • 2 serial wires: Serial Data / Serial Clock • Replicated bus to increase availability IPMB_A IPMB_B

  10. Panda Collaboration Meeting Shelf Manager (3) Example of POWER NEGOTIATION • “COMPUTE POWER PROP.” • Check that the board is there • “GET POWER LEVELS” • Up to 20 level supported • For only 2 • IDLE (P<10W) • RUN (P~200W) • “SET POWER LEVEL” • “RENEGOTIATE POWER” • Issued by the boards • Shelf manager start the negotiation

  11. Panda Collaboration Meeting Remote Upgrade M. Liu, II Physikalisches Intitut (Gießen) • Power on and boot the system normally • New firmware received via Ethernet by the Linux in the PowerPC • The Linux OS has write access to the FLASH memory - New firmware written • Reboot the system and the new HW and OS will start. When failing during the upgrading procedure, a backup mechanism is needed: • Backup firmware selected by a switch • May by the IPMI Controller…

  12. Panda Collaboration Meeting Ethernet Performance M. Liu, II Physikalisches Intitut (Gießen) PERFORMANCE MEASUREMENTS: XILINX evaluation board ML403 with a Virtex-4 FX12 • Point-to-point connection to a PC • Results: • Protocol Direction Throughput (Mbps) • TCP Board -> PC 297.80 • TCP PC -> Board 316.58 • UDP Board -> PC 375.33 • UDP PC -> Board N/A Performance bottleneck: Embedded PowerPC

  13. Panda Collaboration Meeting Compute Node

  14. Panda Collaboration Meeting Compute Node

  15. Panda Collaboration Meeting Compute Node

  16. Panda Collaboration Meeting Compute Node

  17. Panda Collaboration Meeting Summary & Outlook • Panda DAQ requirements cannot be fulfilled with normal computers • The FPGA-based compute node will provide necessary the bandwidth and computational power • Concept is settled • Layout phase already started (Beijing) • Subsystems already tested • Ethernet, Remote Upgrade • But we lack of experience with the IPMI protocol

More Related