Pulsar II Hardware Overview Jamieson Olsen, Fermilab 14 April 2014 - PowerPoint PPT Presentation

tracey
pulsar ii hardware overview jamieson olsen fermilab 14 april 2014 n.
Skip this Video
Loading SlideShow in 5 Seconds..
Pulsar II Hardware Overview Jamieson Olsen, Fermilab 14 April 2014 PowerPoint Presentation
Download Presentation
Pulsar II Hardware Overview Jamieson Olsen, Fermilab 14 April 2014

play fullscreen
1 / 21
Download Presentation
Pulsar II Hardware Overview Jamieson Olsen, Fermilab 14 April 2014
193 Views
Download Presentation

Pulsar II Hardware Overview Jamieson Olsen, Fermilab 14 April 2014

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. Pulsar IIHardwareOverviewJamieson Olsen, Fermilab14 April 2014

  2. Outline • ATCA Overview • Pulsar II Hardware • Front Board • Rear Transition Module • Mezzanine • Data Transfer Options • Pulsar IIa Prototype Results • Conclusion Pulsar II Hardware Overview / J.Olsen

  3. ATCA Overview • Mature telecom standard • IPMI Protocol • Hot swap, redundant modules • Fully instrumented • High power: 300-450W per slot • Front to back airflow • Option for vertical airflow • Boards measure 8U x 280mm • User defined Zone-3 for RTM I/O • 2 to 14 slots per shelf • Full mesh backplane option • 40 Gbps per channel (>7 Tbps) • 100 Gbps per channel coming soon ASIS ATCA shelf at Fermilab. This is a 40G full mesh backplane rated for 450W per slot. Pulsar II Hardware Overview / J.Olsen

  4. Pulsar IIb Front Board • Xilinx Virtex 7 FPGA • XC7VX415T – XC7VX690T • Up to 80 GTH transceivers • 40 for RTM • 28 for Fabric • 12 for Mezzanines • Four FMC Mezzanines • 35W each • LVDS + SERDES up to 60 Gbps • Compatible with LAPP IPMC module • 10/100/1000 Base Interface Port • Backplane clock distribution Pulsar II Hardware Overview / J.Olsen

  5. Block Diagram RTM Optical Transceivers Full Mesh Fabric Interface Dual 48VDC and platform Management (IPMI) Pulsar II Hardware Overview / J.Olsen

  6. Programming the FPGA • JTAG bus driven by the IPMC • Mux to automatically select the Xilinx programmer cable • FPGA also has SPI flash attached • “Indirect” Flash programming • Time to program? • XC7V690T = ~100 Mbits Pulsar II Hardware Overview / J.Olsen

  7. IPMC Module (LAPP) • Dual ARM microcontrollers • IPMI, I2C sensors, AMC/RTM, LEDs, etc. • General User I/O, JTAG, 100BASE-T Ethernet Pulsar II Hardware Overview / J.Olsen

  8. Pulsar IIb Power Distribution 48VDC Yellow = LDO Linears Orange = Switchers, GE/Lineage DLYNX with PMBus Interface Pulsar II Hardware Overview / J.Olsen

  9. Rear Transition Module • Fiber optic transceivers • 8 QSFP+ • 6 SFP+ • Up to 380 Gbps • Intelligent FRU • µC monitors optical modules • Hot Swap • PICMG 3.8 “Zone-3A” compliant Pulsar II Hardware Overview / J.Olsen

  10. Test Mezzanine Card • Kintex XC7K160T FPGA • 4 SFP+ Transceivers • 6.6 Gbps GTX • 128MB DDR3 • Socket for testing ASICs • FPGA Mezzanine Card (FMC) connector is compatible with FPGA development boards • 74 mm x 149 mm Pulsar II Hardware Overview / J.Olsen

  11. System Integration • All ATCA platforms support a dual star Base Interface network • 10/100/1000BASE-T Ethernet • Used for slow controls, downloading firmware, etc. • COTS “switch blades” support separate networks on the Base and Fabric Interfaces • Pulsar IIb supports up to 40G Ethernet Fabric connections to the switch • We are testing the IPBus firmware now • At Fermilab we are using the Emerson F125 switch (10G Ethernet) Pulsar II Hardware Overview / J.Olsen

  12. Data Sharing • The full mesh backplane is a natural fit for triggers where hits are shared across tower boundaries • Particularly true if data sharing is irregular • Effectively blurs the distinction between FPGAs • First Application: FTK Data Formatter • 16 φ x 4 η towers • 32 boards in 4 crates http://www.youtube.com/watch?v=wgDbm_izOXk Pulsar II Hardware Overview / J.Olsen

  13. Time Multiplexed Transfers • The full mesh backplane is also useful in systems which consist of many parallel processing engines • In this example • 10 input boards • 40 Processor Mezzanines • 1 “Gateway” board • 1 output board Pulsar II Hardware Overview / J.Olsen

  14. Multipath Transfers • If the firmware supports packet retransmission then there are many ways to get from A to B • Effective use of under-utilized backplane channels • Must account for additional latency, however. • Currently we are simulating firmware to support “route through” packet handling. Pulsar II Hardware Overview / J.Olsen

  15. Pulsar IIa Prototype • Designed and tested in 2013 • Dual Kintex K325T FPGAs • 10 Gbps GTX Transceivers • Many Unknowns: • First ATCA board design • First FMC mezzanine design • First Xilinx 7-Series FPGA design • First high speed design using high performance PCB materials Pulsar II Hardware Overview / J.Olsen

  16. Pulsar IIa Bench Testing • A single slot “Mini Backplane” was developed at Fermilab • Provides power, a base interface RJ45 port, and loopback for all backplane channels • IPMC microcontroller software was created to support Telnet and FTP access • Download firmware images to SDHC flash memory card • Board control via command line • FPGA local bus is working well at 10 Gbps • Backplane and RTM loopback tests are working well at 6.25 Gbps • 10 Gbps operation is possible, but requires fine tuning GTX settings 6.25 Gbps 10 Gbps Pulsar II Hardware Overview / J.Olsen

  17. Pulsar IIa Crate Testing • Our first ATCA shelf is 10G • 4 x 2.5 Gbps • ELMA rated for ~4 Gbps • Our crate level testing has demonstrated BER of 4.2E-17 @ 6.5 Gbps • No apparent degradation across length of backplane • Tested communication with F101 and F125 switch blades • Recently we acquired an ASIS 40G shelf and tests are ongoing Seven Pulsar IIa boards installed in our ELMA 10G ATCA shelf at Fermilab Pulsar II Hardware Overview / J.Olsen

  18. Pulsar IIb Layout • Incorporate everything we learned from designing and testing the Pulsar IIa • The goal is to have large receiver margins and wide open eyes at 10 Gbps and beyond • Keep high speed traces direct and clean (“less is more” philosophy) • Compensate for every connector, via, and discrete component in the signal path • At 10 Gbps signal transitions are 10-20ps, so every layer transition and stub matters! • Work closely with the PCB manufacturer to develop a stackup and trace geometry that meets performance requirements and can be manufactured! • 14 Layers • Nelco N4000-13 EP SI materials Pulsar II Hardware Overview / J.Olsen

  19. Conclusion • The ATCA platform is a natural fit for many HEP applications • Excellent bandwidth, power, and telecom reliability • Many interesting data transfer options • Our first ATCA prototype board works quite well • GTX transceivers are solid at 6.25 Gbps, some tuning needed to get to 10 Gbps • The Pulsar IIb prototype boards are being fabricated now Pulsar II Hardware Overview / J.Olsen

  20. Backup Slides Pulsar II Hardware Overview / J.Olsen

  21. Pulsar IIb FPGA Power Pulsar II Hardware Overview / J.Olsen