1 / 16

The ALICE Data-Acquisition Read-out Receiver Card

The ALICE Data-Acquisition Read-out Receiver Card. C. Soós et al. (for the ALICE collaboration) LECC 2004 13-17 September 2004, Boston. Outline. Introduction Hardware Firmware Software Performance Applications Summary. Introduction: ALICE DAQ. Detector. Readout Electronics.

jerom
Download Presentation

The ALICE Data-Acquisition Read-out Receiver Card

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The ALICE Data-AcquisitionRead-out Receiver Card C. Soós et al.(for the ALICE collaboration) LECC 200413-17 September 2004, Boston

  2. Outline • Introduction • Hardware • Firmware • Software • Performance • Applications • Summary LECC 2004, Boston

  3. Introduction: ALICE DAQ Detector Readout Electronics • ALICE requirements with Pb-Pb beams • 25 GB/s aggregate BW from detectors • 2.5 GB/s aggregate event building BW • 1.25 GB/s aggregate BW to tape • About 400 Detector Data Link (DDL) • Full-duplex optical link • Data rate up to 200 MB/s • The Readout Receiver Card (D-RORC) • PCI compatible adapter • Integrates two DDL interfaces • ALICE data-acquisition software: DATE DDL SIU DDL 385 Local Data Concentrator DDL DIU D-RORC 175 Event BuildingNetwork 225 ports Global DataCollector 50 StorageNetwork Transient DataStorage 25 LECC 2004, Boston

  4. Introduction: DAQ and HLT • Some of the detectors requirefiltering or data reduction • DAQ and HLT interface • Standard DDL links • Data splitter in the D-RORC • Each D-RORC hosts two DIUs • First DIU receives detector data • Second DIU transfers the copyof the raw data • HLT data is transferredback using DDL and D-RORC DIU DIU HLT Farm H-RORC H-RORC Detector FEP FEP Readout Electronics SIU SIU SIU SIU DIU DIU DIU DIU DIU DIU D-RORC D-RORC D-RORC D-RORC LDC LDC LDC LDC Event Building Network LECC 2004, Boston

  5. Hardware JTAG interface Conf. Flash • FPGA programming anddebugging • Flash memory programming • EPC4 • Programmable via internalor external JTAG chain Electrical transceivers • Multi-rate transceivers • Serial-Parallel-Serial converters • Integrated 8B/10B endec • Clock recovery CMC interface • Standard extension I/F • About 180 user I/O Optical transceivers • 850 nm VCSEL laser • 2.125 Gbit/s • Pluggable modules Altera FPGA LVDS interface PCI 64-bit/66 MHz • APEX-E device family • EP20K400E • High-speed serial I/F • 2 inputs + 2 outputs • Purpose: detector busy • +3.3V compatible signals • Bus master enabled LECC 2004, Boston

  6. DDL interface Transmitdata path Receivedata path Controlregisters PCI core (64-bit master, memory mapped) Firmware High-speed I/F to the transceivers • PCI interface core • Handles PCI transactions • Performs PCI mastering • Receiver and Transmitter • Buffer data and initiate the DMA • Control registers • Mapped to PCI memory • Provide control interface • Provide status information • DDL interface • Performs DDL transactions • Provides DDL status 64-bit PCI or PCI-X bus, 3.3V signaling (!) LECC 2004, Boston

  7. Receiver DMA Push Buffer Descriptor #5 Buffer Descriptor #4 Buffer Descriptor #3 Buffer Descriptor #2 Software Buffer Descriptor #1 Receive Address FIFO Firmware Receive Report FIFO Receiver DMA controller DMA report #5 DMA report #4 DDL DMA report #3 DMA report #2 DMA report #1 PC memory, managed by PHYSMEM PCI bus D-RORC LECC 2004, Boston

  8. Application (e.g. DATE) D-RORC API layer D-RORC driver Physmem Linux kernel Software • D-RORC driver: Linux device driver, runtime loadable module • Finds the D-RORC cards on the PCI buses • Maps the registers into the user memory space • D-RORC API layer: collection of library routines written in C • Ensures exclusive access to the hardware using device locking • Provides simple programming I/F for higher level applications • Command line executables • Hardware identification • Reset components (DIU, SIU etc.) • Send commands, reads status • Send data blocks • Receives and checks data blocks LECC 2004, Boston

  9. Performance: Test bed • Supermicro server motherboard with dual Xeon CPUs @ 2.4 GHz • Six PCI-X slots, 4 bus segments (3+1+1+1) • Linux OS • ALICE Data-Acquisition software (DATE) LECC 2004, Boston

  10. Performance: Single channel • Bandwidth vs. block size measurements with internal and external (DDL) data source using one D-RORC channel • Steady increase until the maximum bandwidth is reached • Internal: BWmax = Fpci [MHz] x 4 [Bytes] = 264 MB/s • External: BWmax = BWddl = 206 MB/s LECC 2004, Boston

  11. Performance: Dual channel • Bandwidth vs. block size measurements with internal and external (DDL) data source using two D-RORC channel • Steady increase until the maximum bandwidth is reached • Internal: BWmax = Fpci [MHz] x 4 [Bytes] x 2 – Loss = 484 MB/s • External: BWmax = BWddl x 2 = 412 MB/s LECC 2004, Boston

  12. Performance: Dual vs. 2 Single • Bandwidth vs. block size measurement with internal data source • Different maximum (different arbitration) • Dual-channel D-RORC: BWmax = 484 MB/s • Two single-channel D-RORC: BWmax = 464 MB/s LECC 2004, Boston

  13. #4 PCI #6 #2 #3 PCI #5 #2 PCI #4 PCI #3 Controller #1 Segment #1 PCI #2 PCI #1 Performance: Six D-RORCs • Testing the fully populated PC using internal data source • Interoperability test • Measure the maximal input bandwidth Bandwidth [MB/s] Normalized Bandwidth [MB/s/Ch] LECC 2004, Boston

  14. Applications: TPC sector test • Test beam of one complete Inner Read-out Chamber (IROC) of the ALICE TPC detector (May 2004) IROC Detector LDC HLT LDC Detector LDC DDL 1 Si TelescopeTOF D-RORC D-RORCDIU VME processor CAEN VME boards DDL 2 D-RORC RCU 1 RCU 2 DDL 3 DDL 4 Fast Ethernet 10 MB/s HLT (3 PCs) HLT CASTOR 1.5 TB GDC DDL 5 3x 250 GB disk /castor/cern.ch/alice/testbeam2004/T10 LECC 2004, Boston

  15. Summary • D-RORC card has been developed as the high-speed interface between the DDL and the PCI bus • Using two integrated DDL channels • reduces the number of PCI slots • offers data paths from the detectors to the DAQ and HLT systems • Linux device driver and API library based on standard C, as well as Linux executables are available • The card has been tested thoroughly in the lab • 1 CH bandwidth = 264 MB/s • 2 CH bandwidth = 484 MB/s • 4 D-RORC on different PCI segments = 1045 MB/s • Real applications show the stability and reliability of the card LECC 2004, Boston

  16. Thank you!

More Related