1 / 30

ALICE Experiment

DATE the DAQ s/w for ALICE (Birmingham, Budapest, CERN, Istanbul, Mexico, Split, Zagreb collaboration) and its possible application at NA49-future. ALICE Experiment. Computing Center. DAQ. 10Gb Ethernet. Counting rooms. DDLs. Machine: LHC @ CERN, Point 2 ALICE detectors: 17

enoch
Download Presentation

ALICE Experiment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DATE the DAQ s/w for ALICE(Birmingham, Budapest, CERN, Istanbul, Mexico, Split, Zagreb collaboration)and its possible application at NA49-future NA49-future Meeting, January 26, 2007

  2. ALICE Experiment Computing Center DAQ 10Gb Ethernet Counting rooms DDLs Machine: LHC @ CERN, Point 2 ALICE detectors: 17 Online Systems: ECS, DAQ, TRG, HLT, DCS Readout: ~400 DDLs @ 2125 Mbps Storage rate: max. 1.25 GB/s Events: 2.5 MB (p-p), 87 MB (Pb-Pb) DAQ Software: DATE V5 Cavern NA49-future Meeting, January 26, 2007

  3. ALICE DAQ architecture PDS TDS TDS Rare/All CTP L0, L1a, L2 BUSY BUSY LTU LTU DDL H-RORC L0, L1a, L2 HLT Farm TTC TTC FEP FEP FERO FERO FERO FERO Event Fragment Sub-event Event File 10 DDLs 10 D-RORC 10 HLT LDC 123 DDLs 262 DDLs 329 D-RORC 175 Detector LDC LDC LDC LDC LDC LDC Load Bal. Event Building Network EDM 50 GDC 25 TDS GDC GDC GDC DS DS GDC 5 DSS Storage Network NA49-future Meeting, January 26, 2007

  4. ALICE online logical model Detector ALICE Terminology Digitizers Front-end Pipeline/Buffer Trigger Level 0,1 Decision Decision Detector DataLink (DDL) DAQ Read-Out Receiver Card (D-RORC) Readout Buffer Trigger Level 2 Decision Decision Data transfer High-Level Trigger Subevent Buffer Local Data Concentrator (LDC) Decision Decision Data Global DataCollector (GDC) Event Building Event Buffer Transient Data Storage (TDS) Permanent Data Storage (PDS) Storage NA49-future Meeting, January 26, 2007

  5. ALICE Detector Data Link NA49-future Meeting, January 26, 2007

  6. DDL architecture Front-End Read-Out Standard detector/DAQ interface Source Interface Unit Forward Channel (Raw data) Backward Channel (Pedestals, control) Detector Data Link (DDL) : • Source Interface Unit • Transmission media • Destination Interface Unit LDC Destination Interface Unit DAQ Read-out Receiver Card (D-RORC) 100 MHz 64-bit PCI card PCI Bus NA49-future Meeting, January 26, 2007

  7. DAQ Read-Out Receiver Cards (D-RORC) • PCI-X adapter for 2 DDL • PCI master:autonomous DMA NA49-future Meeting, January 26, 2007

  8. Readout System Performance • Motherboard with dual Xeon CPUs @ 2.4 GHz • Six PCI-X slots, 4 bus segments (3+1+1+1), 2 controllers • Linux OS • ALICE Data-Acquisition software (DATE) NA49-future Meeting, January 26, 2007

  9. Performance: 6 D-RORCs #4 PCI #6 #2 #3 PCI #5 #2 PCI #4 PCI #3 Controller #1 Segment #1 PCI #2 PCI #1 • Testing the fully populated PC using data source internal to PCI interface • Interoperability test • Measure the maximal input bandwidth Aggregate Bandwidth [MB/s] Normalized Bandwidth [MB/s/Ch] NA49-future Meeting, January 26, 2007

  10. DAQ software • DAQ software • DDL software we can freely use • DAQ framework (DATE) does we need it? • Performance Monitoring (AFFAIR) • Data quality monitoring (MOOD) • Information dissemination • Documentation: 4 DAQ sw packages + ECS: fully documented. User’s guide released and printed • Regular DAQ trainings (70 people in last 3 years) • Linux • Linux SLC4 now. • Process defined to produce, distribute, install new versions • Code management system: CVS. Release packaging and distribution: RPM (~20 MB) • Automatic installation of Linux and DATE on DAQ nodes NA49-future Meeting, January 26, 2007

  11. Memory Management’s Principle RORC Ready FIFO data length transfer status Page aligned continuous user memory outside Linux’s memory space, reachable for DMA and user as well. (physmem module necessary) Free blocks for events or event fragments Maximum block size: 2Mbyte Firmware Free FIFO start address block size Index of Ready FIFO Transfer status possible values: ffffffff unloaded (set by sw) 00000000 loaded, no DTSTW (set by fw) else loaded, DTSTW (set by fw) Free FIFO / Ready FIFO depth: 128 NA49-future Meeting, January 26, 2007

  12. The Free FIFO D-RORC PC memory bank Firmware Free FIFO page address page address page address PC CPU readout Allocation of free pages NA49-future Meeting, January 26, 2007

  13. Direct Memory Access D-RORC PC memory bank DDL Firmware PC CPU No involvement NA49-future Meeting, January 26, 2007

  14. The Ready FIFO D-RORC PC memory bank page status page status page status address address address DDL Firmware Ready FIFO PC CPU readout Delivery of filled pages NA49-future Meeting, January 26, 2007

  15. DDL Software • All functions accessible as interactive commands or API • Script-based interpreter for sequence of operations: • Sending command to the FEE • Reading FEE status • printing the status • comparing the status • polling the status • Downloading data into the FEE from a file • Reading data from the FEE • writing data into a file • comparing data with data in a file • TPC configuration: < 0.3 s • DDL performance • Parallelism FERO DDL D-RORC LDC define pedestal_addr 0x1FFF define enable_pedestal 0x2C reset SIU write_command enable_pedestal write_block pedestal_addr pedestal.hex %x read_and_check_block pedestal_addr pedestal.hex %x NA49-future Meeting, January 26, 2007

  16. DATE Run Control • Control • Configuration • Main parameters • Display • SMI State of all nodes • Main counters of all nodes • Compatible with ALICE ECS NA49-future Meeting, January 26, 2007

  17. DATE Configuration Database • Database content • DATE RolesActors of DATE system:LDCs, GDCs… • TriggerTrigger masks • DetectorsFront-end equipment of LDCs • Event building controlEvent building rules • BanksMemory banks to operate DATE NA49-future Meeting, January 26, 2007

  18. DATE Infologger (1/2) • Online view • Online selection of visible fields • Online selection of message displayed NA49-future Meeting, January 26, 2007

  19. DATE Infologger (2/2) • Offline view • Queries in the message archives • Selection of visible fields and of message displayed NA49-future Meeting, January 26, 2007

  20. Data quality monitoring: MOOD • MOOD: Monitoring Of Online Data • DATE + ROOT environments • MOOD framework • Interfaces to detector code • Applications: • Raw data integrity • Detector performance NA49-future Meeting, January 26, 2007

  21. Performance monitoring: AFFAIR • Individual nodes view • CPU usage • Input/Ouput • Node status • System view / Aggregated performances • Event building bandwidth • Event numbers NA49-future Meeting, January 26, 2007

  22. Event Building Switch • ALICE baseline: • TCP/IP over switched Ethernet • Computing Data Challenges • Event-Building Switch • CERN frame contracts • Needs of IT and experiments • Good prices • On site maintenance (company+IT) • Force 10 Model 1200 NA49-future Meeting, January 26, 2007

  23. Qualification of event-building switch NA49-future Meeting, January 26, 2007

  24. Storage Arrays Performance • Local disk buffer at Pt2 • Storage network: • Fibre Channel currently FC 2G or 4G(Brocade,QLogic) • 16 ports switches with 4 uplinks for interconnect NA49-future Meeting, January 26, 2007

  25. Storage Arrays Performance • Transient Data Storage • Storage arrays(e.g. Infortrend A16F 2 FC 2G ports, 16 SATA II HD) • Performance for a RAID 5 set of 5 disks NA49-future Meeting, January 26, 2007

  26. DDL and DATE for NA49-future? (1/2) • Data transfer • DDL and D-RORC produced and used for longtime • Match speed requirements • Few (< 5) PCs are enough for the task • We have the knowledge how to implement to NA49-future • About 1 man-year of work • Problems with sub-detectors using VME • In ALICE exist VME boards where DDL SIU cards are attached, but interfaces are specific to the given ALICE sub-detector • Or the design VME – DDL interface is necessary for TOF and CAMACs • Interconnect with the trigger system • Trigger “box” has to be designed NA49-future Meeting, January 26, 2007

  27. DDL and DATE for NA49-future? (2/2) • DAQ software • Software (DATE V5) released and documented.Process defined to produce, distribute, install new versions. • Everything (ECS, monitoring, etc.) included • Can we get it from ALICE? • They give it only with support • They can not support us during this year(integration with 17 ALICE sub-detectors) • Modifications necessary if TOF and CAMACs are used without the DDL • Is it shooting at sparrows with a high caliber gun ? • DAQ fabric hardware • Event building switch is necessary • Simple storage network has to be added NA49-future Meeting, January 26, 2007

  28. Thank you NA49-future Meeting, January 26, 2007

  29. Detector Data Link (DDL) • Detector readout: fast data transfer to PC memory • Electronics configuration: pedestals download • Interface and data-transfer detector/DAQ • Radiation tolerant card All components are radiation tolerant including FPGA (ACTEL) • 10 year doses in ALICE (at TPC inner radius): total ionising dose 1.6 krad neutron fluence 3.9·1011 cm-2 charged hadron fluence 8·109 cm-2 • ACTEL ProASIC adopted as baseline NA49-future Meeting, January 26, 2007

  30. DATE V5 Software ALICE Data Acquisition and Test Environment (DATE): software framework for the ALICE DAQ system • Latest DATE version V5: • DATE V5.x kits:RPM package (~20 MB), User’s Guide (~470 pages) • Platforms:IA32 family, SLC4 using a 2.6 kernel • Development:~20 packages, mostly in C, using CVS • Auxiliary Software:DIM, SMI, MySQL, Tcl/Tk, libshift, ROOT, VMEbus driver • Testing:reference system, on-line data challenges, test beams DATE V5 Packages NA49-future Meeting, January 26, 2007

More Related