1 / 13

AMOS/LUSI Controls and Data Acquisition LCLS eXperimental End Stations (XES)

AMOS/LUSI Controls and Data Acquisition LCLS eXperimental End Stations (XES). Robert C. Sass. XES Slow Controls. No major problems forseen AMOS has ~ 100 movers Ultra high vacuum, power supplies etc. Some requiring special consideration are: Beam inhibit next pulse from the XES via MPS

timothylewis
Download Presentation

AMOS/LUSI Controls and Data Acquisition LCLS eXperimental End Stations (XES)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. AMOS/LUSI Controls and Data AcquisitionLCLS eXperimental End Stations (XES) Robert C. Sass

  2. XES Slow Controls • No major problems forseen • AMOS has ~ 100 movers • Ultra high vacuum, power supplies etc. • Some requiring special consideration are: • Beam inhibit next pulse from the XES via MPS • Use chopper to rate limit • Synch gas jet & Laser to beam pulse • Synch mechanical chopper to the beam

  3. Experimental Data • Generation, storage, retrieval and analysis of experimental data is the “product” of the LCLS. • LSST, if funded, will produce ~30 TB of data per night. • The AMOS experiment may eventually take data @120Hz from: • 6 spectrometers @~15 KB. • 5 CCDs @1 MB each • That’s ~700 MB/second or 2.4 TB/hour or ~58 TB/24 hour running period. • Larger CCD densities are already planned. • LCLS experiments are relatively short term and so will follow the technology more quickly than LSST. • These numbers have nowhere to go but up. • With the possible exception of the LHC or until the ILC comes along, LCLS will be the world driver for Data Acquisition, Storage, Retrieval and Analysis.

  4. How Will We Handle This? • Build on previous SLAC experience. • Initially use commercial digitizer and COTS CPU for AMOS spectrometer data. Only one vendor found that makes an 8GHz digitizer. • Use scalable technology developed for LSST for CCD data. • Capitalize on years of BaBar experience with hierarchical storage and management of HEP data. • Many aspects of traditional HEP computing and data management are applicable to XES/LUSI • Large parts of the BaBar data management system have been developed at SLAC • Significant expertise for this type of system exists at SLAC. • BaBar Stores: • ~1 TB/day raw data. • >>1 TB/day derived data products. • ~1.5 PB total Babar data. Old data purged. • Extend/integrate several SLAC-developed and EPICS Java-based technologies for data retrieval and analysis. • Java Analysis Studio (JAS) • Accelerator Integrated Data Access (AIDA) • EPICS Archive Viewer

  5. eXperimental End Station System Overview Channel Access MPG/LCLS Event Generator MCC Main Control Room Anywhere LCLS Analysis / Viewing LCLS Data Retrieval Linac & LCLS Control GUI 120 Hz Archive Engine Event Receiver SCCS Fast Instrument Archive Other Databases DAQ Archive Linac/LCLS Injector 120Hz IOC Injector EELOG DAQ Config Linac Undulator XTOD Slow Instrument Archive X-Ray Beam EPICS Archive SLAC WAN SLAC WAN XES Hutch XES private subnet Hutch IOCs & Instruments Experimenter Ethernet Fast DAQ Data Visual Data Monitor 120 Hz Beam Line Data MPS Machine Protection (MPS) EELOG Detector Instrument(s) Detector Control XES Controller Data Acquisition Subsystem(s) Digitized Data XES Data Cache EVR Hardware Triggers Beam Line Processor 120 Hz Beam Line & Timestamp Data (dedicated Ethernet) Config/Status Experiment Specific IOCs & Instruments Channel Access Gateway X-Ray Transport to Diagnostics & Next Hutch MPS

  6. XES Detail – Data Acquisition 1. Channel Access (Ethernet) 3. EVR (Fiber) 2. Beam Line 120 Hz Data 6. DAQ Data to SCCS 5. SLAC WAN (Ethernet) 4. MPS (Reflective Memory Fiber) XES Hutch XES private subnet 2 Detector Types 2 DAQ Types EELOG Spectrometer SBC DAQ XES Controller D PC Non-RTOS A TOF/Momentum Instrument cPCI ADC RTOS 1 Gb E 1 Gb E 5 G Spectrometer Monitor 1 Gb E Visual Data Monitor EPICS Config PVs XES Data Cache PC Non-RTOS 1 Gb E 1 Gb E 1 Gb E 6 CCD Pixel Detector CE Module DAQ Beam Line Processor PPC RTOS B E SBC RTOS Front End Board Fast Serial FPGA F 1 Gb E MPS PMC 4 G CCD Monitor 1 Gb E EVR VME 3 EPICS Config PVs Beam Line Data PMC 1 Gb E C 2 1 Gb E 10 Gb Enet Channel Access Gateway Experiment Chamber 1 A. EPICS & Local Control (Hutch Subnet) B. Distributed EVR Hardware Triggers C. Beam Line & Timestamp Data (dedicated Enet) D. ADC Control & Digitized Data G. Visual Monitor Data E. Detector Control & Digitized Data F. DAQ Data to Cache

  7. Right side Memory (512 Mbytes) Micron RLDRAM II Right side Configuration memory 128 Mbytes) Samsung K9F5608 Right side Multi-Gigabit Transceivers (MGT) 8 lanes Left side MFD Reset options JTAG Reset Left side 100-baseT CE Module block diagram Fabric clock Right side PPC-405 (450 MHZ) Right side MGT clock FPGA 200 DSPs Lots of gates Xilinx XC4VFX60

  8. Dual CE Module PHYs (0-16) May mix and match lanes to each CE footprint: • ~ 50 cm2 • power: • ~ 7 watts total + • ~ 3/4 Watt/port • Cost ~1K CE CE reset options 10 GE 10 GE 100B-T 100B-T JTAG reset

  9. CE Module Characteristics • Memory Subsystem • 512 Mbytes of RAM • Sustains 8 Gbytes/sec • “Plug-In” DMA interface (PIC) • Designed as a set of IP cores • Designed to work in conjunction with MGT and protocol cores • Bootstrap loader (with up to 16 boot options and images) • Interface to configuration memory • Open Source R/T kernel (RTEMS) • 10 GE Ethernet interface • 100 base-T Ethernet interface • Full network stack • Utility software to manage I/O

  10. Archive Data Management System Overview 1. EPICS Channel Access LCLS Operations Anywhere on Channel Access Networks Anywhere on SCRAMNet Network 2. DAQ Archive Data 3. Slow Archive Data Slow Archive Engines Fast Archive Engine 1 4. Fast Archive Data Global Archive Configuration Management Archive Configuration Interface 5. DAQ Archive Configuration 6. Slow Archive Configuration 4 3 7. Fast Archive Configuration 6 9 8. Offsite/Cache Archive Transfer Global Archive Dictionary Fast Archives SCCS 9. Global Archive Dictionary Management Slow Archives Slow Archive Configs Fast Archive Config DAQ Archive Configs DAQ Archives Modeling & Other Databases 8 7 5 Experiment Operations Archive Configurations Interface 2 XES Hutch Data Acquisition Module(s) XES DAQ Data Cache

  11. Data Volume Makes DAQ Archive More Complicated C E SBC 10 GigE 1 GigE Ethernet Network Switch Local Cache Farm Ethernet Network Switch Links to Computer Center 10 GigE

  12. Archive Retrieval/Analysis System Overview 1. Global Archive Dictionary Read Anywhere 2. Data Retrieval Request 3. Locate server & storage 1 4. Tape Data Request LCLS Analysis / Viewing Tape Data 5. Database Data 6. Slow Archive Data 2 9 7. Fast Archive Data LCLS Data Retrieval 8. DAQ Archive Data 4 9. Results 3 Relational Database Server Slow Archives Server Fast Archive Server DAQ Archives Server Global Archive Dictionaries SCCS Data Retrieval Temporary Cache 5 Modeling & Other Databases 6 7 8 Slow Archives Fast Archives DAQ Archives

  13. Summary • Minor slow controls issues. • Data Acquisition and Archive are the major LCLS challenges. • With CE for CCD and a commercial digitizer & SBC for spectrometers we can scale the DAQ needs of LCLS. • With BaBar experience we can scale the data archive needs. • In the near future we must: • Specify and design an integrated data retrieval and analysis package that can be used anywhere and is user extensible. • Specify and design a framework and tools for experimenters to use in designing and building their instruments. • The CE provides a standard CCD interface • Existing EPICS tools could be extended to construct the experiment-specific EPICS database and screens.

More Related