90 likes | 90 Views
DATE multi-stream recorder. Irina Makhlyueva CERN PH-AID DATE v5 Training 14-15 March 2005. Event Builder’s intrinsic recording capability is limited to a single stream , written either to a local disk, or to a FIFO (event descriptors)
E N D
DATE multi-stream recorder Irina Makhlyueva CERN PH-AID DATE v5 Training 14-15 March 2005
Event Builder’s intrinsic recording capability is limited to a single stream, written either to a local disk, or to a FIFO (event descriptors) DATE multi-stream recorder (mStreamRecorder) permits to write several outputs stream per GDC. It is primarily intended for CASTOR recording mStreamRecorder: • runs on every GDC, reads events from the EB FIFO • started by the Run Control at “Start processes” phase • configured from $DATE_SITE_CONFIG/mStreamRecorder.config • writes log messages and statistics to infoLogger • supports two output stream formats: • raw data (see slides by Klaus), use RFIO for CASTOR recording • ROOT (this feature is still under development, to be released soon) • Better use of CASTOR throughput • More efficient evacuation of the EB output buffer DATE multi-stream recorder
Event descriptors via SimpleFifo mStreamRecorder Events via EB Consumer API EB buffer stream (1) 1 Fatal condition signals (interrupts) stream (2) Event Builder 2 (optional) raw/ROOT transformation GDC disp ….. Dispatching algorithm stream (N) N File destination assigned to stream n n Config file ……….. 1 2 N Reporting to infoLogger/Stat Recorder shared memory (internal logging etc) GDC host GDC architecture with mStreamRecorer • the recorder consists of one disp and N × streamprocesses • all stream processes are completely independent on each other • disp reads the config file, configures and forks the streams • dispatching of events to streams, via FIFOs: • “round-robin” and “first-available” algorithms DATE multi-stream recorder
How to run mStreamRecorder (1) • For CASTOR recording, install the latest production version of CASTOR client on all GDCs, the rpm can be downloaded from http://lxservb01.cern.ch/swrep/i386_redhat73/ • If needed, compile the mStreamRecorder components by running gmake in the$DATE_ROOT/mStreamRecorderdirectory • The executable files disp and streams are located in$DATE_ROOT/mStreamRecorder/Linux. Make sure that the owner (gid/uid) of these files has a write access to the CASTOR directories to be written to. • Important: set the suid and sgid permission bits for thesefiles: chmod ug+s disp stream This is needed because DATE processes are running under the dummy user “nobody”, having no appropriate write permissions • Prepare the configuration file (see next slides) and store it in $DATE_SITE_CONFIG/mStreamRecorder.config DATE multi-stream recorder
How to run mStreamRecorder (2) • Add these lines to $DATE_SITE_CONFIG/SOR.commands script: if [ “${DATE_HOST_ROLE}“ = “gdc” ]; then ${DATE_ROOT}/mStreamRecorder/Linux/cleanup ${DATE_ROOT}/mStreamRecorder/Linux/disp fi This is needed, because the recorder is not yet fully integrated in the Run Control. • Enable online recording, by setting recordingDeviceparameter(s) to “:” in $DATE_SITE_CONFIG/ALLDETECTORS/RunParameters.rcParam or, via Run Control HI at the initialization stage, set Run Parameters → Define → GDC → Recording Device : DATE multi-stream recorder
How to configure mStreamRecorder • Flexible, scalable configuration, currently defined via the ASCII file $DATE_SITE_CONFIG/mStreamRecorder.config • Almost no built-in default parameters: most of defaults must be explicitly defined • Free format (indentation, tabs, newlines and comments are for better readability only) • Up-to-date version of the reference guide: at www.cern.ch/Irina.M • The configuration file is a sequence of names and attributes: # a commentary line >COMMONattr1 attr2 … attr !! Common attributes >RECORDERS default_recattr1 attr2 … attr !! Default recorder attributes GDCname1 attr1 .. stream=str1name \ !! Special ttributes for … stream=… stream=… attr … !! …GDC gdcname1 GDCname2 … >OSTREAMS default_strattr1 attr2 … attr \ !! Default stream … attr … attr !! … attributes str1nameattr1 attr2 … attr !! Special attributes for stream str1name Attribute: name=value █ compulsory █ optional DATE multi-stream recorder
Example 1 >COMMON method=2 Nstreams=3 loglevel=1 >RECORDERS default_recstream=default_str >OSTREAMS default_str sleep=1 fsize=1024 mxrecl=0 \ pool=alice_stage stager=stagealice \ path=/castor/cern.ch/alice/daq_dev/daq_recorder Examples (1) • The minimal configuration: all GDCs and all streams have the same attributes • Remarks: • All recorder parameters (here – only the stream name) are taken from default_rec (default recorder) definition • all stream parameters here are taken from the default_str definition • alice_stage and stagealice are standard ALICE pool and stager. Any other available stager/pool combination can be used instead • The number of streams per GDC is defined by Nstream=3 attribute in the >COMMON category, which has the highest precedence DATE multi-stream recorder
# Example 2 >COMMON method=1>RECORDERSdefault_rec method=2 stream=default_strpcald04gdc Nstreams=4 \ !! the configuration for pcald04dgc stream=default_str stream=public stream=test>OSTREAMSdefault_str sleep=1 path=/local mxrecl=0 \ pool=public stager=stagepublic fsize=2047 !! fsize in Mbytespublic path=/castor/cern.ch/user/m/makhlyuitest=default_str path=/castor/cern.ch/user/m/makhlyui/test_dir Examples (2) • A more sophisticated configuration: individual GDC specs, a selection of different streams Example 2 • One GDC (pcald04gdc)takes its attributes fromdefinition. If this config file is used with multiple GDCs – the other ones will use the default_rec definition • If less that Nstreams are explicitly specified in the recorder, the appropriate number of default streams (default_str) is automatically added • method=1 (“round-robin”) will be used for all GDCs, because of the higher precedence of attributes in the >COMMON category • Streams public and test will take all their attributes, except path= , from default_str DATE multi-stream recorder
Output file names pcald04gdc_run00000610_0_1.data file number (1,2…) GDC name run number stream number (0,1…) Status of mStreamRecorder • It is currently under test in DC6, with plain raw data streams and RFIO protocol for CASTOR • Still under development: • ROOT recording • Small changes in the configuration file parser (to accommodate ROOT and make a better use of defaults) • Configuration file checker • Integration in the DATE MySQL configuration DB • Full integration in the Run Control DATE multi-stream recorder