1 / 44

RDMS CMS Computing: Russian CMS Tier-2 and Tier-1 for CMS at JINR llya Gorbunov and Sergei Shmatov

RDMS CMS Computing: Russian CMS Tier-2 and Tier-1 for CMS at JINR llya Gorbunov and Sergei Shmatov Joint Institute for Nuclear Research , Dubna shmatov@cern.ch. School on JINR/CERN GRID and Advanced Information Systems “ Joint Institute for Nuclear Research, Dubna May 14-18, 2012.

ziazan
Download Presentation

RDMS CMS Computing: Russian CMS Tier-2 and Tier-1 for CMS at JINR llya Gorbunov and Sergei Shmatov

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. RDMS CMS Computing: Russian CMS Tier-2 and Tier-1 for CMS at JINR llya Gorbunov and Sergei Shmatov Joint Institute for Nuclear Research, Dubna shmatov@cern.ch School on JINR/CERN GRID and Advanced Information Systems “Joint Institute for Nuclear Research, Dubna May 14-18, 2012

  2. Compact Muon Solenoid • goals and performance • participation of Russian and Dubna Member State Institutions • data taking, triggering and selection • data flows • CMS Computing Model • data model • data flows • data processing • Tier-1/Tier-2 requirements and resources • RDMS Computing Model • RDMS CMS Tier-2s (readiness, job status, data transfers) • computing for remote operations (JINR ROC) • CMS Tier-1 at Dubna (plans and prospects) IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  3. Brief Intro: What does physics require from Computing? IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  4. Why we need more accelerators and experiments ? Being the remarkable event on the long way of searching of the true, Standard Model of interactions is powerless to answer on very important questions: • Origin of masses of fundamental blocks (quarks, leptons) • A reason for three generations of matter fermions • A reason of domination matter over antimatter and • formation of observable Universe structure • A source of dark mater in the Universe • Possibility of high energy unification of three forces And after all, what about gravity? We hope we will be able to answer on these questions at least partially with next generations of accelerators IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  5. LHC Physics Perspectives • Open Questions: • Mass: why do fundamental particle have mass and why these masses are such different? • Antimatter: why at the present days matter dominates antimatter? • Dark matter: where is concentrated the hidden mass of Universe? • World geometry: are there extra dimensions? bulk volume geometry? • New particles: do they exist? (sparticles, extra gauge bosons, graviton states etc) • QCD matter: what are the properties of QCD at extreme high densities, does QGP exist? Standard model: validity bounds -> high precision tests of SM ! IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  6. Compact Muon Solenoid IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  7. LHC Experiments Compact Muon Solenoid (CMS) + TOTEM (TOTal Elastic and diffractive cross section Measurement) LHC-B LHCf A Large Ion Collider Experiment (ALICE) A Toroidal LHC ApparatuS (ATLAS)

  8. CMS 14 km 15 min CERN Building 40 CMS at LHC IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  9. Compact Muon Solenoid - CMS sampling brass HCAL Silicon Tracker Detector subsystems are designed to measure: the energy and momentum of photons, electrons, muons, jets, missing ET up to a few TeV Forward Calorimeter, HF Muon Chambers Tracker Electromagnetic calorimeter, ECAL Hadron calorimeter HCAL PbWO4 ECAL MUON Chambers Weight 12 500 t Diameter 15.00 m Length 21.6 m Magnetic Field 3.9 Tesla Superconducting Coil diameter6 m, length13 m Return Yoke Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  10. Russian and Dubna Member State CMS Collaboration Belgium Austria the RDMS CMS Collaboration was founded in Dubna in September 1994 Finland USA CERN France Germany RDMSCMS: Greece Hungary Russia and JINR Member States Uzbekistan Italy Ukraine Georgia 2310 authors 38 countries 175 institutions Bulgaria Belarus Poland UK Armenia Portugal Turkey Brazil Serbia China, PR Spain Pakistan China (Taiwan) Switzerland Lithuania Colombia Mexico Iran Korea New-Zealand Croatia Cyprus Ireland India IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012 Estonia

  11. JINR Participation in CMS Construction RDMS bears Full Responsibility JINR Participates ME1/1 SE HE JINR participates in the CMS in a framework of the RDMS CMS Collaboration Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  12. Cross-sections of physics processes at LHC Energy The fraction of “important” events is ~ 10-8 of total event flow ! The fraction of events from “new physics”is 10-10- 10-15(Higgs, SUSY, BSM) • A very small part of events is a proper for physics analysis • No opportunity to store and to process the full data flow Triggering and Selection IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  13. 40 MHz (1000 TB/sec) Level 1 - Special Hardware 75 KHz (75 GB/sec) Level 2 - Embedded Processors 5 KHz(5 GB/sec) Level 3 – Farm of commodity CPUs 100 Hz (100 MB/sec) Data Recording & Offline Analysis • Online: Trigger System • Multilevel trigger • rejection of background (filtration of events) • reduction of data volume • on-line reduction factor ~107 • Trigger menu • selection of required events The data flow is reduced from 1000 TB/sec (1TB=1012 Bytes) at input of the first-level trigger down to 100-200MB/secon output of the third-level trigger (to record on disks or tapes for further analysis) CMS event – 1-2 MB L1 trigger – 75 GB/с HLT trigger – 0.1 GB/с total flow~3 PByear IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  14. LHC Data Flows ~3 PB/year ~3 PB/year IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  15. High Level-1 Trigger(1 MHz) High No. ChannelsHigh Bandwidth(500 Gbit/s) Level 1 Rate (Hz) 106 1 billion people surfing the Web LHCB ATLAS CMS 105 HERA-B KLOE CDF II 104 High Data Archive(PetaByte) CDF 103 H1ZEUS ALICE NA49 UA1 102 104 105 106 107 LEP Event Size (bytes) The CMS Computing Requests Event rate to tape: 100 Hz (109 events/year) Days of running/year: 200 Data written to tape: ~ 3 PetaBytes/Year Monte Carlo events: 9∙108 CPU (Tier-0 + Tier-1s + Tier-3s): ~ 26 000 kSI2k Typical networks: 1 - 10 Gbps Users: ~ 1000physicists distributed all over the world The full processing and analysis can not be performed by the standard tool kits It is essential to use Distributed Computer Resources IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  16. CMS Computing Model IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  17. Tier Structure of GRID Distributed Computing: Tier-0/Tier-1/Tier-2/Tier-3 Tier-0 (CERN): • accepts data from the CMS Online Data Acquisition and Trigger System (TriDAS) • archives RAW data • the first pass of reconstruction and performs Prompt Calibration • data distribution to Tier-1 Tier-1 (11 centers): • receives a data from the Tier-0 • data processing (re-reconstruction, skimming , calibration etc) • distributes data and MC to the other Tier-1 and Tier-2 • secure storage and redistribution for data and MC Tier-2 (>200 centers): • simulation • user physics analysis 17 IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  18. CMS GRID Distributed Computing Tier-0 at CERN Tier-1 (7 centers): CERN , KIT (Germany), PIC (Spain), CCIN2P3 (France), CNAF (Italy), ASGC (Taipei), RAL (UK), FNAL (US) + two sites under construction at Dubna and India Tier-2 (85 centers): 7 in Russian and DMS IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  19. CMS Data Model Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  20. CMS Data Flows (I) ~ Pbyte/sec ~ 100-1500 MBytes/sec 10 Gbps 2-10 Gbps 10 Gbps 0.1 - 10 Gbps IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  21. CMS Data Flows (II) Detector data flow • T0 to T1: • scheduled, time-critical, • continuous during data-taking • reliable transfer needed for fast a • access to new data, and to ensure • that data is stored safely • T1 to T1: • redistributing data, generally after reprocessing (e.g. processing with improved algorithms) • T1 to T2: • Data for analysis at Tier-2s Monte-Carlo data flow • T2 to T1: • data transfer simulation events • T1 to T1/T2: • data transfer re-processed simulation events (e.g. processing with improved algorithms) Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  22. Data Processing (II) Reconstruction RAW Data Event Selection Final Physics Results! CMS-PAS-HIG-11-032 Calibration Data Base Theory/ Monte Carlo Simulation Physics Analysis IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  23. CMS Data Processing (II) Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  24. CMS Computing Resources Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  25. RDMS CMS Computing Model IlyaGorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  26. RDMS CMS Grid Sector Tier-0 CERN Dubna CMS Tier-1 and other CMS Tier-1 centres CMS RDMS Tier-1 Tier-2 KIPT Kharkov Tier-2 Minsk Tier-2 Erevan Tier-2 JINR Dubna Tier-2 INR Troitsk Tier-2 FIAN Moscow Tier-2 SINP MSU Moscow Tier-2 Sofia Tier-2 Tbilisi Tier-2 ITEP Moscow Tier-2 IHEP Protivino Tier-2 PNPI St.Petersburg DMS Russia RDMS Grid Sector To provide data management and data transfer to RDMS Tier-2and data processing in the framework of RDMS Physics Tasks the special СMS RDMS Tier-1 was created in CERN Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  27. CMS T2 requirements Basic requirements to CMS VO T2 sites for Physics group hosting: • info on contact persons responsible for site operation • site visibility (BDII) and CMS VO support • availability of CMSSW actual version • daily SAM availability > 80% • daily CMS job robot efficiency > 90% • regular file transfer test “OK” • Certified links with CMS T1: 2 up and 4 down • disk space ~ 150-200 TB for: - central space (~30 TB) - analysis space (~60-100 TB) - MC space (~20 TB) - local space (~30-60 TB) - local CMS users space (~2 TB per user) • CPU resources ~ 3-4 KSI2K per 1 TB disk space, 2GB RAM memory per job Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  28. RDMS CMS Tier-2 Availability Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  29. Status of RDMS Site Metrics Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  30. Normalized CPU time (in 1K.SI2K.Hours) consumed by the CMS sites September, 2010 – August, 2011) Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  31. Total Number of CMS Jobs at RDMS CMS sites 1 747 182 jobs 5.8% 2 253 116 jobs 8.7% 31 Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  32. Job Summary at RDMS CMS Sites as in the CMS Dashboard Job Status per Sites Site Activity Job Efficiency Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  33. PHEDEX Transfer Rate & Transfer Volumes for RDMS CMS sites Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  34. RDMS CMS T2 association Now Future interest Analysis Groups Exotica: T2_RU_JINR Exotica: T2_RU_INR HI: T2_RU_SINP QCD: T2_RU_PNPI Electroweak: T2_UA_KIPT Top: T2_RU_SINP FWD: T2_RU_IHEP Object/Performance Groups Muon: T2_RU_JINR e-gamma-ECAL: T2_RU_INR JetMET-HCAL: T2_RU_ITEP Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  35. JINR Monitoring and Analysis Remote Centre • Monitoring of detector systems • Data Monitoring / Express Analysis • Shift Operations (except for run control) • Communications of JINR shifter with personal at CMS Control Room (SX5) and CMS Meyrin centre • Communications between JINR experts and CMS shifters • Coordination of data processing and data management • Training and Information Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  36. RDMS CMS Computing Model: Tier-1 at Dubna Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  37. WLCG Tier1 center in Russia • Proposal to create the LCG Tier1 center in Russia (official letter by Minister of Science and Education of Russia A. Fursenko has been sent to CERN DG R. Heuer in March 2011). • The corresponding point to include in the agenda of next 5x5 meeting Russia-CERN (October 2011) - for all four experiments ALICE, ATLAS, CMS and LHCb - ~10% of the summary Tier1 (without CERN) resources - increase by 30% each year - draft planning (proposal under discussion) to have prototype in the end of 2012, and full resources in 2014 to meet the start of next working LHC session. Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  38. Joint NRC "Kurchatov Institute" – JINR Tier1 Computing Centre Project: «Creation of the automated system of data processing for experiments at the Large Hadron Collider (LHC) of Tier1 level and maintenance of Grid-services for a distributed analysis of these data» Terms: 2011-2013 Type of project: R&D Cost: federal budget - 280 million rubles (~8.5 MCHF), extrabudgetary sources - 50% of the total cost Leading executor: RRC KI «Kurchatov institute» for ALICE, ATLAS, and LHC-B Co-executor: LIT JINR (Dubna) for the CMS experiment Project goal: creation in Russia of a computer-based system for processing experimental data received at the LHC and provision of Grid-services for a subsequent analysis of these data at the distributed centers of the LHC global Grid- system. Core of the proposal: development and creation of a working prototype of the first-level center for data processing within the LHC experiments with a resource volume not less than 15% of the required one and a full set of grid-services for a subsequent distributed analysis of these data. Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  39. Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  40. Tier-1 Services • Security (GSI) • Computing Element (CE) • Storage Element (SE) • Monitoring and Accounting • Virtual Organizations (VOMS) • Workload management (WMS) • Information service (BDII) • File transfer service (FTS + PhEDEx) • VOBOX • SQUID / Frontier Servers • CMS user services (Reconstruction Services, Analysis Services etc) Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  41. CMS Tier-1 Requirements • input flow ~ 7 GB/s • output flow ~ 3-4 GB/s • CPU – 2.5 MSI2k • disks – 1.2 PB • tapes for archives and secure store – 2.8 PB • data loss – 10 GB per 1 PB • access to data ~ 800 MB/s • I/O computing node bandwidth ~ GB Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  42. JINR Resources Ilya Gorbunov and Sergei Shmatov“RDMS CMS Computing”, AIS2012, Dubna, May 18, 2012

  43. SUMMARY • RDMS Tier-2 sites are fully satisfied to CMS Computing Requirements • are involved actively into data processing and analysis (~3.8% of all CMS resources) • New CMS Tier-1 site is under construction in Dubna (in 2012-2013 a prototype need to be designed and commmisined) • CMS Regional Operation Center is founded and operated in JINR for remote monitoring of detector systems and data express-analysis

More Related