1 / 9

Production Status Report Planning and Suggestions Catalin Ristea, Latchezar Betev

Production Status Report Planning and Suggestions Catalin Ristea, Latchezar Betev. Nov 2015, Offline Week, LB. Some figures. Since the beginning of 2016… … as aliprod 8,000,000 jobs have been managed (13.5 Mio jobs in total, ~70% MC productions, 6%error rate)

wmarian
Download Presentation

Production Status Report Planning and Suggestions Catalin Ristea, Latchezar Betev

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Production Status Report Planning and Suggestions Catalin Ristea, Latchezar Betev

  2. Nov 2015, Offline Week, LB

  3. Some figures Since the beginning of 2016… … as aliprod • 8,000,000 jobs have been managed (13.5 Mio jobs in total, ~70% MC productions, 6%error rate) • ~471,877,945 events processed, 4068 years running time • 1 PB data size (x2 by replicas) • 14 productions managed (MC continuing & new & unfinished + Refiltering AOD) ...Overall, things are running pretty smoothly, need to be careful not to break the machinery

  4. The “Battlefield” :) GRID Operations Detector QA Configuration MC Data Processing MC Analysis Reconstruction Calibrations Have clear names for the key positions for each of the blocks, as it necessary to have Good communication at SETUP ↔ Running ↔ Validation stages

  5. Organization • MC department to deal with various stages of the productions • Preparation – simulation, detector configuration • Anchored data preparation – in terms of OCDB calibrations If somebody asks about what settings should use, what is the answer? • Take a similar production… • Some pages from experts where to look for certain settings • Initial requests are either based on prior experience, or looking around... could be better? Is this workflow optimal? OLD vs. NEW

  6. UnClutter JIRA Detector Configuration Common Std Prod MC Coordination JIRA Sub-task #1 SETUP/TEST JIRA Sub-task #2 PROCESSING JIRA Sub-task #3 Final QA MC Configuration • Most of the common productions are having ~200-300 comments in JIRA → we need better planning • Hard to find pieces of informations • Need to reorganize it?

  7. Good things happening • Preproduction MC database – better selection, moving around • Planning in advance because detector experts could be overhelmed with requests • Testing and validation of both macro on different datasets • Automatic trending and QA • MonALISA improvements • Grouping by error types (cherry bomb) • Faster “Job types” tab

  8. Macro for AOD (re)filtering and QA Filtering DATA p-p iCollision muon_calo pass Re-filtering MC A-A iCollision ESDFilter changes MultSelection run_flag/year 2010 → 1000 2011 → 1100 2015 → 1500 AliRoot/AliPhysics versions Centrality PHOS no. modules Different repositories: (outside AliRoot/AliPhysics… NEW for testing/validation inside?) • For each period of data taking OK • Too many switches → reduce to run#, GRP, ENV... MC for each AliRoot/AliPhysics versions (AOD/QA pp & AA for MC, may be different data anchored → needs separate versions → prone to errors) Simple multiple versions ↔One single complex version (when there are too many simple versions, better a complex single one...) infamous periodName appearing from time to time

  9. Thank you!

More Related