1 / 6

ATLAS ADC TIM

ATLAS ADC TIM. Metadata in Distributed Analysis S.Albrand, E.Gallas. 1. Overview. Specific DA Questions News from the Simulation Metadata workshop. Input File Peeker. Can it be replaced by a call to AMI to interpret an AMITag?

zenda
Download Presentation

ATLAS ADC TIM

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ATLAS ADC TIM Metadata in Distributed Analysis S.Albrand, E.Gallas 1

  2. Overview Specific DA Questions News from the Simulation Metadata workshop ATLAS ADC TIM

  3. Input File Peeker Can it be replaced by a call to AMI to interpret an AMITag? Implies some re-specification of the AMI tags. Usage, particularly by prodsys … beyond original specification (double or triple transforms described in the same tag) … need to establish some conventions. Discussion with transforms group needed Implementation? “dump” of AMI tags in SQLite files or pyDict format and adapt pyAMI for reading them? Reminder : T0 tags defined directly in AMI. Production tags (MC & reprocessing) defined in prodsys, and read later into AMI. Rod Walker did some tests of using AMI directly by prodsys. Conclusions? ATLAS ADC TIM

  4. User Datasets in AMI Been discussed many times. We know from the logs that people sometimes look for them in AMI. Only AMI requirement is that the metadata MUST come from reliable sources, i.e. DA tools. ATLAS ADC TIM

  5. pyAMI User Command combining a call to DQ2/Rucio and AMI? pyAMI team has already adopted a prototype script from Johannes, and will develop and maintain it. New version pyAMI-04-00-00 Will go into release 18 Already available on lxplus, cvmfs, and tarball for download. User Interface changed slightly. Minimum changes for pyAMI in scripts. https://end.web.cern.ch/end/projects/pyAMI/ ATLAS ADC TIM

  6. Simulation / Metadata workshop Simulation / Metadata workshop : Tuesday this week. Representatives from all coordination areas. Many issues discussed & action items identified - will be documented. Two issues require ADC attention: The derivation of evgen dataset names from job options is a problem. SUSY  thousands of tasks use the SAME JO file thousands of identical JO files with different names. What are the technical limits to the maximum length of a dataset name? DISCUSS! ATLAS ADC TIM 6

More Related