1 / 17

Living in Two Worlds: Understanding SAM and LCG

Explore the Sequential Access by Metadata (SAM) system, file storage, replica catalog, and efficient file delivery to consumers. Learn about SAMGrid, Job Information and Monitoring (JIM), and Grid computing. Discover how SAM stations optimize file delivery, submit jobs, and manage cluster resources effectively. This comprehensive guide covers the integration of SAM and LCG, user interfaces, submission sites, and monitoring procedures within the Grid environment.

prescott-skinner
Download Presentation

Living in Two Worlds: Understanding SAM and LCG

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Dzero MC production on LCG How to live in two worlds (SAM and LCG)

  2. What is SAM? • SAM: Sequential Access by Metadata • file store • sam stations • replica catalog • metadata database • optimizes file delivery to consumers • batch interface Grid meeting Jan 28, 2008

  3. What is SAMGrid? • SAM: Sequential Access by Metadata • JIM: Job Information and Monitoring • Fermigrid • OSG • LCG • … Grid meeting Jan 28, 2008

  4. Until January 2007 resource broker user interface SAM station dorsvlegel.nikhef.nl Grid meeting Jan 28, 2008

  5. What is SAMGrid? • SAM: Sequential Access by Metadata • JIM: Job Information and Monitoring • Fermigrid • OSG • LCG • … Grid meeting Jan 28, 2008

  6. From January 2007 Wuppertal submission site execution site FNAL jim client cluster (Nikhef) SAM station Lyon Grid meeting Jan 28, 2008

  7. JIM client • Submits job to submission site • input.jdl specifies: • request number • cluster Grid meeting Jan 28, 2008

  8. input.jdl lcg_requirement_string = tbn20.nikhef.nl:2119/jobmanager-pbs-qlong job_type = dzero_monte_carlo runjob_requestid = 59873 d0_release_version = p20.08.02 jobfiles_dataset = sg_p20.08.02-v3_d0r07-05-03_cfv01-00-00 minbias_dataset = zerob_p20.08.01_RunIIbMC_REALsample_2 runjob_numevts = 205000 events_per_file = 250 station_name = ccin2p3-grid1 Grid meeting Jan 28, 2008

  9. Submission site • Submits cluster job to execution site • Collects non physical output (logs, …) • Collects monitoring data • Monitoring web pages Grid meeting Jan 28, 2008

  10. Execution site • User interface • Submits individual jobs to cluster (edg-job-submit) • Resource broker Grid meeting Jan 28, 2008

  11. SAM station • Delivers exec files • delivers input files • stores metadata and data in SAM Grid meeting Jan 28, 2008

  12. Output from WN • To SAM database: • metadata of intermediate files and MC output • To SAM store: • reconstructed MC output • To submission site • log files Grid meeting Jan 28, 2008

  13. Pro’s • Always in line with Fermi lab (no home grown scripts) • less labor intensive • better monitoring Grid meeting Jan 28, 2008

  14. Cons • Not scalable • SAM station @ in2p3 • Resource broker @ wuppertal • User interface@ wuppertal • less flexible (no home grown scripts) Grid meeting Jan 28, 2008

  15. Scalability • SAM station @ in2p3 • dCache @ ic.ac.uk • Resource broker @ wuppertal • Moved to dorsvlegel.nikhef.nl • User interface @ wuppertal Grid meeting Jan 28, 2008

  16. dorsvlegel@nikhef.nl • Until January 2007 • UI + SAM station • Until January 2008 • UI + JIM station • Now • UI + JIM station + RB (dzero only) Grid meeting Jan 28, 2008

  17. Monitoring • Go to http://samgrid.fnal.gov:8080 • Click submission sites • Click samgrid.fnal.gov Grid meeting Jan 28, 2008

More Related