WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers . Alexandre Beche < Alexandre. - PowerPoint PPT Presentation

slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers . Alexandre Beche < Alexandre. PowerPoint Presentation
Download Presentation
WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers . Alexandre Beche < Alexandre.

play fullscreen
1 / 23
WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers . Alexandre Beche < Alexandre.
164 Views
Download Presentation
lorene
Download Presentation

WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers . Alexandre Beche < Alexandre.

- - - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

  1. WLCG Transfers Dashboard A unified monitoring tool for heterogeneous data transfers. AlexandreBeche <Alexandre.beche@cern.ch>

  2. Outlines Mission History of data transfer monitoring tools Data volume and retention policy Main improvements Future work Application usage Alexandre Beche – GDB

  3. Mission ATLAS FTS + XRootD CMS FTS + XRootD LHCb FTS ALICE XRootD Alexandre Beche – GDB

  4. Mission ATLAS FTS + XRootD CMS FTS + XRootD LHCb FTS ALICE XRootD Alexandre Beche – GDB

  5. WLCG Transfers Dashboard User Interface Alexandre Beche – GDB

  6. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API / UI WLCG FTS Alexandre Beche – GDB

  7. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API / UI XRootD Integration WLCG FTS FAX AAA Alexandre Beche – GDB

  8. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API/UI WEB API/UI WEB API / UI XRootD Integration XRootD Detailed Dashboard WLCG AAA FAX FTS FAX AAA Alexandre Beche – GDB

  9. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API/UI WEB API/UI WEB API / UI XRootD Integration XRootD Detailed Dashboard WLCG AAA FAX EOS Integration FTS FAX EOS AAA EOS Alexandre Beche – GDB

  10. History of data transfers monitoring tools WLCG Transfers Dashboard WEB API/UI WEB API/UI WEB API / UI XRootD Integration XRootD Detailed Dashboard WLCG AAA FAX EOS Integration ALICE Integration FTS FAX EOS AAA EOS ALICE Alexandre Beche – GDB

  11. WLCG Transfers DashboardFederated approach WEB API/UI WEB API WEB API/UI WEB API / UI ALICE FTS AAA FAX FTS FAX EOS AAA EOS ALICE Alexandre Beche – GDB

  12. WLCG Transfers DashboardFederated approach WLCG Transfers Dashboard API / UI WEB API/UI WEB API WEB API/UI WEB API / UI ALICE FTS AAA FAX FTS FAX EOS AAA EOS ALICE Alexandre Beche – GDB

  13. Data Volumesince June 2012 • Data are split into 3 schemas • FTS, XRootD (AAA and FAX) • High optimization (Partition, no Global indexes, …) Alexandre Beche – GDB

  14. Data Volumesince June 2012 • Data are split into 3 schemas • FTS, XRootD (AAA and FAX) • High optimization (Partition, no Global indexes, …) Alexandre Beche – GDB

  15. Data Volumesince June 2012 • Data are split into 3 schemas • FTS, XRootD (AAA and FAX) • High optimization (Partition, no Global indexes, …) • Snapshot on Tuesday 11 Feb 2014 • Shows the usage of the DB at this specific time • No conclusion can be drawn on technology usage • See next slide on data retention policy Alexandre Beche – GDB

  16. Data retention policy • Two kind of data: • Raw: • Kept for 3 months for FTS • Kept indefinitely for AAA and FAX • Statistics: • Different granularity (10 minutes, daily) • Less than 50GB of data • Kept indefinitely Alexandre Beche – GDB

  17. Main improvements • ALICE traffic is now included • No more duplication at the raw data level • WLCG view is an aggregation over APIs • FTS Dashboard is much faster • Does not have the XRootD overhead • Long time period can be queried • See next slide Alexandre Beche – GDB

  18. Activity since June 2012 Impossible with the old architecture Alexandre Beche

  19. Future work • Improve the integration of technology specific dashboards Alexandre Beche – GDB

  20. Future work • Integration of the map view • Already available for AAA, FAX and FTS Alexandre Beche – GDB

  21. Application usageincluding WLCG Home page Alexandre Beche – GDB

  22. Summary Thanks for your attention • Completely new architecture • Greater flexibility • Easier integration of heterogeneous data • Optimized technology-specific dashboards • Aggregated together inside a top level UI Alexandre Beche – GDB

  23. Useful links • AAA Dashboard • http://dashb-cms-xrootd-transfers.cern.ch • FAX Dashboard: • http://dashb-atlas-xrootd-transfers.cern.ch • FTS Dashboard: • http://dashb-fts-transfers.cern.ch • WLCG Transfers Dashboard: • http://dashb-wlcg-transfers.cern.ch Alexandre Beche – GDB