1 / 30

Interfacing Interactive Data Analysis Tools with the Grid: PPDG CS-11 Activity

Interfacing Interactive Data Analysis Tools with the Grid: PPDG CS-11 Activity. Doug Olson, LBNL Joseph Perl, SLAC ACAT 2002, Moscow 24 June 2002. Contents. Background on PPDG, CS-11 Who is involved Workshop last week (18,19 June) Themes that emerged Near-term goals Longer term planning

dutch
Download Presentation

Interfacing Interactive Data Analysis Tools with the Grid: PPDG CS-11 Activity

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Interfacing Interactive Data Analysis Tools with the Grid:PPDG CS-11 Activity Doug Olson, LBNLJoseph Perl, SLAC ACAT 2002, Moscow 24 June 2002

  2. Contents • Background on PPDG, CS-11 • Who is involved • Workshop last week (18,19 June) • Themes that emerged • Near-term goals • Longer term planning • Summary D. Olson, PPDG CS-11 for ACAT

  3. The 3 US grid projects for HENP are PPDG, GriPhyN, iVDGL D. Olson, PPDG CS-11 for ACAT

  4. PPDG CS-11 D. Olson, PPDG CS-11 for ACAT

  5. Background • CS-11 long title:Interfacing and Integrating Interactive Data Analysis Tools with the Grid and Identifying Common Components and Services • Subtitle:Consider physicist sitting at home institution; “What does she need from grid to carry out physics analysis?” • CS-11 is: • Not new funding • New work area within PPDG mission of grid enabling end-to-end physics applications for US HENP • Driven by experiments needs, middleware providers want to know if new/different grid services needed D. Olson, PPDG CS-11 for ACAT

  6. 18,19 June workshop in Berkeley • Purpose: • Review experiments’ requirements • Overview of existing tools & technology • Discuss existing/planned activities • Identify opportunities for cooperative work on defining interfaces and prototype integration of analysis tools with common grid services. D. Olson, PPDG CS-11 for ACAT

  7. Review Use Cases for requirements D. Olson, PPDG CS-11 for ACAT

  8. Review Tools & Technology D. Olson, PPDG CS-11 for ACAT

  9. Abstract Interfaces for Data Analysis D. Olson, PPDG CS-11 for ACAT

  10. Java Analysis Studio D. Olson, PPDG CS-11 for ACAT

  11. PROOF D. Olson, PPDG CS-11 for ACAT

  12. Clarens D. Olson, PPDG CS-11 for ACAT

  13. Interactivity in a batched grid environment D. Olson, PPDG CS-11 for ACAT

  14. MCAT – Metadata Catalog in SRB D. Olson, PPDG CS-11 for ACAT

  15. SDM Center – bitmap index D. Olson, PPDG CS-11 for ACAT

  16. Grid Architecture view D. Olson, PPDG CS-11 for ACAT

  17. EDG testbed D. Olson, PPDG CS-11 for ACAT

  18. Experiments thoughts, plans, activities • ATLAS • Python interface between Athena framework and grid services • CMS • Grid Analysis Environment (GAE) • Phobos (& ALICE) • PROOF-based analysis • Others (BaBar, Jlab, STAR) at meeting without presentations • Extraction model probably good for BaBar, Jlab • PROOF likely to work for STAR D. Olson, PPDG CS-11 for ACAT

  19. ATLAS extraction view D. Olson, PPDG CS-11 for ACAT

  20. D. Olson, PPDG CS-11 for ACAT

  21. CMS Analysis Scope D. Olson, PPDG CS-11 for ACAT

  22. D. Olson, PPDG CS-11 for ACAT

  23. CMS – Clarens for interconnect (arrows) D. Olson, PPDG CS-11 for ACAT

  24. PROOF & Grid D. Olson, PPDG CS-11 for ACAT

  25. ALICE (by proxy) D. Olson, PPDG CS-11 for ACAT

  26. Themes (or opinions) • Varying degrees of depth to which grid penetrates interactive analysis: • Select data from grid and extract a local (non-grid) copy (proceed with interactive analysis independent of grid) • Run analysis as grid batch jobs while having intermediate results returned for monitoring • Run analysis as grid jobs while having intermediate results returned and have a control channel to jobs to interrupt or guide processing D. Olson, PPDG CS-11 for ACAT

  27. Sample Requirements • Ability to select/extract data objects from grid at one level below event (raw, ESD, AOD, … components) • Do not need arbitrarily fine-grained objects from grid (hit, track, …) • User interface/interaction should be same with or without network connection • Similar to web browser cache, I.e., same tool, same URL • Ability to debug grid jobs • Distributed databases (metadata, calibration/conditions, …) • Working single sign-on and VO/group/user authorization • Estimate of time & resources to run an analysis • Laundry list of requirements being developed in use-cases document, not all shown here D. Olson, PPDG CS-11 for ACAT

  28. Near-term goals • Interest in common metadata catalog • ATLAS, CMS collaborating on GriPhyN Virtual Data Catalog, others welcome • What about SRB/MCAT, AliEn? • Considering metadata catalog at event-component level • Considering AIDA, HepRep for results collection, extraction interface • Interest in PROOF-Grid • Interest in JAS-Grid • Finish use cases / requirements document • Example demos for SC2002 D. Olson, PPDG CS-11 for ACAT

  29. Longer-term planning • Develop detailed workplan for Sept. 9 (US Physics Grid Projects week in San Diego) • Discuss interaction & cooperation with Crossgrid work on interactive analysis • Consider grid interface to PROOF and JAS as good test of common services D. Olson, PPDG CS-11 for ACAT

  30. Summary • Just beginning to consider grid for interactive data analysis. • Aim at interfacing existing tools to grid services • To identify missing services and collaborate on defining/developing common services • HEP-specific metadata catalog • Interactive control/monitor interface? • Identify a few common projects, possible candidates are: • PROOF + Grid • JAS + Grid • Event component level catalog • … (work in progress) • Want close ties with other grid effort on interactive analysis D. Olson, PPDG CS-11 for ACAT

More Related