1 / 22

PPDG The Particle Physics Data Grid

PPDG The Particle Physics Data Grid. Making today’s Grid software work for HENP experiments, Driving GRID science and technology. (www.ppdg.net) Richard P. Mount. November 15, 2000. PPDG. Who is involved? How is it funded? What has it achieved?

april
Download Presentation

PPDG The Particle Physics Data Grid

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. PPDGThe Particle Physics Data Grid Making today’s Grid software work for HENP experiments, Driving GRID science and technology. (www.ppdg.net) Richard P. Mount November 15, 2000

  2. PPDG • Who is involved? • How is it funded? • What has it achieved? • How does it fit in to the big Grid picture? • How is it relevant for LHC?

  3. PPDG Collaborators

  4. PPDG Collaborators Particle Accelerator Computer Physics Laboratory Science ANL X X LBNL X X BNL X X x Caltech X X Fermilab X X x Jefferson Lab X X x SLAC X X x SDSC X Wisconsin X

  5. BaBar STAR D0 Condor Users BaBar Data Management STACS D0 Data Management Condor SRB Users CDF SRB Team CDF Data Management Globus Team Nuclear Physics Data Management Atlas Data Management CMS Data Management Nuclear Physics Globus Users Atlas CMS PPDG: A Coordination Challenge PPDG

  6. PPDG Funding • FY 1999: • PPDG NGI Project approved with $1.2M ($2M requested) from DoE Next Generation Internet program. • FY 2000 • DoE NGI program not funded • $1.2M funded by DoE/OASCR/MICS ($470k) and HENP ($770k) • FY 2001+ • Proposal (to be written) for DoE/OASCR/MICS and HENP funding in SciDAC context. Likely total FY2001 request: ~$3M.

  7. Initial PPDG Goals Implement and Run two services in support of the major physics experiments at BNL, Fermilab, JLAB, SLAC: • “High-Speed Site-to-Site File Replication Service”;Data replication up to 100 Mbytes/s • “Multi-Site Cached File Access Service”: Based on deployment of file-cataloging, and transparent cache-management and data movement middleware Using middleware components already developed by the collaborators.

  8. PPDG Site-to-Site Replication Service PRIMARY SITE Data Acquisition, CPU, Disk, Tape Robot SECONDARY SITE CPU, Disk, Tape Robot

  9. Progress:100 Mbytes/s Site-to-Site • Focus on SLAC – Caltech over NTON at OC48 (2.5 gigabits/s); • Fibers in place; • SLAC Cisco 12000 with OC48 and 2 ×OC12 in place; • Caltech Juniper M160 with OC48 installed; • 990 Mbits/s achieved between SC2000 and SLAC.

  10. Throughput from SC2000 to SLAC Up to 990 Mbits/s using two machines at each end plus multi-stream TCP with large windows

  11. PPDG Multi-site Cached File Access System PRIMARY SITE Data Acquisition, Tape, CPU, Disk, Robot Satellite Site Tape, CPU, Disk, Robot University CPU, Disk, Users Satellite Site Tape, CPU, Disk, Robot Satellite Site Tape, CPU, Disk, Robot University CPU, Disk, Users University CPU, Disk, Users

  12. PPDG Cached File Access Progress • Demonstration of multi-site cached file access based mainly on SRB*.(LBNL, ANL, U.Wisconsin) • Development of HRM storage management interface and implementation in SRB and SAM (D0 data management) * Storage Resource Broker (SDSC)

  13. Test of PPDG Storage Management API (HRM) • 2 separate Clients request and get files from: • SRB catalog and HPSS – LBL and Wisconsin • D0 SAM catalog, disk cache and Enstore storage system – Fermilab and Wisconsin. Demo’d at SC2000. • Agreed on common Storage Resource Management interface. • Next step – Client that requests and gets files from each/both storage management systems – goal to meet the PPDG “multi-site file caching file access” across 2 existing grid components.

  14. PPDG: Initial Architecture

  15. Initial PPDG “System” Components Middleware Components (Initial Choice): See PPDG Proposal Page 15 Object and File-Based Objectivity/DB (SLAC enhanced) Application Services GC Query Object, Event Iterator, Query Monitor FNAL SAM System Resource Management Start with Human Intervention (but begin to deploy resource discovery & mgmnt tools) File Access Service Components of OOFS (SLAC) Cache Manager GC Cache Manager (LBNL) Mass Storage Manager HPSS, Enstore, OSM (Site-dependent) Matchmaking Service Condor (U. Wisconsin) File Replication Index MCAT (SDSC) Transfer Cost Estimation Service Globus (ANL) File Fetching Service Components of OOFS File Movers(s) SRB (SDSC); Site specific End-to-end Network Services Globus tools for QoS reservation Security and authentication Globus (ANL)

  16. Local Site Manager Remote Services logical request (property predicates / event set) Properties, Events, Files Index Request Interpreter files to be retrieved {file:events} 7 4 2 6 8 1 9 3 5 Request Manager File Replica Catalog Request to move files {file: from,to} 11 10 13 12 Request to reserve space {cache_location: # bytes} Storage Reservation service Storage Access service File Access service Cache Manager Local Resource Manager Logical Index service Matchmaking Service Application (data request) Client (file request) Resource Planner Cache Manager GLOBUS Services Layer To Network Fig 1: Architecture for the general scenario - needed APIs

  17. Current PPDG Focus:File Replication Service • Use cases from BaBar, D0, CMS, etc. • Typical target: BaBar SLAC-Lyon transfers(current low-tech approach absorbs about 2 FTE). • Replica catalog distinct from Objectivity catalogs; • GRIDftp transfer. • Globus inter-site security.

  18. The Big Grid Picture Worldwide Grid Project Coordination Software Configuration Control Grid Architecture and Interface Definition Derived-Object DefinitionDatabase Cost/FeasibilityEstimation Resource Policy Reliable ObjectTransfer Resource Discovery Mobile Agents Matchmaking High-throughput IP Security/AuthenticationArchitecture Distributed Replica Catalog QoS, Reservations Security/AuthenticationTechnology Distributed Transaction Management Instrumentation Modeling Prototypes  Products Error Tracing Deployment in Experiments Testbeds User Support

  19. The Big Grid Picture • Grid projects must become coordinated (in progress); • Progress in the commercial world must be exploited;

  20. PPDG in the Big Grid Picture • Rapid deployment of Grid software in support of HENP experiments; • Drive and contribute to Grid architecture: • Architecture must define interfaces between evolving components; • Design and develop new Grid middleware components (deliverables to be defined in consultation with GriPhyN, EU-DataGrid …): • Focus on rapid delivery to HENP experiments (to validate concepts, get feedback and be useful).

  21. PPDG and LHC? BaBar Example PPDG-SLAC-IN2P3-BaBar plan to implement Grid components allowing SLAC + CCIN2P3 + … to become an (adequately) integrated data analysis resource. Delivery of useful service: scheduled for end 2001 RAL SLAC CCIN2P3 CASPUR

  22. PPDG and LHC • US LHC groups are strong participants in PPDG; • Computer scientists in PPDG see the LHC challenge as the leading opportunity to advance the science of data-intensive Grids; • PPDG, GriPhyN and EU-DataGrid are creating coordinated management and joint working groups: • Interoperable systems with consistent components.

More Related