1 / 17

CHEP2000 Padova, Italy

The GIOD Project ( G lobally I nterconnected O bject D atabases) For High Energy Physics Harvey Newman, Julian Bunn, Koen Holtman and Richard Wilkinson A Joint Project between Caltech (HEP and CACR), CERN and Hewlett Packard http://pcbunn.cacr.caltech.edu/. CHEP2000 Padova, Italy.

amal
Download Presentation

CHEP2000 Padova, Italy

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. The GIOD Project(Globally Interconnected Object Databases)For High Energy PhysicsHarvey Newman, Julian Bunn, Koen Holtman and Richard WilkinsonA Joint Project between Caltech (HEP and CACR), CERN and Hewlett Packardhttp://pcbunn.cacr.caltech.edu/ CHEP2000 Padova, Italy

  2. The GIOD Project - Overview • GIOD Project began 1997, a joint effort of Caltech and CERN with funding from Hewlett Packard for two years • with collaboration from FNAL, SDSC • Leveraging existing facilities at Caltech’s Center for Advanced Computing Research (CACR) • Exemplar SPP2000, HPSS system, high speed WAN, CACR expertise • Build a prototype LHC data processing and analysis Center using: • Object Oriented software, tools and ODBMS • Large scale data storage equipment and software • High bandwidth LAN (campus) and WAN (regional, national, transoceanic) connections • Measure, evaluate and tune the components of the center for LHC data analysis and physics • Confirm the viability of the LHC Computing Models

  3. Components of the GIOD Infrastructure • Supercomputer facilities at CACR • Large pool of fully simulated multi-jet events in CMS • Experienced large-scale systems engineers at CACR • Connections at T3- >OC3 in the Local and Wide Area Networks; Fiberoptic links Caltech HEP/CACR • Strong collaborative ties with CMS, RD45, Fermilab and San Diego Supercomputer Center;CERN, CALREN-2 and Internet2 Network Teams

  4. Generation of CMS multi-jet events • Made possible by 1998, 1999 (NSF-sponsored) NPACI Exemplar allocations • Produced ~1,000,000 fully-simulated multi-jet QCD events since May 98; selected from 2 X 109 pre-selected generated events • Directly study Higgs   backgrounds for first time • Computing power of the HP-Exemplar SPP 2000 (~0.2 TIPs) made this attainable • Events used to populate a GIOD Object Database system • “Tag” database implemented and kept separately; Can be quickly replicated to client machines • In 2000: Proposal to NPACI requesting 25% of the Exemplar has been granted • Targeted at event simulation for ORCA (CMS) • Replicas of this database were installed at FNAL and Padua/INFN (Italy) Simple “Tag” class

  5. Scalability tests using the Exemplar • Caltech Exemplar used as a relatively convenient testbed for multiple client tests with Objectivity • Two main thrusts: • Using simple fixed object data • Using simulated LHC events • Results gave support to the viability of the ODBMS system for LHC data • CMS 100 MB/sec milestonemet (170 MB/sec achieved) > 170 MB/sec writing LHC raw event data to the database Up to 240 clients reading simple objects from the database

  6. Java 3D Applet to view GIOD events • Attaches to the GIOD database: allows to scan all events in the database, at multiple detail levels • Demonstrated at the Internet2 meetings in 1998 and 1999, and at SuperComputing’98 in Florida at the iGrid, NPACI and CACR stands ECAL crystals Java2 GUI HCAL towers Run/event selection widget Tracker geometry and hitmap Reconstructed Tracks Reconstructed Jets

  7. Other ODBMS tests Tests with Versant(fallback ODBMS) DRO WAN Tests with CERN Production on CERN’s PCSF and file movement to Caltech Objectivity/DB Creation of 32000 database federation

  8. Tests with Objy/Java binding and JAS Objy DIM and analysis using Java Analysis Studio Java2D Tracker viewer Java Track Fitter

  9. WAN tests: Caltech  SDSC,FNAL • Client tests between SDSC/CACR, CACR/FNAL and CACR/HEP • ftp, LHC event reconstruction, event analysis, event scanning • Investigated network throughput dependence on: • TCP window size, MSS, round trip time (RTT), etc. • payload (ftp, Objy, Web, telnet etc.) Simple ftp traffic Flattened by staggering client startups Objectivity Schema transfer 8 kB DB Pages

  10. WAN tests Caltech  SDSC,FNAL • Using “out of the box” single-stream ftp, achieved • ~7 MB/sec over LAN ATM @ OC3 • ~3 MB/sec over WAN @ OC3 • Expect to ramp up capability by use of • Tuned ftp (buffer, packet and window sizes) • Jumbo frames • New IP implementations or other protocols • Predict ~1 GB/sec in WAN by LHC 2005 using parallel streams • Measurements to be used as a basis for model parametersin further MONARC simulations

  11. Using the Globus Tools • Tests with “gsiftp”, a modified ftp server/client that allows control of the TCP buffer size • Transfers of Objy database files from the Exemplar to • Itself • An O2K at Argonne (via CalREN2 and Abilene) • A Linux machine at INFN (via US-CERN Transatlantic link) • Target /dev/null in multiple streams (1 to 16 parallel gsiftp sesssions). • Aggregate throughput as a function of number of streams and send/receive buffer sizes ~25 MB/sec on HiPPI loop-back ~4MB/sec to Argonne by tuning TCP window size Saturating available B/W to Argonne

  12. GIOD - Summary • GIOD investigated • Usability, scalability, portability of Object Oriented LHC codes • In a hierarchy of large-servers, and medium/small client machines • With fast LAN and WAN connections • Using realistic raw and reconstructed LHC event data • GIOD has • Constructed a large set of fully simulated events and used these to create a large OO database • Learned how to create large database federations • Developed prototype reconstruction and analysis codes that work with persistent objects • Deployed facilities and database federations as testbeds for Computing Model studies

  13. Associated Projects • MONARC - Models Of Networked Analysis at Regional Centers (CERN) • Caltech, CERN, FNAL, Heidelberg, INFN, KEK, Marseilles, Munich, Orsay, Oxford, Tufts, … • Specify candidate model’s performance: throughputs, latencies • Find feasible models for LHC matched to network capacity and data handling • Develop “Baseline Models” in the “feasible” category • PPDG - Particle Physics Data Grid (DoE Next Generation Internet) • Argonne Natl. Lab., Caltech, Lawrence Berkeley Lab., Stanford Linear Accelerator Center, Thomas Jefferson National Accelerator Facility, University of Wisconsin, Brookhaven Natl. Lab., Fermi Natl. Lab., San Diego Supercomputer Center • Delivery of infrastructure for widely distributed analysis of particle physics data at multi-PetaByte scales by 100s to 1000s of physicists • Acceleration of development of network and middleware infrastructure aimed at data-intensive collaborative science. • ALDAP - Accessing Large Data Archives in Astronomy and Particle Physics (NSF Knowledge Discovery Initiative) • Caltech, Johns Hopkins University, FNAL • Explore data structures, physical data storage hierarchies for archival of next generation astronomy and particle physics data • Develop spatial indexes, novel data organisations, distribution and delivery strategies. • Create prototype data query execution systems using autonomous agent workers

  14. Future Directions: GIOD II • Review the advantages of ODBMS vs. (O)RDBMS for persistent LHC data;in light of recent (e.g. Web-enabled) RDBMS developments, for HEPand other scientific fields • Fast traversal of complex class hierarchies ? • Global (“federation”) schema and transparent access ? • Impedance match between the database and the OO code ? • What are the scalability and use issues associated with implementing a traditional RDBMS as a persistent object store for LHC data? • What benefits would the use of an RDBMS bring, if any ? • Which RDBMS systems, if any, are capable of supporting, or projected to support, the size, distribution and access patterns of the LHC data ?

  15. GIOD II : Other New Investigations • What are the implications/benefits for the Globally-distributed LHC computing systems of: • Having Web-like object caching and delivery mechanisms (distributed content delivery, distributed cache management) • The use of Autonomous Agent query systems • Organizing the data and resources in an N-tiered hierarchy • Choosing (de facto) standard Grid tools as middleware • How can data migration flexibility be built in ? • Schema/data to XML conversion (Wisdom, Goblin) ? • Data interchange using JDBC or ODBC • Known format binary files for bulk data interchange:for simple and efficient transport across WANs

  16. GIOD II and ALDAP Optimizing performance of Objectivity for LHC/SDSS data • Use of Self Organizing Maps (e.g. Kohonen) to recluster frequently accessed data into collections in contiguous storage • Use of Autonomous Agents to carry queries and data in WAN distributed database system • Identify known performance issues: get them fixed by the vendor • Example 1: 11,000 cycles cf. 300 cycles overhead to open an object • Example 2: Selection speeds with simple cuts on Tag objects • Make new performance comparisons between Objectivity and ER database (SQLServer) • on identical platforms, • with identical data, • with identical queries, • with all recommended “tweaks”, • with all recommended coding tricks • We have begun tests with SDSS sky objects, and with GIOD”Tag” objects

  17. GIOD II and PPDG Distributed Analysis of ORCA data • Using Grid middleware (notably gsiftp, SRB) to move database files across the WAN • Custom tools to select subset of database files required in local “replica” federations, and attach them once copied • Making “compact” data collections • Remote requests from clients for sets of DB files • Simple staging schemes that asynchronously make data available, and give ETA for delivery, and migrate “cool” files to tertiary storage • Marshalling of distributed resources to achieve production task goals • Complementary ORCA DB files in Caltech, FNAL and CERN replicas • Full pass analysis involves distributing task to all three sites • Move/compute cost decision • Task and results carried by Autonomous Agents between sites (work in ALDAP)

More Related