1 / 15

Test Results of the EuroStore Mass Storage System

Test Results of the EuroStore Mass Storage System. Ingo Augustin CERN IT-PDP/DM 9.2.2000 Padova. EuroStore. EC funded ESPRIT IV project A High Performance Storage Project EP-26317 March 1998 - August 2000 Participants: CERN (HEP) DESY (HEP) QSW (Super Computer Manufacturer)

fisseha
Download Presentation

Test Results of the EuroStore Mass Storage System

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Test Results of the EuroStore Mass Storage System Ingo Augustin CERN IT-PDP/DM 9.2.2000 Padova

  2. EuroStore • EC funded ESPRIT IV project A High Performance Storage Project EP-26317 • March 1998 - August 2000 • Participants: CERN (HEP) DESY (HEP) QSW (Super Computer Manufacturer) HCSA (Space Applications) AMC (Private Hospitals) HNMS (Meteorological Service) TERA (Medical Foundation) Ingo Augustin, CERN

  3. CERN Role • Definition of User Requirements • LHC Central Data Recording • LHC data storage, administration and handling • Prototype Assessment at CERN • scalability • performance • reliability • manageability • Liaison with HPSS • synergy in data exchange • (lack of interest of the HPSS consortium) Ingo Augustin, CERN

  4. User Requirements • Real Time Capabilities • dedicated disks, tape drives for CDR • dynamic allocation of resources • quota and priorities for users and groups • file size and number only limited by OS • high reliability and availability • tape striping • General • “multi-vendor” • easy manageability • tape vaulting Ingo Augustin, CERN

  5. The EuroStore Software Ingo Augustin, CERN

  6. Parallel File System • Extended CS-2 like file system • Distributed over multiple nodes and disks • POSIX compliant • UNIX based • initial implementation on SUN Solaris, later on DECUnix • Resource Management System • Planned RAID 5 extension Ingo Augustin, CERN

  7. Hierarchical Storage Manager • IEEE MSS model • JAVA based • native THREADS (multi-threaded) • easily portable • Objectivity V5 as metadata DB • Communication via ssh with Blowfish encryption • Configurable priorities, quotas, allocations, migration Ingo Augustin, CERN

  8. Prototype Platform • Hardware, PFS and HSM in April 99 at CERN • Platform (QM-1): • 4 dual-processor SUN Enterprise 450 • connection via ELAN/ELITE network (~ 250 MB/s) • 4 x 4 8GB Cheetah SCSI-disks • Gigabit Ethernet interfaces • Prototype restrictions • movers on QM-1 nodes • STK robotics with Redwood or 9840 only Ingo Augustin, CERN

  9. Standard Test Configuration Ingo Augustin, CERN

  10. Interest of CERN • Not the PFS • one HSM demon per PFS • only efficient with “non-commodity” network with optimized kernel Ingo Augustin, CERN

  11. Networking • A PFS is only useful with optimized TCP-stack • ELAN is far too expensive for large usage (~10 kCHF/port) Ingo Augustin, CERN

  12. Storage Mangement Common problems in existing systems: • Manageability • no static environment (new equipment almost all the time) • monitoring (efficiency, problem detection) • few operators • Scalability • number of demons • database access • Reliability • hundreds of tape and disk servers • fault tolerance • Distributed • performance (PC/Linux?) • heterogeneous? • regional centers? Ingo Augustin, CERN

  13. EuroStore HSM Status of “Proof of principle” • Fault tolerant (eg. mover or PVR dead) • Java works • Data rates are limited by hardware only • Data is safe (only one incident of data loss, cause has been fixed) • Very good monitoring (jobs can be traced throughout the system) All problems found up to now were bugs, not design faults • Several components still to be implemented Ingo Augustin, CERN

  14. Ingo Augustin, CERN

  15. Future • EuroStore will finish in August 2000 • implementation of missing features • stress testing Deployment at DESY (see next talk) • Proposal for follow-up submitted to EC in Jan. 2000 • commodity • Linux, PC, low end storage (DVD-RAM-Jukebox…) • network • scalability • additional clients • SAN support (by STK) Ingo Augustin, CERN

More Related