1 / 9

Overview of HPSS for Archival Storage at SDSC: Usage, Capabilities, and Future Enhancements

This document provides an introduction to the High Performance Storage System (HPSS) at the San Diego Supercomputer Center (SDSC), detailing the configuration, load, usage statistics, and different classes of service. With eight servers for movers and comprehensive tape drive storage, HPSS manages about 100TB across nine million files, ensuring efficient data handling and class-dependent file management. Future enhancements in disk caching and technology upgrades are also discussed to optimize archival data storage and retrieval processes.

mervyn
Download Presentation

Overview of HPSS for Archival Storage at SDSC: Usage, Capabilities, and Future Enhancements

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HPSS for Archival Storage Tom Sherwin Storage Group Leader, SDSC sherwint@sdsc.edu

  2. Outline • Introduction to HPSS at SDSC • HPSS Load and Usage Statistics • HPSS Classes of Service • File Behavior • HPSS Futures

  3. Introduction to HPSS at SDSC • 8 servers for movers (disk and tape) • 1 server for HiPPi mover • 20 3590 tape drives • 1.5 TB disk cache - 1TB MaxStrat, .5TB SSA • SP Switch • HPGN

  4. HPSS Load and Usage Statistics • 100TB stored in 9 million files • Average file size: 11MB • ~ 16,000 transactions per day • ~ 250-300GB moved per day • 650GB peak amount of data moved

  5. tiny 0KB-8KB small 8KB-2MB medium 2MB-200MB large 200MB-6TB tape 6TB+ SRB automatically selects COS based on file (container) size HPSS Classes of Service (COS)

  6. File behavior • data written to disk • data rate varies 4.5MB/sec and up • data copied to tape within one hour • data lifetime on disk cache • COS dependent, ~10 days for medium COS • data retrieval from tape • minimum 20-40 seconds for tape ready • user sees first byte when entire file is on disk

  7. HPSS Futures • HPSS 4.1.1 • General enhancements and fixes • Investigating STK 9840 technology • same data rates as 3590, 20GB per tape • 10-20 drives per silo wall, midpoint load • 3590 Upgrades to 256 tracks

  8. HPSS Futures (continued) • Disk Cache • Teraflop machine requires additional 3-5TB • Networking • HPGN access to CalREN2, vBNS, NTON

More Related