Status of HPSS

1. Status of HPSS New Features, Requirements, and Installations Otis Graf IBM Global Services - Federal Houston, Texas October 1999

2. Topics • HPSS New Features since Release 3.2 • Next incremental updates • Feature to be added in 4.2 • Status of HPSS Installations • Focus on Support Services • Next Requirements Development Cycle

3. Summary of HPSS Releases and Patches

4. FileA FileB (F1) FileC (F2) Level 1 Level 2 F2 Unassigned F1 FileC FileA FileB Features Added Since Rel. 3.2 • Includes through 4.1.1 (July 1999) • File families • Segregate files onto different groups of tapes. • Storage Server uses family id passed in segment create calls for selecting the tape for the segment create

5. Root Fileset \ usr bin home ... goodwinj teaff junction junction Features Added Since Rel. 3.2 (Cont.) • Filsets: A hierarchical collection of files and directories managed as a single unit. fs_goodwinj fs_teaff \ \ work play play games

6. Features Added Since Rel. 3.2 (Cont.) • HPSS Filesets • Managed by name server • Can be either HPSS-only or HPSS/DFS • DFS Filesets • DFS-only • HPSS/DFS archived • HPSS & DFS name spaces not kept in synch • Only way to get data is through DFS • Performance similar to DFS-only if data is resident • HPSS/DFS mirrored • HPSS & DFS name spaces kept in synch • Data can be seen and altered from either DFS or HPSS • HPSS high speed data movement can be used

7. Features Added Since Rel. 3.2 (Cont.) • PFTP Enhancements • Allows the use of multiple network and/or multiple systems for child processes for multi-network stripe transfers • MPI-IO Interface • Implements a subset of the MPI-2 standard • Coordinates access to HPSS files from multiple processes

8. Features Added Since Rel. 3.2 (Cont.) • Scaleable Accounting • Accounting summary records support the following collection options during the accounting period: • By account index & COS, • Number of accesses • Number of files transferred • bytes used • By account index, COS, & storage class • number of files transferred • number of accesses

9. Features Added Since Rel. 3.2 (Cont.) • Performance Enhancements • Support disk partitions > 2 GB • Bigger virtual volumes (up to 16,384 segments per VV) • Faster disk file creates and purges • Support for Shelf Tapes • Provide utility to identify and move volumes to shelf • Mark cartridge metadata with shelf descriptor • Generate operator mount requests via SSM pop-up • Client API Enhancements • Manually migrate/purge on a file basis • Lock/unlock to prevent a file from being purged

10. Features Added Since Rel. 3.2 (Cont.) • Non-DCE client API (Solaris, IRIX & AIX) • Non-DCE Mover • Mover ported to other platforms • Solaris (DCE) • SGI IRIX (non-DCE) • Additional Tape Drive Support • Ampex DST 314 • IBM 3590E • Additional Library Support • ADIC Automated Media Library (AML)

11. Number of HPSS Systems* • *Notes: • “System” is a name space instance intended for production • 1999 data is “year to date”.

12. Listing of HPSS Sitesas of October 1999

13. Listing of HPSS Sites (Cont.)as of October, 1999

14. Listing of HPSS Sites (Cont.)as of October, 1999

15. Current Activities • Next scheduled patch (4.1.1.2 in Nov 99) • Sun Phase 2 • STK PVR and IBM 3494 PVR • PFTP Client and Daemon • MPI-IO • SFS Backup utilities • Some manageability improvements • Follow-on patch (first quarter 00) • IBM 3590 tape drives on Solaris • StorageTek RAIT - testing and prototyping activity, no HPSS software change

16. Current Activities (Cont.) • Development of Release 4.2 • Planned date is 4th Quarter 00 • Enhancement of On-Line Resources • Tools repository • On-line help (FAQ & problem/solution DB) • On-line documentation (HTML & PDF) • New Training Classes • Advanced System Admin Class • HPSS API programming class

17. Current Activities (Cont.) • Enhancement of Customer Support • Better coordination with IBM Transarc • Deployment/support procedures

18. Features in Rel. 4.2 • Multiple Storage Subsystems* • Comprised of Name Server, Bitfile Server, Migration Purge Server, & 1-n Storage Servers • May be used to enhance concurrency • May be used to partition servers to handle parts of the name space Data assigned to Storage Subsystem based on name • Fileset creation established association with Name Server • Junction attached fileset to name space • *Note: HPSS Contract terms are pending for multiple name spaces.

19. Features in Rel. 4.2 (Cont.) • Federated Name Space • Defined as cooperative, distributed HPSS systems linked at the internal server level • Supports use of all HPSS interfaces across multiple HPSS sites • Maintains autonomy of individual HPSS installations • Name spaces linked by junctions which point to filesets

20. Features in Rel. 4.2 (Cont.) • Command Line Utilities • Provide command line interface to SSM to enable management of HPSS from automated scripts • Command-line program (hpssadm) which can be run interactively or in batch mode • Allows user to manage servers, devices, storage classes, PVL jobs, and, to a limited extent, volumes

21. Features in Rel. 4.2 (Cont.) • Non-DCE Client API Security • Note: The majority of the non-DCE Client API has been developed for released in Release 4.1.1 • Support client authorization / authentication by: • None • DCE (for platforms with DCE) • Kerberos • Provide Non-DCE Client Gateway (NDCG) with the ability to authenticate clients’ identities on a per connection basis

22. Features in Rel. 4.2 (Cont.) • Mass Configuration • Provide mechanisms to create multiple server or drive / device records in a single request • Provide mechanism to set persistent default values used in creating new configuration records • A configurable default Log Policy will be provided, which may reduce the number of specific policies which need to be created

23. Features in Rel. 4.2 (Cont.) • Gatekeeper • Provides optional client interface to allow sites to schedule and monitor use of HPSS file resources • Supports file create, open, stage, and close requests • May be associated with one or many Storage Subsystems • Sites will be able to implement site specific user level scheduling of storage requests. This may include: • Limit the number of open files per user/host • Prevent create requests for a user/host • Collect statistics

24. Features in Rel. 4.2 (Cont.) • Account Validation • Maintain integrity of user account information. Users will only be able to use valid accounts • Support cross-linked namespaces (remote sites). Individual sites select their own style of accounting • While still supported, no longer be necessary to have user default account information in DCE registry • Sites that do not need accounting or validation of user accounts may continue to run in the current fashion

25. Features in Rel. 4.2 (Cont.) • Sun Solaris Phase 3 • Port remaining servers • Target OS is Solaris 8 • HPSS back-end the IBM General Parallel File System (GPFS)

26. Post Release 4.2 Discussions • No definitive release plans yet • Features that were delayed from Rel. 4.2 • Tape import/export • Multiple distributed movers - allow multiple, possibly distributed movers to access a single device • Begin new requirements development cycle: 2 Qtr 00 • Features under discussion • Improved manageability features • Steps towards server consolidation and decreasing SFS dependencies • Improved GPFS/HPSS performance through parallel data paths • Backup/archive feature (file servers, workstations) • Critical HEP requirements

27. Next Requirements Development Cycle • Process • Process begins at start of a new release cycle. • Requirements contributed by customer and IBM. • Requirements team consists of a representative from each customer site. • Release contents are prioritized by requirements team. • Developers responsible decomposing requirements into subsystem level.

28. Requirements Development Cycle (Cont.) • Process (Cont.) • Requirements are costed, and then reprioritized as an iterative process. Customer balloting influences release contents, but is not binding. • Project metrics used to determine the number of requirements accepted in the next baseline. • Requirements not making baseline are included in the requirements document, but annotated as future release. • Formal inspection of requirements is performed.

29. Requirements Development Cycle (Cont.) • Process (Cont.) • Any changes to the approved baseline require Technical and Executive Committee approval. • Subsystem Requirements documents are published once a baseline is approved. • Opportunity for the HEP Community • HEP community develop common set of requirements • Provide unified voice on requirements committee

30. Additional Resources • HPSS User Forum Presentations • www5.clearlake.ibm.com“What’s New” • Site Reports • Details for current capabilities • Details of Release 4.2 • Details of DFSS and HPSS • Multi-Platform Capability of HPSS Components • www5.clearlake.ibm.com“Product Information”  ”Multi-Platform Capability of HPSS” • Core Servers and Movers • User Interface Clients