1 / 21

Overview of Database Backup and Hierarchical Storage Management (HSM) Techniques

This document provides a comprehensive overview of database backups and hierarchical storage management (HSM) as implemented in the DSS (Digital Sky Survey) project. It details methodologies for efficient data handling, the importance of backup strategies, and the transition from older to new backup systems using technologies like DLT jukeboxes and BackupExec. The significance of ensuring data integrity and ease of retrieval is emphasized, particularly in managing large volumes of astronomical data. Various backup completion statistics and resource URLs are also included for further reference.

lev-perry
Download Presentation

Overview of Database Backup and Hierarchical Storage Management (HSM) Techniques

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. DSS/Db Backups/HSM Overview David Wolfe

  2. DSS-I • 102 CD ROM set • POSS-I redXE plates • SERC-J blueS plates • Compressed ~10X • http://archive.stsci.edu/cgi-bin/dss_form

  3. DQVS • Palomar Quick-V • GSC-I construction plates • 39 CD ROM set • Compressed ~10X • http://archive.stsci.edu/cgi-bin/dss_form

  4. XDSS • RED ( III-aF) • POSS-II XP images • Completed 897 out of 897, 100% • UK Schmidt XS images • Completed 592 out of 606, 98% • UK Schmidt ER images • Completed 270 out of 288, 94% • Compressed ~10X • Contained on Volumes 0001-0390 • Completed by end of 2000 • http://archive.stsci.edu/cgi-bin/dss_form

  5. XDSS(cont) • BLUE (III-aJ) • POSS-II XJ images • Completed 897 out of 897, 100% • Contained on Volumes 1001-1226 • INFRARED (IVN) • POSS-II XI images • Completed 340 out of 897, 38% • Contained on Volumes 2001-2084 • UK Schmidt IS images • Completed 151 out of 894, 16% • Contained on Volumes 3001-3043 • Compressed ~10X • Completed by mid 2001, telescope limited • http://archive.stsci.edu/cgi-bin/dss_form

  6. XDSS(cont) • Headers • Headers for all XDSS second generation scans • ftp://ftp.stsci.edu/software/casb/xdss/

  7. DB BACKUPS • DataBase about 1TB • How to handle such a large volume • Old method • Stand alone DLT drive, NT Backup • 9-10 hours per tape • 15+ tapes

  8. DB BACKUPS (cont) • New method • BackupExec by Veritas • DLT jukebox, 8 cartridges • Breakup into North and South • High through put • Less user interaction

  9. DB BACKUPS (cont) • Importance of the Backups • Large system, just in case of errors • Already had to use previous backups • Replaced corrupted files • Time Frame • Plans are for monthly • Minimal impact to production

  10. HSM • Hierarchical Storage Management • DiskXtender by OTG • Large storage by using DLT jukeboxes • 2 - 46 DLT tape jukeboxes • ~60GB/tape • Storing processing data • OOP, cutouts, deblend, rank…

  11. HSM (cont) • Ease of retrieval • All files visible from explorer • Right click on file to retrieve • Limited by physical disk space

More Related