60 likes | 149 Views
Control System Status Time Meeting 12/1/09 John Morris. Operations File Server NAS(Network Attached Storage) supports 6 file systems. cfse: logger data cfsf: other run data cfsb: op user areas cfsx: op software cfsy: general app config/storage, petTree archives, process archives
E N D
Control System Status Time Meeting 12/1/09 John Morris
Operations File Server NAS(Network Attached Storage) supports 6 file systems • cfse: logger data • cfsf: other run data • cfsb: op user areas • cfsx: op software • cfsy: general app config/storage, petTree archives, process archives • cfsz: http docs, acc phys storage areas
Operations File Server NAS(Network Attached Storage) containing 5 file subsystems • cfse: logger data • cfsf: other run data • cfsb: op user areas • cfsx: op software • cfsy: general app config/storage, petTree archives, process archives • cfsz: http docs, acc phys storage areas Thanksgiving Day 5pm: disk full 8pm: disk unavailable
First Response • Work with NAS vendor (Hitachi) to recover cfsy file system • Devoted significant time to recovery effort since accelerators were off • Ultimately unsuccessful, so... • Export cfsy file system from our hot backup file server • Reboot key server machines and restart Controls servers
Hot Backup Timing Hot backups of critical ops areas are made daily • 11/25 22:30 cfsy hot backup updated • 11/26 17:00 cfsy full Data saved to cfsy (general app_store, pet tree archives, process tree archives) between 11/25 22:30 and 11/26 17:00 is NOT present on cfsy now. Important question – how important is data saved during that time?
Our goals now • Understand reason for the failure and for difficulty in recovery • Recover Thanksgiving Day data from cfsy • Get system into preferred configuration for operations There is a limit (2 days?) to how long we will delay #3.