1 / 21

Surviving Disaster

Surviving Disaster. Presented by: Wayne Lyle, Project Manager Dilworth Paxson LLP Jenn Steele, Director of Information Technology Donovan Hatem LLP Moderator: Ivaylo Nikolov, Director of IT Davies Ward Phillips & Vineberg LLP. Agenda. Jenn’s Story: Waterfall!

fordon
Download Presentation

Surviving Disaster

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Surviving Disaster Presented by: Wayne Lyle, Project ManagerDilworth Paxson LLP Jenn Steele, Director of Information TechnologyDonovan Hatem LLP Moderator: Ivaylo Nikolov, Director of ITDavies Ward Phillips & Vineberg LLP

  2. Agenda Jenn’s Story: Waterfall! Wayne’s Story: Great Flood of ‘07 Panel Discussion/Q&A Lessons Learned

  3. First, Some Information 75 Attorneys 3 Offices Centralized in Boston Founded March 1, 2001 Moved to Current Location Memorial Day Weekend, 2002

  4. Once Upon a Time… HVAC

  5. Sunday, December 2, 2006, 6:45 PM

  6. Order of Events: Waterfall 1 Initial Call 12/2/20066:45 PM Finished Dismantling Equipment 11:00 PM Reassembled Equipment 12/3/2006 7:00 PM Essential Equipment Up & Running 12/4/2006 4:30 AM HVAC Turned Back On 12/5/2006 5:00 PM Systems Go; Finally Went Home 12/7/2006 6:45 PM

  7. Thursday, December 7, 2006, 7:00 PM

  8. Order of Events: Waterfall 2 Initial Call 12/7/20067:00 PM Dismantled Equipment & Set up Sneakernet 11:30 PM Supported Firm on Sneakernet 12/8/2006 7:00 AM Expert Sealed in Kitchen with Equipment 12/8/2006 6:00 PM Started Reassembling Equipment 12/9/2006 11:00 AM Most Systems Go 12/9/2006 7:12 PM

  9. Consequences Data Loss Client data on one NAS only Sent for recovery Equipment Loss Expert cleaning/assessment Surprisingly low Sanity Loss None to begin with…

  10. About that DR/BC Plan… Pre-flood DR/BC Plan Backup tapes & online backup Cross fingers & pray Post-flood DR/BC Plan Backup tapes & online backup VMWare Implementation as full BC solution in progress Cross fingers & pray “Well, we were fine the first time without one…”

  11. New Set-Up HVAC

  12. January 18, 2007 Email: All Wet From: Wayne J. Lyle Sent: Thursday, January 18, 2007 8:30 AM To: Jennifer E. Steele Jenn:      Now I know how you felt the day you got rained on.  We had a major flood on our 6th floor which has our communications room (Phone / Network Hub) as a couple hundred gallons of hot water came falling out of the ceiling.      Did your incident include any dealing with paper files?  If so did you bring in someone special to deal with recovering wet documents and books?  Wayne

  13. Wayne’s Story Background Information IS Director for a law firm in Portland ME. Approximately 80 Attorneys 4 Offices January 16, 2007 Just your average winter day But then ……

  14. “The Great Flood of January 07”

  15. What Happened • A Major HVAC Failure • Frozen VAV Boxes • System Reset – Ooops • VAV Units Burst • Thousands on Gallons • Went on for 20 minutes • Standing water

  16. Impact • Major loss of facilities • Floor was deemed uninhabitable • 30%+ Loss office space • Minor loss of equipment • Mostly monitors and telephones • Service Losses • Financial System offline for 4 hours • Phone System down for 15 minutes

  17. Business Continuity • Re-Establish • Location • Vendor • Rehab • Reclaiming the office • Recovery • Communicate • Internally • Externally • Return • Never Again • Lessons

  18. About the DR / BC Plans • Before the Flood • Distributed Assets • Multi-Site Disk Based Backup • Threat Matrix • After Flood • Virtualization • Investigate Automated Failover • Enhance Threat Matrix

  19. Panel Discussion/Q&A

  20. Lessons Learned Communications both internally and externally are crucial to restoration Delegate even the most minor issues Work the problem. Establish milestones for restoration. Manage your personnel assets Consider the risks to the personnel

  21. Lessons Learned Prior planning—both while designing and later Know what vendors to call Think about the insurance claims Understand short tempers and other emotional effects Maintain perspective

More Related