1 / 33

Redesigning the Enterprise to Achieve 5-Nines

Redesigning the Enterprise to Achieve 5-Nines. Link Alander , Associate Vice Chancellor Shah Ardalan , Vice Chancellor/CIO September 2010. 1 - Key Objective 37 – Major Projects 18 months. 99.999. 1 - Key Objective 37 – Major Projects 18 months. 152 - OTS Staff Members.

christinesloan
Download Presentation

Redesigning the Enterprise to Achieve 5-Nines

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Redesigning the Enterprise to Achieve 5-Nines Link Alander, Associate Vice Chancellor Shah Ardalan, Vice Chancellor/CIO September 2010

  2. 1 - Key Objective 37 – Major Projects 18 months 99.999

  3. 1 - Key Objective 37 – Major Projects 18 months 152 - OTS Staff Members

  4. 1 - Key Objective 37 - Projects 18 months 6 – Strategic Partners 152 - OTS Staff Members

  5. agenda Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close • Project High Level Overview • Overview Lone Star College System • Our Challenges • Our Approach - Simplify IT • It Isn’t Just Technology - IT Maturity • Dynamic Master Plan • LAN/WAN, AD, SAN, VM ……. • Design Scenario - Modeling • The Outcomes – Into the Cloud • Lessons Learned – Takeaways • Q&A

  6. Overview LSCS - Enterprise Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close • 13 Locations across 1,400 square miles • 5 Colleges • 2 University Centers • 5 Instructional Centers • Systems Office and Training Center • 21 new buildings this year • 5 new geographic locations • Our Customers (90,000 strong) • 69,363 Credit Students • 21,000 Non Credit Students • Average 25% growth per semester • Over 4,800 employees • Supported by 152 OTS staff members

  7. Our Challenges • This was not our only project! • 20 Strategic Initiatives • New ERP • IT Security • Client Mgt System • … much more Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  8. The plan – Simplify IT March 2008 The term “Simplify IT” can be found in every business journal, book, and a couple million times on the web “What does it mean?” • Evaluate Everything – Reduce Complexity • Set Enterprise Standards • Select “Best in Class” Solutions • Develop Detailed Process Methodologies • Increase Staff Knowledge • Reduce Interdependencies Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  9. It isn’t just technology! You cannot achieve 5-nines with technology alone! - It is about People, Process and Technology By raising your IT maturity, you will become a Strategic Business Partner that will add value to the organization IT Maturity Model and Assessment Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  10. Dynamic master Plan • Target Objective - (99.999%) Tier I • Eliminate Single Points of Failures • Expand Replication Technologies • Leverage Automation • Deep Monitoring Strategy 18 months to build new environment and migrate users Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close • Key Objectives • Network Redesign • Active Directory Infrastructure • Maximize Virtualization • Security and Monitoring • Assessments • Enterprise Standards • Storage Area Network • Data Centers

  11. Network Redesign – Starting Point March 2008 • Starting Point - Network • 300MB ONS Ring • 48 Single Points of Failure • 4,599 miles of Dark Fiber • 1 – 500 MB Internet connection • 1 – 10 MB Internet connection DR only Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  12. Network Redesign - Head End December 2009 • New Network Head-End • 2 physical locations 37 miles apart • Logically separated networks (not extended) • Nexus 7000 & 5000 • Dual Internet Connections • Redundant firewalls • Redundant DMZ • ACE Load Balancer • 10 GB SAN Fabric X 4 Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  13. Network Redesign - WAN July 2010 • Final State - Network • Full Mesh 10GB links • 28 VSS Core Switches • 49,240 Network ports • 1,374 Network devices • 4,750,342 Sq ft of Wireless network coverage • Comprehensive network monitoring - SolarWinds Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  14. Active Directory - Exchange June 2008 • Phase I - Build New AD/Exchange Environment • Problems • 14 domains with limited trusts (one-way) • 27 DNS zones • 16 Wins servers • Constant replication errors • No health monitoring • No automated account management Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  15. Active Directory - Exchange March 2009 Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close Phase II – Migrate Users • New AD Environment • 2 Domains (Admin/Student) • Exchange 2007 • Challenges • 10,000 Exchange accounts • 27 DNS zones to 5 • 16 Wins servers to 3 • Automation (Self Healing) • Implemented Change Auditor • Implemented Health Monitoring • Automated Account Provisioning

  16. Active Directory - Exchange November 2009 Phase III – Migrate Servers Decommission Old Domain • Challenges • Over 400 physical servers • 14 locations • 70% of hardware end of life • Deployed Enterprise Virtualization Solution • Decommissioned260Servers Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  17. SAN – Back-up & Recovery • Starting Point - SAN • 2GB Broacade • 20 TB replicated • Limited A/B Paths • Majority single paths • No “Single Pain of Glass” to manage the SAN • No true integration with backup solution Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close Starting Point - SAN 3 Disparate SAN’s in Main Data Center 5 Separate Solutions Deployed on Campuses End-of-Life Arrays No Standard or Reliable Back-up and Recovery Solution High Data Security Risks

  18. SAN – Backup/Recovery September 2008 March 2009 September 2009 Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close • Consolidated all SAN’s • Traditional Tier I,II & III • High Availability Design • Disk to Disk Enterprise Backup • Critical for Short RTO, RPO Windows • Single Enterprise Tape/Off-Site Solution • Integrated EMC Storage Management with VM Storage Virtualization

  19. Main Data Center July 2008 Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close • One Main System Data Center • Tier II design • Inefficient – high risk! • Campus Data Centers • Lacked • Back-up power/cooling • Fire Suppression • Evaluated Hosted Locations • Problems extending co-located hot systems • Typical All Hot, All Cold or Warm (not Active-Active) • Acquired New Facility - LSC-University Park • Formerly Compaq World Headquarters

  20. Main Data Center September 2010 Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close • Decision to Build New Data Center – March 2010 • Selected Design and Construction Firm – June 2010 • Commissioned New Tier IV Data Center – September 2010 • Focus - Green – High Density Design • Dual Commercial Power Feeds • Dual Generator Plants • 240KW load capacity – per A/B Path • 99.9 % Efficient and Scalable UPS • Closed Loop Cooling (water cooled racks) • In-Rack Fire Suppression System • Packed into 970 square feet!

  21. Virtualization • LSCS-VM Farm Capacity Today • 2 Virtual Centers • 14 Virtual Data Centers • 468 VM’s • 327 SRM Cold VM’s • 136 Physical CPU’s • 544 CPU Cores • 1,500 GHz • 1.4 TB Memory • 790 VM CPU Count • 4 HP C7000 Blade Systems • 6 HP C3000 Blade Systems Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close Starting Point - Virtualization • Less than 5% of servers were virtualized • Limited Test-Dev Utilization • A few Production Servers • Limited Deployments • System Office • One Campus • Typical Single Server License Purchased

  22. Virtualization December 2008 September 2009 VM-SRM moves servers VM-SRM moves servers VM-SRM moves servers Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close • VM First Policy • VM Farm Design • Campus VM Solution • VMware ELA (all u can eat) • Lifecycle Manager • Site Recovery Manager • Today 82% Virtualized • Including Tier I applications • Target 95% by May 2011 • Storage Virtualization

  23. Enterprise DEEP Monitoring January 2009 February 2009 May 2009 Implemented Proactive Monitoring - Server and Service Level • Deep Monitoring • Service Level • Quest Foglight/Spotlight • Oracle Enterprise Manager • Alert Bot • Server Level • vFoglight • Solarwinds • Servers Alive • Nexpose` • E-Alert Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close Starting Point – Nothing!

  24. Key Performance Indicators! January 2009 September 2009 January 2010 Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close Starting Point – Measured Nothing! • Key Performance Indicators • Financial (8), Operational (65), Strategic (4) • Risk Performance Indicators (32) • Reporting • Standard Monthly Reports • Executive Overview • Capacity / Demand Management • Agility! • Drills • Predefined Drills to Test Availability

  25. Modeling – PeopleSoft (FSM Failover) 100 Virtual Server – 2 Physical Database Servers Per Side Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  26. Modeling – People Soft (FSM Failover) NORMAL Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  27. Application Level - PeopleSoft ERROR – FSM WEB Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  28. Application Level - PeopleSoft ERROR – FSM WEB and APP Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  29. Application Level - PeopleSoft ERROR – FSM WEB, APP and Database Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  30. Outcome – into the clouds SAS Hybrid Dynamic Services Hybrid Dynamic Services Dynamic Services Hybrid Hybrid/Public Traditional Public Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  31. Lessons Learned It’s not a Train! Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close • Dynamic Master Plan • Enabled us to move faster • Made scope creep difficult to manage • Staff exhaustion • Motivating everyone – staff and customers • There is a light at the end of the tunnel – and - • Communication and Constant Measurement • Training, Training and more Training • Containing VM Sprawl • Continuous Service Improvement (CSI) – Is critical during the life of the project, not just after

  32. today Internet Redundancy WAN Redundancy Data Center & Co-Location Co-located VM Farms Storage Redundancy LAN Redundancy Server Virtualization (SRM) Storage Virtualization Intro Planning LAN/WAN AD/Exchg SAN Datacenter VM Monitoring KPIs Modeling Close

  33. Thank You - Q & A One Major Initiative - “5-nines” – 37 Major Projects • WAN Phase I – Core • Infrastructure Assessment • Wireless Network Phase I • VoIP Rebuild • Unified Infrastructure Project Phase I • SAN Phase I • VM First Policy – Datacenters • Quest Foglight • Quest Spotlight • KPI’s • Orion SolarWinds • Quest vFoglight • EMC - Avamar • Unified Infrastructure Project Phase II • EMC – Networker • Quest Change Auditor • VoIP Phase II • SAN Phase II • ERP Architecture • ERP Side A • Campus VM solutions • Cisco ACE Load Balancer • Unified Infrastructure Project Phase III • Temp Datacenter • SO Datacenter • Nexpose Deployment • Orion SolarWinds P II • Dual Internet Connections • ERP Side B • SAN Assessment • New Datacenter – CoLo • VM Assessment • LAN/WAN Phase II • SAN Phase III • Service Level Monitoring • New Datacenter Network • Datacenter infrastructure monitoring SO/CoLo

More Related