IT Consolidation Savings – Post Implementation

1. IT Consolidation Savings – Post Implementation Neal Weatherspoon, CPA, CISA, CISSP IT Audit Manager Oregon Secretary of State – Audits Division NSAA IT Conference - September 2011

2. IT Environment – The Wild, Wild, West… Highly decentralized governance Prior attempt by legislative order to Consolidate two adjoining centers failed to get off the drawing board. Union contracts IT Consolidation Savings - Post Implementation

3. A New Sheriff Rides Into Town • New CIO • Vulnerability Assessment – not good • Budget woes • Accenture contract to explore possibilities IT Consolidation Savings - Post Implementation

4. The Accenture Contract • Initial contract was approx. $1.7 million – later amended to include an additional $11.6 million. • Contract was deliverables based rather than outcome based. • Accenture wrote the statement of work. • Contained provisions favoring the contractor. IT Consolidation Savings - Post Implementation

5. Things that we Believe In: • Cost savings • Enhanced Security • Disaster recovery • Better service The Pitch - Consolidation Objectives IT Consolidation Savings - Post Implementation

6. The Audit Dilemma How do you measure the success of a consolidation project when the baseline and home plate were never clearly established? IT Consolidation Savings - Post Implementation

7. The Auditor’s Job – Evaluate the Effort “The concept of accountability for use of public resources and government authority is key to our nation’s governing process.” Government Auditing Standards, July 2007 Revision IT Consolidation Savings - Post Implementation

8. Report No. 2006-33Department of Administrative Services: Computing and Networking Infrastructure Consolidation (CNIC) Risk Assessment. #1 - Audit the PlanSeptember 2006

9. The Plan • Consolidate 3 Mainframes to 1 • Consolidate Mid range • Virtualize Distributed systems • Consolidate Storage (SAN) • Establish a – Single network perimeter??? • Reduce staff from 155 – 93 • Reduce costs – Payback in 3-5 years IT Consolidation Savings - Post Implementation

10. Finding: Unrealistic Objectives & Timelines Initial project planning weaknesses led to unrealistic project expectations, objectives and timelines, causing the department to duplicate its efforts to adopt a more feasible consolidation strategy. IT Consolidation Savings - Post Implementation

11. The Moment the Plan Shifted From “Consolidate First” to “Lift and Drop” IT Consolidation Savings - Post Implementation

12. Finding: Planning Incomplete Revised project plans remained incomplete regarding how, when or to what degree consolidation of data center resources would occur or how some critical security and disaster recovery services would be provided. IT Consolidation Savings - Post Implementation

13. It was unclear what value the state received from the Accenture contract. Of $7.7 million spent (as of spring 2006), $3.4 were unjustified or unnecessary. Eg.- $465,000 for a newsletter, guest speaker list, and a status-reporting template. $945,000 for assisting an accounting and budget workgroup. Finding: Questioned Costs IT Consolidation Savings - Post Implementation

14. Finding: Inadequate QA The absence or ineffectiveness of independent quality assurance processes impacted decision makers’ view of project risk, cost, and benefit. IT Consolidation Savings - Post Implementation

15. Finding: Non-compliance with Federal Funding Requirements Accounting and compliance issues may result in loss of federal support and/or misstatements in the financial records. IT Consolidation Savings - Post Implementation

16. Report No. 2008-21 Department of Administrative Services: State Data Center Review #2 – Post Implementation July 2008 IT Consolidation Savings - Post Implementation

17. Finding: Objectives not Likely to Occur Important data center consolidation objectives have not yet been achieved. As a result, it is unlikely that the anticipated savings or operational benefits associated with the CNIC project, such as enhanced enterprise disaster recovery and security solutions, will occur. IT Consolidation Savings - Post Implementation

18. Finding: Operational Issues Unresolved Operational controls did not sufficiently address service level agreements with customers, performance and capacity management, standard operating procedures, configuration management, or software licensing requirements. IT Consolidation Savings - Post Implementation

19. Finding: Inadequate Disaster Recovery The department was ill-prepared to timely resume data center operations or assist agencies in their efforts to restore critical computer applications after a major disruption. IT Consolidation Savings - Post Implementation

20. Finding: Inadequate Security The department had not provided a secure computing environment for SDC clients. IT Consolidation Savings - Post Implementation

21. Report No. 2009-05 Department of Administrative Services: Enterprise Security Office Review #3 - Digging Deeper February 2009 IT Consolidation Savings - Post Implementation

22. Security? • Peeled off security layer, separating it from SDC • Established an Enterprise Security Office (ESO) that assumed responsibility for implementing an enterprise level Identity and Access Management system. IT Consolidation Savings - Post Implementation

23. Finding: Incomplete Security Program The ESO had not: • Developed complete security plans and associated standards, policies, and procedures. • Conducted vulnerability assessments. • Reviewed or verified the security of information systems. • Ensured remedial actions were taken to resolve identified security issues. IT Consolidation Savings - Post Implementation

24. Finding: Failed Security Projects The ESO did not appropriately manage its projects to implement centralized Identity and Access Management (I&AM) or to issue digital certificates. Epilogue: The I&AM project and digital certificate programs were scrapped. Loss totaled $14+ million. IT Consolidation Savings - Post Implementation

25. #4 – Not all Bad News! IT Consolidation Savings - Post Implementation

26. Better Service & Disaster Recovery? Report No 2010-22 State Data Center Operations Are Stable, But Some Areas Need Improvement IT Consolidation Savings - Post Implementation

27. Enhanced Security? Report No 2010-15 State Data Center: Faster Progress Needed on Security Issues IT Consolidation Savings - Post Implementation

28. Cost Savings? “The mainframe, mid-range, server, storage and network capacity requirements have grown to 244% of what was migrated to the data center, while the costs have only increased by 33%. If the operating costs of the data center increased at the same rate as capacity, the total cost would have been $270 million per biennium, instead of the current $135 million. This difference of $135 million in cost avoidance savings was obtained via consolidation, standardization and other efficiencies.” State Data Center Strategic Plan and Annual Report, July 2011 IT Consolidation Savings - Post Implementation

29. Credits A special thanks to Dan Piraro, creator of the syndicated newspaper cartoon, Bizarro, for his written permission to use his work to help liven this otherwise dull audit presentation. To enjoy more of Dan’s work go to http://bizarrocomics.com/ IT Consolidation Savings - Post Implementation