Establishing Change Management Processes to Reduce Business Risk

1. Establishing Change Management Processes to Reduce Business Risk Dwayne Melancon, CISAVice PresidentTripwire Inc. dmelancon@tripwire.com September, 2005

2. Causal Factors of IT Downtime Operator Error 60% System Outages 20% 5% Security Related 15% Non-Security Related Application Failure 20% Source: IDC, 2004

3. Which would you rather have? • 1000 servers configured identically, but configured insecurely • 1000 servers configured randomly, but 50% configured in a secure manner

4. Best In Class Ops and Security

5. The “3 C’s” of High Performers • Culture • Tone at the top, Causality, Accountability • Controls • People+process+technology, Change Auditing • Credibility • Management by fact, Service quality

6. Where’s the leverage? Source: IT Infrastructure Library (ITIL) / BS 15000

7. Organization controls the changes: Changes control the organization: Spectrum of Capability • Continuously Improving • <5% of time spent on unplanned work • Closed-Loop Process • 15-35% of time spent on unplanned work • Using Honor System • 35-50% of time spent on unplanned work • Reactive • Over 50% of time spent on unplanned work Effectiveness Reactive Using The Honor System Closed-Loop Change Mgt ContinuouslyImproving Based on the IT Process Institute’s “Visible Ops” Framework

8. Phase 2: Catch and Release, Find Fragile Artifacts Phase 3:Establish Repeatable Build Library Phase 1: Electrify Fence, Modify First Response Phase 4: Continually improve

9. Weak IT Controls Before: Weak IT Controls Increase Risk and Cost Unplanned work 100% Unauthorized changes & accesses Change rate Time

10. Strong IT Controls After: Change Auditing Reduces Risk And Cost 100% Unplanned work Unauthorized changes & accesses Change rate Time

11. Electrify the Fence • Report that answers: • What changes map to authorized and approved work orders? • What changes do not match expected changes?

12. The Big Four says… • “All change must be auditable, and all unauthorized change must be investigated.”

13. Benefit: Improve Your Performance On Audits Auditors’ perception of assurance Control over change Time spent on audit prep and liaising % of time spent on compliance activities time

14. Before: Drifting Configurations Unplanned work Change success rate # of unique configs Mastery of each configuration time

15. After: Find Fragile Artifacts Change success rate Mastery of each configuration Unplanned work # of unique configs time

16. Phase 3: Establish Repeatable Builds Ability to patch and manage variance Unplanned work # of unique configs time

17. Release Time to provision known good build # turns to a known good build Shelf life of build % of systems that match known good build % of builds that have security sign-off # of fast-tracked builds Ratio of release engineers to sysadmins Controls # of changes authorized per week # of actual changes made per week Change success rate # of emergency changes # of service-affecting outages # of “special” changes # of “business as usual” changes Change management overhead Configuration variance Resolution MTTR, MTBF % of time spent on unplanned work Phase 4 Which Metric Do You Want To Improve?

18. Phase 2 Which would you rather have? • 1000 servers configured identically, but configured insecurely • 1000 servers configured randomly, but 50% configured in a secure manner Most high performing organizations would choose the first – why? The ability to systematically change all configurations, ability to defeat entropy, ability to maintain any desired state…

19. Biggest Mistakes That IT Executives Make • Not locking down change • Not electrifying fence • The continual desire for a technical solution • Rewarding personal heroics, instead of repeatable, predictable discipline

20. Summary • If you don’t electrify fence, you are putting the business at risk • To simultaneously improve availability, security and compliance: • Lock down change • Electrify fence

21. Web Conferencing, Video Conferencing and Online Meeting Services Problem: Solution: • Change Management circumvention was impacting service delivery • Needed the means to enforce its “zero tolerance” policy • Implemented Tripwire for Servers on ~2000 systems • Change reports used as evidence when confronting offenders Benefits: • Availability improved by a “nine” – Less than one hour of downtime a year • MTTR was reduced from 50 minutes to less than 15 minutes • Satisfied auditors’ requirements for Sarbanes-Oxley (§404, §302) and reduced the time necessary to prepare and conduct audits • Able to double IT service output with less than a 10% increase in staffing • Better service to their customers. Better control of their IT environment. Delivering Availability & Compliance

22. Resources www.tripwire.com/visops www.theiia.org/index.cfm?doc_id=5175

23. Thank You • To receive a copy of The Visible Ops Handbook, give me a business card or drop me an email at: • dmelancon@tripwire.com • For more on the study of high performing organizations, or to participate in the study: • Visit the IT Process Institute at:www.itpi.org/home/veesc.php • Check out our Visible Ops webinar series at www.tripwire.com