1 / 44

Activity-Based Security Metrics for Effective Measures

Learn practical methods to define, collect, and analyze security metrics, uncovering key insights for improved processes. Gain tools for assessing IT and security budgets, user accounts, systems, and applications. Explore transaction measurements and impactful uses of security data.

strouth
Download Presentation

Activity-Based Security Metrics for Effective Measures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Security Measures & Metrics Pete Lindstrom, CISSP Research Director Spire Security, LLC www.spiresecurity.com petelind@spiresecurity.com

  2. Security Metrics II Security Metrics (Part 2): Activity-Based Security Part 2 of our mini-workshop on security metrics puts the "Four Disciplines" metrics framework into play in the real world. Pete Lindstrom discusses the hurdles that must be overcome in order to get a program off the ground. Pete highlights the data and knowledge gained from this process and describes the best ways to effectively begin your security metrics initiative. Note: this is a participatory session -- all attendees electing to attend this breakout receive a metrics worksheet to be completed prior to the session, which will be the best way to tailor what you learn to fit your own specific requirements. Learn to: • Define a process for collecting the information metrics • Differentiate the relative merits and drawbacks for data collection and analysis • Identify key insights into the metrics themselves and their surrounding processes

  3. Security Metrics People: Departments Admins Time: Hr/Day Month/Yr Costs: Salaries, Consulting HW, SW, Maint. Resources: User accts, systems, apps

  4. Define the Universe • Collect “Universe” Info • Enterprise Information • IT Organization Information • System Information • Be consistent across the board and continue to be consistent throughout. • This is one place where you can limit the scope of a project.

  5. Enterprise Information • Market Value • Total Revenue (non-profits: Funding Level) • Total Expenses • Number of Employees • Number of Geographic Locations

  6. IT Organization Information • Total Capital Budget / Expense • Total O&M Budget / Expense • Total Salary/Consulting Budget / Expense • Number of IT Employees (incl. contractors) • Security Budget

  7. Survey Results I

  8. System Information • Define the “trusted” network environment • Number of desktops/laptops • Number of servers • By Operating System • Number of applications • Inhouse/packaged • All other components • Databases, network components, appliances

  9. Survey Results II

  10. Gather Security Information • People • Time • Costs • Transactions

  11. Person Information • Identify security FTEs. • Two employees that spend half their time on security equal one FTE. • Security is a collaborative effort, so expect lots of partial FTEs. • Operations • System/Network Admins • Developers • Customer Support

  12. Calculate FTEs

  13. Time Information • Annualize everything • Person information plus consultant time • One FTE = 2000 hours • Allocate security time to Four Disciplines. • By % of time • By hours

  14. Allocated Time by % or Hours

  15. Cost Measurements • Identify salaries (take time information above and apply a dollar value) • Identify capital expenses (H/W; S/W; Consult; Service) • Identify maintenance expenses (Consult; Maint Fee; Service)

  16. Allocated Time = People Costs

  17. Allocated Product Costs * appliances PRODUCTS: • Provision • Pwd Mgt • Authent. • Web Acc Control • SSO • FW / NIPS • Vuln Scan • Patch/Remediation • Shields • SRP • Net Monitor • IDS • SEM • Forensics • Policy Mgt • PKI • VPNs • Crypto • DRM/TOS

  18. Transaction Measurements • Identity Management • Accounts created • Accounts disabled • Passwords changed • Vulnerability Management • Vulnerabilities identified • Vulnerabilities patched

  19. Transactions per Hour

  20. Uses of Security Metrics • Process Effectiveness • Six Sigma • Staff Productivity • ROI / promotions • Cycle Time • Balanced Scorecard • Staff Efficiency • ROI • Cost Effectiveness • Activity-based costing • ROI/TCO

  21. Uses of Security Metrics • Trending – are you getting more or less efficient? • Benchmarking – are you doing better/worse than peers? • Forecasting – how many resources do I need for next year? • Decisionmaking – should I build or buy a solution?

  22. Survey Results III – Security Budget Compared to IT Budget Correlation: 0.99073 Avg Sec/IT Budget: 6%

  23. Survey Results III – Security Budget Compared to Devices Correlation: 0.9366 Avg $ per Device: $433

  24. Survey Results III – Security Budget Compared to Employees Correlation: 0.91177 Avg $ per Employee: $413

  25. Q1: Best Budget Predictor? Which metric do you think should be the best predictor for security budget? • IT Budget • Number of Devices • Number of Employees • Other

  26. Q2: Best Explanation for Variance? What do you think would best explain the huge variance in numbers? • Level of risk tolerance • Costs too distributed to capture • Bad survey data – inconsistencies in answers • Some companies are good, some aren’t

  27. Some Examples • Activity-based Costing • Cost Benefit – ROI • Incident Costs

  28. Example – Cost to create user acct • ID Mgt: User Provisioning • Cost: Salaries – a five-person group of administrators create 2500 accounts annually. Average salary is $50k. • Cost: Product (hw, sw) – a user provisioning solution costs $100k total. (amortize over 5 years). • Cost: Maintenance - 20% (pay in year 1). Salaries 2500/5 = 500 accts/user $50,000/500 = $100/acct Product Costs $100,000/5 = $20k yr $20,000/2500 = $8/acct Maintenance $100,000/5 = $20k/yr $20,000/2500 = $8/acct $116 per user account created

  29. What Is It Good For? • $116 per new user per year. • Allocate costs throughout environment. • Plan budget for new applications. • Measure/compare for cost effectiveness.

  30. Survey Results IV – User Info User Accounts per FTE Correlation: 0.14474 Avg Accts per FTE: 4392

  31. Survey Results IV – User Info User Events per FTE Correlation: 0.051865 Avg Events per FTE: 351

  32. Survey Results IV – User Info User Repositories per FTE Correlation: 0.490393 Avg Rep per FTE: 182

  33. Q3: Best User FTE Predictor? Which metric do you think should be the best predictor for user admin FTEs? • Number of User Accounts • Number of Events (adds/changes/deletes) • Number of Repositories • Other

  34. Q4: Best Explanation for Variance? What do you think would best explain the huge variance in numbers? • Level of risk tolerance • FTEs too distributed to capture • Bad survey data – inconsistencies in answers • Some companies are good, some aren’t

  35. Example 2: Cost/benefit for Patching • 2,000 Systems • $70/hr IT support • 1 hour to patch / 2 hours to recover • 10% likelihood of patch failure • 20% likelihood of compromise (pre-exploit)

  36. Example 2: Cost/benefit for Patching • Pre-exploit, manual patching • Cost to Patch: • 2,000 x 70 = $140,000 • Fail: 10% x 2,000 x 70 = $14,000 • Total cost: $154,000 • Cost not to Patch: • 2,000 x 140 x 20% = $56,000 • Decision: Don’t Patch

  37. Example 2: Cost/benefit for Patching • Post-exploit, manual patching • Increases risk of compromise to 80% • Cost to Patch: • 2,000 x 70 = $140,000 • Fail: 10% x 2,000 x 70 = $14,000 • Total cost: $154,000 • Cost not to Patch: • 2,000 x 140 x 80% = $224,000 • Decision: Patch

  38. Example 2: Cost/benefit for Patching • Pre-exploit, automated patching • Assume 1 patch per month • Cost to Patch: • Software Costs = $48,000 • 1/12 of $48k = $4,000 • Fail: 10% x 2,000 x 70 = $14,000 • Total cost: $18,000 • Cost not to Patch: • 2,000 x 140 x 20% = $56,000 • Decision: Patch

  39. Example 2 – Patching ROI • Compare two patch scenarios: • Manual process: $154,000 • Automated process: $18,000 • ROI: $136,000

  40. Example 3: Cost of an Incident • Loss of value (inherent to the resource) • User Productivity • Stored Asset Value • Intellectual Property Value • Revenue Generation Value • Costs (associated w/ incident) • IT Productivity • Regulatory Fines • Opportunity Costs

  41. Calculate User Productivity s = $2 billion • Identify organization’s annual salary expense (s) from financial statements. • Divide by number of employees (e) = avg salary (a). • Divide avg salary by 2000 = avg hourly rate (h). • Estimate % of employee base that are computer users (u). • Estimate % of time that employees use computers (t). • Estimate length of downtime (d). • Productivity Loss = s / e / h * u * t * d e = 40,000 employees a = $2b/40k = $50,000 h = $50k/2k = $25/hr u = 60% empl. users t = 75% (heavy users) d = 2 hours 40k * .6 * $25 * .75 * 2 = $900,000

  42. Notes on Productivity • Steps 4 and 5 estimates could be replaced with avg number of simultaneous sessions and avg length of session. • Assumes you won’t spend more than something is worth. • The calculated number is a conservative one. • Does not account for indirect revenue generated (only includes person salary costs) • IT Productivity is calculated in a similar way but ends up being an extra allocated cost. • Can include opportunity cost as well.

  43. Conclusions • Metrics are useful in a number of different situations. • Numbers/statistics are suspect without corresponding analysis and sensibility test. • Numbers will get better as the profession matures.

  44. Agree? Disagree? Pete Lindstrom petelind@spiresecurity.com www.spiresecurity.com

More Related