No Victims: How to Measure & Communicate Risk

1. No Victims: How to Measure & Communicate Risk Jared Pfost jared@thirddefense.com thirddefense.wordpress.com @JaredPfost

2. Hello • InfoSec 17 years • Consulting • Practitioner • Microsoft • Washington Mutual • Software Development • Microsoft • Startups • Third Defense • Process Nut

3. Human Motivation Straight Forward Tasks Ambiguous Tasks Autonomy Purpose Mastery

4. Just My Opinion Risk Security $pending 2012+ 90’s 2008 2002 2005

5. Q: What does success look like? • Avoid unacceptable risks in the most efficientmanner? • Just good enough to meet a standard of “due care?” • Be compliant 2 months per year?

6. Infosec Evolution

7. Nothing Miraculous Here!

8. Seeking Acceptance Treatment Decisions Control Performance Prioritize Risk Scope Measurements Mitigation Cost-Benefit Define Target Values Manage Risk Register Optimize Targets

9. Risk Prioritization: Kiss The Ring Of Process

10. The Exercise Is Important

11. Evidence

12. Evidence Drives Treatment • Don’t prioritize risk without it... You find it It finds you

13. Risk Narrative • Grabber • Agent • Action: CIA • Asset • Impact • Details • Vulns • Controls • Occurrence • Evidence Criminals copying payment card data through Internet facing web app. We have 50K records, business owner and IT expect X direct and Y indirect costs. Development practices failed to validate malicious input leading to... We found 3 vulns per assessment. Peers lost 100K records last year.

14. Use Culture to Select Model • Evidence In -> Treatment Decision Out Expert Opinion Distributions For ARO and SLE User Defined Ordinal Values And/Or http://beechplane.wordpress.com/2011/08/17/the-simple-power-of-openpert-ale-2-0/

15. Minimize Ordinal Flaws • Non-linear scales • Reserve Highest Values to reference risk details • Edge Cases • Document edges or create a new risk • Understand previous Treatment Decisions against “Color Bands” • Combine quantitative and qualitative values • Include risk narrative elements • Align to other department models e.g. ERM

16. Narrative Scale Definition Impact Frequency

17. 10 Segregation of Duties 9 8 Access Certification 7 ECom. Device Vulns Incident Response 6 Impact ECom: App. Vulns 5 4 Vendor Security Controls DDoS 3 2 2 3 4 5 6 7 8 9 10 Likelihood Accept Evaluate Act Single Event Risks Criminals copying payment card data through Internet facing web app. We have 50K records, business owner and IT expect X direct and Y indirect costs. Details... Evidence...

18. Tell Me A Story • Vulnerability Attributes Evidence: We found 3 injection vulns per assessment. Vulns are easily identifiedand exploitable from the Internet. Only basic knowledge and a motivated Agent are needed. Peer Company was breached last month by a Criminal Group.

19. Tell Me A Story (cont.) • Control Effectiveness Evidence: Development practices failed to validate malicious input. Training is mandatory but ineffective. Quarterly Assessments occur but site updates occur monthly.

20. Tell Me A Story (cont.) • Impact Evidence: Last year’s breach estimated at $xx,xxx direct and $xxx,xxx indirect costs. Peer Companies breach estimated at $xxx,xxx. However minimal customer departures.

21. Multiple Hop Risks • Advanced Adversary copying intellectual property through “Aurora” style attack Test Test Test Test

22. 10 Adv. Adversary: IP Theft 9 Privilege Escalation 8 Social Engineering: Employee 7 6 Impact 5 Device Compromise 4 Data Exfiltration 3 2 2 3 4 5 6 7 8 9 10 Likelihood Accept Evaluate Act Multi-Hop • Keep it simple • Add a “roll-up” risk to represent chain of events

23. Don’t Forget The Agents Advanced Adversary For IP Full Packet Capture Rock Star Response & Forensics Advanced SDL Criminals For Cash Fraud Detection Basic SDL AAA DoS Chaotic Actors For LOLz Vuln Scans Controls: Spending & Process Maturity

24. Spend Or Accept • Prioritize by “Business Value” Construct • Risk(s) Priority • Team Capability • Business Support • Political Reality • Cost Efficiency Gain Save $110K

25. Spending: No Room For Victims • Risk-Based Decisions, Budgets • Internal Consulting • Process Improvement Discretionary Discretionary “Legally Defensible” Security Mandatory

26. 10 Break Glass Access Unencrypted Tapes Application Vulns 9 Employee Terminations Access Certification Segregation of Duties Paper Statements 8 Network Segmentation SaaS Security Transparency Device Patching 7 Unencrypted PII in Email Vendor Security Controls Incident Response 6 Impact Proliferation of PII Rogue Wireless Access SaaS Storage 5 Rogue Devices Log Retention 4 3 DDoS 2 2 3 4 5 6 7 8 9 10 Likelihood Active Mitigated Other Watching Accepted Risk Register - Skeletons • Authoritative Source • Defined Process • Treatment Status • Mitigating • Mitigated • Accepted • “Watching”

27. Measure Evolution

28. Real Metrics Have Outcomes • Metrics have Winners|Losers • Measure actual performance against target • Benefits • Drives “acceptable risk” conversation with Management • Simplifies reporting e.g. are we above|below?

29. Start With “Easy” • Incidents • # of High, Moderate, Annoying • Application • # of Post-production security bugs • Scanned Vulnerabilities • # Patch & config vulns not mitigated per policy timeframe • e.g. Critical, Ecommerce Vulns mitigated within 30 days

30. Age Distribution (Overall) Overdue Vulns 450 350 Critical Critical 400 300 350 Severity 4 Severity 4 250 300 200 250 Severity 3 Severity 3 Vuln Count Vuln Count 200 150 Severity 2 Severity 2 150 100 100 Severity 1 Severity 1 50 50 0 0 > 90 Workstation 90 60 Servers 30 30 60 90 > 90 ECommerce Days Overdue Days Until Due

31. Expand Measurement • Access Management • % Employee termination within policy • % Role/Access verification • Network • % critical systems monitored • Moving to % of full packet capture • Vendors • % assessed per policy • # overdue findings • Employee • # of duplicate incidents • Change Management • # emergency or unplanned changes • % of changes with a regression Every Metric Must Have A Target

32. Server Patching 100 92 Percent 84 75 67 Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Optimize Cost - Target • Is target optimal? Current Target Proposed Target

33. Find Leading Indicators Integrate Metrics Into Root Cause Analysis

34. Zen • Process • Evidence • Communication • Measurement Accept

35. jared@thirddefense.com thirddefense.wordpress.com @JaredPfost Feedback Survey! https://www.surveymonkey.com/sourceboston12 Questions?

36. Appendix

37. 10 9 8 Post Worm 7 6 DoS Post Malware Post 5 4 3 2 1 1 2 3 4 5 6 7 8 9 10 Cost - Benefit - Accountability Evidence: Incidents, response performance, attack attempts Current Target Proposed Target Or http://code.google.com/p/openpert/

38. Embrace Maturity Deltas • Target Maturity used in Spending Decisions • Hire a Benchmarking Service

39. IT Risk Assessment Deliverables

40. RACI in action R – Responsible A – Accountable C – Contribute I - Informed (There can be only one “A”)

41. Are You Ready For The Answer? Motivating Event