The Need for Metrics and Measurement in Application Security

1. The Need for Metrics and Measurement in Application Security Jack Danahy OWASP Metrics and Measurement Standards Committee Project Lead jack.danahy@ouncelabs.com 781-290-5333

2. CSO/CISO Prexis Vulnerability Analysis Data ProgramManagers DevelopmentManagers Developers Compliance/Audit Managers The Need for Metrics • Identify critical areas of focus • Set security investment priorities • Track effectiveness of remediation and training • Monitor performance of development teams and outsourcers • Set critical priorities and security exit criteria • Publish results • Target critical remediation needs • Evaluate ROI in security training investment • Set and monitor security acceptance criteria • Identify critical vulnerabilities early • Learn how to fix the vulnerability • Confirm vulnerability elimination • Monitor compliance with established thresholds • Publish trend analyses to document security efforts/progress • Evaluate outsourcers’ compliance with contractual requirements

3. OWASP Metrics and Measurement Project Goals • Member survey and outreach to characterize significant and required metrics • Metrics gathering best practices framework • Recommendations for metrics gathering, tool analysis, metrics aggregation and weighting

4. The Case for Measurement The Need for Metrics: • Certification • Prioritization • Remediation • Tracking

5. Metrics for Certification • Governance • Credible, reliable metrics support compliance efforts by demonstrating pervasive security • Stability • Proof of security and lack of excessive patching increase customer confidence and reduce operational risk • Functionality • Validation of appropriate implementation of defined security components ensures that product meets baseline security requirements

6. Metrics for Prioritization • Determine application or project vulnerability • Determine severity of vulnerabilities • Prioritize remediation efforts LowValueHigh low exposureAudience and Exposurehigh exposure

7. Metrics for Remediation • Informed business-level decision support • Legacy applications: Wrap it, rewrite it, or replace it • Outsourced projects: Baselines and thresholds drive acceptance criteria and accountability • Resource allocation: focus investments and attention • Efficient workflow for developers • Specific identification of vulnerability • Explanation of vulnerability including potential impact • Conclusive remediation recommendations

8. Metrics for Tracking • Establish baseline and acceptable thresholds • Set accountability expectations with external vendors • Measure team performance • Provide reliable information to all areas of organization • Monitor progress over time requires: • Granularity of information • Periodicity of data (regulatory and public company requirements)

9. Sample Outsourcer Report Card

10. The Case for Measurement • Certification: Provide quantifiable measurement of security • Prioritization: Make informed resource allocation decisions • Remediation: Identify and eliminate risks caused by vulnerabilities • Tracking: Prove progress against reliable baselines and thresholds

11. Call for Participation • Active recruitment efforts underway • owasp-metrics@lists.sourceforge.net • Questions? Comments? • Contact me at: jack.danahy@ouncelabs.com

12. Thank you