Managing the development of secure electronic banking

1. Managing the development ofsecure electronic banking MSc Thesis GUC Tom Nilsen

2. Problem • We have a security analysis for projects • The analysis should ensure compliance to our security requirements • We think that the analysis contributes to secure systems • We do not know the actual contribution • And security level varies across projects • We want to measure • The the security status of system at delivery • The contribution of the Sec. analysis to security

3. Security requirements The Bank's Security Framework has approx 200 Security requirements

5. Security Analysis Reccomended Security solution or approach Will the project use the solution? If not, please describe…..

6. The relations that leads to secure systems Security analysis Security metrics Sec. requirements Security Status at delivery Research Question 60 questions &answ 200 requirements

7. Research question • How to define a security metric that measures the security status of a new system at delivery • Security status: • Compliance to recommendations in the analysis • Compliance to the security framework and requirements

8. The MSc work • Review of previous work • Choice of methods • Defining a framework for the metric • Defining the content of the metric • Testing and improving the metric • Analysing the project work and findings

9. Review of previous work • Theory on security metrics • Security Metrics guide, 800-55 NIST • A method for security metric design, Frost/Snekkenes • Careful: "simplifying a complex security matter down to a number", McHugh/McCallam • Measuring security in practice • Risk analyses and Checklists • Security SLAs (I.e.the Banks own Security metric) • ISF metrics

10. Choice of methods • The Authors role: The risk of being to involved and proprietary: • Reference group and external reviews • Defining a framework for the metric • Studying Previous work and apply principles • Qualitative Content analysis • Security Analysis, Security Requirements • ISF metrics and ideas • To define the topics to be measured • Analysing cross referencing and structuring findings from the Content analyses • Semi structured interview of relevant managers

11. Security Metric A method for security metric design (Frost/Snekkenes) RISK MGMNT, SECURITY REQ, COMPLIENCE C I A TO SELECTED RISK OF NON-COMPLIANCE "A clear, known, agreed and understood definition…"

12. STAKEHOLDER REASON Project manager / Business developer To document security status and learn from their own performance. System owner / Business owner To know the security status when they take over the responsibility from development. Top management To manage risk and to quality assure facts before they report externally to reduce their personal liability. Management of System development To collect statistics to see trends of systematic problems and flaws in development processes that lead to security problems. Head of IT Security To assist top management in managing risk and to manage and improve the security process and deliverables (i.e. Security analyses). StakeholdersWho needs the metric?

13. "Clear line of sight"Accountability • A clear connection betweenthe stakeholders actions and the measured result • Scope of project • Relevant security requirements • Selected requirements for Systems

14. Security program maturity:Ambitions versus reality NIST 800-55 • ASSESSMENT OF THE BANK • Security program since 1994 • Policy • Baseline Sec requirement 1995 • Security Analysis 1997 • > 400 conducted • Outsourcing and Security 2001 • Security program • Security testing, code review, IDS • CONCLUSION: • between level 3 and 4 • data available at medium difficulties • => focus on AUTOMATION of Data collection and testing Maturity is is influenced by Mergers, Management +++

15. NIST 800-55 Metric detail form IMPLEMENTATION EVIDENCE Does the Audit trail provide a trace of user action ? YES ? NO ? How can we help in assessing?

16. ASPECTS AREAS SECTIONS CONTROLS Ctrl-DETAILS SM SEC MGMNT 7 32 Ca 252 Ca 745 CB CRIT BUS 6 25 121 348 CI COMP INST 6 31 Ca 250 Ca 600 NW NETWORK 5 24 Ca 193 Ca 455 SD SYST DEV 6 23 Ca 143 Ca 399 TOT 30 135 Ca 1000 Ca 2500 ISF Metrics

17. Factors that increase # of incidents ISF FIRM Special circumstances for a system or information recourse: ·Subject to a high degree of change ·Widely extended geographically ·Large in scale ·Complex ·Immature ·Accessible to external parties ·Used to support call centres

18. Security analysis Security metrics Sec. requirements Security Status at delivery Research Question 60 questions &answ 200 requirements The content of Security Analysis • COVERAGE • Authentication & Identity management • Access control & role based access mgmnt • Password process • # 30 • Access control for programs • Securing appl.data • Network & data exchange • Secure code & pentest • Audit, logs & incident mgt • Resilience & contingency • Tech. Platform & infra • System maint.& Operation • Agreements (SLA, 3.party) • RISK SUMMARY • # 80

19. Why 30 Q on Access control? • The Banks approach: • Ease of use • Reduced sign-on • Ease of administration • Single point of admin • Advantage of scale • Standardisation • Re-use secure solutionsand processes

20. Security analysis Security metrics Sec. requirements Security Status at delivery Research Question 60 questions &answ 200 requirements

21. EXTERNAL VALIDITY: INTERNATIONALSECURITY STANDARDISF SOGP AND METRICS HEALTH CHK

22. Security metric work sheet

23. Security Metric Prototype metric • COVERAGE • Authentication & Identity management • Access control & role based access mgmnt • Password process • # 50 • Access control for programs • Securing appl.data • Network & data exchange • Secure code & pentest • Audit, logs & incident mgt • Resilience & contingency • # 90 • Tech. Platform & infra • System maint.& Operation • Agreements (SLA, 3.party) • # 116 • RISK SUMMARY

24. Security Metric Challenges • Are questions asked • On the most important topics • With the right approach tothe mix of security and re-use of secure solutions • Right level of details (Health check 170 <> Survey 2500) • And in a way that we can • Collect answers and verify by testing? • Build on and compare to answers in Sec.Analysis • Use the measurement as a part of security documentation?

25. Conclusions from the Bank • Final metric versus MSc version • "minimal"MSc version in English • Proposal and description for building: • new Security Analysis • and Metric on WEB • The bank needs • A metric that focus the weak security areas • With relevant info for risk management • The bank do not need a metric => 3.7 secure

26. Security Metric Focus in the MSc report • For the sake of the reader: • We have chosen a few metric topics • Discuss them in more detail • Instead of "scratching the surface" of 115 metrics

27. Security Metric Testing and improving the metric • The first test candidate:HR and Salary system run by an external SP • Responsible roles had a need for a status • Logistics • Gather data from Sec.Analysis • Arrange meeting with responsible roles • Establish/Confirm facts • Analyse non compliance areas

28. Challenges measuring

29. Implementation evidenceNIST 800-55

30. Security Metric An evaluation of the metric • Value to the bank  • Validity: Internal , External  • Reliability: • improve by Implementation evidence • Trained ITS consultant to assist user • The workload: => automation  • Timing / project phase is important

31. Evaluation of the MSc work • The authors role • Objectivity, the risk of being too involved: • Reference group but difficult to control  • Proprietary setting - "home grown" orknowledge of general interest? • Cross ref. to an open standard – SOGP • Applied principles from science and ISF practice => "How to generalize the metric"

32. "How to generalize the metric" The business must have defined which Security requirements/standards • Select the requirements to comply with • Define stakeholders and their needs • Evaluate the security program maturity • Define a framework for the metric • Define the actual content of the metric with cross ref. to the requirements for validity • Use implementation evidence in forming questions • Test and improve the metric • Decide a process and IT-support for the metric • Remember that introducing a security metric is developing your organisation

33. Further work • Finish a Norwegian version of the metric • Update the Sec.Analysis and metric tonew version of ITS framework • Decide minimum version with final questions • Decide a process for the metric • Stakeholders support • IT-support/automation for the metric

34. Conclusion • We have developed a prototype metricaccording to the assignment • We have tested and improved it • We have described the steps for a final Norwegian version of the metric • The process and the framework for designing the metric is based on scientific principles and "industry practice" • We hope to inspire others to build security metrics

35. New generation ISF metrics • Web based in stead of Excel • XML cross reference • SOGP, 17799, Cobit • META Database • All controls from important standards

36. New generation SA and S metrics • Web based • On ISF metric • With Bank extensions • XML cross reference • SOGP, 17799, CobiT • SA and metrics cr.ref • META Database • Also BSR controls