The 7 Year Itch - Time To Commit Or Time To Move On?

1. The 7 Year Itch -Time To Commit Or Time To Move On? Shaun Lee Security Evaluations Manager, Global Product Security

2. Agenda • Setting the Scene • Discussions and Dilemmas • Conclusions

3. Why we perform Evaluations • Benefits to the Consumer • Evaluations are standard and independent measures of assurance • Fulfils procurement requirements • Improved product security • Benefits to Oracle • Better products • Confirms the security of the development process • Contributes to culture of security • Competitive advantage

4. The Drawbacks • Costly • Typically several hundred thousand $ to evaluate a complex product to EAL4 • Time consuming • Can be 2-3 years first time through, for EAL4 • 6 - 9 months once proficient • Traditional security analysis, e.g.: • Benign network assumptions permitted * • Hacker style penetration testing can be avoided • ‘Unrealistic’ configurations permitted * • Lots of ‘academic’ proof documents to produce * Some schemes are tightening up on these

5. Oracle and Common Criteria • Evaluating under 3 different national schemes • Each national scheme has a perceived image / reputation • Differences in policies, interpretation, oversight • Different labs for different schemes

6. Agenda • Setting the Scene • Discussions and Dilemmas • Conclusions

7. “Under attack: Common Criteria has loads of critics, but is it getting a bum rap?” (GCN, 13-Aug-2007) “Common Criteria is dead.” Colin Williams, SBL, UK (ICCC, Berlin, 2004)

8. Vendors issue updates to certified products soon after a product has achieved certification. The CC does not credit vendors for investing in and using tools & methods for identifying insecure coding errors, like static source code scanners. The length of time and the cost of evaluations Certification does not mean vulnerability free Unrealistic Protection Profiles being written and evaluated. CC not prepared to acknowledge other official international standards Certifications are for benign environments. CC being updated without vendors input or buy in. Just a paperwork exercise CC and Criticism

9. Do we still agree with this? “This is not to say that vendors should be allowed to "get away with anything". They should not. But evaluation processes should take into account differences among vendors, the need to repair flaws, and the likely impossibility of preventing them totally. They should also allow for process improvement -- a key ingredient in the quest for improved product quality that will yield better security.”

10. Countering a Criticism • The CC paradigm is not at fault, rather the national Schemes for allowing: • Benign environments • The exclusion of Buffer Overflows, SQL injections (Media driven customer concerns) • The abuse of the CC’s flexible language • Various national scheme interpretations • Possible Reason: • Trying to address the criticism on length of time and cost for an evaluation • CC flexibility makes it difficult not to allow

11. Realistic Environments • Realistic Environments require: • Realistic Protection Profile environments not just Department of Defence environments • Vendors performing more penetration testing as a result of the change to the Threat environment • Evaluators to perform more realistic penetration testing • Perception / Danger • More realistic environment will cost more money and extend the length of time for an evaluation further. • Real penetration testers better value for money.

12. Currently Pinpoint insecure coding errors Tools are not endorsed by national schemes Should reduce vendors’ evaluation costs and time, but actually slows evaluation down because the activity is additional work Future CC Scope should take into account the use of static source code scanners Schemes should force evaluators to look at these tools CC officially approved tool ‘collections’ Static source code scanners

13. Agenda • Setting the Scene • Discussions and Dilemmas • Conclusions

14. ICCC (2000) – ICCC (2007):7 Years of Progress? • Major revision of CC, now at V3.1 • CCRA now at 23 participants • Little take-up away from government arena • Continued criticism

15. 2000: Why Common Criteria? • Mutual Recognition • Evaluate once, recognised everywhere • Saves repeating work • Saves time and money • We wanted • To show our commitment to evaluations • A world first • To set a high standard for DBMS vendors

16. 2000: What was our conclusion? • Delta ITSEC to CC evaluations • Solid foundation of ITSEC and TCSEC • More work for vendor, less for evaluator • CEM introduces criteria creep • Other sources (National Scheme rules, Interpretations) introducing criteria creep • Mutual recognition makes it all worthwhile!

17. What is the position in 2007? • Criteria Creep less of a problem • But raising the bar is done through the mechanism of PPs, apparently by people with insufficient product set knowledge, and with little consultation. • We have the criticisms and other issues to concern us • CCRA • Alternative assurance schemes • Management of the CC • Mutual recognition still makes it all worthwhile – for now!

18. The Seven Year Itch – Answering the Question It’s not time to move on (abandon), but: • the stakeholders need to work harder and together to get the commitment to move on (make progress). However • failure to engage sufficiently with vendors and • raising the bar too high too quickly especially above the level of CCRA may cause vendors to experience “negative cost recovery”, which could be an issue if the market for evaluated product remains restricted

19. For More Information Oracle Security Evaluations: http://www.oracle.com/technology/deploy/security/seceval/ General Oracle Security information: http://www.oracle.com/technology/security http://www.oracle.com/solutions/security

20. Q & A

21. Do we still agree with this? IFIP-SEC ’91 Keynote: Criteria, Evaluation and the International Environment: where have we been, where are we going? Steve Lipner, Digital Equipment Corporation