1 / 16

Strategies for Evaluating Complex Software Products

Learn from experts at atsec and IBM about mastering complexity in software evaluations. Discover challenges, strategies, and the influence of complexity in this comprehensive guide.

omarkley
Download Presentation

Strategies for Evaluating Complex Software Products

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. How To Eat A Mammoth Experiences With the Evaluation of Complex Software Products Under the Common Criteria Gerald Krummeck (atsec), Bill Penny (IBM)

  2. Agenda • Our Experience • Challenges from complex systems • Evaluations under the Common Criteria • The influence of complexity • Strategies in mastering complexity • Summary

  3. atsec‘s Experience • Evaluation Labs in Germany, USA, Sweden • More than half of all OS evaluations performed world-wide • z/OS (IBM Mainframes) • z/VM (IBM Mainframes) • Linux (SuSE, Red Hat, Oracle) • AIX • Cray • PR/SM, AIX LPAR • Databases • IBM DB2 • Oracle DB • Tivoli System Management Products

  4. IBM‘s experience • ISO 9001 Certified since 1993 • WW development organization • US, Canada, Germany, Australia, US • Mexico, Russia, China • Historically Independent • Long History of IT Management • Project Management • System Management • Process Control • Large Complex Systems • HW, SW • New Function and Service Models • Support Largest WW Business Requirements • High availability, security, integrity

  5. Challenges from complex systems Dimensions of complexity in evaluations • Size of the product • Size of the TOE (what part will be evaluated) • Amount of security functions • Protection Profiles • Depth of evaluation (EAL) • Global distribution of development • Multi-national • Large number of organisational units

  6. Evaluation under Common Criteria Security Target Design SecurityPolicyModel FunctionalSpecification High-Level Design Low-Level Design Implemen- tation Product Correspondence Guidance documentation Tests Vulnerability Analysis Development Process (Life Cycle) Processes Delivery and Operation Configuration Management

  7. Example: IBM z/OS Version 1Release 8 • Size • Several Millions LOC (Assembler, PL/X, C, Java) • Over 30 years development history • Over 300 Manuals (120.000 pages) • Over 630 Claims on security functions in the ST • 10 development sites distributed globally • 10 CM systems • Common Corporate Standards and Processes • Toute la Gaule est occupée… Toute?

  8. Interim Result • You cannot look at everything • But you don‘t need to • Security functions can be located quite accurately and can be tested thoroughly • Requires sufficient experience and product know-how of the evaluators • Development processes become very important • Build trust in the developer to comply with his duties for every piece that has not been scrutinized by the evaluators • Again: Evaluators need experience and product know-how: • It is an illusion to assume that everybody can perform a good evaluation just by applying the CC methodology (not everybody can eat the mammoth without choking on it) • Customers need to identify the right laboratory for them with evaluators skilled in their type of product

  9. Strategies to master complexity • Not everything at once • How to eat the mammoth • Assistance • Site Certification

  10. Not everything at once • Start modest • Focus on core functionality • Start with lower assurance level (EAL2 or EAL3) • Pro: Get your first certificate in due time • Con: lower assurance level than competition • Example Linux: • Start with EAL2, restrictive configuration • Now EAL4, CAPP/LSPP, almost all packages included • In between: write low-level design, add audit functions

  11. Example z/OS • MVS: Orange Book B1 (in the mist of times…) • V1R6 – 2005 • EAL3, CAPP+LSPP (multilevel security) • Core functions: RACF, BCP, JES2, CS390, … • V1R7 – 2006 • EAL4 • Additional security functions • V1R8 – 2007 • Major expansion of security functionality • V1R9 • …

  12. How to eat a Mammoth? • Bite by bite, of course! • Don‘t become intimidated by the size • Don‘t try to swallow it in one piece, either • Important factors: • Experience • Confidence • Perseverance

  13. Assistance • 2 Teams from evaluation lab • Evaluators • Working on-site with developers is beneficial • Additional testers with product know-how • Consultants • Help developer to gather evidence,prepare required documents • Do not influence product itself or developer‘s decisions • Experienced certifiers help, too

  14. Developer committment • Multi-year committment • Strong project management to coordinate all participating organizations • Strong technical leadership • „Divide and Conquer“ • Strong leaders at distributed locations • Educate, track, report • Focus by area (ST, CM,HLD, Test) • Communicate with Evaluation Team • Open, early and frequent discussions

  15. Conclusion • Evaluation of complex products fits well in CC scheme • Medium to long term strategy (and committment!) • Start modest • Increase assurance level and functionality • Processes must fit • Find the right partner with experience and product know-how • ITSEF and certification body

  16. Questions, Comments

More Related