Face Off: Secure Code, Myth or Reality?

Face Off: Secure Code, Myth or Reality? Gary McGraw, Ph.D., Cigital Fred Cohen, Ph.D., Burton Group Andrew Briney, Moderator

Resolved:“Writing “secure” software is a practical, achievable goal in an enterprise setting. • Introductory Remarks (15 minutes) • McGraw: YES • Cohen: NO • Moderator Question to McGraw • McGraw answer (3 minutes) • Cohen rebuttal and question (1 minute) • McGraw answer (1 minute) • Moderator Question to Cohen • Cohen answer (3 minutes) • McGraw rebuttal and question (1 minute) • Cohen response (1 minute)

Software Security is a Good Idea Gary McGraw, Ph.D. CTO, Cigital http://www.cigital.com

We have a security problem • Reactive solutions are failing • Bad software is the root cause

There is a solution • Awareness (among developers) • Books, training, clue • Resources for builders • Frameworks, languages, patterns • Touchpoints in the SDLC • Tools, processes, knowledge

Software security = good • Software is the target of attack • We must build better software to make progress in computer security • We must involve BUILDERS

Toward More Secure Enterprise Applications – Software, Systems and Surety Processes Fred Cohen, Principal Analyst, Burton Group fcohen@burtongroup.com www.burtongroup.com

Building more secure enterprise applications • Thesis • Software quality has become a critical issue • Tools, techniques, processes, metrics and training exist • But are not widely enough used • Vendors can do much better • But they won’t until you insist • But in the end, software can go only so far • Systems surety approaches are the long-term issue • Standards approaches exist and should be leveraged

Building more secure enterprise applications • Agenda • The state of software quality vs. the risks we face • How do we improve the quality of systems we build/buy • How do we deal with low surety systems being used for medium risk applications • Standards • Recommendations

The state of software quality Threat High risk Manage continuously Manage attentively Avoid this risk Systems analysis Expert Facilitated Analysis Continuous reassessment process Scenario analysis and practice Top level management Tighter controls More expensive High Covering Approaches Scenario-based analysis Medium risk Game theoretic Accept or avoid the risk Manage carefully Manage tightly Protection Posture Assessments Vulnerability Testing Looser controls Simple approaches Mid-level management Limited review process Low cost solutions sought Due Diligence Reassess threat upward Oversee Periodically Easy to accept risk Low risk Probabilistic Risk Analysis Consequence Low High

The state of software quality • Poor at best • Lots of well known vulnerability types persist • We have long known ways to eliminate them • Input overruns – curable with compilers and runtime checks • Input syntax not checked – curable with standard input tools • Race conditions – curable with existing system lock mechanisms • Unchecked returns – curable by language selection or libraries • Trusting outside sources – curable by programmer awareness • All of these and more are commercially detectable • Cost is less than $1 per line of code • Cigital, Fortify, OunceLabs, Reasoning, Secure Software, @Stake, … • Also free open source tools for all of these areas • Is it negligence or gross negligence to not check?

The state of software quality (2) • Vendors can do much better • Demand more to get more • Quality testing/certification by independent laboratories (not vendor paid)? • Certification against Common Criteria (for what properties?) • Contractual mandates and acceptance testing? • Patching like recalls: paid for by the vendor, liability for consequences? • Should enterprises sue vendors? • Merchantability and suitability for purpose are not waiveable • Licenses on software favor vendors too heavily • Is it negligent not to spend $1/line to avoid global catastrophic failures? • Should programmers/firms lose their licenses? • Too many software tickets and you can’t program any more? • Too many software failures and you won’t buy from them any more?

The state of software quality (3) • Trends and where they can get us • The trends are not looking very good • $B+ in US patch costs alone in 2003 • More and more faults despite more and more promises • Despite major efforts by major players software quality remains poor • Nobody can create high quality software at market pace • The market has to change its mind and demand quality over pace • Starting to happen – but major impacts on innovation • Major breakthroughs are needed in R&D (universities) • Far unfunded today - $10M/year give or take • The problem will remain this bad? (unsustainable) • In 10 years major software quality problems will remain Risk Surety Time

The state of software quality (4) • Stupid software notions • Testing will detect enough faults • No amount of testing will produce high quality code • Defensive programming is bad • Defensive programming means checking things carefully • It’s approved by/good enough for [place name here] • References like this sound good but get full details and check them • It’s secure, it was programmed in/by [name] • Everyone makes mistakes – process is critical • It’s certified at level C2 / EAL4 / whatever • Unless you understand these certifications and targets in detail don’t assume these are good things

The state of software quality Threat ●Software quality will only get you so far High risk Manage continuously Manage attentively Avoid this risk Systems analysis Expert Facilitated Analysis High Continuous reassessment process Scenario analysis and practice Top level management Tighter controls More expensive Covering Approaches Scenario-based analysis Medium risk Game theoretic Accept or avoid the risk Manage carefully Manage tightly Protection Posture Assessments Vulnerability Testing Looser controls Simple approaches Mid-level management Limited review process Low cost solutions sought Due Diligence Reassess threat upward Oversee Periodically Easy to accept risk Low risk Probabilistic Risk Analysis Low High Consequence

How do we improve software quality? • Improve the education of our people • NSTSSI-certified programs for managers • Graduate degree in software engineering with security specialization • Security-related education, training, experience • Funded University research programs • Things to look for in qualified people • CISSP*(ISC2*) and similar certifications for managers • Someone who has actually implemented a government approved secure system for implementers • Someone who has led a government approved secure implementation for project lead *National Security Telecommunications System Security Initiative *Certified information system security professional (International Information Systems Security Certification Consortium)

Organizational Perspectives and Business Processes Management Policy Standards Procedures Documentation Auditing Testing Technology Personnel Incidents Legal Physical Knowledge Awareness Organization How do we improve software quality? (2) • Apply systems engineering practices • A suitable design process is necessary • Proper policies, procedures, and standards • External review at all steps of the effort helps • Extensive audit of the process • Proper vetting of the people • Protection testing, not just functional testing • In-depth documentation • Skilled people in the process • Careful technology selection

Lifecycles Data People Systems Business How do we improve software quality? (3) • Capability maturity model for IT security • None – not done • Initial - few processes are defined, and success depends on individual effort talent and heroic effort • Repeatable - the necessary process discipline is in place to repeat earlier successes on projects with similar applications • Defined - the process for both management and engineering activities is documented, standardized, and integrated into an organization-wide process and used by all projects • Managed - both the process and end-products are quantitatively understood and controlled using detailed measures • Optimizing - continuous process improvement is enabled by quantitative feedback from the process and from testing innovative ideas and technologies • Measured against process elements of risk management, engineering, assurance management, and coordination for specific process objectives

Business risk management Attack and Defense Processes Accept / Transfer / Mitigate / Avoid Deter Prevent Detect Vulnerabilities Consequences Threats React Adapt Dealing with low surety components • When should I move to medium surety systems • Management has to define the risk thresholds associated with surety levels • Users need to know what risk level they are operating at so they can use the right systems • Business functions should not be supported on systems with lower surety than the risk warrants • Policy should identify and management should support the sanctions associated with misuse

Objectives Integrity Availability Confidentiality Use Control Dealing with low surety components (2) • If the SW won’t get us there, what do we use instead/addition? • Use physical safeguards • Nuclear power plant control rods held out by electromagnets • Physical separation of networks, digital diodes, etc. • Use programmable logic controllers (PLCs*) • Finite state machine designs (e.g., Johnson Controls, Harris, etc.) • Rate and level limiters (e.g., in SCADA* systems) • Failsafes against important consequence • Lockouts for high energy emitters • Braking systems when space is entered • Carefully designed redundancy is applied • N-version programming (e.g., on the space shuttle) • Timeouts and retries with redundant processors (e.g., space systems) *Programmable Logic Controllers *Supervisory Control And Data Acquisition

Context Time Location Purpose Behavior Identity Method Dealing with low surety components (3) • How to build medium out of low? • Control the context • Identify all medium/high consequences • Use medium/high surety mechanisms to failsafe them • Use medium/high surety systems for all feasible controls • Carefully limit the scope and effect of software systems • Use redundancy and sanity checks on low surety outputs • Control what goes into and out of low surety systems • Have a backup plan for when the low surety stuff fails • Eliminate low surety interdependencies • Be very careful about change controls • Keep the low surety isolated from the world • Identity management in a medium surety system

Organizational Perspectives and Business Processes Management Policy Standards Procedures Objectives Documentation Integrity Availability Confidentiality Use Control Auditing Testing Lifecycles Technology Data People Personnel Systems Business Incidents Context Time Legal Location Physical Purpose Knowledge Behavior Identity Awareness Method Organization Standards • Generally Accepted Information Security Principles (GAISP) and International Standards Organization (ISO) 17799 • Control standard for management • Deal with organizational, contextual, lifecycle, and objectives perspectives at a control level • GAISP identifies pervasive principles (PPs), broad functional principles (BFPs), mapping between PPs and BFPs, rationale, and examples • At a management level only • No technical details, just due diligence issues

Organizational Perspectives and Business Processes Management Policy Standards Procedures Documentation Auditing Testing Lifecycles Technology Data People Personnel Systems Business Incidents Legal Physical Knowledge Awareness Organization Standards (2) • Trusted Computer System Evaluation Criteria (TCSEC) • Based on measuring assurance levels and functions associated with trusted computing bases • Defined a set of specific controls and requirements on those controls for assuring data confidentiality • Levels implied surety and security functionality • D: Minimal Protection (if it doesn’t meet anything better - COTS*) • C: Discretionary Protection (some features - little assurance) • B: Mandatory Protection (many features - good assurance) • A: Verified Protection (same features as B - far better assurance) • The gold standard for operating systems • Only really addresses confidentiality • Produces hard-to-use systems *Commercial Off-the-shelf

Common Criteria (ISO 15408) Process for certifying protection profiles (PP) of systems You define the security target (ST) Authorized independent testers validate your claims Ratings given on several dimensions Common Criteria Recognition Agreement allows globalization of evaluations according to specifications Example: ST is purely a secrecy target Implement with physical separation Gain EAL7 for the ST But it may not meet your integrity needs! Evaluation assurance level (EAL) Measures how certain the evaluators are that the product meets the ST EAL1: functionally tested EAL2: structurally tested EAL3: methodologically tested and checked EAL4: methodologically designed, tested and reviewed EAL5: semiformally designed and tested EAL6: semiformally verified design and tested EAL7: formally verified design and tested Standards (3)

Standards (4) • National Security Telecommunications System Security Initiative (NSTSSI) 4011, 4012, 4013, 4014, 4015 • Training standards for people responsible for building and evaluating systems trusted for national security needs • Define requirements for educational programs that are approvable under the U.S. Centers of Excellence Program • As training standards, they are terrible – unusable – and mandatory • But they address a wide array of worthwhile issues that are key to success at building medium and high assurance systems • They also characterize a process that works well for designing and implementing trustworthy systems

Standards (5) • Other standards to consider • ISO 9000 and similar quality standards • Trusted Computing Group (TCG) Trusted Computing Platform (TCP) standard • Most security software is low surety • Only proper development and operation processes allow medium and high surety levels to be reached • Most enterprises operate medium surety systems, but most vendors do not • Most medium surety systems use standards to create medium surety results using a mix of medium and low surety components

Recommendations • Surety should be mapped against risks • Low risk -> low surety (in aggregation -> medium surety) • Medium risk -> medium surety, High risk -> High surety • Remember that risk and surety levels are continua, not discrete • Most medium surety systems use low surety components • Code validation • Enterprises and others should require validation against known classes of vulnerabilities as a condition for purchase in any • Medium or high surety application • Widely deployed software operating system, or component (aggregated risk makes it medium risk) • How does the vendor prove it? – independent evaluations • Eliminate unfair liability terms in licenses / challenge in court / require contract language

Recommendations (2) • Auditing and testing approaches • Audit approaches should be matched to surety levels according to standards of practice at each level • Black box testing will only get you so far • Wrapper-based approaches (and other containers) • Apply to all surety levels but in different ways • Low surety levels: • Very effective at reducing cost of protection • Should be applied at network, platform, application layers • Medium and high levels: More carefully considered • Trusted systems technologies • Low: TCG applies and should be looked into • Medium: TCG will soon become viable, TCSEC available today • High: TCSEC applies and should be used where feasible and appropriate

Recommendations (3) • Contractual and procurement approaches • All software procurement at medium and high should require appropriate levels of assurance in contracts • Medium: Mandatory validation against known fault types • High: Specific regulations and requirements for systems, proofs wherever possible • Standards approaches • Existing standards at medium and high levels should be followed • Knowledge and awareness requirements • CMM approach is a good one to follow for the enterprise • Low surety: CSI, SANS, MISTI, other training useful • Medium surety: Masters level courses with NSTSSI certified courses in the program • High surety: NSTSSI certified courses mandatory

Organizational Perspectives and Business Processes Business risk management Attack and Defense Processes Management Policy Standards Accept / Transfer / Mitigate / Avoid Procedures Protective Measures Objectives Deter Documentation Integrity Availability Confidentiality Use Control Auditing Prevent Testing Lifecycles Detect Technology Vulnerabilities Consequences Threats Data People Personnel Systems Business V React C T Incidents V Context V Time Legal Adapt C V Location T Physical V V Purpose V Knowledge C Behavior V T Identity Awareness Method Organization A comprehensive approach and process Recommendations (4)

Building more secure enterprise applications • Conclusions • Enterprises should insist on software quality • Widely known and readily curable fault types must be sought and eliminated as a matter of due diligence by vendors • Enterprises should insist with their pocketbooks and through legal means • Software quality will only get you so far • Proper internal systems implementation processes are necessary for medium and high surety systems • Failsafes and wrapper methods are used to integrate low surety components • Standards exist and should be followed for medium and high surety systems

Building more secure enterprise applications • References • Burton Group’s Directory and Security Strategies • Risk Management: Concepts and Frameworks • Enterprise Patch Management: Strategies, Tools, and Limitations • Windows Server 2003 Security: Making Progress, But Serious Concerns Remain • Burton Group’s Application Platform Strategies • Application Security Frameworks: Protecting Applications Consistently

Face Off: Secure Code, Myth or Reality?