Vulnerability in the real world lessons from both sides of the fence
1 / 30

Vulnerability in the Real World Lessons from both sides of the fence - PowerPoint PPT Presentation

  • Uploaded on

Vulnerability in the Real World Lessons from both sides of the fence. Ryan Permeh Manager of Product Security McAfee Security Architecture Group Who I am Why should I listen to this guy?. Several years on the Research side

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Vulnerability in the Real World Lessons from both sides of the fence' - louie

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Vulnerability in the real world lessons from both sides of the fence

Vulnerability in the Real WorldLessons from both sides of the fence

Ryan Permeh

Manager of Product Security

McAfee Security Architecture Group

Who i am why should i listen to this guy
Who I am Why should I listen to this guy?

  • Several years on the Research side

  • Years as a reverse engineer and exploit developer

  • Front lines experience as a vendor and developer

  • Currently manage a large security vendor’s product security.

  • 29 shipping projects with millions of lines of code

Bugs bugs and more bugs an accelerating trend
Bugs, Bugs, and more Bugs An accelerating trend

  • Rates of found vulnerabilities increasing

  • Severity of bugs increasing

  • Why?

    • Research isn’t a black art

    • Attacker tools improving

    • More software

    • Do reasons matter that much?

Graph from IBM/ISS X-force Threat analysis 2007

Examining the players what motivates researchers
Examining the Players What motivates Researchers?

  • Common Good (For the Internet)

  • Fame

  • Money

  • Demonstrate Technical Excellence

  • Responsibility

  • Very Bad Things

Examining the players what motivates vendors
Examining the Players What motivates Vendors?

  • Common Good (for our customers)

  • PR and Brand Image

  • Money

  • Demonstrate Technical Excellence

  • Responsibility

Today s reality it s not your daddy s vulnerability market
Today’s Reality It’s not your Daddy’s Vulnerability Market

  • An emerging economic market for bugs

  • Fame is less likely

  • Vendors that make things hard for researchers may find that researchers prefer the market for their bugs

  • Even efficient vendors may not get reports first

  • Vendors need to “fill the gap”

Lack of reports != Lack of bugs

Researcher relations seek the common ground
Researcher Relations Seek the common ground

  • Money from vendors is not likely

    • Extortion?

  • Fame from vendors is more likely

    • Credit in advisories

    • Acknowledgment in any pr items

  • Hire researchers with good track records

    • Many large software companies hire researchers

  • Avoid antagonism on either side

    • Disclosure policies and communication

Growing a secure organization why secure software is good business
Growing a Secure Organization Why secure software is good business

  • Cost of post release vulnerability is oppressive

  • Implementation in the SDLC is paid for itself by only a few serious bugs

  • Damage to brand, to customers, and lawsuits

    An ounce of prevention is worth

    a pound of cure.

    Benjamin Franklin

~35x more expensive to fix a bug

post release than in design





Selling security in your organization demonstrate a need then fill the need
Selling Security in Your Organization Demonstrate a need, then fill the Need

Building Security into your process is a long process

  • Starts Slow

    • Keep scope realistic for your resources

  • Demonstrate Risk

    • Initial reports

    • Focus on growth, not fear

  • Increase Scope

  • [Daily,Weekly,Monthly,Quarterly] wins add up

  • Keep it Consistent and meet commitments

Calculating roi for security there aint no such thing as a free lunch
Calculating ROI for Security There ‘Aint No Such Thing as a Free Lunch

  • Every activity has a cost

    • Human Capital

    • Technical Capital

    • Cost of Opportunity

  • Focus on high ROI activities

  • Plan for the short term and the long term

    • Some activities have short term ROI, others take longer

  • To justify increases, you must have the data

  • Clear improvements justify costs and allow for additional resources

Tracking security metrics what to measure and why



Track over Time

Security Report Card

Number of external security bugs

Number of internal security bugs

Bugs per KLOC

Fuzzing Coverage %

Automated tools coverage %

Input Validation Coverage %

Cost per Sev 1 (internal and external)

Cost per Developer Trained

Cost per Penetration test

Cost per …

Tracking Security Metrics What to measure and why

How do researchers find bugs code audits

Hand Auditing

Relatively low initial cost

Relatively high ongoing costs

Poor scaling

Variable output levels

Deep analysis possible

Relatively poor ROI

Dependant on quality of


Static Code Analysis

Relatively high initial cost

Relatively low ongoing costs

Good scaling

Consistent output levels

Deep analysis usually not possible

Reasonable ROI

Dependant on quality of


How do researchers find bugs? Code Audits

How do researchers find bugs reverse engineering
How do researchers find bugs? Reverse Engineering

  • Useful bug finding skill

  • Usually not necessary if you have code

  • Good for analysis of 3rd party libraries (check the license)

  • Can find very deep bugs

  • Specialized, expensive skill to keep on staff

  • Audits can take longer

  • Tools are getting better, but still require skills

  • Pretty low ROI

How do researchers find bugs fuzzing
How do researchers find bugs? Fuzzing

  • Large scale fault injection

  • Use a public framework or write your own

  • Great for covering

    • File formats

    • Network servers

    • Web input testing

  • Find bugs while you sleep

  • Not very precise

  • Variable initial cost

  • Low ongoing cost

  • Good ROI

How do researchers find bugs threat models
How do researchers find bugs? Threat Models

  • Structural /Architectural Analysis with threat enumeration

  • Pioneered by Microsoft

  • Discovers architectural flaws and missing pieces

  • Can be very formal or less so

  • Scope can be focused or “big picture”

  • Utilize your architects, developers and QA

  • Very low initital and ongoing costs

  • Very good ROI with quick realization

    • Some items found may require large effort to fix

How vendors fix and reduce bugs penetration testing blackbox
How Vendors Fix and Reduce Bugs Penetration Testing / Blackbox

  • Use standard security testing practice

  • Can be easily integrated into QA cycle

  • Requires some specialized knowledge

  • Relatively mature tools

  • Dependant on quality of testers

  • Good candidate for outsourcing

  • Variable initial costs

  • Relatively low ongoing costs (unless outsourced)

  • Fair ROI with good short term gains

  • Pairs very nicely with a threat model

How vendors fix and reduce bugs training
How Vendors Fix and Reduce Bugs Training

  • Training developers in Secure coding techniques

  • Training QA engineers in security testing techniques

  • Requires time to train

  • Requires repetition to lock in skills

  • Can be sped up with intensives

    • Microsoft’s “stop and train”

  • Ultimately it does good in reducing bugs, but its slow

  • ROI is pretty poor, it takes a long time to recoup

    • Requires trainers and training materials

    • Employee turnover affects costs

    • Screening new employees for security experience helps

How vendors fix and reduce bugs integration into the sdlc
How Vendors Fix and Reduce Bugs Integration into the SDLC

  • Strong integration of security at all points of the SDLC

  • Security Requirements

  • Threat modeling and Secure Design Principles

  • Secure Coding Guidelines and Automated analysis

  • Security Test plans, fuzzing, penetration testing

  • Vulnerability Management

  • The end goal, an accumulation of all else

  • ROI dependant on implementation, gains are cumulative and long term

Disclosure policies full disclosure
Disclosure Policies Full Disclosure

  • Instant information

  • Helps attackers and bleeding edge defenders

  • Theoretically trades instant pain for a potentially quicker remediation cycle

  • Can work with vendors or can be 0day

  • Gives all involved the chance to understand the attack on a deeper level, allowing them to better understand their risks

Disclosure policies non disclosure
Disclosure Policies Non Disclosure

  • No public information, ever

  • May be held by vendor or researcher (or bad guy)

  • Silent fixes (if any), no notification

  • Subject to patch diffing

  • Anti-Sec

  • Legal battles

  • Heavy regulation

Disclosure policies the middle ground
Disclosure Policies The Middle Ground

“Responsible” Disclosure

  • Covers all ground between full and non disclosure

  • Not incompatible with full or non disclosure

  • Dependant on defined policy

  • Researcher waits until patch

    • May release full technical details post patch

    • May not release anything

  • Middle ground that people can work with

Disclosure policies some examples
Disclosure Policies Some Examples

OIS – Organization for Internet Safety

  • Coalition of Security and Software Companies

  • Complete end to end disclosure policy

  • Fully documented

  • McAfee and many other vendors follow this model


  • Full disclosure policy


  • Steve Christey and Chris Wysopal

  • Draft, not RFC

    NIAC – DHS

  • Large scale study

Disclosure response what to do when a bug is found
Disclosure Response What to do when a bug is found

  • Have a coordinated security policy in place

  • Make it easy for researchers to contact you


    • Consider using PGP or GPG for communications

    • Have a public web site with contact details and PGP public key

    • Consider posting your policy

      • Paypal

  • Respond as soon as possible

    • Acknowledge receipt of issue immediately

    • Determine Risk

    • Treat Security bugs as important

Communication with researchers
Communication with Researchers

  • Communication with researchers is important

    • Recommend at least weekly updates

    • Updates should include any status or movement in the fix

  • Keep honesty and integrity as priorities

  • Use this Coordinate release schedule

    Always Remember, researchers are helping you, even if it’s for their own reasons. Communication with researchers keeps them happier and less likely to engage in hostilities.

Releasing details coordination and release process
Releasing Details Coordination and Release Process

  • Credit helpful researchers

  • Try hard not to slip on schedules

  • Coordinate release times

    • Realize time zones may cause difficulties

  • Be prepared for questions

    • Customers

    • Internal resources

    • Media

Baby steps next steps and beyond
Baby StepsNext steps and beyond

Building bridges engaging the research community
Building Bridges Engaging the Research Community

  • Keep building bridges

  • Quality external researchers are good for the software development process

  • Continue to learn from the research community

    • Security Conferences

    • Trainings

    • Microsoft’s Bluehat

  • Stay vigilant of trends and changes



Ryan Permeh