estimating the cost and benefits of software assurance investments l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Estimating the Cost and Benefits of Software Assurance Investments PowerPoint Presentation
Download Presentation
Estimating the Cost and Benefits of Software Assurance Investments

Loading in 2 Seconds...

play fullscreen
1 / 15

Estimating the Cost and Benefits of Software Assurance Investments - PowerPoint PPT Presentation


  • 172 Views
  • Uploaded on

Estimating the Cost and Benefits of Software Assurance Investments . Thomas P. Frazier November 9, 2006. Background. Enemies of our country may try subverting government systems by intentionally embedding malicious code or by remotely exploiting unintentionally vulnerable software.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Estimating the Cost and Benefits of Software Assurance Investments' - gada


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
estimating the cost and benefits of software assurance investments

Estimating the Cost and Benefits of Software Assurance Investments

Thomas P. Frazier

November 9, 2006

background
Background
  • Enemies of our country may try subverting government systems by intentionally embedding malicious code or by remotely exploiting unintentionally vulnerable software.
  • This risk is rising with the broadened use of commercial off-the-shelf products and the increasing globalization of software development.
  • Estimate the costs and benefits associated with specific policies and procedures to mitigate these risks.
business case
Business Case
  • Business case requires that we turn the benefits into dollar values.
  • A software assurance investment should be undertaken if the cost of this protection is less than the expected loss of not protecting
  • This is done by computing the expected value of costs and benefits
  • The expected value is defined as E = p*L, where p is the probability of a possible outcome and L is the payoff or dollar value of such an outcome.
cost benefit criterion
Cost-Benefit Criterion
  • An investment in assurance should be undertaken if

C < p*L

where:

C =the cost associated with the software assurance investment

p = probability of a successful attack by an intruder

L =dollar loss resulting from malevolent intrusion

Software assurance investment should be undertaken if the [present value of the] cost of this protection is less than the [present value of the] expected loss of not protecting

concerns
Concerns
  • Many of the policies and procedures are likely to be so general that merely enumerating all the cost and benefit elements is hard, much less assigning values to them.
  • Policies, procedures, and investment ideas will be correlated, meaning that implementing A could affect the costs and benefits associated with B.
  • Quantifying C, and especially p and L
literature search is not encouraging
Literature Search is Not Encouraging
  • Cashell, Brian, William D. Jackson, Mark Jickling, and Baird Webel, The Economic Impact of Cyber-Attacks, Congressional Research Service, Library of Congress, April 1, 2004

“No one in the field is satisfied with our present ability to measure the costs and probabilities of cyber attacks. There are no standard methodologies for cost measurement, and study of the frequency of attacks is hindered by the reluctance of organizations to make public their experiences with security breaches.”

  • Howard E. Glavin, “A Risk Modeling Methodology,” Computer Security Journal, Vol. 19, No. 3 (Summer 2003).
  • U.S. House of Representatives, Select Committee on Homeland Security, Subcommittee on Cybersecurity, Science, and Research & Development, “Cybersecurity for the Homeland,” December 2004.
proposed approach
Proposed Approach
  • Use some combination of data taken from the U.S. liability insurance industry, combined with estimates from experts to estimate p and L
  • Underwriters essentially estimate p and L every time they write a policy.
  • Employ liability insurance data from firms that write and implement software
liability insurance data
Liability Insurance Data
  • Data from the product liability industry can shed some light on the following:
    • How much liability insurance is purchased by firms of various industries and sizes,
    • How much the firms pay for the insurance, and
    • How much the firms set aside for self-insurance, etc. (i.e., exclusions and other coverage information)
an example
An Example
  • Assume that one of the recommended investments is a “red team” review of key software.
  • C - cost of performing the red team reviews, measured by the fully burdened cost of the red team members plus some estimate of the other direct charges typically incurred in these types of operations (e.g., 15 percent of the direct labor charges). COSECMO model may also prove to be useful in estimating C.
estimating risk
Estimating Risk
  • Assume that the cost of insurance (C) is the expected loss coverage (L) times the probability of loss (p)
    • That is, C = pL or p = C÷L
    • p may overstate the probability since the insurance company makes a profit
    • However, the policyholder retains some risk (deductible) which means the actual loss is usually greater than the expected insured loss in the above relationship
    • For simplicity, we will assume these effects offset each other
  • Insurance data on premiums paid for typical coverage indicate that software companies pay about $5 million in annual premiums (C) for $1 billion of coverage (L).
  • p = C÷L = $5M ÷ $1B = 0.005
  • This leads us to estimate that the probability of code failing (p) in any one year is 0.005 ($5 million/$1 billion)
estimating swa effectiveness
Estimating SwA Effectiveness
  • Gartner results indicate that removing all defects would reduce the malicious attack rate by 35%*
  • Now assume software assurance (SwA) in the form of a red team review can cut defects by 50%
  • If failure p = 0.005 without SwA, then SwA can reduce the malicious attack failure by 0.005 x .35 x .5 in this example
  • Reduction in p due to SwA is0.000875

*Gartner reports that 35% of attacks exploit unintended vulnerabilities in code

estimating loss
Estimating Loss
  • Next, we need an estimate of the loss suffered if a code failure occurs
    • Loss could be estimated by time systems are out of service
      • Ship fleet costs $23M/day (development & operation)
      • An attack might cause a 1-day loss of service ($23M loss)
    • Examples across industry show higher collective losses:
      • Love Bug (2000) $15B damage to 3.9M systems
      • Code Red (2001) $1.2B damage, $740M recovery
      • Slammer (2002) $1B damage
      • Blaster (2003) $50B damage
      • My Doom (2004) $38B damage
comparing risk and benefit
Comparing Risk and Benefit
  • Avoided annual loss probability
    • p(reduced) = 0.000875
  • Cost of loss for one day of a ship system
    • $25M (2006 dollars)
  • Annual expected loss avoided
    • 0.000875 x $25M = $22K
  • System lifetime loss avoided
    • 30 years x $22K = $660K (2006 dollars)
  • Cost of review
    • Can the SwA measure be done for less than $660K?
slide14
Nomograph of C, L, p = 0.005Assuming 30 year system life, 35% of defects lead to attacks, 50% defect reduction

Area that satisfies our cost-benefit criterion

private industry insurance data are not a perfect proxy
Private Industry Insurance Data are Not a Perfect Proxy
  • Firms that make up the sample drawn from the insurance database may not face the same threat and losses that the government faces
  • Private firms probably have different tolerance for risk than the government, and this difference influences how much coverage they buy and the price they are willing to pay
  • But, isn’t such a proxy better than merely guessing at p and L in those instances where the experts cannot provide estimates of the parameters?