slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making PowerPoint Presentation
Download Presentation
Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making

Loading in 2 Seconds...

play fullscreen
1 / 30

Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making - PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on

Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making. Russell Cameron Thomas Principal, Meritology russell.thomas@meritology.com Mini-Metricon, February 5, 2007 San Francisco, CA. To introduce a new approach

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Total Cost of Cyber (In)security – Integrating operational security metrics into business decision-making' - sheri


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Total Cost of Cyber (In)security –Integrating operational security metrics into business decision-making

Russell Cameron Thomas

Principal, Meritology

russell.thomas@meritology.com

Mini-Metricon, February 5, 2007

San Francisco, CA

purpose of this talk
To introduce a new approach

Influence thought leaders, academic research, and professional practice

Stimulate your thinking and inspire hope

Build productive bridges between business and IT

Show how key concepts of each can be made compatible

Take a stand on what will work and what won’t

To get your feedback

Is this on the right rack? Is it worth pursuing?

Does it fit with other approaches to security metrics?

To recruit collaborators and advocates

Non-purposes

Debate the devilish details

Debate politics

Debate acceptability in “Mainstream” and “Late Adopter” organizations

It will take years, of course!

Purpose of this Talk

Mini-Metricon, San Francisco - Feb 5, 2007

the challenge
The Challenge
  • Problem: Disconnect between business decision-makers and security specialists regarding value and risk of InfoSec*
    • “Security directors appear to be politically isolated within their companies”
    • “They face a challenging search for allies when they need to gain support from upper management for new security initiatives.”
    • “Companies reported less alignment of security with long-range strategic objectives of the firm.”
    • “The results suggest that security remains a function that is mired in operations in the eyes of senior executives.”
  • Result: under-spending, over-spending, misallocation, burden-dumping, denial, and worse…
    • Fighting the last war
    • Failures of imagination
    • Unintended consequences

* Conference Board Survey Oct. 2006: “Navigating Risk—The Business Case for Security”

Mini-Metricon, San Francisco - Feb 5, 2007

the simplistic approach is a blind alley rosi ale and variants
The Simplistic Approach is a “Blind Alley”ROSI*, ALE**, and variants

Mini-Metricon, San Francisco - Feb 5, 2007

two viewpoints on economic risk
Two Viewpoints on Economic Risk

#1 “Rational Investor”

(Capital Asset Pricing,

Discounted Cash Flow)

  • What matters:
  • D Mean, D variance
  • Fat part of the curve

p(v)

random walk

value

  • When:
  • Quarterly EPS
  • Earnings volatility
  • Shorter time periods

Normal

distributions

time

change in value

#2 “Insurance Actuary”

(Ruin Theory,

“Iceberg Risk”)

  • What matters:
  • Extreme events
  • Tail of the curve

p(v)

random walk with

“avalanches”

value

  • When:
  • Credit rating
  • Solvency
  • Reserve funds
  • Longer time periods

“Fat Tailed”

and skewed

distributions

time

change in value

99%

“Ruin”

Mini-Metricon, San Francisco - Feb 5, 2007

the core idea three costs categories
The Core Idea: Three Costs Categories

Idealized

“Catastrophic”

“Self-insurance”

“Budgeted”

mean

1s

2s

3s

4s

5s

6s

7s

Annual Probability

1,000x

1x

10x

100x

Total Cost of InfoSec

(borrowed from “Value at Risk” concept in Financial Services Risk Management)

Mini-Metricon, San Francisco - Feb 5, 2007

budgeted costs
Budgeted Costs
  • Q: What is the expected (average) impact of security-related costs on EPS and earnings volatility (+/– budget)?
  • The rule: costs must already be in the budget* somewhere
    • Defined to fit the budget and spending approval processes
    • Results in stable ratio-scale values
    • Theoretically and practically sound
      • Applies Activity-based Costing methods
      • Compatible with accounting practice (GAAP)
      • Fits discounted cash flow assumptions for multi-year analysis
    • Good information available (in principle)
    • Simple Arithmetic ® Tractable and simple to understand
    • Composable across organization units and systems
  • “If you are claiming cost reductions, show me whose budget I should cut. If you are claiming revenue increases, show me whose sales quota I should raise.” (Exec VP)

* Includes both operating and capital budgets, but excludes cyber insurance or reserves

Mini-Metricon, San Francisco - Feb 5, 2007

calculating budgeted costs 1
Calculating Budgeted Costs (1)
  • Aggregate direct costs
    • Security staff, training, awareness, tools, services, technology, management, threat monitoring, assessments, etc.
    • Direct cost of predictable and expected loss events and remediation w/ portfolio effects
  • Use cost driver models for indirect costs
    • Patch testing, installation, upgrades, etc.
    • Vendor support costs, 3rd party support
    • Help desk
    • New employee screening and hiring process
    • Indirect costs of predictable and expected loss events with portfolio effects
  • Negotiate cost allocation rules for bundled and overhead costs
    • Infrastructure software and hardware costs
    • Application software
    • Internal IT development
    • Legal dept.
  • Identify costs from unintended consequences and “business prevention”
    • It’s a judgment call how best to account for these, but they will win credibility!
  • If possible, use incremental cost analysis, not just total costs
    • Compare to a base case (e.g. a “barely legal” budget)

Mini-Metricon, San Francisco - Feb 5, 2007

calculating budgeted costs 2
Calculating Budgeted Costs (2)

Modeling indirect costs using cost drivers: e.g.Desktop/Laptop Incidents and Remediation

Cost #1:

Provisioning

Illustrative

Cost #2:

Help Desk

  • Benefits:
  • Simplicity– many fewer budget categories than incident types, scenarios, etc.
  • Effectiveness – puts attentionon the right levers
  • Focus– most often, a few cost drivers dominate (80/20 rule).

Platform Policy

# devices / yr.

Awareness

Compliance %

  • Method:
  • Identify cost drivers using security metrics combined with business operational metrics (e.g. number of new employees, turnover, etc.).
  • Aggregate and simplify where possible.
  • Only account for budgeted (forward-looking) costs. Use historical costs as a guide, if available.

Mini-Metricon, San Francisco - Feb 5, 2007

calculating budgeted costs 3
Calculating Budgeted Costs (3)

Modeling indirect costs using cost drivers:e.g.Indirect costs of predictable and expected loss events, with portfolio effects

  • Benefits:
  • Simpler calculations
  • More robust to varying assumptions

Abstracted

and

Aggregated

attacks,

breaches,

incidents

Asset:

Customer

DB

Risk

Drivers

Exposure, given defenses

Damage, violations, etc.

Cost Drivers

  • Cost Categories:
  • Staff (extra headcount)
  • Customer Service (damage control)
  • etc.

Detection, remediation, etc..

Mini-Metricon, San Francisco - Feb 5, 2007

decision framework for budgeted costs differential analysis
Decision Framework for Budgeted Costs Differential Analysis

#3 Lifetime

#1 Total

Budgeted

Costs vs.

benchmarks

Higher

#4 Self-insurance Cost

Implications

Same

Indirect

Current

Lower

Time

Direct

Current

Budget

“Barely legal”

Budget

“ Premium”

Budget

#2 Budget Optimization

Mini-Metricon, San Francisco - Feb 5, 2007

self insurance cost
Self-Insurance Cost
  • Q: How much money would you put aside each year into a reserve fund* to avoid a serious decline in credit rating due to low-probability/high-impact losses?
  • The rule: an actuarially-sound self-insurance premium, given…
    • Budget-busting loss events
      • Severe outage, delay in a key new product, loss of major sales contract, etc.
      • Material to quarterly EPS (> 1% )
    • Extreme loss events (short of bankruptcy) that threaten credit rating, etc.
      • Long-lasting business interruption, executive fraud, earnings restatement, regulatory action, punitive damages, etc.
    • Interdependencies, correlations (“avalanche effects”), and portfolio effects
    • Parameters: Maximum risk threshold and time horizon set by top management
    • “Mark to Model” approach, calibrated by history & “wisdom of the crowds”
  • A betting man’s judgment: “The race doesn’t always go to the swiftest, but that’s how you bet.”

*Analogous to the concept of Economic Capital in financial services

Mini-Metricon, San Francisco - Feb 5, 2007

calculating self insurance cost 1
Calculating Self-Insurance Cost (1)

Annual premium ≈ Pool ÷ (Time Period)

Cost distribution curve

(if time period is long enough)

Estimation Parameters

Budget threshold

99th Percentile

threshold

Time period*

1

2

3

Self-insurance pool (“Value at Risk”)

Fund solvency*

5

Shape of the curve

4

Interest rates

6

Magnitude of costs

  • Modeling:
  • Distribution curves from parameters
  • Monte Carlo simulation of self-insurance pool with funding parameters, interest rates, etc. to calculate annual premium
    • Dominated by largest losses

2

* Policy decisions by top management

Mini-Metricon, San Francisco - Feb 5, 2007

calculating self insurance cost 2
Calculating Self-Insurance Cost (2)

Parameter values change with new information

How:A Competitive Marketplace for Models

parameter

Prediction Markets

Bayesian Networks

External data bases, benchmarks

time

Consensus

Estimates

Statistical

analysis of

historical loss data

Qualitative Reasoning (e.g. Inference to the Best Explanation, Reasoning about Uncertainty, etc.)

Simulations

Delphi Technique

Assessments,

Scorecards

Mini-Metricon, San Francisco - Feb 5, 2007

ways to make self insurance cost real
Ways to Make Self-Insurance Cost “Real”
  • Link it to real cyber insurance policies
  • Set up a real self-insurance fund via Finite Risk program* or tradable subordinated debt
  • Use it as the “glue” for multi-firm “risk sharing” pools
    • Focused on information sharing and mutual assistance, with incentive instruments
  • Link to performance management and incentive compensation
    • Subdivide Self-Insurance Cost into a “Risk Budget” for each org. unit, or
    • Use it as a “risk adjustment” factor for other performance metrics
  • Create incentive instruments tied to self-insurance costs or cost drivers for…
    • Security outsource vendors
    • Supply chain partners
    • Channel partners
    • Customers
    • Alliance partners
  • Public disclosure
    • SEC filings, other regulatory filings
    • Stakeholder reports
    • Credit rating agencies
    • “Cap and Trade” markets

*See appendix

Mini-Metricon, San Francisco - Feb 5, 2007

catastrophic costs
Catastrophic Costs
  • Q: How much confidence should we have that the firm can survive InfoSec catastrophes?
  • The rule: prioritized loss scenarios above a significance threshold that cover the space of possibilities.
    • Use for business continuity preparation→ agility and robustness
    • Avoid failures of imagination and “fighting the last war”
    • Root out unintended consequences
    • Categorize and prioritize – don’t waste time on precision estimates
    • Strategic scenario analysis, “war gaming”, etc.
    • Focus on discovery, “out of the box”, and reframing
    • Challenge conventional wisdom!
  • “It’s not what we don’t know that will kill us. It’s what we know that ain’t so”.

Mini-Metricon, San Francisco - Feb 5, 2007

risk management decisions
Risk Management Decisions

Gambling

Prudence

Budgeted

Costs

Catastrophic

Costs

Self-insurance

Costs

Mini-Metricon, San Francisco - Feb 5, 2007

a simple example earthquake preparation
A Simple Example – Earthquake Preparation

Spend an extra $1,440 per year over 30 yearsfor earthquake loss reduction?

*from Monte Carlo simulation

  • ALE same for both
  • Simple average says “no” to extra spending

Mini-Metricon, San Francisco - Feb 5, 2007

self insurance costs 1
Self-insurance Costs (1)

Mini-Metricon, San Francisco - Feb 5, 2007

self insurance costs 2
Self-insurance Costs (2)

Justifies extra spending on maximum preparation

Mini-Metricon, San Francisco - Feb 5, 2007

needed self insurance decision framework
Needed: Self-insurance Decision Framework

A. Like other insurance

Which is more credible?

Which leads to better decisions?

B. Self-borrowing

Mini-Metricon, San Francisco - Feb 5, 2007

summary of the method
Summary of the Method
  • Apply enterprise risk management methods
  • Break InfoSec costs into three categories:
    • “Budgeted”
    • “Self-insurance”
    • “Catastrophic”
  • Establish methods, targets, and decision processes for each category
    • Appropriate to the information and uncertainty involved
    • The nature of decisions that apply
    • Link the categories
  • Use operational metrics plus inference to model costs in each category, as appropriate
  • Focus energy on continuous organization learning

Mini-Metricon, San Francisco - Feb 5, 2007

next steps
Next Steps
  • Need more theoretical development and empirical testing
    • Esp. self-insurance concept, models, and decision rules.
    • Factoring in impact on revenue, market share, profitability (pricing power), and reputation
  • Need to standardize “Budgeted Costs” and map to InfoSec assessments and frameworks
  • Need proofs-of-concept using real companies and real data
  • Make it work politically
    • Enterprise Risk Managers = your new best friends
    • TQM and 6 Sigma Specialists = your allies
    • CFOs = Status excelsior sponsors
    • Neutralize or convert opposition (legal department, auditors, etc.)
    • Lead industries = Financial Services? Supply Chain? other?
    • Political change role model = Indian Gaming?? 
  • Make it acceptable to the mainstream managers
  • Q: is it sufficiently promising to continue pursuing?

Mini-Metricon, San Francisco - Feb 5, 2007

appendix

Appendix

Russell Cameron Thomas

Principal, Meritology

russell.thomas@meritology.com

Mini-Metricon, February 5, 2007

San Francisco, CA

why measuring the value of infosec is hard 1
Why Measuring the Value of InfoSec is Hard (1)
  • Information security (InfoSec) should be seen* as a component of enterprise risk management.
    • "Risk” is a forward-looking estimate of uncertain loss over a time period (same as the timeframe for return on the assets).
    • Must cope with all forms of uncertainty and ignorance that apply to actors, assets, threats, vulnerabilities, and learning/adaptation over that timeframe.
  • InfoSec is a repeating evolutionary game
    • Between threatening actors (incl. nature) and protecting actors (incl. nature)
    • Each with an evolving capability set, which may be emergent, nascent, and/or tacit.
    • The terrain for the security game is threats, vulnerabilities, assets, etc.
    • Thus, "security" is not a state of the system or the assets. It's how the protecting actors define success in the game over time.
    • Economics of repeating evolutionary games aren’t well understood yet. They don’t fit existing static equilibrium investment models. They require emergent, dynamic models, e.g. agent-based simulation

*From the viewpoint of business value

Mini-Metricon, San Francisco - Feb 5, 2007

why measuring the value of infosec is hard 2
Why Measuring the Value of InfoSec is Hard (2)
  • InfoSec* is inextricably part of the cyber trust “fur ball”, including
    • Privacy
    • Digital Rights
    • Intellectual Property, brands, reputation, trade secrets
    • Stakeholder disclosure
  • … and physical security
  • Historical loss data, even if copious and available, has limited use
    • The landscape changes too fast
    • Low frequency / high impact events matter
    • Unique events matter
  • The business value of InfoSec isn’t just loss prevention
    • Value comes from the ability to support profitable risk taking
      • e.g. Brakes, condoms
    • Risk balancing is a reflexive process involving perceptions of risk and reward
  • Varies dramatically by industry and sector
    • E.g. a bank vs. a rock quarry

*From the viewpoint of business value

Mini-Metricon, San Francisco - Feb 5, 2007

blind alleys and dirt roads
Blind Alleys and Dirt Roads
  • “Blind Alleys” look good in concept, but won’t work by themselves
    • Return on Investment (ROI), Net Present Value (NPV), Payback, etc.
    • Annualized Loss Expectancy (ALE)
    • Cyber insurance
    • Product liability and tort laws (“actual damages”)
  • “Dirt Roads” work, but just barely
    • 2x2 or 3x3 matrix categorization of incident types or risks by frequency vs. severity
    • Assessments using scoring and ranking systems
    • Balanced scorecards
    • Strategic scenario analysis and walkthroughs
  • Are there any “Autobahn” approaches out there?
    • The null / “realist” hypothesis is “no”, assuming insurmountable problems
    • “Total Cost of (In)security” might be such an approach

Mini-Metricon, San Francisco - Feb 5, 2007

why ale is dumb
Why ALE is Dumb
  • A Simple Case of Three Loss Event Categories*
    • Firm Equity = $50 million; Annual Earnings = $5 million; ROE = 10%
    • Category A: “Common flood”
      • 50% chance of $10,000 loss = $5,000 ALE
    • Category B: “100 year flood”
      • 1.0% chance of $500,000 loss [10% of earnings, 1% of equity] = $5,000 ALE
      • 26% chance of happening at least once in 30 years
    • Category C: “10,000 year flood”
      • 0.01% chance of $50 million loss [100% of equity] = $5,000 ALE
  • Reason 1: ALE math hides risk drivers
    • A+B+C = A+A+A = B+B+B = C+C+C = $15,000 ALE [1.5% of earnings]
    • Conflates simple random walks with random walks with avalanches
      • “Three independent common risks = three independent catastrophic risks”
  • Reason 2: Unreliable estimates of low probability events dominate
    • Lack of data + psychology means estimation errors for the tail are much higher
      • 50% ® 55% chance for A ® $5,250 ALE
      • 1.0% ® 2.0% chance for B ® $10,000 ALE (45% chance in 30 years!)
      • 0.01% ® 0.05% chance for C ® $25,000 ALE
      • S = $40,250 ALE (2.7 times larger!)

*Pareto Distribution, k=1, min = 5,000

Mini-Metricon, San Francisco - Feb 5, 2007

finite risk programs
Finite Risk Programs

The insurance industry offers multi-year self-insurance plans that are

commonly called finite risk insurance. The name arises from the fact that

the risk transfer is very limited. Therefore, the insured will pay for most (or all)

the losses

Year 1

time

Balance carry-forward

Fund established

$$$

Operational

losses

From: “Applying Insurance Modeling

Techniques to Quantify OR”

Dr Marcelo Cruz, RiskMaths,

presented at

GARP OR Seminar

18-19 October 2001 London

Interest paid

Mini-Metricon, San Francisco - Feb 5, 2007

ruin theory applied to finite risk
Ruin Theory applied to Finite Risk

Losses following a

certain stochastic process

Finite Risk hedging

needs

Initial Finite Risk capital

Percentage of gross

income allocated against Finite Risk

From: “Applying Insurance Modeling

Techniques to Quantify OR”

Dr Marcelo Cruz, RiskMaths,

presented at

GARP OR Seminar

18-19 October 2001 London

“ruin”

Mini-Metricon, San Francisco - Feb 5, 2007