CS 589 Information Risk Management

1. CS 589Information Risk Management 23 January 2007

2. Today’s Discussion • Start with risk • Discuss types of information risk • Start with systematic, modeling-based framework for assessing alternatives when risks are known • Continue with the hard part – specification of risk when risks are unknown

3. Next Week • Discuss specification of risks using probability distributions • Discuss incorporation of this information into a decision tree • Discuss ways to apply these techniques to Information Risk scenarios

4. After Next Week • Discuss the Expected Utility decision criterion • Discuss Multiple Objectives and Expected Value and Expected Utility • Discuss Applications in Information Risk Analysis and Management

5. References for Today • Clemen, R. L. and T. Reilly, Making Hard Decisions. Duxbury, 2001. • Gaffney Jr., J. E., J. W. Ulvila, “Evaluation of Intrusion Detectors: A Decision Theory Approach”, Proceedings of the IEEE Symposium on Security and Privacy. 2001.

6. Risk • ??? • Chance of something bad happening? • Having something bad happen? • Anything else?

7. Risk • The probability of an event occurring combined with the consequences of that event • Just about everything is risky • How do we actually measure risk?

8. Risk vs Uncertainty • Uncertainty • We don’t know what the key variables are • We don’t know how they relate to alternatives • Risk • Specify probability distributions • Connect them with alternatives • One goal: Uncertainty  Risk via Modeling

9. Thinking About Risk • Probabilities and Outcomes • Which is riskier? • Living near a large power generation station • International flight • Driving to Albuquerque • We have to define factors, events, outcomes, and associated probabilities

10. Dealing with Risk • Define Risk • Assess Risk • Define Alternatives for Handling the Risk • Evaluate Alternatives • Evaluate your Evaluation Model • Sensitivity Analysis • Implementation

11. Evaluation • Choosing among Alternatives • Should be Evaluated on the same dimension(s) • Expected Value • Expected Utility • Value at Risk (VAR) • Multiple criteria • Measurement of Alternatives on criteria dimensions is key – and another modeling issue

12. Sensitivity Analysis • Checking on the evaluation of each alternative by varying individual variables • Find the variable(s) that have the largest impact(s) on the ordering of alternatives • Goal: robust solutions

13. Visual Representation • Influence Diagrams • Connect factors, events • Help us define risks • Decomposition • Decision Trees • Ordering of decisions, risky events • Easy to see and present – and solve

14. Visual Representations • Squares denote Decisions • Circles denote Risks • Influence Diagrams – arcs connect decision and risk (aka chance) nodes • Decision Trees – decision and chance nodes are sequentially ordered from left to right

15. A Very Simple Example • Coin Flip Game • Decisions: Play/No Play • Risks: Heads/Tails • Outcomes Must be Specified

16. Coin Flip Game Decision Tree With $0 Outcomes

17. If All Outcomes are $0 • We are Indifferent between Play and No Play based on the Expected Value criterion • We Prefer Play to No Play if E(Play) > E(No Play) • Which means that the sum of the outcomes (if we have a fair coin) must be positive • Generally, Play if

18. What if we can play twice? • Sequential decision – we see the result of the first coin flip, and decide to continue • This leads to the notion of Strategies – we can make a plan contingent upon resolution of risks that are resolved between decision nodes • Everything is still based on Expected Value

20. Suppose • O(H) = $10, O(T) = -$7 • p(H) = p(T) = .5 (Fair coin) • We can easily see that we would choose to Play in the one-game case • What about the 2-game case?

22. Strategy • It’s pretty simple – keep playing • Would you really do this? • Do you believe this? • Why or why not??

23. Simple Example • Suppose we are assessing two alternative intrusion detection systems. • What’s the problem? • What are the key risks for this decision? • What are the decisions? • What are the outcomes? • How would we measure the outcomes? • What is the decision criterion?

24. Key Point • The optimal choice will be the one that is associated with the best expected criterion value – such as expected total cost • This will be determined by how we define the outcomes – in terms of total costs – and probabilities • When we roll back a decision tree, we assume that the downstream decision is the best one

25. Expected Value • Random Variable with possible discrete outcomes

30. What do we need to know? • Probabilities • P(Detection|An Intrusion)  P(D|I) • Associated Info • P(I) • And, finally, P(I|D) • Outcomes • Individually, these will not be stochastic – for now • They will still lead to an expectation for each decision node

31. Conditional Probability • P(D|I) and P(D| Not I) • P(Not D|I) and P(Not D|Not I) • Where would we get this information? • What about P(I)?

32. Bayes Rule – Simple Version

33. Interpretation • Two types of Accuracy • Two types of Error

34. Solving the Tree • Establish the Outcomes • Compute the Probabilities – the conditionals on the endpoints and others • Find Expected Values and roll back the tree

36. Sensitivity Analysis • What are the strategies given the numbers we used in the example? • What are the key variables? • How would we assess the base-case outcome of this example?

37. Different Conditional Information • What if we don’t know P(D|I)? • We can flip the tree according to what we do know • Outcomes should remain the same • And the decision should remain the same

38. Another Way – Info Dependent

39. Modeling • Decisions, chance events • Probability distributions for chance events • Lack of data  Bayesian methods • Expert(s) • Lots of data  Distribution model(s) • Outcomes • Financial, if possible • Multiple measures/criteria/attributes

40. Decision Situation • In the context of Firm or Organization Goals, Objectives, Strategies • A complete understanding should lead to a 1-2 sentence Problem Definition • Could be risk-centered • Could be oriented toward larger info issues • Problem Definition should drive the selection of Alternatives and, to some degree, how they are evaluated

41. Information Business Issues • Integrity and reliability of information stored and used in systems • Preserve privacy and confidentiality • Enhance availability of other information systems

42. Risk Management • Process of defining and measuring or assessing risk and developing strategies to mitigate or minimize the risk • Defining and assessing • Data driven • Other sources • Developing strategies • Done in context of objectives, goals