Quantifying Risk – How hard can it be?

1. Quantifying Risk – How hard can it be? Mark Swabey Managing Director Risk Reasoning Ltd

2. The Challenges • “What contingency do we need to include in our bid?” • “What’s our exposure?” • “Do we really need to take that action?” • “I’ve got to spend that money, even if the risk doesn’t happen?!?!” • “What about timescales?” • “…. and our reputation….” • “…. and environmental impact …..”

3. The Potential Benefits • See your total risk exposure • See your residual risk exposure • See how much you need to spend in actions to achieve the residual exposure • Show that your actions are cost-effective • Use other measurement criteria • Time • Environmental Impact • Reputation • Provide the information directors need • For any type of risk assessment

4. The “qualitative” risk management process Identify Assess Manage Respond

5. Elementary Risk Calculation • Basic Risk calculation • Risk Severity = Risk Probability x Risk Impact • So, a simple example: • Risk Severity = 20% x £10,000 = £2,000 • Risk Management is about controlling risk. • To control a risk • Reduce the Probability and/or • Reduce the Impact • Note: For illustration purposes, risk is assumed to be a threat. The maths works just as well for opportunities.

6. Risk Priority Matrix – Heat Map Limiting Actions reduce the impact of the risk - pushes risk down Psychological trick: In western cultures, we concentrate first on top left, since we read like that, so area to concentrate upon is top left. Preventative Actions reduce the chance of risk occurring - pushes risk to right

7. Actions and their effects • Preventative Actions • To minimise the chance of the risk occurring • Action will have to be taken before the risk occurs, whether the risk occurs or not • So we have to spend what it costs • Focus – insurance never prevented a risk! • Limiting Actions • To limit, or mitigate, the impact of the risk • Action may have to be taken, only if the risk occurs • So we may have to spend what it costs • The action can be taken before the risk occurs, but then the whole cost of the action is spent. • Insurance? 2 limiting actions: • paying the premium before the risk (which doesn’t affect the risk!), and • claiming if the risk occurs. • Effect • The chance of success in eliminating the risk

8. Total residual risk severity • One risk, with one preventative action and one limiting action (used if the risk occurs) • Total Residual Risk Severity • = (Cr x Pr x (1-Ppa) x (1-Pla)) + Cpa + (Cla x Pr x (1-Ppa)) • The total residual cost is: • the cost of the risk x the risk probability x the probability that the preventative action doesn’t work x the probability that the limiting action doesn’t work, plus • the cost of the preventative action plus • the cost of the limiting action x risk probability x the probability that the preventative action doesn’t work.

9. Total residual risk severity - example • A risk with 20% probability and £100,000 impact, with • one preventative action costing £1,000 with a 40% chance of success and • one limiting action costing £4,000 with a 30% chance of success • Risk Severity = Cr x Pr • = 100,000 x 20% = £20,000 • Total Residual Risk Severity • = (Cr x Pr x (1-Ppa) x (1-Pla)) + Cpa + (Cla x Pr x (1-Ppa)) • = (100,000 x 20% x 60% x 70%) + 1,000 + (4,000 x 20% x 60%) • = 8,400 + 1,000 + 480 = £9,880

10. Preventative Actions Lower cost Wider range of alternatives Can be included in normal plan Limiting Actions Higher cost Narrow range of alternatives Risk-driven, needs change of plan Preventative vs Limiting Actions Features

11. Risk Risk Action Risk Action Risk Action Risk Action Risk Taking more than one an action into account • A risk may need a number of actions • An action may have an effect on more than one risk Risk Action Action • The chance of each risk and of each action being needed (and when) is now a more complex function – a probability network. • Very difficult in spreadsheets!

12. Risk Priority Matrix revisited Risks - No Actions Risks with Actions

13. BUT – Uncertainty in Estimates • If a man begin with certainties, he shall end in doubt; • But if he will be content to begin with doubts, he shall end in certainties. • Francis Bacon, 1561-1626 • Why? • We don’t know the detail • We can think of alternatives • We know there are aspects that we don’t know (Donald Rumsfeld!) Unknown unknowns Known knowns Known unknowns Reducing uncertainty, surely a risk management objective?

14. Describing Uncertainty • The Problems • If we give a single figure, we will be bound to it What is the chance of us being right? • Expressing uncertainty is difficult Bell Curves? Standard Deviations? Distributions? How many people really understand them? • Statistical methods need lots of data • We haven’t got any data, or • The available data isn’t relevant to our situation • Conclusion: Traditional statistical approaches are not helpful

15. Introduction to the third international workshop on Soft Methods in Probability and Statistics (5-7 Sept 2006) • “Over the last thirty years there has been a growing interest in extending the theory of probability and statistics to allow for more flexible modelling of uncertainty, ignorance and fuzziness. Most such extensions result in a "softening" of the classical theory, to allow for imprecision in probability judgements and to incorporate fuzzy constraints and events. Many approaches utilise concepts, tools and techniques developed in theories such as fuzzy set theory, possibility theory, imprecise probability theory and Dempster-Shafer theory. • The need for soft extensions of probability theory is becoming apparent in a wide range of applications areas. For example, in data analysis and data mining it is becoming increasingly clear that integrating fuzzy sets and probability can lead to more robust and interpretable models that better capture both the inherent uncertainty and fuzziness of the underlying data. Also, in science and engineering the need to analyse and model the true uncertainty associated with complex systems requires a more sophisticated representation of ignorance than that provided by uninformative Bayesian priors. ”

16. Fuzzy sets • Use multiple values and alternatives • Handle uncertainty • Easier to understand by humans, not just mathematicians! • Applications: • Passive sonar classification – interpreting sound spectrum with extremely low signal-noise ratio • Real-time modelling of pilot decision-making in attack helicopter combat • Intelligence analysis with unreliable sources • Wide range of engineering applications • Techniques derived from FRIL (originally Fuzzy Relational Inference Language) from the Engineering Maths Department of Bristol University

17. Medium Definite VHigh vLow None Low High Low Medium-High Don’t Know Expressing uncertainty – sets and ranges Describing Impact Describing Probability

18. Estimating using Fuzzy Sets • Easy to understand • Impact: “It’s definitely going to cost £10,000, a medium chance of costing £30,000, and it couldn’t cost more than £100,000” • Probability/chance: “Somewhere between very low and medium”, “I don’t know” • Overcomes the fear of a single value • Psychological trick: Ask about the impact first. When you have an answer, then ask about probability. The probability range is likely to be narrower. Reason: Participant has thought through various alternatives in defining impact, so has formed a better view of the probability range.

19. Preventative Actions Graphs show improvement, effect of actions and uncertainty Limiting Actions Impact uncertainty increased due to uncertainty of effectiveness of Limiting Action Probability uncertainty reduced

20. Assessment Summary • Total Exposure can be seen • Residual Exposure can be seen • The benefits of the actions are very clear • Clear justification for spending money on actions • The same principles apply to time and other criteria e.g. reputation

21. Assessment Summary - Uncertainty Add the fuzzy sets to give: • Whole uncertainty profile is same as S curve from Monte-Carlo simulation (except that axes are swapped around)

22. Showing the benefit of an action (1) • One action isn’t authorised (29 actions, 28 authorised) so isn’t included in the calculations • What happens if we authorise it?

23. Showing the benefit of an action (2) • Action Authorised: • Risk cost decrease: £467,270 to £391,335 = -£75,935 • Action cost increase: £187,208 to £206,993 = £19,725 • Action plans can now be optimised for effectiveness • Alternative actions can be tested to find the most effective action

24. Other Measurement Criteria • Time - Calculated in a similar manner to cost • Qualitative Measures • A pragmatic shortcut to complex measures • The mathematics still works the same way, if limited-disastrous are equated to a scale • Uncertainty can be expressed as a range e.g. Limited-Significant

25. Summary • Residual risk exposure calculation is simple in theory, but complex in practice • Multiple inter-related risks and actions dramatically increase complexity • Standard statistical methods are not helpful in many cases • Representing uncertainty using fuzzy sets and probability ranges works well • Fuzzy sets and probability ranges easier than conventional statistics to understand and use • General-purpose mathematics that can be extended to express time and other criteria

26. Conclusions • Quantifying risk is possible in the standard risk management process • It can rapidly become complex, so most people don’t explore it • Very difficult to model robustly on spreadsheets • Programmed solutions can offer considerable benefits and business decision support at all levels • Users can see the benefits of good risk management • Managers can understand their organisation’s risk exposure • Everyone can see the potential effectiveness of taking specific actions • Matrices and comparative graphs help users to concentrate on priorities All illustrations used in this workshop generated by our products, collaborative risk management environments which use these principles. RiskAid

27. & RiskAid RiskAid Enterprise Thank youmark.swabey@riskreasoning.co.uktel: 07966 548123 www.riskreasoning.co.uk Supporting best practice risk management by normal people