Challenges in Validating Catastrophe Models

1. Challenges in Validating Catastrophe Models Dr. Paul Rockett • Casualty Actuaries in Europe, 31 May 2013

2. Agenda • Introduction • Validation tools • Validating catastrophe models • The bigger picture • Summary Challenges in Validating Catastrophe Models

3. Introduction Challenges in Validating Catastrophe Models

4. What are catastrophe models? • Tools designed to evaluate the risk profile of a risk or set of risks exposed to catastrophe events Over Limit Event Set Module Event Set Module Loss Ratio Hazard Module Hazard Module Insurance Cover Vulnerability Module Exposure Module Exposure Module Financial Analysis Module Intensity Vulnerability Module Deductible Financial Analysis Module Challenges in Validating Catastrophe Models

5. What are catastrophe models? • They combine the frequency of catastrophe events with their severity to produce annual exceedance probability (EP) curves • EP Curves can be thought of like VAR. SII SCR is a VAR measure corresponding to the 99.5% conf level of non-exceedance in 1-year Convolution Challenges in Validating Catastrophe Models

6. What do they cover? • There are catastrophe models covering • Earthquakes • Windstorms, Tropical Cyclones • Storm Surge, River Floods, ... • Severe Convective Storm (Tornado, Hail, ...) • Bushfire, wild fire, ... • And man-made catastrophes • Terrorism, Explosion, Conflagration • Pandemic • Most important models for the insurance industry are US Hurricane, US Earthquake and Europe Windstorm • But it varies by company Challenges in Validating Catastrophe Models

7. What are catastrophe models used for? Challenges in Validating Catastrophe Models

8. Who builds catastrophe models? Challenges in Validating Catastrophe Models

9. Catastrophe models can drive the Solvency Capital Requirement Why validate catastrophe models? Material Uncertainty • Limited data • Many assumptions Why Validate? • Extreme events and subsequent losses e.g. • Hurricane Katrina • Hurricane Ike • Christchurch Earthquakes • Solvency II directive: • Article 124 (Validation Standards) • Article 229 TSIM18 (Validation Process) History Regulatory pressure Challenges in Validating Catastrophe Models

10. Validation Tools Challenges in Validating Catastrophe Models

11. Solvency II Validation tools Tools for use by all firms (Article 230 TSIM19) Model robustness Profit & loss attribution Stress & scenario testing Results against experience • Purpose: • Are risks covered by the model complete? • Explain Profits and Losses • Applications • Explains actual results using the model. • Assess level of unexplained profits and ability to reflect risk profile of the business • Purpose: • Modelling and estimation errors, • Any model weaknesses • Applications • Back-testing • Goodness of fit • Dependency assumptions • Supplement with qualitative analysis • Purpose: • Are capital requirements produced robust? • Applications • Sensitivity of results to changes in the key underlying assumptions • Stability of the model • Purpose: • Impact of single or multiple events • Applications • Challenge results and parameters • Dependencies and tails of distributions • Understanding of the risk profile • Supports capital allocation decisions • Exceptional but plausible large-loss events • Reverse stress testing • Limitations of the model Challenges in Validating Catastrophe Models

12. Solvency II Validation tools: Further (optional) tools set out by EIOPA Analysis of change Qualitative review Hypothetical portfolio Benchmarking • Purpose: • Reinforce the appropriateness of Pillar I and II results • Applications • Consider results of peers • to investigate whether alternative modelling approaches could improve quality • Note differences in company risk profiles • Purpose: • Additional qualitative validation tools • Applications • Use test • Using the model in BAU demonstrates sufficient level of comfort that the results are appropriate for use • Qualitative review • With dialogue and review, gain comfort: • Theoretical basis of the internal model • Any part of the internal model • Purpose: • Analyse how the results of the model have changed • Applications • Reconciles results • Validates that the model is working as expected • Purpose: • Improve comparisons between different models or versions • Applications • Hypothetical portfolios of assets and/or liabilities • chosen by the company to analyse outcomes from different versions of internal models • Care needs to be taken to avoid creating systemic risk Challenges in Validating Catastrophe Models

13. Validating Catastrophe Models Challenges in Validating Catastrophe Models

14. Input Cat Modules Output Validating catastrophe models Event Set Exposure Hazard EP Curves Exposure Data ELTs Vulnerability • Each component needs validation Financial Analysis • Main focus is on model output Challenges in Validating Catastrophe Models

15. Exposure data Exposure Data • Key Aspects • Typical Approach • Qualitative review inc. data audit • Validation Challenge • Ensuring processes in place track all the exposure information, and that it passes quality control checks before modelling Output Cat Modules Input Event Set Exposure Hazard Vulnerability Financial EP Curves ELTs Challenges in Validating Catastrophe Models

16. Event set module Exposure Data • Key Aspects • Typical Approach • Back-testing, Sensitivity Testing, Benchmarking • Validation Challenges • Need an understanding of the physical mechanism to define suitable tests Output Cat Modules Input Event Set Exposure Hazard Vulnerability Financial EP Curves ELTs Challenges in Validating Catastrophe Models

17. Hazard module Exposure Data • Key Aspects • Typical Approach • Back-testing, Qualitative Review • Validation Challenge • Sensitivity testing very difficult Output Cat Modules Input Event Set Exposure Hazard Vulnerability Financial EP Curves ELTs Challenges in Validating Catastrophe Models

18. Exposure module Exposure data • Key Aspects • Typical Approach • Benchmarking, Audit • Validation Challenge • Very involved to properly check Output Cat Modules Input Event Set Exposure Hazard Vulnerability Financial EP Curves ELTs Challenges in Validating Catastrophe Models

19. Vulnerability module Exposure Data • Key Aspects • Typical Approach • Qualitative Review, Sensitivity Testing • Validation Challenge • Limited data Output Cat Modules Input Event Set Exposure Hazard Vulnerability Financial EP Curves ELTs Challenges in Validating Catastrophe Models

20. Financial module Exposure Data • Key Aspects • Typical Approach • Qualitative Review, Audit • Validation Challenge • Understanding the real level of uncertainty Output Cat Modules Input Event Set Exposure Hazard Vulnerability Financial EP Curves ELTs Challenges in Validating Catastrophe Models

21. Output Exposure Data • Key Aspects • Typical Approach • Back-testing, Sensitivity Testing, Qualitative Reviews, Benchmarking • Validation Challenge • Limited data Output Cat Modules Input Event Set Exposure Hazard Vulnerability Financial EP Curves ELTs Challenges in Validating Catastrophe Models

22. Bigger Picture Challenges in Validating Catastrophe Models

23. Scope of validation • Validating catastrophe models should be seen within the context of internal model validation • Each aspect needs validation Challenges in Validating Catastrophe Models

24. How to validate non-modelled perils? Corporate Governance • Non-modelled Risks • Key Aspects • Typical Approach • Qualitative Review • Validation Challenge • Subjective • Model Risk Management • Understanding Challenges in Validating Catastrophe Models

25. Corporate governance Corporate Governance • Non-modelled Risks • Key Aspects • Typical Approach • Qualitative Review, Audit • Validation Challenge • Checking the right structures, processes and controls are in place to ensure the IM is robust • Model Risk Management • Understanding Challenges in Validating Catastrophe Models

26. Understanding cat models Corporate Governance • Non-modelled Risks Knowledge of Catastrophe Models • Challenge • Improving knowledge flow • Model Risk Management • Understanding Challenges in Validating Catastrophe Models

27. Model Risk Management. Is the cat model really reasonable? Corporate Governance • Non-modelled Risks • Scenario: After a validation exercise, model C is selected for the IM. • Model C is revised. The company doesn’t approve. • Validation Challenge Robustness of the IM • Model Risk Management • Understanding Challenges in Validating Catastrophe Models

28. Summary Validating catastrophe models is challenging • Catastrophe models are complex, and comprise a number of sub-modules • Each sub-module is subject to significant uncertainties • Data to validate them is generally limited, but there are tools to help Validation needs to be seen in a broad context • Internal Model perspective includes adequate understanding by stakeholders, corporate governance, model risk management and non-modelled risks • It also considers the interface between cat models and capital models, pricing and accumulation management tools • Effort needs to be proportional to materiality Challenges in Validating Catastrophe Models

29. Contact Paul Rockett Senior Manager T: +44 (0)20 7951 1098 M: +44 (0)78 2408 4806 E: prockett@uk.ey.com Challenges in Validating Catastrophe Models

30. Reliance • This document is provided on the basis that you agree that Ernst & Young LLP (including its partners and staff) accepts no responsibility and shall have no liability in contract, tort or otherwise in relation to the contents of this document and that any use you make of this document is entirely at your own risk. • The information in this presentation pack is confidential and contains proprietary information of Ernst & Young LLP.  It should not be provided to anyone other than the intended recipients without our written consent. • The information in this pack is intended to provide only a general outline of the subjects covered.  It should not be regarded as comprehensive or sufficient for making decisions, nor should it be used in place of professional advice. • Accordingly, Ernst & Young LLP accepts no responsibility for loss arising from any action taken or not taken by anyone using this pack. • The information in this pack will have been supplemented by matters arising from any oral presentation by us, and should be considered in the light of this additional information. Challenges in Validating Catastrophe Models

31. Thank you