Using Models Outside the Actuary’s Area of Expertise (Property and Casualty )

1. Using Models Outside the Actuary’s Area of Expertise(Property and Casualty) DFA Seminar July 17 & 18, 2000 Task Force On Complex Models

2. Development History Review of Standard Application to DFA Standard of Practice - Using Models Outside the Actuary’s Area of Expertise (Property and Casualty)

3. 11-15-95 American Academy of Actuaries Casualty Practice Council Meeting “ . . . Use of Outside Experts by Actuaries. We anticipated that this standard would provide guidance when an actuary is working with sophisticated, complex models involving scientific theories . . .” Development Background

4. . . . Believes that a new standard on the Use of Complex Models in Actuarial Practices is required. . . . Address the level of understanding of the model and any underlying theories . . . Also address what control processes must be used by the actuary in establishing a reliance on the model’s output. ASB Casualty Council Proposal to ASB Board

5. Why is a standard needed: Regulators and other users . . .question the applicability of these analyses when they are based in large part on the results of models that are either outside of the normal range of actuarial work or for which key parts of the model contain proprietary information and are not subject to normal disclosure processes. ASB Casualty Council Proposal to Board

6. Karen F. Terry, Chair Kay A. Cleary Alice H. Gannon Paul E. Kinson Ronald T. Kozlowski Godfrey Perrott David A. Lalonde Jeffrey F. McCarty Daniel M. Scheibenreif A. Eric Thorlacius Joan M. Weiss Kurt Reichle ASB Task Force on Complex Models

7. Issues • Is a standard needed? • Should it address when to use a model? • Who should standard apply to? • Raising the bar • Reliance on experts • Proprietary issues

8. First Exposure Draft • Released May 1998 • Comment Deadline September 1, 1998 • 38 postcards & 52 letters • 29 SOA affiliation • 21 CAS affiliation • 1 other • Significant Issues Raised

9. Definition of Actuarial vs. Nonactuarial How to define - include examples Practice is ever evolving Don’t put artificial fence around what is/isn’t actuarial Change focus to outside individual actuary’s area of expertise Applicability to Practice Areas Other Than P/C Examples in other Practice Areas - less need Relative benefits of more inclusive standard vs. pressing need in P/C area Significant Issues

10. Second Exposure Draft • Released September 1999 • Comment Deadline March 1, 2000 • 10 responses • 3 SOA affiliation • 7 CAS affiliation • Few Issues Raised

11. Revised by the Task Force on Complex Models Approved by the Casualty Committee of the ASB Adopted by the Actuarial Standards Board June 2000 Actuarial Standard of Practice No. 37

12. Section 1 - Purpose, Scope, Cross References, and Effective Date Section 2 - Definitions Section 3 - Analysis of Issues and Recommended Practices Section 4 - Communications & Disclosures Appendices Using Models Outside the Actuary’s Area of Expertise (P/C)

13. The purpose of this standard is to provide guidance to the actuary in using models that incorporate specialized knowledge outside of the actuary’s own area of expertise when developing an actuarial work product. This guidance addresses the actuary’s obligation to review the model and make appropriate disclosures. Section 1 - Purpose

14. This standard applies to actuaries who use models that incorporate specialized knowledge outside of the actuary’s own area of expertise when performing professional services in connection with property and casualty insurance coverages . . . This standard applies to all models whether or not they are proprietary in nature. The standard is intended to be used in conjunction with other actuarial standards of practice (ASOPs) providing guidance for actuarial work, . . . If a conflict exists between this standard and applicable law, compliance with applicable law is not considered to be a deviation from this standard. Section 1 - Scope

15. Expert (same as ASOP 17) “One who is qualified by specialized knowledge, skill, experience, training, or education to render an opinion concerning the matter at hand.” Model “An information structure, such as a set of mathematical equations, logic, or algorithms, that is used to represent the behavior of specified phenomena.” Section 2 - Definitions

16. When using a modelthat incorporates specialized knowledge outside the actuary’s own area of expertise,the actuary should: Determine appropriate reliance on experts Have basic understanding of the model Evaluate whether model is appropriate for intended application Confirm appropriate validation has occurred Determine appropriate use of model Level of effort in understanding and evaluating should be consistent with intended use and its materiality to results of actuarial analysis Section 3.1 - Introduction

17. Actuary should consider whether: individual is an expert model has been reviewed by experts in field there are standards that apply to the model or to the testing and validation of the model . . . certified as having met such standards Section 3.2 - Appropriate Reliance on Experts

18. “The actuary should be familiar with the basic components of the model and understand both the user input and the model output . . .” Model Components (familiar with basic components, basic understanding, is it generally accepted, how tested or validated, level of independent expert review) User Input (detail required to produce results consistent with intended use) Model Output (consistent with intended use) Section 3.3 - Understanding of the Model

19. Is the model appropriate for particular actuarial analysis (e.g., interest rate models built by academic finance community) May consider: Applicability of historical data Developments in relevant fields.Impossible to remain up to date if not an expert in the field Section 3.4 - Appropriateness of the Model for the Intended Application

20. The actuary should evaluate the user input and reasonableness of the model output Directed to ASOP 23, Data Quality Reasonability may consider Results of alternative models or methods Historical observations Consistency of relationships Sensitivity to variations in assumptions Section 3.5 - Appropriate Validation

21. Having completed sections 3.2-3.5, the actuary should use judgment to determine whether it is appropriate to use the model results, making any compensating adjustments as necessary. Does this clause provide an escape from the entire standard making it irrelevant? Section 3.6 - Appropriate Use of the Model

22. An actuary may rely on another actuary who followed this standard Satisfied that other actuary’s evaluation performed in accordance and is appropriate for intended purpose Disclose such reliance Section 3.7 - Reliance on Model Evaluation by Another Actuary

23. Document efforts demonstrating compliance Proprietary Information Disclosure Prescribed Statement of Actuarial Opinion Deviation from Standard Section 4 - Communications and Disclosures

24. When does the standard apply? What efforts are required? What are the implications to DFA practice? Application to DFA

25. If the actuary is not a DFA expert and is using a DFA model - yes If the actuary is a DFA expert using an economic model of which he is not an expert - yes If the actuary is a DFA expert, using a proprietary DFA model provided by another party, whose design is open to the actuary - no When does the standard apply in DFA?

26. A P&C actuary, active in the development of DFA models building a DFA model to analyze company investment and reinsurance strategies - considering use of CIR interest rate model as basis for economic simulation The actuary is not an expert on interest rate modeling so standard applies Let’s examine the standards analysis requirements (section 3) Case Situation: Using CIR in a DFA model

27. CIR paper is a standard in financial modeling Highly cited and reviewed (considered groundbreaking, but problems are noted) No professional standards are obvious that can be relied on in this area 3.2 Appropriate Reliance on Experts

28. Model Components - model described by a stochastic equation describing development of interest rates - developers highly regarded - seems simple but reasonable (it is widely quoted and reviewed but now rarely used for security pricing) User input - yield curve and volatility Model output - yield curves 3.3 Understanding of the Model

29. CIR is a one factor interest rate model (all bonds act as leveraged versions of one) This is a very weak assumption if examining investment strategies to control interest risk It is one of the standard early stochastic interest rate models developed for derivative pricing More advanced models have been developed since A few papers exist on use in DFA 3.4 Appropriateness of the Model for the Intended Application

30. Input is starting interest rates and volatility Model output Output yield curves do not look like actual yield curves Rates go up or down together Can get a reasonable range of yield levels, but not yield curve shapes Output does not change significantly for modest input parameter changes 3.5 Appropriate Validation

31. Based on the prior analysis “the actuary should use his or her professional judgement to determine whether it is appropriate to use the model results” 3.6 Appropriate use of the model

32. If the application does not examine alternative investment strategies or interest rate risk (e.g. examining alternative reinsurance arrangements) - probably yes due to low materiality to the results If considering alternative investment strategies or interest rate risk - in my opinion, no. Does use of CIR in DFA comply? “The actuary’s level of effort in understanding and evaluating the model should be consistent with the intended use of the model and its materiality to the results of the actuarial analysis” Section 3.1

33. “ … should be satisfied that the other actuary’s evaluation was performed in accordance with this standard …” Ask for a copy of the documentation required by 4.1 “ … be satisfied that the other actuary’s evaluation was performed in accordance with this standard … “ 3.7 Reliance on Model Evaluation by Another Actuary

34. How will this effect DFA practice? • Who is a DFA expert? • Requires greater discipline • Likely lead to more standardization • DFA model providers will likely need to provide documentation to support compliance • Actuaries will have to determine what they are experts on • Will make it very hard to justify using models which are out of date