testing models on simulated data presented at the casualty loss reserve seminar september 19 2008 l.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Testing Models on Simulated Data Presented at the Casualty Loss Reserve Seminar September 19, 2008 PowerPoint Presentation
Download Presentation
Testing Models on Simulated Data Presented at the Casualty Loss Reserve Seminar September 19, 2008

Loading in 2 Seconds...

play fullscreen
1 / 26

Testing Models on Simulated Data Presented at the Casualty Loss Reserve Seminar September 19, 2008 - PowerPoint PPT Presentation


  • 231 Views
  • Uploaded on

Testing Models on Simulated Data Presented at the Casualty Loss Reserve Seminar September 19, 2008. Glenn Meyers, FCAS, PhD ISO Innovative Analytics. The Application Estimating Loss Reserves. Given a triangle of incremental paid losses

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Testing Models on Simulated Data Presented at the Casualty Loss Reserve Seminar September 19, 2008' - arnold


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
testing models on simulated data presented at the casualty loss reserve seminar september 19 2008

Testing Models on Simulated DataPresented at the Casualty Loss Reserve Seminar September 19, 2008

Glenn Meyers, FCAS, PhD

ISO Innovative Analytics

the application estimating loss reserves
The ApplicationEstimating Loss Reserves
  • Given a triangle of incremental paid losses
    • Ten years with 55 observations arranged by accident year and settlement lag
  • Estimate the distribution of the sum of the remaining 45 accident years/settlement lags
  • Loss reserve models typically have many parameters
    • Examples in this presentation – 9 parameters
danger possible overfitting
Danger – Possible Overfitting!
  • Model describes the sample, but not the population
  • Understates the range of results
  • The range is the goal!
  • Is overfitting a problem with loss reserve models?
  • If so, what do we do about it?
outline
Outline
  • Illustrate overfitting with a simple example
    • Example – fit a normal distribution with three observations
    • Illustrate graphically the effects overfitting
  • Illustrate overfitting with a loss reserve model
    • Reasonably good loss reserve model
    • Show similar graphical effects as normal example
normal distribution
Normal Distribution
  • MLE for parameters m and s
  • n = 3 in these examples
simulation testing strategy
Simulation Testing Strategy
  • Select 3 observations at random

Population - m = 1000, s = 500

  • Predict a normal distribution using the maximum likelihood estimator
  • Select 1,000 additional observations at random from the same population
  • Compare distribution of additional observations with the predicted distribution
pp plots
PP Plots

Plot

Predicted Percentiles x Uniform Percentiles

  • If predicted percentiles are uniformly distributed, the plot should be a 45o line.
view maximum likelihood as an estimation strategy
View Maximum Likelihood As an Estimation Strategy
  • If you estimate distributions by maximum likelihood repeatedly, how well do you do in the aggregate?
  • Consider a space of possible parameters for a model
  • Select parameters at random
    • Select a sample for estimation (training)
    • Select a sample for post-estimation (testing)
continuing prior slide
Continuing Prior Slide
  • Select parameters at random
    • Select a sample for estimation (training)
    • Select a sample for post-estimation (testing)
  • Fit a model for each training sample
  • Calculate the predicted percentiles of the testing sample.
  • Combine for all samples.
  • In the aggregate, the predicted percentiles should be uniformly distributed.
pp plots for normal distribution n 3
PP Plots for Normal Distribution (n = 3)

S-Shaped PP Plot - Tails are too light!

problem how to fit distribution
Problem – How to Fit Distribution?
  • Proposed solution – Bayesian Analysis
  • Likelihood = Pr{Data|Model}
  • We need Pr{Model|Data} for each model in the prior
bayesian fits
Predictive distributions are spread out more than the MLE’s.

On individual fits, they do not always match the testing data.

Bayesian Fits
bayesian fits as a strategy
Parameters of model were selected at random from the prior distribution

Near perfect uniform distribution of predicted percentiles

At least in this example, the Bayesian strategy does not overfit.

Bayesian Fits as a Strategy

Combined PP Plot for

Bayesian Fitting Strategy

analyze overfitting in loss reserve formulas
Analyze Overfitting in Loss Reserve Formulas
  • Many candidate formulas - Pick a good one
  • Paid LossAY,Lag ~ Collective Risk Model
  • Claim count distribution is negative binomial
  • Claim severity distribution is Pareto
    • Claim severity increases with settlement lag
  • Calculate likelihood using FFT
simulation testing strategy18
Simulation Testing Strategy
  • Select triangles of data at random
    • Payment pattern at random
    • ELR at random
    • {Loss|Expected Loss} for each cell in the triangle from Collective Risk Model
  • Randomly select outcomes using the same payment pattern and ELR
  • Evaluate the Maximum Likelihood and Bayesian fitting methodology with PP plots.
background on formula
Background on Formula
  • Fit a Bayesian model to over 100 insurers and produced an “acceptable” combined PP plot on test data from six years later.
  • This paper tests the approach to simulated data, rather than real data.
summary
Summary
  • Examples illustrate the effect of overfitting
  • Bayesian approach provides a solution
  • These examples are based on simulated data, with the advantage that the “prior” is known.
  • Previous paper extracted prior distributions from maximum likelihood estimates of similar claims of other insurers
conclusion
Conclusion
  • It is not enough to know if assumptions are correct.
  • To avoid the light tails that arise from overfitting, one has to get information that is:

Outside

The

Triangle