1 / 21

Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models

Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models. John A. Major, ASA Guy Carpenter & Company, Inc. Example: Exponential Distribution. N=20 observations T = sample mean; l =1 true mean MLE EP curve: q-exceedance point (PML, VaR) X .01 = 4.605 actual.

Download Presentation

Taking Uncertainty Into Account: Bias Issues Arising from Uncertainty in Risk Models

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Taking Uncertainty Into Account:Bias Issues Arising from Uncertainty in Risk Models John A. Major, ASA Guy Carpenter & Company, Inc.

  2. Example: Exponential Distribution • N=20 observations • T = sample mean; l=1 true mean • MLE EP curve: • q-exceedance point (PML, VaR) • X.01 = 4.605 actual

  3. Sampling Distribution of T

  4. Estimated PDFs

  5. Client Questions • What is the 1 in 100-yr PML (1% VaR)? • What is probability of exceeding 4.605? • Can you give me an EP curve to answer these and similar questions? • Does sampling error affect the answer? • Can I get unbiased answers?

  6. 3 Kinds of Bias • “dollar” or X-bias: • the average of PML dollar estimates • “probabilistic” or P-bias: • the average true exceedance probability of estimated PML points • “exceedance” or Q-bias: • the average estimated exceedance probability

  7. Exponential MLE is X-unbiased

  8. Exponential MLE is X-unbiased

  9. Exponential MLE is P-biased • for small q • Expected actual risk is greater than nominal • Uncertainty increases risk!

  10. Exponential MLE is P-biased

  11. Correcting for P-bias • Predictive distribution • “Prediction interval” in regression • Mix randomness and uncertainty • integrate model pdf over parameter distribution • Exponential model: • Predictive result:

  12. Predictive vs. Model Density

  13. Which to use? • MLE curve is X-unbiased • no uncertainty adjustment, but... • on average, gets right $ answer • Predictive curve is P-unbiased • “takes uncertainty into account” and... • on average, reflects true exceedance pr • But they disagree... • and it gets worse...

  14. Exponential MLE is Q-biased • for small q • Expected estimated risk is greater than the true risk (at the specified threshold) • Uncertainty now causes risk to be overstated!

  15. Exponential MLE is Q-biased

  16. Correcting for Q-bias • Minimum Variance Unbiased Estimator • standard procedure in classical statistics • Rao-Blackwell Theorem • Expectation of unbiased estimator, conditional on sufficient statistic • Exponential model: • MVUE result:

  17. MVUE vs. Model Density

  18. Paradox • Say we get an estimated T=1 (correct) • MLE says X.01=4.605, Pr{X>4.605}=1% • Predictive: X.01=5.179 is p-unbiased • risk is greater than MLE answer because impact of uncertainty • MVUE: Pr{X>4.605}=.69% is q-unbiased • risk is less because MLE tends to overstate exceedance probability

  19. How the Paradox Arises

  20. Conclusions • Uncertainty induces bias in estimators • Biases operate in different directions • depends on the question being asked • There is no monolithic “fix” for taking uncertainty into account • Predictive distribution fixes p-bias, • while making q-bias worse

  21. Recommendations • First: Show modal estimates (MLE etc.) • Second: Show effect of uncertainty • Keep uncertainty distinct from randomness • Sensitivity testing w.r.t. parameters • Confidence intervals on estimators • Third: Adjust for bias only as necessary • Carefully attend to the question asked • Advise that bias adjustment is equivocal

More Related