1 / 19

Quantifying Uncertainty using Classical Methods (Likelihood Profile, Bootstrapping)

Quantifying Uncertainty using Classical Methods (Likelihood Profile, Bootstrapping). Fish 458, Lecture 12. Quantifying Uncertainty (an overview). Uncertainty comes in several forms: Process uncertainty (e.g. recruitment variability, natural mortality variability, birth-death processes).

oona
Download Presentation

Quantifying Uncertainty using Classical Methods (Likelihood Profile, Bootstrapping)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Quantifying Uncertainty using Classical Methods(Likelihood Profile, Bootstrapping) Fish 458, Lecture 12

  2. Quantifying Uncertainty(an overview) • Uncertainty comes in several forms: • Process uncertainty (e.g. recruitment variability, natural mortality variability, birth-death processes). • Observation uncertainty (e.g. CVs for abundance estimates). • Model uncertainty (is the model we chose correct; how many alternative models fit the data adequately?) • Estimation uncertainty – given a model and some data, how well do the data determine the parameters (and predictions) of the model. • Implementation uncertainty – given a management decision, it be enforced?

  3. Quantifying Uncertainty(an overview-II) • The various types of uncertainties can be distinguished by: • Can they be reduced by additional research or are they inherent to the system. • Can we quantify them using classical statistical methods. • Today we address “estimation uncertainty”. We defer the other types of uncertainties to future lectures.

  4. Estimation Uncertainty • We are going to quantify uncertainty about the estimates of the model parameters (and its predictions of state variables) under the assumption that the model (and likelihood) are correct. • Typical ways to quantify estimation uncertainty include computing standard errors and confidence intervals.

  5. Hint: Generic Solver Macro-I Sub ApplySolver(Minpars, TheSheet, TheFunctionValue) Worksheets(TheSheet).Activate SolverReset ' Small precision and automatic scaling SolverOptions Precision:=0.00001, Scaling:=True ' Specify the cell to minimize (MaxMilVal=2) and the parameters to change SolverOK SetCell:=Range(TheFunctionValue), MaxMinVal:=2, ByChange:=Range(Minpars) ‘ Add a constraint (in this case the cells must be positive) SolverAdd CellRef:=Range(Minpars), Relation:=3, FormulaText:=0 ' Don't ask anything at the end of the call SolverSolve UserFinish:=True End Sub

  6. Hint: Generic Solver Macro-II • Notes: • Many EXCEL versions do not have all the Visual Basic libraries needed to call SOLVER from a macro loaded. • Within the Visual Basic editor you will need to click “Tools” - “References” - “Solver.xls” to make this library accessible.

  7. Likelihood Profile (one parameter) • Fit the model to find the ML parameter estimates and the corresponding negative log-likelihood. • Select a set of fixed values for the parameter of interest. • Minimize the negative log-likelihood fixing the parameter to each value in turn. • Plot the difference between the negative log-likelihood from step 1 and those from step 3.

  8. A First Example-I Note: t0=0 and  is assumed known The problem: We are fitting a growth curve to some age and length data. We want to compute the likelihood profile for 

  9. A First Example-II We can compute confidence intervals from likelihood profiles Approximate 95% CI

  10. Likelihood profiles and confidence intervals • An 100-x% confidence interval for p parameters is determined by finding the values for the parameter(s) for which: • is the negative log-likelihood corresponding to the maximum likelihood estimates.

  11. Likelihood Profile (multiple parameters) • Fit the model to find the ML parameter estimates and the corresponding negative log-likelihood. • Select a set of fixed parameter combinations for the set of parameters of interest. • Minimize the negative log-likelihood fixing the values for the set of parameters of interest to each set of values in turn. • Plot the difference between the negative log-likelihood from step 1 and those from step 3 (this creates a surface).

  12. What about State Variables-I • Model outputs often include population size, harvest rate, etc. We are usually more interested in these quantities than about the parameters themselves. • However, the state variables are seldom parameters of the model (and cannot be made to be parameters of the model). This makes computing a likelihood profile for them difficult.

  13. What about State Variables-II • For each (target) value of the State Variable: • Add a penalty to the negative log-likelihood that increases as the difference between the target value and the model estimate is larger. • It is often a good idea to change the size of the penalty, w, as we get closer to the target (i.e. apply Solver several times, each time increasing w)

  14. A Likelihood Profile for Current Biomass

  15. Bootstrapping-I • Likelihood profile has some major disadvantages: • Dealing with derived quantities (e.g. state variables) can be difficult. • Dealing with multiple parameters simultaneously is very computationally intensive. • It is impossible to compute likelihood profiles for all of the state variables simultaneously. • These problems can be overcome through bootstrapping.

  16. Bootstrapping-II • Bootstrapping deserves a course of its own but, in (very) simple terms, it involves: • Generate a large number of pseudo data sets, each based on the original data set. • Fit the model to each such data set. • Compute summary statistics of interest (standard deviations, confidence intervals, etc.) from the results for each model fit.

  17. Bootstrapping III • The most common form of bootstrapping involves developing the pseudo data set by resampling the residuals (with replacement) and adding these to the model predictions: • is the index for year y in pseudo data set U, • is the model prediction of the index for year y, and • y* is selected at random from 1:n.

  18. Bootstrapping – IV(the biomass trajectory for Cape Hake)

  19. Readings • Hilborn and Mangel, Chapter 7 • Haddon, Chapter 3

More Related