Skip this Video
Download Presentation
Brooks, Legault, Nitschke, O’Brien, Sosebee, Rago, and Seaver

Loading in 2 Seconds...

play fullscreen
1 / 20

Brooks, Legault, Nitschke, O’Brien, Sosebee, Rago, and Seaver - PowerPoint PPT Presentation

  • Uploaded on

Nothing gives rest but the sincere search for truth. Blaise Pascal (French philosopher). WP 2.4 Evaluation of NMFS Toolbox Assessment Models on Simulated Groundfish Data Sets Comparative Simulation Tests Overview. Brooks, Legault, Nitschke, O’Brien, Sosebee, Rago, and Seaver. What did we do?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Brooks, Legault, Nitschke, O’Brien, Sosebee, Rago, and Seaver' - arawn

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Nothing gives rest but the sincere search for truth.

Blaise Pascal (French philosopher)

WP 2.4Evaluation of NMFS Toolbox Assessment Models on Simulated Groundfish Data SetsComparative Simulation Tests Overview

Brooks, Legault, Nitschke, O’Brien, Sosebee, Rago, and Seaver

what did we do
What did we do?
  • Evaluated 5 NFT stock assessment models for three stocks under 4 scenarios meant to examine potential difficulties in real assessments
    • GB yt (retro), GB cod (domes), white hake (ageing)
  • PopSim used to generate true conditions and create 100 datasets with the same random errors for all 5 models
  • Evaluated Accuracy and Precision of the 100 point estimates from the models
    • Did not examine precision of each of the 100 runs

60 scenarios

6000 assessments

  • Test hypothesis that all models are impacted similarly when presented with the same underlying problem
  • A priori know that some models will not perform well under the test conditions because limiting data to VPA years
  • NOT trying to declare one model “winner”
  • NOT trying to declare any model “bad”
popsim primer
PopSim Primer

Age and Length Based Population Simulator

User defines

  • Dimensions (Years, Ages, Plus Group Age, Lengths)
  • Initial NAA
  • Recruitment time series (or SRR)
  • Annual Fmult and selectivity
  • Biological Characteristics
    • M, von B, L-W
  • Fishery Sampling
  • Surveys
  • Sets Template for Stock Assessment Model
surveys vs indices
Surveys vs Indices
  • Surveys
    • Are a property of the true population
    • Catchability defined for all ages and years
    • Uncertainty added to true values at age and length
  • Indices
    • Are a property of the model
    • Sum values from surveys
    • Can be either number or biomass based
    • Can be limited age range or entire age range
    • Can be changed between models without impacting underlying truth
  • Initial NAA distributed according to stdev1
  • Growth transfer matrices created for each age based on expected von B growth for age and stdev2
  • Fish not allowed to decrease in size
  • Allows fishing to change distribution of length at age
market sampling
Market Sampling
  • Markets declared by user
  • Sampling conducted per 100 mt of landings in each market each year


popsim limitations
PopSim Limitations
  • PopSim is not reality
  • Annual Time Steps
  • Does not contain spatial components
  • Does not allow gender differences
  • Does not allow density dependent effects
  • No integrated management
    • Developing MSE wrapper to use PopSim, VPA, AgePro, and Control Rules
this exercise
This Exercise
  • Used utility to convert VPA run to PopSim
    • Gets Nyears, plus group age from VPA
    • Sets initial NAA and R from VPA
    • Sets annual Fmult from VPA
    • Estimates one logistic selectivity from VPA
  • Length and biology stuff from user
  • Market stuff from user
  • Surveys and Indices defs from user
  • Tuned markets, sampling, and surveys to represent actual assessments by lead
farmed out assessments
Farmed Out Assessments
  • AIM – Rago
  • ASPIC – Brooks
  • SCALE – Nitschke
  • VPA – Legault, O’Brien, Sosebee
  • ASAP – Legault
  • Used base case to get template settings reasonable
  • Applied this base case to each of the test cases
  • Some models did additional runs with modified templates to “fix” the problem
  • PopSim compares the distribution of 100 assessments with the known true values
  • Exactly what is compared depends on model
    • E.g. VPA NAA & FAA, ASPIC B & F
  • Many, many runs and scenarios
    • PopSim creates tables and graphs
    • R program to gather results and automatically create plots
started by looking at direct results
Started by looking at direct results

Black Line True

Circles and Grey Line Median

Red dashed Lines 5 and 95%iles

general conclusions
General Conclusions
  • Given failure of all models tested (simple & complex), we suspect other models would also be vulnerable to “retrospective agents”
  • Use of age-specific indices is robust to uncertainty in survey selectivity
  • If ageing is uncertain, these simulations support using models w/o age or models which allow uncertainty in catch at age
general conclusions cont
General Conclusions (cont.)
  • VPA and ASAP ‘failures’ were similar in pattern
  • Magnitude of bias was less for ASAP
  • Precision usually somewhat better for ASAP
  • Given these similarities, we suggest that ASAP may offer some advantages to VPA (esp. in terms of flexibility)