hydrologic modeling verification validation calibration and sensitivity analysis n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Hydrologic Modeling: Verification, Validation, Calibration, and Sensitivity Analysis PowerPoint Presentation
Download Presentation
Hydrologic Modeling: Verification, Validation, Calibration, and Sensitivity Analysis

Loading in 2 Seconds...

play fullscreen
1 / 22

Hydrologic Modeling: Verification, Validation, Calibration, and Sensitivity Analysis - PowerPoint PPT Presentation


  • 111 Views
  • Uploaded on

Hydrologic Modeling: Verification, Validation, Calibration, and Sensitivity Analysis. Fritz R. Fiedler, P.E., Ph.D. Definitions (review). Verification: check if code solves equations correctly Validation: check if model reasonably represents physical process

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Hydrologic Modeling: Verification, Validation, Calibration, and Sensitivity Analysis' - afya


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
hydrologic modeling verification validation calibration and sensitivity analysis

Hydrologic Modeling: Verification, Validation, Calibration, and Sensitivity Analysis

Fritz R. Fiedler, P.E., Ph.D.

definitions review
Definitions (review)
  • Verification: check if code solves equations correctly
  • Validation: check if model reasonably represents physical process
  • Calibration: adjust model parameters to match observations
  • Sensitivity Analysis: relative effect of parameter changes on output
verification
Verification
  • Compare numerical results to analytical results
level 1 validation
Level 1 Validation
  • Compare model results to simple experiments (can estimate parameters a priori)
calibration
Calibration
  • Adjust parameters to match observations
level 2 validation
Level 2 Validation
  • Compare model results to observations for a different input data set post-calibration
    • Reserve some data (do not use in calibration)
    • After finding parameters that result in “best fit,” run model with reserved input and compare to output
  • Problems with this?
  • What happens in practice?
sensitivity analysis
Sensitivity Analysis
  • Explore how parameter changes affect output
  • Sensitivity index:
calibration targets
Calibration Targets

Can physically based model parameters be measured? Why or why not?

goodness of fit
Goodness of Fit
  • Visual comparison between simulated and observed – look for trends in errors
    • A learned art
    • Use appropriate graph scales
  • Statistical performance measures
    • Consider mean daily discharge as calibration target
    • Q = observed
    • S = simulated
means and bias
Means and Bias

Common calibration strategy: fix bias first, revisit periodically, goal of no bias

slide11

Maximum Error:

  • Percent Average Absolute Error
sum of squares of errors
Sum of Squares of Errors
  • Most common basis for statistical goodness of fit
    • e.g., least squares regression, seek to minimize
root mean squared error
Root Mean Squared Error
  • Size of error usually related to size of events or values, thus RMSE typically smaller for dry periods, small watersheds (for example)
  • How would you modify RMSE to facilitate comparison?
percent rmse
Percent RMSE
  • Normalize RMSE by mean observed
  • Because the magnitude of RMSE varies with magnitude of values, by minimizing RMSE only, which part of hydrographs are primarily best fit in calibration?
  • How can this tendency be addressed?
nash sutcliffe
Nash-Sutcliffe
  • Very popular method of evaluating calibration
  • Reading: McCuen, R. H., Evaluation of the Nash—Sutcliffe efficiency index, Journal of Hydrologic Engineering, 11(6), 597-602, 2006 (note: author uses different variables)
line of best fit
Line of Best Fit

Analyze as in regression: hypothesis testing on A and B, residual analysis, correlation coefficient…

how to use statistical measures
How to Use Statistical Measures
  • For a given time period, e.g., 1 year, and/or averages over multiple years
  • Look for seasonal trends
how to use statistical measures1
How to Use Statistical Measures
  • By flow interval (value interval)
  • Errors as f(Q) – aim for no systematic variation
  • How would you pick the intervals?
exceedance plots
Exceedance Plots

x

x

x

x

Q, S

x

x

x

x

x

x

0

percent days exceeded

100

generalized calibration strategies
Generalized Calibration Strategies
  • Set realistic parameter bounds before starting
  • Fix insensitive parameters first; focus on most sensitive
  • Eliminate most bias early in process, revisit
  • Use regionalized variables as appropriate
  • Combine manual and automatic techniques
equifinality
Equifinality
  • Multiple combinations of parameters can lead to similar results
  • Issue with both multi-parameter lumped models (e.g., SAC-SMA) and spatially distributed models (e.g., CASC-2D)
  • Reading: Ebel, B. A. and K. Loague, Physics-based hydrologic-response simulation: Seeing through the fog of equifinality, Hydrological Processes, 20(13), 2887–2900, 2006