Model uncertainty
1 / 33

Model uncertainty - PowerPoint PPT Presentation

  • Updated On :

Model uncertainty. How uncertain are your modelling results?. This Session (#19):. Session 19 divides into two sections:. Overview of uncertainty analysis aims & objectives Analysis of uncertainty in breach modelling Findings and conclusions for breach modelling Mark Morris ~ 25 mins.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Model uncertainty' - oberon

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Model uncertainty l.jpg

Model uncertainty

How uncertain are your modelling results?

This session 19 l.jpg
This Session (#19):

Session 19 divides into two sections:

  • Overview of uncertainty analysis aims & objectives

  • Analysis of uncertainty in breach modelling

    • Findings and conclusions for breach modelling

      Mark Morris ~ 25 mins

  • Analysis of uncertainty in flood propagation

    • Findings and conclusions for propagation modelling

  • Overall conclusions & observations

  • Q&A

    Francisco Alcrudo ~ 25+10 mins

4th IMPACT Workshop - Zaragoza

Impact modelling uncertainty l.jpg
IMPACT: Modelling uncertainty:

Breach Formation





Sediment Movement


Geophysics /

Field Data


Assessment of modelling uncertainty

Implications for End User Applications


4th IMPACT Workshop - Zaragoza

Why do this l.jpg
Why do this?

  • Dambreak modelling requires a high degree of modelling expertise / experience. Uncertainties in the modelling process are higher than for ā€˜normalā€™ river modelling work.

  • Flood risk management may be performed more effectively if additional information is given supporting ā€˜best estimatesā€™.

4th IMPACT Workshop - Zaragoza

Aims objectives l.jpg
Aims & objectives:

  • Investigate uncertainty within modelling predictions for breach, flood propagation and sediment transport

  • Demonstrate how uncertainty within each of these modelling domains contributes towards overall uncertainty in predicting final conditions (e.g. water levels at specific locations)

  • Consider the implications of the magnitude of uncertainty found in terms of use by ā€˜end usersā€™

4th IMPACT Workshop - Zaragoza

Scope of work under impact l.jpg
Scope of work under IMPACT

The scope of work under IMPACT:

  • does not allow for an investigation of uncertainty in the impact of flooding or the assessment / management of flood risk

  • does not allow for development of extensive, statistical analysis techniques

    [Simple, practical assessment of likely order of magnitude of uncertainty]

4th IMPACT Workshop - Zaragoza

Progress to date l.jpg
Progress to date


  • Understanding the problem

  • Development of approach

  • Recognition of limitations for sediment modelling

  • Selection of case studies

  • Implementation for breach modelling

  • Implementation for propagation modelling

  • Analysis and conclusions







4th IMPACT Workshop - Zaragoza

Development of approach l.jpg
Development of approach...

  • A slow process that ended with a simple practicable but perhaps non rigorous approach balancing the need to analyse the problem with limitations in time and budget (the real world!)

  • The main arguments:

    • Fully rigorous statistically based analysis model by model and also linking of models

    • Removal of subjective - expert judgement type stages

    • Simple, practicable approach allowing indicative answer within acceptable time / budgets

  • Natural


4th IMPACT Workshop - Zaragoza

Approach taken l.jpg
Approach taken...


  • Recognition that current ability to model sediment movement under dambreak / extreme flood conditions was limited

  • Progress has been made (Yves Zech / Sandra Soares) but not sufficient to allow assessment of uncertainty

4th IMPACT Workshop - Zaragoza

Simple overview of approach l.jpg
(Simple) Overview of approach

1 Adopt the concept of using upper, mid and lower estimates of modelling results parameter (e.g. flood hydrograph, water level)

2 Assess uncertainty in breach model leading to upper / mid / lower flood hydrographs

3 Assess uncertainty in propagation models using 3 flood hydrographs from breach modelling as input conditions (leading to nine propagation predictions)

4 Select upper, mid, lower estimates for presentation of results to end user

4th IMPACT Workshop - Zaragoza

Some issues l.jpg
Some issues

  • Need to balance expert judgement (subjectivity):

    • What does upper, mid, lower mean?

    • What model parameters do you vary to create these?

    • Different models use different parameters - how do you compare these?

  • Run time and flexibility of models dictates approach:

    • Differences between similar model types (rel small)

    • Differences between breach and propagation (significant)

4th IMPACT Workshop - Zaragoza

Selection of case studies l.jpg
Selection of case studies:

  • Very difficult to find extensive, reliable data setsā€¦

    • Ongoing problemā€¦forensics team?

  • Two case studies selected:

    • Tous Dam Failure (breach & propagation)

    • Lake Ha!Ha! (sediments)

4th IMPACT Workshop - Zaragoza

Uncertainty in breach modelling l.jpg
Uncertainty in breach modelling

Basic process:

  • Sensitivity analysis to modelling parameters

  • Selection of top 3-5 parameters; identification of realistic parameter range and distribution

  • Monte Carlo analysis

  • Review and selection of upper, mid & lower hydrographs

  • Modelling undertaken by 3 organisations / 3 different models

  • Selection of overall upper, mid, lower

4th IMPACT Workshop - Zaragoza

Modelling with hr breach l.jpg
Modelling with HR BREACH

  • 1D flow model - piping and overtopping

  • Soil erosion through shear stress / sediment transport; no predefinition of breach growth

  • Integrated soil mechanics for lateral and longitudinal discreet slope failure (undercutting etc)

  • Surface protection simulation

  • Composite structure and associated failure mechanisms

  • Variable sediment equations, adjustable probability distribution for slope failure (uncertainty in soils),

  • Monte Carlo facility

4th IMPACT Workshop - Zaragoza

Sensitivity analysis l.jpg
Sensitivity analysis

4th IMPACT Workshop - Zaragoza

Monte carlo simulations l.jpg

Failure distribution

Monte Carlo simulations

  • 5 parameters varied for MC analysis

  • 1600 MC runs undertaken

    • < than statistical requirement

    • little difference in results from 3 parameter MC analysis

No failure - only overtopping

4th IMPACT Workshop - Zaragoza

All upper mid lower results l.jpg
All upper-mid-lower results

4th IMPACT Workshop - Zaragoza

Modelling with rupro cemagref l.jpg
Modelling with Rupro (Cemagref)

  • Aim: providing an outflow hydrograph from piping or overtopping

  • Simplified assumptions: average breach cross-section, sediment transport using Meyer Peter MĆ¼ller equation with sediment described by only one representative diameter and one friction coefficient

  • Advantages:

    • very rapid calculation, few parameters to test

    • can be integrated in 1-D and 2-D propagation models thus cascade breaching or dike breaching easy to calculate

  • Disadvantages: difficulty to assess parameters in case of complex structure of the dam

4th IMPACT Workshop - Zaragoza

Modelling with rupro cemagref uncertainty calculations l.jpg

Parameter considered

Values of parameter

Peak outflow (m3/s)

Time of peak outflow

Breach bottom elevation (m)

53 / 70

20300 / 15300

21 : 36 / 20 : 53

Breach Strickler friction coefficient

26.5 / 20

17070 / 20300

24 : 19 / 21 : 36

sediment diameter (mm)

250 / 0.08

6196 / 16700

21 : 51 / 25 : 02

Maximum breach width

not limited / 36 m

24924 / 13600

18 : 45 /19 : 33

Modelling with Rupro (Cemagref) Uncertainty calculations

  • Step 1: what are the parameters to which outflow hydrograph is sensitive?

  • Step 2: select the three main parameters and 5 values for each (very low, low, average, high, very high) to which a probability is associated

4th IMPACT Workshop - Zaragoza

Modelling with rupro cemagref uncertainty calculations20 l.jpg
Modelling with Rupro (Cemagref) Uncertainty calculations

  • Step 3: calculation for all combinations (125) and ranking hydrographs by peak outflows to obtain 5, 10, 50, 90 and 95 % occurrences.

4th IMPACT Workshop - Zaragoza

Breach model deich p l.jpg
Breach Model DEICH_P

Breach Formation

4th IMPACT Workshop - Zaragoza

Breach model deich p22 l.jpg
Breach Model DEICH_P

Breach Formation without core

4th IMPACT Workshop - Zaragoza

Breach model deich p23 l.jpg
Breach Model DEICH_P

Breach Formation with core

4th IMPACT Workshop - Zaragoza

Selected upper mid lower results l.jpg
Selected upper-mid-lower results

4th IMPACT Workshop - Zaragoza

Some observations l.jpg
Some observationsā€¦

  • Qp varies between +50% (~22,500m3/s) and -17% (~12,500m3/s)

    • BUT, what donā€™t forget that non failed data has been removed

  • Timing influenced by inflow data, but upper-lower still shows ~2.75hr difference

4th IMPACT Workshop - Zaragoza

Some observations26 l.jpg
Some observationsā€¦

  • How do our best estimate results compare to mid values?

    • Mid ļ‚¹ Best estimate

4th IMPACT Workshop - Zaragoza

Some observations27 l.jpg
Some observationsā€¦

  • Choice of sediment transport equation worth a closer lookā€¦

    • analysis was prompted by a bug in our MC routine - looking for way to model in homogeneous rather than composite form

    • considered 2 sediment transport equations (cohesive / non cohesive); core and outer layer material parameters

    • highlights the importance of selecting the right sediment relationship

    • highlights the effect of assuming a homogeneous bank

4th IMPACT Workshop - Zaragoza

Some conclusions breach uncertainty l.jpg
Some conclusions (breach uncertainty)

  • Uncertainty range for Qp for Tous study was ~ +50% -17%

    • Note - this includes uncertainty from 3 models / modellers. Range from 1 model / modeller will appear less.

  • Modeller best estimate was better than ā€˜midā€™ estimate (for this case).

    • Skill of modeller will play a significant part

    • Routine use of mid value not necessarily the best

4th IMPACT Workshop - Zaragoza

Some conclusions breach uncertainty29 l.jpg
Some conclusions (breach uncertainty)

  • Understanding the sensitivity of models to different parameters is essential

    • different models will include different parameters

    • model sensitivity to parameters will vary from model to model

  • All breach models have one or more parameters relating distribution of shear stress / sediment / erosion / breach growth.

    • This parameter is usually hidden but has a major influence on results

    • Care is required to validate particular parameter values for each application

4th IMPACT Workshop - Zaragoza

Some conclusions breach uncertainty30 l.jpg
Some conclusions (breach uncertainty)

  • Application of Monte Carlo approach is ā€˜betterā€™ than sensitivity analysis, but the process requires a quick running model for practicality

    • Current breach models are on the limit of practicability for MC analysis

  • Current breach models are not well designed to predict the timing of breach initiation

    • current models are very sensitive to boundary conditions (ie defined flow, water level etc) rather than actually predicting the onset of breach

4th IMPACT Workshop - Zaragoza

Some conclusions breach uncertainty31 l.jpg
Some conclusions (breach uncertainty)

  • Breach model results are highly dependent upon choice of sediment transport equation

    • No existing equations are ideal (steady state, long termā€¦)

    • selection of most appropriate equation for case application should be done (look at conditions that equation was developed for)

4th IMPACT Workshop - Zaragoza

Some conclusions breach uncertainty32 l.jpg
Some conclusions (breach uncertainty)

  • Simulation of composite or homogeneous structures and assumptions of averaged soil properties can significantly affect results

    • Variations in Qp of >300% can be seen when different assumptions / sed equations are made (compared to +- 50% for earlier uncertainty range)

    • Therefore use appropriate model (composite or homogeneous) & appropriate equation. Be wary of assuming homogeneous for a composite structure...

4th IMPACT Workshop - Zaragoza

Beyond impact l.jpg
Beyond IMPACTā€¦

  • Awaiting DSIG work with interest

  • FLOODsite - Task 20

    ā€œDevelopment of framework for the influence and impact of uncertaintyā€

    ā€œā€¦develop an approach and prototype software for propagating uncertainty through integrated flood risk modelsā€¦ā€

    • University Bristol (UK) & IHE (Delft) under theme lead from HR - > 60 mm research



4th IMPACT Workshop - Zaragoza