Analyzing input and structural uncertainty of a hydrological model with stochastic, time-dependent parameters

Download Presentation

Analyzing input and structural uncertainty of a hydrological model with stochastic, time-dependent parameters

Loading in 2 Seconds...

- 171 Views
- Uploaded on
- Presentation posted in: General

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Peter Reichert

Eawag Dübendorf and ETH Zürich,Switzerland

- Motivation
- Approach
- Implementation
- Application
- Discussion

Motivation

Approach

Implementation

Application

Discussion

Motivation

Approach

Implementation

Application

Discussion

Motivation

Motivation

Approach

Implementation

Application

Discussion

- Environmental modelling is often based on deterministic models that describe substance and organism mass balances in environmental compartments.
- Statistical inference with such models is often based on the assumption that the data is independently and identically distributed around the predictions of the deterministic model at „true“ parameter values.
- The concept underlying this approach is that the deterministic model describes the „true“ system behaviour and the probability distributions centered at the model predictions the measurement process.

- Empirical evidence often demonstrates the invalidity of these statistical assumptions:
- Residuals are often heteroscedastic and autocorrelated.
- The residual error is usually (much) larger than the measurement error.

- This leads to incorrect results of statistical inference. In particular, parameter and model output uncertainty are usually underestimated.
- These obviously wrong results lead to abandoning of the statistical approach and to the development of conceptually poorer techniques in applied sciences.
- We are interested in a statistically satisfying approach to this problem.

Motivation

Approach

Implementation

Application

Discussion

Suggested solution (Kennedy and O‘Hagan, 2001, and many earlier, more case-specific approaches):

Extend the model by a discrepancy or bias term.

Replace:by:where yM = deterministic model, x = model inputs, q = model parameters, Ey = observation error, B = bias or model discrepancy, YM = random variable representing model results.

Motivation

Approach

Implementation

Application

Discussion

The bias term is usually formulated as a non-parametric statistical description of the model deficits (typically as a Gaussian stochastic process).

Advantage of this approach: Statistical description of model discrepancy improves uncertainty analysis.

Disadvantage: Lack of understanding of the cause of the discrepancy makes it still difficult to extrapolate.

Motivation

Approach

Implementation

Application

Discussion

We are interested in a technique that supports identification of the causes and reduction of these discrepancies.

There are three generic causes of failure of the description of nature with a deterministic model plus measurement error:

Motivation

Approach

Implementation

Application

Discussion

- Errors in deterministic model structure.
- Errors in model input.
- Inadequateness of a deterministic description of systems that contain intrinsic non-deterministic behaviour due to
- influence factors not considered in the model,
- model simplifications (e.g. aggregation, adaptation, etc.),
- chaotic behaviour not represented by the model.

Because of these deficits we cannot expect a deterministic model to describe nature appropriately.

Motivation

Approach

Implementation

Application

Discussion

Pathway for improving models:

- Reduce errors in deterministic model structure to improve average behaviour.
- Add adequate stochasticity to the model structure to account for random influences.

This requires the combination of statistical analyses with scientific judgment.

This talk is about support of this process by statistical techniques.

Motivation

Approach

Implementation

Application

Discussion

Approach

Questions:

- How to make a deterministic, continuous-time model stochastic?
- How to distinguish between deterministic and stochastic model deficits?

Motivation

Approach

Implementation

Application

Discussion

- Replacement of differential equations (representing conservation laws) by stochastic differential equations can violate conservation laws and does not address the cause of stochasticity directly.
- It seems to be conceptually more satisfying to replace model parameters (such as rate coefficients, etc.) by sto-chastic processes, as stochastic external influence factors usually affect rates and fluxes rather than states directly. The model consists then of an extended set of stochastic differential equations of which some have zero noise.

Motivation

Approach

Implementation

Application

Discussion

Note that the basic idea of this approach is very old.

The original formulation was, however, limited to linear or weakly nonlinear, discrete-time systems with slowly varying driving forces (e.g. Beck 1987).

The bias term approach is a special case of our approach that consists of an additive output parameter.

Motivation

Approach

Implementation

Application

Discussion

Our suggestion is to

- extend this original approach to continuous-time and nonlinear models;
- allow for rapidly varying external forces;
- embed the procedure into an extended concept of statistical „bias-modelling“ techniques.

Motivation

Approach

Implementation

Application

Discussion

Implementation

Deterministc model:

Motivation

Approach

Implementation

Application

Discussion

Consideration of observation error:

Model with parameter i time-dependent:

Motivation

Approach

Implementation

Application

Discussion

The time dependent parameter is modelled by a mean-reverting Ornstein Uhlenbeck process:

Motivation

Approach

Implementation

Application

Discussion

This has the advantage that we can use the analytical solution:

or, after reparameterization:

We combine the estimation of

- constant model parameters, , with
- state estimation of the time-dependent parameter(s), , and with
- the estimation of (some of the) (constant) parameters of the Ornstein-Uhlenbeck process of the time dependent parameter(s), .

Motivation

Approach

Implementation

Application

Discussion

Gibbs sampling for the three different types of parameters. Conditional distributions:

Motivation

Approach

Implementation

Application

Discussion

simulation model (expensive)

Ornstein-Uhlenbeck process (cheap)

Ornstein-Uhlenbeck process (cheap)

simulation model (expensive)

Tomassini et al. 2007

Metropolis-Hastings sampling for each type of parameter:

Motivation

Approach

Implementation

Application

Discussion

Multivariate normal jump distributions for the parameters qM and qP. This requires one simulation to be performed per suggested new value of qM.

The discretized Ornstein-Uhlenbeck parameter, , is split into subintervals for which OU-process realizations conditional on initial and end points are sampled. This requires the number of subintervals simulations per complete new time series of .

Tomassini et al. 2007

Motivation

Approach

Implementation

Application

Discussion

Application

Simple Hydrological Watershed Model (1):

Motivation

Approach

Implementation

Application

Discussion

Kuczera et al. 2006

3

A

4

6

5

1

B

2

C

7

8

Simple Hydrological Watershed Model (2):

Motivation

Approach

Implementation

Application

Discussion

8 model parameters

3 initial conditions

1 standard dev. of obs. err.

3 „modification parameters“

Kuczera et al. 2006

Simple Hydrological Watershed Model (3):

Motivation

Approach

Implementation

Application

Discussion

- Data set of Abercrombie watershed, New South Wales, Australia (2770 km2), kindly provided by George Kuczera (Kuczera et al. 2006).
- Box-Cox transformation applied to model and data to decrease heteroscedasticity of residuals.
- Step function input to account for input data in the form of daily sums of precipitation and potential evapotranspiration.
- Daily averaged output to account for output data in the form of daily averaged discharge.

Motivation

Approach

Implementation

Application

Discussion

- Estimation of 11 model parameters: 8 rate parameters 3 initial conditions 1 measurement standard deviation
- Priors: Independent lognormal distributions for all parameters with the exception of the measurement standard deviation (1/s).
- Modification factors (frain, fpet, fQ) kept equal to unity.

Motivation

Approach

Implementation

Application

Discussion

Motivation

Approach

Implementation

Application

Discussion

Motivation

Approach

Implementation

Application

Discussion

Motivation

Approach

Implementation

Application

Discussion

Motivation

Approach

Implementation

Application

Discussion

The results show the typical deficiencies of deterministic models:

- Residuals are heteroscedastic and autocorrelated.
- The standard deviation of the residuals is larger than the measurement error (increasing from 0.24 m3/s at a discharge of zero to 30 m3/s at 100 m3/s).
- Model predictions are overconfident.
In addition: ground water level trend seems unrealistic.

Motivation

Approach

Implementation

Application

Discussion

Step 1: Estimation of time-dependent parameters

- Estimation of 11 time-dependent parameters: 8 rate parameters 3 modification factors (frain, fpet, fQ)
- Ornstein-Uhlenbeck process applied to the log of each parameter sequentially. Hyperparameters: t =1d, s =0.2 (22%) fixed, only estimation of initial value and mean (0 for log frain, fpet, fQ).
- Constant parameters as before.

Motivation

Approach

Implementation

Application

Discussion

Step 2: Analyzing Degree of Bias Reduction

- As quality of fit is insufficient (residual standard deviation larger than measurement error), quality of fit is a primary indicator of bias (when being careful with regard to overfitting).
- Reduction of autocorrelation can be checked as a secondary criterion (it is likely to be accompanied by reduction of residual standard deviation).

Motivation

Approach

Implementation

Application

Discussion

Improvement of fit:

Nash-Sutcliffe indices:

frain0.90

ks0.84

fQ0.67

sF0.63

fpet0.60

kr0.57

ket0.54

qlat,max0.54

kdp0.53

kgw,max0.52

kbf0.52

base0.51

Motivation

Approach

Implementation

Application

Discussion

Assessment:

- Input (frain) and output (fQ) modifications.
- Potential for soil / runoff model (ks, SF) improvements.
- Some potential for river and evaporation improvements.

Random or deterministic?

Step 3: Identification of Potential Dependences

- Despite doing an exploratory analysis of the values of time dependent parameters on all model states and inputs, no significant dependences could be found.
- This is an indication that it may be difficult to improve the deterministic model, or that the improvement will be restricted to a small number of data points.

Motivation

Approach

Implementation

Application

Discussion

Step 4: Improvement of Deterministic Model :

Motivation

Approach

Implementation

Application

Discussion

Extension 1: Modification of runoff flux:

Extension 2: Modification of sat. area funct.:

Extentsion 1 has two, extension 2 three additional model parameters.

Model Extensions:

Motivation

Approach

Implementation

Application

Discussion

Previous results:

Nash-Sutcliffe indices:

frain0.90

ks0.84

fQ0.67

sF0.63

fpet0.60

kr0.57

ket0.54

qlat,max0.54

kdp0.53

kgw,max0.52

kbf0.52

base0.51

Motivation

Approach

Implementation

Application

Discussion

Extended models:

Nash-Sutcliffe indices:

ext. 10.73

ext. 20.51

Original Model:

Motivation

Approach

Implementation

Application

Discussion

Modified Model:

Original Model:

Motivation

Approach

Implementation

Application

Discussion

Modified Model:

Original Model:

Motivation

Approach

Implementation

Application

Discussion

Modified Model:

Conclusions of Step 4

- The significant increase in the Nash-Sutcliffe index is caused by the elimination of a small number of outliers.
- All other deficiencies remain.
- This is the reason why the improvement could not have been detected in the exploratory analysis.
- It seems questionable that the remaining deficiencies could be significantly reduced by improvements of the deterministic model.

Motivation

Approach

Implementation

Application

Discussion

Step 5: Addition of Stochasticity to the Model

Major sources of indeterminism:

- Spatial aggregation: Aggregation of distributed reservoirs in a much smaller number of reservoirs in the model leads to the same model results for different „states of nature“ (that lead to different results in nature).
- Rainfall uncertainty:Spatial heterogeneity of rainfall intensity is not well captured by point rainfall measurements.

Motivation

Approach

Implementation

Application

Discussion

It seems reasonable to summarize these sources of indeterminism by a stochastic rain modification factor frain.

To quantify input uncertainty (combined with aggregation error) we need an informative prior for the measurement error.

We choose sQ,trans ~ N(0.5,0.05).0.5 corresponds to a standard deviation in original units increasing from 0.1 m3/s at a discharge of zero to 12.6 m3/s at a discharge of 100 m3/s.

The standard deviation of the Ornstein-Uhlenbeck process for log frain is now estimated from the data.

Motivation

Approach

Implementation

Application

Discussion

Time-dependent parameter frain:

Motivation

Approach

Implementation

Application

Discussion

Motivation

Approach

Implementation

Application

Discussion

Original Model:

Motivation

Approach

Implementation

Application

Discussion

Modified Model with Time-Dependent Parameter frain:

Original Model:

Motivation

Approach

Implementation

Application

Discussion

Modified Model with Time-Dependent Parameter frain:

Original Model:

Motivation

Approach

Implementation

Application

Discussion

Modified Model with Time-Dependent Parameter frain:

Motivation

Approach

Implementation

Application

Discussion

Discussion

- The suggested procedure seems to fulfil the expectations of supporting the identification of model deficits and of introducing stochasticity into a deterministic model.
- It is related to and can be viewed as a generalization of previous work on
- Time-dependent parameters using Kalman filtering (e.g. Beck and Young 1976, etc.)
- Modelling of bias of deterministic models(Craig et al. 1996, Kennedy and O‘Hagan 2001, Bayarri et al. 2005, etc.)
- Rainfall multipliers(Kuczera 1990, Kavetski et al. 2001, etc.)

Motivation

Approach

Implementation

Application

Discussion

- There is need for future research in the following areas:
- Explore alternative ways of learning from the identified parameter time series.
- Different formulation of time-dependent parameters (for some applications smoother behaviour).
- Include multiple time-dependent parameters into the analysis.
- Use a more specific model to represent input uncertainty.
- Improve efficiency (linearization, emulation).
- Learn from more applications.

Motivation

Approach

Implementation

Application

Discussion

- Collaboration for this paper:Johanna Mieleitner
- Development of the technique:Hans-Rudolf Künsch, Roland Brun, Christoph Buser , Lorenzo Tomassini, Mark Borsuk.
- Hydrological example and data:George Kuczera.
- Interactions at SAMSI:Susie Bayarri, Tom Santner, Gentry White, Ariel Cintron, Fei Liu, Rui Paulo, Robert Wolpert, John Paul Gosling, Tony O‘Hagan, Bruce Pitman, Jim Berger, and many more.

Motivation

Approach

Implementation

Application

Discussion