Using fluxnet data to evaluate land surface models
Download
1 / 22

Using FLUXNET data to evaluate land surface models - PowerPoint PPT Presentation


  • 77 Views
  • Uploaded on

Using FLUXNET data to evaluate land surface models. Ray Leuning and Gab Abramowitz 4 – 6 June 2008. Land surface model evaluation framework. Reto Stockli’s ‘Model farm’. Schematic diagram of model components from a systems perspective. system boundary, B inputs, u initial states, x 0

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' Using FLUXNET data to evaluate land surface models' - sheba


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Using fluxnet data to evaluate land surface models

Using FLUXNET data to evaluate land surface models

Ray Leuning and Gab Abramowitz

4 – 6 June 2008


Land surface model evaluation framework
Land surface model evaluation framework

Reto Stockli’s ‘Model farm’

CSIRO.Using FLUXNET data to evaluate land surface models


Schematic diagram of model components from a systems perspective
Schematic diagram of model components from asystems perspective

  • system boundary, B

  • inputs, u

  • initial states, x0

  • parameters, θ

  • model structure, M

  • model states, x

  • outputs, y

Errors in each component affects model performance

Liu, Y. Q. and Gupta, H. V. (2007). Uncertainty in Hydrologic Modeling: Toward an Integrated Data Assimilation Framework. Water Resources Research 43, W07401, doi:10.1029/2006/WR005756.

CSIRO.Using FLUXNET data to evaluate land surface models


Parameter estimation multiple objective functions possible
Parameter estimation Multiple objective functions possible

CSIRO.Using FLUXNET data to evaluate land surface models


Parameter estimation multiple criteria possible e g e nee
Parameter estimation Multiple criteria possible, e.g. λE, NEE

The dark line between the two criteria’s minima, α and β, represents the Pareto set

CSIRO.Using FLUXNET data to evaluate land surface models


Comparing rmse of models of varying complexity across sites after parameter optimization
Comparing RMSE of models of varying complexity across sites after parameter optimization

λE

Sites

Ideal result (0,0)

Models

H

Hogue, T. S., Bastidas, L. A., Gupta, H. V., and Sorooshian, S. (2006). Evaluating Model Performance and Parameter Behavior for Varying Levels of Land Surface Model Complexity. Water Resources Research 42, W08430, doi:10.1029/2005WR004440.

CSIRO.Using FLUXNET data to evaluate land surface models


Solo neural network cluster analysis
SOLO neural network - cluster analysis

Abramowitz, G., Gupta, H., Pitman, A., Wang, Y.P., Leuning, R. and Cleugh, H.A. (2006). Neural Error Regression Diagnosis (NERD): A tool for model bias identification and prognostic data assimilation. Journal of Hydrometeorology, 7:160-177.

CSIRO.Using FLUXNET data to evaluate land surface models


Poor model performance not just due to poor parameter estimation

observed

cable

solo

Poor model performance not just due to poor parameter estimation

CABLE with 4 different parameter sets

SOLO – cluster analysis

CSIRO.Using FLUXNET data to evaluate land surface models


No model or single performance measure is best for all fluxes
No model or single performance measure is best for all fluxes

CABLE, ORCHIDEE, CLM,

MLR multiple linear regression,

ANN artificial neural network

CSIRO.Using FLUXNET data to evaluate land surface models


Model comparisons average seasonal cycle
Model comparisons - average seasonal cycle fluxes

NEE

λE

Global default parameters for each PFT used

H

CSIRO.Using FLUXNET data to evaluate land surface models


Model comparisons average daily cycle
Model comparisons - average daily cycle fluxes

NEE

λE

Global default parameters for each PFT used

H

CSIRO.Using FLUXNET data to evaluate land surface models


Pdf s for nee e h across 6 sites
PDF’s for NEE, fluxesλE & H across 6 sites

CSIRO.Using FLUXNET data to evaluate land surface models


Nee perturbed parameter ensemble simulations
NEE Perturbed-parameter ensemble simulations fluxes

Average diurnal cycle

Monthly averages

CSIRO.Using FLUXNET data to evaluate land surface models


E perturbed parameter ensemble simulations
λ fluxesE Perturbed-parameter ensemble simulations

Average diurnal cycle

Monthly averages

CSIRO.Using FLUXNET data to evaluate land surface models


H perturbed parameter ensemble simulations
H Perturbed-parameter ensemble simulations fluxes

Average diurnal cycle

Monthly averages

CSIRO.Using FLUXNET data to evaluate land surface models


Partitioning climate space into 9 som nodes
Partitioning climate space into 9 SOM nodes fluxes

S↓ Tair qair

S↓ Tair qair

S↓ Tair qair

night

night

night

CSIRO.Using FLUXNET data to evaluate land surface models


Nee pdfs at nodes 7 9 at tumbarumba
NEE PDFs at nodes 7 -9 at Tumbarumba fluxes

S↓ Tair qair

S↓ Tair qair

S↓ Tair qair

9

8

7

night

CSIRO.Using FLUXNET data to evaluate land surface models


Suggested set of discussion topics
Suggested set of discussion topics fluxes

  • Primary objectives

  • Establish a framework that provides standardised data sets and an agreed set of analytical tools for LSM evaluation

    • Analytical tools should provide a wide range of diagnostic information about LSM performance

    • Datasets specifically formatted for LSM execution and evaluation

  • Specific objectives

  • To detect and eliminate systematic biases in several LSMs in current use

  • To obtain optimal parameter values for LSMs after biases have been diminished or eliminated

  • To evaluate the correlation between key model parameters and bioclimatic space

CSIRO.Using FLUXNET data to evaluate land surface models


Tasks for meeting 1
Tasks for meeting 1 fluxes

  • Discuss what form the LSM evaluation framework should take

    • PILPS style?

    • What will be asked of data providers?

    • What will be asked of LS modellers?

  • Agree on a minimal set of LSM flux performance measures (model vs observations vs benchmark):

    • Average diurnal cycle?

    • Average annual cycle (monthly means)?

    • Some type of frequency analysis (wavelet, power spectrum etc)?

    • Conditional analysis (SOM node analysis):

      • Overlap of pdfs

      • Multiple criteria cost function set (mean, rmse, rsq, regression gradient and intercept)

  • Discuss other LSM outputs and datasets useful for process evaluation

  • Discuss ways to include parameter uncertainty in LSM evaluation (c.f. Abramowitz et al., 2008)

CSIRO.Using FLUXNET data to evaluate land surface models


Tasks for meeting 2
Tasks for meeting 2 fluxes

  • Discuss options for the most effective way to provide these services

    • Will individual groups do benchmarking, evaluation of model states?

    • Preference for an automated web-based interface and data server

      • Automatic processing through a website?

      • Abramowitz suggests automation of basic LSM performance measure plots, including benchmarking (as in Abramowitz, 2005).

      • Uploaded output from LSM runs in ALMA format netcdf could return standard plots to the user and/or post on website.

    • Model detective work and improvement to be done by individual groups

CSIRO.Using FLUXNET data to evaluate land surface models


Data analysis will use
Data analysis will use: fluxes

  • Several current LSMs

  • Quality controlled Fluxnet datasets

  • SOFM (Self-organizing feature maps) analysis

    • to classify bioclimatic data into n2 nodes

    • to evaluate model biases for each node to help the ‘detective work’ of identifying areas of model weaknesses

    • to identify upper-boundary surfaces for stocks of C and N and P in global ecosystems as a function of the n2 climate nodes

  • Benchmarking

    • to compare model predictions at each climate node against multiple linear regression (MLR) estimates

CSIRO.Using FLUXNET data to evaluate land surface models


Tools currently available from abramowitz
Tools currently available from Abramowitz fluxes

  • SOLO (SOFM + MLR) software (Fortran)

  • LSMs ‘Model Farm’ of Reto Stöckli plus CABLE

  • CSV to ALMA netcdf conversion routine (Fortran)

  • Plotting routines in R

  • Fluxnet database in CSV and netcdf formats

CSIRO.Using FLUXNET data to evaluate land surface models


ad