Climate Test Bed Seminar Series
This presentation is the property of its rightful owner.
Sponsored Links
1 / 47

Climate Test Bed Seminar Series 24 June 2009 PowerPoint PPT Presentation


  • 51 Views
  • Uploaded on
  • Presentation posted in: General

Climate Test Bed Seminar Series 24 June 2009. Bias Correction & Forecast Skill of NCEP GFS Ensemble Week 1 & Week 2 Precipitation & Soil Moisture Forecasts. Yun Fan & Huug van den Dool. Acknowledgement: Jae Schemm, John Janowiak, Doug Lecomte , Jin Huang , Pingping Xie,

Download Presentation

Climate Test Bed Seminar Series 24 June 2009

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Climate test bed seminar series 24 june 2009

Climate Test Bed Seminar Series

24 June 2009

Bias Correction & Forecast Skillof NCEP GFS Ensemble Week 1 & Week 2Precipitation & Soil Moisture Forecasts

Yun Fan & Huug van den Dool

Acknowledgement: Jae Schemm, John Janowiak, Doug Lecomte, Jin Huang, Pingping Xie,

Viviane Silva, Peitao Peng, Vern Kousky, Wayne Higgins


Outline

Outline

  • Motivation

  • Methodology

  • Performance of NCEP GFS Week1 & Week2 Ensemble Precipitation Forecasts

  • Analysis of Week1 & Week2 Biases & Errors

  • Application: land model forced with bias corrected week1 week2 P & T2m forecast

  • Future Work


Climate test bed seminar series 24 june 2009

History of Soil Moisture “Dynamical” OutlookCPC Leaky Bucket Hydrological ModelForced With Week 1 & Week 2 GFS Forecasts

Single member HR MRF (started around 1997 & CONUS)

Ensemble GFS (started late 2001 & CONUS)

Bias corrected Ensemble GFS (started late 2003 & CONUS)

Bias corrected Ensemble GFS (started late 2007 & global land)

:The prediction skill of soil moisture crucially depends on our ability to predict precipitation

Early stage (both good and bad comments) 

Recent years (more & more good comments)

So its time to verify & quantify:

daily GFS ensemble week 1 & week 2 precip forecast skills & statistics


Climate test bed seminar series 24 june 2009

The quality of soil moisture prediction largely or almost entirely depends on the quality of precipitation prediction


Climate test bed seminar series 24 june 2009

Daily bias correction based on

last 30 (or 7) day forecast errors

Today

Last 30 day

Week1

Week2

Past

Future

1/N Σ [ Pf (week1) – Po (week1) ] = Bias1

1/N Σ [ Pf (week2) – Po (week2) ] = Bias2

Pf : GFS ensemble week1 & week2 precip forecast

Po: Observed week1 & week2 precip from CPC daily global Unified Precip

N = ( 30, 7..….)


Climate test bed seminar series 24 june 2009

North America

Seasonal cycle with

Large day to day fluctuation

On 0.5x0.5 obs grid


Climate test bed seminar series 24 june 2009

South America

On 0.5x0.5 obs grid


Climate test bed seminar series 24 june 2009

Asia-Australia

On 0.5x0.5 obs grid


Climate test bed seminar series 24 june 2009

Africa

On 0.5x0.5 obs grid


Climate test bed seminar series 24 june 2009

How good is GFS?

Seasonal cycle with

Large day to day fluctuation

On 0.5x0.5 obs grid


Climate test bed seminar series 24 june 2009

How good is GFS?

Seasonal cycle with

Large day to day fluctuation

On 0.5x0.5 obs grid


Climate test bed seminar series 24 june 2009

Comparison(based on last 30-day forecast errors)Obs grids(regrid model grids to 0.5x0.5 obs grids) Model grids (regrid obs grids to 2.5x2.5 model grids) Question: Does grid matter for skills assessment?


Climate test bed seminar series 24 june 2009

Skill does not depend much on the grid


Climate test bed seminar series 24 june 2009

Today

Comparisonbias corrected skills(based on last 30-day forecast errors) bias corrected skills (based on last 7-day forecast errors) Question: Does the bias estimate influence skill?

Last 30 day

Week1

Week2

Past

Future

1/N Σ [ Pf (week1) – Po (week1) ] = Bias1

1/N Σ [ Pf (week2) – Po (week2) ] = Bias2

Pf : GFS ensemble week1 & week2 precip forecast

Po: Observed week1 & week2 precip from CPC daily global Unified Precip

N = ( 30, 7..….)


Climate test bed seminar series 24 june 2009

1) Skill depends on the definition of bias

2) 30-day bias correction better than 7-day bias correction


Climate test bed seminar series 24 june 2009

Comparisonbias corrected skills(based on last 30-day forecast errors) raw forecast skills (no bias correction applied)Question: Does bias correction improve skill in terms ofSpatial CorrelationandRMSE?


Climate test bed seminar series 24 june 2009

Bias correction is time &

location dependent


Climate test bed seminar series 24 june 2009

Bias correction helps everywhere


Climate test bed seminar series 24 june 2009

Table 1. Averaged (May 1, 2008 – June 7, 2009) spatial

correlations over different monsoon regions

Increased by 80%

The effectiveness of bias correction is mainly space dependent.

Bias correction can correct spatial distribution of Pf& reduce its error.

Increased by 67%

Table 2. Averaged (May 1, 2008 – June 7, 2009) RMSE over different monsoon regions (unit: mm/week)

Similarity of Pf & Po

Reduced by 23%

Reduced by 28%

Distance of Pf & Po


Climate test bed seminar series 24 june 2009

30-day running mean

CONUS


Climate test bed seminar series 24 june 2009

In terms of Spatial Anomaly Correlation, bias correction helps:

1) very little over North America

2) considerably over South America & Africa

3) a little over Asia-Australia

In terms of RMSE:

Bias correction helps everywhere

Questions:

Why bias correction works but varies in space and time?

What biases look like?

Are biases removable & to what extent are they removable?


Climate test bed seminar series 24 june 2009

Temporal-spatial structures of last 30-daybiases:Daily Bias1 & Bias2 used to correct GFS ensemble week1 & week2 forecasts

Today

Last 30 day

Week1

Week2

Past

Future

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1

1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2


Climate test bed seminar series 24 june 2009

Annual Mean Bias or Raw Forecast Error

Week-1 mean Bias

Week-2 mean Bias


Climate test bed seminar series 24 june 2009

Mean Bias of Daily R2 & Observed Precip (1979-2006)


Climate test bed seminar series 24 june 2009

summer

winter


Climate test bed seminar series 24 june 2009

winter

summer


Climate test bed seminar series 24 june 2009

Temporal-spatial structures of last 30-daybiases: Daily Bias1 & Bias2 used to correct GFS ensemble week1 & week2 forecasts

Today

Last 30 day

Week1

Week2

Past

Future

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1

1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2

Large-scale & low-frequency (annual or semi-annual cycles) are prominent

First two EOF modes of Bias1 & Bias2 explain about 60% total variances

GFS has prominent annual cycle errors (lesson for model development?)


Climate test bed seminar series 24 june 2009

Temporal-spatial structures of real timeraw forecast errors:DailyGFS week1 & week2 forecast errorswithout bias correction

Today

Last 30 day

Week1

Week2

Past

Future

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1

1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2

No bias correction applied

Pf (week1) – Po (week1) = Error1

Pf (week2) –Po (week2) = Error2


Climate test bed seminar series 24 june 2009

Temporal-spatial structures of real time raw forecast errors:DailyGFS week1 & week2 forecast errorswithout bias correction

Today

Last 30 day

Week1

Week2

Past

Future

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1

1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2

Pf (week1) – Po (week1) = Error1

Pf (week2) –Po (week2) = Error2

No bias correction applied

Raw forecast errors are dominated by the 1st, 2nd or 3rd EOFs in Bias1 & Bias2

First two EOF modes of Error1 & Error2 explain about 23~35% total variances

At least this amount of error is removable. But so far bias correction was not done by EOF analysis


Climate test bed seminar series 24 june 2009

Temporal-spatial structures of real timeforecast errors:GFS week1 & week2 forecast errorswith last 30-day bias correction

Today

Last 30 day

Week1

Week2

Past

Future

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1

1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2

Bias correction:

Error1= Error1 – Bias1

Error2= Error2 – Bias2

Pf (week1) – Po (week1)= Error1

Pf (week2) –Po (week2)= Error2


Climate test bed seminar series 24 june 2009

Annual Mean ForecastError after bias correction

5 times smaller than mean bias or raw forecast error

Week-1 mean forecast error

Week-2 mean forecast error


Climate test bed seminar series 24 june 2009

Temporal-spatial structures of real timeforecast errors:GFS week1 & week2 forecast errorswith last 30-day bias correction

Today

Last 30 day

Week1

Week2

Past

Future

1/30 Σ [ Pf (week1) – Po (week1) ] = Bias1

1/30 Σ [ Pf (week2) – Po (week2) ] = Bias2

Bias correction:

Error1= Error1 – Bias1

Error2= Error2 – Bias2

Pf (week1) – Po (week1)= Error1

Pf (week2) –Po (week2)= Error2

Bias Corrected ForecastErrors are much more random (in time mainly, EOFs more “white”).

Leading EOF modes of Bias1, Bias2, & Error1, Error2 Show that GFS has prominent large-scale & low-frequency errors or GFS has difficulty to reproduce those observed Precip patterns & their evolution. However, to some extent they can be corrected through bias correction, especially in winter season.


Climate test bed seminar series 24 june 2009

Application

Soil Moisture “Dynamical” Outlook

CPC Leaky Bucket Hydrological ModelForced With Week-1 & Week-2 GFS EnsembleForecasts

(Daily data from 01Nov2003 to present)

All initial conditions & verification datasets are from leaky

bucket model forced with daily observed P & T2m


Climate test bed seminar series 24 june 2009

Some Thoughts:

  • Once this (SST, w) was the lower boundary….

  • Both SST and w have (high) persistence

  • Old ‘standard’ in meteorology: If you cannot beat persistence …..

  • For instance: dw/dt = P – E - R = F

    or w(t+1)=w(t) + F

  • Clearly if we do not know F with sufficient skill, the forecast loses against persistence (F=0).


Climate test bed seminar series 24 june 2009

30-day running mean

P1=0.9511, C1=0.9512

PR1=16.27, FR1=18.02

P2=0.9015, C2=0.8957

PR2=23.67, FR2=26.56


Climate test bed seminar series 24 june 2009

Precip

30-day running mean


Climate test bed seminar series 24 june 2009

Even moderate forecast skill at right time still help a lot


Climate test bed seminar series 24 june 2009

30-day running mean for week-2

Hybrid persistence = week-1 forecast persists to week-2


Climate test bed seminar series 24 june 2009

Summary

  • Moderate week-1 & week-2 GFS P forecast skills

  • Last 30-d biases dominated by low-frequency & large-scale errors

  • Bias corrections are time & location dependent

  • Soil moisture forecast skill hardly beats its persistence over CONUS

  • The inability to outperform persistence relates to the skill of precipitation not being above a threshold (AC>0.5 is required)


Climate test bed seminar series 24 june 2009

Future Work

  • Is PDF bias correction better?

  • GFS Week3 & Week4 Precip Assessment

  • GFS hindcasts?

  • How about New CFSRR?


  • Login