slide1
Download
Skip this Video
Download Presentation
Motivation

Loading in 2 Seconds...

play fullscreen
1 / 15

Motivation - PowerPoint PPT Presentation


  • 255 Views
  • Uploaded on

Recent Reduction in the Area of “EC” in CPC Outlooks and its Relationship to Value Ed O’Lenic, Kenneth Pelman, and David Unger NOAA-NWS-Climate Prediction Center 34th Climate Diagnostics and Prediction Workshop, October 26-30, 2009, Monterey, California. Motivation.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Motivation' - rumor


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1
Recent Reduction in the Area of “EC” in CPC Outlooksand its Relationship to ValueEd O’Lenic, Kenneth Pelman, and David UngerNOAA-NWS-Climate Prediction Center34th Climate Diagnostics and Prediction Workshop, October 26-30, 2009, Monterey, California
motivation
Motivation

This paper updates our efforts to improve the skill and utility of CPC official (OFF) 3-Month Temperature and Precipitation Outlooks.

  • We demonstrate that the skill and value of CPC’s .5-month-lead 3-Month Temperature and Precipitation Outlooks has improved since we began using an objective method to consolidate the forecast tools (CON) in our operations starting in 2006.
  • To do this we first compare the performance of

Official (OFF) outlooks, made in real-time from1995-2004, with that of outlooks made by objective consolidation (CON) retrospectively over the same time period.

skill measures
Skill Measures

We examine measures of performance including:

  • the fractional improvement by the forecast over random forecasts, sn,

2) the frequency (average area), n/T, of non-EC (equal chances) probabilities, and

  • sa , the area-weighted sn.
  • There is evidence of a sustained improvement in these metrics.
  • We also make a case for sa as a measure of value.
s fractional improvement over random forecasts
s =fractional improvement over random forecasts

Scores only non-EC areas

Scores non-EC and EC areas

n/T = frequency non-EC = % coverage

T = # all forecasts

n= # non-EC forecasts

C = # correct forecasts

E = # forecasts, out of T, expected

correct by chance

e= # forecasts, out of n, expected

correct by chance

EC= # EC forecasts

the case for using s a as a metric
The Case for Using sa as a Metric
  • The official metric, sn, rewards larger EC regions, this is not in the interest of users.
  • sa = (n/T) * sn, rewarding larger non-EC areas.
  • Users have made clear that forecasts with fewer EC forecasts have higher overall value.
  • Problem: sa is smaller, on average, than sn.

Premise for this paper: sa is a measure of value

consolidation versus official forecasts over 1995 2004
Consolidation versus Official ForecastsOver 1995-2004
  • We applied the objective consolidation technique, previously used for our official SST forecasts, to CONUS temperature and precipitation.
  • We compared the skill of consolidated forecasts (CON) with official forecasts (OFF) for 30 forecasts in each of 4 seasons from 1995-2004.
  • OFF forecasts were those made in real-time.
  • CON forecasts used the same 4 tools as OFF.
  • Results are reported in O’Lenic et al, 2008, and Unger et al, 2009.
slide8
sa Decomposed into sn (contours), and n/T (colors)

for .5 mo-lead 3-Month Precipitation Forecasts

Change in n/T (CON-minus-OFF) is given in % at bottom.

1995-2004 mean sn*100is at lower right in each figure.

Skill,Official (OFF) .5 mo-lead 3-Month Precipitation Outlook, 1995-2004

FMA, MAM, AMJ (30 fcsts)

ASO, SON, OND (30 fcsts)

19

2

MJJ, JJA, JAS (30 fcsts)

NDJ, DJF, JFM (30 fcsts)

4

FMA, MAM, AMJ (30 fcsts)

4

Skill, Consolidation (CON) .5 mo-lead 3-Month Precipitation Outlook, 1995-2004

+8%

+18%

+20%

+16%

13

ASO, SON, OND (30 fcsts)

17

NDJ, DJF, JFM (30 fcsts)

MJJ, JJA, JAS (30 fcsts)

12

NDJ, DJF, JFM (30 fcsts)

MJJ, JJA, JAS (30 fcsts)

FMA, MAM, AMJ (30 fcsts)

ASO, SON, OND (30 fcsts)

9

FMA, MAM, AMJ (30 fcsts)

0.5 Month Lead 3-Mo Precipitation Outlooks, 1995-2004:

CON Raises U.S. Annual Mean sn from 9 to 12,

and Increases non-EC Forecasts in All Seasons

slide9
sa Decomposed into sn (contours), and n/T (colors)

for 0.5 mo-lead 3-Month Temperature Outlooks

Change in n/T (CON-minus-OFF) is given in % at bottom,

1995-2004 mean sn*100 is at lower right in each figure.

Official (OFF) .5 mo-lead 3-Month Temperature Outlook, 1995-2004

FMA, MAM, AMJ (30 fcsts)

ASO, SON, OND (30 fcsts)

MJJ, JJA, JAS (30 fcsts)

NDJ, DJF, JFM (30 fcsts)

FMA, MAM, AMJ (30 fcsts)

23

22

16

37

Consolidation (CON) .5 mo-lead 3-Month Temperature Outlook, 1995-2004

+40%

+55%

+11%

+31%

ASO, SON, OND (30 fcsts)

NDJ, DJF, JFM (30 fcsts)

MJJ, JJA, JAS (30 fcsts)

25

33

23

33

FMA, MAM, AMJ (30 fcsts)

0.5 Month Lead 3-Mo Temperature Outlooks, 1995-2004:

CON Raises U.S. Annual Mean sn from 18 to 24,

and Increases non-EC Forecasts in All Seasons

slide11
CONUS Spatial Average, 48-Month Running Mean of area covered ,sn, sa, sn-sa 1995-2005 compared with post-2005, Temperature

CONSOLIDATION

IMPLEMENTED

n/T

sd = 34

sd = 38

sn

30

27

Fraction x 100

sa

12

11

sa

sn

sn-sa

n/T

slide12
CONUS Spatial Average, 48-Month Running Mean of area covered ,sn, sa, sn-sa 1995-2005 compared with post-2005, Precipitation

CONSOLIDATION

IMPLEMENTED

n/T

Fraction x 100

sn

sd = 31

sd = 34

sa

2.5

sa

sn

sn-sa

n/T

summary
SUMMARY
  • Retrospective verification shows increase in the skill of objective Consolidation (CON) forecasts compared with Official (OFF), over 1995-2004. CONUS mean sn, n/T rise in all seasons, except winter for both T, P. For temperature, the sa OFF vs CON difference is significant at the 95% level, the sn rise is not. For precipitation, both sn and sa OFF vs CON differences are significant at the 95% level.
  • OFF forecasts made after 2005:

1. sa (Area-weighted sn) is a user-centric skill and value metric.

2. Mean sn ~ same/higher than for 1995-2005, without strong ENSO.

3. Sustained increase in skill, frequency (sa), and therefore, value.

4. The variability of sn drops, another measure of value.

5. Skill of L3MTO has risen as a direct result of use of CON.

6. Better forecasts can be made by focusing on value.

slide14
CONUS Spatial Average, 48-Month Running Mean of area covered ,sn, sa, sn-sa 1995-2005 compared with post-2005, Temperature

sa

sn

sn-sa

n/T

CONSOLIDATION

IMPLEMENTED

  • T, post 2005:
  • N/T +
  • sn rises, stays at levels above those during period including a
  • strong ENSO
  • sa rises from 8 to 12
  • Coverage rises to above strong ENSO levels
  • sd falls 38 to 34
  • Mean sn ~ same as that for 1995-2005, but without strong ENSO

n/T

Fraction x 100

sn

sd = 34

sd = 38

31

27

12

11

sa

slide15
CONUS Spatial Average, 48-Month Running Mean of area covered ,sn, sa, sn-sa1995-2005 compared with post-2005, Precipitation

sa

sn

sn-sa

n/T

CONSOLIDATION

IMPLEMENTED

  • P, post 2005:
  • Coverage rises
  • sn rises to levels
  • comparable to those during strong ENSO
  • sa rises from 0.5 to 2.5
  • Mean sn ~ same as that for 1995-2005, but without strong ENSO
  • sd falls 34 to 31

Fraction x 100

n/T

sn

sd = 34

sd = 31

2.5

sa

ad