slide1 n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Probabilistic Forecast PowerPoint Presentation
Download Presentation
Probabilistic Forecast

Loading in 2 Seconds...

play fullscreen
1 / 66

Probabilistic Forecast - PowerPoint PPT Presentation


  • 311 Views
  • Uploaded on

Probabilistic Forecast. Kiyoharu Takano Climate Prediction Division JMA. 3-month(Dec.-Feb.) forecast issued by JMA at 25,Nov. Contents (1) Predictability of Seasonal Forecast (2) Example of Probability Forecast (3) Quality of Probability Forecast Reliability Resolution

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Probabilistic Forecast' - aurek


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
slide1

Probabilistic Forecast

Kiyoharu Takano

Climate Prediction Division

JMA

slide2

3-month(Dec.-Feb.) forecast

issued by JMA at 25,Nov.

slide3

Contents

(1) Predictability of Seasonal Forecast

(2) Example of Probability Forecast

(3) Quality of Probability Forecast

Reliability

Resolution

(4) Economical Benefit of Forecast

Deterministic forecast

Probabilistic forecast

slide4

(1)Predictability of Seasonal Forecast

cf. Dr. Sugi’s presentation

< Predictability of 1st kind >Originates from Initial conditionDeterministic forecast fails beyond a few weeks due to the growth of errors contained in the initial states (Lorenz, 1963; 1965). < Predictability of 2nd kind > Lorenz (1975) Originates from boundary condition

Effective for longer time scale; Month to season

There remains internal variability which is not controlled by boundary conditions

slide5

Growth of forecast error(Predictability of 1st kind)

1. Error growth due to imperfectness of numerical prediction

⇒Improvement of numerical model

2. Growth of initial condition error

 ⇒ Improvement of objective analysis

However………...

(1) There remains finite (non zero) error in initial condition,

(although it will be reduced as improvement of objective analysis).

We cannot know ‘true’ initial condition

(2)Small initial error(difference) grows fast(exponentially) as time progresses and the magnitude of error becomes the same order as natural variability after a certain time. ⇒Chaos

Deterministic forecast becomes meaningless after sufficiently long time

slide6

Chaos

Lorenz,E. N. ,1963, J.A.S. 20, 130-

Equations for simplified role type convection.                               ‘The Lorenz system’

X: Fourier component of stream function

Y,Z:Fourier component of temperature

Nonlinear terms :XZ,XY

r:Stability parameter

slide7

X

Time

Z

Y

X

  • Findings of Lorenz
  • ‘The solution (x(t),y(t),z(t))
  • behaves strangely’
  • The solution (x(t),y(t),z(t))
  • is bound.
  • Non-periodic
  • Slight initial difference causes
  • large difference. Chaos

Observation

Numerical prediction

Time steps (x0.01))

slide8

What can we do on the predictability limit?

Predictability limit varies depending on flow pattern.

slide9

0:100

slow initial errors growth

fast initial errors growth

70:30

50:50

  • What can we do on the predictability limit?
  • Predictability limit varies depending on atmospheric flow pattern and initial condition
  • Prediction of uncertaintyof forecast are important

warm

cold

slide10

The Ensemble Prediction

7/24

a forecast +prediction of uncertainty of forecast

Temperature anomalies at 850hPa over the Northern Japan

(7day running mean)

6/ 5

Initial:5 June 2003

Initial:24 July 2003

slide11

Longer than one month forecast

Seasonal forecast

One-month forecast

One-week forecast

Longer than one month forecast based on initial condition is impossible at least generally!

slide12

Predictability of second kind

In seasonal time scale, forecast based on initial condition is impossible at least generally

The forecast is based on the influences of boundary conditions such as SSTs or soil wetness.

Prediction of second kind

However ………..

Regression OLR map with Nino3 SSTs

(DJF)

slide13

Atmospheric variation is not fully controlled by variation of boundary condition such as SSTs but there are internal variation..

Examples of internal variations are baroclinic instability , typhoon, Madden-Julian oscillation e.t.c. and these can be predicted as initial value problem in short time scale but they are unpredictable in seasonal time scale.

Then a variation X is writtenas

X=Xext+Xin

Xext : variation controlled by boundary

conditions(Signal)

Xin : internal variation(Noise)

(cf. Mr. Sugi’s presentation)

slide14

30day mean 850hPa Temperature prediction at the Nansei Islands

Noise

Signal

Individual numerical prediction

Average of individual predictions (ensemble mean)

Observation

slide15

Reduction of noise

Since the internal variation can be reduced by time mean but the signal is not reduced, Time-mean is taken in seasonal forecast.

Unpredictable

reduced by time mean

predictable

This time mean is effective especially in tropics.

In addition, main SST signals such as ENSO are in tropics.

Seasonal forecast in tropics is easier than in mid-latitude.

slide16

The noise reduction effect of time mean

5day mean

Individual numerical

prediction

Average of individual

predictions (ensemble mean)

Observation

30day mean

90day mean

slide17

Though time mean is effective to reduce noise,

the noise is not removed completely.

Therefore there remains uncertainty from internal variation

and again probabilistic forecast is necessary.

Individual numerical

prediction

Average of individual

predictions (ensemble mean)

Observation

90day mean

slide18

(2) Example of probabilistic forecast

(a) One-month

forecast

NJ

WJ

Temperature at 850hPa

EJ

Nansei

Surface Temperature Forecast

1 month 1st week 2nd week 3rd-4th week

Forecast Period11.22-12.21 11.22-11.28 11.29-12.5 12.6-12.19

category - 0 + - 0 + - 0 + - 0 +

Northern Japan 20 40 40 20 40 40 20 50 30 20 40 40

Eastern Japan 20 30 50 20 30 50 20 40 40 20 40 40

Western Japan 10 40 50 10 40 50 20 30 50 20 40 40

Nansei Islands 10 40 50 10 30 60 20 30 50 20 40 40

( category : below normal, 0 : near normal, + : above normal, Unit : %)

slide19

(2) 3 month forecast

Temperature Forecast

Period 3 months 1st month 2nd month 3rd month

Nov.-Jan. Nov. Dec. Jan.

Category - 0 + - 0 + - 0 + - 0 +

Northern Japan 30 50 20 30 50 20 30 50 20 20 50 30

Eastern Japan 20 5030 20 50 30 20 50 30 20 40 40

Western Japan 20 40 40 20 50 30 20 40 40 20 40 40

Nansei Islands 20 30 50 20 40 40 20 30 50 20 30 50

( category ? : below normal, 0 : near normal, + : above normal, Unit : %)

slide20

Precipitation Forecast (3 month forecast)

Period 3 months 1st month 2nd month 3rd month

Nov.-Jan. Nov. Dec. Jan.

category - 0 + - 0 + - 0 + - 0 +

Northern Japan

Japan Sea side 20 50 30 20 50 30 20 50 30 30 40 30

Pacific side 30 50 20 30 50 20 30 50 20 30 40 30

Eastern Japan

Japan Sea side 20 50 30 20 50 30 20 50 30 30 50 20

Pacific side 20 50 30 20 50 30 20 50 30 20 40 40

Western Japan

Japan Sea side 20 50 30 20 50 30 20 50 30 30 40 30

Pacific side 20 50 30 20 50 30 20 50 30 20 40 40

Nansei Islands 20 50 30 40 40 20 20 40 40 30 40 30

( category -: below normal, 0 : near normal, + : above normal, Unit : %)

slide23

Examples of probabilistic forecast excluding seasonal forecast at JMA

JMA uses probabilistic expression not only in seasonal forecast but in many forecasts where there is uncertainty of forecast.

(1) Short-range forecast

Tokyo District

Today

North-easterly wind, fine, occasionally cloudy,

Wave 0.5m

Probability of Precipitation

12-18 10%

18-00 0%

Temperature forecast

today’s maximum in Tokyo 14 degrees centigrade

slide24

Date 24Mon 25Tue26Wed 27Thu 28Fri 29Sat 30Sun

Max.,Min. T(℃)

Probability of Precipitation

(2) One-week forecast

         10 / 1611/15 7/15 9/16 9/17 11/18

70 50 40 30 30 40

Issuedat23,NOV

slide25

(3) Typhoon Forecast

The probability that a district will be in storm warming area

is also being issued.

slide26

Quality of Probabilistic Forecast

What is good probabilistic forecast?

‘A good probabilistic forecast must express the uncertainty of forecast exactly and have large dispersion from ‘climatic proportion of frequency’

(a) Reliability

(b) Resolution

slide27

(a) Reliability

Probability forecast P was issued ‘M(P)’ times for a event ‘E’.

In M(P) times, event ‘E’ occurred N(P) times.

If probability forecast is reliable

for large number of M(P).

Ex. Probability 30% was issued 50times.

Event ‘E’ is expected to occur about 15timesin them.

slide28

The reliability diagram

Event E

Ideal reliability diagram

Relative frequency

Forecast probability

slide29

Reliability Diagram

Monthly mean surface temperature forecast in Japan

Relative frequency

Forecast probability

slide30

monthly mean Psea anomalies >0

Reliability diagram

Example of

probability by ensemble one month forecast

slide31

(b) Resolution

Climatorogical relative frequency for a event is perfectly reliable

so far as there is no climatic Change.

ex. It is known that relative frequency of rainy day is about 30% at Tokyo from historical data set.

Can we issue probability of 30% as a tomorrow's probability of precipitation at Tokyo every day?

When the reliability is perfect, dispersion of probability from climatorogical relative frequency is another measure of probabilistic forecast quality ‘resolution’.

The best resolution probabilistic forecasts are those of 100% or 0% provided that reliability is perfect. =perfect forecast.

slide32

Resolution

Monthly mean surface temperature forecast in Japan

Relative frequency

Forecast probability

Frequency

slide33

Resolution of probability forecast

7day mean Z500 anomalies>0

1st week

Reliability diagram

Relative frequency(resolution)

1st week probabilistic forecast is better than 2nd week in ‘resolution’ measure

2nd week

From one-month ensemble forecast

slide34

Quantitative evaluation of probabilistic forecast

-------The Brier score-------

The Brier score is defined as

pi : forecast probability

vi=1: if event E occurred

=0 : if not occurred

N: total number of forecast

bis mean square error of probability

slide35

After some algebraic transformations,

b can be rewritten as

brel

bres

bunc

where

Nt :frequency of forecast probability pt

Mt: frequency of occurrence of event E within Nt

slide36

represents ‘Reliability’

represents ‘Resolution’

represents ‘Uncertainty’

(not depends on the forecast.

shows difficulty of forecast)

Murphy’s decomposition(1973)

slide37

Brier skill score

The absolute value of brier score is difficult to understand except perfect reliability and resolution case (=0).

Then usually following Brier skill score is used

where bc is the brier score of climatorogical relative frequency( probabilistic climate forecast).B=1 when the forecast is perfect.

Additionally, following skill score is also used.

Brel=Bres=1 when the forecast is perfect.

slide38

Example of Brier skill score

Ensemble one-month forecast

7day mean Z500anomalies>0

2nd week

1st week

All scores are expressed in %

slide40

(4) Economical value of forecast

----a consideration with Cost/Loss model-----

The Cost/Loss model

an event E

①If E occurs, the loss is Ldue to a damage

②To protect form the damage, a action costs C (<L)

ex.

If the temperature exceeds a threshold, a crop is damaged by a pest. The damage is L.

To protect the crop from a pest , spraying agrochemical is necessary and it costsC.

slide41

The cost/benefit model

a Event E

if the event E occurs , a benefit is B.

If not occurs, we lose C.

Example、

If it is fine, a benefit B is earned by selling lunch boxes, but if it rains, no benefit is earned.

The cost to make lunch boxes is C.

The discussion of Cost/Loss is almost the same as that of Cost/Benefit model.

slide42

The case without forecast (Cost/Loss model)

Considering D times operations.

The climatorogical proportion of occurrence of event E is R.

When always taking action to protect from damage,

the expense is ,

If no action is done, the expense is

Then,

when R>C/L

If the event E occurs frequently, it is better to take action always.

slide43

The case with perfect forecast (Cost/Loss model)

If forecast is perfect, we take action only when event E is forecasted. Then the expense is

of course,

Perfect forecast always reduces the expense

slide44

The case with actual deterministic forecast

Actual forecast sometimes fails. Then we make following contingency matrix

Occurs

Forecast

( ) shows in case of perfect forecast

If we use these forecasts, the expenses corresponding individual boxes above are,

Occurs

Take Action

slide45

The relationships with pre-defined variables are

Occurs

Forecast

We newly define following variables which express forecast skill

Hit rate

False-alarm rate

The lager H is and the smaller F is , the better forecast is.

H=1 and F=0, for perfect forecast

slide46

The expense with these forecasts is

If forecast is worse, Mp sometimes become lager than Mclim2 or Mclim1 .

IF H=0 and F=1(the worst forecast!)

Mp=DLR+D(1-R)C>Mclim1,Mclim2

slide47

It should be also noted that Mp depends not only on H and F but C/Land R .

An imperfect deterministic forecast is not always useful for all users.

slide48

The case with probabilistic forecast

(Cost/Loss model)

How do we use probabilistic forecast?

Consider to take action always when the probabilistic forecast is P0 for the occurrence of event E. The expense is,

Mp0=DpC

where Dp is total frequency that the event E was forecasted with probability P0

If probabilistic forecast is perfectly reliable, the frequency that event E occurred is DpP0 and then the expense without taking action is

Mc=DpP0L

slide49

Probabilistic forecast is useful

when,

Mp0=DpC < Mc=DpP0L

Therefore, we should take action when

P0>C/L

This is the simplest way to use probabilistic forecast for decision making.

Note the threshold probabilities are different for individual users with different C/L and all users can get some gain with their own threshold.

In case of Cost-benefit model, with similar calculations, the criterion is,

P0 >C/(B+C)

slide50

Ex. 1(Cost-Loss model)

If the temperature exceeds a threshold =33℃, a crop is damaged by a pest. The damage is L=10,000$

To protect the crop from a pest , spraying agrochemical is necessary and it costsC=3,000$

C/L=3000/10000=0.3

Consider 10 times forecast of above 33℃ with probability 20%(30%,40%) .

When we take action the expense is

3000x10=30,000$

When we do not take action, the expense is

10000x(10x0.2)=20,000 $ for probability 20%

10000x(10x0.3)=30,000 $ for probability 30%

10000x(10x0.4)=40,000 $ for probability 40%

We had better take action when Probability>30%=C/L

slide51

Example for cost-benefit model

If it is fine, a benefit B is earned by selling lunch boxes, but if it rains, no benefit is earned.

The cost to make lunch boxes is C ,which is the loss when it rains.

Price of lunch box=10$ and 100 lunch boxes are sold in a fine day.

The cost to make one lunch box=5$

The benefit in a fine day is B=(10-5)x100=500$

The cost is C=5x100=500$,which is the loss when it rains

C/(B+C)=0.5

10 times forecast of fine with probability 40%(50%,60%)

When we sell thelunch boxes, The expected cost is

500x(10x(1-0.4))=30,000 $ for probability 40%

500x(10x(1-0.5))=25,000 $ for probability 50%

10000x(10x(1-0.6)))=20,000 $ for probability 60%

The expected benefit is

500x(10x0.4)=20,000 $ for probability 40%

500x(10x0.5)=25,000 $ for probability 50%

500x(10x0.6)=30,000 $ for probability 60%

We had better sell when Probability>50%=C/(B+C)

slide52

Verification of probabilistic forecast

-----How to use an actual probabilistic forecast---------

We used a important assumption to derive the threshold probability to take action in the previous section.

Assumption

probabilistic forecast is perfectly reliable.

Although this condition would be satisfied approximately in most practical probabilistic forecasts, there is no guarantee that it is always satisfied.

In addition, the expense reduction (Mp-Mclim) with probabilistic forecast also depends on the ‘resolution’ of probabilistic forecast and we cannot know how much it is without verification.

Therefore verification is important to use probabilistic forecast actually.

slide53

A Verification of probabilistic forecast from the economical view point

We assume

E will occur when P>Pt and

E will not occur when P<Pt

where Pt is a threshold probability. And again we make contingency matrix as follows.

Occurs

Forecast

slide54

As similar to before, the expense is,

The expense for perfect forecast is,

That for climatic forecast is,

slide55

We define the ‘Value of forecast’ as the reduction in Mp over Mclim normalized by the maximum possible reduction. That is,

V(Pt)=1 for the perfect forecast and negative for bad forecast.

slide56

We calculate V for various threshold probabilities and C/L.

For a given C/L, and a event E, the optimal value is,

slide57

Probability

of T850anomalies >0

From Palmer(2000)

For a user

with C/L=0.6,

V=12% and

best threshold

probability

is 70%,

although threshold

60%(=C/L) gives

some benefit(7%).

60% line

20%

40%

10%

30%

  • From this graph,
  • The user can choose the threshold which brings maximum benefit( Note reliability is not completely perfect).
  • A user with a C/L can estimate maximum benefit

50%

90%

60%

70%

80%

slide59

Vopt for

Probability forecast

V for

deterministic forecast

The graph of Vopt for probabilistic forecast is higher and wider than that of V for the deterministic forecast.

Because, for deterministic forecast, only one contingency matrix is made and then only one graph of V is drawn. On the other hand, Vopt is the maximum of the graphs of V with various probability threshold.

slide60

A verification example of monthly surface temperature anomaly probability forecast in Japan (Statistical down scaling from ensemble forecast))

Surface temperature (below normal)

Surface temperature (above normal)

Surface temperature

Reliability diagram for above normal

Reliability diagram for below normal

slide61

A verification of dymamical one-month ensemble forecast Anom.(Z500)>0

28day mean ( winter 2001)

slide62

1st week

2nd week

slide64

Conclusions

  • Seasonal forecast has uncertainty due to chaos of atmospheric flow. The probabilistic forecast is the best method to express this uncertainty.
  • In probabilistic forecast, a user can use his/her own threshold probability to take action depending on his/her own C/L in Cost-Loss model. For a sufficiently ‘reliable’ probabilistic forecast, the threshold probability is equal to C/L. If ‘reliability’ is not enough, user can know the best threshold probability by a verification.
  • In general, the probabilistic forecast is superior to the deterministic forecast at least from the economical point of view so far as the forecast is not perfect although it seems difficult in some degree. More dissemination of probabilistic forecast is necessary.
slide66

Monthly mean surface temperature forecast at Japan

Surface temperature above normal

Vopt