Anthony Barnston, Lisa Goddard, Simon Mason and Andrew Robertson International Research Institute

1. Consolidation of Predictions of Seasonal Climate by Several Atmospheric General Circulation Models at IRI • Anthony Barnston, Lisa Goddard, Simon Mason and Andrew Robertson • International Research Institute • for Climate and Society (IRI)

2. IRI DYNAMICAL CLIMATE FORECAST SYSTEM 2-tiered OCEAN ATMOSPHERE GLOBAL ATMOSPHERIC MODELS ECPC(Scripps) ECHAM4.5(MPI) CCM3.6(NCAR) NCEP(MRF9) NSIPP(NASA) COLA2 GFDL PERSISTED GLOBAL SST ANOMALY Persisted SST Ensembles 3 Mo. lead 10 POST PROCESSING MULTIMODEL ENSEMBLING -Bayesian -Caninical variate 24 24 FORECAST SST TROP. PACIFIC: THREE scenarios (multi-models, dynamical and statistical) TROP. ATL, INDIAN (ONE statistical) EXTRATROPICAL (damped persistence) 12 Forecast SST Ensembles 3/6 Mo. lead 24 24 30 12 30 30

3. IRI DYNAMICAL CLIMATE FORECAST SYSTEM 2-tiered OCEAN ATMOSPHERE GLOBAL ATMOSPHERIC MODELS ECPC(Scripps) ECHAM4.5(MPI) CCM3.6(NCAR) NCEP(MRF9) NSIPP(NASA) COLA2 GFDL PERSISTED GLOBAL SST ANOMALY FORECAST SST TROP. PACIFIC:THREE scenarios: 1) CFS prediction 2) LDEO prediction 3) Constructed Analog prediction TROP. ATL, and INDIAN oceans CCA, or slowly damped persistence EXTRATROPICAL damped persistence

4. Six GCM Precip. Forecasts, JAS 2000

5. RPSS Skill of Individual Models: JAS 1950-97

6. Goals To combine the probability forecasts of several models, with relative weights based on the past performance of the individual models To assign appropriate forecast probability distribution: e.g. damp overconfident forecasts toward climatology

7. Probabilities and Uncertainty Climatological Probabilities GCM Probabilities k = tercile number t = forecast time m = no. ens members Above Normal 1/3 6/24 Tercile boundaries are identified for the models’ own climatology, by aggregating all years and ensemble members. This corrects overall bias. 1/3 Near-Normal 8/24 Below Normal 1/3 10/24

8. Bayesian Model Combination Combine climatology forecast (“prior”) and an AGCM forecast, with its evidence of historical skill, to produce weighted (“posterior”) forecast probabilities, by maximizing the historical likelihood score.

9. Aim to maximize the likelihood score k=tercile category t=year number The multi-year product of the probabilities that were hindcast for the category that was observed. (Could maximize other scores, such as RPSS) Prescribed, observed SST used to force AGCMs. Such simulations used in absence of ones using truly forecasted SST for at least half of AGCMs.

10. 1. Calibration of each model, individually, against climatology Optimize likelihood score k=tercile category (1,2, or 3) t=year number j=model number (1 to 7) w=weight for climo (c) or for model j PMMkt= weighted linear comb of Pjkt over all j, normalized by Σ(wj) 2. Calibration of the weighted model combination against climatol Optimize likelihood score where wMM uses wj proportional to results of the first step above

11. Algorithm used to maximize the designated score: Feasible Sequential Quadratic Programming (FSQP) “Nonmonotone line search for minimax problems” C M : TOTAL NUMBER OF CONSTRAINTS. C ME : NUMBER OF EQUALITY CONSTRAINTS. C MMAX : ROW DIMENSION OF A. MMAX MUST BE AT LEAST ONE AND GREATER C THAN M. C N : NUMBER OF VARIABLES. C NMAX : ROW DIMENSION OF C. NMAX MUST BE GREATER OR EQUAL TO N. C MNN : MUST BE EQUAL TO M + N + N. C C(NMAX,NMAX): OBJECTIVE FUNCTION MATRIX WHICH SHOULD BE SYMMETRIC AND C POSITIVE DEFINITE. IF IWAR(1) = 0, C IS SUPPOSED TO BE THE C CHOLESKEY-FACTOR OF ANOTHER MATRIX, I.E. C IS UPPER C TRIANGULAR. C D(NMAX) : CONTAINS THE CONSTANT VECTOR OF THE OBJECTIVE FUNCTION. C A(MMAX,NMAX): CONTAINS THE DATA MATRIX OF THE LINEAR CONSTRAINTS. C B(MMAX) : CONTAINS THE CONSTANT DATA OF THE LINEAR CONSTRAINTS. C XL(N),XU(N): CONTAIN THE LOWER AND UPPER BOUNDS FOR THE VARIABLES. C X(N) : ON RETURN, X CONTAINS THE OPTIMAL SOLUTION VECTOR. C U(MNN) : ON RETURN, U CONTAINS THE LAGRANGE MULTIPLIERS. THE FIRST C M POSITIONS ARE RESERVED FOR THE MULTIPLIERS OF THE M C LINEAR CONSTRAINTS AND THE SUBSEQUENT ONES FOR THE C MULTIPLIERS OF THE LOWER AND UPPER BOUNDS. ON SUCCESSFUL C TERMINATION, ALL VALUES OF U WITH RESPECT TO INEQUALITIES C AND BOUNDS SHOULD BE GREATER OR EQUAL TO ZERO.

12. Circumventing the effects of sampling variability • Sampling variability appears to be an issue: noisy weight distribution with large number of zero weights and some unity weights • Bootstrap the optimization, omitting contiguous 6-year blocks of the 48-yr time series • yields 43 samples of 42 years • shows the sampling variability of the likelihood over subsets of years • We average the weights across the samples

13. Example • Six GCMs’ Jul-Aug-Sep precipitation simulations • Training period: 1950–97 • Ensembles of between 9 and 24 members

14. Model Weights – initially, by individual model

15. Climatological Weights – Multi-model

16. Model Weights – after second (damping) step

17. Model Weights – step 2, and Averaged over Subsamples

18. For more spatially smooth results, the weighting of each grid point is averaged with that of its 8 neighbors, using binomial weighting. X X X X X X X X X

19. Climatological Weights

20. Combination Forecasts of July-Sept Precipitation After first stage only After second (damping) stage After sampling subperiods After spatial smoothing

21. ReliabilityJAS Precip., 30S-30N Above-Normal Below-Normal Bayesian Pooled Observed relative Freq. Observed relative Freq. Individual AGCM Forecast probability Forecast probability (3-model) from Goddard et al. 2003

22. RPSS Precip. from Roberson et al. (2004): Mon. Wea. Rev., 132, 2732-2744

23. RPSS 2-m Temp. from Roberson et al. (2004): Mon. Wea. Rev., 132, 2732-2744

24. The “climatological” (equal-odds) forecast provides a useful prior for combining multiple ensemble forecasts Sampling problems become severe when attempting to combine many models from a short training period (“noisy weights”) A two-stage process combines the models together according to a pre-assessment of each against climatology Smoothing of the weights across data sub-samples and spatially appears beneficial Conclusions - Bayesian

25. IRI’s forecasts use also a second consolidation scheme, whose result is averaged with the result of the Bayesian scheme. 1. Bayesian scheme 2. Canonical Variate scheme

26. Canonical Variate Analysis (CVA) • A number of statistical techniques involve calculating linear combinations (weighted sums) of variables. The weights are defined to achieve specific objectives: • PCA – weighted sums maximize variance • CCA – weighted sums maximize correlation • CVA – weighted sums maximize discrimination

27. Canonical Variate Analysis

28. Canonical Variate Analysis The canonical variates are defined to maximize the ratio of the between-category (separation between the crosses) to the within-category (separation of dots from like-colored crosses) variance.

29. Conclusion IRI presently using a 2-tiered prediction system. It is interested in using fully coupled systems also, and is looking into incorporating those. Within its 2-tiered system it uses 4 SST prediction scenarios, and combines the predictions of 7 AGCMs. The merging of 7 predictions into a single one uses two multi-model ensemble systems: Bayesian and canonical variate. These give somewhat differing solutions, and are presently given equal weight.