Quantifying efficiency of homogenisation methods

1. Quantifying efficiency of homogenisation methods Dr. Peter Domonkos dopeter@t-online.hu COST HOME ES0601

2. Measuring efficiencyour expectations • Gaining the real climatic trends, • Gaining the real trends and fluctuations, • Identifying large inhomogeneity-shifts one-by-one, • Identifying as many shifts as we can

3. Measuring efficiencygeneral practice • Usually the rate of correct detection is examined (Ducré-Robitaille, Mestre, Menne and Williams, etc.) • Menne and Williams (2005) apply the hit rate (or power, = H), false detection rate (F), false alarm rate (FAR), bias of detection frequency (B), and the improvement in skill compared to random forecasts (HSS).

4. Measuring efficiencygeneral practice

5. Measuring efficiencythis presentation • Arbitrary, but reasonable choices • 1 = standard deviation of estimated noise • Factual shift: Shift with MM0 magnitude between two adjacent 3 year long periods. M0 = 2 or M0 = 3 here. • Right detection: A shift with M1.5 for M0 = 2 (M2 for M0 = 3) is detected with maximum 1 year lapse. • False detection: A shift with M1.5 for M0 = 2 (M2 for M0 = 3) is detected at year j, but there is no shift of the same direction than the detected one with M > 0 within the (j-2,j+2) period.

6. Measuring efficiencythis presentation • Let the number of the time series be m, the total of the factual shifts is k, the number of right detections is DR, that of false detections is DF, then

7. Measuring efficiencythis presentation • Reliability of trends!? • Let the mean bias of trend slopes, caused by inhomogeneities is t0 before the homogenisation, and t after the homogenisation. Then the improvement in trend reliability is indicated by • General (combined) efficiency (Domonkos, 5th Seminar, 2006)

8. Properties of time series • Five versions of simulated datasets are examined here. Each dataset has 10,000, one hundred year long time series. The scale of the properties is wide from a single inhomogeneity per time series to the inclusion of very complex inhomogeneity-structures „Hungarian standard” (Domonkos, 5th Seminar, 2006). • (1) 1 shift with M = 3; (2) 1 shift with M = 3 and 4 shifts with M = 1.5; (3) and (4) Shifts with 1/ decade frequency, exponential distribution of M above 1, and uniform distribution of M below 1. (3) Mmax<2; (4) Mmax<3; (5) Hungarian standard

9. Distribution of difference (percentage) between the detected inhomogeneity-properties of simulated and real climatic time series for HUSTANDARD.k : simple, wk : weighted with sample size

10. Homogenisation methods • 15 objective homogenisation methods: 2-2 versions of Bayes-test [Bay, Ba1], Buishand-test [Bu1, Bu2], SNHT [SNH, SNT] and t-test [tt1, tt2]; Caussinus-Mestre test [C-M], Easterling-Peterson test [E-P], Mann-Kendall test [M-K], MASH [MAS], Multiple Linear Regression [MLR], Pettitt-test [Pet] and Wilcoxon Rank Sum test [WRS].

11. Method parameterisation • With original parameterisations the chance of detecting at least 1 inhomogeneity is ~5% in pure white noise. • Minimum length of subperiods for calculating own statistical properties: usually 5 years, but in C-M and MAS 1 year, and in E-P 3 years. • Outliers are prefiltered; Concerning multiple inhomogeneities the semihierarchic algorithm of Moberg and Alexandersson (1997) is included in Bay, Ba1, Bu1, Bu2, M-K, MLR, Pet, SNH, SNT and WRS. • In a few experiments optimisedparameterisation is applied (its use is indicated).

12. Red = C-MBlue = MASHGreen = E-P Black = t-test (tt1) Brown = SNHT for shiftsLila = MLR

13. Identification A, 1 shift (M=3)

14. Identification A, 1 shift (M=3)+ 4 small shifts

15. Identif. A of M3, Exp. M<6

16. Identif.A of M2, Hu standard

17. Identif.A of M3, Hu standard

18. Identif.B of M2, Exp. M<2

19. Identif.B of M3, Exp. M<2

20. Absence of large shiftsnumber of kinds: 7, best: tt1, C-M, Bay

21. Trends, 1 shift (M=3)filled columns = optimised parameters

22. Trends, 1 shift + 4 small shifts

23. Trends, Exp. M<2

24. Trends, Exp. M<6

25. Trends, Hu standard

26. Identification A, 1 shift

27. Identif.A, 1 shift + 4 small shifts

28. Identif.B of M2, Exp. M<2

29. Identif.B of M3, Exp. M<2

30. Identif.A of M2, Hu standard

31. Identif.A of M3, Hu standard

32. Identif.A of M3, Exp. M<6

33. Discussion • Identification of M>3 shifts is best with MASH, but its reproduction of climatic trends is not among the best results. This drawback of MASH can be reduced with parameter-optimisation. • Many results with C-M are on the top, except for cases of very low rate of large inhomogeneities. If the evaluations of shorter than 3-year sections are excluded, and detection results with M<2 are not considered, all the possible disadvantages with C-M are avoidable, even the skill in detecting shifts of M>3 exceeds the performance of MASH.

34. Conclusions • The efficiency-order of homogenisation methods strongly depends on the properties of time series, the purposes/priorities of the homogenisation, and on the way of the efficiency evaluation. • Direct methods for identifying multiple inhomogeneities (C-M and MASH) usually perform better, than the other methods. When the avoidance of false detection has enhanced importance t-test and E-P methods are also competitive. • Parameter-optimisation may yield improved results.

35. Thank you for your attention!COST HOME ES0601