Alexander M. Sterin, Ph.D

1. От данных наблюдений к новым знаниям об изменениях климата: подходы и основные этапыFrom Observational Data to New Knowledge about Climate Change: Approaches and Main Stages Alexander M. Sterin, Ph.D Russian Research Institute for Hydrometeorological Information – World Data Center (RIHMI-WDC) 6, Korolyov str., Obninsk, Kaluga region, 249035 E-mail:sterin@meteo.ru

2. Problems to be mentioned: • 10+ principles of climatic monitoring • Eleventh principle of climatic monitoring • Pyramid of climate information products • Large climate data projects • What do we want to detect? What is climate signal? • The main questions by TAR IPCC • Observational data vs Reanalysis outputs • Traditional and robust (resistant) techniques in data processing and analysis • Inhomogeneity detection, estimation, correction (or “correction”?), and trend re-estimation • Why “ensemble” of data series?

3. Ten principles of climate monitoring: sources: • Karl, T.R., V.E. Derr, D.R. Easterling, C.K. Folland, D.J. Hoffman, S. Levitus, N.Nicholls, D.E. Parker, and G.W. Withee, 1995: Critical issues for long-term climate monitoring. Climatic Change, 31, 185-221 • National Research Council, 1999: Adequacy of Climate Observing Systems, National Academy Press, Washington, DC, 51 pp • Guidelines on Climate Observation Networks and Systems. WMO WCP WCDMP-No.52 WMO-TD No.1185 December 2003, 57 pp.

4. Ten principles of climate monitoring • 1. Management of Network Change • 2. Parallel Testing • 3. Metadata • 4. Data Quality and Continuity • 5. Integrated Environmental Assessment • 6. Historical Significance • 7. Complementary Data • 8. Climate Requirements • 9. Continuity of Purpose • 10. Data and Metadata Access

5. The PYRAMID of derivative data sets for the climate studies VOLUME • Global climate generalization (separate figures) • Climate data of high degree of generalization (small volume) • Data sets derived from next lower level (limited volume) • The data sets derived from observational data of lowest level (moderate volume) • Lowest level – observational data before and after quality check (huge volume) MONADS

6. The PYRAMID of derivative data sets for the climate studies Requirements for moderate volume derivative data sets (MONADS for U/A data, in particular): To include maximal amount set of monthly statistics To provide minimization of needed accesses to the lower level data (observational data) To provide the statistics that are correct to be generalized when upper-level data sets are obtained To minimize the heuristic considerations and heuristic statistics To provide a “bridge” between the lower level data (observational data) and data sets of higher level of generalization

7. What do we want to detect? What is climate signal? The definitions of signal and noise largely depend on interests of researcher In climate research, the signal is defined by the interest of researcher and by consistency with physical ideas; noise is everything else unrelated to this object of interest Von Storch and Frankignoul (1998): “The large amounts of data that are usually studied in climate exhibit a complex mixture of signals and noise. The purpose of statistical analysis is to disentangle this mixture to find the needle (the signal) in the haystack (the noise). This allegory has two sides. First, it is difficult to find the needle in the haystack. Second, after it has been found, it should be easily recognizable… To identify the climate signal, advanced techniques must be required, but after the identification, the signal usually may be described by means of simple techniques such as composites, correlations, etc.”

8. The main questions by TAR IPCC: seven questions related to detection of climate change How much is the world warming? Is the recent warming unusual? How rapidly did the climate change in the distant past? Have precipitation and atmospheric moisture changed? Are the atmospheric/oceanic circulations changing? Has climate variability, or have climate extremes, changed? Are the observed trends internally consistent?

9. Large climate data projects The results must be open, for testing by max user groups Serious errors and gaps are inevitable – users must understand this and decline from criticism addressed to authors! Authors must announce about their gaps immediately – they must feel proud but not feel guilty!!!!!! Based on: previous errors, previous experience, previous analysis, - the new stages (cycles) of large data projects must be planned and repeated. These repetitions must be every six to ten years…

10. Observational data vs Reanalysis outputs • Reanalysis outputs are used widely – avalanche of publications, avalanche of applications • In numerous publications, reanalysis outputs are mentioned as “DATA” (in contrast to “MODEL OUTPUTS”) • Reanalysis outputs are NOT “DATA”!!!!! • We need to study the difference between observational data and reanalysis outputs in all spatial considerations and temporal scales • A special talk at ENVIROMIS 2004 will be devoted to trend differences in U/A data and in reanalysis outputs

11. Traditional and robust (resistant) techniques in climate data processing and analysis Robust, resistant and non-parametric techniques could be applied as alternative to traditional statistics on all stages of data processing and analysis: In elementary statistical calculations – robust parameters of scale, location and association In trend analysis – robust measures of association between predictant and predictors In datasets of statistics, it is better to stop “one step before the final step” in including the statistics We must not limit to one robust techniques against traditional techniques – the robust methods are numerous, and we may choose the worst of them We must give reference, what statistical techniques were used for calculations (for trend calculations, in particular)

12. Traditional and robust (resistant) techniques in climate data processing and analysis

13. Inhomogeneity detection, estimation, correction (or “correction”?), and trend re-estimation The main principle: do not do harm to time series How to identify “inhomogeneity candidate points” in climate time series? By statistical methods, from metadata, by physical models and considerations, by comparing with: baseline device, neighbor stations, other detectors, etc Are the “corrected” data correct and are the “non-corrected” data incorrect? A step back must be always provided Need to re-calculate trends without necessary getting so called “corrected” series – wide “what-if” analysis

14. Why “ensemble” of data series for climate signal evaluation? Primarily – vast intercomparison of series The “ensemble” of independently obtained data sets from various independent platforms, is aimed to estimate group-based value of climate signal A robust criterion R is introduced to evaluate the ratio of uncertainty measures: MSE – median of individual values of standard error in signal estimates (associated with uncertainties in estimating by individual datasets) PSD – pseudo standard deviation of signal amplitudes (PSD= IQR/1.349) – associated with the spread of signal estimates from all available datasets Ratio R=2*MSE/PSD R>>1 – uncertainty in signal is large enough in each individual dataset to encompass the spread between datasets R<=1 – use of multiple (ensemble) estimates datasets gives more complete characterization of uncertainty than individual estimates R – non-parametric criterion, non-sensitive to outliers in individual datasets, if any Tropics Tropics

15. Why “ensemble” of data series for climate signal evaluation? Tropics Tropics

16. Ten principles of climate monitoringadditional eleventh principle • 11. Redundancy: Multiple and independent observing systems should provide measurements, and mulitiple, independent research groups should analyze the data and provide climate monitoring data products. Such redundancy exists for many key climate variables, and enhances our understanding of uncertainty in estimates of climate variations • (FORMULATED BY Seidel, Sterin et al., 2004, Journ. Climate, no.11)

17. THANK YOU!

18. Ten principles: • 1. Management of Network Change: Assess how and the extent to which a proposed change could influence the existing and future climatology obtainable from the system, particularly with respect to climate variability and change. Changes in observing times will adversely affect time series. Without adequate transfer functions, spatial changes and spatially dependent changes will adversely affect the mapping of climatic elements.

19. Ten principles: • 2. Parallel Testing: Operate the old system simultaneously with the replacement system over a sufficiently long time period to observe the behavior of the two systems over the full range of variation of the climate variable observed. This testing should allow the derivation of a transfer function to convert between climatic data taken before and after the change. When the observing system is of sufficient scope and importance, the results of parallel testing should be documented in peer-reviewed literature.

20. Ten principles: • 3. Metadata: Fully document each observing system and its operating procedures. This is particularly important immediately prior to and following any contemplated change. Relevant information includes: instruments, instrument sampling time, calibration, validation, station location, exposure, local environmental conditions, and other platform specifics that could influence the data history. The recording should be a mandatory part of the observing routine and should be archived with the original data. Algorithms used to process observations need proper documentation. Documentation of changes and improvements in the algorithms should be carried along with the data throughout the data archiving process..

21. Ten principles: • 4. Data Quality and Continuity: Assess data quality and homogeneity as a part of routine operating procedures. This assessment should focus on the requirements for measuring climate variability and change, including routine evaluation of the long-term, high-resolution data capable of revealing and documenting important extreme weather events.

22. Ten principles: • 5. Integrated Environmental Assessment: Anticipate the use of data in the development of environmental assessments, particularly those pertaining to climate variability and change, as a part of a climate observing system's strategic plan. National climate assessments and international assessments, (e.g., international ozone or IPCC) are critical to evaluating and maintaining overall consistency of climate data sets. A system's participation in an integrated environmental monitoring program can also be quite beneficial for maintaining climate relevancy. Time series of data achieve value only with regular scientific analysis..

23. Ten principles: • 6. Historical Significance: Maintain operation of observing systems that have provided homogeneous data sets over a period of many decades to a century or more. A list of protected sites within each major observing system should be developed, based on their prioritized contribution to documenting the long-term climate record.

24. Ten principles: • 7. Complementary Data: Give the highest priority in the design and implementation of new sites or instrumentation within an observing system to data-poor regions, poorly observed variables, regions sensitive to change, and key measurements with inadequate temporal resolution. Data sets archived in non-electronic format should be converted for efficient electronic access.

25. Ten principles: • 8. Climate Requirements: Give network designers, operators, and instrument engineers climate monitoring requirements at the outset of network design. Instruments must have adequate accuracy with biases sufficiently small to resolve climate variations and changes of primary interest. Modeling and theoretical studies must identify spatial and temporal resolution requirements.

26. Ten principles: • 9. Continuity of Purpose: Maintain a stable, long-term commitment to these observations, and develop a clear transition plan from serving research needs to serving operational purposes.

27. Ten principles: • 10. Data and Metadata Access: Develop data management systems that facilitate access, use, and interpretation of data and data products by users. Freedom of access, low cost mechanisms that facilitate use (directories, catalogs, browse capabilities, availability of metadata on station histories, algorithm accessibility and documentation, etc.), and quality control should be an integral part of data management. International cooperation is critical for successful data management.