CROP-CIS User utility assessment of Geoland2 BioPar products

1. CROP-CISUser utility assessment of Geoland2 BioPar products Comparison of G2 BioPar vs. JRC-MARSOP SPOT- VGT NDVI & fAPAR products M. Meroni, C. Atzberger, O. Leo. JRC-MARS

2. Index • Objective of the analysis • Methods (spatial and temporal analysis) • Data and study areas • Main results of the comparison • Ongoing activities on BP full archive (JRC + IGIK) G2 Interim Review Meeting, JRC Ispra

3. Objective To provide a first assessment of new BioPar products by comparison with the “well known” JRC-MARSOPusing a comprehensive statistical protocol • The analysis can: • describe existing differences between the two datasets • identify and point out inconsistencies in a specific product • provide a basis for more in-depth analysis at specific locations / times • The analysis can’t: • say which product is best! (validation is required for this purpose) G2 Interim Review Meeting, JRC Ispra

4. Methods • Analysis of spatial and temporal agreement separately • Spatial comparison y x z Data cube 1 Data cube 2 Land cover • Compare layer by layer • Summarize this comparison by a metric (e.g. R2) • Plot the metric across time (possibly stratified by land use classes) class a class b class c data2 data2 data2 data1 data1 data1 The result of the spatial comparison is a time series G2 Interim Review Meeting, JRC Ispra

5. Methods • Temporal comparison • Compare pixel by pixel • Summarize this comparison by a metric (e.g. R2) • Plot the metric across space (possibly deriving some summary statistics of such maps) Data cube 1 Data cube 2 Z2 Z1 Z2 Z1 Land cover The result of the temporal comparison is a map G2 Interim Review Meeting, JRC Ispra

6. Data: SPOT-VGT 10-day • Geoland2 BioPar data vs. • JRC-MARS data • MARSOP-FS for the global window (FOODSEC action) • MARSOP-A4C for the extended European window (AGRI4CAST action), original and filtered (mod-SWETS) G2 Interim Review Meeting, JRC Ispra

7. Biopar compositing window • dekadal products: composites updated every 10 days; • 30 days compositing* window is asymmetric around the “most representative” day (16 day before and 13 after it, equally weighted); Considering the required processing time, the overall delayfor data delivery is 16 days (MARSOP delay = 8 days) An issue for MARS NRT applications *Note that for BP the term “compositing” is not fully appropriate because the value assigned to the dekad is derived from the inversion of the linear reflectance model of Roujean et al. (1992) applied to normalize the bidirectional effects during the synthesis period of 30 days. G2 Interim Review Meeting, JRC Ispra

8. Data: SPOT-VGT 10-day • Time domain: • 2 years of BP GEOV1 demo products available (2003 and 2004) • Spatial domain: Three 10° x 10° BioPar tiles (1120 x 1120 pixels) selected in different agro-climatic regions monitored by JRC-MARS: • France (temperate - Mediterranean); • Brazil (tropical); • Niger (arid). G2 Interim Review Meeting, JRC Ispra

9. RESULTS – Cloud screening • Fraction of valid observations (examples using fAPAR) MARSOP BIOPAR NIGER (semi-arid) BRAZIL (tropical-humid) Similar in arid areas, MARSOP>BIOPAR in humid areas G2 Interim Review Meeting, JRC Ispra

10. RESULTS – Cloud screening • Temporal profile of fraction of valid observations MARSOP BIOPAR NIGER (semi-arid) NIGER (semi-arid) Cropland, Niger BRAZIL (tropical-humid) BRAZIL (tropical-humid) Cropland, Brazil MARSOP>>BIOPAR in cloudy/rainy periods G2 Interim Review Meeting, JRC Ispra

11. RESULTS – Cloud screening • Temporal profile of fAPAR (pixel of forest, Brazil) When MARSOP shows unrealistic drops, BIOPARis missing or not/less affected • Both NDVI and fAPAR, MARSOP-FS and -A4C: • BP has consistently lower fraction of valid observations; • Difference is large for Brazil (severe cloud cover) and small for Niger (low cloudiness); • Unrealistic drops in MARSOP temporal profiles. Cloud screening algorithm applied by BIOPAR is more conservative and realistic (.. larger compositing window for BIOPAR..) G2 Interim Review Meeting, JRC Ispra

12. RESULTS – Overall agreement (space and time pooled together) • Example for fAPAR ECDF, example for all land cover classes (pooled together) BPVs. MARSOP-FS BPVs. MARSOP-A4C FRANCE BRAZIL NIGER ~70% FAPAR < 0.5 ~30% FAPAR < 0.5 • FS and A4C data: fAPARMARS < fAPARBIOPAR • Largest differences observed for France (A4C) • statistically significant differences between distributions were found for Brazil (MARSOP-FS) and France (MARSOP- A4C) % of pixels showing statistically different data distribution G2 Interim Review Meeting, JRC Ispra

13. RESULTS – Overall agreement (space and time pooled together) • Correlation (example for fAPAR) Density scatter plot and linear regression (BIOPAR = intercept + slope * MARS) BP Vs. MARSOP-FS BP Vs. MARSOP-A4C FRANCE BRAZIL NIGER Regional differences in OLS coefficients: • Niger: very small offset and slope greater than 1; similar profile minima, larger BIOPAR maxima; • Brazil: positive offset and slope close to 1. BIOPAR is consistently higher than MARS; • France: large offset and slope smaller than 1. Highest differences between the two datasets are found for low fAPAR (wintertime values). G2 Interim Review Meeting, JRC Ispra

14. RESULTS – Spatial comparison • Temporal evolution of spatial agreement • (fAPAR of forest land cover) • Mean profiles • Large systematic component of the difference • Seasonality in spatial Agreement Coefficient (Ji & Gallo, 2006) BPVs. MARSOP-FS BPVs. MARSOP-A4C Spatial AC varying over time. What’s the source of this variability/scatter? rainy season winter time rainy season G2 Interim Review Meeting, JRC Ispra

15. RESULTS – Spatial comparison • Factors contributing to the scatter: • Different cloud screening effectiveness (example on fAPAR) BIOPAR FS G2 Interim Review Meeting, JRC Ispra

16. RESULTS – Spatial comparison • Factors contributing to the scatter: • Different cloud screening effectiveness (example on fAPAR) • Presence of brightness contrast in MARS-FS (example for NDVI, Woodland, Niger) due to BRDF MARSOP-FS BIOPAR Dekad 22 G2 Interim Review Meeting, JRC Ispra

17. RESULTS – Spatial comparison • Factors contributing to the scatter: • Different cloud screening effectiveness (example on fAPAR) • Presence of brightness contrast in MARS-FS (example for NDVI, Woodland, Niger) due to BRDF • «Unexpected» wintertime BIOPAR signal (France) G2 Interim Review Meeting, JRC Ispra

18. RESULTS – Temporal comparison • Temporal agreement Spatial distribution of Agreement Coefficient (example for fAPAR) BP Vs. MARSOP-FS BP Vs. MARSOP-A4C FRANCE BRAZIL NIGER AC Arid area with very low fAPAR variability Areas with high cloud cover Low agreement over large areas • Regions of very low agreement in Niger and Brazil can be explained G2 Interim Review Meeting, JRC Ispra

19. RESULTS – Temporal comparison • Temporal smoothness, example of Brazil (MARSOP-FS) Starting from the assumption that fAPAR varies smoothly over vegetated landwe investigated the temporal smoothness of the two datasets. mean absolute value of the first derivative of fAPAR over time • 40% of FS absolute dekadal variation > 0.05 FAPAR units • such frequency is implausible in the given geographical setting. BP appears more realistic. MARSOP-FS BioPar Mean |fAPAR′| Forest Cropland G2 Interim Review Meeting, JRC Ispra

20. Conclusions on 2 years of demo data • BP “Compositing” window may be problematic for MARS NRT application • Significant differences between BIOPAR and MARSOP (both spatial and temporal variability) • The differences in cloud screening effectiveness and compositing method make BioPar products more realistic than MARSOP-FS • Same holds true for MARSPO-A4C. However, positive BP anomalies in wintertime deserve further investigation • Scientific paper submitted to INTERNATIONAL JOURNAL OF APPLIED EARTH OBSERVATION AND GEOINFORMATION G2 Interim Review Meeting, JRC Ispra

21. Ongoing activities [JRC] • Intercomparison of the products extended to the full archive (1999-2011): • Statistical approach (similar to that described so far) • Operational approach (simulating actual MARS operations): • Analysis of vegetation anomalies • Bulletin production: differences in data quality (with BP taking more observations into account) against delivery time (with MARSOP data being in principle “more recent”) • Within-season crop yield predictions in Tunisia: evaluate possible performance improvements using BioPar data instead of MARSOP G2 Interim Review Meeting, JRC Ispra

22. Anomaly analysis, preliminary results • fAPAR from HIST archive (1998-2010) • Focus on anomalies as z-scores, i.e. normalization of each dekad as distance from mean expressed in SD units where x is the fAPAR profile of a given dekad • Overall correlation (example for France, MARSOP-A4C) • Low correlation of z-scores G2 Interim Review Meeting, JRC Ispra

23. Anomaly analysis, preliminary results • Example of time profiles • MARSOP and BP: roughly parallel development, but important scatter G2 Interim Review Meeting, JRC Ispra

24. Anomaly analysis, preliminary results • Trend analysis (France) • For each dekad, the z-score values of all pixels are averaged (one line for each dekad). • MARSOP BP 2 z 2 z 0 0 -2 z -2 z • BP shows a clear positive trend with time, not visible in MARSOP. Is this greening really happening? G2 Interim Review Meeting, JRC Ispra

25. Anomaly analysis, preliminary results • Example of application: detection of known droughts • Monthly averages of fAPAR over France 2003 (heat waves between May and August) • MARSOP BP April April May May Z-score 2 0 June June July July -2 August August Sept Sept • Good agreement between datasets, both see the anomaly, spatial pattern more plausible for BP G2 Interim Review Meeting, JRC Ispra

26. User utility assessment of Geoland2 BioPar products Wheat yield forecasting in Europe. Comparison of performances using G2 BioPar and MARSOP time series, preliminary results. KatarzynaDabrowska-Zielinska IGIK, Institute of Geodesy and Cartography,Warsaw(Poland)

27. Objective Test the performance of MARSOP and BioPar for wheat yield monitoring/forecasting in Europe Data RS: dekadal SPOT-VGT NDVI and fAPAR from HIST archive (1999-2009) Yield: Regional Agricultural Statistics Database of EUROSTAT

28. Methods Calibration of a Partial Least Square model tuned at NUTS2 level The explanatory variables are all the RS dekadal observations extracted from the growing season period as defined on the basis of an agro-climatic classification. Agro-climatic zones in Europe (Iglesias, A. et al., 2009) SOS & EOS Two modes of operation of the model: Monitoring mode: (yield estimation after EOS) all dekadal RS indices of growing season are available Forecasting mode: (yield estimation within season) unknown dekadal indices are set to their long term average values

29. Results in monitoring mode Yield estimates over 1999-2009, comparison BioPar, MARSOP and “null model” (mean yield): Cross-validation (Jackknifing) prediction errors (RMSE, MPE, MAPE) for agro-climatic zones Best performances are marked in yellow • The model doesn’t outperform the “null model” in all regions • Small performance differences using either MARSOP or BioPar

30. Results in monitoring mode Spatial distribution of the error (MPEs) • The largest errors in absolute terms are observed in Southwest of Europe and in the most northern region of Finland; • Again small performance differences using either MARSOP or BioPar.

31. Results in forecasting mode Example of forecast for year 2009 using data 1999-2008. Model performances (DecMAPE, mean absolute forecast error) and compared to the “null model” (the mean yield). • No substantial differences between MARSOP and BioPar • Forecasted yield performs better than simple average in few regions only (red bars shorter than the blue ones)

32. SUMMARY • No statistical differences in predicting wheat yield using either MARSOP or BioPar data. • The differences in crop yield prediction are minimal and in favour of BioPar [MARSOP] in monitoring [forecasting] mode; • Overall, poor performances of the model, especially when used in “forecasting mode”. This behaviour could be explained by the short RS time series available (11 years) and the huge gaps in EUROSTAT yield data; • Current activities: investigation of different methods for region grouping (period of forecast); collection of more ground truth EUROSTAT data