Supporting the Validation of MODIS Land Products

1. Supporting the Validation ofMODIS Land Products Larry Voorhees, Dick Olson, Bob Cook, ORNL* DAAC Jeff Morisette, NASA GSFC John Dwyer, LP DAAC EOS Data Products Review NASA HQ March 10-11, 2003 This presentation should be viewed using ‘Slide Show’ with PowerPoint 2002 Much of the information in this image prepared by the NASA Goddard Space Flight Center came from MODIS. * Oak Ridge National Laboratory is managed by UT-Battelle, LLC, for the U.S. Dept. of Energy under contract DE-AC05-00OR22725.

2. Jeff Privette, Nazmi El Saleous, and Robert Wolfe, NASA GSFC Tom Kalvelage, LP DAAC Steve Running, U Montana Ranga Myneni, Boston U Alfredo Huete, U Arizona Dennis Baldocchi, UC Berkeley Warren Cohen and the BigFoot Team Many other Science Teams and EOS Validation Investigators Contributors and Participants

3. Outline • Validation Approach • Data Available for Validation • high resolution imagery • in situ data • User Statistics • Communications and Future Activities • What works, what’s ‘in the works’, and what needs to be improved?

4. Why validate global land products? • Good science and resource management require understanding product accuracy / uncertainty • Explicit statements of uncertainty foster an informed user community and improved use of data • International environmental agreements imply products may be independently evaluated and possibly challenged • As more, and similar, global products are produced by CEOS members, inter-use will require characterization of each product’s uncertainty

5. ? Field and tower data are combined with high resolution imagery to produce high resolution products. Correlate Aggregate Data Product the challenge . . . ORNL DAAC MODIS Scene Field Measurements LP DAAC Some graphics courtesy of BigFoot project

6. validation in ‘stages’ . . . Stage 2 Product accuracy has been assessed over a widely distributed set of locations and time periods via several ground-truth and validation efforts. Stage 3 Product accuracy has been assessed and the uncertainties in the product well established via independent measurements in a systematic and statistically robust way representing global conditions. Stage 1 Product accuracy has been estimated using a small number of independent measurements obtained from selected locations and time periods and ground-truth/field program effort.

7. EOS Land Validation Core Sites • 26 sites stratified by six biomes • Based on existing stations, resources, experts • FLUXNET, AERONET, BigFoot, LTER, etc. • Long-term monitoring • Jointly nominated by Instrument and Validation PIs • Remote Sensing Data • Landsat 5, ETM+, IKONOS, • ASTER, airborne sensors, • subsetted over the core sites • in situ Site Data • Collected by various groups • at core sites, available via Mercury • Networks • AERONET, FLUXNET, etc. • Ancillary Site Data/GIS Layers • - site variables • - elevation • - land cover • - reference layers (e.g., political • boundaries, airports, water • bodies)

8. in situ Data for Core Sites Remote Sensing Data for Core Sites

9. Landsat downloads from LP DAAC by Core Site Since October 1999 Number of files downloaded Year represents the date of the ftp delivery, not necessarily the year of data acquisition

10. MODIS downloads from LP DAAC by Core Site Since October 1999 LP DAAC also provides ftp access to MODIS and ETM+ over GOFC network, SAFARI 2000, and CEOS LAI-intercomparison sites, statistics not shown here Year represents the date of the ftp delivery, not necessarily the year of data acquisition

11. > 200 towers and growing www.daac.ornl.gov/ FLUXNET/fluxnet.html Over 100 site-years of data in-situ data from ORNL for comparison with MODIS land products;NPP example • FLUXNET(a collection of networks) • A global network of micrometeorological tower sites that measure the exchanges of CO2, water vapor, and energy between terrestrial ecosystem and atmosphere • Aggregated data available from ORNL • Ecosystem site characteristics for core sites • Historical data from literature • Net Primary Productivity (NPP) • point (~1,000) and grid (7,000 0.5 deg) • Leaf Area Index (~1000) • Regional and Global data sets • ORNL DAAC archive • Climate (7) • Vegetation (~100) • Soils (9) • Hydrology (3) • >180 data sets registered in Mercury (data held by others) regarding vegetation, land use, soil, climate, hydrology, etc.

12. FPAR for Walker Branch November 2000 – July 2002 Red – Median of 5x5 PixelBlue – Good QAGreen – Failed QA from Collection 3 from Collection 3 MODIS ASCII Subsets ( 7 x 7 km) • MODAPS at GSFC (NASA) provides 11x 31 km subsets of selected products in HDF and SIN Projection for 274 sites in Collection 4 • ORNL DAAC subsets to 7 x 7 km and reformats from HDF images into ASCII files of pixel values and QA flags, expanding as data are collected • ASCII files are posted on ORNL DAAC’s ftp site “. . . the MODIS fluxtower cutouts are the best damn thing we ever did. People [ecologists] down here [Australia] are piling on that data, and it is producing way more use of MODIS data than otherwise would have happened!!” -- Steve Running E-mail, Feb 26, 2003 --

13. Validation Support at ORNL: FY 2002 • Developed process to produce MODIS ASCII subsets • Tested and provided feedback on the MODIS Reprojection Tool • Documented MODIS products (Collection 3) and provided QA flags for subsets, and identified improvements for the existing documentation (3 products for 25 sites) • Participated in MODIS Vegetation and Albedo Workshops • Described processing and availability of MODIS ASCII subsets • Added 249 additional sites for validation (total 274) • More flux tower, field, and albedo sites • Provided data for validation via Mercury

14. Validation Support at ORNL: FY 2003 • Expanding number of sites and products for the MODIS subsets • Posting MODIS Collection 4: new algorithms, SIN projection • Supporting MODIS Validation Workshop – August 2003 • Expanding in-situ data collection – FLUXNET, Mercury MODIS ASCII Subsets: 8 products, 8-day & annual 274 sites, 7x7 km

15. User Statistics for Validation Data • Use of Mercury • Referrals to registered data and documentation • Data downloads of in situ measurements (ORNL DAAC) • Data downloads of remote sensing images of core sites (LP DAAC) – presented earlier

16. Mercury Searches 2001 2002 Validation Data through Mercury • 57 validation data sets registered in Mercury • Land Validation and SAFARI 2000 • User statistics:July 2001 – January 2003 • 1,409 users • 4,488 searches

17. Types of Referrals* July 1, 2001 – January 30, 2003 (953 referrals) *‘Referrals’ are links to where the data, documentation, and home pages reside.

18. Numerous user accolades . . . “I believe this to be the most outstanding archive and method of distribution around. The government and labs have a lot to be proud of here. Thanks” “I have been receiving information from ORNL during the last year, and I am really very grateful . . . Thanks.” “excellent web site !” ORNL DAAC User Statistics in CY 2002 • NPP, FLUXNET, and Climate Collections data are among the most often requested products • > 40% of all users request NPP data sets

19. Reports,e.g., • MODLAND “Special Issue” • Morisette, J. T., J.L. Privette, and C.O. Justice, A framework for the validation of MODIS land products”, Remote Sensing of Environment, 83 (1-2) 77-96, 2002. • Earth Observer report from the CEOS / WGCV Albedo workshop • “Summary of the International Workshop on Albedo Product Validation”Nov./Dec. 2002, v.14, n.6, p.17-18 (available on-line at http://eospso.gsfc.nasa.gov/eos_observ/11_12_02/nov_dec_02.pdf September 19, 2001 January 21, 2003 September 24, 2001 Communications • Publishing validation results • Web sites, e.g., • http://modis.gsfc.nasa.gov/MODIS/LAND/VAL/ • http://eosdatainfo.gsfc.nasa.gov/eosdata/ssinc/modland_dataprod.shtml • EMDI Workshop, April 2002 • SAFARI 2000 Meeting, October 2002 • ‘Popular articles’ for the general user community (Earth Observatory Web site) • Collaborating with CEOS / WGCV

20. Land Product Validation Subgroup • Increase the quality and economy of global satellite product validation • develop and promote international standards and protocols for field sampling, scaling, error budgeting, data exchange, and product evaluation • Advocate mission-long validation programs for current and future earth observing satellites Using EOS Land Validation Core Sites as an example to establish the expanded ‘CEOS Land Validation Core sites’ through a joint CEOS WGISS / WGCV activity • 2002 activities • LAI Intercomparison “on-line” • Fire/Burned area regional activities • Albedo validation workshop • Web-posted “Case studies”

21. Validation . . . what works • Growing buy-in to the ‘core sites’ concept • where the field, airborne, satellite data suite is helping to address “scaling issues” • Exploitation of in situ networks (e.g. flux towers, AERONET, LTER/ILTER) and field investigations (e.g., LBA, SAFARI 2000) • International partnerships and coordination, mainly through CEOS/IGOS • Simplified data access with dedicated archives and ‘easy to use’ data format (e.g., ASCII subsets) • Frequent communication between Product PIs, Validation PIs, and data facilitators • Best validation sites are those where investigators taking ground measurements are also interested in satellite products

22. Validation . . . what’s in the works • Data for Stage 2 validation of MODLAND products continue to be compiled and made available • the number of sites is growing through coordination with science networks and CEOS members • Improving formal documentation and making validation results more readily available to users • Users can provide important feedback over and above formal ‘validation investigators’(e.g., through DAAC surveys and user services, Fall 2003 Workshop) • CEOS LPV topical meetings on Global Land Cover Validation (April 2003 and Fall 2003) • FLUXNET Workshop, August 2003

23. Validation . . . on-going needs • Promote stronger interface to the user community (e.g., through Validation Workshops) • Develop stronger connection with the scientific community and NASA Roadmaps to establish how accurate the products need to be and how to relay products’ uncertainty so this information can be incorporated into how they are used • Continue to promote collaboration between NASA’s Cal/Val program and the CEOS Working Group on Calibration and Validation

24. Comments . . . . Questions? Much of the information in this image prepared by the NASA Goddard Space Flight Center came from MODIS.