1 / 20

HDF-EOS Overview and Status

HDF-EOS Overview and Status. Larry Klein Emergent Information Technologies, Inc. September 19, 2000 larry@eos.hitc.com. What is HDF-EOS?. An extension to HDF, built with HDF objects. A standard for data structures for remote sensing data and in-situ measurements with:

mallika
Download Presentation

HDF-EOS Overview and Status

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. HDF-EOS Overview and Status Larry Klein Emergent Information Technologies, Inc. September 19, 2000 larry@eos.hitc.com

  2. What is HDF-EOS? • An extension to HDF, built with HDF objects. • A standard for data structures for remote sensing data and in-situ measurements with: • Tightly coupled geolocation information • Access based on geolocation • Incorporates ECS metadata attributes. • Used for many EOS standard products.

  3. HDF-EOS File Contents • Global HDF attributes • Core Metadata - searchable attributes, copied in database • Archive Metadata - non-searchable attributes • Any number of HDF objects specifying HDF-EOS objects (Grid, Point, Swath) • Structural metadata - links geolocation to science data • Additional HDF objects.

  4. HDF-EOS Grid Structure • Usage - Data which is organized by regular geographic spacing, specified by projection parameters. • Structure • Any number of 2-D to 8-D data arrays per structure, one per data type (e.g. temperature) • Geolocation information contained in projection formula, coupled by structural metadata. • Any number of Grid structures per file allowed.

  5. Grid Data Example

  6. Grid Data: 1 Km. Digital Elevation Model South Pole

  7. HDF-EOS Swath Structure • Usage - Data which is organized by time, monotonic geolocation, track parameter. Spacing can be irregular. • Structure • Geolocation information stored explictly in Geolocation Field (2-D array) • Data stored in 2-D or 3-D arrays • Time stored in 1-D or 2-D array, TAI is standard. • Geolocation/science data connected by structural metadata

  8. Swath Data Example

  9. HDF-EOS Point Structure • Usage - Data which is specified temporally and/or spatially, but with no particular organization. • Structure • Tables (Vdatas) used to store science data at a particular Lat/Long/Height. • Up to eight levels of data allowed. Structural metadata specifies relationship between levels.

  10. Point Data Example

  11. HDF-EOS Interface Schematic UserApplication HDF-EOS Interface ECS Metadata Interface Grid Point Swath Low Level HDF File I/O

  12. User Interface • Access - Open, close objects/files • Definition - Set key features of data sets • Basic I/O - Read write data • Index I/O - Link tables (Point only) • Inquiry - Return information about data • Subset - Read data from specific region • Metadata access (SDP toolkit)

  13. Current Archive Holdings • GSFC DAAC: MODIS L1, Atmos./Ocean • 50 TB, 105,000 granules, 300 GB/day • will be ~1.3 TB / day with addition of MODIS L2 and EOS Aqua data. • EDC DAAC: Landsat/ASTER/MODIS Land • 20 TB, 106,000 granules, 274 GB/day • Langely DAAC: MISR • 9 TB, 39,000 granules, 88 GB/day

  14. Availability • HDF library - ftp hdf.ncsa.uiuc.edu • HDF-EOS V2.6 library (HDF4 based) • http://newsroom.gsfc.nasa.gov/sdptoolkit/toolkit.html • SDP Toolkit library - same location as HDF-EOS • Platform Support • DEC, HP, IBM, Sun, SGI, Windows, Linux (soon) • Language Support - C, C++, F77, F90 • HDF4-based library will be supported indefinitely

  15. HDF5 - Based Development • NCSA has introduced HDF5 • A complete rewrite of HDF4, a new user interface. • Simplified data model • Breaks 2 GB file size limit • Potential for parallel I/O, multi-threading

  16. HDF5 - Based Development HDF-EOS 3.0 • Development overview • Maintain current HDF-EOS interface to maximum extent • Allow all HDF5 features through interface • Build HDF-EOS (4 to 5) conversion function • Data production executable can be built with either or both HDF4 and HDF5 - based library • Support heterogeneous ECS archive (future)

  17. Design Schematic Common Functions Definition, Access, I/O, Inquiry, Subset User Application HDF-EOS Interface Grid, Point, Swath ECS Metadata Interface HDF4 Interface* HDF5 Interface Combined HDF4/5 Interface* Build Options *Implemented if required

  18. Applications • H5EOSView - EOSView for HDF5 • Java Browser - Browser for both HDF4 and 5 based granules • HDF-EOS 4 to 5 converter, HDF-EOS 2.x to HDF-EOS 3.x

  19. Schedule • Jan., 2000 - Beta Version of library • H5EOSView, Sun, SGI, C support • Oct. 2000 - Beta2 Version • Add F77, F90, DEC support • Local Attributes, ProfileInterface, Chunking/Tiling • HDF-EOS 4 to 5 converter and Java Browser

  20. Schedule (cont.) • Jan. 2001 - SCF version • add HP, IBM, Windows, Linux (maybe a few months later), C++ support • DAAC version (ECS Drop 6B) - Summer 2001. • Archives can contain both HDF4 and HDF5 - based granules. • ECS subsystems don’t crack granules, but regression testing needed. • 2 GB size limit is an issue.

More Related