improving long term preservation eos data by independently mapping hdf4 data objects n.
Download
Skip this Video
Loading SlideShow in 5 Seconds..
Improving long-term preservation EOS data by independently mapping HDF4 data objects PowerPoint Presentation
Download Presentation
Improving long-term preservation EOS data by independently mapping HDF4 data objects

Loading in 2 Seconds...

play fullscreen
1 / 18

Improving long-term preservation EOS data by independently mapping HDF4 data objects - PowerPoint PPT Presentation


  • 102 Views
  • Uploaded on

Improving long-term preservation EOS data by independently mapping HDF4 data objects. Mike Folk, Ruth Aydt, Peter Cao, Kent Yang Ruth Duerr Christopher Lynnes. Phase 2: Productizing HDF4 Mapping schema and tools for deployment. Phase 2 tasks.

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about 'Improving long-term preservation EOS data by independently mapping HDF4 data objects' - vanig


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
improving long term preservation eos data by independently mapping hdf4 data objects

Improving long-term preservation EOS data by independently mapping HDF4 data objects

Mike Folk, Ruth Aydt, Peter Cao, Kent YangRuth DuerrChristopher Lynnes

phase 2 tasks
Phase 2 tasks
  • Investigate integration of mapping schema with existing standards
  • Determine HDF-EOS 2 requirements
  • Redesign the XML schema
  • Implement production quality HDF4 map writer
  • Develop demo HDF4 map reader and optional general reader
  • Develop test suite and validation utility for layouts and datatypes
  • Deploy tools at other NASA data centers.

Annual HDF Briefing to ESDIS

hdf4 hdf eos2 mapping workflow
HDF4/HDF-EOS2 mapping workflow

Annual HDF Briefing to ESDIS

task a investigate integration of mapping schema with existing standards

Task AInvestigate integration of mapping schema with existing standards

Annual HDF Briefing to ESDIS

task a
Task A
  • Status: complete except for report
  • Results
    • Standards investigated included METS, PREMIS, ESML and NcML
    • None really meet our needs.
    • PREMIS is about the file but doesn’t address the contents.
    • MIX approach harmonizes with it.
    • Will try MIX approach, but it doesn’t
    • For contents, it will deviate from PREMIS
    • Elements will align with corresponding subset of PREMIS
    • We have content that is a stretch for PREMIS
    • Conclusion: We will not adopt any particular standard, but will try to harmonize with them and leverage familiarity with other things, such as the use of the Common Data Model (CDM).

Annual HDF Briefing to ESDIS

task b determine hdf eos2 requirements

Task B: Determine HDF-EOS2 requirements

Annual HDF Briefing to ESDIS

categorize hdf eos2 data products
Categorize HDF-EOS2 data products
  • Status: Complete
  • Results:
    • Create a data pool
      • Data from GES DISC, NSIDC, LAADS, LP-DAAC and LaRC
    • Detailed description of sample data
    • Investigate HDF-EOS2 Swath, Grid and Point
    • Summarize the requirements
    • (next slide)

Annual HDF Briefing to ESDIS

requirements for hdf eos2 objects
Requirements for HDF-EOS2 objects
  • Results (continued)
    • Report to listing all options for adding HDF-EOS2 contents to the mapping file
    • Documents and reports under the wiki page:

http://wiki.hdfgroup.org/MappingPhase2_TaskB

Annual HDF Briefing to ESDIS

task c redesign schema

Task CRedesign Schema

Annual HDF Briefing to ESDIS

task c redesign schema1
Task C: Redesign Schema
  • Duration: Begin in January; Finish in June
  • Assigned to Ruth
  • Activities:

1) Gather background information and knowledge:

2) Formulate overall schema design and identify corner cases

3) Detailed Schema design

4) Documentation

Annual HDF Briefing to ESDIS

task d implement writer

Task DImplement Writer

Annual HDF Briefing to ESDIS

task d implement writer1
Task D: Implement Writer
  • Duration: Began in March; Finish in Sept
  • Assigned to: Binh-Minh Ribler and Joe Lee
  • Activities:
    • Implement API to for creating maps (Binh-Minh)
      • Develop new writer requirements, based on new XML schema and additional deployment needs
      • Design APIs
      • Implement new features
      • Document
    • Implement writer (Joe)
      • Design writer tool
      • Implement writer
      • Implement test suite for writer
      • Document

Annual HDF Briefing to ESDIS

status of task d
Status of Task D
  • Implement API to for creating maps
      • Requirements and design nearly complete
      • Some implementation has been completed
      • A bit behind schedule, but no threat to schedule
  • Implement writer (Joe)
      • Design process is underway
      • On schedule for completion by September

Annual HDF Briefing to ESDIS

task e implement demo reader
Task E: Implement demo reader
  • Duration: Begin in June, complete in Sept
  • Assigned to: Ruth Duerr
  • Activities:
    • Develop requirements, based on new schema and identification of additional deployment needs.
    • Design reader, based on requirements, and from a review of the prototype design.
    • Implement reader.
    • Document reader.
    • Test reader on EOS file “zoo”
    • Deposit reader, documentation, and tests in open source repository, probably SourceForge.

Annual HDF Briefing to ESDIS

task f implement validation utility
Task F: Implement validation utility

Optional task

No plans currently to perform

Annual HDF Briefing to ESDIS

task g deploy
Task G: Deploy
  • Duration: Begin in Jan 2010, complete in April
  • Assigned to: NSDC and GES DISC
  • Activities:
    • GES DISC
      • Incorporate into the existing archive ingest system
      • Manage the retrofit into existing metadata files
    • NSIDC
      • Implementation effort for the V0 data
      • Support implementation in NSIDC’s ECS system
    • Other ESDCs will be encouraged to join in the effort, but it is assumed that deployment to other centers will occur for the most part subsequently to the project.

Annual HDF Briefing to ESDIS