Contribution of safe to xfdu
1 / 54

Contribution of SAFE to XFDU - PowerPoint PPT Presentation

  • Uploaded on

Contribution of SAFE to XFDU. CCSDS/MOIMS/IPR Workshop (11-15/04/2005 Athens). Stéphane MBAYE GAEL Consultant [email protected] Rationale. SAFE Format Specifications 1.5 – [SAFE-FMT] ESA/CNES XFDU I/O Library – [XFDU-I/O] SAFE I/O library – [SAFE-I/O]

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Contribution of SAFE to XFDU' - theta

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Contribution of safe to xfdu
Contribution of SAFE to XFDU

CCSDS/MOIMS/IPR Workshop(11-15/04/2005 Athens)

Stéphane MBAYEGAEL [email protected]


  • SAFE Format Specifications 1.5 – [SAFE-FMT]

  • ESA/CNES XFDU I/O Library – [XFDU-I/O]

  • SAFE I/O library – [SAFE-I/O]

  • XFDU Specifications – Issues and Proposals

Safe origin and goals reminder
SAFE - Origin and goals (reminder)

  • Standard Archive Format for Europe

  • Provide ESA with a single format for all EO data

    • All Level 0 (Raw) and Level 1

    • Level 2 (as far as applicable)

    • Ground Segment Auxiliary files)

  • Should become an ESA AIP

  • Candidate for SIP and DIP (TBV)

  • Shall be compatible with XFDU

Safe 111 data types to be supported
SAFE – 111 Data types to be supported


IRS P3 (1)MOS (1)

Nimbus (2)CZCS (2)

MOS (1)MESSR (1)

SPOT (3)HRV (1), HRVIR (1), HRG (1)

  • ENVISAT (59)ASAR (23), AATSR (5), DORIS (3), GOMOS (7), MERIS (13), MIPAS (6), MWR (1), RA2 (8), SCIAMACHY (5)

  • ERS (25)RA (4), AMI (12), ATSR (3), MWR (1), GOME (2), PRARE (3)

  • Landsat (3)MSS (1), TM (1), ETM+(1), RBV (0)

  • JERS (3)OPS (1), SAR (2)

  • Terra/Aqua (1)MODIS (1)

  • NOAA (6)AVHRR/TOVS (6)

  • Orbview (5)SeaWifs (5)

Safe a restriction of xfdu
SAFE, a restriction of XFDU

<!-- SAFE Root element type. The type of the SAFE root derives from XFDU root type. -->

<xsd:complexType name="safeType">


<xsd:restriction base="XFDUType">


<xsd:element form="unqualified" name="packageHeader«  type="safe:packageHeaderType"

minOccurs="1« /> 

<xsd:element form="unqualified" name="informationPackageMap" type="safe:informationPackageMapType"

maxOccurs="1« /> 

<xsd:element form="unqualified" name="metadataSection" type="safe:metadataSectionType"

minOccurs="1« /> 

<xsd:element form="unqualified" name="dataObjectSection" type="dataObjectSectionType"

minOccurs="0« /> 

<xsd:element form="unqualified" name="behaviorSection" type="behaviorSectionType"

minOccurs="0« />





<!-- SAFE Root element -->

<xsd:element name="safe" type="safe:safeType"/>

Xml schemas namespace policy
XML Schemas / Namespace policy


Safe information package map section
SAFE Information Package Map Section

One and only one Information Package Map Section

Safe core metadata objects
SAFE – Core Metadata Objects

  • Acquisition Period Object

  • Platform Objects

  • Processing Log Objects

  • Quality Information Objects

  • Orbit Reference Objects

  • FrameSet Objects

  • XML Schemas Objects

Safe acquisition period object
SAFE - Acquisition Period Object

Safe specialized core objects
SAFE - Specialized Core Objects

<!-- Platform description -->

<metadataObject ID="platform"

classification="DESCRIPTION" category="DMD">

<mdWrap textInfo="Platform Description"

vocabluaryMdType="OTHER" otherMdType="SAFE"




<!-- Platform identification -->



<!-- Instrument identification -->



<meris:mode vcid="6" opsMode="4"/>






Safe quality information
SAFE – Quality Information

Use of XPath/XQuery foridentifying the corrupted/

Missing areas of theData Objects

<!-- Quality information of the measurements -->

<metadataObject ID="measurementQualityInformation"

classification="DESCRIPTION" category="DMD">

<mdWrap textInfo="Quality Information"

vocabluaryMdType="OTHER" otherMdType="SAFE"




<!-- 24 records are missing from after the 1233rd record -->





<!-- The records 456 to 468 have a bad application ID -->


mdsr[456 to 468]/isp/pkHd/PkId/appId






Safe xml schema objects
SAFE – XML Schema Objects

  • All Data Objects shall have an associated XML Schemas

    • Post-processing (sub-setting, reformatting, …)

    • Annotation (quality information, cloud cover)

    • Indexing

    • Documentation / Presentation

Safe preservation of binary objects
SAFE - Preservation of Binary Objects

  • Annotated XML Schemas

  • Structured Data File (SDF) markups



<xsd:element name="record" type="xs:int"

minOccurs="0" maxOccurs="unbounded">




<sdf:occurrence query="../header/numRec + 1"/>

<sdf:length unit=‘bit’>12</sdf:length>







Sdf example of binary interpreter derby
SDF - Example of Binary interpreter (Derby)

Detailed content of an ENVISAT MERIStelemetry Source Packet(CCSDS standard)described by SDF

Active sensors metadata objects
Active Sensors Metadata Objects

  • Pulse Repetition Frequency

  • Polarization

  • Pulse Length

Safe behavior objects
SAFE Behavior Objects

  • No SAFE Behavior Object has been defined

  • Some may be added for configuring Stitching?

  • Should be investigated with CNES

Getting xfdu i o library web site
Getting XFDU I/O library –Web site

Xfdu i o basic principles
XFDU I/O Basic principles

  • Based on interactions with the MANIFEST file

  • Each interaction is not automatically comitted (possible rollback): the result shall be explicitely saved

  • 100% Java

  • C++ Wrapper based on Java Native Interface (JNI)

  • No support of Behavior section (to be investigated with CNES)

Xfdu i o implemented usecases
XFDU I/O Implemented Usecases

  • Creation of a new XFDU packet

  • Opening of an existing XFDU packet

  • Removal of a XFDU packet

  • Moving a XFDU packet (with or without the attached objects)

  • Management of the Content Units

  • Management of Metadata Objects (only XML wrap)

  • Management of Data Objects (only Flocat references)

  • Browse Data Object Content (if associated XML Schema)

  • Validation of XFDU packet

  • Dumping XFDU Manifest in XML

Xfdu i o configuration status
XFDU I/O – Configuration Status

  • Configuration identifiers:

    • safe-1-0-beta-2: first functional release

    • safe-1-0-beta-1: API only

  • Package identifier: esa.xfdu (TBV CNES)

Safe i o basic principles cont
SAFE I/O Basic principles (Cont’)

  • Delegates standard XFDU access to ESA XFDU I/O

  • Based on interactions with the MANIFEST file

  • Each interaction is not automatically comitted (possible rollback): the result shall be explicitely saved

Safe i o implemented usecases
SAFE I/O Implemented Usecases

  • Creation of a new SAFE packet

  • Opening of an existing SAFE packet

  • Removal of a SAFE packet

  • Moving a SAFE packet (with or without the attached objects)

  • Management of the Content Units

  • Management of the Core Metadata Objects

  • Adding specialized Metadata

  • Management of Data Objects

  • Extraction or pointing to a subpart of a Data Object

  • Identifying missing units and corrupted areas

  • Browse Data Object Content

  • Validation of SAFE packet

  • Dumping SAFE Manifest in XML

Safe i o configuration status
SAFE I/O – Configuration Status

  • Configuration identifiers:

    • safe-1-0-release : an official release of version 1.0

    • safe-1-0-rc2 : second release candidate of version 1.0

    • safe-1-0-rc1 : a release candidate of version 1.0

    • safe-1-0-beta-1: unconsolidated distribution of version 1.0

  • Package identifier:

Xfdu specifications issues and proposals

XFDU SpecificationsIssues and Proposals

Xfdu environment information
XFDU – Environment Information

  • What is teh exact definition of the environmentInfo/@specVersion attribute?

  • May the environmentInfo contain package history (e.g. SAFE processing logs)?

Xfdu information package map
XFDU – Information Package Map

  • What is the presice role of contentUnit/@order ?

    • GAEL understanding: user defined order

    • CNES understanding: package sequence number?

  • A Map section ?shall? Reference all metadata/data objects of the package?

  • Meaning of multiple Map sections?

Xfdu single file packages
XFDU – Single file packages

  • For DIPs ESA requests single file products

  • ZIP, Jar or other standard archive format are not suitable for large data sets (e.g. 5 Gb for ENVISAT-ASAR format)

  • ESA proposes to append binary data objects to the MANIFEST file (not fully compliant with XML 1.0 or XML 1.0)

Xfdu how to design restrictions
XFDU – How to design restrictions?

  • The XML Schema requires global elements and attributes for restricting types in a different target namespace (E.g. SAFE)

  • May xfdu.xsd propose more global element for that purpose?

  • Should implementers redefine the xfdu.xsd?

  • May the XFDU specification provide a clarification/recommendation on that subject?

Xfdu metadata data object references
XFDU – Metadata/Data Object references

  • metadataObject/@mdRef =? bytestream/@FLocat

  • metadataObject/@mdWrap =? bytestream/@FContent

  • May these differences be homogeneized?

Xfdu checksums
XFDU – Checksums

  • Should the CRC moved or add to the Flocat element (i.e. one CRC per file)?

  • What is the difference between dataObject/@checksum and bytestream/@checksum?

Xfdu reference to xlink
XFDU – Reference to XLink

  • Requires an xlink.xsd in the same directory as the xfdu.xsd

  • References

    • ?

    • ?

  • Where to download the right XML Schema?

Xfdu typos
XFDU - Typos

  • vocabluaryMdType ?

  • Comments of all items of the enumerations: e.g.EAD?

  • mdSecType may be renamed metadataSectionType?

Xfdu package header
XFDU - Package Header

Xfdu information package map section
XFDU - Information Package Map Section

Xfdu metadata object section
XFDU - Metadata Object Section

Xfdu data object section
XFDU - Data Object Section

Xfdu behaviour object section
XFDU - Behaviour Object Section