Contribution of safe to xfdu
This presentation is the property of its rightful owner.
Sponsored Links
1 / 54

Contribution of SAFE to XFDU PowerPoint PPT Presentation


  • 97 Views
  • Uploaded on
  • Presentation posted in: General

Contribution of SAFE to XFDU. CCSDS/MOIMS/IPR Workshop (11-15/04/2005 Athens). Stéphane MBAYE GAEL Consultant [email protected] Rationale. SAFE Format Specifications 1.5 – [SAFE-FMT] ESA/CNES XFDU I/O Library – [XFDU-I/O] SAFE I/O library – [SAFE-I/O]

Download Presentation

Contribution of SAFE to XFDU

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Contribution of SAFE to XFDU

CCSDS/MOIMS/IPR Workshop(11-15/04/2005 Athens)

Stéphane MBAYEGAEL [email protected]


Rationale

  • SAFE Format Specifications 1.5 – [SAFE-FMT]

  • ESA/CNES XFDU I/O Library – [XFDU-I/O]

  • SAFE I/O library – [SAFE-I/O]

  • XFDU Specifications – Issues and Proposals


SAFE Format Specifications 1.5[SAFE-FMT]


SAFE - Origin and goals (reminder)

  • Standard Archive Format for Europe

  • Provide ESA with a single format for all EO data

    • All Level 0 (Raw) and Level 1

    • Level 2 (as far as applicable)

    • Ground Segment Auxiliary files)

  • Should become an ESA AIP

  • Candidate for SIP and DIP (TBV)

  • Shall be compatible with XFDU


SAFE – 111 Data types to be supported

NOAA (6)AVHRR/TOVS (6)

IRS P3 (1)MOS (1)

Nimbus (2)CZCS (2)

MOS (1)MESSR (1)

SPOT (3)HRV (1), HRVIR (1), HRG (1)

  • ENVISAT (59)ASAR (23), AATSR (5), DORIS (3), GOMOS (7), MERIS (13), MIPAS (6), MWR (1), RA2 (8), SCIAMACHY (5)

  • ERS (25)RA (4), AMI (12), ATSR (3), MWR (1), GOME (2), PRARE (3)

  • Landsat (3)MSS (1), TM (1), ETM+(1), RBV (0)

  • JERS (3)OPS (1), SAR (2)

  • Terra/Aqua (1)MODIS (1)

  • NOAA (6)AVHRR/TOVS (6)

  • Orbview (5)SeaWifs (5)


SAFE, a restriction of XFDU

<!-- SAFE Root element type. The type of the SAFE root derives from XFDU root type. -->

<xsd:complexType name="safeType">

<xsd:complexContent>

<xsd:restriction base="XFDUType">

<xsd:sequence>

<xsd:element form="unqualified" name="packageHeader«  type="safe:packageHeaderType"

minOccurs="1« /> 

<xsd:element form="unqualified" name="informationPackageMap" type="safe:informationPackageMapType"

maxOccurs="1« /> 

<xsd:element form="unqualified" name="metadataSection" type="safe:metadataSectionType"

minOccurs="1« /> 

<xsd:element form="unqualified" name="dataObjectSection" type="dataObjectSectionType"

minOccurs="0« /> 

<xsd:element form="unqualified" name="behaviorSection" type="behaviorSectionType"

minOccurs="0« />

</xsd:sequence>

</xsd:restriction>

</xsd:complexContent>

</xsd:complexType>

<!-- SAFE Root element -->

<xsd:element name="safe" type="safe:safeType"/>


XML Schemas / Namespace policy

http://www.esa.int/safe/1.0

http://www.esa.int/safe/1.0/core-1.0.xsd

http://www.esa.int/safe/1.0/passive-sensor-1.0.xsd

http://www.esa.int/safe/1.0/active-sensor-1.0.xsd

http://www.esa.int/safe/1.0/impl/xxx

Specializations…


SAFE Information Package Map Section

One and only one Information Package Map Section


SAFE – Core Metadata Objects

  • Acquisition Period Object

  • Platform Objects

  • Processing Log Objects

  • Quality Information Objects

  • Orbit Reference Objects

  • FrameSet Objects

  • XML Schemas Objects


SAFE - Acquisition Period Object


SAFE – Platform Object


SAFE - Specialized Core Objects

<!-- Platform description -->

<metadataObject ID="platform"

classification="DESCRIPTION" category="DMD">

<mdWrap textInfo="Platform Description"

vocabluaryMdType="OTHER" otherMdType="SAFE"

mimeType="text/xml">

<xmlData>

<safe:platform>

<!-- Platform identification -->

<safe:nssdcId>2003-009A</safe:nssdcId>

<safe:familyName>ENVISAT</safe:familyName>

<!-- Instrument identification -->

<safe:instrument>

<safe:familyName>MERIS</safe:familyName>

<meris:mode vcid="6" opsMode="4"/>

</safe:instrument>

</safe:platform>

</xmlData>

</mdWrap>

</metadataObject>


SAFE – Orbit Reference Object


SAFE – Processing Log Object


SAFE – Quality Information

Use of XPath/XQuery foridentifying the corrupted/

Missing areas of theData Objects

<!-- Quality information of the measurements -->

<metadataObject ID="measurementQualityInformation"

classification="DESCRIPTION" category="DMD">

<mdWrap textInfo="Quality Information"

vocabluaryMdType="OTHER" otherMdType="SAFE"

mimeType="text/xml">

<xmlData>

<safe:qualityInformation>

<!-- 24 records are missing from after the 1233rd record -->

<safe:missingUnits>

<safe:location>mdsr[1234]</safe:location>

<safe:count>24</safe:count>

</safe:missingUnits>

<!-- The records 456 to 468 have a bad application ID -->

<safe:corruptedUnits>

mdsr[456 to 468]/isp/pkHd/PkId/appId

</safe:corruptedUnits>

</safe:qualityInformation>

</xmlData>

</mdWrap>

</metadataObject>


SAFE – Frame Sets


SAFE – XML Schema Objects

  • All Data Objects shall have an associated XML Schemas

    • Post-processing (sub-setting, reformatting, …)

    • Annotation (quality information, cloud cover)

    • Indexing

    • Documentation / Presentation


SAFE - Preservation of Binary Objects

  • Annotated XML Schemas

  • Structured Data File (SDF) markups

<xsd:complexType>

<xsd:sequence>

<xsd:element name="record" type="xs:int"

minOccurs="0" maxOccurs="unbounded">

<xsd:annotation>

<xsd:appinfo>

<sdf:block>

<sdf:occurrence query="../header/numRec + 1"/>

<sdf:length unit=‘bit’>12</sdf:length>

</sdf:block>

</xsd:appinfo>

</xsd:annotation>

</xsd:element>

</xsd:sequence>

<xsd:complexType/>


SDF – Hierarchy of data blocks


SDF - Location of a block within the stream


SDF - Physical representation of a node


SDF - Physical representation of a complex content


SDF - Markups


SDF - Example of Binary interpreter (Derby)

Detailed content of an ENVISAT MERIStelemetry Source Packet(CCSDS standard)described by SDF


Active Sensors Metadata Objects

  • Pulse Repetition Frequency

  • Polarization

  • Pulse Length


SAFE Behavior Objects

  • No SAFE Behavior Object has been defined

  • Some may be added for configuring Stitching?

  • Should be investigated with CNES


XFDU I/O library – [XFDU-I/O]


XFDU I/O - General architecture


Getting XFDU I/O library –Web site

http://www.gael.fr/xfdu


XFDU I/O Basic principles

  • Based on interactions with the MANIFEST file

  • Each interaction is not automatically comitted (possible rollback): the result shall be explicitely saved

  • 100% Java

  • C++ Wrapper based on Java Native Interface (JNI)

  • No support of Behavior section (to be investigated with CNES)


XFDU I/O Implemented Usecases

  • Creation of a new XFDU packet

  • Opening of an existing XFDU packet

  • Removal of a XFDU packet

  • Moving a XFDU packet (with or without the attached objects)

  • Management of the Content Units

  • Management of Metadata Objects (only XML wrap)

  • Management of Data Objects (only Flocat references)

  • Browse Data Object Content (if associated XML Schema)

  • Validation of XFDU packet

  • Dumping XFDU Manifest in XML


XFDU I/O – Configuration Status

  • Configuration identifiers:

    • safe-1-0-beta-2: first functional release

    • safe-1-0-beta-1: API only

  • Package identifier: esa.xfdu (TBV CNES)


SAFE I/O library – [SAFE-I/O]


SAFE I/O - General architecture


Getting SAFE I/O library – SAFE Web site

http://www.gael.fr/safe


SAFE I/O Basic principles (Cont’)

  • Delegates standard XFDU access to ESA XFDU I/O

  • Based on interactions with the MANIFEST file

  • Each interaction is not automatically comitted (possible rollback): the result shall be explicitely saved


SAFE I/O Implemented Usecases

  • Creation of a new SAFE packet

  • Opening of an existing SAFE packet

  • Removal of a SAFE packet

  • Moving a SAFE packet (with or without the attached objects)

  • Management of the Content Units

  • Management of the Core Metadata Objects

  • Adding specialized Metadata

  • Management of Data Objects

  • Extraction or pointing to a subpart of a Data Object

  • Identifying missing units and corrupted areas

  • Browse Data Object Content

  • Validation of SAFE packet

  • Dumping SAFE Manifest in XML


SAFE I/O – Configuration Status

  • Configuration identifiers:

    • safe-1-0-release : an official release of version 1.0

    • safe-1-0-rc2 : second release candidate of version 1.0

    • safe-1-0-rc1 : a release candidate of version 1.0

    • safe-1-0-beta-1: unconsolidated distribution of version 1.0

  • Package identifier: esa.safe


XFDU SpecificationsIssues and Proposals


XFDU – Environment Information

  • What is teh exact definition of the [email protected] attribute?

  • May the environmentInfo contain package history (e.g. SAFE processing logs)?


XFDU – Information Package Map

  • What is the presice role of [email protected] ?

    • GAEL understanding: user defined order

    • CNES understanding: package sequence number?

  • A Map section ?shall? Reference all metadata/data objects of the package?

  • Meaning of multiple Map sections?


XFDU – Single file packages

  • For DIPs ESA requests single file products

  • ZIP, Jar or other standard archive format are not suitable for large data sets (e.g. 5 Gb for ENVISAT-ASAR format)

  • ESA proposes to append binary data objects to the MANIFEST file (not fully compliant with XML 1.0 or XML 1.0)


XFDU – How to design restrictions?

  • The XML Schema requires global elements and attributes for restricting types in a different target namespace (E.g. SAFE)

  • May xfdu.xsd propose more global element for that purpose?

  • Should implementers redefine the xfdu.xsd?

  • May the XFDU specification provide a clarification/recommendation on that subject?


XFDU – Metadata/Data Object references

  • [email protected] =? [email protected]

  • [email protected] =? [email protected]

  • May these differences be homogeneized?


XFDU – Checksums

  • Should the CRC moved or add to the Flocat element (i.e. one CRC per file)?

  • What is the difference between [email protected] and [email protected]?


XFDU – Reference to XLink

  • Requires an xlink.xsd in the same directory as the xfdu.xsd

  • References

    • http://www.w3.org/TR/XLink ?

    • http://www.w3.org/1999/XLink ?

  • Where to download the right XML Schema?


XFDU - Typos

  • vocabluaryMdType ?

  • Comments of all items of the enumerations: e.g.EAD?

  • mdSecType may be renamed metadataSectionType?


Thank you for your attention !


XFDU – XML Schema Overview


XFDU - Package Header


XFDU - Information Package Map Section


XFDU - Metadata Object Section


XFDU - Data Object Section


XFDU - Behaviour Object Section


  • Login