Contribution of safe to xfdu
This presentation is the property of its rightful owner.
Sponsored Links
1 / 54

Contribution of SAFE to XFDU PowerPoint PPT Presentation


  • 85 Views
  • Uploaded on
  • Presentation posted in: General

Contribution of SAFE to XFDU. CCSDS/MOIMS/IPR Workshop (11-15/04/2005 Athens). Stéphane MBAYE GAEL Consultant [email protected] Rationale. SAFE Format Specifications 1.5 – [SAFE-FMT] ESA/CNES XFDU I/O Library – [XFDU-I/O] SAFE I/O library – [SAFE-I/O]

Download Presentation

Contribution of SAFE to XFDU

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript


Contribution of safe to xfdu

Contribution of SAFE to XFDU

CCSDS/MOIMS/IPR Workshop(11-15/04/2005 Athens)

Stéphane MBAYEGAEL [email protected]


Rationale

Rationale

  • SAFE Format Specifications 1.5 – [SAFE-FMT]

  • ESA/CNES XFDU I/O Library – [XFDU-I/O]

  • SAFE I/O library – [SAFE-I/O]

  • XFDU Specifications – Issues and Proposals


Safe format specifications 1 5 safe fmt

SAFE Format Specifications 1.5[SAFE-FMT]


Safe origin and goals reminder

SAFE - Origin and goals (reminder)

  • Standard Archive Format for Europe

  • Provide ESA with a single format for all EO data

    • All Level 0 (Raw) and Level 1

    • Level 2 (as far as applicable)

    • Ground Segment Auxiliary files)

  • Should become an ESA AIP

  • Candidate for SIP and DIP (TBV)

  • Shall be compatible with XFDU


Safe 111 data types to be supported

SAFE – 111 Data types to be supported

NOAA (6)AVHRR/TOVS (6)

IRS P3 (1)MOS (1)

Nimbus (2)CZCS (2)

MOS (1)MESSR (1)

SPOT (3)HRV (1), HRVIR (1), HRG (1)

  • ENVISAT (59)ASAR (23), AATSR (5), DORIS (3), GOMOS (7), MERIS (13), MIPAS (6), MWR (1), RA2 (8), SCIAMACHY (5)

  • ERS (25)RA (4), AMI (12), ATSR (3), MWR (1), GOME (2), PRARE (3)

  • Landsat (3)MSS (1), TM (1), ETM+(1), RBV (0)

  • JERS (3)OPS (1), SAR (2)

  • Terra/Aqua (1)MODIS (1)

  • NOAA (6)AVHRR/TOVS (6)

  • Orbview (5)SeaWifs (5)


Safe a restriction of xfdu

SAFE, a restriction of XFDU

<!-- SAFE Root element type. The type of the SAFE root derives from XFDU root type. -->

<xsd:complexType name="safeType">

<xsd:complexContent>

<xsd:restriction base="XFDUType">

<xsd:sequence>

<xsd:element form="unqualified" name="packageHeader«  type="safe:packageHeaderType"

minOccurs="1« /> 

<xsd:element form="unqualified" name="informationPackageMap" type="safe:informationPackageMapType"

maxOccurs="1« /> 

<xsd:element form="unqualified" name="metadataSection" type="safe:metadataSectionType"

minOccurs="1« /> 

<xsd:element form="unqualified" name="dataObjectSection" type="dataObjectSectionType"

minOccurs="0« /> 

<xsd:element form="unqualified" name="behaviorSection" type="behaviorSectionType"

minOccurs="0« />

</xsd:sequence>

</xsd:restriction>

</xsd:complexContent>

</xsd:complexType>

<!-- SAFE Root element -->

<xsd:element name="safe" type="safe:safeType"/>


Xml schemas namespace policy

XML Schemas / Namespace policy

http://www.esa.int/safe/1.0

http://www.esa.int/safe/1.0/core-1.0.xsd

http://www.esa.int/safe/1.0/passive-sensor-1.0.xsd

http://www.esa.int/safe/1.0/active-sensor-1.0.xsd

http://www.esa.int/safe/1.0/impl/xxx

Specializations…


Safe information package map section

SAFE Information Package Map Section

One and only one Information Package Map Section


Safe core metadata objects

SAFE – Core Metadata Objects

  • Acquisition Period Object

  • Platform Objects

  • Processing Log Objects

  • Quality Information Objects

  • Orbit Reference Objects

  • FrameSet Objects

  • XML Schemas Objects


Safe acquisition period object

SAFE - Acquisition Period Object


Safe platform object

SAFE – Platform Object


Safe specialized core objects

SAFE - Specialized Core Objects

<!-- Platform description -->

<metadataObject ID="platform"

classification="DESCRIPTION" category="DMD">

<mdWrap textInfo="Platform Description"

vocabluaryMdType="OTHER" otherMdType="SAFE"

mimeType="text/xml">

<xmlData>

<safe:platform>

<!-- Platform identification -->

<safe:nssdcId>2003-009A</safe:nssdcId>

<safe:familyName>ENVISAT</safe:familyName>

<!-- Instrument identification -->

<safe:instrument>

<safe:familyName>MERIS</safe:familyName>

<meris:mode vcid="6" opsMode="4"/>

</safe:instrument>

</safe:platform>

</xmlData>

</mdWrap>

</metadataObject>


Safe orbit reference object

SAFE – Orbit Reference Object


Safe processing log object

SAFE – Processing Log Object


Safe quality information

SAFE – Quality Information

Use of XPath/XQuery foridentifying the corrupted/

Missing areas of theData Objects

<!-- Quality information of the measurements -->

<metadataObject ID="measurementQualityInformation"

classification="DESCRIPTION" category="DMD">

<mdWrap textInfo="Quality Information"

vocabluaryMdType="OTHER" otherMdType="SAFE"

mimeType="text/xml">

<xmlData>

<safe:qualityInformation>

<!-- 24 records are missing from after the 1233rd record -->

<safe:missingUnits>

<safe:location>mdsr[1234]</safe:location>

<safe:count>24</safe:count>

</safe:missingUnits>

<!-- The records 456 to 468 have a bad application ID -->

<safe:corruptedUnits>

mdsr[456 to 468]/isp/pkHd/PkId/appId

</safe:corruptedUnits>

</safe:qualityInformation>

</xmlData>

</mdWrap>

</metadataObject>


Safe frame sets

SAFE – Frame Sets


Safe xml schema objects

SAFE – XML Schema Objects

  • All Data Objects shall have an associated XML Schemas

    • Post-processing (sub-setting, reformatting, …)

    • Annotation (quality information, cloud cover)

    • Indexing

    • Documentation / Presentation


Safe preservation of binary objects

SAFE - Preservation of Binary Objects

  • Annotated XML Schemas

  • Structured Data File (SDF) markups

<xsd:complexType>

<xsd:sequence>

<xsd:element name="record" type="xs:int"

minOccurs="0" maxOccurs="unbounded">

<xsd:annotation>

<xsd:appinfo>

<sdf:block>

<sdf:occurrence query="../header/numRec + 1"/>

<sdf:length unit=‘bit’>12</sdf:length>

</sdf:block>

</xsd:appinfo>

</xsd:annotation>

</xsd:element>

</xsd:sequence>

<xsd:complexType/>


Sdf hierarchy of data blocks

SDF – Hierarchy of data blocks


Sdf location of a block within the stream

SDF - Location of a block within the stream


Sdf physical representation of a node

SDF - Physical representation of a node


Sdf physical representation of a complex content

SDF - Physical representation of a complex content


Sdf markups

SDF - Markups


Sdf example of binary interpreter derby

SDF - Example of Binary interpreter (Derby)

Detailed content of an ENVISAT MERIStelemetry Source Packet(CCSDS standard)described by SDF


Active sensors metadata objects

Active Sensors Metadata Objects

  • Pulse Repetition Frequency

  • Polarization

  • Pulse Length


Safe behavior objects

SAFE Behavior Objects

  • No SAFE Behavior Object has been defined

  • Some may be added for configuring Stitching?

  • Should be investigated with CNES


Xfdu i o library xfdu i o

XFDU I/O library – [XFDU-I/O]


Xfdu i o general architecture

XFDU I/O - General architecture


Getting xfdu i o library web site

Getting XFDU I/O library –Web site

http://www.gael.fr/xfdu


Xfdu i o basic principles

XFDU I/O Basic principles

  • Based on interactions with the MANIFEST file

  • Each interaction is not automatically comitted (possible rollback): the result shall be explicitely saved

  • 100% Java

  • C++ Wrapper based on Java Native Interface (JNI)

  • No support of Behavior section (to be investigated with CNES)


Xfdu i o implemented usecases

XFDU I/O Implemented Usecases

  • Creation of a new XFDU packet

  • Opening of an existing XFDU packet

  • Removal of a XFDU packet

  • Moving a XFDU packet (with or without the attached objects)

  • Management of the Content Units

  • Management of Metadata Objects (only XML wrap)

  • Management of Data Objects (only Flocat references)

  • Browse Data Object Content (if associated XML Schema)

  • Validation of XFDU packet

  • Dumping XFDU Manifest in XML


Xfdu i o configuration status

XFDU I/O – Configuration Status

  • Configuration identifiers:

    • safe-1-0-beta-2: first functional release

    • safe-1-0-beta-1: API only

  • Package identifier: esa.xfdu (TBV CNES)


Safe i o library safe i o

SAFE I/O library – [SAFE-I/O]


Safe i o general architecture

SAFE I/O - General architecture


Getting safe i o library safe web site

Getting SAFE I/O library – SAFE Web site

http://www.gael.fr/safe


Safe i o basic principles cont

SAFE I/O Basic principles (Cont’)

  • Delegates standard XFDU access to ESA XFDU I/O

  • Based on interactions with the MANIFEST file

  • Each interaction is not automatically comitted (possible rollback): the result shall be explicitely saved


Safe i o implemented usecases

SAFE I/O Implemented Usecases

  • Creation of a new SAFE packet

  • Opening of an existing SAFE packet

  • Removal of a SAFE packet

  • Moving a SAFE packet (with or without the attached objects)

  • Management of the Content Units

  • Management of the Core Metadata Objects

  • Adding specialized Metadata

  • Management of Data Objects

  • Extraction or pointing to a subpart of a Data Object

  • Identifying missing units and corrupted areas

  • Browse Data Object Content

  • Validation of SAFE packet

  • Dumping SAFE Manifest in XML


Safe i o configuration status

SAFE I/O – Configuration Status

  • Configuration identifiers:

    • safe-1-0-release : an official release of version 1.0

    • safe-1-0-rc2 : second release candidate of version 1.0

    • safe-1-0-rc1 : a release candidate of version 1.0

    • safe-1-0-beta-1: unconsolidated distribution of version 1.0

  • Package identifier: esa.safe


Xfdu specifications issues and proposals

XFDU SpecificationsIssues and Proposals


Xfdu environment information

XFDU – Environment Information

  • What is teh exact definition of the [email protected] attribute?

  • May the environmentInfo contain package history (e.g. SAFE processing logs)?


Xfdu information package map

XFDU – Information Package Map

  • What is the presice role of [email protected] ?

    • GAEL understanding: user defined order

    • CNES understanding: package sequence number?

  • A Map section ?shall? Reference all metadata/data objects of the package?

  • Meaning of multiple Map sections?


Xfdu single file packages

XFDU – Single file packages

  • For DIPs ESA requests single file products

  • ZIP, Jar or other standard archive format are not suitable for large data sets (e.g. 5 Gb for ENVISAT-ASAR format)

  • ESA proposes to append binary data objects to the MANIFEST file (not fully compliant with XML 1.0 or XML 1.0)


Xfdu how to design restrictions

XFDU – How to design restrictions?

  • The XML Schema requires global elements and attributes for restricting types in a different target namespace (E.g. SAFE)

  • May xfdu.xsd propose more global element for that purpose?

  • Should implementers redefine the xfdu.xsd?

  • May the XFDU specification provide a clarification/recommendation on that subject?


Xfdu metadata data object references

XFDU – Metadata/Data Object references

  • [email protected] =? [email protected]

  • [email protected] =? [email protected]

  • May these differences be homogeneized?


Xfdu checksums

XFDU – Checksums

  • Should the CRC moved or add to the Flocat element (i.e. one CRC per file)?

  • What is the difference between [email protected] and [email protected]?


Xfdu reference to xlink

XFDU – Reference to XLink

  • Requires an xlink.xsd in the same directory as the xfdu.xsd

  • References

    • http://www.w3.org/TR/XLink ?

    • http://www.w3.org/1999/XLink ?

  • Where to download the right XML Schema?


Xfdu typos

XFDU - Typos

  • vocabluaryMdType ?

  • Comments of all items of the enumerations: e.g.EAD?

  • mdSecType may be renamed metadataSectionType?


Thank you for your attention

Thank you for your attention !


Xfdu xml schema overview

XFDU – XML Schema Overview


Xfdu package header

XFDU - Package Header


Xfdu information package map section

XFDU - Information Package Map Section


Xfdu metadata object section

XFDU - Metadata Object Section


Xfdu data object section

XFDU - Data Object Section


Xfdu behaviour object section

XFDU - Behaviour Object Section


  • Login