1 / 20

Implementation of E2EDM Technology for WIS Development – Insights from NODC Meeting, Russia

The E2EDM (End-to-End Data Management) project was showcased at the recent NODC meeting in Russia, focused on integrating various marine meteorological and oceanographic data sources. Developed as a pilot project under the JCOMM/ETDMP initiative, E2EDM seeks to enhance data accessibility and interoperability across disciplines. The meeting presented functionalities such as local data wrapping, real-time data retrieval, and climate data mapping. Key representatives from the Met Office (UK), IFREMER (France), and NODC (Russia) contributed to discussions on operationalizing this technology for better data management in the North Atlantic region.

rosa
Download Presentation

Implementation of E2EDM Technology for WIS Development – Insights from NODC Meeting, Russia

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GISC & DCPC developers meeting E2EDM technology implementation for WIS GISC development S. Sukhonosov, S. Belov Langen, 29 – 31 March NODC, Russia

  2. History VLIZ, Belgium • E2EDM was originated as one of the JCOMM/ETDMP pilot projects in 2003 MetOffice,UK IFREMER, France NODC, Russia NODC, Russia

  3. Pilot project data sources The following data sources are involved in the E2EDM prototype • Historical marine meteorological data • Historical ocean cruise data • Real-time GTS ocean (BATHY and TESAC) data • Real-time GTS ocean (TESAC/ARGO data) • Real-time GTS (SHIP) data • Monthly climatic fields of ocean parameters (imageries) • Ocean SST satellite data Geographic area – North Atlantic (test-area) NODC, Russia

  4. Scope E2EDM provides following functionality: • Wrapping local data by means of E2EDM components • Mapping mechanisms (codes, structures) • Data search and retrieving from distributed data sources on user requests (real-time and scheduled) • Data obtaining and further processing, visualization and exchange E2EDM is integrating operational and delay-mode time scale data; across oceanographic and marine meteorological disciplines; from multiple data source formats and multiple data providers in different geographic regions. NODC, Russia

  5. Basis • Concept model • Namespace • Data exchange protocol NODC, Russia NODC, Russia

  6. E2EDM concept model • Providing formalization, identification, description and interpretation for model objects. Model is based on the following semantic objects: • Element • Class (set of elements) • Record (set of classes) • In terms of the model each technological object is described by means of listed semantic objects. E.g. object called “Resource” is record, which contains features – classes. Each resource record (or description) contains granularity structures for the source data, e.g. first level of granularity – platform, second level – date and time, third – measurements (sea surface temperature, water salinity, etc.). NODC, Russia NODC, Russia

  7. E2EDM namespace • Defines metadata and data structures. Within E2EDM consists of two sub-namespaces: global and conceptual (thematic). • Global namespace – structures for the model objects • Conceptual (thematic domain) namespace – list of thematic elements dealing with some discipline, e.g. meteorology, and available for discovery and exchange. • Both based on XML Schema. • Global XML Schema (ISO 19115 + WMO Core Metadata) • Concept XML Schema(DiGIR) NODC, Russia NODC, Russia NODC, Russia

  8. E2EDM data exchange protocol Data exchange protocol is responsible for searching and data retrieving. Request message (XML, DiGIR-protocol) Response message (XML, DiGIR-protocol) Transport data file (NetCDF) NODC, Russia NODC, Russia NODC, Russia NODC, Russia

  9. Request message example <request xmlns="http://digir.net/schema/protocol/2003/1.0" xmlns:e2edm="http://data.meteo.ru/digir/schema/conceptual/cruise" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://digir.net/schema/protocol/2003/1.0 http://digir.net/schema/protocol/2003/1.0/digir.xsd http://data.meteo.ru/digir/schema/conceptual/cruise http://data.meteo.ru/digir/schema/conceptual/cruise/pilot2cruise.xsd"> <header> <version>0.95</version> <sendTime>2005-08-03T14:47:52+03:00</sendTime> <source>127.0.0.1</source> <destination resource="RU_RIHMI_01">http://data.meteo.ru:80/digir/DiGIR.php</destination> <type>search</type> </header> <search> <filter> <and> <lessThanOrEquals> <e2edm:Latitude>78.54</e2edm:Latitude> </lessThanOrEquals> </and> </filter> <records limit="100" start="0"> <structure> <xsd:element name="record" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> <xsd:complexType> <xsd:sequence> <xsd:element ref="e2edm:dateTime"/> <xsd:element ref="e2edm:DEPH"/> <xsd:element ref="e2edm:TEMP"/> </xsd:sequence> </xsd:complexType> </xsd:element> </structure> </records> <count>false</count> </search> </request> NODC, Russia

  10. Response message example <responseWrapper> <response xmlns="http://digir.net/schema/protocol/2003/1.0"> <header> <sendTime>2005-09-29T15:35:17+04:00</sendTime> <source resource="RU_RIHMI_03"/> <type>search</type> </header> <content> <record> <E2ETransportSpecification> <transportFileURL>http://data.meteo.ru:8090/dpms/ncResults/1127993716743-result.nc</transportFileURL> </E2ETransportSpecification> <count>7</count> </record> </content> </response> </responseWrapper> NODC, Russia

  11. E2EDM prototype components • E2EDM prototype specifications -- structures, metadata/data formats, codes and other means of data unification from the local data systems • E2EDM Prototype software – E2EDM Data Provider system and E2EDM Integration Server NODC, Russia

  12. Diagram of prototype • Functionality: • Integration server • Data discovery (through resource catalog) • Monitoring of distributed data sources • Request/reply mechanism • Data distribution from the multiple data sources • Metadata synchronization • Data Provider • Request handling • Access to the local data source (DBMS, files) • Data extraction based on search criteria • Transport data file generation NODC, Russia

  13. General scheme User Interface layer (HTTP, Web-services) reply request Integration Server Transport data file Response message Request message Transport data file Data Provider Data Provider Data Provider NODC, Russia

  14. Integration Server user web-interface Resource catalogue Parameters Additional search options NODC, Russia

  15. Integration Server user web-interface (cont.) Search status NODC, Russia

  16. Integration Server user web-interface (cont.) Point data profile data grid data • E2EDM Web site http://data.meteo.ru/e2edm/ • E2EDM Integration Server - IODE Project Office (VLIZ) http://e2edm.vliz.be - RIHMI-WDC (mirror) http://data.meteo.ru:8080/iserv/ NODC, Russia

  17. Current status • Released technical specification document • Builded software framework using SOA • Extended community of national Data Providers • Solved performance problems (e.g. response time) • Implemented caching mechanism • Defined user policy NODC, Russia

  18. Future… • It is important to develop administrative services for component management and maintenance • To build up predefined queries, push/pull and notification mechanisms • System metadata mapping into WMO Core Metadata Profile • To develop web-services in order to improve compatibility with other systems NODC, Russia

  19. Thank you! Questions? NODC, Russia

  20. NODC, Russia

More Related