Eumetsat unified meteorological archive and retrieval facility and services u marf
1 / 16

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF) - PowerPoint PPT Presentation

  • Uploaded on
  • Presentation posted in: General

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF). Franck Cade (U-MARF Project Manager) ( Outline. An introduction to U-MARF: What is U-MARF ? Project drivers and objectives Development approach & schedule overview

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Download Presentation

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF)

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

Presentation Transcript

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF)

Franck Cade (U-MARF Project Manager)(


  • An introduction to U-MARF: What is U-MARF ?

    • Project drivers and objectives

    • Development approach & schedule overview

    • U-MARF technical overview

    • Performances issues

  • U-MARF standard services :How can users gain access to it ?

    • U-MARF standard services overview

    • Access to U-MARF data: Product search and ordering

    • U-MARF distribution formats

    • User documents

  • Conclusions

1. What is the U-MARF ?

  • An introduction to U-MARF: what is U-MARF ?

    • Project drivers and objectives

    • Development approach & schedule overview

    • U-MARF technical overview

U-MARF Project drivers and objectives

  • EUMETSAT decision in 1997 to develop a multi-mission, archiving and retrieval facility, including on-line web user access capability, supporting all EUMETSAT programmes: the Unified Meteorological Archive and Retrieval Facility “U-MARF”

  • Offer integrated user access to the whole of EUMETSAT data products and services:

    • GEO data :

      • Meteosat Transition Programme (MTP) (including historical archive from METEOSAT 1 to 6),

      • Meteosat Second Generation (MSG-1 to MSG-3)

    • LEO data : EUMETSAT Polar System (EPS) (METOP satellites),

    • Plus multi-mission products ( Level 2 and above) from Satellite Application Facilities (SAF)

  • Exploit and take advantage of new technologies (e.g. archiving, on-line web access)

  • Built in flexibility and scalability to accommodate future new data and archive growth potential

  • Objective is to provide access to a wide range of external users (from novice users, to expert meteorological and climate communities)

U-MARF Development approach

  • U-MARF procured from industry : contract allocated to Consortium including Matra Systemes & Information (F) as Prime, with ACS (I) approved by EUMETSAT Council in November 1998

  • Incremental development performed in two major versions:

    • U-MARF V1:

      • phased on MSG development schedule

      • providing common multi-mission functions and services for MSG and MTP data and products

      • MTP data to be migrated from current MARF to U-MARF in 3 years minimum

    • U-MARF V2 :

      • phased on EPS development schedule

      • extending services to EPS data sets and implementing connectivity with the SAF

U-MARF Schedule Overview

  • U-MARF Development Milestones::

    • Project start Early January 1998

    • Invitation To Tender July 1998

    • V1 Industrial development kick-off Mid-December 1998

    • End of V1 developmentPlanned December 2001

    • End of integration with GS Planned May 2002

    • V2 Industrial development kick-off Planned January 2002

    • End of V2 developmentPlanned March 2004

  • U-MARF services milestones:

    • Ingestion, archiving and cataloguing MSG data: operational for MSG-1 launch

    • Standard user services : fully operational after end of MSG-1 commissioning

    • End of migration of MTP data: MSG-1 launch + 3 years (TBC)

U-MARF Technical Overview (1/3)


& Control





MSG Ground


EPS Ground






SAF Local



SAF Local Archive




External Users

UMARF Catalogue Management

UMARF Storage Management


Formatting &


UMARF User services

External Users

External Users

  • User Services

  • Product formatting & delivery

  • Monitoring & Control

  • Ground Segment Front-End

  • Storage management

  • Catalogue management

U-MARF Technical Overview (2/3)

  • U-MARF Functions:

    • Ingestion - Archiving & cataloguing chain:

      • Ingestion from Ground Segments (processing facilities), and Remote Centres

      • Archiving & Cataloguing

    • Retrieval - Ordering & Delivery chain :

      • User Services

      • Product Formatting and Delivery

    • Monitoring & Control & Administration:

      • Monitoring & Control

      • User administration

  • Excluded functions:

    • Processing, [Re-processing]

Performances Issues

  • Data ingestion:

    • For V1: capacity to ingest 3 MSG data stream in parallel : 180 Gbytes/day - 2.4 Mbytes/s

    • For V2 :capacity to ingest 2.2 EPS data stream in parallel: 420 Gbytes/day - 5.5 Mbytes/s

  • User data retrieval :

    • For V1: 100 Gbytes/day

    • For V2: 600 Gbytes/day then 1.2 Tbytes/day

  • Capacity requirement:

    • see next

  • On-line user connections:

    • For V1: capacity to support at least 10 simultaneous users for on-line data retrieval for V1

    • For V2: capacity to support at least 30 simultaneous users for on-line data retrieval

  • Availability of data for user retrieval

    • Data accessible to users 2 hours after ingestion into the U-MARF

    • SAF catalogue records accessible to users 3 hours after ingestion

U-MARF Archive ingestion and accumulation

  • MTP: ~ 15 Terabytes/year=> total of ~ 45 Terabytes

  • MSG: ~25 Terabytes/year=> total of ~300 Terabytes

  • EPS: total of ~100 Terabytes/year => total of ~ 1.5 Petabytes

U-MARF Technical Overview (3/3)Summary

  • Characteristics & constraints summary:

    • Technical features:

      • Near-real time ingestion constraints from processing facilities

      • Throughput performances: ingestion + retrieval

    • Operational aspects:

      • Drastic availability & reliability constraints: 10 minutes downtime

      • Automated archive and automated production for retrieval including for media delivery to users (small robots/stackers providing 48 autonomy of operations)

  • Open points:

    • Inter-operability features with other EO providers I.e. CEOS,…

    • User feedback on user services definition and performances

2. How can users gain access to U-MARF?

  • U-MARF standard services presentation :How can users gain access to U-MARF data?

    • U-MARF standard services overview

    • Access to U-MARF data: Product search (Inventory & Browse) and ordering

    • U-MARF distribution formats

    • User documents

U-MARF standard User Services

  • U-MARF services will be integrated with EUMETSAT user services and accessed from the EUMETSAT home page (

  • U-MARF Core Services :

    • Information and Guide: aiming at providing users with general information and documents (about U-MARF, products characteristics, file formats, …)

    • Product search (Inventory and Browse): allowing users to search for archived products (simple or advanced queries) and to obtain information (metadata , thumbnails, browse images) on the selected products

    • Ordering and Follow-up: allowing users to order selected products (further to Inventory search services)

  • U-MARF Supporting Auxiliary Services:

    • User Registration : allowing a user to log-in as known user or to request for a registration (user account and user profile)

    • User Services Administration: users account management, EUM data policy implementation, invoicing management...

    • Help Desk: comprehensive help desk services supported by automated functions

Access to U-MARF data: product search & ordering

  • Inventory search

    • V1/V2 search features => GEO and common search features:

      • Temporal search criteria

      • Metadata search criteria :satellite, instrument, product type, processing level, version of algorithm, quality indicators

    • V2 additional LEO specific features: orbit, geographical search

  • Ordering features:

    • Elementary orders (on archived products) or Standing Orders (subscription on future selected products in a pre-defined time range)

    • Product subsetting capabilities:

      • spectral bands subsetting

      • geographical subsetting (ROI for GEO image products, orbit subset for LEO orbit based products)

      • specific subsetting (header and/or trailer extraction)

  • Delivery options :

    • Electronic (ftp)

    • Physical media delivery : media includes : CD-ROM, [DVD], DAT, Exabyte, DLT

    • High quality printouts

U-MARF Distribution Formats

  • U-MARF V1 Distribution Format:

    • Native format (as generated by MTP or MSG Ground Segment )

    • Band Sequential Format for SEVIRI Level 1.5

    • Open MTP for:

      • Level 1.0 and Level 1.5

      • Cloud Analysis, Cloud Motion Winds, Sea Surface Temperature, Upper Tropospheric Humidity, Climate Data Set

    • WMO / BUFR for MTP MET Products and for the following MSG MET Products :

      • Atmospheric Motion Vectors (AMV), Cloud Analysis (CLA) , Clear Sky Radiance (CSR), Tropospheric Humidity (TH), Climate Data Set (CDS), Total Ozone (TOZ)

    • JPEG for METEOSAT and SEVIRI level 1.5

    • Other Distribution Formats (TBD): including formats requested by last STG OPS WG :McIdas, and other evolutions of the MTP MARF

  • Additional U-MARF V2 Distribution Formats:

    • Native format (as generated by EPS Ground Segment )

    • WMO formats (BUFR/GBRIB) for all EPS products

    • HDF (Hierarchical Data Format) for MTP, MSG and EPS products

    • PNG (Portable Network Graphics) for MTP, MSG and EPS products


  • U-MARF is part of a EUMETSAT effort to :

    • build an affordable and maintainable multimission infrastructure by exploiting commonalty between the different programmes

    • Enhance user access to EUMETSAT data products and services, as they become relevant to a wider range of user communities

  • Login