eumetsat unified meteorological archive and retrieval facility and services u marf
Download
Skip this Video
Download Presentation
EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF)

Loading in 2 Seconds...

play fullscreen
1 / 16

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF) - PowerPoint PPT Presentation


  • 119 Views
  • Uploaded on

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF). Franck Cade (U-MARF Project Manager) ([email protected]). Outline. An introduction to U-MARF: What is U-MARF ? Project drivers and objectives Development approach & schedule overview

loader
I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
capcha
Download Presentation

PowerPoint Slideshow about ' EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF)' - armand


An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.


- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
eumetsat unified meteorological archive and retrieval facility and services u marf

EUMETSAT Unified Meteorological Archive and Retrieval Facility and Services (U-MARF)

Franck Cade (U-MARF Project Manager)([email protected])

outline
Outline
  • An introduction to U-MARF: What is U-MARF ?
    • Project drivers and objectives
    • Development approach & schedule overview
    • U-MARF technical overview
    • Performances issues
  • U-MARF standard services : How can users gain access to it ?
    • U-MARF standard services overview
    • Access to U-MARF data: Product search and ordering
    • U-MARF distribution formats
    • User documents
  • Conclusions
1 what is the u marf
1. What is the U-MARF ?
  • An introduction to U-MARF: what is U-MARF ?
    • Project drivers and objectives
    • Development approach & schedule overview
    • U-MARF technical overview
u marf project drivers and objectives
U-MARF Project drivers and objectives
  • EUMETSAT decision in 1997 to develop a multi-mission, archiving and retrieval facility, including on-line web user access capability, supporting all EUMETSAT programmes: the Unified Meteorological Archive and Retrieval Facility “U-MARF”
  • Offer integrated user access to the whole of EUMETSAT data products and services:
    • GEO data :
      • Meteosat Transition Programme (MTP) (including historical archive from METEOSAT 1 to 6),
      • Meteosat Second Generation (MSG-1 to MSG-3)
    • LEO data : EUMETSAT Polar System (EPS) (METOP satellites),
    • Plus multi-mission products ( Level 2 and above) from Satellite Application Facilities (SAF)
  • Exploit and take advantage of new technologies (e.g. archiving, on-line web access)
  • Built in flexibility and scalability to accommodate future new data and archive growth potential
  • Objective is to provide access to a wide range of external users (from novice users, to expert meteorological and climate communities)
u marf development approach
U-MARF Development approach
  • U-MARF procured from industry : contract allocated to Consortium including Matra Systemes & Information (F) as Prime, with ACS (I) approved by EUMETSAT Council in November 1998
  • Incremental development performed in two major versions:
    • U-MARF V1:
      • phased on MSG development schedule
      • providing common multi-mission functions and services for MSG and MTP data and products
      • MTP data to be migrated from current MARF to U-MARF in 3 years minimum
    • U-MARF V2 :
      • phased on EPS development schedule
      • extending services to EPS data sets and implementing connectivity with the SAF
u marf schedule overview
U-MARF Schedule Overview
  • U-MARF Development Milestones::
    • Project start Early January 1998
    • Invitation To Tender July 1998
    • V1 Industrial development kick-off Mid-December 1998
    • End of V1 development Planned December 2001
    • End of integration with GS Planned May 2002
    • V2 Industrial development kick-off Planned January 2002
    • End of V2 development Planned March 2004
  • U-MARF services milestones:
    • Ingestion, archiving and cataloguing MSG data: operational for MSG-1 launch
    • Standard user services : fully operational after end of MSG-1 commissioning
    • End of migration of MTP data: MSG-1 launch + 3 years (TBC)
u marf technical overview 1 3
U-MARF Technical Overview (1/3)

Monitoring

& Control

MSG

Front-End

MTP

Front-End

MSG Ground

Segment

EPS Ground

Segment

MTP MARF

EPS

Front-End

SAF

SAF Local

User

Services

SAF Local Archive

&Catalogue

U-MARF

Facility

External Users

UMARF Catalogue Management

UMARF Storage Management

Product

Formatting &

Delivery

UMARF User services

External Users

External Users

  • User Services
  • Product formatting & delivery
  • Monitoring & Control
  • Ground Segment Front-End
  • Storage management
  • Catalogue management
u marf technical overview 2 3
U-MARF Technical Overview (2/3)
  • U-MARF Functions:
    • Ingestion - Archiving & cataloguing chain:
      • Ingestion from Ground Segments (processing facilities), and Remote Centres
      • Archiving & Cataloguing
    • Retrieval - Ordering & Delivery chain :
      • User Services
      • Product Formatting and Delivery
    • Monitoring & Control & Administration:
      • Monitoring & Control
      • User administration
  • Excluded functions:
    • Processing, [Re-processing]
performances issues
Performances Issues
  • Data ingestion:
    • For V1: capacity to ingest 3 MSG data stream in parallel : 180 Gbytes/day - 2.4 Mbytes/s
    • For V2 :capacity to ingest 2.2 EPS data stream in parallel: 420 Gbytes/day - 5.5 Mbytes/s
  • User data retrieval :
    • For V1: 100 Gbytes/day
    • For V2: 600 Gbytes/day then 1.2 Tbytes/day
  • Capacity requirement:
    • see next
  • On-line user connections:
    • For V1: capacity to support at least 10 simultaneous users for on-line data retrieval for V1
    • For V2: capacity to support at least 30 simultaneous users for on-line data retrieval
  • Availability of data for user retrieval
    • Data accessible to users 2 hours after ingestion into the U-MARF
    • SAF catalogue records accessible to users 3 hours after ingestion
u marf archive ingestion and accumulation
U-MARF Archive ingestion and accumulation
  • MTP: ~ 15 Terabytes/year => total of ~ 45 Terabytes
  • MSG: ~25 Terabytes/year => total of ~300 Terabytes
  • EPS: total of ~100 Terabytes/year => total of ~ 1.5 Petabytes
u marf technical overview 3 3 summary
U-MARF Technical Overview (3/3)Summary
  • Characteristics & constraints summary:
    • Technical features:
      • Near-real time ingestion constraints from processing facilities
      • Throughput performances: ingestion + retrieval
    • Operational aspects:
      • Drastic availability & reliability constraints: 10 minutes downtime
      • Automated archive and automated production for retrieval including for media delivery to users (small robots/stackers providing 48 autonomy of operations)
  • Open points:
    • Inter-operability features with other EO providers I.e. CEOS,…
    • User feedback on user services definition and performances
2 how can users gain access to u marf
2. How can users gain access to U-MARF?
  • U-MARF standard services presentation : How can users gain access to U-MARF data?
    • U-MARF standard services overview
    • Access to U-MARF data: Product search (Inventory & Browse) and ordering
    • U-MARF distribution formats
    • User documents
u marf standard user services
U-MARF standard User Services
  • U-MARF services will be integrated with EUMETSAT user services and accessed from the EUMETSAT home page (www.eumetsat.de)
  • U-MARF Core Services :
    • Information and Guide: aiming at providing users with general information and documents (about U-MARF, products characteristics, file formats, …)
    • Product search (Inventory and Browse): allowing users to search for archived products (simple or advanced queries) and to obtain information (metadata , thumbnails, browse images) on the selected products
    • Ordering and Follow-up: allowing users to order selected products (further to Inventory search services)
  • U-MARF Supporting Auxiliary Services:
    • User Registration : allowing a user to log-in as known user or to request for a registration (user account and user profile)
    • User Services Administration: users account management, EUM data policy implementation, invoicing management...
    • Help Desk: comprehensive help desk services supported by automated functions
access to u marf data product search ordering
Access to U-MARF data: product search & ordering
  • Inventory search
    • V1/V2 search features => GEO and common search features:
      • Temporal search criteria
      • Metadata search criteria :satellite, instrument, product type, processing level, version of algorithm, quality indicators
    • V2 additional LEO specific features: orbit, geographical search
  • Ordering features:
    • Elementary orders (on archived products) or Standing Orders (subscription on future selected products in a pre-defined time range)
    • Product subsetting capabilities:
        • spectral bands subsetting
        • geographical subsetting (ROI for GEO image products, orbit subset for LEO orbit based products)
        • specific subsetting (header and/or trailer extraction)
    • Delivery options :
      • Electronic (ftp)
      • Physical media delivery : media includes : CD-ROM, [DVD], DAT, Exabyte, DLT
      • High quality printouts
u marf distribution formats
U-MARF Distribution Formats
  • U-MARF V1 Distribution Format:
    • Native format (as generated by MTP or MSG Ground Segment )
    • Band Sequential Format for SEVIRI Level 1.5
    • Open MTP for:
      • Level 1.0 and Level 1.5
      • Cloud Analysis, Cloud Motion Winds, Sea Surface Temperature, Upper Tropospheric Humidity, Climate Data Set
    • WMO / BUFR for MTP MET Products and for the following MSG MET Products :
      • Atmospheric Motion Vectors (AMV), Cloud Analysis (CLA) , Clear Sky Radiance (CSR), Tropospheric Humidity (TH), Climate Data Set (CDS), Total Ozone (TOZ)
    • JPEG for METEOSAT and SEVIRI level 1.5
    • Other Distribution Formats (TBD): including formats requested by last STG OPS WG :McIdas, and other evolutions of the MTP MARF
  • Additional U-MARF V2 Distribution Formats:
    • Native format (as generated by EPS Ground Segment )
    • WMO formats (BUFR/GBRIB) for all EPS products
    • HDF (Hierarchical Data Format) for MTP, MSG and EPS products
    • PNG (Portable Network Graphics) for MTP, MSG and EPS products
conclusions
Conclusions
  • U-MARF is part of a EUMETSAT effort to :
    • build an affordable and maintainable multimission infrastructure by exploiting commonalty between the different programmes
    • Enhance user access to EUMETSAT data products and services, as they become relevant to a wider range of user communities
ad