1 / 17

SIPAD New Generation:

SIPAD New Generation:. A multi-field system for accessing scientific data and added-value services. Dominique Heulet CNES 18, Avenue E.Belin 31401 Toulouse Cedex 9 France e-mail:Dominique.Heulet@cnes.fr. Introduction. SIPAD-NG What it is:

keagan
Download Presentation

SIPAD New Generation:

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. SIPAD New Generation: A multi-field system for accessing scientific data and added-value services Dominique HeuletCNES 18, Avenue E.Belin 31401 Toulouse Cedex 9 France e-mail:Dominique.Heulet@cnes.fr

  2. Introduction • SIPAD-NG • What it is: • a complete operational system for preserving and accessing data, • implements proven off-the-shelf products and a sophisticated data model, • meets the requirements of a data center with a long-term mission, • a centralized system with its own catalogue • Why it has been developed: • CNES wants to offer a complete generic software to be used as a basis for building data centers

  3. SIPAD-NG: follow-on of the SIPAD software • The SIPAD system • Developed in the 1990s for the CDPP • CDPP: Space Plasma Physics Data Centre • SIPAD enables users to retrieve data and metadata through a web server • SIPAD software designed to be used by any scientific discipline: • Plasma Physics (CDPP & CASSINI projects), • Oceanography (MERCATOR project), • Atmospheric Chemistry & Astronomy • Advantages of SIPAD: • SIPAD implements all the basic functions of a data access system • Open-ended Data Model based on a Data Dictionary allowing the control of delivered metadata • Disadvantages: • Limitations to implement new functions (e.g. interoperability), • Web user interface not flexible enough (not easily customizable)

  4. The Data Model: graphs • For each project, definition of the Data Model • First goal of the Data Model: organize the catalogue in order to make data search & selection by navigation easy • Definition of a data graph, a browse graph and a document graph • Data graph: storage objects (archived data – e.g. a ‘tar’ file –) data objects (data files seen by end users) data sets (groups of data objects – e.g. instrument level 1 data –) collections (groups of data sets or collections) • Browse graph: browse products, browse sets, browse collections • Document graph: documents, document collections • Definition of links between these graphs • Web server pages are automatically generated from the Data Model

  5. The Data Model: graphs Project Browse collection Data collection Document collection Data collection Data set Browse collection Browse set Document collection Data set Data set Data Object Browse set Browse product Document Document Data object Data object Browse product Browse product Storage object Storage object Storage object

  6. The Data Model: attributes • Elements of the graphs can be described by attributes • Generic attributes • Size of a data object • Number of data objects in a data set • Specific attributes according to scientific discipline • Plasma Physics: start time & stop time of an observation • Earth Observation: geographical coordinates • Use of attributes • To present information associated to data • To select data by criteria • The web server provides selection pages for entering criteria values

  7. The Data Model: attributes Project Mission Browse collection Mission documents Name Description Measured parameters etc … Experiment Browse set Experiment documents Dataset Id Level etc … Data set Browse product Document Title Author Data Object Id Start time Stop time Version etc … Selection criteria Data object

  8. SIPAD & SIPAD-NG common functions • Data search & selection • Navigation in the Data Model, • Search on criteria, • Selection of product browse • User management • Access rights, • Quotas • Order management • Data extraction from an archive, • Media production, • Network transfer, • User workspace

  9. New functions implemented in SIPAD-NG • Integration of Added Value Services (AVSs) • SIPAD-NG allows to: • associate AVSs to Data Model elements in order to propose them to users, • call AVSs by giving them parameters, • manage access rights and user quotas • SIPAD-NG implements a specific interface for: • Batch AVSs: process data which have already been ordered and deliver the result to the user (e.g. coordinate transformation), • Interactive AVSs: process data available in the user workspace(e.g. graphic representation of data from several data sets) • Interoperability • SIPAD-NG provides interfaces to send requests to or to receive requests from another Data Centre • Web Services & SOAP protocol

  10. Adaptability • Adaptation of the web server to the project needs • SIPAD-NG web server easily customizable to project needs • SIPAD-NG web server can be replaced by a project specific web server • Implementation in Data Centres located inside CNES • SIPAD-NG compliant with CNES Computer Centre services: • SUN SOLARIS operating system, • interfaces with CNES archiving and media production facilities • SIPAD-NG compliant with CNES security rules • Implementation in Data Centres located outside CNES • SIPAD-NG can run on a single machine or its components can be distributed over several machines • SIPAD-NG can run on both SOLARIS and LINUX operating systems

  11. Architectural design Catalogue Web Server Added Value Service Remote Data Centre Client applications Order Management User space Catalogue consultation Administration & Supervision SIPAD-NG kernel Basic services User management Ingestion Administration Web Server Local Data Centre services Administrator On line archive Near line archive Media production

  12. Client applications provided by SIPAD-NG • The SIPAD-NG software system is composed of: • a kernel: • the six basic services, • the Administration Web Server, • the data base • several client applications: • complementary to the basic services (e.g. automatic ingestion), • main client application: User Web Server • Characteristics of the SIPAD-NG User Web Server: • compliant with the six basic services interfaces, • easily customizable, • A project can replace it by its own web server and only reuse the communication layer with the basic services

  13. Implementation examples Time selection Geographic selection Oracle server CDPP catalogue Internet machine STAF Solaris server SEM SIPAD-NG User Web Server & Catalogue Consultation service Other SIPAD-NG services & Administration Web Server CDPP at CNES Computer Centre Linux server SIPAD-NG basic services SIPAD-NG user web server ICARE catalogue ICARE Data Centre File server On-line data

  14. Technology ‘Client’ layer User Web Server & Administration Web Server • JAVA • Apache Tomcat • Struts →MVC2 • Axis → Web Services ‘Application Logic’ layer • JAVA • RMI →Inter-machines • communication • XML →Ingestion & • Basic services • interfaces ‘Business’ layer Basic services ‘Object-Relational Mapping’ layer ‘Database’ layer ORACLE 9i

  15. Conclusion • Schedule • 2003: requirements, specification • February 2004: beginning of development • July 2005: first release • December 2005: final release • SIPAD-NG contractor: CS SI • SIPAD-NG users • those already using SIPAD • CDPP, MERCATOR, CASSINI • Migration of the catalogues from SIPAD to SIPAD-NG • Future potential users • ICARE: Data Centre for Clouds & Aerosols • SSALTO: Altimetry & Orbitography

More Related