James casey cern james casey@cern ch on behalf of eu datagrid wp2
1 / 12

James Casey - PowerPoint PPT Presentation

  • Updated On :

Next-Generation EU DataGrid Data Management Services. James Casey CERN [email protected] On behalf of EU DataGrid WP2. Talk Outline. Introdution to EU DataGrid workpackage 2 WP2 Service Design and Interactions Spitfire Replication Services Security Conclusions and outlook. Authors

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'James Casey' - Olivia

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript

Talk outline l.jpg
Talk Outline

  • Introdution to EU DataGrid workpackage 2

  • WP2 Service Design and Interactions

    • Spitfire

    • Replication Services

    • Security

  • Conclusions and outlook


Diana Bosio, James Casey, Akos Frohner, Leanne Guy, Wolfgang Hoschek, Peter Kunszt, Erwin Laure, Levi Lucio, Heinz Stockinger, Kurt Stockinger - CERN

Giuseppe Andronico, Federico DiCarlo, Andrea Domenici, Flavia Donno, Livio Salconi – INFN

William Bell, David Cameron, Gavin McCance, Paul Millar, Caitriona Nicholson – PPARC, University of Glasgow

Joni Hahkala, Niklas Karlsson, Ville Nenonen, Mika Silander, Marko Niinimaki – Helsinki Institute of Physics

Olle Mulmo, Gian Luca Volpato – Swedish Research Council

Grid middleware architecture hourglass l.jpg
Grid middleware architecture hourglass

Current Grid architectural functional blocks:

Specific application layer





Common application layer

Grid Application Services (LCG)

EU DataGrid middleware

High Level Grid Services


Basic Grid Services

OS, Storage & Network services

Eu datagrid wp2 data management work package l.jpg
EU DataGrid WP2Data Management Work Package

Responsible for

  • Transparent data location and secure access

  • Wide-area replication

  • Data access optimization

  • Metadata access

    NOTresponsible for (but partially relying on other WPs for)

  • Data storage

  • Proper Relational Database bindings

  • Remote I/O

  • Security infrastructure

Wp2 service paradigms l.jpg
WP2 Service Paradigms

  • Choice of technology:

    • Java-based servers using Web Services

      • Tomcat, Oracle 9iAS

    • Interface definitions in WSDL

    • Client stubs for many languages (Java, C, C++)

      • Axis, gSOAP

    • Persistent service data in Relational Databases

      • MySQL, Oracle

  • Modularity

    • Modular service design for pluggability and extensibility

    • No vendor specific lock-ins

  • Evolvable

    • Easy adaptation to OGSA foreseen, based on the same technology

    • Largely independent of underlying OS, RDBMS

Spitfire grid enabling rdbms l.jpg
Spitfire: Grid-enabling RDBMS

  • Capabilities:

    • Simple Grid enabled front end to any type of local or remote RDBMS through secure SOAP-RPC

    • Sample generic RDBMS methods may easily be customized with little additional development, providing WSDL interfaces

    • Browser integration

    • GSI authentication

    • Hooks in place for local authorization

  • Status: current version 2.1

    • Used by EU DataGrid Earth Observation and Biomedical applications.

    • Not suitable for the retrieval of LARGE result sets

Replication services basic functionality l.jpg
Replication Services: Basic Functionality

Each file has a unique Grid ID.

Locations corresponding to the GUID are kept in the Replica Location Service.

Users may assign aliases to the GUIDs. These are kept in the Replica Metadata Catalog.

Files have replicas stored at many Grid sites on Storage Elements.

Replica Metadata


Replica Location


Replica Manager

The Replica Manager provides atomicity for file operations, assuring consistency of SE and catalog contents.





Higher level replication services l.jpg
Higher Level Replication Services

The Replica Subscription Service issues Replication commands automatically, based on a set of subscription rules defined by the user.

Hooks for user-defined pre- and post-processing for replication operations are available.

Replica Subscription


The Replica Manager may call on the Replica Optimization service to find the best replica among many based on network and SE monitoring.

Replica Metadata


Replica Location


Replica Manager

Replica Optimization








Network Monitor

Interactions with other grid components l.jpg
Interactions with other Grid components

Virtual Organization

Membership Service

User Interface or

Worker Node

Resource Broker

Replica Subscription


Information Service

Replica Metadata


Replica Location


Replica Manager

Replica Optimization


Applications and users interface to data through the Replica Manager either directly or through the Resource Broker. Management calls should never go directly to the SE.







Network Monitor

Replication services status l.jpg
Replication Services Status

  • Current Status

    • All components are deployed right now

    • Initial tests show that expected performance can be met

    • Need proper testing in a ‘real user environment’ – EDG2; LCG1

  • Features for next release

    • Currently Worker Nodes need outbound connectivity – Replica Manager Proxy Service needed. Needs proper security delegation mechanism.

    • Logical collections support

    • Service-level authorization

    • Subscription Service does not handle individual users – due to missing delegation.

Security infrastructure for java based web services l.jpg
Security: Infrastructure for Java-based Web Services

  • Trust Manager

    • Mutual client-server authentication using GSI (ie PKI X509 certificates) for all WP2 services

    • Supports everything transported over SSL

  • Authorization Manager

    • Supports coarse grained authorization: Mapping user->role->attribute

    • Fine grained authorization through policies, role and attribute maps

    • Web-based Admin interface for managing the authorization policies and tables

  • Status:

    • Fully implemented, authentication is enabled on the service level

    • Delegation implementation needs to be finished

    • Authorization needs more integration, waiting for deployment of VOMS

Conclusions and outlook l.jpg
Conclusions and outlook

  • The second generation Data Management services have been designed and implemented based on the Web Service paradigm

  • Flexible, extensible service framework

  • Deployment choices : robust, highly available commercial products supported (eg. Oracle) as well as open-source (MySQL, Tomcat)

  • First experiences with these services show that their performance meets the expectations

  • Real-life usage will show its strengths and weaknesses on the LCG-1 and EDG2.0 testbeds during the rest of this year.

    Thanks to the EU and our national funding agencies for their support of this work