1 / 39

GRIP: Interoperability between UNICORE and Globus

GRIP: Interoperability between UNICORE and Globus. D. Erwin, M. Rambadt, Ph. Wieder Zentralinstitut für Angewandte Mathematik Forschungszentrum Jülich Terena 2002, Limerick, Ireland June 5, 2002 - Session 7A. Contents. GRIP Partners UNICORE and Globus GRIP Objectives

bat
Download Presentation

GRIP: Interoperability between UNICORE and Globus

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. GRIP: Interoperability between UNICORE and Globus D. Erwin, M. Rambadt, Ph. Wieder Zentralinstitut für Angewandte Mathematik Forschungszentrum Jülich Terena 2002, Limerick, Ireland June 5, 2002 - Session 7A

  2. Contents GRIP Partners UNICORE and Globus GRIP Objectives The GRIP Architecture Experience with an early prototype GRIP and OSGA Terena 2002

  3. GRIP: GRid Interoperability Project Funded in part by EU grant IST 2001-32257 Duration: January 2002 – December 2003 Budget: 1.9 Mio € URL: www.interoperability-project.org Terena 2002

  4. Forschungszentrum Jülich, D (Coordinator) • Pallas GmbH, D • University of Manchester, UK • University of Warsaw, PL • Deutscher Wetterdienst, D • University Southampton, UK • Fecit, UK • Argonne National Laboratory, US Foils created by partners of the GRIP project Partners Terena 2002

  5. GRIP Partners UNICORE and Globus GRIP Objectives The GRIP Architecture Experience with an early prototype GRIP and OSGA Terena 2002

  6. UNICORE Goals UNICORE: UNiform Interface to COmputing Resources • conceived prior to the ‘invention’ of the GRID • UNICORE has both a modest and an ambitious goal: • Create • Seamless • Secure • Intuitive access to distributed German HPC resources • Create a system that can be used in production at the German HPC centers Terena 2002

  7. PC² ZIB FZJ DWD TUD Pallas RUS RUKa LRZ LRZ: SR 8000, VPP RUS: Hitachi, SX-5 FZJ: 2xT3E, T90, SV1 ZIB: T3E, SGIDWD: T3E, IBM SP RUKA: IBM SP PC²: hpcLine TUD: SGI Pallas: Sun fecit: VPP, Sun Terena 2002

  8. UNICORE Functions • Creation of system-independent jobs • Execution at different sites • Creation of complex interdependent multi-system and multi-site jobs • Monitoring and control of jobs • Dependencies • Workflow Terena 2002

  9. UNICORE Functions • Transfer of data to and from the workstation • Automatic data transfer between systems and sites • Access to data in Unix file systems and archives • Application support • No changes to third party applications • Support for legacy job scripts • Support for Metacomputing • Performance analysis using VAMPIR Terena 2002

  10. UNICORE Functions • Single sign-on • No changes to local user management and naming conventions • Secure communication over the Internet • Support for all platforms, operating systems, and batch systems at partner sites • Retaining of administrative autonomy and security policies of participating sites Terena 2002

  11. UNICORE Functions • Continued development and support after the end of the project • Basis for additional national and international projects • Production quality prototype Objects will be achieved by year end 2002 URL: www.unicore.de Terena 2002

  12. EUROGRID • European Grid Testbed • Based on UNICORE technology • Development of additional UNICORE functions • Funded in part by EU grant IST-1999-29247 • Duration: November 2000 - October 2003 URL: www.eurogrid.org Terena 2002

  13. Geography Terena 2002

  14. EUROGRID Partner • Forschungszentrum Jülich, D • Pallas GmbH, D(Project Coordinator) • Parallab - University of Bergen, N • CNRS - IDRIS, F • Warsaw University - ICM, PL • Victoria University of Manchester, UK • Deutscher Wetterdienst, D • GIE EADS CCR, F • ETH Zürich (CSCS Manno), CH • Fecit, UK • T-Systems, D Terena 2002

  15. Globus • Development by Argonne National Laboratory and other US partners • Defines Protocols and APIs for GRIDs • Toolbox to enable development of GRID aware applications • Basis for many international Grid projects Terena 2002

  16. Grid Architecture Application Layer Collective Layer Resource Layer Fabric Layer Applications, Portals, Environments Higher Services, APIs, Protocols APIs, Protocols, SDKs Computers, Networks, Devices Terena 2002

  17. Grid Architecture Application Layer Collective Layer Resource Layer Fabric Layer Applications, Portals, Environments Higher Services, APIs, Protocols Globus APIs, Protocols, SDKs Computers, Networks, Devices Terena 2002

  18. Grid Architecture Legion Application Layer Collective Layer Resource Layer Fabric Layer Applications, Portals, Environments Higher Services, APIs, Protocols APIs, Protocols, SDKs Computers, Networks, Devices Terena 2002

  19. Grid Architecture UNICORE Application Layer Collective Layer Resource Layer Fabric Layer Applications, Portals, Environments Higher Services, APIs, Protocols APIs, Protocols, SDKs Computers, Networks, Devices Terena 2002

  20. Grid Architecture UNICORE Application Layer Collective Layer Resource Layer Fabric Layer Applications, Portals, Environments Higher Services, APIs, Protocols Globus APIs, Protocols, SDKs Computers, Networks, Devices Terena 2002

  21. GRIP Partners UNICORE and Globus GRIP Objectives The GRIP Architecture Experience with an early prototype GRIP and OSGA Terena 2002

  22. GRIP Objectives • Develop software to facilitate interoperation between UNICORE and Globus combining the unique strength of each system (Pallas, fecit, UoM, FZJ, ANL) • Build and demonstrate biomolecular (MD & QC codes) and meteorological (RLM) inter-grid applications (ICM, DWD, SOTON) • Create wrappers to support commercial applications in a Grid (no code changes) • Contribute to and influence international Grid standards through the Global Grid Forum Terena 2002

  23. GRIP Objectives • Run a UNICORE job on Globus resources • Run Jobs using Globus and UNICORE resources together • Prototype proposed standards in future versions of UNICORE Terena 2002

  24. Key UNICORE Technologies • Seamless Computing Model • Job abstraction • Incarnation • File staging and transfer support • Security Model • X509 certificate based security model • Consigner plus endorser model • Several CA policies • Fully flexible firewall support • Generic Client • No changes to application needed • Handy GUI environment • Portable Server • Java + Perl Terena 2002

  25. Parallels and Contrasts • Resource Description • UNICORE: Same model for discovery and request • Globus: Different models for discovery and request • Jobs vs Applications • UNICORE: Workflow environment • Globus: Application APIs and toolkit • Security • UNICORE: End-to-End security model • Globus: Requires transitive trust • Incarnation and Grounding • UNICORE: Incarnation of abstractions at server • Globus: Client side grounding (substitution to ‘normal form’)? • Protocol Complexities • UNICORE: Polling (request/response) • Globus: Call back Terena 2002

  26. Missing Technologies • Missing Models in UNICORE • Distributed directory services • Multi-language Implementations • Application level message passing • Event management • Synchronized job start • Implementation to be completed in UNICORE • Resource reservation • Interactive processing • Dynamic support for new Vsites on the fly Terena 2002

  27. GRIP Partners UNICORE and Globus GRIP Objectives The GRIP Architecture Experience with an early prototype GRIP and OSGA Terena 2002

  28. Network Job Supervisor Network Job Supervisor Network Job Supervisor Batchsystem Batchsystem Batchsystem Batchsystem Target System Interface Target System Interface Batch SubSystem Batch SubSystem UNICORE Architecture User Workstation UNICORE GUI Site List HTTP SSL UNICORE Server UNICORE Server Gateway Gateway SSL Network Job Supervisor Network Job Supervisor Network Job Supervisor TCP/IP TCP/IP UNICORE Site 1 UNICORE Site n Terena 2002

  29. User@Workstation UNICORE Client X.509User Certificate Job Preparation Agent(JPA) Job Monitor Controller(JMC) Abstract jobs, status requests Authentication UNICORE Login optional firewall Server@Usite Site-specificauthentication Gateway User validation Network Job Supervisor (NJS)(Incarnation/Scheduling) Network Job Supervisora (NJS) (Incarnation/Scheduling) Localuser DB Localuser DB Incarnation DB Incarnation DB optional firewall batch jobs, status requests, data system1 systemn hostsin a Vsite TS Interface TS Interface batch subsystema batch subsystemx UNICORE Architecture Terena 2002

  30. Client grid-proxy-init plugin Gateway (Logical) Globus Usite Generic Globus IDB Vsite A NJS Vsite B NJS Globus TSI IDB Host A IDB Host B Globus TSI IDB Maker Globus JM Host A Globus GIIS Globus JM Host B GRIS GRIS Proposed Architecture Terena 2002

  31. GRIP Partners UNICORE and Globus GRIP Objectives The GRIP Architecture Experience with an early prototype GRIP and OSGA Terena 2002

  32. GRAM job request Batch Job Callback Data Data Request Creation Enhanced Target System Interface Terena 2002

  33. Simplified Architecture implemented by M. Rambadt as part of his Master thesis: • No changes to UNICORE or Globus • Fixed Globus resource as a Visite • Extended TSI interfaces UNICORE and Globus: • To create proxy certificates • To map AJO to RSL • To acts as Globus client • To return results and status from Globus to NJS and Client  Proof of concept Experience Terena 2002

  34. GRIP Partners UNICORE and Globus GRIP Objectives The GRIP Architecture Experience with an early prototype GRIP and OSGA Terena 2002

  35. Outlook • First GRIP deliverable (end 2002) will interface UNICORE and Globus 2.0 • Project partners work with GGF on future developments • OGSA is an important topic • OSGA will be considered by GRIP project for 2003 (requires agreement by EU) Terena 2002

  36. Outlook • Architecturally OGSA and UNICORE are very compatible • WDSL service types for AJO classes • Semi-automatic translation for Web Services • More work for OGSA extension • Demonstrated by Dave Snelling in Manchester • Authorization and Security • Not yet fully defined in OGSA • UNICORE expects X.509 to Unix login mapping Terena 2002

  37. Outlook • UNICORE will continue to contribute to Grid developments • Grid development will benefit from experience gained at UNICORE Test and Production sites • Accepted standards with proven implementations are essential Terena 2002

  38. Web addresses: • http://www.unicore.deInformation about the project • http://www.fz-juelich.de/unicore-testFree UNICORE Client Software • http://www.unicore.orgDownload for Software + Sources • http://www.eurogrid.orgInformation about EUROGRID • http://www.grid-interoperability.orgInformation about GRIP Terena 2002

  39. http://www.unicore.de http://www.fz-juelich.de/unicore-test http://www.eurogrid.org http://www.grid-interoperability.org ? Terena 2002

More Related