Globus Toolkit 2

1. Globus Toolkit 2 Peter Kacsuk – Sipos Gergely MTA SZTAKI {kacsuk,sipos}@sztaki.hu

2. Progress in Grid Systems Client/server Cluster computing Super- computing Network Computing High-throughput computing High-performance computing Web Services 2nd Gen. Condor Globus OGSA/OGSI OGSA/WSRF Grid Systems

3. Client program moves to resource(s) Security is a serious problem! The Globus-2 model Resource description GIIS (MDS-2) Publish MDS-2 API (configuration description) Resource Resource requestor provider GRAM API

4. Solutions by Globus (GT-2) • Dynamic creation of Virtual Organizations (VOs) • Clients can directly choose resources • Standardprotocols are used to connect Globus sites • Security issues are basically solved • Firewalls are allowed between Grid sites • PKI: CAs and X.509 certificates • SSL for authentication and message protection • The client does not need account on every Globus site: • Proxies and delegation for secure single Sign-on • Still: • provides metacomputing facilities (MPICH-G2) • Not service-oriented either

5. Globus Layered Architecture Applications Application Toolkits MPICH-G2 DUROC globusrun Condor-G Nimrod/G GAT Basic Grid Services – Globus Toolkit 2 Replica Mngt GSI-FTP GRAM GASS GSI MDS-2 Grid Fabric Condor MPI TCP UDP DiffServ Solaris LSF PBS NQE Linux NT

6. Collaboration Tools Data Mgmt Tools Distributed simulation . . . Information services Resource mgmt Data mgmt . . . Remote access Remote monitor The Role of Grid Middleware and Tools net Credit to Ian Foster

7. Globus Approach A p p l i c a t i o n s • Focus on architecture issues • Provide implementations of grid protocols and APIs as basic infrastructure • Use to construct high-level, domain-specific solutions • Design principles • Keep participation cost low • Enable local control • Support for adaptation Diverse global services Core Globus services Local OS

8. Resource brokers, Resource co-allocators TCP, FTP, HTTP, etc. Internet protocol GRAM protocol Condor, LSF, NQE, LoadLeveler, etc. Ethernet, ATM, FDDI, etc. Globus Approach: Hourglass High-level services Low-level tools

9. GRAM Components MDS client API calls to locate resources Client MDS: Grid Index Info Server Site boundary MDS client API calls to get resource info GRAM client API calls to request resource allocation and process creation. MDS: Grid Resource Info Server Query current status of resource GRAM client API state change callbacks Globus Security Infrastructure Local Resource Manager Allocate & create processes Request Job Manager Create Gatekeeper Process Parse Monitor & control Process RSL Library Process

10. Resource Specification Language • Much of the power of GRAM is in the RSL • Common language for specifying job requests • A conjunction of (attribute=value) pairs • GRAM understands a well defined set of attributes

11. Resource A Resource B GRIS GRIS “Standard” MDS Architecture (v1.1.3) • Resources run a standard information service (GRIS) which speaks LDAP and provides information about the resource (no searching). • GIIS provides a “caching” service much like a web search engine. Resources register with GIIS and GIIS pulls information from them when requested by a client and the cache as expired. • GIIS provides the collective-level indexing/searching function. Client 1 Clients 1 and 2 request infodirectly from resources. Client 2 GIIS requests information from GRIS services as needed. Client 3 uses GIIS for searching collective information. Client 3 GIIS Cache contains info from A and B

12. GASS Architecture for file staging Submit machine Execution machine &(executable=https://…) main( ) { fd = globus_gass_open(…) … read(fd,…) … globus_gass_close(fd) } (b) RSL extensions GRAM GASS Server HTTP Server (a) GASS file access API FTP Server Cache (c) Remote cache management (d) Low-level APIs for customizing cache & GASS server % globus-gass-cache

13. program GASS server stdout 1 Host name Contact string jobmanager globus-job-run 2 Command Line Args RSL string GRAM & GASS: Putting It Together 1. Derive Contact String 2. Build RSL string 3. Startup GASS server 4. Submit to request 5. Return output Execution machine Submit machine 5 4 5 3 4 4 gatekeeper

14. GRAM Client GRAM Client GSI GSI Globus Components In Action Local Machine User Proxy Cert X509 User Cert Machines RSL string mpirun grid-proxy-init RSL multi-request globusrun RSL single request RSL parser DUROC GASS Server GRAM Job Manager GRAM Job Manager GRAM Gatekeeper GRAM Gatekeeper GSI GASS Client GASS Client GSI PBS Unix Fork App App Nexus Nexus AIX MPI Solaris MPI Remote Machine Remote Machine

15. What is Condor-G? • Condor-G is a Personal-Condor enhanced with Globus services • It knows how to speak to Globus resources via GRAM • It can be used to submit jobs to remote Globus resources • It makes Condor keep track of their progress

16. Condor-G: Condor for the Grid • Condor is a high-throughput scheduler • Condor-G uses Globus Toolkit libraries for: • Security (GSI) • Managing remote jobs on Grid (GRAM) • File staging & remote I/O (GASS) • Grid job management interface & scheduling • Robust replacement for Globus Toolkit programs • To implement a reliable, crash-proof, checkpointable job submission service • Supports single or high-throughput apps on Grid • Personal job manager which can exploit Grid resources

17. The Use of Condor-G Globus resource Condor Master Condor GridManager Condor Schedd Globus resource condor_submit condor_q condor_rm Globus resource

18. Condor-G as user job submission service Condor-G condor_submit condor_q condor_rm Globus GRAM Globus GRAM Globus GRAM Globus GRAM fork CONDOR LSF PBS

19. Globus-based production Grids • LHC Grid(LCG-2) • A homogeneous Grid developed by CERN • Restrictive policies (global policies over rule local policies) • A dedicated Grid to the Large Hydron Collider experiments • Works 24 hours/day and used in EGEE • UK-NGS • A homogeneous Grid deployed in the UK • Restrictive policies • Non-dedicated • Works 24 hours/day