Globus Overview

1. Globus Overview CS-780-3 Lecture Notes In courtesy of Andy Kowalski

2. Overview • What Defines a Grid • Globus • GT1, GT2 (pre OGSA) • GT3 (post OGSA) • Issues

3. What Defines a Grid? “We will probably see the spread of ‘computer utilities, which, like present electric and telephone utilities, will service individual homes and offices across the country” -Len Kleinrock. 1969 [1]

4. What Defines a Grid? “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities.” - Ian Foster and Carl Kesselman, 1998 [2]

5. Ian Foster’s Three Point Checklist (2002) A Grid is a system that: • “coordinates resources that are not subject to centralized control” [2] • “using standard, open, general-purpose protocols and interfaces” [2] • “to deliver nontrivial quantities of service” [2]

6. Globus Project • Multi-institutional research effort on the technology required to build a grid or grids • Mathematics and Computer Science Division at Argonne National Laboratory • Information Sciences Institution at the University of Southern California • University of Chicago • University of Edinburgh • Center for Parallel Computers • Principle Investigators • Ian Foster - Argonne National Laboratory and University of Chicago • Carl Kesselman - University of Southern California

7. Globus Project • Seek to understand application requirements for a usable grid and … • Conduct research and development to create the technologies required to build a grid • Globus toolkit

8. Globus Toolkit • Bag of services • Developers can use these to build grid applications • Makes use of existing standards when possible • Interfaces and implementations • Distinguishes between local and global services • Defines interfaces to manage heterogeneity • Does not hide heterogeneity • Information service to deal with constant changes • Utilization, failures, new resources, etc.

9. Hourglass Principle • A layered architecture with and hourglass shape • Grid services should provide a simple well-defined interface between applications and local services

10. GRAM • Globus Resource Allocation Manager • Provides a single interface for requesting and using remote system resources for executing jobs • Each GRAM is responsible for the resource management systems local to it • LSF (Load Sharing Facility), Condor, PBS • Provides a standard interface to these different systems • Applications can access these different systems using a standard API • Within GRAM resource requests are described by the Resource Specification Language (RSL)

11. GRAM Broker RSL Application Queries & Info Information Service Ground RSL Co-allocator Single Ground RSL • Ground RSL is an expanded description of the application’s request. It may represent more than on job. • Single Ground RSL is a description of a singe job created from the Ground RSL. There may be more than one created from the original Ground RSL. GRAM GRAM LSF PBS

12. RSL • Resource Specification Language • Provides a common language to describe resources • Uses attribute, value pairs • A simple example & (executable = a.out (* <-- that is an unquoted literal *)) (directory = /home/nobody ) (arguments = arg1 "arg 2") (count = 1)

13. Nexus • Low-level communications API that supports other communication libraries • Can select alternative low-level protocols based on network topology • Message passing, IP, shared memory • Can introduce encryption based on source and destination of a message • Intended for use by compiler writers and library developers • CC++, HPC++, and MPICH-G

14. MDS • Metacomputing Directory Service (GT1) • Monitoring and Discovery Service (GT2&3) • Provides information about the systems in the grid • CPU type, memory, OS, network, etc. • Queue information, utilization, etc • Basically a local LDAP server • MDS ties all the LDAP servers together so everyone has access to the same information • Has an API for discovering, publishing, and accessing information

15. MDS Architechture • Grid Index Information Service (GIIS) • Provides an aggregate directory of lower level data • Grid Resource Information Service (GRIS) • Runs on a resource and acts like a content gateway to the resource • Information Provider (IP) • Interface to data collection services

16. GSI • Globus Security Infrastructure • Deals only with authentication • Users and services • Uses x.509 certificates • Also used username and password (rlogin) • Uses Secure Socket Layer (SSL) for the connection and authentication • No longer true with GT3 • All Globus services use GSI to authenticate users

17. HBM • Heartbeat Monitor • API to register with the service and collect data • Service expects updates from clients that have registered • If no update is received, attempts to determine if the service itself is down or if there are other factors

18. GASS • Global Access to Secondary Storage • Allows programs to use the standard C I/O library to read and write files on remote computers • Reads • File is copied locally and then read • Writes • File is written locally and then copied to their destination • Appends • Communicated to the remote file as soon as it is written

19. GEM • Globus Executable Management • Support identification, location, and creation of executables in heterogeneous environments • Pre-stages executables to multiple remote sites • Removes them after execution

20. GridFTP • Secure data transfer protocol • GSI authentication • Optimized for high-bandwidh wide-area networks • Parallel streams - multiple data channels • Partial file transfer • Third-party transfers • Server to server

21. Globus Toolkit 3 • Based on the Open Grid Services Architecture (OGSA) • Attempts to leverage industry standards • Makes use of web services • Preserves backward compatibility with GT2 • Future releases will begin eliminating compatibility • Client interfaces to services are now WSDL/OGSI based

22. Issues • Protocols and interfaces • Where the GT1 and GT2 implementation of a grid service was inadequate for certain grids, others would developed competing services using different interfaces and protocols • Defeated the goal of compatibility • OGSA/OGSI and web services are hoped to resolve this • Don’t need the whole thing • Want to use GSI, need to install all of Globus • COGs helped eliminate this burden • Use only what you need

23. Issues • Globus originally seemed focused on compute grids • Other groups were building data grids • data grids represent a larger customer base • GridFTP was over extended • Was once proposed that it be used for more than file transfers • Message passing • VO Management

24. References [1] Ian Foster. What is the Grid? A Three Point Checklist. July 20, 2002. [2] Ian Foster and Carl Kesselman, editiors. “The Grid: Blueprint for a New Computing Infrastructure.” 1998. [3] Ian Foster and Carl Kesselman. The Globus Project: A Status Report. Proceedings of IPPS/SPDP’98 Heterogeneous Computing Workshop. 1998. [4] Ian Foster and Carl Kesselman. Globus: Ametacomputing Infrastructure Toolkit. International Journal of Supercomputer Applications. 1997 [5] Globus Project. Globus Toolkit 2.2 MDS Technology Brief. January 30, 2003. [6] Globus Project. Community Authorization Service (CAS) Overview. December 18, 2001. [7] Ian Foster. Open Grid Services Architecture. Plenary Talk at CHEP 2003. March 26, 2003. [8] Ian Bird. Deploying the LHC Computing Grid. Plenary Talk at CHEP 2003. March 26, 2003.