1 / 11

GRAM: Globus Resource Allocation and Management

GRAM: Globus Resource Allocation and Management. GRAM designed to provide a single common protocol and API for requesting and using remote system resources uniform, extensible interface local job scheduling systems. API for submitting and canceling a job request

emilie
Download Presentation

GRAM: Globus Resource Allocation and Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. GRAM: Globus Resource Allocation and Management • GRAM designed to provide a single common protocol and API for requesting and using remote system resources • uniform, extensible interface local job scheduling systems. • API for • submitting and canceling a job request • checking the status of a submitted job. • Specifications written by user using Resource Specification Language (RSL) • processed by GRAM as part of the job request. • By design, GRAM does not guarantee user environments on remote hosts.

  2. GRAM Architecture (GT2)

  3. GRAM: • Resource • An entity capable of running one or more processes on behalf of a user. • Client • The process that is using the resource allocation client-side API. • Job • A process or set of processes resulting from a job request. • Job Request • A request to gatekeeper to create one or more job processes, expressed in the RSL. • Gatekeeper • A process, running as root, which begins the process of handling allocation requests. It exists on the remote computer before any request is submitted. • When the gatekeeper receives an allocation request from a client, it mutually authenticates with the client, maps the requestor to a local user, starts a job manager on the local host as the local user, and passes the allocation arguments to the newly created job manager. • Job Manager • One job manager is created by the gatekeeper to fulfill every request submitted to the gatekeeper. • It starts the job on the local system, and handles all further communication with the client.

  4. RSL Resource Specification Language • RSL is a type of formal language • Has its own syntax and parsing rules • Works like unix regular expressions and shell scripts • We will cover in more detail next week • Ugly…users hate it… • & (rsl_substitution = (TOPDIR "/home/nobody") (DATADIR $(TOPDIR)"/data") (EXECDIR $(TOPDIR)/bin) ) (executable = $(EXECDIR)/a.out (* ^-- implicit concatenation *)) (directory = $(TOPDIR) ) (arguments = $(DATADIR)/file1 (* ^-- implicit concatenation *) $(DATADIR) # /file2 (* ^-- explicit concatenation *) '$(FOO)' (* <-- a quoted literal *)) (environment = (DATADIR $(DATADIR))) (count = 1) Performing all variable substitution and removing comments yields an equivalent RSL string: • & (rsl_substitution = (TOPDIR "/home/nobody") (DATADIR "/home/nobody/data") (EXECDIR "/home/nobody/bin") ) (executable = "/home/nobody/bin/a.out" ) (directory = "/home/nobody" ) (arguments = "/home/nobody/data/file1" "/home/nobody/data/file2" "$(FOO)" ) (environment = (DATADIR "/home/nobody/data")) (count = 1)

  5. GRAM Job Execution Environment • HOME • The user's home directory. • LOGNAME • The user's login name. • X509_USER_PROXY • The path to the job manager's delegated credential. (GSI only). • GLOBUS_GRAM_JOB_CONTACT • The job manager's contact string for this job. • GLOBUS_GRAM_MYJOB_CONTACT • The GRAM MyJob contact string for intrajob communication. • GLOBUS_LOCATION • The path to the Globus installation on the job manager host. • X509_CERT_DIR* • The path to a trusted certificate directory. This variable will only be set if the -x509-cert-dir argument is given to the job manager. • GLOBUS_GASS_CACHE_DEFAULT* • The path to the job's GASS cache (if the gass_cache RSL attribute is present). • GLOBUS_TCP_PORT_RANGE* • A system-specific range of TCP ports which may be used by the job. Globus I/O will automatically honor this range. Only present if the related configuration option is present in the job manager configuration file. • GLOBUS_REMOTE_IO_URL* • The path to a file containing a URL string of a GASS server which the job may access (if the remote_io_url attribute is present).

  6. Globus client tools • Use these to submit jobs to GRAM and get remotes tasks done: • globusrun – most basic way • globus-job-run  powerful • Test authentication (effectively ‘ping’): $ globusrun -a -r cab047.info.uvt.ro GRAM Authentication test successful • Execute remote simple Unix commands • $globus-job-run blue.info.uvt.ro /bin/uname -a Linux blue 2.6.16-hardened-r10 #2 SMP Fri Sep 1 22:36:46 EEST 2006 i686 Intel(R) Xeon(TM) CPU 2.40GHz GenuineIntel GNU/Linux

  7. Globus Jobs: complex but powerful • Most of these require RSL input • Typically used for batch job submission (e.g. jobs submitted to a queuing system on a cluster) • globus-job-get-output • globus-job-run • globus-job-status • globus-job-submit • globus-job-cancel • globus-job-clean

  8. Globus-job-run: used to run code remotely • Run on single node of remote cluster [dana@Hport ~]$ globus-job-run cab047 ./hello.sh Hello from cab047 [dana@Hport ~]$ • Run on multiple CPU’s: [dana@Hport ~/.globus]$ globus-job-run cab047 -np 4 /home/dana/hello.sh Hello from cab047 Hello from cab047 Hello from cab047 Hello from cab047

  9. globus-job-run: Examples • Run multiple commands: globus-job-run cab047 /bin/sh -c “cd my_dir ; ls” • Run several mpi jobs: globus-job-run \ -: wn01 -np 64 -s my-aix-exec \ -: nanosim1 -np 128 -s my-linux-exec • For help: globus-job-run -help Examples taken from NPACI training class (L. Brieger)

  10. globus-job-submit: Remote batch jobs • For help: globus-job-submit -help • To submit jobs to the remote batch scheduler %globus-job-submit \ cab047/jobmanager-batch \ -queue normal -np 4 /home/dana/mpi/little https://cab047.info.uvt.ro:44864/68982/1047069851/ ( jobID in response to submission )

  11. Job management • Use jobID to check on job status: globus-job-status https://cab047.info.uvt.ro:44864/68982/1047069851 PENDING …ACTIVE…DONE • Use jobID to retrieve output or cancel job globus-job-get-output \ https://cab047.info.uvt.ro:44864/68982/1047069851 globus-job-cancel \ https://cab047.info.uvt.ro:44864/68982/1047069851 • Use jobID to clean up cached output from job (on remote machine): globus-job-clean https://ca047.info.uvt.ro:44864/68982/1047069851

More Related