1 / 20

Job Submission

Job Submission. Using PBSPro and Globus Job Commands. Overview. Computational Resources Queuing Systems PBSPro Globus Toolkit. Computational Resources. Hard Disks (permanent storage) Number of CPUs (processing power) CPU time (processing time) Physical memory (program size)

patty
Download Presentation

Job Submission

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Job Submission Using PBSPro and Globus Job Commands

  2. Overview • Computational Resources • Queuing Systems • PBSPro • Globus Toolkit

  3. Computational Resources • Hard Disks (permanent storage) • Number of CPUs (processing power) • CPU time (processing time) • Physical memory (program size) • All computers have limited resources • These resources may span across multiple processors/machines • How to allocate these resources fairly, amongst the many users?

  4. Queuing Systems • Holding area for pending requests • A method for allocating the needed resources based on a user request • Processing of requests dynamically, quickly and fairly • Many differentimplementations

  5. PBSPro • Queuing System used to control the allocation of computational resources to user submitted jobs • Allows optimal sharing of all resources • Ensures that the limited resources aren’t over-run and exceeded • Unattended processing of requests

  6. PBS Queues • Allows distribution of resources into clearly defined groups, called queues • Queues can be defined by: • Maximum CPU time • Number of CPUs available • Memory needed • Concurrently executing jobs • Queuing schemes evolve over time as user requests and workload vary

  7. Interacting With PBS • qsub – Submit a job to the queues • qstat – Check the status of a job • qdel – Delete a job from the queues • qmgr – Create/modify queue settings • xpbs – Monitor all queues and jobs

  8. qsub – Submit A Job • To submit a job to the Queuing System, use “qsub” • E.g. qsub test.sub • Test.sub is a script file containing the commands to be executed • qsub returns your “QueueID”, if the job was submitted successfully:61606.master

  9. Submit File For A Serial Job • !/bin/bash#PBS -l walltime=2:00:00#PBS -l mem=5mb#PBS -j oe#PBS –m becd Serial./test

  10. Submit File For A Parallel Job • #!/bin/sh#PBS -l nodes=4:ppn=2#PBS -l walltime=48:00:00#PBS -j oecd ./MPI/Examplesmpiexec ./test

  11. qstat – Check Job Status • To check the status of a job submitted to the Queuing System, use “qstat”Job id Name User Time S Queue------------------ ---------- ------------ ------------ - --------61395.master G5D2C ngs0140 70:00:40 R cpu1661494.master STDIN ngs0227 17:58:40 R cpu161495.master STDIN ngs0227 18:15:42 R cpu161496.master STDIN ngs0227 18:15:02 R cpu161497.master STDIN ngs0227 18:13:42 R cpu161498.master STDIN ngs0227 18:14:13 R cpu861555.master Co_V ngs0133 20:57:53 R cpu2461567.master 20_12 ngs0234 00:31:17 R cpu161576.master 20_21 ngs0234 00:11:59 R cpu461578.master 20_23 ngs0234 00:05:51 R cpu161580.master 20_25 ngs0234 00:03:09 R cpu1

  12. qdel – Delete A Job • To delete a job submitted to the Queuing System, use “qdel” • E.g. qdel QueueID qdel QueueID1 QueueID2

  13. Globus Toolkit • An open source toolkit for developing Grid based applications and connectivity • Allocating computational resources on remote (Globus aware) machines for the execution of user submitted jobs

  14. Globus Job Commands • globus-job-run <options> • globus-job-submit <options> • globus-job-status URL • globus-job-get-output URL

  15. globus-job-run • Allows you to run a job as though it were interactive, on a local or remote machine • Don’t actually need to log on to the machine itself • Not submitted to the Queuing System • Returns the programs output as though you were running interactively

  16. globus-job-run Examples • globus-job-run grid-data.man.ac.uk /bin/date • globus-job-run grid-data.rl.ac.uk ./test • globus-job-run \grid-compute.leeds.ac.uk/jobmanager-pbs \-np 8 -x ‘(jobtype=mpi)(environment= \(NGSMODULES clusteruser))’ ./MPI/test • globus-job-run grid-data.rl.ac.uk -s ./test

  17. globus-job-submit • To submit a job through a Globus Job Manager • Commands returns a URL to the program’s outputhttps://grid-compute.leeds.ac.uk:64167/5291/1094639422/ • The status and output of the job can be tested through this URL

  18. globus-job-submit Examples • globus-job-submit \grid-data.rl.ac.uk/jobmanager-pbs ./test • globus-job-submit \grid-compute.leeds.ac.uk/jobmanager-pbs \-x ‘(jobtype=mpi)(directory=/home/bob/mpi) \(environment=(NGSMODULES clusteruser)) \(count=8)’ ./mpi_program • globus-job-submit \grid-data.man.ac.uk/jobmanager-pbs -s ./test

  19. globus-job-status URL • To get the status of a submitted job • globus-job-status \https://gridcompute.leeds.ac.uk:64167/5291/1094639422/ • Returns: • Pending, Active, Done, Failed

  20. globus-job-get-output URL • To retrieve the output of a job submitted through a Globus Job Manager • globus-job-get-output \https://gridcompute.leeds.ac.uk:64167/5291/1094639422/ • Returns the output of the program to the console

More Related