1 / 10

Resource Management

Ewa Deelman. Resource Management. Runtime-based Scheduling. Priority Queue Phase Eligible jobs are determined – based on fairness, system limitations, job state, etc...

mora
Download Presentation

Resource Management

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Ewa Deelman Resource Management

  2. Runtime-based Scheduling • Priority Queue Phase • Eligible jobs are determined – based on fairness, system limitations, job state, etc... • The list of eligible jobs is then sorted by job priority and jobs are scheduled to run until a job cannot be scheduled due to resource limitations • Resources and start/end times are reserved for the first job that could not be scheduled • Backfill Scheduling Phase • Jobs following the reserved job which would not delay the reserved job are scheduled on the remaining idle rsources

  3. Single Queue Backfilling • A job is allowed to jump in the queue ahead of jobs that are delayed due to lack of resources • Non-preemptive • Conservative backfilling • A job is allowed to jump ahead provided it does not delay any previous job in the queue • Aggressive backfilling • A job is allowed to jump ahead if the first job is not affected • Better performing then conservative backfilling

  4. Aggressive backfilling • Job (arrival time, number of procs, expected runtime)‏ • Any job that executes longer than expected runtime is killed • Define pivot—first job in the queue • If enough resources for pivot, execute pivot and define a new pivot • Otherwise sort all currently executing jobs in order of their expected completion time • Determine pivot time—when sufficient resources for pivot are available

  5. Aggressive backfilling • At pivot time, any idle processors not required for pivot job are defined as extra processors • The scheduler searches for the first queued job that • Requires no more than the currently idle processors and will finish by the pivot time, or • Requires no more than the minimum currently idle processors and the extra processors • Once a job becomes a pivot, it cannot be delayed • Possible that a job will end sooner and pivot starts before pivot time

  6. Aggressive backfilling example Job A (10 procs,40mins)‏ 10 processors 8 time Queue Job B (12 procs, 1 hour)‏ Job C (20 procs, 2 hours)‏ Job D (2 procs, 50 mins)‏ Job E (6 procs, 1 hour)‏ Job F (4 procs, 2 hours)‏ Running Job A(10 procs, 40 mins)‏

  7. Aggressive backfilling example Job A (10 procs,40mins)‏ 10 PIVOT Job B processors 8 JOB D time queue Job B (12 procs, 1 hour)‏ Job C (20 procs, 2 hours)‏ Job E (6 procs, 1 hour)‏ Job F (4 procs, 2 hours)‏ Running Job A(10 procs, 40 mins)‏ Job D (2 procs, 50 mins)‏

  8. Aggressive backfilling example Job A (10 procs,40mins)‏ 10 PIVOT Job B processors 8 Job F Job D time Queue Job B (12 procs, 1 hour)‏ Job C (20 procs, 2 hours)‏ Job E (6 procs, 1 hour)‏ Running Job A(10 procs, 40 mins)‏ Job D (2 procs, 50 mins)‏ Job F (4 procs, 2 hours)‏

  9. Aggressive backfilling example Job A (10 procs,40mins)‏ 10 Job B processors 8 Job F Job D time Queue Job C (20 procs, 2 hours)‏ Job E (6 procs, 1 hour)‏ Running Job D (2 procs, 50 mins)‏ Job F (4 procs, 2 hours)‏ Job B (12 procs, 1 hour)‏

  10. Aggressive backfilling example Job A (10 procs,40mins)‏ Pivot Job C (20 procs, 2 hours)‏ processors 10 Job B 8 Job F Job D time Queue Job C (20 procs, 2 hours)‏ Job E (6 procs, 1 hour)‏ Running Job D (2 procs, 50 mins)‏ Job F (4 procs, 2 hours)‏ Job B (12 procs, 1 hour)‏

More Related