Towards Provision of Quality of Service Guarantees in Job Scheduling

1. Towards Provision of Quality of Service Guarantees in Job Scheduling Mohammad Islam Pavan Balaji P. Sadayappan D. K. Panda Computer Science and Engineering The Ohio State University

2. Job Schedulers Today • Publicly Usable Supercomputer Centers and Compute Clusters • Becoming increasingly common (OSC, SDSC, etc) • Allow people to run jobs and charge the user based on the job requirements • Job requirements include Number of CPUs,CPU time, memory, etc. • Jobs are submitted by the user with resource requirements • Scheduled to execute at a later time • Dedicated job scheduler schedules the submitted jobs (Ex: PBS, Maui, Silver) • Jobs provide an estimate of the run-time together with the job • Independent Parallel Job Scheduling Model • Dynamically arriving Independent Parallel Jobs • Resource mapping: Submitted Jobs to Resources • Viewed as a two dimensional chart: Processors vs. Time

3. Running Jobs Two Dimensional Scheduling Grid J4 Job Queue J5 J6 J3 Processors J2 J1 Time Current Time

4. Job Scheduling Overview • Submit jobs have several associated parameters • Resources Required • Arrival Time (time with the job is submitted by the user) • Run-time estimate of the job (users need to provide a run-time estimate) • Queuing Order • FCFS (First Come First Served) • SJF (Shortest Job First) • Reservations are made for some of the jobs • Conservative backfilling model • EASY backfilling model

5. Running Jobs Backfilling Models J4 Job Queue J5 J6 Reservation Violation J5 J4 J6 J3 Processors J2 J1 J5 Time Current Time Conservative gives reservations to all jobs; EASY gives reservations to just one job

6. Guarantees in Service • Number of Techniques studied over the years • Backfilling (Ex: Conservative, EASY, No Guarantee) • Priority based scheduling • Differentiated service to different classes of jobs • Soft Real-time or Best Effort guarantees to the completion time • Hard Real-Time or “Deadline-based” scheduling • Allow Users to specify the deadline they desire • Cost model based on Resources Used AND Deadline Specified • Requires a deadline-based scheduling algorithm

7. QoS for Job Scheduling • Two Components in providing QoS • Job Scheduling Component • Admission Control • Can we meet the specified deadline? • Once admitted, cannot miss the specified deadline • Studied as a part of our previous work [islam03:qops] • Cost Model Component • Based on Resources Used AND Deadline Specified • More urgent jobs are charged more • Role of user characteristics in the charging model • Tolerance of users to missed deadlines • We deal with this component of QoS in this paper [islam03:qops] “QoPS: A QoS based scheme for Parallel Job Scheduling”, M. Islam, P. Balaji, P. Sadayappan and D. K. Panda. Published in JSSPP ’03 and LNCS ‘04.

8. Overview • Introduction and Motivation • Cost Model for Supercomputer Centers • Understanding User-Tolerance • Dealing with User-Tolerance • Artificial Slack • Kill-and-Restart • Experimental Results • Conclusions and Future Work

9. Cost Model in Supercomputer Centers • Current Cost Model based on Resources used • Cost independent of the response time of the job • User cannot request for a quicker service • Some schedulers allow differentiated service (Ex: NERSC) • Use high priority, normal and low priority queues • High Priority queue charges more; Low Priority queue charges less • No guarantees provided to the user about the response time • Idea of charging the user based on the service/priority: Still Relevant ! • Two components in the Cost Model • Resource Charge • QoS Charge

10. Cost Model Components • Resource Charge • Depends on the CPU, Memory, Disk Space, etc. • We only consider the CPU resource; extendable to other resources • QoS Charge • Depends on the urgency of the job • Depends on the difficulty in scheduling the job; based on two components • Current load in the system • Slowdown of the category to which the job belongs (Ex: Short-Wide) • Slowdown depicts the factor by which the job is delayed compared to its runtime Resource Charge = Processors x Run-Time QoS Charge = Category Slowdown / Requested Slowdown

11. Overview • Introduction and Motivation • Cost Model for Supercomputer Centers • Understanding User-Tolerance • Dealing with User-Tolerance • Artificial Slack • Kill-and-Restart • Experimental Results • Conclusions and Future Work

12. Understanding User Tolerance • If the requested deadline cannot be met? • Some users might submit another job or to another supercomputer center • Some users might re-submit the job with another deadline • What new deadline? Hit-and-Trial? • We modified QoPS to provide some feedback about the best possible deadline • We model user-tolerance based on a Tolerance Factor (TF) • Factor of extension in the requested deadline, the user can accept • If the (Earliest Possible Deadline) < (TF x Requested Deadline) • The user is assumed to accept the new deadline • Else • The user is assumed to reject the new deadline

13. Feedback based QoPS Algorithm • This is for the case when the user requested deadline cannot be met • Say this deadline is D1 • When a new job arrives: • Try to place the new job without disturbing any other job • Calculate the completion time of the job • This is a definitely feasible deadline (D2) • Use a binary search for N iterations to find a better deadline • For each iteration, run QoPS with the new deadline • If feasible, try a tighter deadline; else try a looser deadline

14. Impact of User Tolerance • Increase in Resource Charge: Intuitive • Decrease in QoS Charge; as the users get more tolerant, the center loses QoS money ! • This is because of accepting cheaper jobs which would have otherwise been rejected • Overall profit of the Supercomputer Center depends on the ratio of the two charges

15. Overview • Introduction and Motivation • Cost Model for Supercomputer Centers • Understanding User-Tolerance • Dealing with User-Tolerance • Artificial Slack • Kill-and-Restart • Experimental Results • Conclusions and Future Work

16. Dealing with User-Tolerance • User-Tolerance is not an always win situation! • Depends on the QoS charge to Resource charge ratio • If QoS charge were 0, user-tolerance is always beneficial • If Resource charge were 0, user-tolerance is never beneficial • We need some way to deal with this behavior of user-tolerance • Allows us to maximize profit depending on the QoS and Resource charges • We propose two approaches to counter user-tolerance • Providing Artificial Slack • Kill and Restart Mechanism

17. Artificial Slack • If a requested deadline cannot be met • Provide the best possible deadline to the job • Maximizes the revenue for the job • Creates a very tight schedule; difficulty in accepting later arriving jobs • Provide the job with an artificial slack • Loses out on the revenue for the job • The overall schedule is loose enough to accept later arriving jobs • Offered Deadline extended • Arrival Time + (Earliest Deadline – Arrival Time) x Slack Factor

18. Kill and Restart • Some supercomputer centers support a kill-and-restart mechanism • A running job can be killed and restarted: No permanent modifications • Some centers such as the OSC support such mechanisms • Our approach utilizing the kill-and-restart mechanism • If a jobs requested deadline cannot be met • Kill a running job and try scheduling this job • Verify that both the new job and the killed job can be scheduled • If they can be scheduled, accept the schedule • It is to be noted that this approach might result in wastage of resources

19. Overview • Introduction and Motivation • Cost Model for Supercomputer Centers • Understanding User-Tolerance • Dealing with User-Tolerance • Artificial Slack • Kill-and-Restart • Experimental Results • Conclusions and Future Work

20. Impact of Slack Factor (SF) • Counter Impact as User Tolerance ! • Drop of Resource charge; more smaller jobs accepted, force larger jobs to be rejected • Increase in QoS charge; additional slack forces significantly delayed jobs to be rejected

21. Impact of Kill-and-Restart (TF = 4.0) • Kill-and-Restart loses out on resource charge due to wastage of resources • Gains on the QoS charge due to its ability to accept later arriving expensive jobs

22. Impact of Non-deadline Jobs (20% deadline) • Benefits of Kill-and-Restart are even higher • Benefits seen in both cost components

23. Overview • Introduction and Motivation • Cost Model for Supercomputer Centers • Understanding User-Tolerance • Dealing with User-Tolerance • Artificial Slack • Kill-and-Restart • Experimental Results • Conclusions and Future Work

24. Concluding Remarks • Considerable Research on the topic of Parallel Job Scheduling • The issue of provision of QoS has received little attention • In this paper, we extend our previous QoS based scheme, QoPS • A feedback mechanism to provide the best possible deadline • Study the impact of user-tolerance on overall revenues • Propose schemes to minimize the negative impacts of user-tolerance • Demonstrated capabilities and issues based on simulation studies

25. Future Work • Incorporate the QoPS scheme into Maui/Moab at the OSC • Study QoS aspects for multi-site schedulers • Clusters with heterogeneous systems • Extend the scheme to multiple resources • Study QoS aspects for network flow scheduling

26. Thank You !http://www.cse.ohio-state.edu/~saday{islammo, balaji, saday, panda}@cse.ohio-state.edu

27. Backup Slides

28. The Basic QoPS Algorithm • When a job (JN+1) arrives • If J1, J2, …, JN are already present and scheduled in that order • Place the job (JN+1) at the start of all jobs • Try scheduling the jobs in that order • If all jobs are able to meet their deadlines, Great ! Admit it ! • If some job fails, we have two options: • Option1: • Consider the failed job as a critical job • Push the failed job to the start of the schedule and retry • ‘k’ number of such re-orderings of existing jobs are allowed • If (number of re-orderings > k) switch to option 2 • Option2: • Back off exponentially in the position at which you try placing job (JN+1) and retry

29. Working of the QoPS Algorithm J13 J12 J11 J10 J9 J8 J7 J13 J6 J5 J6 J4 J5 J3 J2 J1 J1 J13 J2 J1 J3 J2 J1 J3 J13 J4 J13 J2 J3 Current Violations = 0 Current Violations = 1 Current Violations = 0 Current Violations = 2 J4 J3 J2 J1 J2 J1 J13 J1 J13 J3 J13 J3 J2 Max. Violations Allowed = 2

30. Simulation Approach CTC/SDSC Trace Duplication/Expansion Load Variation Deadline Calculator Deadline-based Trace QoPS Simulation MSB Simulation MRT Simulation EASY Simulation

31. Trace Generation • Many job logs available, but no associated deadlines • Synthetic Deadline Generation • Generate a schedule for the job trace using EASY • For any job J, if the Turnaround time in this schedule is T • Deadline for J = Arrival Time + max (runtime, (1-SF) x T) • SF is the “Stringency factor” (0 < SF < 1) • 0 would give the least stringent deadlines and 1 the most stringent • Some jobs might not come with deadlines • Very lax deadlines to prevent starvation • If ‘T’ is the current expected Turnaround time, • Deadline = Arrival Time + max (24hrs, R x T) • R is the “Relaxation Factor” of the schedule