A CLASS OF POLICIES THAT PRIORITIZE SMALL JOBS Adam Wierman

1. A CLASS OF POLICIES THAT PRIORITIZE SMALL JOBSAdam Wierman Carnegie Mellon University Pittsburgh, PA USA

2. Web Server Use Shortest Remaining Processing Time First (SRPT) A MOTIVATIONAL EXAMPLE: Web servers Bandwidth is the bottleneck Processor Sharing (PS) Goal Minimize user response times requests for files [Harchol-Balter, Schroeder, Bansal, Agrawal, TOCS 2003]

3. PS mean response time ? Why SRPT? SRPT load 0 1 A MOTIVATIONAL EXAMPLE: Web servers [Harchol-Balter, Schroeder, Bansal, Agrawal, TOCS 2003]

4. users CPUs Internet Databases Routers Disks Prioritize small jobs Locks SCHEDULING SUCCESS STORIES ACROSS COMPUTER APPLICATIONS Web Servers …also p2p, wireless, operating systems… Rai, Urvoy-Kelley, Biersack, Harchol-Balter, Schroeder, McWherter, Rawat, Dinda, & many others

5. SRPT RS PSJF PS LCFS SJF DPS PLJF FCFS LJF FSP PLCFS FB LRPT SCHEDULING SUCCESS STORIES ACROSS COMPUTER APPLICATIONS RENEWED INTEREST IN THEORETICAL WORK Harchol-Balter, Biersack, Ayesta, Nunez-Queija, Nuyens, Boxma, Zwart, Borst, & many others

6. There is a gap between the theoretical results and the practical implemplentations

7. Slowdown & Fairness are also important Hybrid policies outperform SRPT on these Theory a small set of individual, idealized policies are analyzed multiple metrics are important Practice real policies are hybrids policies studied in theory are “tweaked” for use in practice

8. Prioritize small jobs BRIDGING THE GAP Analyze classes of policiesinstead of individual policies SRPT RS PSJF PS LCFS SJF DPS PLJF • Classes include practical hybrids • Classes allow optimization of secondary metrics FCFS LJF PLCFS FSP FB LRPT

9. OUR GOAL TODAYDefine a class of policies that “prioritizes small jobs” and is bothBROADandTIGHT behaves like SRPT includes practical hybrid policies

10. THE SMART CLASS SMART PSJF RS SRPT SMAll Response Times • Bias Property • Consistency Property • Transitivity Property PS LJF PLCFS ROS FCFS LRPT

11. TWO NOTIONS OF “SMALL” JOBS small original size small remaining size

12. BIAS PROPERTY If OriginalSize(a) < RemainingSize(b) then a has priority over b a b [Wierman, Harchol-Balter, Osogami 2005]

13. remaining size 0 0 original size BIAS PROPERTY If OriginalSize(a) < RemainingSize(b) then a has priority over b [Wierman, Harchol-Balter, Osogami 2005]

14. BIAS PROPERTY If OriginalSize(a) < RemainingSize(b) then a has priority over b lower priority remaining size ? higher priority 0 0 original size [Wierman, Harchol-Balter, Osogami 2005]

15. SRPT PSJF ! RS remaining size Partial ordering allows time varying policies ? original size EXAMPLES If OrigSize(a) < RemSize(b) then a has priority over b Many others! e.g. iRj+S

16. ! Partial ordering allows time varying policies B B A C A D remaining size *PSJF D C time

17. ! Partial ordering allows time varying policies B A ? B A D remaining size *PSJF *PSJF D C *SRPT time

18. ! Partial ordering allows time varying policies • Use a parameterized policy setthat is (nearly) dense in SMART,e.g. iRj + S • Search (i,j) space for policy thatoptimizes secondary objectives,e.g. fairness and predictability ONLINE MULTI-OBJECTIVE SCHEDULING USING SMART

19. THE BIAS PROPERTYISN’T ENOUGH CONSISTENCY TRANSITIVITY remaining size ? orig. size [Wierman, Harchol-Balter, Osogami 2005]

20. THE BIAS PROPERTYISN’T ENOUGH CONSISTENCY If a is served ahead of b then a will always have priority over b + TRANSITIVITY If an arriving job b preempts c, then until b leaves, every arriving job a with original size smaller than b has priority over c. remaining size ? at most 1 has higher priority orig. size [Wierman, Harchol-Balter, Osogami 2005]

21. DOING THE “SMART” THING lower priority SMAll Response Times remaining size • Bias Property • Consistency Property • Transitivity Property at most 1 job with higher priority higher priority 0 0 original size

22. ? Which policies are notSMART? SMART PSJF RS SRPT FB SJF PS LJF PLCFS ROS FCFS LRPT Blind policies Non-preemptive policies

23. OUR GOAL TODAYDefine a class of policies that “prioritizes small jobs” and is bothBROADandTIGHT behaves like SRPT includes practical hybrid policies

24. M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)

25. Residence time Residence time Waiting time Waiting time T(x) RESULT, plot for E[T(x)] CONDITIONAL RESPONSE TIME UNDER SMART POLICIES Theorem: Under the M/GI/1, for all SMART policies P, Response time for a job of size x [Wierman, Harchol-Balter, Osogami 2005]

26. SRPT SMART PSJF original size T(x) RESULT, plot for E[T(x)] CONDITIONAL RESPONSE TIME UNDER SMART POLICIES Theorem: Under the M/GI/1, for all SMART policies P, Picture “proof”: Waiting time remaining size ?

27. SRPT PSJF T(x) RESULT, plot for E[T(x)] CONDITIONAL RESPONSE TIME UNDER SMART POLICIES Theorem: Under the M/GI/1, for all SMART policies P, Picture “proof”: Residence time SMART remaining size No higher priority jobs ? ? original size

28. M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)

29. Theorem: In the M/GI/1, mean response time PS E[T] ! These bounds are tight SMART load, ρ 0 1 SMART POLICIES ARE 2-COMPETITIVE [Wierman, Harchol-Balter, Osogami 2005]

30. Theorem: In the M/GI/1, ! These bounds are tight SMART POLICIES ARE 2-COMPETITIVE Consider the M/D/1 SRPT does FCFS (only in M/D/1). So as ρ1 PLCFS is in SMART (only in M/D/1) As ρ1, E[T]PLCFS  2 E[T]SRPT

31. Theorem: In the M/GI/1, ? Tight under Deterministic Tight under Pareto (as ρ1) What is the heavy traffic behavior? SMART POLICIES ARE 2-COMPETITIVE With work, the bounds can be rewritten as

32. M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)

33. λ1 Nμ λ2 λN NB Many sources Large buffer ! SMART policies are asymptotically equivalent in both TAIL BEHAVIOR OF SMART Pr(T>y)is difficult to study directly so it is typically studied asymptotically [Nuyens, Wierman, Zwart 2007] [Yang, Wierman, Shakkattai, Harchol-Balter 2006]

34. OUR GOAL TODAYDefine a class of policies that “prioritizes small jobs” and is bothBROADandTIGHT • behaves like SRPT: • E[T] • Pr(T>y) includes practical hybrid policies But…

35. Theory a small set of individual, idealized policies are analyzed multiple metrics are important implementation restrictions Practice real policies are hybrids policies studied in theory are “tweaked” for use in practice

36. Theory a small set of individual, idealized policies are analyzed implementationsonly have job size estimates implementation restrictions Practice implementations only use 7-10 priority levels policies studied in theory are “tweaked” for use in practice

37. ? What is the impact of inexact job size information? no k such that use a SMART policy know exact job sizes 1 ? have an estimate of job sizes prioritize jobs with small estimated sizes 2 know only the distribution of job sizes if sizes are variable use FB otherwise use FCFS 3

38. OUR NEW GOALBROADENtheSMART class while keeping itTIGHT 2 k E[T]P ≤ E[T]SRPT include policies that use job size estimates

39. rem. size ? orig. size rem. size ? Weaken the Bias Property orig. size

40. remaining size ε(x) ? ? original size original size SMART SMARTε If OrigSize(a) = x and ε(x) < RemSize(b) then a has priority over b If OrigSize(a) < RemSize(b) then a has priority over b

41. ε(x) ? original size SMARTε EXAMPLES If OrigSize(a) = x and ε(x) < RemSize(b) then a has priority over b How do you get back SMART? ε(x) = x How can you characterize job size estimates? ε(x) = (1+σ) x You can also capture policies with a finite number of priority levels...

42. OUR NEW GOALBROADENtheSMART class while keeping itTIGHT k E[T]P ≤ E[T]SRPT include policies that use inexact job sizes

43. M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)

44. ε(x) rem. size ? ! Under SMART σ=δ=0, so E[T] ≤ 2E[T]SRPT orig. size ! x (1+σ) = (1-δ)1-α under Pareto(α) distributions SMARTε POLICIES ARE NEAR OPTIMAL Theorem: In an M/GI/1 under SMARTεpolicy P σbounds the SIZE of larger jobs that get higher priority δbounds the LOAD of larger jobs that get higher priority

45. ? What is the effect of inexact job size information? Theorem: In an M/GI/1 with Pareto(α) job sizes and SMARTεpolicy P where for all x, 20 E[T] / E[T]SRPT 10 2 0 200% 100% estimate accuracy (σ) INTERPRETING THE THEOREM Take α=1.1

46. ? What is the effect of inexact job size information? Theorem: In an M/GI/1 with Pareto(α) job sizes and SMARTεpolicy P where for all x, PS INTERPRETING THE THEOREM mean response time SMARTε SRPT load 0 1

47. M/GI/1 preempt-resume queue ANALYSIS SETTING: E[T]SMART APPROACH: Bound T(x)SMART Pr(TSMART>y)

48. λ1 Nμ λ2 λN NB Many sources Large buffer ! SMARTε policies are asymptotically equivalent for unbounded service distributions TAIL BEHAVIOR OF SMARTε POLICIES Pr(T>y)is difficult to study directly so it is typically it is studied asymptotically

49. WRAPUP

50. ! Other forms of ε(x) are also useful SMARTε when PSJF RS E[T]≤ k E[T]SRPT SRPT Pr(T>x)~ Pr(TSRPT>x) PS LJF FCFS Show that SMARTε policies behave like SRPT Formalize the heuristic of “prioritizing small jobs” • Bias Property • Consistency Property • Transitivity Property