Download
1 / 21
210 likes | 311 Views
GRASS : Trimming Stragglers in Approximation Analytics. Ganesh Ananthanarayanan, Michael Hung, Xiaoqi Ren, Ion Stoica, Adam Wierman, Minlan Yu. Approximation Jobs. Data deluge timely answers from part of dataset are often good enough
E N D
GRASS: Trimming Stragglers in Approximation Analytics Ganesh Ananthanarayanan, Michael Hung, Xiaoqi Ren, Ion Stoica, Adam Wierman, Minlan Yu
Approximation Jobs • Data deluge timely answers from part of dataset are often good enough • Deadline-bound Jobs: Maximize accuracy within a deadline • E.g., get me the best ad to show within 1s • Error-bound Jobs: Minimize time taken to achieve desired accuracy • E.g., get me the #cars sold to the nearest 1000
Prioritize Tasks • Spawn jobs on large dataset; subset of tasks need to complete • Accuracy α (Subset of completed tasks) • Prioritize tasks • Jobs run only part of their tasks at a time (multi-waved) • NP-Hard but many known heuristics…
Stragglers • Occur despite prevention mechanisms • Speculation: Spawn a duplicate, earliest wins Challenge:dynamically prioritize while choosing between speculative/unscheduled tasks
Opportunity Cost • Speculative copies consume extra resources T1 Is speculation worth the payoff? Slot 1 Slot 2 T2 Slot 3 • T1 T3 time 0 5 10 9
Roadmap • Two extreme approaches • Switching between the two approaches • Evaluation
Greedy Scheduling (GS) • Schedule the task that greedilyimproves accuracy, i.e., finish earliest Accuracy = 7/9 Slot 1 T1 T1 Deadline = 6 Slot 2 T2 • T3 • T4 • T5 • T6 • T7 time 3 0 1 6
Resource Aware Scheduling (RAS) • Speculate only if (c)trem > (c+1)tnew Accuracy = 8/9 Slot 1 T1 • T4 • T6 • T7 T1 Deadline = 6 Slot 2 T2 • T1 • T3 • T5 • T8 time 3 0 1 6
GS vs. RAS Accuracy = 7/9 Accuracy = 3/9 Slot 1 T1 T1 T1 GS Deadline = 6 Deadline = 3 Slot 2 T2 • T3 • T4 • T5 • T6 • T7 time 3 0 1 6 Accuracy = 2/9 Accuracy = 8/9 Slot 1 T1 • T4 • T6 • T7 T1 RAS Deadline = 6 Deadline = 3 Slot 2 T2 • T1 • T3 • T5 • T8 time
How to pick between GS and RAS? • When to consider opportunity cost? • Analytical model [TBD]
Guideline • Be “conservative” early in the job, and “aggressive” near its completion For jobs with >2 waves of tasks use RAS, otherwise use GS
GRASS = GS+ RAS • Start with RAS and switch to GSwhen 2 waves of tasks remain • How to estimate two remaining waves? • Wave boundaries are not strict • Task durations are uncertain • Wave-width is not constant
Learning switching point in GRASS • GRASS starts with RAS and switches to GS close to the deadline/error-bound • Learn the switching point from completed jobs • Generate GS-only and RAS-only samples • Switch when the GS-only samples are better for the remaining deadline
GRASS Scheduling • GS Greedy Scheduling • RAS Opportunity cost aware Scheduling • Switch from RAS to GS close to deadline/error-bound • Learn switching point
Implementation Details • Implemented inside Hadoop 0.20.2 and Spark 0.7.3 • Modified Fair Scheduler • Task Estimators • trem is extrapolated from remaining data; progress reports at 5% intervals • tnew is estimated from completed tasks
Evaluating GRASS • Workload from Facebook and Bing traces • 1000’s of nodes, Hadoop and Dryad jobs • How was it generated? • Baselines: LATE orMantri • 200 node EC2 deployment; trace-driven sim.
