Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

1. Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths Mugurel Ionut Andreica Polytechnic University of Bucharest Computer Science Department

2. Summary • Motivation – Scheduling File Transfers • Offline Scheduling • maximum profit scheduling (files with divisible sizes) – heuristic algorithm • minimum cost scheduling – optimal algorithm • Online Resource Management • algorithmic framework for the block partitioning method (=data structure) Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

3. Motivation • world-wide development and deployment of distributed systems, services and applications • competition for bottleneck resources => poor performance • inefficient usage of available resources => poor performance • scheduling techniques => performance improvements Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

4. Maximum Profit Scheduling • n files (divisible sizes) • szi – size of the ith file • sz1≤sz2 ≤... ≤ szn • szi=qi·szi-1, qi≥1 is an integer • pi=profit of the ith file (transfer request) • k paths • path j – available in the interval [0,Tj] • unit speed • schedule the requests non-preemptively + at most one request per path per time unit • equivalent to multiple knapsack with divisible item sizes Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

5. Multiple Knapsack with Divisible Item Sizes • NP-hard • can be solved in O(n·max{Tj}k) (extension from single knapsack problem) • heuristics • Greedy1MultipleKnapsack(n,k) • pack the items optimally in the first knapsack • remove the q packed items and the knapsack • call Greedy1MultipleKnapsack(n-q,k-1) with the remaining items and knapsacks • Greedy2 • sort the items according to some criterion (e.g. Profit/size) • insert the items into knapsack using the First Fit heuristic • new heuristic algorithm – this paper Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

6. New Heuristic Algorithm (1/2) (Multiple Knapsack with Divisible Item Sizes) • split the items into groups • all the items in the same group (i) have the same size (sgi) • sg1>sg2>...>sgG (G=total # of groups) • within a group, sort the items in decreasing order of their profits • pri,1≥pri,2≥…≥pri,ni (ni=the # of items in group i) • insert the items into knapsacks using the First Fit heuristic • traverse the items in inreasing order of group number; within a group, in increasing order of item number • choice of knapsack for an item (i,j) – not important => because of divisible item sizes, we insert the same set of items Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

7. New Heuristic Algorithm (2/2) (Multiple Knapsack with Divisible Item Sizes) • repeatedly improve the initial solution • replace an item (i,j) in the knapsack with a subset Q of items outside the knapsack, s.t. pr(i,j)<profit(Q) && profit(Q)-pr(i,j)=maximum • ignore item (i,j) from now on • time complexity • O(n·S·min{n,S·log(S)}) • S=maximum size of an item Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

8. Performance Evaluation(Multiple Knapsack with Divisible Item Sizes) • tested algorithms • new heuristic • Greedy1MultipleKnapsack • Greedy2 (with multiple criteria) • direct solution (extension from single knapsack) • results • most of the cases – solved optimally by the new heuristic • in terms of quality of solution and running time => the new heuristic = clear winner • followed by Greedy1MultipleKnapsack Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

9. Minimum Cost Scheduling (1/3) • sequence of n identical files • to be sent consecutively from the same source to the same destination • sending time per file=1 time unit • k data transfer providers • provider j • cost: Cj per time unit • lease at most ONE time interval of duration at most Tmax,i which includes [T1,j, T2,j] • default network link: cost Li for transferring the ith file • minimize the total cost Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

10. Minimum Cost Scheduling (2/3) • O(n·k) dynamic programming • sort the data transfer providers in increasing order of T2,j: T2,1≤T2,2≤…≤T2,n • Cmin[i,j]=the minimum total cost for sending the first j files using a subset of the first i providers (in the sorted order) Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

11. Minimum Cost Scheduling (3/3) • compute an auxiliary array minpi in O(n) time • for j in [T2,i, T1,i+Tmax,i]: • Cmin[i,j]=min{Cmin[i-1,j], Cmin[i,j-1]+Lj, (j-T1,i)·Ci+minpi[j-Tmax,i]} • Cmin[k,n] – the answer • O(n·k) overall Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

12. Online Resource Management(1/4) • scenario • a resource manager receives resource allocation and reservation requests • request=amount of resource (bandwidth) + time constraints (fixed duration, earliest start time, latest finish time) • model • time – divided into equally-sized time slots • many requests simultaneously => we need low response times => efficient data structure Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

13. Online Resource Management(2/4) • algorithmic framework • block partitioning method • array of n cells • each cell – value vi • divide the cells: n/k blocks of size k • update and query functions: O(k+n/k) • k=sqrt(n) => O(k+n/k)=O(sqrt(n)) • point and range queries • vi=? • qFunc(va, va+1, ..., vb)=? • point and range updates • vi=u • vi=uFunc(u, vi), a≤i≤ b Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

14. Online Resource Management(3/4) • update functions • BPpointUpdate • BPrangeUpdate • BPrangeUpdatePoints • BPrangeUpdatePartialBlock • BPrangeUpdateFullBlock • query functions • BPpointQuery • BPrangeQuery • BPrangeQueryPoints • BPrangeQueryPartialBlock • BPrangeQueryFullBlock Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

15. Online Resource Management(4/4) • solves efficiently many problems • range addition update+range min query • range set update+range maximum sum segment query • ... many more • typical resource reservation requests • range addition update (reserve bandwidth for a full time interval) • range minimum query (check to see if enough bandwidth is available in every time slot) Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

16. Conclusions • offline scheduling problems • maximum profit scheduling • multiple knapsack with divisible item sizes – efficient heuristic • minimum cost scheduling • optimal dynamic programming algorithm • online resource management • algorithmic framework (new) for the block partitioning method (well known) Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths

17. Thank You ! Optimal Scheduling of File Transfers with Divisible Sizes on Multiple Disjoint Paths