Greedy Algorithms

Greedy Algorithms

, n activities. start time：Si Activity i, finish time：fi Any two activitiesi , j are compatible if their intervals do not overlap, i.e. or • Activity-selection problem: • Problem： • Want to schedule as many compatible activities as possible.

Greedy-Activity-Selector(s,f)： • Complexity： • Thm: The above algorithm schedules optimally. Assume , by quicksort Greedy-Activity-Selector(s, f) n = s.length; { A = {a1} ; k=1; for m=2 to n do return A }

Elements of the Greedy strategy： • Greedy-choice property: Assemble globally optimal solution by making locally optimal choice. • Optimal substructure: An optimal solution to the problem contains within it optimal solutions to subproblems.

worth dollars i-th item： weight units • 0-1 knapsack problem：( NP complete ) • Problem： • Goal：to carry as much value as possible. • Fractional knapsack problem： • Problem： n items n items A person can carry W units. Can take fractions of items ( eg. 1/4, …. ) in decreasing order Example：W=50 item1：取 item1 , item2 及 item2： 2/3 的 item3. item3：

a b c d e f Frequency (x1000) 9 45 13 12 16 5 Fixed-length codeword 000 001 010 011 100 101 variable-length codeword 0 101 100 111 1101 1100 • Huffman codes： Use fixed-length codeword： Total Use variable-length codeword： Total

100 a : 45 c : 12 d : 16 b : 13 No codeword is also a prefix of some other codes. • Prefix codes：字首 T： 0 1 55 1 0 cost of the tree T. 30 25 0 0 1 1 14 0 1 f : 5 e : 9

e : 9 e : 9 f : 5 f : 5 e : 9 f : 5 f : 5 e : 9 f : 5 e : 9 f : 5 c : 12 b : 13 d : 16 a : 45 c : 12 b : 13 d : 16 b : 13 c : 12 b : 13 a : 45 b : 13 a : 45 d : 16 d : 16 c : 12 c : 12 a : 45 c : 12 b : 13 d : 16 a : 45 d : 16 a : 45 14 14 14 14 14 1 1 1 1 1 0 0 0 0 0 25 25 1 1 0 0 30 30 30 55 1 1 1 0 0 0 1 0 25 1 0 (a) e : 9 (b) • Constructing a Huffman code： (c) (d) (f) (同前頁：T) 55 1 0 (e) 55 1 0 25 1 0

2 4 3 3 2 2 5 1 1 1 6 6 2 5 3 2 5 4 2 2 3 4 3 3 2 3 5 1 1 1 2 4 2 3 11 23 0 1 11 12 1 0 0 1 6 6 I N 1 0 1 0 21 3 1 0 O B A 1 0 11 10 1 0 F G 5 1 E 0 2 1 0 T C P Example：” A SIMPLE STRING TO BE ENCODED USING A MINIMAL NUMBER OF BITS.” source： A B C D E F G I L M N D P R S T U 60 0 1 37 0 1 16 1 0 8 8 1 1 0 0 4 4 1 0 M S 1 0 U L D R 0 1 1 0 1 1 1 1 0 0 1 0 0 1 1 0 1 0 1 1 0 1 0 1 1 0 0 0 1 1 1 0 0 1 1 1 1 0 0 1 1 1 0 1 1 1 0 1 1 1 0 0 1 0 0 0 0 1 1 1 1 1 1 1 1 1 0 1 1 0 1 0 0 1 1 1 0 1 0 1 1 1 0 0 1 1 1 1 1 1 0 0 0 0 1 1 0 1 0 0 0 1 0 0 1 0 1 1 0 1 1 0 0 1 0 1 1 0 1 1 1 1 0 0 0 0 1 0 0 1 0 0 1 0 0 0 0 1 1 1 1 1 1 1 0 1 1 0 1 1 1 1 0 1 0 0 0 1 0 0 0 0 0 1 1 0 1 0 0 1 1 0 1 0 0 0 1 1 1 1 0 0 0 1 0 0 0 0 1 0 1 0 0 1 0 1 1 1 0 0 1 0 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 1 1 1 0 1 0 1 0 0 1 1 1 0 1 1 1 0 0 1

Huffman(C)： • Algorithm： • Complexity： Huffman(C) { // Q：priority queue for i=1 to n-1 do { z = allocate-Node() ; x = left(z) = Extract-min(Q) ; y = right(z) = Extract-min(Q) ; f(z) = f(x) + f(y) ; Insert(Q , z) ; } return Extract-min(Q) ; }

C C • Lemma 2： C : alphabet set,each c in C with frequency f(c). Let x, y in C with the lowest frequencies f(x) and f(y). Then there exists an optimal prefix code, where the code words for x and y have the same length and differ in the last bit. T ( optimal ) Proof： T’ T’’ = = 同理

T 是 C 之最佳之 prefix code. T’：full binary tree representing an optimal prefix code over C’ • Lemma 3： leaves T, obtained from T’ by replacing z with an internal node with children x and y, is an optimal prefix code. Proof： For any 若 T 不是 C 的最佳 prefix code. 則可找到 T’’ 使 B(T’’) < B(T)

Thm： Procedure Huffman produces an optimal prefix code. Proof: By Lemma 2 and 3.

Matroids： • S：a finite set • I：a nonempty family of subsets of S ( 即 I的元素為集合 ) 並滿足：若且則 • 若且則存在使得 • Graphic matroid： • andA：acyclic. independent subset ( hereditary property ) ( exchange property ) Example：Matric matroid if columns in A are linearly independent. is a graph. ( i.e. A forms a forest )

Example: Matric Matroid: Example: Graphic Matroid:

Thm 5： IfG is anundirected graph thenis a matroid. Proof： 1. E：finite 2. is hereditary, acyclic graph 之部分仍為 acyclic. 3. 假設 A,B 為 G 中之 forests 且 A：包含個 trees. B：包含個 trees. ( B 的樹較少棵) ( A single node is counted as a tree.) 故在 B 中有一樹 T 包含 A 中的 2棵樹的vertices. Why? 亦即 T 中存在一 edge (u,v) 使得 u 和 v 分佈在 A 中的兩棵樹. 故將 (u,v) 加到 A 中不會產生 cycle. 故由 1. 2. 3. 定理得證.

Thm 6： All maximal independent subsets in a matroid have the same size. Proof： , A is maximal if it has no extension. 即不存在使得假設 A：maximal independent subset. B：maximal independent subset 且存在使得 A 為 maximal.

Given , find has maximal possible weight. • Weighted Matroid： • Definition： • Greedy algorithms on a weighted matroid • Problem： , weight function：w(x) for each Example：Minimum Spanning Tree ：weight function defined on E. Define , and Let A be a maximal independent set in I. Then A corresponds to a spanning tree in G.

Algorithm： Greedy(M,w) { Sort M.S into monotonically decreasing order by weight w ; For each , taken in monotonically decreasing order by w(x) ; do if then Return A } 若則上述需步驟。

令為一加權 ( weight ) matroid. S 依加權函數 w, 排列由大到小. • Lemma 7：令 x為 S 中第一個使之independent元素 ( 但此 x並不一定存在 ). 若此 x存在，則存在一最佳( 即 w(A) 最大 ) 之 A Proof： Why ? 若沒有上述 x存在，則為唯一的 independent set. 現假設 B 為一非空之 optimal subset. 若 , 則取 A 等於 B 故得證. 若 , 則 B 中之元素的 weight 不會比 x之 weight 大. 假設但已知令由 exchange property, 可將 A 擴充至而且 A 仍保持為 independent. 取 B 為 optimal A 為 optimal 且含 x.

Lemma 8-9： 令為任一 matroid. 若且 x不為之 extension, 則 x不為任一 independent set 之 extension. Proof：假設 x為 A 之 extension 但不為之 extension. independent. x為 A 之 extension. 由假設 x不是之 extension. independent.

Lemma 10(Matroids optimal-substructure)： 令 x為 Greedy 演算法中第一個被選入的元素. 尋找 M中包含 x 之 maximum-weight independent subset 可以被轉化為尋找 matroid 之 maximum weight ind. subset Proof：若且且 A：maximum weighted 又故由 M 中含 x 之 maximum-weight solution 可找到 M’ 之 maximum-weight solution. 反之亦然.

Thm 11： 給定 , 則 Greedy(M,w) 可以找到 optimal solution. Proof: By Lemma 16.9, pass over all elements that are not extensions of . By lemma 16.7, once the first x is selected, Greedy is safe to add x to A. Lemma 16.10, implies that the remaining problem is one of finding an Optimal subset in the matroid M’ which is the contraction of M by x.

1 2 3 4 5 6 7 4 2 4 3 1 4 6 70 60 50 40 30 20 10 • A task-scheduling problem： • S={1,2,…..,n} n unit-time tasks. • Deadlines： task i 需在 di前完成. • Penalties：若 task i 不能在 di前完成，則罰 wi 若 task i 能在 di前完成者無 penalty. 目標：安排一執行順序使 penalty 最小. Example： Schedule：< 2 4 1 3 7 5 6 > penalty 20+30=50.

k k+1 j i • Def： • 在一 schedule 中： • late task：if it finishes after its deadline. • early task：if it finishes before its deadline. • Early-first form：early tasks precede the late tasks. • Canonical form：same as early-first form and the early tasks are scheduled in order of monotonically increasing deadlines. • a set A of task is independent：若存在一 schedule 使得 A 中無 late schedule. • 故任一 schedule 中之 early tasks 形成一 independent set. 令I表所有 independent set 之集合. • 如何判定一 task 集合是否為 independent？ k k+1 i j 若 i,j：early task t=1,2,…,n ，令 Nt(A) 表 A 中之 tasks 其 deadline t 之 task 個數.

Lemma 12： 令 A：tasks 所形成之集合. 則下列敘述為等價 1. A：independent. 2. For t=1,2,…n， 3. 若 A 中之 tasks 依 deadlines ( nondecreasing ) 排序，則無 late task. Proof：若 , 則 A 中存在 late task. 故顯然.

Thm 13： S={ unit tasks with deadline } (S,I ) is a matroid. I={ independent sets of tasks } Proof： • Clearly. • 由上述 independent set 之定義知其滿足 matroid 之第 2 個條件。 • Suppose A, B  I and |B| > |A|. • Let k be the largest t s.t. Such t exist. It holds at least for t=0. Since Nn(B)=|B| and Nn(A)=|A|, but |B|>|A|, we must have k < n and for all k+1  j  n, Nj(B) > Nj(A). • Thus, B contains more tasks with deadline k+1 than A does. • Let task i be in B-A with deadline k+1 and A’ = A {i}. • By property 2 of lemma 12, we show A’ is ind. For t in [0, k], Nt(A’)=Nt(A)  t, since A is ind. For t in (k, n], we have Nt(A’) Nt(B) t, since B is ind. Thus A’ is ind. And (S, I ) is a matroid.

Solution by Greedy Algorithm • Sort w1,….., wn in decreasing order • Check A  {i}  I ? I.e. is A  {i} independent? I.e does Nt(A  {i})  t hold for t=0,…,n? • Time complexity: O(n2).

Greedy Algorithms