1 / 57

Greedy Algorithms

Greedy Algorithms. Greed is good. (Some of the time). Outline. Elements of greedy algorithm Greedy choice property Optimal substructures Minimum spanning tree Kruskal’s algorithm Prim’s algorithm Huffman code Activity selection. Introduction. Concepts

persephone
Download Presentation

Greedy Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Greedy Algorithms Greed is good. (Some of the time)

  2. Outline • Elements of greedy algorithm • Greedy choice property • Optimal substructures • Minimum spanning tree • Kruskal’s algorithm • Prim’s algorithm • Huffman code • Activity selection Chapter 3: Greedy Algorithms

  3. Introduction • Concepts • Choosing the best possible choice at each step. • This decision leads to the best over all solution. • Greedy algorithms do not always yield optimal solutions. Chapter 3: Greedy Algorithms

  4. Elements of Greedy Algorithms

  5. Greedy-choice property • A globally optimal solution is derived from a locally optimal (greedy) choice. • When choices are considered, the choice that looks best in the current problem is chosen, without considering results from subproblems. Chapter 3: Greedy Algorithms

  6. Optimal substructures • A problem has optimal substructure if an optimal solution to the problem is composed of optimal solutions to subproblems. • This property is important for both greedy algorithms and dynamic programming. Chapter 3: Greedy Algorithms

  7. Dynamic programming A choice is made at each step. The choice made at each step usually depends on the solutions to subproblems. Dynamic-programming problems are often solved in a bottom-up manner. Greedy algorithm The best choice is made at each step and after that the subproblem is solved. The choice made by a greedy algorithm may depend on choices so far, but it cannot depend on any future choices or on the solutions to subproblems. A greedy strategy usually progresses in a top-down fashion, making one greedy choice after another, reducing each given problem instance to a smaller one. Greedy Algorithm v.s. Dynamic Programming Chapter 3: Greedy Algorithms

  8. Steps in Design Greedy Algorithms • Determine the optimal substructure of the problem. • Develop a recursive solution. • Prove that at any stage of the recursion, one of the optimal choices is the greedy choice. Thus, it is always safe to make the greedy choice. • Show that all but one of the subproblems induced by having made the greedy choice are empty. • Develop a recursive algorithm that implements the greedy strategy. • Convert the recursive algorithm to an iterative algorithm. Chapter 3: Greedy Algorithms

  9. Shortcuts Design • Form the optimization problem so that, after a choice is made and there is only one subproblem left to be solved. • Prove that there is always an optimal solution to the original problem that makes the greedy choice, so that the greedy choice is always safe. • Demonstrate that, having made the greedy choice, what remains is a subproblem with the property that if we combine an optimal solution to the subproblem with the greedy choice we have made, we arrive at an optimal solution to the original problem. Chapter 3: Greedy Algorithms

  10. Minimum Spanning Tree

  11. Definitions • Let G = (V, E)be an undirected graph. T is a minimum spanning tree of G if T ⊆ E is an acyclic subset that connects all of the vertices and whose total weight w(T )=  w(u, v) is minimized. (u,v)∈T • Let A be a subset of some minimum spanning tree. An edge (u, v) is a safe edge for A if A{(u, v)} is also a subset of a minimum spanning tree. Chapter 3: Greedy Algorithms

  12. Definitions • Let G = (V, E)be an undirected graph. A cut (S, V − S)of G is a partition of V. An edge (u, v)E crosses the cut (S, V − S)if one of its endpoints is in S and the other is in V − S. A cut respects a set A of edges if no edge in A crosses the cut. An edge is a light edge crossing a cut if its weight is the minimum of any edge crossing the cut. Chapter 3: Greedy Algorithms

  13. Theorem Let G = (V, E)be a connected, undirected graph with a real-valued weight function w defined on E, A be a subset of E that is included in some minimum spanning tree for G, (S, V − S)be any cut of G that respects A. If (u, v)is a light edge crossing (S, V − S), thenedge (u, v)is safe for A. S A u x light edge v y V-S Chapter 3: Greedy Algorithms

  14. Proof Let T be a minimum spanning tree that includes A. If (u,v) is in T, then (u,v) is safe for A. If (u,v) is not in T, then T contains an edge (x,y) crossing S and V-S. Since both (u,v) and (x,y) cross S and V-S, there is a cycle for edges in T  (u,v). Another spanning tree T’ can be created from T. S A u x light edge T v y V-S Chapter 3: Greedy Algorithms

  15. S A u x light edge T’ v y V-S Proof S A u x light edge T v y V-S Remove edge (x,y) -> cut the cycle Add edge (u,v) -> create a cycle T’ is created from T. Chapter 3: Greedy Algorithms

  16. Proof Thus,T’is a spanning tree that includes A. Next, we need to show that T’is a minimum spanning tree. From the construction of T’, T’= T-{(x,y)}  {(u,v)}. Thus, w(T’ ) = w(T) - w(x,y) +w(u,v). Since (u,v) is a light edge cross S and V-S, w(u,v)  w(x,y). Thus, w(T’ ) w(T). Since T is a minimum spanning tree, w(T’ ) = w(T). Then, T’isalso a minimum spanning tree . As a result, (u, v)is safe for A. Chapter 3: Greedy Algorithms

  17. Corollary Let G = (V, E)be a connected, undirected graph with a real-valued weight function w defined on E, A be a subset of E that is included in some minimum spanning tree for G, and C = (VC, EC)be a connected component (tree) in the forest GA= (V, A). If (u, v)is a light edge connecting C to another tree in GA, then (u, v)is safe for A. Proof (VC, V−VC)respects A, and (u, v)is a light edge for this cut. Therefore, (u, v)is safe for A. Chapter 3: Greedy Algorithms

  18. Kruskal’s Algorithm • Concept • Build a forest of minimum spanning trees. • Repeatedly connect the trees to create a subset of a minimum spanning tree, until all nodes are covered. • In connecting two trees, choose the edge of the least weight. • Let C1 and C2 denote the two trees that are connected by (u, v). Since (u, v)must be a light edge connecting C1 to some other tree, we need to prove that (u, v)is a safe edge for C1. • Kruskal’s algorithm is a greedy algorithm, because at each step it adds to the forest an edge of least possible weight. Chapter 3: Greedy Algorithms

  19. Example of Kruskal’s Algorithm 12 8 1 9 2 9 5 3 11 7 5 8 3 1 10 Chapter 3: Greedy Algorithms

  20. Kruskal’s Algorithm MST-KRUSKAL(G,w) A =  for each vertex v in V [G] do MAKE-SET(v) sort the edges of E in nondecreasing order by weight w for each edge (u, v)in E, taken in nondecreasing order by weight do if FIND-SET(u) FIND-SET(v) then A = A  {(u, v)} UNION(u, v) return A Chapter 3: Greedy Algorithms

  21. Kruskal’s Algorithm: Complexity MST-KRUSKAL(G,w) A =  for each vertex v in V [G] do MAKE-SET(v) sort the edges of E in nondecreasing order by weight w for each edge (u, v)in E, taken in nondecreasing order by weight do if FIND-SET(u) FIND-SET(v) then A = A  {(u, v)} UNION(u, v) return A O(v) O(e lg e) O((v+e) lg v) e  v2; O(e lg v) Chapter 3: Greedy Algorithms

  22. Prim’s Algorithm • Prim’s algorithm has the property that the edges in the set A always form a single tree. • The tree starts from an arbitrary root vertex r. • Grow the tree until it spans all the vertices inV. • At each step, a light edge is added to the tree A that connects A to an isolated vertex of GA =(V, A). Chapter 3: Greedy Algorithms

  23. Example of Prim’s Algorithm 12 8 1 9 2 9 5 3 11 7 5 8 3 1 10 Chapter 3: Greedy Algorithms

  24. Prim’s Algorithm PRIM(G,w, r) for each u in V[G] do key[u] = ∞ π[u] = NIL key[r] = 0 Q = V[G] while Q  do u = EXTRACT-MIN(Q) for each v in Adj[u] do if v in Q and w(u, v) < key[v] then π[v] = u key[v] = w(u, v) Chapter 3: Greedy Algorithms

  25. Execution of Prim’s Algorithm 12 8 0 9 12 3 8 1 9 2 9 2 1 3 5 3 11 7 5 8 3 5 7 1 10 5 1 10 Chapter 3: Greedy Algorithms

  26. Prim’s Algorithm:Complexity PRIM(G,w, r) for each u in V[G] do key[u] = ∞ π[u] = NIL key[r] = 0 Q = V[G] while Q  do u = EXTRACT-MIN(Q) for each v in Adj[u] do if v in Q and w(u, v) < key[v] then π[v] = u key[v] = w(u, v) Also build min-heap O(v) O(e lg v) O(lg v) O(v lg v) O(lg v) Chapter 3: Greedy Algorithms

  27. Huffman Code

  28. Problem • Find an optimal prefix code representing a set of characters in a file, where each character has a frequency of occurrences. • Prefix code • Codes in which no codeword is also a prefix of some other codeword. • {0, 110, 101, 111, 1000, 1001} is a prefix code. • {0, 110, 101, 111, 1010, 0111} is not prefix code • Optimality • The code yields a file with the minimum number of bits. Chapter 3: Greedy Algorithms

  29. Creating Huffman Code: Example 100 1 64 0 1 0 30 34 1 15 0 1 0 1 0 ฉ: 5 ก: 36 ข: 17 ค: 17 ง: 15 จ: 10 0 100 101 110 1110 1111 Chapter 3: Greedy Algorithms

  30. Optimal Code • An optimal code for a file is always represented by a full binary tree, in which every nonleaf node has two children. • If C is the alphabet from which the characters are drawn and all character frequencies are positive, then the tree for an optimal prefix code has exactly |C| leaves, one for each letter of the alphabet, and exactly |C|−1 internal nodes. Chapter 3: Greedy Algorithms

  31. Tree for 1 letter Tree for 2 letters Tree for 4 letters A 1 1 2 B Tree for 3 letters C A B 4 3 1 2 3 A C 1 2 B 4 A 3 A 1 B 1 2 2 3 Full Binary Trees for Prefix Code Chapter 3: Greedy Algorithms

  32. Tree for 5 letters Tree for 4 letters B B C A C A 4 3 1 2 D 3 1 2 5 4 C D B 4 5 B 3 A C A 1 2 4 3 1 2 Full Binary Trees for Prefix Code Chapter 3: Greedy Algorithms

  33. Creating Huffman Code: Example กขคงจฉ: 100 1 ขคงจฉ: 64 0 1 0 งจฉ: 30 ขค: 34 1 จฉ: 15 0 1 0 1 0 ฉ: 5 ก: 36 ข: 17 ค: 17 ง: 15 จ: 10 100 101 110 1110 1111 0 Chapter 3: Greedy Algorithms

  34. Theorem • A full binary tree for an optimal prefix code for C letters has exactly C leaves, and exactly C−1 internal nodes. Proof by induction. Basis: C=1. If there is one letter, the binary tree requires only 1 leaf node, and 0 internal node. Induction hypotheses: For C< n, the full binary tree for an optimal prefix code for C letters has exactly C leaves, and exactly C−1 internal nodes. Chapter 3: Greedy Algorithms

  35. Theorem • Induction Step: Let T be a full binary tree for an optimal prefix code for C+1 letters. To create a full binary tree or optimal prefix code, we can take a full binary tree for an optimal prefix code for C letters, and add another leaf node L by either • adding a new root node R and put L and the old root of T as its children, or • replacing a leaf node N of T by an internal node and put L and N as its children. In either case, the number of leaf nodes of T is C+1 and the number of internal nodes is C. Chapter 3: Greedy Algorithms

  36. Creating Huffman Code: Algorithms HUFFMAN(C) n = |C| Q = C for i = 1 to n − 1 do allocate a new node z z.left = x = EXTRACT-MIN(Q) z.right = y = EXTRACT-MIN(Q) z.f =x.f + y.f INSERT(Q, z) ►Return the root of the tree. return EXTRACT-MIN(Q) Use min-heap for Q O(n) to build min-heap O(n lg n) O(lg n) Chapter 3: Greedy Algorithms

  37. Greedy-choice property of Huffman Code Let C be an alphabet in which each character c C has frequency f [c], and x and y be two characters in C having the lowest frequencies. • Then, there exists an optimal prefix code for C in which the codewords for x and y have the same length and differ only in the last bit. Chapter 3: Greedy Algorithms

  38. Proof • The idea of the proof is to : • take the tree T representing an arbitrary optimal prefix code • modify it to make a tree representing another optimal prefix code such that the characters x and y appear as sibling leaves of maximum depth in the new tree. • If we can do this, then their codewords will have the same length and differ only in the last bit. Chapter 3: Greedy Algorithms

  39. Proof Let a and b be two characters that are sibling leaves of maximum depth in T . Assume that f [a] ≤ f [b] and f [x] ≤ f [y]. Since f [x] and f [y] are the two lowest leaf frequencies, in order, and f [a] and f [b] are two arbitrary frequencies, in order, we have f [x] ≤ f [a] and f [y] ≤ f [b]. Chapter 3: Greedy Algorithms

  40. x y a b a y x b Proof Exchange the positions of a and x in T to produce T’ . B(T) − B(T’ ) =  f(c) dT (c)−  f(c)dT’ (c) cCcC = f [x] dT (x)+ f [a] dT (a)− f [x] dT’ (x)− f [a] dT’ (a) = f [x] dT (x) + f [a] dT (a)− f [x] dT (a) − f [a] dT (x) = ( f [a] − f [x])( dT (a)− dT (x)) ≥ 0 T ’ T Chapter 3: Greedy Algorithms

  41. a y x b Proof Then, exchange the positions of b and y in T’to produce T’’. Similarly, itdoes not increase the cost, and so B(T’ )− B(T’’ ) 0. Therefore, B(T’’ )≤ B(T). Since T is optimal, B(T)≤ B(T’’ ). Then, B(T’’ )= B(T). Thus, T’’is an optimal tree in which x and y appear as sibling leaves of maximum depth. T ’ T’’ a b x y Chapter 3: Greedy Algorithms

  42. Optimal-substructure Property Let C be a given alphabet with frequency f [c] defined for each character c C, x and y be two characters in C with minimum frequency, C’ be the alphabet C with characters x, y removed and (new) character z added, so that C’ = C − {x, y}  {z}; define f for C’ as for C, except that f [z] = f [x] + f [y], and T be any tree representing an optimal prefix code for the alphabet C ’. • Then the tree T , obtained from T ’ by replacing the leaf node for z with an internal node having x and y as children, represents an optimal prefix code for the alphabet C. Chapter 3: Greedy Algorithms

  43. Proof Show that the cost B(T)can be expressed in terms of the cost B(T’)by considering the component costs. For each c C − {x, y}, we have • dT(c)= dT ’ (c). • f [c] dT (c)= f [c] dT ’ (c). Since dT(x)= dT(y)= dT ’(z)+ 1, we have f [x]dT(x)+ f [y]dT(y)= ( f [x] + f [y])( dT’ (z)+ 1) = f [z] dT ’(z)+ ( f [x] + f [y]) Thus, B(T)= B(T ’)+ f [x] + f [y]. That is, B(T ’)= B(T)− f [x] − f [y] . Chapter 3: Greedy Algorithms

  44. Proof We now prove by contradiction. Suppose T does not represent an optimal prefix code for C. Then there exists a tree T ’’ such that B(T ’’) < B(T). Without loss of generality, T ’’ has x and y as siblings. Chapter 3: Greedy Algorithms

  45. Proof Let T ’’’ be the tree T ’’ with the common parent of x and y replaced by a leaf z with frequency f [z] = f [x] + f [y]. Then, B(T ’’’)= B(T ’’)− f [x] − f [y] < B(T)− f [x] − f [y] = B(T ’). We reach a contradiction to the assumption that T ’ represents an optimal prefix code for C’. Thus, T must represent an optimal prefix code for the alphabet C. Chapter 3: Greedy Algorithms

  46. Interval Scheduling

  47. Problem definition • Let S be {a1, a2, . . . , an} of n proposed activities that wish to use the same resource. • Only one activity can use the resource at a time. • Each activity aihas a start time siand a finish time fi, where 0 ≤ si < fi < ∞. • If selected, activity aitakes place during the half-open time interval [si , fi). • Activities aiand ajare compatible if the intervals [si , fi)and [s j , f j)do not overlap . • The activity-selection problem is to select a largest subset of mutually compatible activities. Chapter 3: Greedy Algorithms

  48. A1 A2 A3 Example S Chapter 3: Greedy Algorithms

  49. Subproblems • Si j= {akS : fi≤ sk < fk≤ s j} • the subset of activities in S that can start after activity aifinishes and finish before activity ajstarts. • Add activities a0 and an+1 such that f0 = 0 and sn+1 =∞. • Then S = S0,n+1, and 0 ≤ i, j ≤ n + 1. Chapter 3: Greedy Algorithms

  50. ak Optimal Solution • Let Ai jbean optimal solution to Si j . • Let c[i, j ] be the number of activities in a maximum-size subset of mutually compatible activities in Si j. • c[i, j ] = 0 for i ≥ j (i.e. Si j= ) • c[i, j ] = c[i, k] + c[k, j ] + 1 if ak be an activity in Ai j. Then, • c[i, j ] =  0 if Si j=  max {c[i, k] + c[k, j ] + 1} if Si j i<k<j akSi j Chapter 3: Greedy Algorithms

More Related