Download
1 / 52
620 likes | 1.33k Views
Greedy Algorithms. Intuition: At each step, make the choice that is locally optimal. Does the sequence of locally optimal choices lead to a globally optimal solution ? Depends on the problem Sometimes guarantees only an approximate solution Examples: Shortest paths: Dijkstra
E N D
Greedy Algorithms • Intuition: At each step, make the choice that is locally optimal. • Does the sequence of locally optimal choices lead to a globally optimal solution? • Depends on the problem • Sometimes guarantees only an approximate solution • Examples: • Shortest paths: Dijkstra • Minimum spanning trees: Prim and Kruskal • Compression: Human coding • Memory allocation: First fit, Best fit
Greedy Method • Greedy algorithms are typically used to solve optimization problem. Most of these problems have n inputs and require us to obtain a subset that satisfies some constraints. Any subset that satisfies these constraints is called a feasible solution. We are required to find a feasible solution that either minimizes or maximizes a given objective function. In the most common situation we have: • C: A set (or list) of candidates; • S: The set of candidates that have already been used; • feasible(): A function that checks if a set is a feasible solution; • solution(): A function that checks if a set provides a solution; • select(): A function for choosing most promising candidates; • An objective function that we are trying to optimize.
The Generic Procedure • function greedy(C: set): set; • begin • S :=Ø; /* S is the set in which we construct the solution */ • while (not solution(S) and C ≠ Ø ) do • begin • x := select(C); • C := C - {x}; • if feasible(S{x}) then S := S {x}; • end; • if solution(S) then return(S) else return(Ø); • end; • The selection function use the objective function to choose the most promising candidates from C
Example: Coin Change • We want to give change to a customer using the smallest possible number of coins (of units 1, 5, 10 and 25, resp.). • Greedy algorithm will always find the optimal solution in this case. • If 12-unit coins are added, it will not necessarily find the optimal solution. • Greedy method might even fail to find a solution despite the fact that one exist. (Consider coins of 2, 3, and 5 units.) E.g., 15 = (12, 1, 1, 1), (10, 5) is optimal E.g., 6 = 5 + ? (3, 3) is optimal
The Knapsack Problems • 0-1 knapsack problem: A thief robbing a store finds n items, the ith item has weight ci and is worth vi dollars (ci & vi are integers) If the thief can carry at most B weight in his knapsack, what items should he take to make the most profitable? • Fractional knapsack problem: same as above, except that the thief can take fractions of items (e.g., an item might be gold dust) • Integer knapsack problem: same as 0-1 knapsack problem, except that the number of each item is unlimited. What is the input size of the problem? n * log B
The Knapsack Problems • Optimal substructure property: consider the most valuable load that weights pounds in 0-1 problem, if we remove item j from this load, the remaining load must be the most valuable load weighting at most B - cjthat can be taken from the n - 1original items excluding j. • In fractional problem, if we remove w pounds of item j, the remaining load must be the most valuable load weighting at most B - wthat can be taken from the n - 1original items plus cj - wpounds of item j.
The Knapsack Problems • The Integer Knapsack Problem Maximize vi ≥ 0, xi: nonnegative integers Subject to ≤ B ci ≥ 0, B > 0 • The0-1 Knapsack Problem: same as integer knapsack except that the values of xi's are restricted to 0 or 1. • The Fractional Knapsack Problem Maximize vi ≥ 0, 1 ≥ xi ≥0 Subject to ≤ B ci ≥ 0, B > 0
The Knapsack Problems • Let f(k, a) = , 0 ≤ k ≤ n, 0 ≤ a ≤ B. • For integer knapsack problems, we have f(k, a) = max{f(k - 1, a), f(k, a - ck) + vk}, and • For 0-1 knapsack problems, we have f(k, a) = max{f(k - 1, a), f(k - 1, a - ck) + vk}. • What we want to compute is f(n, B), which depends recursively on at most nB previous terms: f(k, a), 0 ≤ k ≤ n, 0 ≤ a ≤ B.
The Knapsack Problems • For the fractional knapsack problem, a greedy approach can solve it: Rearrange the objects so that Then, for items i= 1 to n, take as much of item i as there is while not exceeding weight limit B. • Running time is O(n log n) • Remark: Dynamic programming is not applicable to the fractional knapsack problem (why?), while greedy method may fail to find optimal solutions for the integer/0-1 knapsack problems.
Greedy does not work for 0-1 problem! Fractional Knapsack: Optimal solution is $240.
Job Scheduling • We want to schedule given n jobs. Job i requires running time ti • Find the best order to execute the jobs to minimize the average completion time • Sample input: (J1, 10), (J2, 4), (J3, 5), (J4, 12), (J5, 7) • One possible schedule: J3, J2, J1, J5, J4. Total completion time = 5+9+19+26+38= 97 • Optimal schedule: J2, J3, J5, J1, J4 Total completion time = 4+9+16+26+38 = 93
Greedy Scheduling Algorithm • Schedule the job with smallest running time first. • Schedule jobs in increasing order of running times. • Correctness: Suppose scheduling order is i1, , in. If tij > tij+1, then swap jobs ij and ij+1: • Completion times for jobs before ij and ij+1 do not change • Time of job ij increases by tij+1 • Time of job ij+1 decreases by tij • So total completion time decreases by tij- tij+1 • Thus, optimal completion time can happen only if the list i1, , in is sorted.
Multiprocessor Case • Suppose that jobs can be scheduled on k processors. • Same intuition: since running times of earlier jobs contribute to completion times of latter jobs, schedule shorter jobs first. • Algorithm: • Sort jobs in increasing order of running times: i1, , in. • Schedule i1 on processor P1, i2 on Pi2, , ik on Pk, ik+1 on Pk+1, and so on in a cycle • Sample input: (J1, 10), (J2, 4), (J3, 5), (J4, 12), (J5, 7) and 2 processors • Solution: J2, J5, J4 on P1; J3, J1 on P2 Total completion time: (4+11+23)+(5+15)=58
Final Completion Time • Suppose we want to minimize the maximum completion time instead of total (or average) completion time • makes sense only in multiprocessor case • Scheduling shorter job first does not seem to have any advantage • In fact, previous greedy algorithm does not give optimal solution • Optimal solution: J1, J2, J3 on P1; J4, J5 on P2. Final completion time =19 • In fact, no greedy strategy works. We may be forced to try out all possible ways to split jobs among processors. This is an NP-complete problem!
More on Scheduling • We have looked at a very simple variant • In practice, there are many complications: • Jobs are not known a priori, but they arrive in real-time • Different jobs have different priorities • It may be OK to preempt jobs • Jobs have different resource requirements, not just processing time • A hot research topic: scheduling for multimedia applications
Minimum Spanning Trees (MST) • A tree is a connected graph with no cycles. • A spanning tree of a connected graph G is a subgraph of G which has the same set of vertices of G and is a tree. • A minimum spanning tree of a weighted graph G is the spanning tree of G whose edges sum to minimum weight. (If G is not connected, we can talk about a minimum spanning forest.) • There can be more than one minimum spanning tree in a graph (consider a graph with identical weight edges.) • The minimum spanning tree problem has a long history, the 1st algorithm dates back at least to 1926!.
Minimum Spanning Trees (MST) • Minimum spanning tree is always taught in algorithm courses since (1) it arises in many applications, (2) it is an important example where greedy algorithms always give the optimal answer, and (3) Clever data structures are necessary to make it work. • A set of edges is a solution if it constitutes a spanning tree, and it is feasible if it does not include a cycle. A feasible set of edges is promising if it can be completed to form an optimal solution. • An edge touches a set if exactly one end of the edge is in it.
Lemma • Let G = (V, E) be a connected undirected graph where the length of each edge is given. Let V' ⊆V and E' ⊆ E be a promising set of edges such that no edges in E' touches V'. Let e be the shortest edge that touches V'. Then E' ∪{e} is promising. E’
Kruskal's Algorithm • Idea: Using union and find algorithms. • T = ; • Sort all edges by weight. • Make each node a singleton set. • For all e = (u, v) E in sorted order do: • If find(u) find(v) then add e to T and union(u; v). (else discard e)
Example sorted
Analysis of Kruskal's Algorithm • Let m = |E|. • O(m log m) = O(m log n) to sort edges. • O(n) for initialize n sets. • The repeat-loop is executed at most m times. • In the worst case O((2m+n-1)log*n) for all the find and union operations, since there are at most 2m find operations and n-1 union operations. • At worst, O(m) for the remaining operations. • Total time complexity is: O(m log n).
Exercises • Prove that Kruskal's algorithm works correctly. The proof which uses the lemma in the previous page, is by induction on the number of edges selected until now. • What happens, if by mistake, we run the algorithm on a graph that is not connected? • What is the complexity of the algorithm if the list of edges is replaced by an adjacent matrix?
Prim's Algorithm Select an arbitrary vertex to start. While (there are fringe vertices) select minimum weight edge between tree and fringe add the selected edge and vertex to the tree • The main loop of the algorithm is executed n-1 times; each iteration takes O(n) time. Thus Prim's algorithm takes O(n2) time. • Compare the above two algorithm according to the density of the graph G = (V, E). • What happens to the above two algorithms if we allow edges with negative lengths?
Single-Source Shortest Paths (SSSP) • The Problem: Given an n-vertex weighted graph G = (V, E) and a vertex v in V, find the shortest paths from v to all other vertices. • Edges in G may have positive, zero or negative weights, but there is no cycle of negative weight.
D&C approach • procedure SP(i, j, d); • if i ≠ j • then begin • d := D[i, j]; • for k := 1 to n do d := min(d, SP(i,k)+SP(k,j)); • end • else d := 0; • Needs EXPONENTIAL time.
Floyd's algorithm: DP approach Idea: the shortest path from i to j without passing through nodes numbered > k • for k := 1 to n do • for i: = 1 to n do • for j := 1 to n do • D[i, j] := min(D[i, j], D[i, k]+D[k, j]); • Dynamic programming approach. • Takes O(n3) time. • Solve all-pairs shortest path problem.
Dijkstra's algorithm: Greedy approach • C := {2, 3, ..., n}; • for i := 2 to n do near[i] := D[l, i]; • repeat n-2 times • v := some element of C minimizing near[v]; • C := C - {v}; • for each w in C do near[w] := min(near[w], near[v]+D[v,w]); • Takes O(n2) time.
Dijsktra's Shortest-Path Algotithm • Algoritllm SHORTEST- PATH(u) • begin • for i = 1 to n do • near[i] = D[u, i]; • P[i] = u; • V' = V - {u}; • near[u] = 0; • while (V' is not empty) do • Select v such that near[v] = min{near[w]: w ∈V'}; • V' = V' - {v}; • for w ∈ V' do
Algorithm Continued 12. if (near[w] > near[v] + D[v, w]) 13. then near[w] = near[v] + D[v, w] 14. P[w] = v; (* P[w] is the parent of w *) 15. for w ∈V do 16. (* print the shortest path from w to u. *) 17. print w; 18. q = w; 19. while (q ≠u) do 20. q = P[q]; print q; 21 print u; 22. end.
Greedy Heuristics • Often used in situations where we can (or must) accept an approximate solution instead of an exact optimal solution. • Graph Coloring Problem: Given G = (V, E), use as few colors as possible to color the nodes in E so that adjacent nodes are of different color.
Greedy Solutions • Algorithm 1: • Arrange the colors according to some order. • For each node v in V, find the smallest color which has not yet been used to paint any neighbors of v and paint v by this color. • Algorithm 2: • Choosing a color and an arbitrary starting node, and then considering each other node in turn, painting it with this color if possible. • When no further nodes can be painted, we choose a new color and a new starting node that has not yet been painted. Then we repeat as in step 1 until every node is painted.
Example of Graph Coloring • Greedy: 5 colors Optimal: 2 colors • Greedy approach may find optimal solution in some cases, but it may also give an arbitrary answer.
Traveling Salesperson Problem • The Problem: Find, in an undirected graph with weights on the edges, a tour (a simple cycle that includes all the vertices) with the minimum sum of edge-weights. • Algorithm: • Choose the edge with minimum weight. • Accept the edge (under the consideration together with already selected edges) if it • does not cause a vertex to have degree 3 or more, and • does not form a cycle, unless the number of selected edges equals the number of the vertices of the graph. • Repeat the above steps until n edges have been selected.
Example • Edges are chosen in the order: 3 4 5 6 (1, 2) (3, 5) (4, 5) (2, 3) 7 8 9 10 (1, 5) (2, 5) (3, 4) (1, 3) 11 12 1525 (1, 4) (2, 4) (4, 6)(1, 6) • Thus, the solution is 1-2-3-5-4-6-1 with length = 58. • The optimal solution is 1-2-3-6-4-5-1 with length = 56.
Matroids • Pair (S, I) where S is a nonempty, finite set, and I is a family of subsets of S such that • I; • If J I and I J, then I I (hereditary property) • If I, J I and |I| < |J|, then there exists an x J - I such that I {x} I (exchange property). • Elements of I : independent sets • Subsets of S not in I : dependent sets
Examples • Graphic matroid: G = (V, E) connected undirected graph • S := E • I := set of forests (=acyclic subgraph) in G • Claim: (S; I) is a matroid. • Proof: • The graph (V, ) is a forest. • Any subset of a forest is a forest. • Let I and J be forests with |I| < |J|:
