Greedy Algorithms

Greedy Algorithms CSC 331: Algorithm Analysis Greedy Algorithms

A 1 C E 3 4 4 2 4 5 B D F 4 6 Cable TV Network Suppose you are asked to connect cable TV to houses in a new neighborhood. You may only bury the cables along certain paths. Some of these paths may be more expensive because they are longer or the cable must be buried deeper. CSC 331: Algorithm Analysis Greedy Algorithms

A A C C E E 3 4 4 B B D D F F 6 Cable TV Network 1 2 4 5 4 What is the cheapest possible way to connect all of the houses to each other? The solution must be connected and acyclic. Undirected graphs of this kind are called trees. The particular tree we want is the one with minimum total weight, known as the minimum spanning tree. CSC 331: Algorithm Analysis Greedy Algorithms

Output: A tree T=(V,E’), with E’ ⊆ E, that minimizes weight(T) = Minimum Spanning Tree Input: An undirected graph G=(V,E); edge weights we. CSC 331: Algorithm Analysis Greedy Algorithms

1 1 A A C C E E 3 4 4 2 2 4 4 5 5 B B D D F F 6 4 4 Minimum Spanning Tree The minimum spanning tree has a cost of 16. However, this is not the only optimal solution. CSC 331: Algorithm Analysis Greedy Algorithms

Greedy Algorithms Greedy algorithms build up a solution piece by piece, always choosing the next piece that offers the most obvious and immediate benefit. Although such an approach can be disastrous for some computational tasks, there are many for which it is optimal. CSC 331: Algorithm Analysis Greedy Algorithms

Kruskal’s Algorithm Start with an empty graph and then select edges from E according to the following rule: Repeatedly add the next lightest edge that doesn’t produce a cycle. CSC 331: Algorithm Analysis Greedy Algorithms

Kruskal’s Algorithm 6 5 C E A 4 1 5 2 3 4 B D F 2 4 So how does your algorithm know if an edge will produce a cycle? CSC 331: Algorithm Analysis Greedy Algorithms

Kruskal’s Algorithm Cut property Suppose edges X are part of a minimum spanning tree of G=(V,E). Pick any subset of vertices S for which X does not cross between S and V - S, and let e be the lightest edge across this partition. Then X ∪ {e} is part of some minimum spanning tree. CSC 331: Algorithm Analysis Greedy Algorithms

A 1 A C 3 C E E A C E A C E A C E Edges X: MST T: The cut: MST T’: e 2 2 1 2 3 B B D D F F B D F B D F B D F 1 4 S V - S Kruskal’s Algorithm CSC 331: Algorithm Analysis Greedy Algorithms

Kruskal’s Algorithm • procedure kruskal (G,w) • Input: A connected undirected graph G=(V,E) with • edge weights we • Output: A minimum spanning tree defined by the • edges X • for all u ∈ V: • makeset(u) • X = {} • sort the edges E by weight • for all edges {u,v} ∈ E, in increasing weight order • if find(u) ≠ find(v): • add edge {u,v} to X • union (u,v) CSC 331: Algorithm Analysis Greedy Algorithms

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1 David Luebke 128/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1? David Luebke 158/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2? 19 9 14 17 8 25 5 21 13 1 David Luebke 178/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5? 21 13 1 David Luebke 198/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8? 25 5 21 13 1 David Luebke 218/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9? 14 17 8 25 5 21 13 1 David Luebke 238/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13? 1 David Luebke 258/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14? 17 8 25 5 21 13 1 David Luebke 278/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17? 8 25 5 21 13 1 David Luebke 298/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19? 9 14 17 8 25 5 21 13 1 David Luebke 308/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21? 13 1 David Luebke 318/28/2014

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25? 5 21 13 1 David Luebke 328/28/2014

Kruskal’s Running Time Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } David Luebke 358/28/2014

Kruskal’s Running Time • Sort edges: O(E lg E) • O(V) MakeSet()’s • O(E) FindSet()’s • O(V) Union()’s • Upshot: • Best disjoint-set union algorithm makes above 3 operations take O(E(E,V)),  almost constant • Overall thus O(E lg E), almost linear w/o sorting • Note that this is also O(E lg V), since E = O(V2) David Luebke 368/28/2014

The Cut Property The cut property tells us in general terms is that any algorithm conforming to the following greedy schema is guaranteed to work: X = { } (edges picked so far) repeat until |X| = |V| - 1: pick a set S ⊂ V for which X has no edges between S and V-S let e ∈ E be the minumum-weight edges between S and V-S X = X ∪ {e} CSC 331: Algorithm Analysis Greedy Algorithms

Prim’s Algorithm In Prim’s algorithm, the intermediate set of edges X always forms a subtree, and S is chosen to be the set of this tree’s vertices. On each iteration, the subtree defined by X grows by one edge (the lightest edge between a vertex in S and a vertex outside of S). CSC 331: Algorithm Analysis Greedy Algorithms

Prim’s Algorithm • procedure prim (G, w) • Input: A connected undirected graph G = (V,E) with • edge weights we • Output: A minimum spanning tree defined by the prev • for all u ∈ V: • cost(u) = ∞ • prev(u) = nil • pick any initial node u0 • cost(u0) = 0 • H = makequeue(V) (priority queue - cost values) • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v CSC 331: Algorithm Analysis Greedy Algorithms

Prim’s Algorithm • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v 6 4 9 5 14 2 10 15 3 8 David Luebke 408/28/2014

Prim’s Algorithm • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v  6 4 9 5    14 2 10 15    3 8  David Luebke 418/28/2014

Prim’s Algorithm • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v  6 4 9 5    14 2 10 15 u 0   3 8  Pick a start vertex u David Luebke 428/28/2014

Prim’s Algorithm • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v  6 4 9 5    14 2 10 15 u 0   3 8  Red vertices have been removed from Q David Luebke 438/28/2014

Prim’s Algorithm • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v  6 4 9 5    14 2 10 15 u 0   3 8 3 Red arrows indicate parent pointers David Luebke 448/28/2014

Prim’s Algorithm • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v  6 4 9 5 14   14 2 10 15 u 0   3 8 3 David Luebke 458/28/2014

Prim’s Algorithm • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v  6 4 9 5 14   14 2 10 15 0   3 8 3 u David Luebke 468/28/2014

Prim’s Algorithm • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v  6 4 9 5 14   14 2 10 15 0 8  3 8 3 u David Luebke 478/28/2014

Prim’s Algorithm • while H is not empty: • v = deletemin(H) • for each {v,z} ∈ E: • if cost(z) > w(v,z): • cost(z) = w(v,z) • prev(z) = v  6 4 9 5 10 2  14 2 10 15 0 8  3 8 3 u David Luebke 508/28/2014

Greedy Algorithms