Week 3: Minimum Spanning Trees

Week 3: Minimum Spanning Trees Definition of MST Generic MST algorithm Kruskal's algorithm Prim's algorithm

Problem: Laying Telephone Wire Central office

Wiring: Naïve Approach Central office Expensive!

Wiring: Better Approach Central office Minimize the total length of wire connecting the customers

Definition of MST • Input: connected, and undirected Graph G=(V,E) • Each edge (u,v) in Ecarries a weight w(u,v) • Also referred to as cost (length of edge) to connect u and v. • Output: A subset T of E that connects all of the vertices in V and whose total weight is minimized. • Since the total weight is minimized, the subset T must be acyclic (no circuit). • Thus, T is a tree. We call it a spanning tree. • The problem of determining the tree T is called the minimum-spanning-tree problem.

What is a Minimum-Cost Spanning Tree • Example: The graph Has 16 spanning trees. Some are: The graph has two minimum-cost spanning trees, each with a cost of 6:

Complete Graph All 16 of its Spanning Trees

Minimum Spanning Trees The Minimum Spanning Tree for a given graph is the Spanning Tree of minimum cost for that graph. Complete Graph Minimum Spanning Tree 7 2 2 5 3 3 4 1 1

Application of MST: an example • In the design of electronic circuitry, it is often necessary to make a set of pins electrically equivalent by wiring them together. • To interconnect n pins, we can use n-1 wires, each connecting two pins. • We want to minimize the total length of the wires. • Minimum Spanning Trees can be used to model this problem.

Electronic Circuits:

8 7 9 4 2 11 14 i 4 g h c d e f b a 7 6 10 8 1 2 Here is an example of a connected graph and its minimum spanning tree: Notice that the tree is not unique: replacing (b,c) with (a,h) yields another spanning tree with the same minimum weight.

Growing a MST(Generic Algorithm) GENERIC_MST(G,w) 1 A:={} 2 while A does not form a spanning tree do 3 find an edge (u,v) that is safe for A 4 A:=A∪{(u,v)} 5 return A • Set A is always a subset of some minimum spanning tree. This property is called the invariant Property. • An edge (u, v) is a safe edge for A if adding the edge to A does not destroy the invariant. • A safe edge is just the CORRECT edge to choose to add to T.

8 7 9 4 2 S↑ 11 14 ↑ S i 4 h c d e g a b f 7 6 10 8 V-S↓ ↓ V-S 1 2 • This figure shows a cut (S,V-S) of the graph. • The edge (d,c) is the unique light edge crossing the cut.

Safe edge We need some definitions and a theorem. • A cut (S,V-S) of an undirected graph G=(V,E) is a partition of V. • An edge crosses the cut (S,V-S) if one of its endpoints is in S and the other is in V-S. • An edge is a light edge crossing a cut if its weight is the minimum of any edge crossing the cut.

1 1 1 2 1 1 1 1 Theorem 1 Given: • G=(V, E) be a connected, undirected graph with a real-valued weight function w defined on E. • A be a subset of E that is included in some minimum spanning tree for G. • Let (S,V-S) be any cut of G such that for any edge e=(u, v) in A, {u, v}  S or {u, v}  (V-S). • Let (x, y) be a light edge crossing (S,V-S). Then, edge (x, y) is safe for A. Proof: Let Topt be a minimum spanning tree. (Blue) A --a subset of Topt and (x, y)-- a light edge crossing (S, V-S) • If (x, y) is NOT safe, then(x, y) is not in T. • (See the red edge in Figure) • There MUST be another edge (x’, y’) in Topt crossing (S, V-S).(Green) • We can replace (x’, y’) in Topt with (x, y) and get another tree T’opt • Since (x, y) is light , T’optis also optimal.

Corollary Given: Let G=(V, E) be a connected, undirected graph with a real-valued weight function w defined on E. • Let A be a subset of E that is included in some minimum spanning tree for G. • Let C be a connected component (tree) in the forest GA=(V, A). • Let (u, v) be a light edge (shortest among all edges connecting C with others components) connecting C to some other component in GA. • Then, edge (u,v) is safe for A. (For Kruskal’s algorithm)

The algorithms of Kruskal and Prim • The two algorithms are elaborations of the generic algorithm. • They each use a specific rule to determine a safe edge in line 3 of GENERIC_MST. • In Kruskal's algorithm, • The set A is a forest. • The safe edge added to A is always a least-weight edge in the graph that connects two distinct components. • In Prim's algorithm, • The set A forms a single tree. • The safe edge added to A is always a least-weight edge connecting the tree to a vertex not in the tree.

Kruskal’s Algorithm

Complete Graph 4 B C 4 2 1 A 4 E F 1 2 D 3 10 5 G 5 6 3 4 I H 3 2 J

4 1 A B A D 4 4 B C B D 4 10 2 B C B J C E 4 2 1 1 5 C F D H A 4 E F 1 6 2 2 D J E G D 3 10 5 G 3 5 F G F I 5 6 3 4 3 4 I G I G J H 3 2 J 2 3 H J I J

Sort Edges (in reality they are placed in a priority queue - not sorted - but sorting them makes the algorithm easier to visualize) 1 1 A D C F 2 2 C E E G 4 2 3 B C H J F G 4 2 1 3 3 G I I J A 4 E F 1 4 4 2 A B B D D 3 10 5 G 4 4 B C G J 5 6 3 4 5 5 I F I D H H 3 2 J 6 10 D J B J

Add Edge 1 1 A D C F 2 2 C E E G 4 2 3 B C H J F G 4 2 1 3 3 G I I J A 4 E F 1 4 4 2 A B B D D 3 10 5 G 4 4 B C G J 5 6 3 4 5 5 I F I D H H 3 2 J 6 10 D J B J

Cycle Don’t Add Edge 1 1 A D C F 2 2 C E E G 4 2 3 B C H J F G 4 2 1 3 3 G I I J A 4 E F 1 4 4 2 A B B D D 3 10 5 G 4 4 B C G J 5 6 3 4 5 5 I F I D H H 3 2 J 6 10 D J B J

Minimum Spanning Tree Complete Graph 4 B C 4 B C 4 4 2 1 2 1 A E A 4 F E 1 F 1 2 D 2 D 3 10 G 5 G 3 5 6 3 4 I I H H 3 2 3 J 2 J

Kruskal's algorithm(basic part) • (Sort the edges in an increasing order) • A:={} • while E is not empty do { • take an edge (u, v) that is shortest in E and delete it from E • if u and v are in different components then add (u, v) to A } Note: each time a shortest edge in E is considered.

Kruskal's algorithm MST_KRUSKAL(G,w) 1 A:={} 2 for each vertex v in V[G] 3 do MAKE_SET(v) 4 sort the edges of E by nondecreasing weight w 5 for each edge (u,v) in E, in order by nondecreasingweight 6 doif FIND_SET(u) != FIND_SET(v) 7 then A:=A∪{(u,v)} 8 UNION(u,v) • returnA

Disjoint-Set • Keep a collection of sets S1, S2, .., Sk, • Each Si is a set, e,g, S1={v1, v2, v8}. • Three operations • Make-Set(x)-creates a new set whose only member is x. • Union(x, y) –unites the sets that contain x and y, say, Sx and Sy, into a new set that is the union of the two sets. • Find-Set(x)-returns a pointer to the representative of the set containing x. • Each operation takes O(log n) time.

Our implementation uses a disjoint-set data structure to maintain several disjoint sets of elements. • Each set contains the vertices in a tree of the current forest. • The operation FIND_SET(u) returns a representative element from the set that contains u. • Thus, we can determine whether two vertices u and v belong to the same tree by testing whether FIND_SET(u) equals FIND_SET(v). • The combining of trees is accomplished by the UNION procedure. • Running time O(|E| lg (|E|)).

i a b h c d e f g 8 7 9 4 2 11 14 4 7 6 10 8 1 2 The execution of Kruskal's algorithm (Moderate part) • The edges are considered by the algorithm in sorted order by weight. • The edge under consideration at each step is shown with a red weight number.

i i b h c d e f h g g f e d c a a b 8 8 7 7 9 9 4 4 2 2 11 11 14 14 4 4 7 7 6 6 10 10 8 8 1 1 2 2

Prim’s Algorithm

Complete Graph 4 B C 4 2 1 A 4 E F 1 2 D 3 10 5 G 5 6 3 4 I H 3 2 J

Old Graph New Graph 4 B C 4 4 B C 2 1 4 2 1 A 4 E F 1 A 4 E F 1 2 D 3 2 10 D 3 5 G 10 5 G 5 6 3 5 6 3 4 I 4 H I H 3 2 J 3 2 J

Week 3: Minimum Spanning Trees