# Minimum Spanning Tree Algorithms - PowerPoint PPT Presentation

1 / 48

Minimum Spanning Tree Algorithms. Prim’s Algorithm Kruskal’s Algorithm Runtime Analysis Correctness Proofs. What is A Spanning Tree?. A spanning tree for an undirected graph G=(V,E) is a subgraph of G that is a tree and contains all the vertices of G

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.

Minimum Spanning Tree Algorithms

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -

## Minimum Spanning Tree Algorithms

Prim’s Algorithm

Kruskal’s Algorithm

Runtime Analysis

Correctness Proofs

### What is A Spanning Tree?

• A spanning tree for an undirected graph G=(V,E) is a subgraph of G that is a tree and contains all the vertices of G

• Can a graph have more than one spanning tree?

• Can an unconnected graph have a spanning tree?

a

b

u

e

c

v

f

d

(u,v)  T w(u,v ) is minimized

Mst T: w( T )=

### Minimal Spanning Tree.

• The weight of a subgraph is the sum of the weights of it edges.

• A minimum spanning tree for a weighted graph is a spanning tree with minimum weight.

• Can a graph have more then one minimum spanning tree?

a

4

4

9

3

b

u

e

14

2

10

15

c

v

f

3

8

d

### Example of a Problem that Translates into a MST

The Problem

• Several pins of an electronic circuit must be connected using the least amount of wire.

Modeling the Problem

• The graph is a complete, undirected graph G = ( V, E ,W ), where V is the set of pins, E is the set of all possible interconnections between the pairs of pins and w(e) is the length of the wire needed to connect the pair of vertices.

• Find a minimum spanning tree.

### Greedy Choice

We will show two ways to build a minimum spanning tree.

• A MST can be grown from the current spanning tree by adding the nearest vertex and the edge connecting the nearest vertex to the MST. (Prim's algorithm)

• A MST can be grown from a forest of spanning trees by adding the smallest edge connecting two spanning trees. (Kruskal's algorithm)

### Notation

• Tree-vertices: in the tree constructed so far

• Non-tree vertices: rest of vertices

Prim’s Selection rule

• Select the minimum weight edge between a tree-node and a non-tree node and add to the tree

a

4

6

5

b

u

14

2

10

c

v

3

8

15

d

f

### The Prim algorithm Main Idea

Select a vertex to be a tree-node

while (there are non-tree vertices) {

if there is no edge connecting a tree node with a non-tree node

return “no spanning tree”

select an edge of minimum weight between a tree node and a non-tree node

add the selected edge and its new vertex to the tree}

return tree

### Implementation Issues

• How is the graph implemented?

• Assume that we just added node u to the tree.

• The distance of the nodes adjacent to u to the tree may now be decreased.

• So using adjacency lists seems better

• How should the set of non-tree vertices be represented?

• The operations are:

• build set

• delete node closest to tree

• decrease the distance of a non-tree node from the tree

• check whether a node is a non- tree node

### Implementation Issues

• How should the set of non-tree vertices be represented?

• A priority queue PQ may be used with the priority D[v] equal to the minimum distance of each non-tree vertex v to the tree.

• Each item in PQ contains: D[v], the vertex v, and the shortest distance edge (v, u) where u is a tree node

• This means:

• build a PQ of non-tree nodes with initial values -

• fast build heap O (V )

• building an unsorted list O(V)

• building a sorted list O(V) (special case)

### Implementation Issues

• delete node closest to tree (extractMin)

• O(lg V ) if heap and

• O(V) if unsorted list

• O(1) sorted list

• decrease the distance of a non-tree node to the tree

• We need to find the location i of node v in the priority queue and then execute (decreasePriorityValue(i, p)) where p is the new priority

• decreasePriorityValue(i, p)

• O(lg V) for heap,

• O(1) for unsorted list

• O(V ) for sorted list (too slow)

### Implementation Issues

• What is the location i of node v in a priority queue?

• Find in Heaps, and sorted lists O(n)

• Unsorted – if the nodes are numbered 1 to n and we use an array where node v is the v item in the array O(1)

Extended heap

• We will use extended heaps that contain a “handle” to the location of each node in the heap.

• When a node is not in PQ the “handle” will indicate that this is the case

• This means that we can access a node in the extended heap in O(1), and check v  PQ in O(1)

• Note that the “handle” must be updated whenever a heap operation is applied

### Implementation Issues

2. Unsorted list

• Array implementation where node v can be accesses as PQ[v] in O(1), and the value of PQ[v] indicates when the node is not in PQ.

Lines 1-5 initialize the priority queue PQ to contain all Vertices. Ds for all vertices except r, are set to infinity.

r is the starting vertex of the TThe T so far is empty

Add closest vertex and edge to current T

Get all adjacent vertices v of u,update D of each non-tree vertex adjacent to u

Store the current minimum weight edge, and updated distance in the priority queue

1. for each uV2.doD [u ]  3. D[ r ] 0

4. PQ make-heap(D,V, {})//No edges5. T 6.7. whilePQ  do8.(u,e )  PQ.extractMin() 9. add (u,e) to T10. for each v  Adjacent (u )

// execute relaxation11. do if v PQ && w( u, v ) < D [ v ]12. then D [ v ]  w (u, v) 13. PQ.decreasePriorityValue( D[v], v, (u,v )) 14. return T // T is a mst.

### Prim’s AlgorithmInitialization

Prim (G )

1. for each uV2. doD [u ]  3. D[ r ] 0

4. PQ make-heap(D,V, {})//No edges5. T 

### Building the MST

// solution check

7. whilePQ  do//Selection and feasibility 8.(u,e )  PQ.extractMin() // T contains the solution so far . 9. add (u,e) to T10. for each v  Adjacent (u )11. do if v  PQ && w( u, v ) < D [ v ]12. then D [ v ]  w (u, v) 13. PQ.decreasePriorityValue

(D[v], v, (u,v) ) 14. return T

1. for each uV2.doD [u ]  3. D[ r ] 0

4. PQ make-PQ(D,V, {})//No edges5. T 6.

7. whilePQ  do8.(u,e )  PQ.extractMin() 9. add (u,e) to T10. for each v  Adjacent (u )11. do if v PQ && w( u, v ) < D [ v ]12. then D [ v ]  w (u, v) 13. PQ.decreasePriorityValue(D[v], v, (u,v)) 15. return T // T is a mst.

Assume a node in PQ can be accessed in O(1)

** Decrease key for v requires O(lgV ) provided the node in heap with v’s data can be accessed in O(1)

### Time Analysis

Using Extended Heap implementation

Lines 1 -6 run in O (V )

Max Size of PQ is | V |

Count7 =O (V )

Count7(8) = O (V )  O( lg V )

Count7(10) = O(deg(u ) ) =O( E )

Count7(10(11)) = O(1)O( E )

Count7(10(11(12))) = O(1) O( E )

Count7(10(13)) = O( lg V)  O( E ) Decrease- Key operation on the extended heap can be implemented in O( lg V)

So total time for Prim's Algorithm is O ( V lg V + E lg V )

What is O(E ) ?

Sparse Graph, E =O(V) , O (E lg V)=O(V lg V )

Dense Graph, E=O(V2), O (E lg V)=O(V2 lg V)

### Time Analysis

1. for each uV2.doD [u ]  3. D[ r ] 0

4. PQ make-PQ(D,V, {})//No edges5. T 6.7. whilePQ  do8.(u,e )  PQ.extractMin() 9. add (u,e) to T10. for each v  Adjacent (u )11. do if v PQ && w( u, v ) < D [ v ]12. then D [ v ]  w (u, v) 13. PQ.decreasePriorityValue (D[v], v, (u,v)) 15. return T // T is a mst.

Using unsorted PQ

Lines 1 - 6 run in O (V )

Max Size of PQ is | V |

Count7 = O (V )

Count7(8) = O (V )  O(V )

Count7(10) = O(deg(u ) ) =O( E )

Count7(10(11)) = O(1)O( E )

Count7(10(11(12))) = O(1) O( E )

Count7(10(13)) =O( 1)  O( E )

So total time for Prim's Algorithm is O (V + V2 +E) = O (V2 )

For Sparse/Dense graph : O( V2 )

Note growth rate unchanged for adjacency matrix graph representation

handle

A

B

C

1

2

3

### Prim - extended HeapAfter Initialization

PQ

T

A

1

0, (A, {})

2

6

B

5

C

G

, (B, {})

, (C, {})

2

3

Prim (G, r)1. for each uV2.do D [u ]  3. D[ r ] 0

4. PQ make-heap(D,V, { })5. T 

G

A

B

C

C 6

B 2

A 2

C 5

A 6

B 5

handle

A

B

C

Null

2

1

### Prim - extended Heap Build tree - after PQ.extractMin

PQ

A

T

(A, {})

2

1

, (C, {})

6

B

5

C

G

, (B, {})

7. whilePQ  do 8.(u,e)  PQ.extractMin()9. add (u,e) to T

2

handle

A

B

C

Null

1

2

### Update B adjacent to A

PQ

T

(A, {})

A

1

2, (B, {A, B})

2

6

B

5

C

G

, (C, {})

2

10. for each v  Adjacent (u )11.// relaxation operation

// relaxation

11. do if v PQ && w( u, v ) < D [ v ]12. then D [ v ]  w (u, v) 13. PQ.decreasePriorityValue

( D[v], v, (u,v))

handle

A

B

C

Null

1

2

PQ

T

(A, {})

1

2, (B, {A, B})

6, (C, {A, C})

A

2

2

6

B

5

C

G

handle

A

B

C

Null

Null

1

### Build tree - after PQ.extractMin

PQ

T

(A, {})

(B, {A, B})

6, (C, {A, C})

1

A

7. whilePQ  do 8.(u,e)  PQ.extractMin()9. add (u,e) to T

2

6

B

5

C

G

handle

A

B

C

Null

Null

1

### Update C adjacent to B

PQ

T

(A, {})

T

(A, {})

(B, {A, B})

5, (C, {B, C})

1

A

10. for each v  Adjacent (u )11.// relaxation operation

2

6

B

5

C

G

handle

A

B

C

Null

Null

Null

### Build tree - after PQ.extractMin

PQ

T

(A, {})

T

(A, {})

(B, {A, B})

(C, {B, C})

A

7. whilePQ  do 8.(u,e)  PQ.extractMin()9. add (u,e) to T

2

6

B

5

C

G

### Prim - unsorted listAfter Initialization

PQ

T

A

B

A

C

12

0, (A, {})

, (B, {})

, (C, {})

4

B

5

C

G

G

A

B

C

C 4

B 12

Prim (G, r)1. for each uV2.do D [u ]  3. D[ r ] 0

4. PQ make-PQ(D,V, { })5. T 

A 12

C 5

A 4

B 5

### Build tree - after PQ.extractMin

PQ

T

(A, {})

A

B

A

C

12

Null

, (B, {})

, (C, {})

4

B

5

C

G

7. whilePQ  do 8.(u,e)  PQ.extractMin()9. add (u,e) to T

### Update B, C adjacent to A

PQ

T

(A, {})

A

A

B

C

12

Null

12, (B, {A, B})

4, (C, {A, C})

4

B

5

C

G

10. for each v  Adjacent (u )11.// relaxation operation

### Build tree - after PQ.extractMin

T

(A, {})

(C, {A, C})

A

12

PQ

4

B

5

A

B

C

C

Null

Null

12, (B, {A, B})

G

7. whilePQ  do 8.(u,e)  PQ.extractMin()9. add (u,e) to T

### Update B adjacent to C

PQ

T

(A, {})

T

(A, {})

(C, {A, C})

A

12

4

B

5

A

B

C

C

Null

Null

5, (B, {C, B})

G

10. for each v  Adjacent (u )11.// relaxation operation

### Build tree - after PQ.extractMin

PQ

T

(A, {})

T

(A, {})

(C, {A, C})

(B, {C, B})

A

12

4

B

5

A

B

C

C

Null

Null

Null

G

7. whilePQ  do 8.(u,e)  PQ.extractMin()9. add (u,e) to T

D = [ 0, , …,  ]

Prim (G)1. for each uV2.do D [u ]  3. D[ r ] 0

4. PQ make-heap(D,V, { })5. T 

PQ = {( 0,(a,)), (,(b,?)), ...(,(h,?))}

T = { }

r

G =

a

4

6

9

5

b

e

g

14

2

10

15

c

f

h

3

8

d

// relaxation

11. do if v PQ && w( u, v ) < D [ v ]12. then D [ v ]  w (u, v) 13. PQ.decreasePriorityValue

( D[v], v, (u,v))

7. whilePQ  do 8.(u,e)  PQ.extractMin()9. add (u,e) to T10. for each v  Adjacent (u )11.// relaxation operation15. return T

D = [ 0,

r

G =

T = {

a

4

6

9

5

b

e

g

PQ = {

14

2

10

15

c

f

h

3

8

d

### Lemma 1

Let G = ( V, E) be a connected, weighted undirected graph. Let T be a promising subset of E. Let Y be the set of vertices connected by the edges in T.

If e is a minimum weight edge that connects a vertex in Y to a vertex in V - Y, then T { e } is promising.

Note: A feasible set is promising if it can be extended to produce not only a solution , but an optimal solution.

In this algorithm: A feasible set of edges is promising if it is a subset of a Minimum Spanning Tree for the connected graph.

### Outline of Proof of Correctness of Lemma 1

T is the promising subset, and e the minimum cost edge of Lemma 1

Let T ' be the MST such that T T '

We will show that if e T' then there must be another MST T" such that T {e}  T".

Proof has 4 stages:

1. Adding e to T', closes a cycle in T' {e}.

2. Cycle contains another edge e' T' but e' T

3. T"=T'  {e} - {e’ }is a spanning tree

4. T" is a MST

Lemma 1

T

 Y

X

X

 V -Y

e

X

X

X

X

X

X

X

X

X

X

X

X

X

### The Promising Set of Edges Selected by Prim

MST T' but e  T'

Lemma 1

 Y

 V -Y

X

X

X

X

X

X

X

X

X

X

X

X

X

X

X

### Since T' is a spanning tree, it is connected. Adding the edge, e, creates a cycle.

T'  {e}

Stage 1

u

v

e

In T' there is a path from uY to v  V -Y. Therefore the path must include another edge e' with one vertex in Y and the other in V-Y.

Stage 2

e'

Lemma 1

 Y

 V -Y

X

T "

e

X

X

X

X

X

X

X

• If we remove e’ from T’ { e } the cycle disappears.

• T”=T’ {e } - {e’} is connected. T’ is connected. Every pair of vertices connected by a path that does not include e’ is still connected in T”. Every pair of vertices connected by a path that included e’ , is still connected in T” because there is a path in T" =T’  { e } - { e’ } connecting the vertices of e’.

Stage 3

w( e )  w( e' )By the way Prim picks the next edge

Lemma 1

 Y

 V -Y

X

T"

e

X

X

X

X

X

X

X

• w( e )  w( e' ) by the way Prim picks the next edge.

• The weight of T", w(T") = w (T' ) + w( e ) - w( e' )  w(T').

• But w (T' )  w(T")because T' is a MST.

• So w (T' ) = w(T")and T" is a MST

Stage 4

Conclusion T { e } is promising

### Theorem : Prim's Algorithm always produces a minimum spanning tree.

Proof by induction on the set T of promising edges.Base case: Initially, T = is promising.Induction hypothesis: The current set of edges T selected by Prim is promising.Induction step: After Prim adds the edge e,T {e } is promising.Proof: T {e } is promising by Lemma 1.

Conclusion: When G is connected, Prim terminates with |T | = |V | -1, and T is a MST.

Kruskal’s Algorithm

a

4

6

5

b

u

14

2

10

c

v

3

8

15

d

f

### Kruskal's Algorithm:Main Idea

solution = { } while ( more edges in E) do// Selection select minimum weight edgeremove edge from E// Feasibility if (edge closes a cycle with solution so far)then reject edgeelse add edge to solution// Solution check if |solution| = |V | - 1 return solution

return null // when does this happen?

a

4

6

9

5

b

e

g

14

2

10

15

c

f

h

3

8

d

### Kruskal's Algorithm:

1. Sort the edges E in non-decreasing weight2. T  3. For each v V create a set.4. repeat 5.Select next {u,v} E, in order 6.ucomp  find (u) 7. vcomp  find (v) 8. ifucomp  vcomp then8. add edge (u,v) to T9. union ( ucomp,vcomp )10.untilT contains |V | - 1 edges11. returntree T

C= { {a}, {b}, {c}, {d}, {e}, {f}, {g}, {h} }C is a forest of trees.

### Kruskal - Disjoint setAfter Initialization

A

Sorted edges

T

2

A B 2

B

6

B C 5

5

A C 6

7

C

G

B D 7

D

1. Sort the edges E in non-decreasing weight2. T 

3. For each v V create a set.

D

A

B

C

Disjoint data set for G

### Kruskal - add minimum weight edge if feasible

A

T

Sorted edges

2

A B 2

(A, B)

6

B

B C 5

5

7

A C 6

C

G

B D 7

D

5. for each {u,v} in ordered E6. ucomp  find (u) 7. vcomp  find (v) 8. ifucomp  vcomp then9. add edge (v,u) to T10. union( ucomp,vcomp )

Find(A)

Find(B)

A

B

C

D

Disjoint data set for G

A

C

D

B

After merge(A, B)

### Kruskal - add minimum weight edge if feasible

A

T

Sorted edges

2

A B 2

(A, B)

(B, C)

6

B

B C 5

5

7

A C 6

C

G

B D 7

D

5. for each {u,v} in ordered E6. ucomp  find (u) 7. vcomp  find (v) 8. ifucomp  vcomp then9. add edge (v,u) to T10. union ( ucomp,vcomp )

Find(B)

Find(C)

A

C

D

B

After merge(A, C)

A

B

C

D

### Kruskal - add minimum weight edge if feasible

A

T

Sorted edges

2

A B 2

(A, B)

(B, C)

6

B

B C 5

5

7

A C 6

C

G

B D 7

D

5. for each {u,v} in ordered E6. ucomp  find (u) 7. vcomp  find (v) 8. ifucomp  vcomp then9. add edge (v,u) to T10. union ( ucomp,vcomp )

Find(A)

Find(C)

A

D

B

C

A and C in same set

### Kruskal - add minimum weight edge if feasible

A

T

Sorted edges

2

A B 2

(A, B)

(B, C)

(B, D)

6

B

B C 5

5

7

A C 6

C

G

B D 7

D

5. for each {u,v} in ordered E6. ucomp  find (u) 7. vcomp  find (v) 8. ifucomp  vcomp then9. add edge (v,u) to T10. union ( ucomp,vcomp )

Find(B)

Find(D)

A

D

B

C

A

After merge

B

C

D

### Kruskal's Algorithm: Time Analysis

Count1 = ( E lg E )

Count2= (1)

Count3= ( V )

Count4 = O( E )

Using Disjoint set-height andpath compression

Count4(6+7+10)=

O((E +V) (V))

Sorting dominates the runtime. We get T( E,V ) = ( E lg E), so for a sparse graph we get ( V lg V)for a dense graph we get( V2lg V2) = ( V2lg V)

Kruskal ( G )1. Sort the edges E in non-decreasing weight2. T  3. For each v V create a set.4. repeat 5.{u,v} E, in order 6.ucomp  find (u) 7. vcomp  find (v) 8. ifucomp  vcomp then9. add edge (v,u) to T10. union ( ucomp,vcomp )11.untilT contains |V | - 1 edges12. returntree T