CS 3343: Analysis of Algorithms

CS 3343: Analysis of Algorithms Minimum Spanning Tree

Outline • Review of last lecture • Prim’s algorithm and Kruskal’s algorithm for MST in detail

Graph Review • G = (V, E) • V: a set of vertices • E: a set of edges • Degree: number of edges • In-degree vs out-degree • Types of graphs • Directed / undirected • Weighted / un-weighted • Dense / sparse • Connected / disconnected

Graph representations • Adjacency matrix 1 2 4 3 How much storage does the adjacency matrix require? A: O(V2)

Graph representations • Adjacency list 1 2 3 3 2 4 3 3 How much storage does the adjacency list require? A: O(V+E)

A 1 2 3 4 1 0 1 1 0 2 1 0 1 0 3 1 1 0 1 4 0 0 1 0 Graph representations • Undirected graph 1 2 4 3 2 3 1 3 1 2 4 3

A 1 2 3 4 1 0 5 6 0 2 5 0 9 0 3 6 9 0 4 4 0 0 4 0 Graph representations • Weighted graph 1 5 6 2 4 9 4 3 2,5 3,6 1,5 3,9 1,6 2,9 4,4 3,4

Analysis of time complexity • Convention: • Number of vertices: n, |V|, V • Number of edges: m, |E|, E • O(V+E) is the same as O(n+m) or O(|V|+|E|)

Tradeoffs between the two representations |V| = n, |E| = m Both representations are very useful and have different properties, although adjacency lists are probably better for most problems

6 4 5 9 14 2 10 15 3 8 Minimum Spanning Tree • Problem: given a connected, undirected, weighted graph:

Minimum Spanning Tree • Problem: given a connected, undirected, weighted graph, find a spanning tree using edges that minimize the total weight 6 4 5 9 • A spanning tree is a tree that connects all vertices • Number of edges = ? • A spanning tree has no designated root. 14 2 10 15 3 8

T2 T1 T1’ v u Minimum Spanning Tree • MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees • Let T be an MST of G with an edge (u,v) in the middle • Removing (u,v) partitions T into two trees T1 and T2 • w(T) = w(u,v) + w(T1) + w(T2) • Claim 1:T1 is an MST of G1 = (V1, E1), and T2 is an MST of G2 = (V2, E2) • Proof by contradiction: • if T1 is not optimal, we can replace T1 with a better spanning tree, T1’ • T1’, T2 and (u, v) form a new spanning tree T’ • W(T’) < W(T). Contradiction.

y x Minimum Spanning Tree • MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees • Let T be an MST of G with an edge (u,v) in the middle • Removing (u,v) partitions T into two trees T1 and T2 • w(T) = w(u,v) + w(T1) + w(T2) • Claim 2:(u, v) is the lightest edge connecting G1 = (V1, E1) and G2 = (V2, E2) • Proof by contradiction: • if (u, v) is not the lightest edge, we can remove it, and reconnect T1 and T2 with a lighter edge (x, y) • T1, T2 and (x, y) form a new spanning tree T’ • W(T’) < W(T). Contradiction. T2 T1 v u

Algorithms • Generic idea: • Compute MSTs for sub-graphs • Connect two MSTs for sub-graphs with the lightest edge • Two of the most well-known algorithms • Prim’s algorithm • Kruskal’s algorithm • Let’s first talk about the ideas behind the algorithms without worrying about the implementation and analysis

Prim’s algorithm • Basic idea: • Start from an arbitrary single node • A MST for a single node has no edge • Gradually build up a single larger and larger MST 6 5 Not yet discovered 7 Fully explored nodes Discovered but not fully explored nodes

Prim’s algorithm • Basic idea: • Start from an arbitrary single node • A MST for a single node has no edge • Gradually build up a single larger and larger MST 2 6 5 9 Not yet discovered 4 7 Fully explored nodes Discovered but not fully explored nodes

Prim’s algorithm • Basic idea: • Start from an arbitrary single node • A MST for a single node has no edge • Gradually build up a single larger and larger MST 2 6 5 9 4 7

Prim’s algorithm in words • Randomly pick a vertex as the initial tree T • Gradually expand into a MST: • For each vertex that is not in T but directly connected to some nodes in T • Compute its minimum distance to any vertex in T • Select the vertex that is closest to T • Add it to T

Example a 6 12 9 5 b f g 7 14 15 8 c e h 10 3 d

Pseudocode for Prim’s algorithm Given G = (V, E). Output: a MST T. Randomly select a vertex v S = {v}; A = V \ S; T = {}. While (A is not empty) find a vertex u  A that connects to vertex v  S such that w(u, v) ≤ w(x, y), for any x  A and y  S S = S U {u}; A = A \ {u}; T = T U (u, v). End Return T

Time complexity Given G = (V, E). Output: a MST T. Randomly select a vertex v S = {v}; A = V \ S; T = {}. While (A is not empty) find a vertex u  A that connects to vertex v  S such that w(u, v) ≤ w(x, y), for any x  A and y  S S = S U {u}; A = A \ {u}; T = T U (u, v). End Return T n vertices • Time complexity:n * (time spent on purple line per vertex) • Naïve: test all edges => Θ(n * m) • Improve: keep the list of candidates in an array => Θ(n2) • Better: with priority queue => Θ(m log n)

Idea 1: naive find a vertex u  A that connects to vertex v  S such that w(u, v) ≤ w(x, y), for any x  A and y  S min_weight = infinity. For each edge (x, y)  E if x  A, y  S, and w(x, y) < min_weight u = x; v = y; min_weight = w(x, y); • time spent per vertex: Θ(m) • Total time complexity: Θ(n*m)

Idea 2: distance array // For each vertex v, d[v] is the min distance from v to any node already in S // p[v] is the parent node of v in the spanning tree For each v  V d[v] = infinity; p[v] = null; Randomly select a v, d[v] = 1; // d[v]=1 just to ensure proper start S = {}; A = V; T = {}. While (A is not empty) Search d to find the smallest d[u] > 0. S = S U {u}; A = A \ {u}; T = T U (u, p[u]); d[u] = 0. For each v in adj[u] if d[v] > w(u, v) d[v] = w(u, v); p[v] = u; End

a 6 12 9 5 b f g 7 14 15 8 c e h 10 3 d

a 6 12 9 5 b f g Discovered 7 14 15 8 c e h 10 3 Explored d Not discovered

a 6 12 9 5 b f g 7 14 15 8 c e h 10 3 d

Time complexity? // For each vertex v, d[v] is the min distance from v to any node already in S // p[v] is the parent node of v in the spanning tree For each v  V d[v] = infinity; p[v] = null; Randomly select a v, d[v] = 1; // d[v]=1 just to ensure proper start S = {}. T = {}. A = V. While (A is not empty) Search d to find the smallest d[u] > 0. S = S U {u}; A = A \ {u}; T = T U (u, p[u]); d[u] = 0. For each v in adj[u] if d[v] > w(u, v) d[v] = w(u, v); p[v] = u; End n vertices Θ(n) per vertex O(n) per vertex if using adj list (n) per vertex if using adj matrix Overall complexity: Θ (n2)

Idea 3: priority queue (min-heap) • Observation • In idea 2, we need to search the distance array n times, each time we only look for the minimum element. • Distance array size = n • n x n = n2 • Can we do better? • Priority queue (heap) enables fast retrieval of min (or max) elements • log (n) for most operations

CS 3343: Analysis of Algorithms