RAIK 283 Data Structures & Algorithms

RAIK 283Data Structures & Algorithms Greedy Algorithms and MST Dr. Ying Lu ylu@cse.unl.edu

RAIK 283Data Structures & Algorithms • Giving credit where credit is due: • Most of slides for this lecture are based on slides created by Dr. David Luebke, University of Virginia. • Some examples and slides are based on lecture notes created by Dr. Chuck Cusack, Hope College, Dr. Jim Cohoon, University of Virginia, and Dr. Ben Choi, Louisiana Technical University. • I have modified them and added new slides

Greedy Algorithms • Main Concept: Make the best or greedy choice at any given step. • Choices are made in sequence such that • Each individual choice is best according to some limited “short-term” criterion, that is not too expensive to evaluate • Once a choice is made, it cannot be undone! • Even if it becomes evident later that it was a poor choice • Sometimes life is like that  • The goal is to • take a sequence of locally optimal actions, hoping to yield a globally optimal solution.

When to be Greedy • Greedy algorithms apply to problems with • Optimal substructures: optimal solutions contain optimal sub-solutions. • The greedy choice property: an optimal solution can be obtained by making the greedy choice at each step. • Many optimal solutions, but we only need one such solution.

Minimum Spanning Tree (MST) • A spanning tree for a connected, undirected graph, G=(V, E) is • a connected subgraph of G that forms an • undirected tree incident with each vertex. • In a weighted graph G=(V, E, W), • the weight of a subgraph is the sum of the weights of the edges in the subgraph. • A minimum spanning tree (MST) for a weighted graph is • a spanning tree with the minimum weight.

Minimum Spanning Tree • Problem: given a connected, undirected, weighted graph: find a spanning tree using edges that minimize the total weight 6 4 5 9 14 2 10 15 3 8

Minimum Spanning Tree • Which edges form the minimum spanning tree (MST) of the graph? A 6 4 5 9 H B C 14 2 10 15 G E D 3 8 F

Minimum Spanning Tree • Answer: A 6 4 5 9 H B C 14 2 10 15 G E D 3 8 F

Another Example • Given a weighted graph G=(V, E,W), find a MST of G 7 3 6 4 5 3 5 2 6 3

Finding a MST • MSTs satisfy the optimal substructure property: an optimal tree is composed of optimal subtrees • Principal greedy methods: algorithms by Prim and Kruskal • Prim • Grow a single tree by repeatedly adding the least cost edge that connects a vertex in the existing tree to a vertex not in the existing tree • Intermediary solution is a subtree • Kruskal • Grow a tree by repeatedly adding the least cost edge that does not introduce a cycle among the edges included so far • Intermediary solution is a spanning forest

Prime’s Algorithm (High-Level Pseudocode) • Prime(G) //Input: A weighted connected graph G = <V, E> //Output: ET --- the set of edges composing MST of G VT = {v0} ET=  for i = 1 to |V| - 1 do find a minimum-weight edge e* = (u*, v*) among all the edges (u, v) such that u is in VT and v is in V-VT VT = VT  {v*} ET = ET  {e*} return ET

Prime’s Algorithm (High-Level Pseudocode) • Prime(G) //Input: A weighted connected graph G = <V, E> //Output: ET --- the set of edges composing MST of G VT = {v0} ET=  for i = 1 to |V| - 1 do find a minimum-weight edge e* = (u*, v*) among all the edges (u, v) such that u is in VT and v is in V-VT VT = VT  {v*} ET = ET  {e*} return ET A 6 4 9 5 H B C 14 2 10 15 G E D 3 8 F

Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); • Grow a single tree by repeatedly adding the least cost edge that connects a vertex in the existing tree to a vertex not in the existing tree Intermediary solution is a subtree

Prim’s Algorithm 6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 14 2 10 15 3 8 Run on example graph

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5    14 2 10 15    3 8  Run on example graph

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5    14 2 10 15 r 0   3 8  Pick a start vertex r

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5    14 2 10 15 u 0   3 8  Red vertices have been removed from Q

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5    14 2 10 15 u 0   3 8 3 Red arrows indicate parent pointers

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 14   14 2 10 15 u 0   3 8 3

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 14   14 2 10 15 0   3 8 3 u

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 14   14 2 10 15 0 8  3 8 3 u

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 10   14 2 10 15 0 8  3 8 3 u

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 10 2  14 2 10 15 0 8  3 8 3 u

Prim’s Algorithm  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 10 2  14 2 10 15 0 8 15 3 8 3 u

Prim’s Algorithm u  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 10 2  14 2 10 15 0 8 15 3 8 3

Prim’s Algorithm u  6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 10 2 9 14 2 10 15 0 8 15 3 8 3

Prim’s Algorithm u 4 6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 10 2 9 14 2 10 15 0 8 15 3 8 3

Prim’s Algorithm u 4 6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 5 2 9 14 2 10 15 0 8 15 3 8 3

Prim’s Algorithm 4 6 4 MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); 9 5 5 2 9 u 14 2 10 15 0 8 15 3 8 3

Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); What is the hidden cost in this code?

Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; DecreaseKey(v, w(u,v)); delete the smallest element from the min-heap decrease an element’s value in the min-heap (outline an efficient algorithm for it)

Review: Prim’s Algorithm MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; DecreaseKey(v, w(u,v)); How often is ExtractMin() called? How often is DecreaseKey() called?

Review: Prim’s Algorithm What will be the running time? MST-Prim(G, w, r) Q = V[G]; for each u Q key[u] = ; key[r] = 0; p[r] = NULL; while (Q not empty) u = ExtractMin(Q); for each v Adj[u] if (v Q and w(u,v) < key[v]) p[v] = u; key[v] = w(u,v); There are n=|V| ExtractMin calls and m=|E| DecreaseKey calls. The priority Q implementation has a large impact on performance. E.g., O((n+m)lg n) = O(m lg n) using min-heap for Q (for connected graph, n – 1  m)

Finding a MST • Principal greedy methods: algorithms by Prim and Kruskal • Prim • Grow a single tree by repeatedly adding the least cost edge that connects a vertex in the existing tree to a vertex not in the existing tree • Intermediary solution is a subtree • Kruskal • Grow a tree by repeatedly adding the least cost edge that does not introduce a cycle among the edges included so far • Intermediary solution is a spanning forest

MST Applications? A 6 4 5 9 H B C 14 2 10 15 G E D 3 8 F

MST Applications • Network design • telephone, electrical power, hydraulic, TV cable, computer, road • MST gives a minimum-cost network connecting all sites • MST: the most economical construction of a network

Kruskal’s Algorithm (High-Level Pseudocode) • Kruskal(G) //Input: A weighted connected graph G = <V, E> //Output: ET --- the set of edges composing MST of G Sort E in nondecreasing order of the edge weight ET= ; encounter = 0 //initialize the set of tree edges and its size k = 0 //initialize the number of processed edges while encounter < |V| - 1 k = k+1 if ET  {ek} is acyclic ET = ET  {ek}; encounter = encounter + 1 return ET

Kruskal’s Algorithm (High-Level Pseudocode) • Kruskal(G) //Input: A weighted connected graph G = <V, E> //Output: ET --- the set of edges composing MST of G Sort E in nondecreasing order of the edge weight ET= ; encounter = 0 //initialize the set of tree edges and its size k = 0 //initialize the number of processed edges while encounter < |V| - 1 k = k+1 if ET  {ek} is acyclic ET = ET  {ek}; encounter = encounter + 1 return ET A 6 4 9 5 H B C 14 2 10 15 G E D 3 8 F

Kruskal’s Algorithm Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } • Grow a tree by repeatedly adding the least cost edge that does not introduce a cycle among the edges included so far Intermediary solution is a spanning forest

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1?

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2 19 9 14 17 8 25 5 21 13 1

Kruskal’s Algorithm Run the algorithm: Kruskal() { T = ; for each v  V MakeSet(v); sort E by increasing edge weight w for each (u,v)  E (in sorted order) if FindSet(u)  FindSet(v) T = T U {{u,v}}; Union(FindSet(u), FindSet(v)); } 2? 19 9 14 17 8 25 5 21 13 1

RAIK 283 Data Structures & Algorithms