CS 225 Data Structures and Software Principles

CS 225 Data Structures and Software Principles Section 11 Graph Algorithms

Discussion Topics • Shortest Path Problems • Applications, problems • Solutions • Dijkstra’s Algorithm • Minimum Spanning Trees • Solutions • Prim’s Algorithm • Kruskal’s Algorithm

Shortest Path Applications • Find best path to travel from one city to another • Map/route generation in traffic • Network routing (CS 438): Open Shortest Path First (OSPF) • intradomain routing protocol • Lets routers learn their forwarding table & accommodate to changes in network topology B 5 3 A 10 C 11 2 D

Shortest - Path Problems • Single-source (single-destination): Find a shortest path from a given source (vertex s) to each of the vertices. • Single-pair: Given two vertices, find a shortest path between them. Solution to single-source problem solves this problem efficiently, too. • All-pairs: Find shortest-paths for every pair of vertices. Dynamic programming algorithm. • Unweighted shortest-paths: BFS

Dijkstra’s Algorithm • Solves shortest path problems in two stages: • Greedily computes and stores shortest path from any given vertex to all others in the graph • Uses the data structure to find a shortest path • Assumptions: • Positive weights on edges (Can modify to work even with negative weights, but this is much slower.) • Graph is connected

d(s, s) = 0  • d(s, s1) = 5  • d(s, s2) = 6  • d(s, s3) = 8  • d(s, s4) = 15 s4 8 15 s3 2 10 s s2 3 4 5 1 s1 Intuition • Assume that the shortest distances from the starting node s to the rest of the nodes are • d(s, s)  d(s, s1)  d(s, s2)  …  d(s, sn-1) • Now a shortest path from s to si may include any of the vertices {s1, s2,…, • si-1} but NOT any sj where j > i. • Dijkstra’s algorithm: select nodes compute shortest distances in the order s, s1, s2 ,…, sn-1

Dijkstra’s Greedy Selection Rule • Assume s1, s2… si-1 have been selected, and their shortest distances have been stored in Solution • Select node si and save d(s, si) if si has the shortest distance from s on a path that may include only s1, s2… si-1 as intermediate nodes • Need to maintain for each unselected node v the distance of the shortest path from s to v, D[v]

s4 8 15 s3 2 10 s s2 3 4 5 1 s1 Dijkstra’s Example Solution = { (s, 0) } D[s1]=5 for path [s, s1] D[s2]=  for path [s, s2] D[s3]=10 for path [s, s3] D[s4]=15 for path [s, s4] Solution = { (s, 0), (s1, 5) } D[s2]= 6 for path [s, s1, s2] D[s3]=9 for path [s, s1, s3] D[s4]=15 for path [s, s4] Solution = { (s, 0), (s1, 5), (s2, 6) } D[s3]=8 for path [s, s1, s2, s3] D[s4]=15 for path [s, s4] Solution = { (s, 0), (s1, 5), (s2, 6),(s3, 8), (s4, 15) }

s4 8 15 s3 2 10 s s2 3 4 5 1 s1 Implementing the Selection Rule • Node near is selected and added to Solution if D(near)  D(v) for any v Ï Solution Solution = {(s, 0)} D[s1]=5  D[s2]=  D[s1]=5  D[s3]=10 D[s1]=5 D[s4]=15 Node s1 is selected Solution = { (s, 0), (s1, 5) }

Updating D[] • After adding near to Solution, update D[v] of all nodes v Ï Solution and adjacent to near if there is a shorter special path from s to v that contains node near : if (D[near] + w(near, v) < D[v]) then D[v]=D[near] + w(near, v) D[near] = 5 2 Solution after adding near 2 3 s D[v] = 9 is updated to D[v]=5+2=7 3 6

s4 8 15 s3 2 10 s s2 3 4 5 1 s1 Updating D[] Example Solution = {(s, 0)} D[s1]=5, D[s2]=  , D[s3]=10, D[s4]=15. Solution = { (s, 0), (s1, 5) } D[s2]= D[s1]+w(s1, s2)=5+1=6, D[s3]= D[s1]+w(s1, s3)=5+4=9, D[s4]=15 Solution = { (s, 0), (s1, 5), (s2, 6) } D[s3]=D[s2]+w(s2, s3)=6+2=8, D[s4]=15 Solution = { (s, 0), (s1, 5), (s2, 6), (s3, 8), (s4, 15) }

Dijkstra’s Algorithm Step 0: At the start, all distance fields d of the vertices in the graph are set to  (infinity), and each vertex’s confirmed flag is set to 0 Step 1: Set distance field of source node to 0 Step 2: for (i = 0; i < |V|; i++) { Let x = the unconfirmed vertex (flag == 0) with minimum distance field. Set confirmed flag of x = 1. For all (unconfirmed) neighbors y of x if (dy > dx + Costx,y) dy = dx + Costx,y }

Dijkstra’s Algorithm • For each iteration • Choose the vertex with minimum distance among all unconfirmed vertices • Update the distances to its neighbors (if it’s a shorter path going through this vertex) • How can we reconstruct the shortest paths? • For each destination, keep track of the vertex just prior to a destination vertex along the shortest path to that destination • Two ways to implement • using a heap • using an array

Dijkstra’s Implementationusing Heaps • Store unknown vertices in a heap H using distance fields as keys • Finding the minimum vertex: H.delete_min() operation  O(log V) • Do this once per vertex in the graph • Updating distances to neighbors of min vertex: H.increase_key_value(n, newdist)  O(log V) • Do this once per edge in the graph, since we visit the outgoing edges of every vertex only once • Running time (assuming adj. list): O(V log V + E log V) • Works well for sparse graphs

Dijkstra’s Implementationusing Arrays • Maintain an array for the vertices • Finding the min vertex now involves searching the array  O(V) • Updating the distance  O(1) • Running time: O(V2 + E)  O(V2) • Works well for dense graphs

Dijkstra’s AlgorithmRunning Times Summary Graph Implementation Adj. Matrix Adj. List Dijkstra’s Implementation Heap Array How long does it take to find a vertex’s neighbors in our matrix?

Discussion Topics • Shortest Path Problems • Applications, problems • Solutions • Dijkstra’s Algorithm • Minimum Spanning Trees • Solutions • Prim’s Algorithm • Kruskal’s Algorithm

Spanning Trees • Here we will consider only undirected weighted graphs • Spanning tree: a tree that spans the entire graph. Contains every vertex of the original graph in itself (i.e. it has |V| vertices) and has |V| - 1 edges. Since it is a tree, it has no cycles. • Application Example: bridging algorithm • constructing a spanning tree over an extended LAN with redundant links, so bridges know which ports to forward data over while avoiding loops (CS 438)

Minimum Spanning Trees • Every Spanning Tree has an associated Cost =  edge weights in the tree • Minimum Spanning Tree (MST): a spanning tree with the minimum cost among all the possible spanning trees for a given graph • Solutions: we consider two greedy algorithms • Prim’s Algorithm • Kruskal’s Algorithm

Prim’s Algorithm • Conceptually maintain two sets of vertices: IN = { some start vertex } OUT = { all other vertices } • MST starts with one vertex and no edges. for (i = 0; i < |V| - 1; i++) { /* Each iteration add one edge and one vertex */ Pick an edge with lowest cost connecting a vertex from IN to a vertex from OUT. Add the edge and endpoint from OUT into tree. Move that vertex from OUT to IN. }

Prim’s Implementation • Use modified Dijkstra’s algorithm • Ignore the total path cost to the vertex. Use only edge cost from x to its neighbor y. For all (unconfirmed) neighbors y of x if (dy > Costx,y) dy = Costx,y • Running time = same as Dijkstra’s  O(V log V + E log V) OR O(V2)

Kruskal’s Algorithm • MST starts with all vertices but no edges. Mark all edges as unconsidered. for (i = 0; i < |V| - 1; i++) { /* Each iteration add one edge */ Pick an unconsidered edge with lowest cost. Mark it considered. if (adding edge doesn’t cause a cycle in the tree) add edge to tree }

Kruskal’s Implementation • Use a heap & disjoint sets • Heap: Insert all edges & remove them one by one. • Disjoint sets: Used to test for cycles. Initialize each vertex to a set. • Running time = make heap + remove edges + (finds + unions) = O(E) + O(E log E) + O(2E+V-1)(log*V)  O(E log E) equiv to:  O(E log V)

Remember • Dijkstra’s algorithm finds the shortest distance from one source vertex to all other vertices • Prim’s and Kruskal’s algorithms find the Minimum Spanning Tree for a given graph

CS 225 Data Structures and Software Principles