Introduction to Graphs

1. Campus map Travelling salesperson Circuit layout Project scheduling Oil flow Flight scheduling Introduction to Graphs • In many cases we are faced with a problem that is defined in terms of a number of objects or entities that have some relationships among them. We then try to answer interesting questions. • Graphs are the basic mathematical formulation we use too tackle such problems. • Basic definitions and properties. • Computer representation. • Some applications. Graphs, part I

2. v1 e1 v2 e2 v3 e3 v4 e4 v5 What is a Graph? A Graph G consists of a set V of vertices or nodes and a set E of edges that connect the vertices. We write G=(V,E). G=(V,E) V={v1,v2,v3,v4,v5} E={e1,e2,e3,e4} e1=(v1,v2) e2=(v2,v3) e3=(v2,v4) e4=(v3,v5) Graphs, part I

3. v1 v2 e1 v3 e2 v4 v5 v1 e1 e2 e6 v2 e3 v3 e4 e5 v4 Graphs --- Examples G=(V,E) V={v1,v2,v3,v4,v5} E={e1,e2} e1=(v2,v3) e2=(v2,v4) G=(V,E) V={v1,v2,v3,v4} E={e1,e2,e3,e4,e5,e6} e1=(v1,v2) e2=(v1,v3) e3=(v2,v3) e4=(v2,v4) e5=(v3,v4) e6=(v1,v4) Graphs, part I

4. v1 e1 e2 v2 e3 v3 e4 v4 Directed Graphs In some cases we want the edges to have directions associated with them; we call such a graph a directed graph or a digraph. G=(V,E) V={v1,v2,v3,v4} E={e1,e2,e3,e4} e1=(v2,v1) e2=(v1,v3) e3=(v1,v4) e4=(v4,v3) e5=(v3,v4) e5 ordered pair (predecessor, successor) Graphs, part I

5. v1 40 G=(V,E) V={v1,v2,v3,v4} E={e1,e2,e3,e4,e5} ………………….. ………………….. 50 55 v2 75 v3 15 v4 63 Weighted Graphs In some cases, we want to associate a weight with each edge in the graph. Such a graph is known as a weighted graph. Graphs with no weights are called unweighted graphs (or simply graphs). Directed graphs can also be weighted (directed weighted graphs). Graphs, part I

6. More Graph Terminology • A vertex vj is said to be adjacent to a different vertex vi if an edge connects vi to vj, i.e., if there exists and edge e Î E such that e=(vi,vj). • A path is a sequence of vertices in which each vertex is adjacent to the next one. That is, a path p = v1, v2, … , vn(n > 1) such that each vertex vi+1 is adjacent to vi, 1 £ i < n. • The length of a path is the number of edges in it. • A cycle is a path of length greater than one that begins and ends at the same vertex. In other words, a cycle is a path p = v1, v2, … , vn, such that v1 = vn. • A graph with no cycles is called an acyclic graph. A directed acyclic graph is called a DAG. Graphs, part I

7. More Graph Terminology (Cont’d) • A simple cycle is a cycle formed from three or more distinct vertices in which no vertex is visited more than once along the simple cycle’s path (except for the starting and ending vertex).That is, if p = v1, v2, … , vn (n > 3) is a path, then p is a simple cycle if v1 = vn , and vi¹ vj for different i and j in the range 1 £ i,j < n. • Two different vertices are connected if there is a path between them. • A subset of vertices S is said to be a connected componentof G if there is a path from each vertex vi to any other distinct vertex vj of S. If S is the largest such subset, then it is called a maximal connected component. • The degree of a vertex is the number of edges connected to it. Graphs, part I

8. 5 1 4 3 15 2 6 7 12 14 8 11 13 9 10 Graph Terminology --- Example Graphs, part I

9. Representation of Graphs • The adjacency matrix for a graph G=(V,E) with n (or |V|) vertices numbered 0, 1, …, n-1 is an n x n array M such that M[i][j] is 1 if and only if there is an edge from vertex i to vertex j. • The adjacency list for a graph G=(V,E) with n vertices numbered 0, 1, …, n-1 consists of n linked lists. The ith linked list has a node for vertex j if and only if the graph contains and edge from vertex i to vertex j. • Which one is better? Graphs, part I

10. 8 0 1 2 3 4 5 6 7 8 0 0 0 1 0 0 1 0 0 0 1 0 0 0 0 0 0 1 0 0 7 2 0 0 0 0 0 0 1 0 0 3 0 0 0 0 1 0 0 0 0 5 3 4 0 0 0 0 0 1 0 0 0 5 0 0 0 1 0 0 0 1 0 2 0 4 6 0 0 0 0 0 0 0 0 0 7 0 0 1 0 0 0 0 0 1 8 0 0 0 0 0 0 0 0 0 6 1 Adjacency Matrix --- Example 1 Graphs, part I

11. 0 8 0 1 2 3 0 ¥ 8 ¥ 6 1 6 1 8 ¥ 9 ¥ 9 2 ¥ 9 ¥ ¥ 3 3 6 ¥ ¥ ¥ 2 Adjacency Matrix --- Example 2 The matrix is symmetric for undirected graphs. Graphs, part I

12. 8 0 2 5 1 6 2 6 7 3 4 4 5 5 3 5 3 7 6 7 2 8 2 0 4 8 6 1 Adjacency List --- Example 1 Graphs, part I

13. 0 8 0 1 8 3 6 1 6 1 0 8 2 9 2 1 9 9 3 3 0 6 2 Adjacency List --- Example 2 Graphs, part I

14. Matrix List M[i][j] O(1) Search List O(d) Operation 1 Traverse row O(n) Traverse List O(d) Operation 2 Which is Better? • Operation 1: Is there an edge from vertex i to vertex j? • Operation 2: Find all vertices adjacent to vertex i. • Time: • Determine which operation is most frequent. Graphs, part I

15. Matrix: n2 x size of integer; i.e., O(n2). n x size of pointer + O(|E|) x (size of integer + size of pointer) O(n+|E|) = O(|V| + |E|) List: Which is Better? • Space: How big is this? • Consider space given graph properties. Graphs, part I

16. 1 2 3 4 5 6 7 8 9 1 1 1 7 9 2 1 1 1 2 3 1 1 1 1 3 4 1 1 8 5 1 1 1 1 6 1 1 4 7 1 1 8 1 1 5 6 9 1 1 Graph Traversals Traversal: visit each vertex in the graph once and only once. Graphs, part I

17. Graph Traversals --- Basic Algorithm while ($ un-VISITED vertex) { Initialize a DS with un-VISITED vertex vI ;mark as VISITED while (DS is not empty) { remove a vertex vj from DS visit vj add un-VISITED vertices adjacent to vj to DS; mark each as VISITED } } • DS is a queue --- breadth-first traversal (search) • DS is a stack --- depth-first traversal (search) Graphs, part I

18. 1 1 2 3 6 3 6 2 4 5 2 4 5 4 5 4 5 6 5 Breadth-First Traversal • Spanning tree. Graphs, part I

19. 1 2 3 4 6 5 2 2 4 1 3 3 3 3 3 5 6 Depth-First Traversal • Spanning tree. Graphs, part I

20. 1 2 3 4 5 6 Recursive Depth-First Traversal DFS(vi) visit vj; mark vi as VISITED for each un-VISITED vertex vj adjacent to vi { DFS (vj) } } Graphs, part I

21. A H I F B C G J K D E M L A H I non-Recursive DF: A G E D F C B H I J M L K F B C G J K D E M L A H I Recursive DF: A B C F D E G H I J K L M F B C G J K D E M L Graph Traversals --- Example BF: A B C F G D E H I J K L M Graphs, part I

22. Complexity of Graph Traversals • Each vertex must be visited exactly once. • At a vertex, we must determine all other vertices connected to the vertex. • Adjacency matrix: O(|V|2). • Adjacency list: O(|V| + |E|). • Each edge is examined once (directed) or twice (undirected). • Typically, lists are better than matrices. The complexity of the traversal is linear in the number of edges. Graphs, part I

23. Elementary Graph Operations • Graph traversals provide the basis for many elementary graph operations: • Spanning trees on graphs • Graph cycles • Connected components of a graph Graphs, part I

24. Minimum-Cost Spanning Trees • A minimum-cost spanning tree of a connected weighted graph is a collection of edges connecting all vertices such that the sum of the weights of the edges is the smallest possible. 6 6 b b 7 7 a a 9 9 c c 2 2 3 3 e e 4 4 h h 4 4 8 8 d d g g 5 5 f f 2 2 • Prim’s algorithm: always pick the edge with the smallest weight to any node. It is a greedy algorithm.

25. MCST --- Example 6 6 b b 7 7 a a 9 9 c c 2 2 3 3 e e 4 4 h h 4 4 8 8 d d g g 5 5 f f 2 2 6 6 b b 7 7 a a 9 9 c c 2 2 3 3 e e 4 4 h h 4 4 8 8 d d g g 5 5 f f 2 2

26. MCST --- Example (cont’d) 6 6 b b 7 7 a a 9 9 c c 2 2 3 3 e e 4 4 h h 4 4 8 8 d d g g 5 5 f f 2 2 6 6 b b 7 7 a a 9 9 c c 2 2 3 3 e e 4 4 h h 4 4 8 8 d d g g 5 5 f f 2 2

27. Prim’s Algorithm for MCST S = {0} /* Set of edges in current MCST */ for i = 0 to n-1 D[i] = M[0][i] /* Weight of smallest edge from i to S */ V[i] = 0 /* Vertex in S that i connects to */ for i = 1 to n-1 find the smallest D[v] such that v  S S = S  {v} MCST = MCST + ( v, V[v] ) for all vertices u  S if( D[u] > M[u][v] ) then D[u] = M[u][v] V[u] = v Graphs, part I

28. Prim’s Algorithm: Heap Implementation • The following implements Prim’s algorithm with a heap: S = {0} /* Set of edges in current MCST */ ……. loop 1 for i = 0 to n D[i] = M[0][i] /* */ Weight of smallest edge from i to S V[i] = 0 /* */ Vertex in S that i connects to ……. loop 2 for i = 1 to n …Extract_Min() Ï S find the smallest D[v] such that v È S = S {v} MCST = MCST + ( v, V[v] ) ……. loop 3 Ï for all vertices u S if( D[u] > M[u][v] ) then … Decrease_Key() D[u] = M[u][v] V[u] = v Graphs, part I

29. Prim’s Algorithm: Heap Implementation • Consider that you use a heap to hold the keys D[0…n-1]. In • this heap children are larger than the root (heap property): • To find the minimum in loop 2 you need O(log V) time • with procedure Extract_Min(heap) • To update the values of D[0…n-1] in loop 3 you need a • procedure Decrease_Key(heap, node, new_key) • that takes a pointer to a node in the heap and decreases • its key. This can be done in O(logV) time ….. • HOW? Graphs, part I

30. Prim’s Algorithm: Heap Implementation Loop 1: Executed V times. Total time: O(V) Loop 2: Executed V times. To extract the minimum we need O(log V) time at each iteration. Total time: O(VlogV) Loop 3: It is executed a total of 2E times and it takes O(log V) time at each iteration to decrease the key. Therefore, total time is O(E logV) Heap Implementation: O(V+VlogV +ElogV) = O(ElogV) Graphs, part I

31. 7 7 2 3 2 3 5 5 3 8 3 8 1 1 1 4 1 4 10 10 2 2 5 5 6 6 6 5 6 5 7 7 Single Source Shortest Paths Given a weighted connected graph G=(V,E), and given a pair of vertices vs (source) and vd (destination) Î V what is the shortest path from vs to vd? That is, what is the path that has the smallest sum of edge weights? Source vertex vs is vertex 1 Bold line is shortest path from 1(vs) to 5 (vd)

32. B F C E G D Single Source Shortest Paths A B E F H 15 A B E G H 14 A C E F H 16 A C E G H 15 A D E F H 26 A D E G H 25 SP from A to H = SP from A to E + SP from E to H. SP from A to H = SP from A to B + SP from B to H. SP from A to H = SP from A to C + SP from C to H. In general: SP from A to H = SP from A to vi + SP from vi to H; "vi. 1 2 5 7 1 3 A H 5 6 6 8

33. Use Dijkstra’s(greedy) algorithm if graph has only positive weights • Use Bellman-Ford’s algorithm if graph has positive and negative weights (but not negative cycles…) • Both algorithms can run on directed or undirected graphs Single Source Shortest Paths • Let D[vs, v] be the distance from the source vertex vs to some vertex v and let M[u, v] be the weight of edge uv for vertices u and v. At the end of any correct shortest path algorithm you should have: D[vs, v] D[vs, u] + W(v, u) (why?) Graphs, part I

34. B F -- 2 1 6 ? ? ? ? C E H G D B F B D E F G H 2 6 4 ? ? ? E H G D Dijkstra’s Algorithm Cost of SP from A to vi through S 1 2 5 A B C D E F G H 7 1 3 A 5 6 6 8 S = {A} 1 2 5 A C 7 1 3 -- 1 A C 5 6 6 8 S = {A,C}

35. F 6 ? ? ? E H G D F 6 8 ? H G D Dijkstra’s Algorithm (Cont’d) Cost of SP from A to vi through S B 1 2 5 A B C D E F G H 7 1 3 -- 2 1 3 A C 5 6 6 8 S = {A,C,B} B 1 2 5 A B C D E F G H 7 1 3 -- 2 1 3 10 A C E 5 6 6 8 S = {A,C,B,E}

36. F -- 2 1 6 3 H F -- 2 1 6 3 H Dijkstra’s Algorithm (Cont’d) Cost of SP from A to vi through S B 1 2 5 A B C D E F G H 7 1 3 10 8 ? A C E 5 6 6 8 G S = {A,C,B,E,D} D B 1 2 5 A B C D E F G H 7 1 3 10 8 14 A C E 5 6 6 8 G S = {A,C,B,E,D,G} D

37. -- 2 1 6 3 8 H -- 2 1 6 3 8 Dijkstra’s Algorithm (Cont’d) Cost of SP from A to vi through S B 1 2 F 5 A B C D E F G H 7 1 3 10 14 A C E 5 6 6 8 G S = {A,C,B,E,D,G,F} D B 1 2 F 5 A B C D E F G H 7 1 3 10 14 A C E H 5 6 6 8 G S = {A,C,B,E,D,G,F,H} D

38. Dijkstra’s Algorithm for Shortest Paths S = {0} /* Current MST */ for i = 0 to n D[i] = M[0][i] /* Shortest path length from 0 to i */ for i = 1 to n-1 find the smallest D[v] such that v  S S = S  {v} for all vertices u  S if (D[u] > D[v] + M[v][u]) then D[u] = D[v] + M[v][u]

39. 7 2 3 5 3 8 1 1 4 10 2 5 6 6 5 7 Dijkstra’s Algorithm --- Example 1 2 3 4 5 6 -- 3 ? ? ? 5 1 2 3 4 5 6 -- 3 10 ? ? 5 1 2 3 4 5 6 -- 3 10 7 12 5 1 2 3 4 5 6 -- 3 10 7 12 5 1 2 3 4 5 6 -- 3 10 7 11 5 1 2 3 4 5 6 -- 3 10 7 11 5

40. Bellman-Ford’s Algorithm • If the graph also contains negative weights then Dijkstra’s will not work. Use Bellman-Ford’s algorithm • The graph should never contain negatives cycles! (because the idea of a shortest path cannot be defined) • Bellman-Ford’s algorithm returns FALSE in this case • Recall, let D[vs, v] be the distance from the source vertex vs to some vertex v and let M[u, v] be the weight of edge uv for vertices u and v. Graphs, part I

41. Bellman-Ford’s Algorithm --- Algorithm for i = 0 to n D[i] = INFINITE /* Shortest path length from 0 to i */ end for for i = 1 to n-1 for each edge u  v in the graph do if (D[v] > D[u] + M[u][v]) then D[v] = D[u] + M[v][u] for each edge u  v in the graph do if (D[v] > D[u] + M[u][v]) then return FALSE /* negative cycle exists */ Graphs, part I

42. 5 vs 6 INF INF INF vs 6 5 -2 0 6 0 -3 -2 8 7 -3 -4 8 7 7 2 7 INF INF INF -4 7 9 2 9 Initial graph after i=1 Bellman-Ford’s Algorithm --- Example • Edges are directed with positive and negative weights • Each vertex holds D[v] value • Bold edges the ones if-statement executes in 1st loop Graphs, part I

43. Bellman-Ford’s Algorithm --- Example 5 6 4 5 vs 6 -2 vs 2 4 6 -2 0 -3 8 7 0 -3 8 -4 7 7 2 -4 7 2 7 2 9 7 2 9 after i=2 after i=3 5 5 vs 2 4 vs 2 4 6 6 -2 -2 0 -3 0 -3 8 8 7 7 -4 -4 7 7 2 2 7 -2 7 -2 9 9 after i=4 Graphs, part I after i=5… (final)

44. Bellman-Ford’s Algorithm • In the 1st nested loop, the algorithm repeats n (= V) times. Each time it visits every vertex E • It takes O(VE + E) = O(VE) time (VE time to execute the 1st nested loop and E time to execute the 2nd loop ) • If the graph has negative weighted cycles then the idea of a shortest path cannot be defined: • One could possibly traverse the cycle infinitely and keep reducing the length of the shortest path • Bellman-Ford returns FALSE in this case Graphs, part I