Algorithm & Application

Algorithm & Application Algorithm : A step-by-step procedure for solving a problem Prof. Hyunchul Shin shin@hanyang.ac.kr Hanyang University

Foundations of Algorithms • Richard Neapolitan and KumarssNaimipour • 3rd Edition. Jones and Bartlett Computer Science, 2004 • Time : CPU cycles • Storage: memory • Instance: Each specific assignment of values to parameters

Problem Is the number x in the list S of n numbers? The answer is yes if x is in S and no if it is not. (ex) S={10,7,11,5,13,8} , n=6 , and x=5 . Solution “yes” Algorithm : search ( S, n, x ) { for ( i=1; i<=n; i++ ) if S[i]==x, return ( “yes” ); return ( “no” ); } /* cf. text P5 */

Exchange Sort Problem : Sort n keys in nondecreasing order Inputs : n, S[1],…,S[n] Outputs : Sorted keys in the array S. Algorithm: Exchange Sort { for( i=1; i<=n; i++ ) for( j=i+1; j<=n; j++ ) if( S[j] < S[i]) exchange S[i] and S[j] }

Algorithm Exchange Sort Algorithm: Exchange Sort (ex) n=4 S=[ 4 3 1 5] { for( i=1; i<=n; i++ ) for( j=i+1; j<=n; j++ ) if( S[j] < S[i]) exchange S[i] and S[j] } Homework Show i , j , S , for exchange sort of S=[ 3 8 5 9 7]. Due 1 week 3 4 1 3 1 5 3 4 3 5 4 5

Matrix Multiplication Cn×n=An×n. Bn×n Cij= aik. Bkj , for i<=n, j <=n. (ex) = Algorithm { /*Matrix multiplication*/ for( i=1; i<=n; i++ ) for( j=1; j<=n; j++ ) { C[i][j]=0; for(k=1;k<=n; k++) C[i][j]= C[i][j] + A[i][k] ×B[k][j]; } }

Fibonacci Sequence f0=0 f1=1 fn= fn-1 + fn-2 for n>=2. (ex) f2 =f1 + f0 =1 + 0=1 f3 =f2 + f1 =1 + 1=2 f4 =f3 + f2 =2 + 1=3 f5 =f4 + f3 =3 + 2=5 …

Fibonacci (Recursive) int fib (int n) { /*divide-and-conquer : chap2 */ if(n<=1) return n; else return( fib(n-1) + fib(n-2) ); } (ex) fib(5) computation

Fibonacci (Iterative) Intfib_iter (int n){/*dynamic programing:chap3*/ Index i; int f[0..n]; f[0]=0; If(n>0){ f[1]=1; for( i=2; i<=n; i++ ) f[i]=f[i-1]+f[i-2]; } Return f[n]; } Complexity (cf text p16) Fib(100) takes 13 days. Fib_iter(100) takes 101 n sec

Complexity: Exchange Sort Algorithm: Exchange Sort { for( i=1; i<=n; i++ ) for( j=i+1; j<=n; j++ ) if( S[j] < S[i]) exchange S[i] and S[j] } Basic operation: Comparison of S[j] with S[i] Input size: n, the number of items to be sorted. Complexity: the number of basic operations T(n)=(n-1)+(n-2)+(n-3)+…+1 =(n-1).n/2 ЄO(n2)

Complexity: Matrix Multiplication Algorithm { /*Matrix multiplication*/ for( i=1; i<=n; i++ ) for( j=1; j<=n; j++ ) { C[i][j]=0; for(k=1;k<=n; k++) C[i][j]= C[i][j] + A[i][k] ×B[k][j]; } } Basic operation: multiplication (innermost for loop) Input size: n, #rows and #columns Complexity: T(n)=n×n×n =n3 ЄO(n3)

Memory Complexity Analysis of algorithm efficiency in terms of memory. Time complexity is usually used. Memory complexity is occasionally useful.

Order : Big O • Definition For a given complexity function f(n), O(f(n)) is the set of complexity functions g(n) for which there exists some positive real constant c and some nonnegative integer N such that for all n ≤ N, . (ex) T1(n)=(n-1).n/2 Є O(n2) T2(n)=n3 Є O(n3) T3(n)=10000++1000 ЄO(n2) (cf. p29)

Divide and Conquer Top-Down Approach (p47) Divide the problem into subproblems Conquer subproblems Obtain the solution from the solutions of subproblems Binary search Problem: Is x in the sorted array S of size n ? Inputs: Sorted array S, a key x. Outputs: Location of x in S (0 if x is not in S)

Binary Search Locationout=location(1,n); Index location (index low, index high) { index mid; if(low>high) return 0 ; else{ mid= ; if (x==S[mid]) return mid; else if (x<S[mid]) return location (low,mid-1); else return location (mid+1,high); } }

Worst-Case Complexity: Binary Search Locationout=location(1,n); Index location (index low, index high){ index mid; if(low>high) return 0 ; else{ mid= (low+high)/2; if (x==S[mid]) return mid; else if (x<S[mid]) return location (low,mid-1); else return location (mid+1,high); } } Basic operation: Comparison of x with S[mid] Input size: n (#items in the array S) W(n)=W(n/2) + 1 recursive top call level

Complexity: Binary Search W(n)=W(n/2)+1 , for n>1 , n a power of 2 W(1)=1 It appears that W(n) = log n + 1 (Induction base) For n=1, t1=1=log1+1 (Induction hypothesis) Assume that W(n)=logn + 1 (Induction step) L=W(2n)=log(2n)+1 R=W(2n)=W(n)+1=(logn + 1)+1 =logn + log2 + 1=log(2n) + 1

Merge Sort (O(nlogn))

Quick Sort Sort by dividing the array into two partitions Using a pivot item. (ex)(first item) Quick sort(index low , index high) {index pivot;/*index of the pivot*/ if(high>low){ partition (low, high, pivot ); quicksort (low, pivot-1); quicksort (pivot+1, high); } }

Homework Given 20 15 25 22 11 20 30 27 (n=8) • Mergesort as in Fig 2.2 P54 • Quicksort as in Fig 2.3 P61 • Partition as in Table 2.2 P62 Due 1 week

Worst-case complexity: Quick sort • Worst case: When the array is already sorted • Time to partition: Tp (x) = n – 1 • Time to sort left subarray = T(0) • Time to sort right subarray = T(n-1) • Quick Sort T(n) = T(0) + T(n-1) + (n-1) for n > 0 T(n) = T(n-1) + (n-1), since T(0) = 0 T(n) = n(n-1)/2 ϵ O() • Average-case complexity: O (nlogn)

Dynamic Programming (Bottom-up) • Dynamic programming • Establish a recursive property • Solve in bottom-up fashion by solving smaller instances first • (ex): Fibonacci (Iterative) • Divide-and-conquer • Divide a problem into smaller instances • Solve these smaller instances (blindly) • Examples: • Fibonacci (Recursive): Instances are related • Merge sort: Instances are unrelated

Binomial coefficient • Frequently, n! is too large to compute directly • Proof:

Binomial coefficients: Divide-and-conquer • Algorithm /* Inefficient */ intbin (intn, intk) { if ( k = = 0 || n = = k) return 1; else returnbin (n-1, k - 1)+bin (n - 1, k); }

Binomial coefficients Figure 3.1: The array B used to compute the binomial coefficient Complexity : O(nK)

Example 3.1: Compute Compute row 0: {This is done only to mimic the algorithm exactly.} {The value B [0] [0] is not needed in a later computation.} B [0] [0] = 1 Compute row 1: B [1] [0] = 1B [1] [1] = 1 Compute row 2: B [2] [0] = 1B [2] [1] = B [1] [0] + B [1] [1] = 1+1 = 2B [2] [2] = 1 Compute row 3: B [3] [0] = 1B [3] [1] = B [2] [0] + B [2] [1] = 1+2 = 3B [3] [2] = B [2] [1] + B [2] [2] = 2+1 = 3 Compute row 4: B [4] [0] = 1B [4] [1] = B [3] [0] + B [3] [1] = 1+3 = 4B [4] [2] = B [3] [1] + B [3] [2] = 3+3 = 6

Binomial Coefficient: Dynamic Programming • Establish a recursive property • Solve in bottom up fashion Algorithm: intbin2 (intn, intk){ index i, j;intB[0..n][0..k]; for (i = 0; i < = n; i ++) for (j = 0; j < = minimum(i, k); j ++) if (j == 0 || j == i)B[i][j] = 1; elseB[i][j] = B[i - 1][j - 1] + B[i - 1][j]; return B[n][k];}

HOMEWORK • Use dynamic programming approach to compute B[5][3]. • Draw diagram like figure 3.1 (Page 94) • Due in 1 week

Binary Search Tree • Definition: For a given node n, • Each node contains one key • Key (node in the left subtree of n) <= Key (n) • Key(n) <= Key(node in the right subtree of n) • Optimality depends on the probability

Binary Search Tree • Depth(n): # edges in the unique path from the root to n. • (Depth=level) • Search time = depth(key) + 1 • The root has a depth of 0.

Binary Search Algorithm structnodetype{ Key type key; Nodetype* left; Nodetype* right; }; typeofnodetype* node_pointer; Void search (node_pointer tree, keytypekeyin, node_pointer & p) { {bool found=false; p=tree; while(!found) if (p->key==keyin) found=true; elseif(keyin < p->key) p=p->left; else p=p->right; }

Example

Greedy Approach Start with an empty set and add items to the set until the set represents a solution. Each iteration consists of the following components: A selection procedure A feasibility check A solution check

Spanning Tree A connected subgraph that contains all the vertices and is a tree.

Graph G=(V , E) Where V is a finite set of vertices and E is a set of edges (pairs of vertices in V). (ex) V={v1, v2, v3, v4, v5} E={(v1, v2,), (v1, v3,), (v2, v3,), (v2, v4,), (v3, v4,), (v3, v5,), (v4, v5,), }

Weight of a Graph

Prim’s Algorithm Figure 4.4: A weighted graph (in upper-left corner) and the steps in Prim's algorithm for that graph. The vertices in Y and the edges if F are shaded at each step.

Prim’s Algorithm F = Ø; for (i = 2; i <= n; i++){ // Initialize nearest [i] = 1; // v1 is the nearest distance [i] = W[1] [i] ; // distance is the weight } repeat (n - 1 times){ // Add all n - 1 vertices min = ∞ for (i = 2; i <= n; i++) if (0 ≤ distance [i] < min) { min = distance [i]; vnear = i; } e = edge connecting vnear and nearest [vnear]; F=F ∪{e} //add e to F distance [vnear] = - 1; for (i = 2; i <= n; i++) //update distance if (W[i] [vnear] < distance [i]){ distance = W[i] [vnear]; nearest [i] = vnear; } }

Prim’s Algorithm

Prim’s Spanning Tree Complexity : O(n2) (n-1) iterations of the repeat loop (n-1) iterations in two for loops T(n)= 2(n-1) (n-1) Theorem Prim’s algorithm always produces a minimum Spanning tree.

Dijkstra’s Shortest Paths Figure 4.8: A weighted, directed graph (in upper-left corner) and the steps in Dijkstra's algorithm for that graph. The vertices in Y and the edges in F are shaded in color at each step.

Dijkstra’s Algorithm F = Ø; for (i = 2; i<= n; i++){ //Initialize touch [i] = 1; // paths from V1 length [i] = W[1] [i]; } repeat (n - 1 times){ min = ∞; for (i = 2; i < = n; i++) if ( 0 ≤ length [i] < min) { min = length [i]; vnear = i; } e = edge from from[vnear]to vnear; F=F ∪{e} //add e to F for (i = 2; i < = n; i++) if (length [vnear] + W[vnear] [i] < length [i]){ length[i] = length[vnear] + W[vnear][i]; touch[i] = vnear; } length[vnear] = -1; }

Complexity Prim’s and Dijkstra’s : O (n2) Heap implementation : O (mlogn) Fibonacci heap implementation : O (m + nlogn) 1.Find a minimum spanning tree for the following graph 2.Find the shortest paths from V4 to all the other vertices

Scheduling Minimizing the total time (waiting + service) (ex) Three jobs : t1=5, t2=10, t3=4. Schedule Total Time in the System [1, 2, 3]5+(5+10)+(5+10+4) = 39 [1, 3, 2]5+(5+4)+(5+4+10) = 33 [2, 1, 3]10+(10+5)+(10+5+4) = 44 . . . [3, 1, 2]4+(4+5)+(4+5+10) = 32 3! cases

Optimal Scheduling for Total Time Smallest service time first. // Sort the jobs in nondecreasing order of service time // Schedule in sorted order. Complexity (sorting) w(n)Є O ( nlogn )

Schedule with Deadlines Schedule to maximize the total profit. Each job takes one unit of time to finish. (ex) JobDeadlineProfit 1 2 30 2 1 35 3 2 25 4 1 40 [1,3] : TP=30+25=55 [2,1] : TP=35+30=65 … [4,1] : TP=40+30=70.(optimal) … Is highest profit first optimal?

Algorithm & Application