1 / 106

Analysis & Design of Algorithms (CSCE 321)

Analysis & Design of Algorithms (CSCE 321). Prof. Amr Goneid Department of Computer Science, AUC Part 7. Divide & Conquer Algorithms. Divide & Conquer Algorithms. Divide & Conquer Algorithms. The General Method Time Complexity Common Recurrence relations

meena
Download Presentation

Analysis & Design of Algorithms (CSCE 321)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Analysis & Design of Algorithms(CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 7. Divide & Conquer Algorithms Prof. Amr Goneid, AUC

  2. Divide & Conquer Algorithms Prof. Amr Goneid, AUC

  3. Divide & Conquer Algorithms • The General Method • Time Complexity • Common Recurrence relations • A General Divide & Conquer Recurrence • Merge Sort Algorithm • Quicksort Algorithm • An Application of Quicksort: Convex Hull • Selection by Partitioning • Closest Pair Problem Prof. Amr Goneid, AUC

  4. 1. The General Method To solve a problem using Divide and Conquer: • Divide it into smaller problems • Solve the smaller problems recursively • Combine their solutions into a solution for the big problem Prof. Amr Goneid, AUC

  5. The General Method • The base case for recursion is a sub-problem of constant (and small) size. • Analysis of D&Q algorithms is done by constructing a recurrence relation • We can obtain T(n) in a closed form by solving the recurrence relation Prof. Amr Goneid, AUC

  6. The General Method • For problem p, small(p) = true if p is simple. In this case S(p) is the solution. Otherwise, Divide, Conquer, then Combine. • Divide & Conquer Algorithm DQ(p) { if small(p) return S(p); else { 1. Divide p into smaller instances p1,p2,..,pk 2. Apply DQ to each pi 3. Return Combine(DQ(p1),DQ(p2),…,DQ(pk))} } Prof. Amr Goneid, AUC

  7. 2. Time Complexity Let g(n) = time for S(p) when p is small f(n) = time to divide p and combine solutions then for p small T(n) = g(n) otherwise Prof. Amr Goneid, AUC

  8. 3. Common Recurrence Relations • A recursive algorithm that halves the input in one step then processes the two halves: T(n) = 2T(n/2) + 1 for n >1 with T(1) given • A recursive algorithm that halves the input in one step then processes one half and ignores the other: T(n) = T(n/2) + 1 for n >1 with T(1) given Prof. Amr Goneid, AUC

  9. Common Recurrence Relations • A recursive algorithm that examines every element before dividing and then ignores one half: T(n) = T(n/2) + n for n >1 with T(1) given • A recursive algorithm that makes a linear pass through the input, before, during or after it is split into two halves then processes the two halves: T(n) = 2 T(n/2) + n for n >1 with T(1) given Prof. Amr Goneid, AUC

  10. Example: Recursive Binary Search Algorithm BinSearch (a , s , e , x) { if (s == e) // if small (p), i.e. one element { if (x ==as) return s ; else return -1 ;} // return S(p) else { // two or more elements mid = (s + e)/2 ; // Divide if (x ==amid) return mid ; else if (x <amid) // Conquer return BinSearch (a , s , mid-1 , x) ; // n/2 or else return BinSearch (a , mid+1 , e , x) ; } // n/2 } // No Combine Prof. Amr Goneid, AUC

  11. Binary Search Demo http://www.cosc.canterbury.ac.nz/people/mukundan/dsal/BSearch.html Prof. Amr Goneid, AUC

  12. Worst Case of Binary Search Let T(n) be the worst case number of comparisons used on an array of n items. Hence Now we seek a closed form solution for T(n) Prof. Amr Goneid, AUC

  13. Worst Case of Binary Search Prof. Amr Goneid, AUC

  14. 4. A General D&Q Recurrence Prof. Amr Goneid, AUC

  15. A General Solution The secondary recurrence is solved as a first order linear recurrence. For T(1) i.e. F(0) given, we have as shown before: Prof. Amr Goneid, AUC

  16. Very Common: f(n) = b nx Prof. Amr Goneid, AUC

  17. Case(a): f(n) = b nx, a = cx Prof. Amr Goneid, AUC

  18. Summary Prof. Amr Goneid, AUC

  19. Case(a): f(n) = b nx, a = cx Prof. Amr Goneid, AUC

  20. Other examples for a = cx Recursive Power Function The following function computes xn Pow (x , n) { if (n == 0) return 1.0; else if (n == 1) return x; else if (n mod 2) return Pow (x*x , n / 2) * x; else return Pow (x*x , n / 2); } Show that the number of arithmetic operations performed by this algorithm in the best case is T(n) = log n Prof. Amr Goneid, AUC

  21. Solution In the best case n = 2m , so that n is always even and we obtain the recurrence relation: T(n) = T(n/2) + 1 , for n > 1 with T(1) = 0 Here, a = 1, b = 1, c = 2, x = 0 with the solution : T(n) = bnx log n = log n Prof. Amr Goneid, AUC

  22. Other examples for a = cx Recursive log Function A recursive function to compute the largest integer less than or equal to log2 n. (Hint: for n > = 2 , the value of this function for n is one greater than for n/2 ). int Log2(int n) { if (n == 1) return 0; else return (1 + Log2(n/2)); } The number of integer divisions has the recurrence relation: T(n) = T(n/2) + 1 , for n > 1 with T(1) = b = 0 with the solution : T(n) = log n = O(log n) Prof. Amr Goneid, AUC

  23. Other examples for a = cx Find the Number of Binary Digits in the Binary Representation of a Positive Decimal Integer using a D&Q recursive algorithm. int BinDig (int n) { if (n == 1) return 1; else return (1 + BinDig(n/2)); } T(n) = number of arithmetic operations. T(n) = T(n/2) + 2 for n > 1 with T(1) = 0 a = 1, b = 2, c = 2, x = 0 T(n) = 2 log n = O(log n) Prof. Amr Goneid, AUC

  24. Other examples for a = cx Other Recurrences with a = cx • T(n) = 4 T(n/2) + n2 for n > 1 with T(2) = 1 Hence, T(n) = n2(log n – ¾) = O(n2 log n) • T(n) = 9T(n/3) + n2 for n > 1 with T(1) = 0 Hence, T(n) = n2 log3 n • T(n) = 16 T(n/4) + n2 for n > 1 with T(1) = 0 Hence, T(n) = n2 log4 n Prof. Amr Goneid, AUC

  25. Case(b): f(n) = b nx, a cx Example: a Fractal Algorithm The following function draws recursive squares (called a fractal star). The drawing primitive is Box (x , y , n) which draws a square of side (n) pixels centered at (x,y) : void STAR( int x, int y, int n) { if (n > 1) { STAR(x-n , y+n , n/2); STAR(x+n , y+n , n/2); STAR(x-n , y-n , n/2); STAR(x+n , y-n , n/2); Box(x , y , n); } } Prof. Amr Goneid, AUC

  26. Fractal Algorithm The recurrence relation is: T(n) = 4 T(n/2) + 4 n for n > 1 with T(1) = 0 Here, a = 4, b = 4, c = 2, x =1 so that a  cx The general solution is: Prof. Amr Goneid, AUC

  27. Example: MaxMinStraightforward Algorithm // Return max and min values in locations s to e in array a[ ] MaxMin (a, s , e , mx, mn) { mx = mn = as; for i = s+1 to e { if (ai > mx) mx = ai ; if (ai < mn) mn = ai ; } } Invoke as MaxMin(a , 1 , n , mx , mn) ; Let T(n) be the number of comparisons of array elements. Hence, T(n) = 2 ( n – 1) independent of the data in a[ ]. Prof. Amr Goneid, AUC

  28. MaxMin D&Q Algorithm MaxMin (a, s , e , mx , mn) { if (s  e-1) // one or two elements if (as > ae) {mx = as; mn = ae;} else {mx = ae; mn = as;} else { m = (s+e)/2; MaxMin (a,s,m,mx,mn); Maxmin (a,m+1,e,mx1,mn1); mx = max(mx,mx1); mn = min(mn,mn1); } } // The functions max/min(a,b) find the larger/smaller of (a,b). Each will cost one comparison. Prof. Amr Goneid, AUC

  29. MaxMin Analysis The recurrence relation is: T(n) = 2 T(n/2) + 2 for n > 2 with T(2) = 1 Here, a = 2, b = 2, c = 2, x =0 so that a  cx The general solution is: This is only 75% of the cost of the straightforward algorithm. Prof. Amr Goneid, AUC

  30. Other Examples with a  cx Copy Binary Tree Algorithm The following function receives a pointer (t) to a binary tree and returns a pointer to an exact copy of the tree : treeNode *CopyTree ( treeNode *t ) { treeNode *p; if (t) { p = new treeNode ; p->left = CopyTree(t->left) ; p->right = CopyTree(t->right); p->info = t->info ; return p ; }; else return NULL ; } Prof. Amr Goneid, AUC

  31. Other Examples with a  cx Copy Binary Tree Algorithm In the worst case, the tree is full and the number of visits is: T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1 Hence T(n) = 2n – 1 = O(n) Prof. Amr Goneid, AUC

  32. Other Examples with a  cx Pre-Order Traversal of a Binary Tree void preorder (tree t) { if (t) { visit (t); preorder (t -> left); preorder (t -> right); } } In the worst case, the tree is full and the number of visits is: T(n) = 2 T(n/2) + 1 for n > 1 with T(1) = 1 Hence T(n) = 2n – 1 = O(n) Prof. Amr Goneid, AUC

  33. Other Examples with a  cx Other Algorithms • T(n) = T(n/2) + n2 for n > 1 with T(1) = 0 Hence T(n) = (4/3) (n2 – 1) • T(n) = 2 T(n/2) + n2 for n > 1 with T(1) = 0 Hence T(n) = 2n(n – 1) • T(n) = 8 T(n/2) + n2 for n > 1 with T(1) = 0 Hence T(n) = n2 (n – 1) Prof. Amr Goneid, AUC

  34. 5. MergeSort(a) Merging • Definition: Combine two or more sorted sequences of data into a single sorted sequence. • Formal Definition: The input is two sorted sequences, A={a1, ..., an} and B={b1, ..., bm} The output is a single sequence, merge(A,B), which is a sorted permutation of {a1, ..., an, b1, ..., bm}. Prof. Amr Goneid, AUC

  35. Merging merge(A, B) is A, if B is empty, B, if A is empty, {a1}.merge({a2, ..., an}, B) if a1 <= b1, and {b1}.merge(A, {b2, ..., bm}) otherwise. The symbol "." stands for concatenation, for example, {a1, ..., an}.{b1, ..., bm} = {a1, ..., an, b1, ..., bm}. Prof. Amr Goneid, AUC

  36. Merge Algorithm array B Merge(A,p,q,r) { copy Ap..r to Bp..r and set Ap..r to empty while (neither L1 nor L2 empty) { compare first items of L1 & L2 remove smaller of the two from its list add to end of A } concatenate remaining list to end of A return A } L1 q p q+1 L2 r Prof. Amr Goneid, AUC

  37. Example i j B q r p A z Prof. Amr Goneid, AUC

  38. Example i j B q r p A z Prof. Amr Goneid, AUC

  39. Example i j B q r p A z Prof. Amr Goneid, AUC

  40. Example i j B q r p A z Prof. Amr Goneid, AUC

  41. Example i j B q r p A z Prof. Amr Goneid, AUC

  42. Example i j B q r p A z Prof. Amr Goneid, AUC

  43. Example i j B q r p A z Prof. Amr Goneid, AUC

  44. Example i j B q r p A z Prof. Amr Goneid, AUC

  45. Worst Case Analysis • |L1| = size of L1, |L2| = size of L2 • In the worst case |L1| = |L2| = n/2 • Both lists empty at about same time, so everything has to be compared. • Each comparison adds one item to A so the worst case is T(n) = |A|-1 = |L1|+|L2|-1 = n-1 = O(n) comparisons. Prof. Amr Goneid, AUC

  46. (b) MergeSort Methodology • Invented by Von Neumann in 1945 • Recursive Divide-And-Conquer Prof. Amr Goneid, AUC

  47. MergeSort Methodology • Divides the sequence into two subsequences of equal size, sorts the subsequences and then merges them into one sorted sequence • Fast, but uses an extra space Prof. Amr Goneid, AUC

  48. Methodology (continued) • Divide: Divide n element sequence into two subsequences each of n/2 elements • Conquer: Sort the two sub-arrays recursively • Combine: Merge the two sorted subsequences to produce a sorted sequence of n elements Prof. Amr Goneid, AUC

  49. Algorithm MergeSort (A, p, r) // Mergesort array A[ ] locations p..r { if (p < r) // if there are 2 or more elements { q = (p+r)/2; // Divide in the middle // Conquer both MergeSort (A , p , q); MergeSort(A , q+1 , r); Merge(A , p , q , r); // Combine solutions } } Invoke as MergeSort(a , 1 , n) Prof. Amr Goneid, AUC

  50. Merge Sort Example Prof. Amr Goneid, AUC

More Related