1 / 46

CS 3343: Analysis of Algorithms

Learn how to solve recurrence relations using recursion tree method. Analyses of algorithms, merge sort, binary search, insertion sort, factorial, and power computation. Discover various methods to solve complex recurrences.

leomaw
Download Presentation

CS 3343: Analysis of Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 3343: Analysis of Algorithms Lecture 5: Solving recurrence by recursion-tree method

  2. 316 =3x 3 x 3 …. x 3 Answer: 15 316 =38 x 38 38 =34 x 34 34 =32 x 32 32 =3x 3 Problem of the day • How many multiplications do you need to compute 316? Answer: 4

  3. Pseudo code int pow (b, n) // compute bn m = n >> 1; p = pow (b, m); p = p * p; if (n % 2) return p * b; else return p;

  4. Pseudo code int pow (b, n) m = n >> 1; p = pow (b, m); p = p * p; if (n % 2) return p * b; else return p; int pow (b, n) m = n >> 1; p = pow(b,m) * pow(b,m); if (n % 2) return p * b; else return p;

  5. Outline • Review of last lecture – analyzing recursive algorithms • Defining recurrence relation • Today: analyzing recursive algorithms • Solving recurrence relation

  6. Sloppiness:Should be T( n/2 ) + T( n/2) , but it turns out not to matter asymptotically. Analyzing merge sort T(n) Θ(1) 2T(n/2) f(n) MERGE-SORTA[1 . . n] • If n = 1, done. • Recursively sort A[ 1 . . n/2 ] and A[ n/2+1 . . n ] . • “Merge” the 2 sorted lists

  7. Recurrence for merge sort • Divide: Trivial. • Conquer: Recursively sort 2 subarrays. • Combine: Merge two sorted subarrays T(n) = 2T(n/2) + Θ(n) # subproblems Dividing and Combining subproblem size

  8. Binary Search BinarySearch (A[1..N], value) { if (N == 0) return -1; // not found mid = (1+N)/2; if (A[mid] == value) return mid; // found else if (A[mid] > value) return BinarySearch (A[1..mid-1], value); else return BinarySearch (A[mid+1, N], value) }

  9. Recursive Insertion Sort RecursiveInsertionSort(A[1..n]) 1. if (n == 1) do nothing; 2.RecursiveInsertionSort(A[1..n-1]); 3. Find index i in A such that A[i] <= A[n] < A[i+1]; 4. Insert A[n] after A[i];

  10. Compute factorial Factorial (n) if (n == 1) return 1; return n * Factorial (n-1); • Note: here we use n as the size of the input. However, usually for such algorithms we would use log(n), i.e., the bits needed to represent n, as the input size.

  11. Recurrence for computing power int pow (b, n) m = n >> 1; p = pow (b, m); p = p * p; if (n % 2) return p * b; else return p; int pow (b, n) m = n >> 1; p=pow(b,m)*pow(b,m); if (n % 2) return p * b; else return p; T(n) = ? T(n) = ?

  12. Recurrence for computing power int pow (b, n) m = n >> 1; p = pow (b, m); p = p * p; if (n % 2) return p * b; else return p; int pow (b, n) m = n >> 1; p=pow(b,m)*pow(b,m); if (n % 2) return p * b; else return p; T(n) = T(n/2)+(1) T(n) = 2T(n/2)+(1)

  13. What do they mean? Challenge: how to solve the recurrence to get a closed form, e.g. T(n) = Θ (n2) or T(n) = Θ(nlgn), or at least some bound such as T(n) = O(n2)?

  14. Solving recurrence • Running time of many algorithms can be expressed in one of the following two recursive forms or Both can be very hard to solve. We focus on relatively easy ones, which you will encounter frequently in many real algorithms (and exams…)

  15. Solving recurrence • Recursion tree or iteration method - Good for guessing an answer • Substitution method - Generic method, rigid, but may be hard • Master method - Easy to learn, useful in limited cases only - Some tricks may help in other cases

  16. T(n) = Q(1)ifn = 1; 2T(n/2) + Q(n)ifn > 1. Recurrence for merge sort We will usually ignore the base case, assuming it is always a constant (but not 0).

  17. Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant.

  18. Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. T(n)

  19. dn T(n/2) T(n/2) Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant.

  20. dn dn/2 dn/2 T(n/4) T(n/4) T(n/4) T(n/4) Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant.

  21. Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn/2 dn/2 dn/4 dn/4 dn/4 dn/4 … Q(1)

  22. Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn/2 dn/2 h = log n dn/4 dn/4 dn/4 dn/4 … Q(1)

  23. Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn dn/2 dn/2 h = log n dn/4 dn/4 dn/4 dn/4 … Q(1)

  24. Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn dn/2 dn dn/2 h = log n dn/4 dn/4 dn/4 dn/4 … Q(1)

  25. Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn dn/2 dn dn/2 h = log n dn/4 dn/4 dn dn/4 dn/4 … … Q(1)

  26. Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn dn/2 dn dn/2 h = log n dn/4 dn/4 dn dn/4 dn/4 … … Q(1) #leaves = n Q(n)

  27. Recursion tree for merge sort Solve T(n) = 2T(n/2) + dn, where d > 0 is constant. dn dn dn/2 dn dn/2 h = log n dn/4 dn/4 dn dn/4 dn/4 … … Q(1) #leaves = n Q(n) Total Q(n log n) Later we will usually ignore d

  28. Recurrence for computing power int pow (b, n) m = n >> 1; p = pow (b, m); p = p * p; if (n % 2) return p * b; else return p; int pow (b, n) m = n >> 1; p=pow(b,m)*pow(b,m); if (n % 2) return p * b; else return p; T(n) = T(n/2)+(1) T(n) = 2T(n/2)+(1) Which algorithm is more efficient asymptotically?

  29. log(n) Time complexity for Alg1 Solve T(n) = T(n/2) + 1 • T(n) = T(n/2) + 1 = T(n/4) + 1 + 1 = T(n/8) + 1 + 1 + 1 = T(1) + 1 + 1 + … + 1 = Θ (log(n)) Iteration method

  30. Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1.

  31. Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1. T(n)

  32. 1 T(n/2) T(n/2) Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1.

  33. 1 1 1 T(n/4) T(n/4) T(n/4) T(n/4) Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1.

  34. Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1. 1 1 1 1 1 1 1 … Q(1)

  35. Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1. 1 1 1 h = log n 1 1 1 1 … Q(1)

  36. Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1. 1 1 1 1 h = log n 1 1 1 1 … Q(1)

  37. Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1. 1 1 1 2 1 h = log n 1 1 1 1 … Q(1)

  38. Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1. 1 1 1 2 1 h = log n 1 1 4 1 1 … … Q(1)

  39. Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1. 1 1 1 2 1 h = log n 1 1 4 1 1 … … Q(1) #leaves = n Q(n)

  40. Time complexity for Alg2 Solve T(n) = 2T(n/2) + 1. 1 1 1 2 1 h = log n 1 1 4 1 1 … … Q(1) #leaves = n Q(n) Total Q(n)

  41. n - 1 More iteration method examples • T(n) = T(n-1) + 1 = T(n-2) + 1 + 1 = T(n-3) + 1 + 1 + 1 = T(1) + 1 + 1 + … + 1 = Θ (n)

  42. More iteration method examples • T(n) = T(n-1) + n = T(n-2) + (n-1) + n = T(n-3) + (n-2) + (n-1) + n = T(1) + 2 + 3 + … + n = Θ (n2)

  43. 3-way-merge-sort 3-way-merge-sort (A[1..n]) If (n <= 1) return; 3-way-merge-sort(A[1..n/3]); 3-way-merge-sort(A[n/3+1..2n/3]); 3-way-merge-sort(A[2n/3+1.. n]); Merge A[1..n/3] and A[n/3+1..2n/3]; Merge A[1..2n/3] and A[2n/3+1..n]; • Is this algorithm correct? • What’s the recurrence function for the running time? • What does the recurrence function solve to?

  44. Unbalanced-merge-sort ub-merge-sort (A[1..n]) if (n<=1) return; ub-merge-sort(A[1..n/3]); ub-merge-sort(A[n/3+1.. n]); Merge A[1.. n/3] and A[n/3+1..n]. • Is this algorithm correct? • What’s the recurrence function for the running time? • What does the recurrence function solve to?

  45. More recursion tree examples (1) • T(n) = 3T(n/3) + n • T(n) = T(n/3) + T(2n/3) + n • T(n) = 2T(n/4) + n • T(n) = 2T(n/4) + n2 • T(n) = 3T(n/2) + n • T(n) = 3T(n/2) + n2

  46. More recursion tree examples (2) • T(n) = T(n-2) + n • T(n) = T(n-2) + 1 • T(n) = 2T(n-2) + n • T(n) = 2T(n-2) + 1

More Related