1 / 61

Part 5. Computational Complexity (3)

Part 5. Computational Complexity (3). CS 200 Algorithms and Data Structures. Outline. Recurrence Relations Divide and Conquer Understanding the Complexity of Algorithms. Recurrence Relations: An Overview. What is a recurrence relation? A recursively defined sequence Example

jermaine-holcomb
Download Presentation

Part 5. Computational Complexity (3)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Part 5.Computational Complexity (3) CS 200 Algorithms and Data Structures

  2. Outline • Recurrence Relations • Divide and Conquer • Understanding the Complexity of Algorithms

  3. Recurrence Relations: An Overview • What is a recurrence relation? • A recursively defined sequence • Example • Arithmetic progression: a, a+d, a+2d, …, a+nd • a0 = a • an = an-1 + d

  4. Recurrence Relations:Formal Definition • Sequence = Recurrence relation + Initial conditions (“base case”) • Example: an = 2an-1 + 1, a1 = 1

  5. Compound Interest • You deposit $10,000 in a savings account that yields 10% yearly interest. How much money will you have after 30 years? (b is balance, r is rate)

  6. Recursively defined functions and recurrence relations • A recursive function • Example: f(0) = a(base case) f(n) = f(n-1) + dfor n > 0 (recursive step) • The above recursively defined function generates the sequence • a0 = a • an = an-1 + d • A recurrence relation produces a sequence, an application of a recursive function produces a value from the sequence

  7. How to Approach Recursive Relations Substitution Sequence of Values Recursive Functions • f(0) = 0(base case) • f(n) = f(n-1) + 2for n > 0 (recursive part) • f(0) = 0 • f(1) = f(0) +2 = 2 • f(2) = f(1)+2 = 4 • f(3) = f(2) +2 = 6 Closed Form?(solution, explicit formula)

  8. Find a recursive function(1/5) • Give an inductive definition of the factorial function F(n)= n!

  9. *Is recursively defined functions well-defined? • For every positive integer, the value of the function at this integer is determined in an unambiguous way. • With given any of two parts of the definition, we obtain the same value no matter how we apply the two parts of the definition.

  10. Find a recursive function(2/5) • Give a recursive definition of an, where a is a nonzero real number and n is a nonnegative integer. • Rosen Chapter 5 example 3-2 pp. 346

  11. Find a recursive function(3/5) • Give a recursive definition of, • Rosen Chapter 5 example 3-3 pp. 346

  12. Find a recursive function(4/5) • Give a recursive definition of l(w), the length of the string w. • Rosen Chapter 5 example 3-7 pp. 350

  13. Find a recursive function(5/5) • Find a recurrence relation and give initial conditions for the number of bit strings of length n that do not have two consecutive 0s. How many such bit strings are there of length five?

  14. Solving recurrence relations (1) Solve a0 = 2; an = 3an-1 , n > 0 (1) What is the recursive function? (2) What is the sequence of values? Hint 1: Solve by substitution

  15. Solving recurrence relations (2) Solve a0 = 2; an = 3an-1 + 2 for n > 0 (1) What is the recursive function? (2) What is the sequence of values? Hint 1: Solve by substitution Hint 2: Use formula for summing geometric series 1 + r + r2 + … + rn = (rn+1- 1)/(r – 1) if r ≠ 1

  16. Solving recurrence relations (3) Solve a0 = 1; an = an-1 + nfor n > 0 Hint 1: Solve by substitution

  17. Modeling with Recurrence • Suppose that the number of bacteria in a colony triples every hour • Set up a recurrence relation for the number of bacteria after n hours have elapsed. • 100 bacteria are used to begin a new colony.

  18. Linear Recurrence Relations A linear homogeneous recurrence relation of degree k with constant coefficients is a recurrence relation of a form an = c1an-1 + c2an-2+ … + ckan-k where, c1,c2 ,c3 …. ,ck are real numbers and ck is not 0.

  19. Is this linear homogeneous recurrence relation? • fn = fn-1 + fn-2 • an = an-1 + a2n-2 • Hn = 2Hn-1+1 • Modeling of problems • They can be systematically solved. Linear homogeneous recurrence relations? 1 B. 2 C. 3 D. 1,2,3

  20. Theorem 1 Letc1andc2 be real numbers. Suppose that r2 –c1r – c2 = 0 has two distinct rootsr1 and r2 . Then the sequence {an} is a solution of the recurrence relationan = c1an-1 +c2an-2 if and only if an= α1r1n+ α2r2n for n= 0,1,2,… where α1 and α2 are constants. Rosen section8.2 theorem 1, pp.515

  21. What is the solution of the recurrence relation • an = an-1 + 2an-2 with a0 = 2 and a1 = 7? Rosen Section 8-2 Example 3 pp. 516

  22. Outline • Recurrence Relations • Divide and Conquer • Understanding the Complexity of Algorithms

  23. Divide-and-Conquer Basic idea: Take large problem and divideit into smaller problems until problem is trivial, then combine parts to make solution. Recurrence relation for the number of steps required: f(n) = a f(n / b) + g(n) n/b: the size of the sub-problems solved a: number of sub-problems g(n): steps necessary to combine solutions to sub-problems

  24. Example: Binary Search public static intbinSearch (intmyArray[], int first, intlast, int value) { // returns the index of value or -1 if not in the array int index; if (first > last) { index = -1; } else { intmno = (first + last)/2; if (value == myArray[mno]) { index = mno; } else if (value < myArray[mno]) { index = binSearch(myArray, first, mno-1, value); } else { index = binSearch(myArray, mno+1, last, value); } } return index; } What are a, b, and g(n)?

  25. Estimating big-O (Master Theorem) From section 8.3 (for 6th Edition 7.3) in Rosen

  26. Example: Binary Search using the Master Theorem f(n) = a f(n / b) + nd for the binary search algorithm f(n) = a f(n / b) + nd = 1 f(n / 2) + 3 Therefore, d = 0 (to make nd a constant), b = 2, a = 1. bd = 20 = 1, It satisfies the second condition of the Master theorem. So, f(n) = O(ndlog2n) = O(n0log2n) = O(log2n)

  27. Step-by-step Master Theorem • Modeling the divide-conquer with a recurrence relation. • f(n) is a recurrence function representing the number of operations with size of input n.

  28. Step-by-step Master Theorem Method_A(array){ Method_A(left_half); Method_A(right_half); Complete_the_Method(array); } Method_A(array) Method A Call Method_A(left_half) Call Method_A(right_half) Method A Method A Method A Method A Method A Method A

  29. Step-by-step Master Theorem Method_B(array){ Method_A(possible_half); Do_nothing_Method(array); } Method_A(array) Method B Call Method_B(left_half) Method B Method B

  30. Method_C(array){ Method_C(first_one_third); Method_C(second_one_third); Method_C(last_one_third); Complete_method(array); } Step-by-step Master Theorem Method_C(array) Method C Call Method_C(last_one_thrid) Call Method_A(right_half) Call Method_C(first_one_thrid) Call Method_C(second_one_thrid) Method C Method C Method C Method C Method C Method C Method C Method C Method C Method C Method C Method C

  31. LEVEL 1 Method … LEVEL 2 Method Method Method LEVEL 3 Method Method Method Method Method Method Method Method Method Method Method Method … LEVEL 4 Goal 1: What is the complexity per level? Goal 2: What is the total complexity?

  32. LEVEL 1 Method … Method Method Method Method Method Method Method Method Method Method Method Method Method Method Method … “Method” is called with input size n. “Method” calls “Method” recursively a times “Method” calls “Method” recursively with input size n/b

  33. LEVEL 1 Method … Method Method Method Method Method Method Method Method Method Method Method Method Method Method Method … “Method” includes software fragment on top of calling methods recursively, and it takes g(n) operations.

  34. LEVEL 1 Method … Method Method Method Method Method Method Method Method Method Method Method Method Method Method Method … Therefore, in LEVEL 1, “Method” requires, f(n) = a f(n / b) + g(n)

  35. Method … Method Method Method LEVEL 2 Method Method Method Method Method Method Method Method Method Method Method Method … This is also true for LEVEL 2 f(n) = a f(n / b) + g(n)

  36. Method … Method Method Method Method Method Method Method Method Method Method Method Method Method Method Method … LEVELi This is also true for LEVEL i f(n) = a f(n / b) + g(n)

  37. More generally, O(nd) if a < bd f(n) is O(ndlogn) if a = bd O(nlogba) if a > bd

  38. Base case • We will set d = 1 • The bottom level of the tree is equally well computed. • Base case • It is straightforward to extend the proof for the case when d ≠ 1.

  39. Goal 1. complexity per level • Let’s think about the recursion tree. • There will be logbn levels. • At each level, the number of subproblems will be multiplied by a. • Therefore, the number of subproblems at level i will be ai. • Each subproblem at level i is a problem of size (n/bi). • A subproblem of size (n/bi) requires (n/bi)d additional work. logbn levels

  40. The total number of units of work on level i • ai(n/bi)d = nd(ai/bdi) =nd(a/bd)i For the level i , the work per level is decreasing, constant, or increasing exactly when (a/bd)i is decreasing, constant, or increasing. logbn levels

  41. Observation • Now, observe that, (a/bd) = 1 a = bd • Therefore, the relations, • a < bd (2) a = bd, (3) a > bd are the conditions deciding the types of the growth function

  42. Goal 2: Bounding f(n) in the different cases. • In general, we have that the total work done is, • a < bd (2) a = bd (3) a > bd

  43. Case 1. a < bd • nd times a geometric series with a ratio of less than 1. • First item is the biggest one.

  44. Case 2. a = bd • (a/bd) = 1 • nd(1+1+1+…+1) = nd(logbn)

  45. Case 3. a > bd • (a/bd) > 1 • Therefore the largest term is the last one.

  46. Complexity of MergeSort with Master Theorem (1/2) • Mergesort splits a list to be sorted twice per level. • Uses fewer than n comparisons to merge the two sorted lists of n/2 items each into one sorted list. • Function M(n) satisfying the divide-and-conquer recurrence relation • M(n) = 2M(n/2) + n

  47. Complexity of MergeSort with Master Theorem (2/2) M(n) = 2M(n/2) + n for the mergesort algorithm f(n) = a f(n / b) + nd = 2 f(n / 2) + n1 Therefore, d = 1, b = 2, a = 2. bd = 21 = 2 It satisfies the second condition of the Master theorem. So, f(n) = O(ndlog2n) = O(n1log2n) = O(nlog2n)

  48. Quicksort?

  49. The Closest-Pair Problem • Consider the problem of determining the closest pair of points in a set of n points (x1, y1)…(xn, yn)in the plain, where the distance between two points (xi, yi)and (xj, yj) is the usual Euclidean distance, ________________ √(xi - xj)2 + (yi - yj)2

  50. Solution (1/5) • Calculate the distance between every pair of points and then find the smallest of these distances • It will takeO(n2)

More Related