1 / 51

Merge Sort Solving Recurrences The Master Theorem

Merge Sort Solving Recurrences The Master Theorem. Review: Asymptotic Notation. Upper Bound Notation: f(n) is O(g(n)) if there exist positive constants c and n 0 such that f(n)  c  g(n) for all n  n 0

brendabenson
Download Presentation

Merge Sort Solving Recurrences The Master Theorem

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Merge Sort Solving Recurrences The Master Theorem David Luebke 11/4/2020

  2. Review: Asymptotic Notation • Upper Bound Notation: • f(n) is O(g(n)) if there exist positive constants c and n0such that f(n)  c  g(n) for all n  n0 • Formally, O(g(n)) = { f(n):  positive constants c and n0such that f(n)  c  g(n)  n  n0 • Big O fact: • A polynomial of degree k is O(nk) David Luebke 21/4/2020

  3. Review: Asymptotic Notation • Asymptotic lower bound: • f(n) is (g(n)) if  positive constants c and n0such that 0  cg(n)  f(n)  n  n0 • Asymptotic tight bound: • f(n) is (g(n)) if  positive constants c1, c2, and n0 such that c1 g(n)  f(n)  c2 g(n)  n  n0 • f(n) = (g(n)) if and only if f(n) = O(g(n)) AND f(n) = (g(n)) David Luebke 31/4/2020

  4. Other Asymptotic Notations • A function f(n) is o(g(n)) if  positive constants c and n0such that f(n) < c g(n)  n  n0 • A function f(n) is (g(n)) if  positive constants c and n0such that c g(n) < f(n)  n  n0 • Intuitively, • o() is like < • O() is like  • () is like > • () is like  • () is like = David Luebke 41/4/2020

  5. Merge Sort MergeSort(A, left, right) { if (left < right) { mid = floor((left + right) / 2); MergeSort(A, left, mid); MergeSort(A, mid+1, right); Merge(A, left, mid, right); } } // Merge() takes two sorted subarrays of A and // merges them into a single sorted subarray of A // (how long should this take?) David Luebke 51/4/2020

  6. Merge Sort: Example • Show MergeSort() running on the arrayA = {10, 5, 7, 6, 1, 4, 8, 3, 2, 9}; David Luebke 61/4/2020

  7. Analysis of Merge Sort Statement Effort • So T(n) = (1) when n = 1, and 2T(n/2) + (n) when n > 1 • So what (more succinctly) is T(n)? MergeSort(A, left, right) { T(n) if (left < right) { (1) mid = floor((left + right) / 2); (1) MergeSort(A, left, mid); T(n/2) MergeSort(A, mid+1, right); T(n/2) Merge(A, left, mid, right); (n) } } David Luebke 71/4/2020

  8. Recurrences • The expression:is a recurrence. • Recurrence: an equation that describes a function in terms of its value on smaller functions David Luebke 81/4/2020

  9. Solving Recurrences • The substitution method (CLR 4.1) • A.k.a. the “making a good guess method” • Guess the form of the answer, then use induction to find the constants and show that solution works • Examples: • T(n) = 2T(n/2) + (n)  T(n) = (n lg n) • T(n) = 2T(n/2) + n  ??? David Luebke 91/4/2020

  10. Solving Recurrences • The substitution method (CLR 4.1) • A.k.a. the “making a good guess method” • Guess the form of the answer, then use induction to find the constants and show that solution works • Examples: • T(n) = 2T(n/2) + (n)  T(n) = (n lg n) • T(n) = 2T(n/2) + n  T(n) = (n lg n) • T(n) = 2T(n/2 )+ 17) + n  ??? David Luebke 101/4/2020

  11. Solving Recurrences • The substitution method (CLR 4.1) • A.k.a. the “making a good guess method” • Guess the form of the answer, then use induction to find the constants and show that solution works • Examples: • T(n) = 2T(n/2) + (n)  T(n) = (n lg n) • T(n) = 2T(n/2) + n  T(n) = (n lg n) • T(n) = 2T(n/2+ 17) + n (n lg n) David Luebke 111/4/2020

  12. s(n) = n + s(n-1) = n + n-1 + s(n-2) = n + n-1 + n-2 + s(n-3) = n + n-1 + n-2 + n-3 + s(n-4) = … = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k) David Luebke 121/4/2020

  13. s(n) = n + s(n-1) = n + n-1 + s(n-2) = n + n-1 + n-2 + s(n-3) = n + n-1 + n-2 + n-3 + s(n-4) = … = n + n-1 + n-2 + n-3 + … + n-(k-1) + s(n-k) = David Luebke 131/4/2020

  14. So far for n >= k we have David Luebke 141/4/2020

  15. So far for n >= k we have • What if k = n? David Luebke 151/4/2020

  16. So far for n >= k we have • What if k = n? David Luebke 161/4/2020

  17. So far for n >= k we have • What if k = n? • Thus in general David Luebke 171/4/2020

  18. T(n) = 2T(n/2) + c 2(2T(n/2/2) + c) + c 22T(n/22) + 2c + c 22(2T(n/22/2) + c) + 3c 23T(n/23) + 4c + 3c 23T(n/23) + 7c 23(2T(n/23/2) + c) + 7c 24T(n/24) + 15c … 2kT(n/2k) + (2k - 1)c David Luebke 181/4/2020

  19. So far for n > 2k we have • T(n) = 2kT(n/2k) + (2k - 1)c • What if k = lg n? • T(n) = 2lg n T(n/2lg n) + (2lg n - 1)c = n T(n/n) + (n - 1)c = n T(1) + (n-1)c = nc + (n-1)c = (2n - 1)c David Luebke 191/4/2020

  20. Sorting Revisited • So far we’ve talked about two algorithms to sort an array of numbers • What is the advantage of merge sort? • What is the advantage of insertion sort? • Next on the agenda: Heapsort • Combines advantages of both previous algorithms David Luebke 201/4/2020 David Luebke 201/4/2020

  21. 16 14 10 8 7 9 3 2 4 1 Heaps • A heap can be seen as a complete binary tree: • What makes a binary tree complete? • Is the example above complete? David Luebke 211/4/2020 David Luebke 211/4/2020

  22. Heaps • A heap can be seen as a complete binary tree: • The book calls them “nearly complete” binary trees; can think of unfilled slots as null pointers 16 14 10 8 7 9 3 2 4 1 1 1 1 1 1 David Luebke 221/4/2020 David Luebke 221/4/2020

  23. 16 14 10 8 7 9 3 2 4 1 Heaps • In practice, heaps are usually implemented as arrays: A = 16 14 10 8 7 9 3 2 4 1 = David Luebke 231/4/2020 David Luebke 231/4/2020

  24. 16 14 10 8 7 9 3 2 4 1 Heaps • To represent a complete binary tree as an array: • The root node is A[1] • Node i is A[i] • The parent of node i is A[i/2] (note: integer divide) • The left child of node i is A[2i] • The right child of node i is A[2i + 1] A = 16 14 10 8 7 9 3 2 4 1 = David Luebke 241/4/2020 David Luebke 241/4/2020

  25. Referencing Heap Elements • So… Parent(i) { return i/2; } Left(i) { return 2*i; } right(i) { return 2*i + 1; } • An aside: How would you implement this most efficiently? • Another aside: Really? David Luebke 251/4/2020 David Luebke 251/4/2020

  26. The Heap Property • Heaps also satisfy the heap property: A[Parent(i)]  A[i] for all nodes i > 1 • In other words, the value of a node is at most the value of its parent • Where is the largest element in a heap stored? • Definitions: • The height of a node in the tree = the number of edges on the longest downward path to a leaf • The height of a tree = the height of its root David Luebke 261/4/2020 David Luebke 261/4/2020

  27. Heap Height • What is the height of an n-element heap? Why? • This is nice: basic heap operations take at most time proportional to the height of the heap David Luebke 271/4/2020 David Luebke 271/4/2020

  28. Heap Operations: Heapify() • Heapify(): maintain the heap property • Given: a node i in the heap with children l and r • Given: two subtrees rooted at l and r, assumed to be heaps • Problem: The subtree rooted at imay violate the heap property (How?) • Action: let the value of the parent node “float down” so subtree at i satisfies the heap property • What do you suppose will be the basic operation between i, l, and r? David Luebke 281/4/2020 David Luebke 281/4/2020

  29. Heap Operations: Heapify() Heapify(A, i) { l = Left(i); r = Right(i); if (l <= heap_size(A) && A[l] > A[i]) largest = l; else largest = i; if (r <= heap_size(A) && A[r] > A[largest]) largest = r; if (largest != i) Swap(A, i, largest); Heapify(A, largest); } David Luebke 291/4/2020 David Luebke 291/4/2020

  30. Heapify() Example 16 4 10 14 7 9 3 2 8 1 A = 16 4 10 14 7 9 3 2 8 1 David Luebke 301/4/2020 David Luebke 301/4/2020

  31. Heapify() Example 16 4 10 14 7 9 3 2 8 1 A = 16 4 10 14 7 9 3 2 8 1 David Luebke 311/4/2020 David Luebke 311/4/2020

  32. Heapify() Example 16 4 10 14 7 9 3 2 8 1 A = 16 4 10 14 7 9 3 2 8 1 David Luebke 321/4/2020 David Luebke 321/4/2020

  33. Heapify() Example 16 14 10 4 7 9 3 2 8 1 A = 16 14 10 4 7 9 3 2 8 1 David Luebke 331/4/2020 David Luebke 331/4/2020

  34. Heapify() Example 16 14 10 4 7 9 3 2 8 1 A = 16 14 10 4 7 9 3 2 8 1 David Luebke 341/4/2020 David Luebke 341/4/2020

  35. Heapify() Example 16 14 10 4 7 9 3 2 8 1 A = 16 14 10 4 7 9 3 2 8 1 David Luebke 351/4/2020 David Luebke 351/4/2020

  36. Heapify() Example 16 14 10 8 7 9 3 2 4 1 A = 16 14 10 8 7 9 3 2 4 1 David Luebke 361/4/2020 David Luebke 361/4/2020

  37. Heapify() Example 16 14 10 8 7 9 3 2 4 1 A = 16 14 10 8 7 9 3 2 4 1 David Luebke 371/4/2020 David Luebke 371/4/2020

  38. Heapify() Example 16 14 10 8 7 9 3 2 4 1 A = 16 14 10 8 7 9 3 2 4 1 David Luebke 381/4/2020 David Luebke 381/4/2020

  39. Analyzing Heapify(): Informal • Aside from the recursive call, what is the running time of Heapify()? • How many times can Heapify() recursively call itself? • What is the worst-case running time of Heapify() on a heap of size n? David Luebke 391/4/2020 David Luebke 391/4/2020

  40. Analyzing Heapify(): Formal • Fixing up relationships between i, l, and r takes (1) time • If the heap at i has n elements, how many elements can the subtrees at l or r have? • Draw it • Answer: 2n/3 (worst case: bottom row 1/2 full) • So time taken by Heapify() is given by T(n) T(2n/3) + (1) David Luebke 401/4/2020 David Luebke 401/4/2020

  41. Analyzing Heapify(): Formal • So we have T(n) T(2n/3) + (1) • By case 2 of the Master Theorem, T(n) = O(lg n) • Thus, Heapify()takes linear time David Luebke 411/4/2020 David Luebke 411/4/2020

  42. Heap Operations: BuildHeap() • We can build a heap in a bottom-up manner by running Heapify() on successive subarrays • Fact: for array of length n, all elements in range A[n/2 + 1 .. n] are heaps (Why?) • So: • Walk backwards through the array from n/2 to 1, calling Heapify() on each node. • Order of processing guarantees that the children of node i are heaps when i is processed David Luebke 421/4/2020 David Luebke 421/4/2020

  43. BuildHeap() // given an unsorted array A, make A a heap BuildHeap(A) { heap_size(A) = length(A); for (i = length[A]/2 downto 1) Heapify(A, i); } David Luebke 431/4/2020 David Luebke 431/4/2020

  44. BuildHeap() Example • Work through exampleA = {4, 1, 3, 2, 16, 9, 10, 14, 8, 7} 4 1 3 2 16 9 10 14 8 7 David Luebke 441/4/2020 David Luebke 441/4/2020

  45. Analyzing BuildHeap() • Each call to Heapify() takes O(lg n) time • There are O(n) such calls (specifically, n/2) • Thus the running time is O(n lg n) • Is this a correct asymptotic upper bound? • Is this an asymptotically tight bound? • A tighter bound is O(n) • How can this be? Is there a flaw in the above reasoning? David Luebke 451/4/2020 David Luebke 451/4/2020

  46. Analyzing BuildHeap(): Tight • To Heapify() a subtree takes O(h) time where h is the height of the subtree • h = O(lgm), m = # nodes in subtree • The height of most subtrees is small • Fact: an n-element heap has at most n/2h+1 nodes of height h • CLR 7.3 uses this fact to prove that BuildHeap() takes O(n) time David Luebke 461/4/2020 David Luebke 461/4/2020

  47. Heapsort • Given BuildHeap(), an in-place sorting algorithm is easily constructed: • Maximum element is at A[1] • Discard by swapping with element at A[n] • Decrement heap_size[A] • A[n] now contains correct value • Restore heap property at A[1] by calling Heapify() • Repeat, always swapping A[1] for A[heap_size(A)] David Luebke 471/4/2020 David Luebke 471/4/2020

  48. Heapsort Heapsort(A) { BuildHeap(A); for (i = length(A) downto 2) { Swap(A[1], A[i]); heap_size(A) -= 1; Heapify(A, 1); } } David Luebke 481/4/2020 David Luebke 481/4/2020

  49. Analyzing Heapsort • The call to BuildHeap() takes O(n) time • Each of the n - 1 calls to Heapify() takes O(lg n) time • Thus the total time taken by HeapSort()= O(n) + (n - 1) O(lg n)= O(n) + O(n lg n)= O(n lg n) David Luebke 491/4/2020 David Luebke 491/4/2020

  50. Priority Queues • Heapsort is a nice algorithm, but in practice Quicksort (coming up) usually wins • But the heap data structure is incredibly useful for implementing priority queues • A data structure for maintaining a set S of elements, each with an associated value or key • Supports the operations Insert(), Maximum(), and ExtractMax() • What might a priority queue be useful for? David Luebke 501/4/2020 David Luebke 501/4/2020

More Related