1 / 61

Divide and Conquer Algorithms with Recurrence Equations

Understand and analyze divide-and-conquer algorithms with recurrence equations for efficient problem-solving. Learn algorithms design concepts.

rosab
Download Presentation

Divide and Conquer Algorithms with Recurrence Equations

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 11Sorting Acknowledgement: These slides are adapted from slides provided with Data Structures and Algorithms in C++, Goodrich, Tamassia and Mount (Wiley 2004) and slides from Jory Denny and Mukulika Ghosh

  2. Divide and Conquer Algorithmsanalysis with recurrence equations • Divide-and conquer is a general algorithm design paradigm: • Divide: divide the input data into (disjoint) subsets • Recur: solve the sub-problems recursively • Conquer: combine the solutions for into a solution for • The base case for the recursion are sub-problems of constant size • Analysis can be done using recurrence equations (relations)

  3. Divide and Conquer Algorithmsanalysis with recurrence equations • When the size of all sub-problems is the same (frequently the case) the recurrence equation representing the algorithm is: • Where • is the cost of dividing into the sub-problems • There are sub-problems, each of size that will be solved recursively • is the cost of combining the sub-problem solutions to get the solution for

  4. ExerciseRecurrence Equation Setup • Algorithm – transform multiplication of two -bit integers and into multiplication of -bit integers and some additions/shifts • Where does recursion happen in this algorithm? • Rewrite the step(s) of the algorithm to show this clearly. Algorithm Input: -bit integers Output: • if • Split and into high and low order halves: • ; ; ; • else • return

  5. ExerciseRecurrence Equation Setup • Algorithm – transform multiplication of two -bit integers and into multiplication of -bit integers and some additions/shifts • Assuming that additions and shifts of -bit numbers can be done in time, describe a recurrence equation showing the running time of this multiplication algorithm Algorithm Input: -bit integers Output: • if • Split and into high and low order halves: • ; ;; • else • return

  6. ExerciseRecurrence Equation Setup • Algorithm – transform multiplication of two -bit integers and into multiplication of -bit integers and some additions/shifts • The recurrence equation for this algorithm is: • The solution is which is the same as naïve algorithm Algorithm Input: -bit integers Output: • if • Split and into high and low order halves: • ; ;; • else • return

  7. 7 2  9 4  2 4 7 9 7  2  2 7 9  4  4 9 7  7 2  2 9  9 4  4 Merge Sort

  8. Merge-Sort • Merge-sort is based on the divide-and-conquer paradigm. It consists of three steps: • Divide: partition input sequence into two sequences and of about elements each • Recur: recursively sort and • Conquer: merge and into a sorted sequence Algorithm Input: Sequence of elements, Comparator Output: Sequence sorted according to • if • return

  9. Merge-Sort Execution Tree (recursive calls) • An execution of merge-sort is depicted by a binary tree • Each node represents a recursive call of merge-sort and stores • Unsorted sequence before the execution and its partition • Sorted sequence at the end of the execution • The root is the initial call • The leaves are calls on subsequences of size 0 or 1 7 2  9 4  2 4 7 9 7  2  2 7 9  4  4 9 7  7 2  2 9  9 4  4

  10. 7 2 9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 2  2 7 9 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Partition 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

  11. 7 2 | 9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 2  2 7 9 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Recursive Call, partition 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

  12. 7 2 | 9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 | 2 2 7 9 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Recursive Call, partition 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

  13. 7 2 | 9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 | 2 2 7 9 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Recursive Call, base case 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

  14. 7 2 | 9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 | 2 2 7 9 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Recursive Call, base case 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

  15. 7 2 | 9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 | 2 2 7 9 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Merge 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

  16. 7 2 | 9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 | 2 2 7 9 | 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Recursive call, …, base case, merge 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

  17. 7 2 | 9 4  2 4 7 9 3 8 6 1  1 3 8 6 7 | 2 2 7 9 | 4  4 9 3 8  3 8 6 1  1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Merge 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

  18. 7 2 | 9 4  2 4 7 9 3 8 | 6 1  1 3 8 6 7 | 2 2 7 9 | 4  4 9 3 | 8 3 8 6 | 1 1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Recursive call, …, merge, merge 7 2 9 4  3 8 6 1  1 2 3 4 6 7 8 9

  19. 7 2 | 9 4  2 4 7 9 3 8 | 6 1  1 3 8 6 7 | 2 2 7 9 | 4  4 9 3 | 8 3 8 6 | 1 1 6 7  7 2  2 9  9 4  4 3  3 8  8 6  6 1  1 Execution Example • Merge 7 2 9 4  3 8 6 1 1 2 3 4 6 7 8 9

  20. Analysis Using Recurrence relation • The running time of Merge Sort can be expressed by the recurrence equation: • We need to determine , the time to merge two sorted sequences each of size . Algorithm Input: Sequence of elements, Comparator Output: Sequence sorted according to • if • return

  21. Merging Two Sorted Sequences • The conquer step of merge-sort consists of merging two sorted sequences and into a sorted sequence containing the union of the elements of and • Merging two sorted sequences, each with elements and implemented by means of a doubly linked list, takes time Algorithm Input: Sequences with elements each Output: Sorted sequence of • while • if • ; • else • ; • while • ; • while • ; • return

  22. And the complexity of mergesort… Algorithm Input: Sequence of elements, Comparator Output: Sequence sorted according to • if • return • So, the running time of Merge Sort can be expressed by the recurrence equation:

  23. Another Analysis of Merge-Sort • The height of the merge-sort tree is • at each recursive call we divide in half the sequence, • The work done at each level is • At level , we partition and merge sequences of size • Thus, the total running time of merge-sort is

  24. Summary of Sorting Algorithms (so far)

  25. 7 4 9 6 2  2 4 6 7 9 4 2  2 4 7 9 7 9 2  2 9  9 Quick-Sort

  26. Quick-Sort x • Quick-sort is a randomized sorting algorithm based on the divide-and-conquer paradigm: • Divide: pick a random element (called pivot) and partition into • - elements less than • - elements equal • - elements greater than • Recur: sort and • Conquer: join , , and x L G E x

  27. Partition • We partition an input sequence as follows: • We remove, in turn, each element from and • We insert into , , or , depending on the result of the comparison with the pivot Algorithm Input: Sequence , position of the pivot Output: Subsequences of the elements of less than, equal to, or greater than the pivot, respectively • while • if • else if • else // • return

  28. Quick-Sort Tree • An execution of quick-sort is depicted by a binary tree • Each node represents a recursive call of quick-sort and stores • Unsorted sequence before the execution and its pivot • Sorted sequence at the end of the execution • The root is the initial call • The leaves are calls on subsequences of size 0 or 1 7 4 9 6 2  2 4 6 7 9 4 2  2 4 7 9 7 9 2  2 9  9

  29. Execution Example • Pivot selection 7 2 9 4 3 7 6 1  1 2 3 4 6 7 7 9 2 4 3 1  1 2 3 4 7 9 77 7 9 4 33 4 9 9 1  1 4  4

  30. Execution Example • Partition, recursive call, pivot selection 7 2 9 4 3 7 6 1  1 2 3 4 6 7 7 9 2 4 3 1  1 2 3 4 7 9 77 7 9 4 33 4 9 9 1  1 4  4

  31. Execution Example • Partition, recursive call, base case 7 2 9 4 3 7 6 1  1 2 3 4 6 7 7 9 2 4 3 1  1 2 3 4 7 9 77 7 9 4 33 4 9 9 1  1 4  4

  32. Execution Example • Recursive call, …, base case, join 7 2 9 4 3 7 6 1  1 2 3 4 6 7 7 9 2 4 3 1  1 2 3 4 7 9 77 7 9 4 33 4 9 9 1  1 4  4

  33. Execution Example • Recursive call, pivot selection 7 2 9 4 3 7 6 1  1 2 3 4 6 7 7 9 2 4 3 1  1 2 3 4 7 9 77 7 9 4 33 4 9 9 1  1 4  4

  34. Execution Example • Partition, …, recursive call, base case 7 2 9 4 3 7 6 1  1 2 3 4 6 7 7 9 2 4 3 1  1 2 3 4 7 9 77 7 9 4 33 4 9 9 1  1 4  4

  35. Execution Example • Join, join 7 2 9 4 3 7 6 1  1 2 3 4 6 7 7 9 2 4 3 1  1 2 3 4 7 9 77 7 9 4 33 4 9 9 1  1 4  4

  36. In-Place Quick-Sort • Quick-sort can be implemented to run in-place • In the partition step, we use replace operations to rearrange the elements of the input sequence such that • the elements less than the pivot have indices less than • the elements equal to the pivot have indices between and • the elements greater than the pivot have indices greater than • The recursive calls consider • elements with indices less than • elements with indices greater than Algorithm Input: Array , indices Output: Array with the elements between and sorted • if • return • //random integer between and • return

  37. In-Place Partitioning • Perform the partition using two indices to split into and (a similar method can split into and ). • Repeat until and cross: • Scan to the right until finding an element . • Scan to the left until finding an element . • Swap elements at indices and j k 3 2 5 1 0 7 3 5 9 2 7 9 8 9 7 6 9 (pivot = 6) 3 2 5 1 0 7 3 5 9 2 7 9 8 9 7 6 9

  38. Analysis of Quick Sort using Recurrence Relations • Assumption: random pivot expected to give equal sized sub-lists • The running time of Quick Sort can be expressed as: • - time to run on input of size Algorithm Input: Sequence , indices Output: Sequence with the elements between and sorted • if • return • //random integer between and • return

  39. Analysis of Partition • Each insertion and removal is at the beginning or at the end of a sequence, and hence takes time • Thus, the partition step of quick-sort takes time Algorithm Input: Sequence , position of the pivot Output: Subsequences of the elements of less than, equal to, or greater than the pivot, respectively • while • if • else if • else // • return

  40. So, the expected complexity of Quick Sort • Assumption: random pivot expected to give equal sized sub-lists • The running time of Quick Sort can be expressed as: Algorithm Input: Sequence , indices Output: Sequence with the elements between and sorted • if • return • //random integer between and • return

  41. Worst-case Running Time • The worst case for quick-sort occurs when the pivot is the unique minimum or maximum element • One of and has size and the other has size • The running time is proportional to: • Alternatively, using recurrence equations: …

  42. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Expected Running Timeremoving equal split assumption • Consider a recursive call of quick-sort on a sequence of size • Good call: the sizes of and are each less than • Bad call: one of and has size greater than • A call is good with probability 1/2 • 1/2 of the possible pivots cause good calls: 7 2 9 4 3 7 6 1 9 7 2 9 4 3 7 6 1 1 7 2 9 4 3 7 6 2 4 3 1 7 9 7 6 Bad call Good call Bad pivots Bad pivots Good pivots

  43. Expected Running Time • Probabilistic Fact: The expected number of coin tosses required in order to get heads is (e.g., it is expected to take 2 tosses to get heads) • For a node of depth , we expect • ancestors are good calls • The size of the input sequence for the current call is at most • Therefore, we have • For a node of depth , the expected input size is one • The expected height of the quick-sort tree is • The amount or work done at the nodes of the same depth is • Thus, the expected running time of quick-sort is

  44. Summary of Sorting Algorithms (so far)

  45. Sorting Lower Bound

  46. Comparison-Based Sorting • Many sorting algorithms are comparison based. • They sort by making comparisons between pairs of objects • Examples: bubble-sort, selection-sort, insertion-sort, heap-sort, merge-sort, quick-sort, ... • Let us therefore derive a lower bound on the running time of any algorithm that uses comparisons to sort elements, . Is ? no yes

  47. Counting Comparisons • Let us just count comparisons then. • Each possible run of the algorithm corresponds to a root-to-leaf path in a decision tree

  48. Decision Tree Height • The height of the decision tree is a lower bound on the running time • Every input permutation must lead to a separate leaf output • If not, some input …4…5… would have same output ordering as …5…4…, which would be wrong • Since there are leaves, the height is at least

  49. The Lower Bound • Any comparison-based sorting algorithm takes at least time • That is, any comparison-based sorting algorithm must run in )time.

  50. Bucket-Sort and Radix-Sort 1, c 3, a 3, b 7, d 7, g 7, e        0 1 2 3 4 5 6 7 8 9 Can we sort in linear time? B

More Related