1 / 48

Lectures on Recursive Algorithms

Lectures on Recursive Algorithms. COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski. Announcements. Web page: www.csc.liv.ac.uk/~darek/comp523.html Slides constantly updated based on your feedback – check for a new version a few hours after the lecture(s). Overview.

shae
Download Presentation

Lectures on Recursive Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lectures onRecursive Algorithms COMP 523: Advanced Algorithmic Techniques Lecturer: Dariusz Kowalski Lectures on Recursive Algorithms

  2. Announcements • Web page: www.csc.liv.ac.uk/~darek/comp523.html • Slides constantly updated based on your feedback – check for a new version a few hours after the lecture(s) Lectures on Recursive Algorithms

  3. Overview Previous lectures: • Algorithms: correctness, termination, performance • Asymptotic notation • Popular asymptotic complexities: log n, n, n log n, n2, ... • Graphs - basic definitions and examples These lectures: • Recursive algorithms - basic recursion • Searching algorithm in time O(log n) • Finding majority in time O(n) • Sorting in time O(n log n) • Other examples Lectures on Recursive Algorithms

  4. Algorithms based on recurrence Algorithmic scheme: • Reduce theinput to smaller sub-input(s) • Solve the problem for (some) sub-inputs • Merge the obtained solution(s) into one global solution Lectures on Recursive Algorithms

  5. Complexity analysis of recursive process Let T(m) denote time (or other complexity measure) of solving a given problem by a given algorithm working on input length m Simple recursive formula: T(n)  qT(n/2) + f(n) Solve the problem for half of the input q times, and spend at most f(n) time for partitioning and/or merging Lectures on Recursive Algorithms

  6. Example 1:Searching in time O(log n) Lectures on Recursive Algorithms

  7. Searching: problem and algorithm • Input: sorted array A of n numbers and number x • Problem: find if x is in array A • Solution: Algorithm SEARCH(x,1,n) • Procedure SEARCH(x,i,j): • If i = j then • if x=A[i] then answer YES • if xA[i] then answer NO • Else compare x with element A[(i+j)/2]: • If x = A[(i+j)/2] then answer YES • If x < A[(i+j)/2] then SEARCH(x, i, (i+j)/2-1) • If x > A[(i+j)/2] then SEARCH(x, (i+j)/2+1, j) Lectures on Recursive Algorithms

  8. Tree structure of searching root 4 find if value 5 is there 2 6 height = 3 1 3 5 7 1 2 3 4 5 6 7 8 8 leaves Lectures on Recursive Algorithms

  9. Time and memory consumption Each recursive call • needs additional constant memory, and • the size of input decreases by factor 2 Formula for time (# comparisons): T(n)  T(n/2) + 4 Additional memory size: log n = O(log n) In other words: recursive algorithm must store a path from the root to the searched leaf, and it has logarithmic length Lectures on Recursive Algorithms

  10. Case q = 1, f(n) = c T(n) T(n/2) + c , T(2) c T(n) T(n/2) + c (T((n/2)/2) + c)+ c = T(n/4) + 2c (T((n/4)/2) + c)+ 2c = T(n/8) + 3c … (T(n/2i) + c)+ (i-1). c = T(n/2i) + i . c …  (T(n/2log n – 1) + c) + (log n - 2). c = T(n/(n/2)) + (log n - 2). c = T(2) + (log n – 1) . c c + (log n – 1) . c= c log n Lectures on Recursive Algorithms

  11. Case q = 1, f(n) = cformal analysis T(n) T(n/2) + c T(2) c Solution:T(n) c log n Proof: by induction. • For n = 2 straightforward: T(2) c c log 2 • Suppose T(n/2) c log (n/2); then T(n) T(n/2) + c  c log (n/2) + c  c log n - c + c c log n Lectures on Recursive Algorithms

  12. Example 2:Finding a majority in time O(n) Lectures on Recursive Algorithms

  13. Finding a majority • Input: array A of n elements • Problem: find if there is an element x that occurs in array A more than n/2 times (amajority value) • Solution: Algorithm Majority(A) returning a majority element in A, if it exists, or null otherwise Lectures on Recursive Algorithms

  14. Finding a majority: algorithm Algorithm Majority(A[1..n]): If |A| = 0 then output null, else if |A| = 1 then output A[1] ; else: • If n = |A| is odd then • check whether A[n] is a majority in Aby counting the number of occurrences of value A[n]; if yes then output A[n], otherwise decrease n by one • Initialize additional array Bof size |A|/2 • Set j to 0 • For i = 1,2,…,n/2 do: • if A[2i-1] = A[2i] then • increase j by one • set B[j] to A[2i] • Find if there is a majority in B[1..j] by executing Majority(B[1..j]) • If a majority value x in B[1..j] is returned then check whether xis a majority in A, by going through array Aand counting the number of occurrences of value x inA; if successful outputx; otherwise null Lectures on Recursive Algorithms

  15. Tree structure of finding majority root 2 odd, no majority height = 2 2 2 1 2 2 2 2 1 2 1 1 8 leaves Lectures on Recursive Algorithms

  16. Correctness of Majority Algorithm Lemma: (to be proved later) If xis a majority in Athen xis a majority in B. Correctness: (proof by induction, based on the Lemma) Inductive assumption: • Majority(B) works correctly for all B’s of size smaller than |A| Case 1: There is a majority in A. • Then it is a majority in B, by Lemma, therefore it will be found in the recursive stage of the algorithm Majority(B) (by inductive argument), and will be checked (successfully) in the last point of the algorithm Majority(A). Case 2: There is no majority in A. • Then even if there is a candidate value found in the recursive stage Majority(B) of the algorithm, it will be rejected anyway in the last point of the algorithm Majority(A). Lectures on Recursive Algorithms

  17. Proof of the Lemma Suppose, to the contrary, that xis a majority in Abut is not a majority in B. Let • m be the number of occurrences of xin A, and • k be the number of occurrences of xin B. It follows that the other values appear at least k times in B. Consequently, the other values appear in A: • at least 2k times: those pairs that are represented in Bby a value different than x; plus • m-2k times: each occurrence of xin Athat is not paired by another x(there are k pairs xxin A, thus m-2k of other occurrences of x inA) is paired by some other value (different than x) which gives at least 2k+(m-2k) =m occurrences in total, and contradicts the fact that xis a majority in A. Contradiction! Therefore xmust be also a majority in B. Lectures on Recursive Algorithms

  18. Memory consumption Each call to the recursive procedure • needs additional memory of half of the current input size • reduces the size of the input by factor at least two Formula for time (# comparisons): T(n)  n -1 + 3n/2 + T(n/2) + n = T(n/2) + 7n/2 Additional memory used (for smaller arrays B): M(n)  n/2 + M(n/2)  n/2 + n/4 + M(n/4)  n/2 + n/4 + … + 1 = n - 1 = O(n) Intuition about memory size: the algorithm needs a binary tree of linear size, in the worst case, to store “winners” Lectures on Recursive Algorithms

  19. Case q = 1, f(n) = cn T(n) T(n/2) + cn , T(1) c T(n) T(n/2) + cn  (T(n/4) + cn/2) + cn = T(n/4) + (3/2)cn  (T(n/8) + cn/4) + (3/2)cn = T(n/8) + (7/4)cn …  (T(n/2i) + cn/2i-1) + ((2i-1-1)/2i-2). cn = T(n/2i) + ((2i-1)/2i-1). cn … T(n/2log n) + ((2log n-1)/2log n-1). cn = T(1) + (n – 1)/(n/2) . cn c + 2cn – 2c  2cn Lectures on Recursive Algorithms

  20. Case q = 1, f(n) = c nformal analysis T(n) T(n/2) + c n T(1) c Solution:T(n)  2cn Proof: by induction. • For n = 1 straightforward: T(1) c 2c  1 • Suppose T(n/2)  2c(n/2); then T(n) T(n/2) + cn  2c(n/2) + c n = 2cn Lectures on Recursive Algorithms

  21. Textbook and exercises READING: Chapter 5, up to Section 5.2 EXERCISES: • Solve the following recursive formulas the best you can: • T(n)  T(n/2) + 4 • T(n)  T(n/2) + 5n • T(n)  T(n/2) + 3n2 • T(n)  (3/2)T(n/2) + 1 Compare and sort the obtained (asymptotic) formulas according to the asymptotic order. For each of them try to find an algorithmic example with performance giving by this formula. • Sort the following formulas according to big/small Oh order: log (n1/2), log (9n), log (n3), 2log n, 23log n, 2log (9n), n2, n log n • Propose and analyse an algorithm for checking if there is a majority in a given sorted array A. Lectures on Recursive Algorithms

  22. Example 3:Sorting in time O(n log n) Lectures on Recursive Algorithms

  23. Divide and conquer Algorithm: • Partition input into two halves in (at most) linear time • Solve the problem for each half of the input separately • Merge solutions into one in (at most) linear time Recursive formula on time complexity: T(n)  2T(n/2) + cn Lectures on Recursive Algorithms

  24. Case q = 2, f(n) = cn T(n)  2T(n/2) + cn , T(2) c T(n)  2T(n/2) + cn  2(2T(n/4) + cn/2) + cn = 4T(n/4) + 2cn  4(2T(n/8) + cn/4) + 2cn = 8T(n/8) + 3cn …  2i-1 (2T(n/2i) + cn/2i-1) + (i-1). cn = 2i T(n/2i) + i . cn …  2log n - 1 T(2) + (log n - 1) . cn cn log n Lectures on Recursive Algorithms

  25. Solution for timeformal analysis T(n)  2 T(n/2) + cn T(2) c Solution: T(n) cnlog n Proof: by induction. • For n = 2 straightforward: T(2) c c2log 2 • Suppose T(n/2) c(n/2)log (n/2); then T(n)  2T(n/2) + cn  2c(n/2)log (n/2) + cn = 2c(n/2)log n –2cn/2 + cn= cnlog n Lectures on Recursive Algorithms

  26. Example - sorting by merging • Input: list L of n numbers • Problem: Sort list L • Algorithm MergeSort(L): If L has at most two elements then sort them by comparison and exit; Else: • Split L into two lists prefix and suffix, each of size n/2 • Sort, by merging, the prefix and the suffix separately: MergeSort(prefix) and MergeSort(suffix) • Merge sorted prefix with sorted suffix as follows: • Initialize final list as empty • Repeat until either prefix or suffix is empty: • Compare the first elements on the lists prefix and suffix and then move the smaller one to the end of the final list • Concatenate final list with the remaining non-empty list (prefix or suffix) Lectures on Recursive Algorithms

  27. MergeSort procedure 1 2 4 7 8 3 5 6 9 10 1 2 3 4 5 6 7 8 9 10 Lectures on Recursive Algorithms

  28. Tree structure of sorting root merging operations height = 3 2 4 1 7 8 3 6 5 8 leaves Lectures on Recursive Algorithms

  29. Time and memory consumption Each recursive call • needs additional constant memory, and • the number of inputs increases by factor 2, and • the size of each input decreases by factor 2 Formula for time (# comparisons and list operations): T(n)  n/2 + 2T(n/2) + 2n = 2T(n/2) + 5n/2 Additional memory size: M(n)  2 M(n/2) + 2 M(n)  2 M(n/2) + 2 4 M(n/4) + 4 + 2  8M(n/8) + 8 + 4 + 2  … n/2 +n/4 + … + 4 + 2 = n - 2 = O(n) In other words: divide and conquer algorithm must store the binary tree of pointers during the computation Lectures on Recursive Algorithms

  30. Quick sort - algorithmic scheme Generic Quick Sort: • Select one element x from the input • Partition the input into part containing elements not greater than x and part containing elements bigger than x • Sort each part separately • Concatenate these two sorted parts Problem: how to choose element x to balance the sizes of these two parts? (to get the similar recursive equations as for MergeSort) Lectures on Recursive Algorithms

  31. Why parts should be balanced? Suppose they are not balanced, but, say, the smallest element is chosen: T(n) T(1) + T(n-1) + c n T(1) c Solution: T(n)  (c/2) . n2 Proof: by induction. • For n = 1 straightforward: T(1) c  (c/2) . 12 • Suppose T(n-1)  (c/2) . (n-1)2; then T(n)  T(n-1) + c + cn  (c/2) . (n-1)2 + c (n+1)  (c/2) . (n-1)2+ (c/2) . 2n (c/2) .n2 Lectures on Recursive Algorithms

  32. Two approaches to be fast Deterministic: • Additional tree structure of size O(n) needed for identifying the median value • Time: O(n log n), additional memory O(n) Randomized: • Select separation element x uniformly at random • Time (expected): O(n log n), additional memory O(n) Lectures on Recursive Algorithms

  33. Randomized approach - analysis Let T(n) denote the expected time: sum of all possible values of time weighted by the probabilities of these values T(n)  1/n([T(n-1)+T(1)] + [T(n-2)+T(2)] + … +[T(0)+T(n)]) + cn T(0) = T(1) = 1, T(2) c Solution: T(n) d n log n, for some constant d 8c Proof: by induction. • For n = 2 straightforward: T(2) c d .2 . log 2 • Suppose T(m) d m log m,for every m<n; then (1-1/n)T(n)  (2/n)(T(0) + … + T(n-1)) + c n  (2d/n)(1 + 1 + 2log 2 + … + (n-1)log(n-1)) + c n  (2d/n)((n2/2) log n – n2/8) + c n  d n log n - d (n/4) + c n  d n log n - d n/2 T(n)  (n/(n-1))(d n log n - d n/2)  d n log n Lectures on Recursive Algorithms

  34. Tree structure of random execution root 1 6 5 7 height = 5 3 2 4 1 2 3 4 5 6 7 8 8 leaves Lectures on Recursive Algorithms

  35. Textbook and Exercises READING: Section 5.3 EXERCISES: • Solved exercises 1, 2 from the textbook, chapter 5 “Divide and Conquer” • Prove that 1 + 1 + 2 log 2 + … + (n-1)log(n-1)  n(n/2) log(4n) , and (n/(n-1))(d n log n - d n/2)  d n log n , for n > 16 Lectures on Recursive Algorithms

  36. Example 4:Finding closest pair in time O(n log n) Lectures on Recursive Algorithms

  37. Problem Closest pair of points: • Input: n points on the plane • Output: two points having the closest distance Remarks: • Exhaustive search algorithm gives time O(n2) • Similar approach gives ALL pairs of closest points Lectures on Recursive Algorithms

  38. Solution Preprocessing: • Sort points according to the first coordinate (obtain horizontal list H) and according to the second coordinate (obtain vertical list V) Main Algorithm: • Partition input list H into halves (H1,H2) in linear time (draw vertical line L containing medium point); split list V accordingly into V1,V2, where Vi contains the same elements as Hi and inherits the initial order from V (for i = 1,2) • Solve the problem for each half of the input separately, obtaining two pairs of closest points; let  be the smallest distance from the obtained ones • Find if there is a pair which has distance smaller than  - if yes then find the smallest distance pair Lectures on Recursive Algorithms

  39. Main difficulty Find if there is a pair which has distance smaller than : if yes then find the smallest distance pair How to do it in linear time?    L Lectures on Recursive Algorithms

  40. Closest pair in the strip • Find a sub-list P of V containing all points with a distance at most  from the line L: • go through V and remove elements not –close to L 2. For each element in P check its distance to the next 8 elements and remember the closest pair ever found    P Lectures on Recursive Algorithms L

  41. Why to check only 8 points ahead? 1. Partition the strip into squares of length /2 each,as shown in the picture 2. Each square contains at most 1 point - by definition of  3. If there is at least 2 full squares between points then they can not be the closest points 4. There are at most 8 squares to check  /2 /2 /2 /2 /2 /2 Lectures on Recursive Algorithms L

  42. Time complexity Preprocessing: sorting in time O(n log n) Main Algorithm: recursion in time O(n log n) T(n)  3n/2 + 2T(n/2) + 8n = 2T(n/2) + 9.5n T(2) = 1 Lectures on Recursive Algorithms

  43. Example 5:Integer multiplication in time O(n1.59) Lectures on Recursive Algorithms

  44. Beyond (n log n) : integer’s multiplication • Input: two integers x and y, each consisting of at most n bits • Output: multiply these integers • Naïve Algorithm: • Multiply each bit i of x by y, add (i-1) zeroes at the end • Add the obtained at most n numbers • Time: (n2) of bit operations Lectures on Recursive Algorithms

  45. Divide and Conquer approach Let x = x1 + 2n/2x2 and y = y1 + 2n/2y2 Let p = x1 + x2 and q = y1 + y2. Then x y = (x1 + 2n/2x2) (y1 + 2n/2y2) = x1 y1 + 2n/2 (x2 y1+x1 y2) + 2nx2 y2 = x1 y1 + 2n/2 (pq - x1 y1-x2 y2) + 2nx2 y2 Algorithm: • Compute p and q (in linear time) • Compute recursively: x1 y1, x2 y2 , pq • Performx1 y1 + 2n/2 (pq - x1 y1-x2 y2) + 2nx2y2 in linear time Lectures on Recursive Algorithms

  46. Time complexity T(n)  3T(n/2) + c n T(1) c Solution: T(n) dnlog 3, for some constant d 3c Proof: For n = 1 straightforward: T(1) cdnlog 3 General case: T(n)  3T(n/2) + c n  9T(n/4) + 3c(n/2) + c n  … c n (1 + 3/2 + (3/2)2 + … + (3/2)log n) • c n 2(3/2)log n + 1  3c n nlog 3 - 1 = 3c nlog 3 d nlog 3 = O(n1.59) Lectures on Recursive Algorithms

  47. Textbook and Exercises READING: Chapter 5 from Section 5.4 OBLIGATORY EXERCISES: • How to add two n-bit integers in linear number of bit operations? ADDITIONAL EXERCISES: • Solved Exercise 1 from the textbook, chapter 5 “Divide and Conquer” • Exercises 1, 6, 7 from the textbook, chapter 5 “Divide and Conquer” Lectures on Recursive Algorithms

  48. Conclusions • Searching in time O(log n) • Finding majority value in time O(n) • Divide and conquer in time O(n log n) • sorting algorithms (MergeSort, RandQuickSort) • closest points algorithm • Beyond (n log n) : O(nlog 3) time algorithm for integer multiplication Lectures on Recursive Algorithms

More Related