1 / 22

Recurrences

Recurrences. Recurrences. Coming up Merge Sort (Chap 2.3) Recurrences (Chap 4.1-4.3). Merge Sort. Merge Sort : divide-and-conquer approach Insertion Sort : Incremental approach The Divide-and-Conquer Paradigm : Divide : Divide the problem into a number of subproblems.

williambjones
Download Presentation

Recurrences

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Recurrences

  2. Recurrences Coming up • Merge Sort (Chap 2.3) • Recurrences (Chap 4.1-4.3)

  3. Merge Sort Merge Sort : divide-and-conquerapproach Insertion Sort : Incrementalapproach The Divide-and-Conquer Paradigm: Divide: Divide the problem into a number of subproblems. Conquer: If the subproblem sizes are small enough, solve them directly. Otherwise solve them recursively. Combine: Combine the solutions to the subproblems into the solution for the original problem.

  4. Merge Sort • Suppose there are some people called Mr. MergeSort. They are identical. • They don’t know how to do sorting. • But each of them has a secretary called Mr. Merge, who can merge 2 sorted sequences into one sorted sequence.

  5. 5 2 4 7 1 3 2 6 5 2 4 7 1 3 2 6 1 All of them say ‘This is easy. No need to do anything.’ 5 2 4 7 3 2 6 Merge Sort “So complicated!!, I’ll split them and call other Mr. MergeSorts to handle.” At the beginning, a Mr. MergeSort is called to sort: 5 2 4 7 1 3 2 6 Then 2 other Mr. MergeSorts are called to sort: Both of them say “Still complicated! I’ll split them and call other Mr. MergeSorts to handle.” Then 4 other Mr. MergeSorts are called to sort: All of them say “Still complicated! I’ll split them and call other Mr. MergeSorts to handle.” Then 8 other Mr. MergeSorts are called to sort:

  6. 5 2 4 7 1 3 2 6 5 2 4 7 2 4 5 7 1 3 2 6 1 2 3 6 5 2 4 7 1 3 2 6 2 5 4 7 1 3 2 6 1 All of them say ‘This is easy. No need do anything.’ 1 5 2 4 7 3 2 6 5 2 4 7 3 2 6 Merge Sort Then the first Mr. MergeSort succeeds and returns. The first Mr. MergeSort calls his secretary Mr. Merge to merge the returned numbers 1 2 2 3 4 5 6 7 Then each of the 2 Mr. MergeSorts returns the merged numbers. Both Mr. MergeSorts call their secretaries Mr. Merge to merge the returned numbers The 4 Mr. MergeSorts call their secretaries Mr. Merge to merge the returned numbers Then the 4 Mr. MergeSorts returns the merged numbers. Then the 8 Mr. MergeSorts return.

  7. The MERGE-SORT(A,p,r) procedure sorts the elements A[p,..r]: A = .. 5 2 4 7 1 3 2 6 .. pth rth Merge Sort MERGE-SORT (A,p,r) 1 if p < r 2 Then q  (p+r)/2 3 MERGE-SORT(A,p,q) 4 MERGE-SORT(A,q+1,r) 5 MERGE (A,p,q,r) MERGE-SORT (A,p,r) 1 if p < r 2 Then q (p+r)/2 3 MERGE-SORT(A,p,q) 4 MERGE-SORT(A,q+1,r) 5 MERGE (A,p,q,r) x: “Floor” The least integer greater than x x: “Ceiling” The greatest integer less than x

  8. L= R= 1 2 3 6  2 4 5 7  L= R= A = 3 6  4 5 7  .. 2 4 5 7 1 2 3 6 .. A = .. 2 4 5 7 1 2 3 6 .. A = .. 12 2 3 4 5 6 7 .. pth qth rth pth rth pth rth L= R= L= R= 2 4 5 7 1 2 3 6 2 3 6  2 4 5 7  L= R= 6  4 5 7  A = .. 1 4 5 7 1 2 3 6 .. A = .. 12 23 4 5 6 7 .. pth rth L= R= 1 2 3 6  2 4 5 7  pth rth L= R= 2 3 6  4 5 7  L= L= L= L= R= R= R= R=  6   6  7  5 7  7   A = .. 12 2 3 4 5 6 7 .. A = A = A = A = .. 12 234 56 7 .. .. 12 234 56 7 .. .. 12 234 5 6 7 .. .. 12 234 5 6 7 .. pth rth pth pth pth pth rth rth rth rth Merge Sort The MERGE(A,p,q,r) procedure merges 2 sorted sequences: A[p..q] and A[q+1..r] Step 1: Copy A[p..q], A[q+1..r] to 2 temporary arrays L and R. Step 2: Adds a largest value,  (a sentinel: ending condition), to each of L and R. Step 3: Continuously remove the smallest one from L and R back to A until finished.

  9. Merge Sort MERGE (A,p,q,r) 1 n1  q-p+1 2 n2  r-q 3 create L[1..n1+1], R[1..n2+1] 4 for i  1 to n1 5 do L[i]  A[p+i-1] 6 for j  1 to n2 7 do R[j]  A[q+j] 8 L[n1+1]   9 R[n2+1]   10 i  1 11 j  1 12 for k  p to r 13 do if L[i]  R[j] 14 then A[k]  L[i] 15 i  i+1 16 else 17 j  j+1 • The MERGE(A,p,q,r) procedure merges 2 sorted sequences: A[p..q] and A[q+1..r] Step 1: Copy A[p..q], A[q+1..r] to 2 temporary arrays L and R. Step 2: Adds a largest value,  (a sentinel: ending condition), to each of L and R. Step 3: Continuously remove the smallest one from L and R back to A until finished.

  10. Analysis of MERGE Procedure MERGE (A,p,q,r) 1 n1  q-p+1 2 n2  r-q 3 create L[1..n1+1], R[1..n2+1] 4 for i  1 to n1 5 do L[i]  A[p+i-1] 6 for j  1 to n2 7 do R[j]  A[q+j] 8 L[n1+1]   9 R[n2+1]   10 i  1 11 j  1 12 for k  p to r 13 do if L[i]  R[j] 14 then A[k]  L[i] 15 i  i+1 16 else 17 j  j+1 • Let n=n1+ n2 • Line 1-3 and 8-11 takes constant time. So, (1). • Line 4-5 and Line 6-7 takes (n1+n2) = (n) time. • In line 12-17, the loop iterates n times, each of which takes constant time. So, (n). • Conclusion: The MERGE procedure runs in (1) + (n) + (n) = (n) time.

  11. Or simplified: (1) if n=1 c if n=1 T(n) = T(n) = Tn/2 + Tn/2 + (n) if n>1 2T(n/2)+cn if n>1 Analysis of Merge Sort Then, what is the complexity of Merge Sort? To sort A[1..n] using Merge Sort, we call MERGE-SORT(A,1,n) MERGE-SORT has a recursive call to itself, plus a call to MERGE. The Running time: MERGE-SORT (A,p,r) 1 if p < r 2 Then q  (p+r)/2 3 MERGE-SORT(A,p,q) 4 MERGE-SORT(A,q+1,r) 5 MERGE (A,p,q,r) Recurrence Equation

  12. c if n=1 T(n) = 2T(n/2)+cn if n>1 Recursion-tree method Can directly prove or help to guess MI Substitution method guess a bound + prove by mathematical induction Master method Based on the Master Theorem Solving Recurrences => T(n) = (?) When an algorithm contains recursive call(s) to itself, (eg. Divide-and-conquer approaches), its running time can often be described by a recurrence equation. 3 methods to solve recurrences:

  13. c if n=1 T(n) = 2T(n/2)+cn if n>1 cn cn T(n/2) T(n/2) cn/2 cn/2 T(n/4) T(n/4) T(n/4) T(n/4) Solving Recurrence (Recursion-tree method) Recursion-tree method Expanding the recursion tree: T(n)

  14. cn cn cn/2 cn cn cn/2 cn/4 cn/4 cn/4 cn/4 c*1 c*1 c*1 c*1 c*1 c*1 c*1 c*1 n Solving Recurrence (Recursion-tree method) Fully Expanded recursion tree: Total: cn lg n + cn ie. T(n) = (n lg n)

  15. 1 if n=1 T(n) = 2T(n/2)+n if n>1 MI Solving Recurrence (Substitution method) Substitution method Step 1. Guess the form of the solution Example. Given: We guess that T(n)=O(n lg n) To prove this we need to show “There exists positive constants cand n0 such that ...T(n)  cn lg n, for all n  n0” Step 2

  16. T(n)=O(n lg n) “There exists a positive constants cand n0 such that T(n)  cn lg n, for all n  n0” 1 if n=1 T(n) = 2T(n/2)+n if n>1 MI Solving Recurrence (Substitution method) Step 2. Use mathematical induction to find the constants and prove. Assume it holds for n/2, ie. There exists positive constants c and n0 such that T(n/2)  c n/2 lg (n/2). By substitution: T(n)  2 (c n/2 lg (n/2)) + n  cn lg (n/2) + n = cn (lg(n)-lg(2))+n = cn (lg(n)-1)+n = cn lg n -cn + n  cn lg n This holds as long as c  1

  17. T(n)=O(n lg n) “There exists a positive constants cand n0 such that T(n)  cn lg n, for all n  n0” 1 if n=1 T(n) = 2T(n/2)+n if n>1 MI Solving Recurrence (Substitution method) Step 2 (cont’d). Then we need to find n0. Try n0=1 When n=1, T(n)= 1 cn lg n = (c*1) lg (1) = c * 0 = 0 => T(n)  cn lg n ? Try n0=2 When n=2, T(n) = 2T(1) + 2 = 2(1) + 2 = 4 cn lg n = (c*2) lg (2) = c*2*1 = 2c => T(n)  cn lg n ? Try n0=3 When n=3, T(n) = 2T(1) + 3 = 2(1) + 3 = 5 cn lg n = (c*3) lg (3) = 3*1.585*c = 4.755*c => T(n)  cn lg n ? Since n0=2 and n0=3 form the base cases for all n>=2, we thus completed the prove: T(n)  cn lg n is true when c>=2 and n>=2.

  18. Solving Recurrence (Master method) Master method • A cookbook method • For solving recurrences of the form: T(n) = a T(n/b) + f(n) Where a>= 1 and b > 1 and f(n) is an asymptotically positive function • These algorithms work recursively by dividing a problem of size n into a subproblems, each of size n/b. • f(n) = cost of dividing the problem and combining the results. • We omit floors and ceilings (eg. interpret n/b to mean either n/b or n/b.)

  19. Solving Recurrence (Master method) Master Theorem Let T(n) be defined as T(n) = a T(n/b) + f(n) where a >=1 and b > 1. Then If f(n) = O(nlogba-) for some constant >0, then T(n) = (nlogba) If f(n) = (nlogba), then T(n) = (nlogbalg n) If f(n) = (nlogba+)for some constant >0, and if a f (n/b) <= c f(n) for some constant c <1 and all sufficiently large n, then T(n) = (f(n)) The master method can be used only if f(n) satisfies any of the 3 cases.

  20. Solving Recurrence (Master method) Master Theorem For T(n) = a T(n/b) + f(n) where a >=1 and b > 1. If f(n) = O(nlogba-) for some constant >0, then T(n) = (nlogba) If f(n) = (nlogba), then T(n) = (nlogbalg n) If f(n) = (nlogba+)for some constant >0, .. then T(n) = (f(n)) T(n) = 9T(n/3) + n a = 9, b = 3 => nlogba =nlog39 = n2 Since f(n) = n = n(2-1) = O(nlog39 -) Where =1 => Case 1 => T(n) = (nlogba) => T(n) = (n2) T(n) = T(2n/3) + 1 a = 1, b = 3/2 => nlogba = nlog3/21 = n0 = 1 Since f(n) = 1 = (nlogba) => Case 2 => T(n) = (nlogbalg n) => T(n) = (lg n) Example 1: Example 2:

  21. Solving Recurrence (Master method) Master Theorem For T(n) = a T(n/b) + f(n) where a >=1 and b > 1. If f(n) = O(nlogba-) for some constant >0, then T(n) = (nlogba) If f(n) = (nlogba), then T(n) = (nlogbalg n) If f(n) = (nlogba+)for some constant >0, and if a f (n/b) <= c f(n) for some constant c <1 and all sufficiently large n, then T(n) = (f(n)) T(n) = 3T(n/4) + n lg n a=3, b=4 => nlogba = nlog43 = n0.793 Since f(n) = n lg n When n  2, f(n)  n lg 2 = n = n1 = n0.793+, where =0.201 => f(n) = (nlogba+) a f(n/b) = a (n/b * lg(n/b)) = 3 (n/4 * lg (n/4)) = 3/4 n lg(n/4) = 3/4 (n lg(n)) - 3/4 (n lg(4))  3/4 f(n) => a f(n/b)  c f(n) where c=3/4. => Case 3. ie. T(n) = (f(n)) = (n lg n) Example 3:

  22. Recurrences Summary • The Divide-and-Conquer Paradigm • Merge Sort (MERGE-SORT Procedure + MERGE Procedure) • Analysis of MERGE: (n) • Analysis of MERGE-SORT: A recurrence problem (n lg n)) 3 Methods to solve recurrences: • Recursion-tree method • Substitution method • Master method

More Related