한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님

1. Recurrences 한양대학교 정보보호 및 알고리즘 연구실 2008. 2. 12 이재준 담당교수님 : 박희진 교수님

2. Contents of Table A. Growth of function Recurrence solution methods The substitution method The recursion-tree method Relationship between asymptotic notations B. Recurrence

3. 3. Relationship between this notations • Analogy • f(n) = Θ(g(n)) ≈ f(n)= g(n). • f(n) = O(g(n)) ≈ f(n)≤ g(n). • f(n) = Ω(g(n)) ≈ f(n)≥ g(n). • f(n) = o(g(n)) ≈ f(n)<g(n). • f(n) = ω(g(n)) ≈ f(n)>g(n).

4. 3. Relationship between this notations • Transitivity • Transpose symmetry • f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) , • f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , • f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) , • f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) , • f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)). • f(n) = O(g(n)) if and only if g(n) = Ω(f(n)), • f(n) = o(g(n)) if and only if g(n) = ω(f(n)).

5. 3. Relationship between this notations • Reflexivity • f(n) = Θ(f(n)) • f(n) = O(f(n)) • f(n) = Ω(f(n)) • Symmetry • f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).

6. Recurrences • Recurrence When should we use the recurrence? When algorithm contains a recursive call to itself, Its running time can often be described by a recurrence.

7. Recurrences • Recurrence - Definition Equation or inequality that describes a function in terms of its value on smaller inputs.

8. Recurrences • RecurrenceExample Worst-case running time of T(n)could be described by the recurrence Merge-Sort procedure Show that the solution of T(n) = Θ(n log n) if n=1 if n>1

9. Recurrences • RecurrenceSolution method to example 1) The substitution method guess a bound and prove our guess correct. 2) The recursion-tree method convert the recurrence into a tree whose nodes represent the costs incurred at various level of recursion 3) The master method determining asymptotic bounds for many simple recurrences of the form ( a ≥ 1, b ≥ 1, and f(n) is given function )

10. 1. Technicalities for abbreviation • Technicalities 1) assumption of integer The running time T(n) of algorithm is only defined when n is an integer 2) floors /ceilings We often omit floors and ceilings 3) boundary condition The running time of an algorithm on a constant-sized input is a constant that we typically ignore.

11. 1. Technicalities for abbreviation if n=1 if n>1 • Technicalities Example The recurrence describing the worst-case running time of Merge-Sort if n=1 if n>1 if n=1 assumption of integer if n>1 Omit floors and ceilings Omit boundary condition

12. 2. Recurrence solution methods • Substitution Method - Definition 1. Guess the form of the solution 2. Use mathematical induction to find the constant and show that the solution works. - It can be used to establish either upper or lower bounds on a recurrence.

13. 2. Recurrence solution methods • Substitution Method Example Let us determine an upper bound on the recurrence

14. 2. Recurrence solution methods • Substitution Method Solution to example We need to find constant c and n0 to meet the requirement about assumption T(n) ≤ cnlg n ( c>0 ) We using the Mathematical induction to show that our assumption holds for the boundary condition. 1. guess that the solution is 2. prove that ( c > 0 )

15. 2. Recurrence solution methods • Substitution Method Solution to example Mathematical induction We can prove something stronger for a given value by assuming something stronger for smaller values. • → T(1), T(2), T(3), … ….. T(⌊n/2⌋), T(n) k k+1 • k means ⌊n/2⌋ • k+1means n

16. 2. Recurrence solution methods • Substitution Method Solution to example Recurrence : Guess : T(n)= O(n) Prove : T(n) ≤ cn Substitution T (⌊n/2⌋) ≤ c ⌊n/2⌋ lg (⌊n/2⌋) • T(n) ≤ 2(c ⌊n/2⌋lg(⌊n/2⌋)) + n • ≤ cnlg(n/2) + n ( because, ⌊n/2⌋ < n/2 ) • = cnlgn - cnlg 2 + n • = cnlgn -(c+1)n ( It’s our assumed fact) -> • k means • k+1means n

17. 2. Recurrence solution methods • Substitution Method Solution to example Assumed Fact : cnlgn –(c+1)n Prove : T(n) ≤ cn Assumed Fact ≤ Prove • T(n) ≤ cnlgn – (c+1)n • ≤ cnlgn (as long as c ≥ 1)

18. 2. Recurrence solution methods • Substitution Method Solution to example • - Find the constant n0 - we can take advantage of asymptotic notation, we can replace base case T(1) ≤ cnlgn T(1) = 1 but, c1 lg1 = 0 - So, we only have to prove T (n) = cnlgn for n ≥ n0 for n (n0 =2) Replace T(1) byT(2) andT(3) as the base cases by letting n0 =2.

19. 2. Recurrence solution methods • Substitution Method Solution to example • -Find the constantc • - choosing c large enough T (2) = 4 and T (3) = 5. T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ). Any choice of c ≥ 2 suffices for base case of n=2 and n =3

20. 2. Recurrence solution methods • Substitution Method Skill - Subtleties We can prove something stronger for given value by assuming something stronger for a smaller values. Overcome difficulty by subtracting a lower-order term from our previous guess

21. 2. Recurrence solution methods • Substitution Method Skill - Subtleties example Recurrence : Guess : T(n)= O(n) Prove : T(n) ≤ cn Substitution → → • k means • k+1 means n

22. 2. Recurrence solution methods • Substitution Method Skill - Subtleties example ( It’s Impossible. Then T(n)= O(n) is wrong? ) Subtracting a lower-order term from our previous guess T(n) ≤ cn So, we now prove T(n) ≤ cn-b

23. 2. Recurrence solution methods • Substitution Method Skill - Subtleties example T(n) Recurrence : Guess : T(n)= O(n) Prove : T(n) ≤ cn-b ( b≥1, c must be chosen large enough )

24. 2. Recurrence solution methods • Substitution Method Skill - Avoid pitfalls We need to prove the exact form of the inductive hypothesis T(n) ≤ cn T(n) ≤ (c+1)n

25. 2. Recurrence solution methods • Substitution Method Skill - Avoid pitfalls example Recurrence : Guess : T(n)= O(n) Prove : T(n) ≤ cn ( Wrong !!! exact form :T(n)≤ cn)

26. 2. Recurrence solution methods • Substitution MethodSkill - Simplify this recurrence bychanging variable Renaming m = lg n S(m) = T(2m) This new recurrence has same solution : S(m) = O(m log m)

27. 2. Recurrence solution methods • Substitution MethodSkill - Simplify this recurrence bychanging variable This new recurrence has same solution : S(m) = O(m log m) Changing back from S(m)to T(n) T(n) = T(2m) = S(m) = O(m lg m) = O( lg n lg(lg n) )

28. 2. Recurrence solution methods • Substitution MethodSkill - Making a good guess prove loose upper and lower bounds on the recurrence and then reduce the range of uncertainty. - Guessing a solution takes : experience, occasionally, creativity We can make a good guess by using recursion tree!!

29. 2. Recurrence solution methods • Recursion-tree - Definition For come up with good guess, convert the recurrence into a tree that each node represents the cost of a single subproblem somewhere in the set of recursive function invocations. We sum all the per-level costs to determine the total cost of all levels of the recursion

30. 2. Recurrence solution methods • Recursion-tree Example The recurrence is T (n) = 3T (⌊n/4⌋) + Θ(n2) = 3T (n/4) + cn2 Use the recursion tree to making a good guess for upper bound for the solution.

31. 2. Recurrence solution methods • Recursion-tree Solution to example Draw recursion tree 3T (n/4) + cn2 cn2 cn2 T(n)

32. 2. Recurrence solution methods • Recursion-tree Solution to example cn2 c(n/4)2 c(n/4)2 c(n/4)2 log4n When depth ithen subprogram size is n/4i= 1so, i = log4 n and it means tree’s height and tree has log4 n + 1 levels. The number of nodes at depth iis3iand finally … T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1) T(1)

33. 2. Recurrence solution methods • Recursion-tree Solution to example Total Cost Sum of cost for idepth Sum of cost for root to i-1 depth

34. 2. Recurrence solution methods • Recursion-tree Solution to example Total Cost is… ( When 0<x<1 then, )

35. 2. Recurrence solution methods T (n) = 3T (⌊n/4⌋) + Θ(n2) = 3T(⌊n/4⌋) + cn2 Recurrence : Guess : T (n) = O(n2) Prove : T (n) ≤ dn2 (for some d > 0 and for the same c > 0) • Recursion-tree Solution to example ProveT (n) = O(n2) by the substitution method

36. 2. Recurrence solution methods Recursion-tree Solution to example Find constantd T (n) = 3T(⌊n/4⌋) + cn2 T(n) ≤ 3T(⌊n/4⌋) + cn2 ≤ 3d⌊n/4⌋2 + cn2 ≤ 3d(n/4)2 + cn2 = 3/16 dn2 + cn2 ≤ dn2(where the last step holds as long as d ≥ (16/13)c )

37. 2. Recurrence solution methods • Recursion-tree Example The recurrence T (n) = 3T (n/3)+T(2n/3)+O(n) Use the recursion tree to making a good guess for upper bound for the solution.

38. 2. Recurrence solution methods • Recursion-tree Solution to example Draw recursion tree T (n) = 3T (n/3)+T(2n/3)+ cn cn cn cn log3/2n cn Total : O(n log n) This recursion tree is not a complete binary tree. Not all leaves contribute a cost of exactlycn

39. 2. Recurrence solution methods • Recursion-tree Solution to example Draw recursion tree T (n) = 3T (n/3)+T(2n/3)+ cn cn cn cn log3/2n cn The longest path from root to a leaf is n→(2/3)n → (2/3)2n → … → 1. Since, (2/3)kn= 1 when k=log3/2 nand it means tree’s height is log3/2 n . The total cost = Cost ofeach level x Height = cn xlog3/2 n = O(cnlog3/2n) = O(n lgn)

40. 2. Recurrence solution methods Recurrence : Guess : T (n) = O(n lg n) Prove : T (n) ≤ d n lg n(for some d > 0 and for the same c > 0) T(n/3) + T(2n/3) + cn • Recursion-tree Solution to example ProveT (n) = O(n lg n) by the substitution method

41. 2. Recurrence solution methods Recursion-tree Solution to example Find constantd T(n)=T(n/3) + T(2n/3) + cn T(n) ≤ T(n/3) + T(2n/3) + cn ≤ d(n/3)lg(n/3) + d(2n/3)lg(2n/3) + cn = (d(n/3)lgn- d(n/3)lg 3) + (d(2n/3)lgn– d(2n/3)lg(3/2)) + cn = dnlgn- d((n/3)lg3 + (2n/3)lg(3/2)) + cn = dnlgn- d((n/3)lg3 + (2n/3)lg(3/2)) + cn = dnlgn- d((n/3)lg3 + (2n/3)lg3 - (2n/3)lg2) + cn = dnlgn- dn(lg3 - 2/3) + cn ≤ dnlgn(as long as d ≥ c/(lg 3 - (2/3)) )

42. Content of next week Ch4. Recurrence Recurrence solution methods The substitution method The recursion-tree method The master method