Asymptotics and Recurrence Equations

1. Asymptotics and Recurrence Equations Analysis of Algorithms Prepared by John Reif, Ph.D.

2. Asymptotics and Recurrence Equations • Computational Complexity of a Program • Worst Case and Expected Bounds • Definition of Asymptotic Equations • Solution of Recurrence Notation

3. Readings • Main Reading Selection: • CLR, Chapter 3, 4 and Appendix A

4. Goal of Asymptotic Notation • To estimate and compare growth rates of functions • Ignore constant factors of growth

5. “f(n) is asymptotically equal to g(n)”

6. ”f(n) is little o g(n) ” • f(n) is o(g(n)) if

7. ”f(n) is big O g(n) ” • f(n) is O(g(n)) if

8. Example of f(n) is O(g(n))

9. “f(n) is order at least g(n)” • f(n) is Ω(g(n)) if

10. Example of f(n) is Ω(g(n))

11. “f(n) is order tight with g(n)” • f(n) is Θ(g(n)) if

12. Suppose my algorithm runs in time O(n) • Don’t say: • “his algorithm runs in time O(n2) so is worse” • But prove: • “his algorithm runs in time Ω(n2) so is worse” • Must find a worst case input of length n for which his algorithm takes time > cn2 for all n > n0

13. Use of O Notation • n is O(n2) sometimes written n = O(n2) • But n2 is not O(n) so can’t use identities! •  The two sides of the equality do not play a symmetric role

14. Use of Asymptotic Notation • Write • As • Example

15. Asymptotic Bounds of Sums and Products

16. Convergent Power Sum • A polynomial is asymptotically equal to its leading term as x  ∞

17. Sums of Powers • for n  ∞ • Or equivalently

18. Examples • 2nd order asymptotic expansion

19. Asymptotic Expansion of f(n) as n  n0 • If • and

20. Bounding Sums by Integrals

21. Bounding Sums by Integrals (cont’d) • So • Example then

22. Bounding Sums by Integrals (cont’d) • So • Since • So

23. Other Approximations Derived from Integrals • Harmonic Numbers

24. Stirling’s Approximation for Factorial • Factorial n! = 1·2 · 3 · · · (n-1) · n • So

25. Recurrence Equations Approximate Solution of Recurrence Relations

26. Recurrence Equations (over integers) • Homogenous of degree d n > d • Given constant coefficients a1, …, ad initial values x1, x2 ,…, xd

27. Example: Fibonacci Sequence • n > 2 Fn = Fn-1 + Fn-2 F0 = 0, F1 =1

28. Solution of Fibonacci Sequence “golden ratio” where

29. Solution of Fibonacci Sequence (cont’d) • Hence

30. A Useful Theorem • c > 0, d > 0 • If • then

31. Proof • Is solution

32. Cases

33. Example: Mergesort input list L of length N if N=1 then return L else do let L1 be the first elements of L let L2 be the last elements of L M1 Mergesort (L1) M2 Mergesort (L2) return Merge (M1 , M2)

34. Time Bounds of Mergesort • Initial Value T(1) = c1 for N > 1 for some constants c1, c2> 1

35. Time Bound (cont’d) • N > 1 guess Holds if Solution

36. Time Bound (cont’d) • N > 1 • Transform Variables • Recurrence equation:

37. Solve by usual methods for recurrence equations

38. Advanced Material Exact Solution of Recurrence Relations

39. Homogenous Recurrence Relations (no constant additive term) • Solve: try Multiply by • Get characteristic equation:

40. Case of Distinct Roots • Distinct Roots r1, r2, …, rd • Where ri is dominant root

41. Other Case • Roots are not distinct r1= r2 = r3 • Then solutions not independent, so additional terms:

42. Inhomogenous Recurrence Equations • Nonzero constant term • Solution Method (1) Solve homogenous equation

43. Solution Method • Solve homogenous equation • Case , add particular solution

44. Solution Method (cont’d) Case , add particular solution • Add particular and homogeneous solutions, and solve for constants This is all we usually need!!

45. Asymptotics and Recurrence Equations Analysis of Algorithms Prepared by John Reif, Ph.D.