Growth of functions

1. Growth of functions

2. Function Growth • The running time of an algorithm as input size approaches infinity is called the asymptotic running time • We study different notations for asymptotic efficiency. • In particular, we study tight bounds, upper bounds and lower bounds.

3. The “sets” and their use – big Oh • Big “oh” - asymptotic upper bound on the growth of an algorithm • When do we use Big Oh? • Theory of NP-completeness • To provide information on the maximum number of operations that an algorithm performs • Insertion sort is O(n2) in the worst case • This means that in the worst case it performs at most cn2 operations • Insertion sort is also O(n6) in the worst case since it also performs at most dn6 operations

4. The “sets” and their use – Omega Omega - asymptotic lower bound on the growth of an algorithm or a problem* When do we use Omega? 1. To provide information on the minimum number of operations that an algorithm performs • Insertion sort is (n) in the best case • This means that in the best case its instruction count is at least cn, • It is (n2) in the worst case • This means that in the worst case its instruction count is at least cn2

5. The “sets” and their use – Omega cont. 2. To provide information on a class of algorithms that solve a problem • Sort algorithms based on comparison of keys are (nlgn) in the worst case • This means that all sort algorithms based only on comparison of keys have to do at least cnlgn operations • Any algorithm based only on comparison of keys to find the maximum of n elements is (n) in every case • This means that all algorithms based only on comparison of keys to find maximum have to do at least cn operations

6. The “sets” and their use - Theta • Theta - asymptotic tight bound on the growth rate of an algorithm • Insertion sort is (n2) in the worst and average cases • The means that in the worst case and average cases insertion sort performs cn2 operations • Binary search is (lg n) in the worst and average cases • The means that in the worst case and average cases binary search performs clgn operations • Note: We want to classify an algorithm using Theta. • In Data Structures used Oh • Little “oh” - used to denote an upper bound that is not asymptotically tight. n is in o(n3). n is not in o(n)

7. The functions • Let f(n) and g(n) be asymptotically nonnegative functions whose domains are the set of natural numbers N={0,1,2,…}. • A function g(n) is asymptotically nonnegative, if g(n)³0 for all n³n0 where n0ÎN

8. Asymptotic Upper Bound: big O c g (n) Graph shows that for all n N, f(n)  c*g(n) f (n) Why only for n N ? What is the purpose of multiplying by c > 0? N f (n) = O ( g ( n ))

9. Asymptotic Upper Bound: O Definition: Let f (n) and g(n) be asymptotically non-negative functions. We sayf (n) is in O( g ( n )) if there is a real positive constant c and a positive Integer N such that for every n ³ N 0 £f (n)£ cg (n ). Or using more mathematical notation O( g (n) ) = { f (n )| there exist positive constant c and a positive integer N such that 0 £f( n) £ cg (n ) for all n³ N}

10. n2 + 10 nO(n2) Why? take c = 2N= 10 2n2 > n2 + 10 n for all n>=10 1400 1200 1000 n2 + 10n 800 600 2 n2 400 200 0 0 10 20 30

11. Does 5n+2 ÎO(n)? Proof: From the definition of Big Oh, there must exist c>0 and integer N>0 such that 0 £ 5n+2£cn for all n³N. Dividing both sides of the inequality by n>0 we get: 0 £ 5+2/n£c. 2/n £ 2, 2/n>0 becomes smaller when n increases There are many choices here for c and N. If we choose N=1 then c³ 5+2/1= 7. If we choose c=6, then 0 £ 5+2/n£6. So N ³ 2. In either case (we only need one!) we have a c>o and N>0 such that 0 £ 5n+2£cn for all n ³ N. So the definition is satisfied and 5n+2 ÎO(n)

12. Does n2Î O(n)? No. We will prove by contradiction that the definition cannot be satisfied. Assume that n2Î O(n). From the definition of Big Oh, there must exist c>0 and integer N>0 such that 0 £n2£cn for all n³N. Dividing the inequality by n>0 we get 0 £n£ c for all n³N. n £c cannot be true for any n >max{c,N }, contradicting our assumption So there is no constant c>0 such that n£c is satisfied for all n³N, and n2 O(n)

13. O( g (n) ) = { f (n )| there exist positive constant c and positive integer N such that 0 £f( n) £ cg (n ) for all n³ N} • 1,000,000 n2O(n2) why/why not? • (n - 1)n / 2 O(n2) why /why not? • n / 2 O(n2) why /why not? • lg (n2) O( lgn ) why /why not? • n2O(n) why /why not?

14. Asymptotic Lower Bound, Omega: W f (n) c * g (n) N f(n) =( g ( n ))

15. Asymptotic Lower Bound: W Definition: Let f (n) and g(n) be asymptotically non-negative functions. We sayf (n) is W( g ( n )) if there is a positive real constant c and a positive integer N such that for every n ³ N0 £ c* g (n ) £ f ( n). Or using more mathematical notation W( g ( n )) = { f (n)| there exist positive constant c and a positive integer N such that 0 £c* g (n ) £f ( n) for all n³ N}

16. Is 5n-20ÎW (n)? Proof: From the definition of Omega, there must exist c>0 and integer N>0 such that 0 £ cn£ 5n-20 for all n³N Dividing the inequality by n>0 we get: 0 £ c £ 5-20/n for all n³N. 20/n £ 20, and 20/n becomes smaller as n grows. There are many choices here for c and N. Since c > 0, 5 – 20/n >0 and N >4 For example, if we choose c=4, then 5 – 20/n ³ 4 and N ³ 20 In this case we have a c>o and N>0 such that 0 £ cn£ 5n-20 for all n ³ N. So the definition is satisfied and 5n-20 Î W (n)

17. W( g ( n )) = { f (n)| there exist positive constant c and a positive integer N such that 0 £c* g (n ) £ f ( n) for all n³ N} • 1,000,000 n2W (n2) why /why not? • (n - 1)n / 2W (n2) why /why not? • n / 2W (n2) why /why not? • lg (n2)W ( lg n ) why /why not? • n2W (n) why /why not?

18. Asymptotic Tight Bound: Q d g (n) f (n) c g (n) f (n) =Q( g ( n )) N

19. Asymptotic Bound Theta: Q Definition: Let f (n) and g(n) be asymptotically non-negative functions. We say f (n) is Q( g ( n )) if there are positive constants c, d and a positive integer N such that for every n ³ N0 £cg (n ) £ f ( n) £ dg ( n ). Or using more mathematical notation Q( g ( n )) = { f (n)| there exist positive constants c, d and a positive integer N such that 0 £ cg (n ) £ f ( n) £ dg ( n ). for alln³ N}

20. More on Q • We will use this definition:Q (g (n)) =O(g (n) ) Ç W (g (n) )

21. We show:

24. More Q • 1,000,000 n2Q(n2) why /why not? • (n - 1)n / 2Q(n2) why /why not? • n / 2Q(n2) why /why not? • lg (n2)Q( lg n ) why /why not? • n2Q(n) why /why not?

25. Qualified Notation Ω Omega Notation • best case Ω - describes a lower bound for all input (it can't get any better than this). Example: the array is already correctly sorted. • worst case Ω - describes a lower bound for worst case input, possibly greater than best case. Example: the array is sorted in reverse order. • just Ω - same as best case Ω

26. Qualified Notation O Big-O Notation • best case O - describes an upper bound for best case input, possibly lower than worst case. Example: the array is already correctly sorted. • worst case O - describes an upper bound for all input (it can't get any worse than this). Example: the array is sorted in reverse order. • just O - same as worst case O

27. Qualified Notation Θ Theta Notation • best case Θ - not used • worst case Θ - describes asymptotic bounds for worst case input • just Θ - same as worst case Θ

28. Comparing functions • Similarly for • Reflexivity • Symmetry • Transpose symmetry