1 / 43

Asymptotic Growth Rate

Asymptotic Growth Rate. Function Growth. The running time of an algorithm as input size approaches infinity is called the asymptotic running time We study different notations for asymptotic efficiency. In particular, we study tight bounds, upper bounds and lower bounds. Outline.

Jims
Download Presentation

Asymptotic Growth Rate

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Asymptotic Growth Rate Cutler/Head

  2. Function Growth • The running time of an algorithm as input size approaches infinity is called the asymptotic running time • We study different notations for asymptotic efficiency. • In particular, we study tight bounds, upper bounds and lower bounds. CS575 / Class 1

  3. Outline • Why do we need the different sets? • Definition of the sets O, Omega and Theta • Classifying examples: • Using the original definition • Using limits • General Properties • Little Oh • Additional properties Cutler/Head

  4. The “sets” and their use – big Oh • Big “oh” - asymptotic upper bound on the growth of an algorithm • When do we use Big Oh? • Theory of NP-completeness • To provide information on the maximum number of operations that an algorithm performs • Insertion sort is O(n2) in the worst case • This means that in the worst case it performs at most cn2 operations • Insertion sort is also O(n6) in the worst case since it also performs at most dn6 operations Cutler/Head

  5. The “sets” and their use – Omega Omega - asymptotic lower bound on the growth of an algorithm or a problem* When do we use Omega? 1. To provide information on the minimum number of operations that an algorithm performs • Insertion sort is (n) in the best case • This means that in the best case its instruction count is at least cn, • It is (n2) in the worst case • This means that in the worst case its instruction count is at least cn2 Cutler/Head

  6. The “sets” and their use – Omega cont. 2. To provide information on a class of algorithms that solve a problem • Sort algorithms based on comparison of keys are (nlgn) in the worst case • This means that all sort algorithms based only on comparison of keys have to do at least cnlgn operations • Any algorithm based only on comparison of keys to find the maximum of n elements is (n) in every case • This means that all algorithms based only on comparison of keys to find maximum have to do at least cn operations Cutler/Head

  7. The “sets” and their use - Theta • Theta - asymptotic tight bound on the growth rate of an algorithm • Insertion sort is (n2) in the worst and average cases • The means that in the worst case and average cases insertion sort performs cn2 operations • Binary search is (lg n) in the worst and average cases • The means that in the worst case and average cases binary search performs clgn operations • Note: We want to classify an algorithm using Theta. • In Data Structures used Oh • Little “oh” - used to denote an upper bound that is not asymptotically tight. n is in o(n3). n is not in o(n) Cutler/Head

  8. The functions • Let f(n) and g(n) be asymptotically nonnegative functions whose domains are the set of natural numbers N={0,1,2,…}. • A function g(n) is asymptotically nonnegative, if g(n)³0 for all n³n0 where n0ÎN CS575 / Class 1

  9. Asymptotic Upper Bound: big O c g (n) Graph shows that for all n N, f(n)  c*g(n) f (n) Why only for n N ? What is the purpose of multiplying by c > 0? N f (n) = O ( g ( n )) Cutler/Head

  10. Asymptotic Upper Bound: O Definition: Let f (n) and g(n) be asymptotically non-negative functions. We sayf (n) is in O( g ( n )) if there is a real positive constant c and a positive Integer N such that for every n ³ N 0 £f (n)£ cg (n ). Or using more mathematical notation O( g (n) ) = { f (n )| there exist positive constant c and a positive integer N such that 0 £f( n) £ cg (n ) for all n³ N} Cutler/Head

  11. n2 + 10 nO(n2) Why? take c = 2N= 10 2n2 > n2 + 10 n for all n>=10 1400 1200 1000 n2 + 10n 800 600 2 n2 400 200 0 0 10 20 30 Cutler/Head

  12. Does 5n+2 ÎO(n)? Proof: From the definition of Big Oh, there must exist c>0 and integer N>0 such that 0 £ 5n+2£cn for all n³N. Dividing both sides of the inequality by n>0 we get: 0 £ 5+2/n£c. 2/n £ 2, 2/n>0 becomes smaller when n increases There are many choices here for c and N. If we choose N=1 then c³ 5+2/1= 7. If we choose c=6, then 0 £ 5+2/n£6. So N ³ 2. In either case (we only need one!) we have a c>o and N>0 such that 0 £ 5n+2£cn for all n ³ N. So the definition is satisfied and 5n+2 ÎO(n) Cutler/Head

  13. Does n2Î O(n)? No. We will prove by contradiction that the definition cannot be satisfied. Assume that n2Î O(n). From the definition of Big Oh, there must exist c>0 and integer N>0 such that 0 £n2£cn for all n³N. Dividing the inequality by n>0 we get 0 £n£ c for all n³N. n £c cannot be true for any n >max{c,N }, contradicting our assumption So there is no constant c>0 such that n£c is satisfied for all n³N, and n2 O(n) Cutler/Head

  14. O( g (n) ) = { f (n )| there exist positive constant c and positive integer N such that 0 £f( n) £ cg (n ) for all n³ N} • 1,000,000 n2O(n2) why/why not? • (n - 1)n / 2 O(n2) why /why not? • n / 2 O(n2) why /why not? • lg (n2) O( lg n ) why /why not? • n2O(n) why /why not? Cutler/Head

  15. Asymptotic Lower Bound, Omega: W f (n) c * g (n) N f(n) =( g ( n )) Cutler/Head

  16. Asymptotic Lower Bound: W Definition: Let f (n) and g(n) be asymptotically non-negative functions. We sayf (n) is W( g ( n )) if there is a positive real constant c and a positive integer N such that for every n ³ N0 £ c* g (n ) £ f ( n). Or using more mathematical notation W( g ( n )) = { f (n)| there exist positive constant c and a positive integer N such that 0 £c* g (n ) £f ( n) for all n³ N} Cutler/Head

  17. Is 5n-20ÎW (n)? Proof: From the definition of Omega, there must exist c>0 and integer N>0 such that 0 £ cn£ 5n-20 for all n³N Dividing the inequality by n>0 we get: 0 £ c £ 5-20/n for all n³N. 20/n £ 20, and 20/n becomes smaller as n grows. There are many choices here for c and N. Since c > 0, 5 – 20/n >0 and N >4 For example, if we choose c=4, then 5 – 20/n ³ 4 and N ³ 20 In this case we have a c>o and N>0 such that 0 £ cn£ 5n-20 for all n ³ N. So the definition is satisfied and 5n-20 Î W (n) Cutler/Head

  18. W( g ( n )) = { f (n)| there exist positive constant c and a positive integer N such that 0 £c* g (n ) £ f ( n) for all n³ N} • 1,000,000 n2W (n2) why /why not? • (n - 1)n / 2W (n2) why /why not? • n / 2W (n2) why /why not? • lg (n2)W ( lg n ) why /why not? • n2W (n) why /why not? Cutler/Head

  19. d g (n) Asymptotic Tight Bound: Q f (n) c g (n) f (n) =Q( g ( n )) N Cutler/Head

  20. Asymptotic Bound Theta: Q Definition: Let f (n) and g(n) be asymptotically non-negative functions. We say f (n) is Q( g ( n )) if there are positive constants c, d and a positive integer N such that for every n ³ N0 £cg (n ) £ f ( n) £ dg ( n ). Or using more mathematical notation Q( g ( n )) = { f (n)| there exist positive constants c, d and a positive integer N such that 0 £ cg (n ) £ f ( n) £ dg ( n ). for all n³ N} Cutler/Head

  21. More on Q • We will use this definition:Q (g (n)) =O(g (n) ) Ç W (g (n) ) Cutler/Head

  22. We show: Cutler/Head

  23. Cutler/Head

  24. Cutler/Head

  25. More Q • 1,000,000 n2Q(n2) why /why not? • (n - 1)n / 2Q(n2) why /why not? • n / 2Q(n2) why /why not? • lg (n2)Q( lg n ) why /why not? • n2Q(n) why /why not? Cutler/Head

  26. Limits can be used to determine Order { c then f (n) = Q ( g (n)) if c > 0 if lim f (n) / g (n) = 0 or c > 0 then f (n) = o ( g(n)) ¥ or c > 0then f (n) = W ( g (n)) • The limit must exist n®¥ Cutler/Head

  27. Example using limits Cutler/Head

  28. L’Hopital’s Rule If f(x) and g(x) are both differentiable with derivatives f’(x) and g’(x), respectively, and if Cutler/Head

  29. Example using limits Cutler/Head

  30. Î O ( n ) lg n æ ö ln n ln n 1 = = lg n ( lg n )' ' = ç ÷ è ø ln 2 ln 2 nln2 lg n (lg n )' 1 = = = lim lim lim 0 n n ' n ln 2 ® ¥ ® ¥ ® ¥ n n n Example using limits Cutler/Head

  31. n ( 2 ) Example using limits Î k n O where k is a positiv e integer. = ln 2 n n 2 e = = ln 2 ln 2 n n n n ( 2 )' ( e )' = ln2 e 2 ln 2 - 1 k k n kn = = lim lim n n 2 2 ln 2 ® ¥ ® ¥ n n - - 2 k k ( k 1 ) n k ! = = = = lim . . . lim 0 2 n n k 2 ln 2 2 ln 2 ® ¥ ® ¥ n n Cutler/Head

  32. Another upper bound “little oh”: o Definition: Let f (n) and g(n) be asymptotically non-negative functions.. We say f ( n ) is o ( g ( n ))if for every positive real constant cthere exists a positive integer N such that for all n ³ N 0 £f(n)<cg (n ). o( g (n) ) = {f(n): for any positive constant c >0, there exists a positive integer N > 0 such that 0 £ f( n) < cg (n ) for all n³ N} • “little omega” can also be defined Cutler/Head

  33. main difference between O and o O( g (n) ) = { f (n )|there exist positive constant c and a positive integer N such that 0 £f( n) cg (n ) for all n³ N} o( g (n) ) = { f(n)| for any positive constantc >0, there exists a positive integer N such that 0 £ f( n) < cg (n ) for all n³ N} For ‘o’ the inequality holds for all positive constants. Whereas for ‘O’ the inequality holds for some positive constants. Cutler/Head

  34. Lower-order terms and constants • Lower order terms of a function do not matter since lower-order terms are dominated by the higher order term. • Constants (multiplied by highest order term) do not matter, since they do not affect the asymptotic growth rate • All logarithms with base b >1 belong to (lg n) since Cutler/Head

  35. Asymptotic notation in equations What does n2 + 2n + 99 = n2 + Q(n) mean? n2 + 2n + 99 = n2 + f(n) Where the function f(n)is from the setQ(n). In fact f(n) = 2n + 99 . Using notation in this manner can help to eliminate non-affecting details and clutter in an equation. ? 2n2 + 5n + 21= 2n2 + Q (n ) = Q (n2) Cutler/Head

  36. Transitivity: If f (n) = Q (g(n )) and g (n) = Q (h(n )) then f (n) = Q (h(n )) . If f (n) = O (g(n )) and g (n) = O (h(n )) then f (n) =O (h(n )). If f (n) = W (g(n )) and g (n) = W (h(n )) then f (n) = W (h(n )) . If f (n) = o (g(n )) and g (n) = o (h(n )) then f (n) = o (h(n )) . If f (n) = w (g(n )) and g (n) = w (h(n )) then f (n) = w (h(n )) Cutler/Head

  37. Reflexivity: f (n) = Q (f (n )). f (n) = O (f (n )). f (n) = W (f (n )). “o” is not reflexive Cutler/Head

  38. Symmetry and Transpose symmetry: • Symmetry:f (n) = Q ( g(n )) if and only if g (n) = Q (f (n )) . • Transpose symmetry:f (n) = O (g(n )) if and only if g (n) = W (f (n )). f (n) = o (g(n )) if and only if g (n) = w (f (n )). Cutler/Head

  39. Analogy between asymptotic comparison of functions and comparison of real numbers. f (n) = O( g(n)) »a £ b f (n) = W ( g(n)) » a ³ b f (n) = Q ( g(n)) » a = b f (n) = o ( g(n)) » a < b f (n) = w ( g(n)) » a > b Cutler/Head

  40. Not All Functions are Comparable The following functions are not asymptotically comparable: Cutler/Head

  41. Cutler/Head

  42. Is O(g(n)) = (g(n))  o(g(n))? We show a counter example: The functions are: g(n) = n and f(n)O(n) but f(n)  (n) and f(n)  o(n) Conclusion: O(g(n))  (g(n))  o(g(n)) Cutler/Head

  43. General Rules • We say a function f (n ) is polynomially bounded if f (n ) = O ( nk ) for some positive constant k • We say a function f (n ) is polylogarithmic bounded if f (n ) = O ( lgk n) for some positive constant k • Exponential functions • grow faster than positive polynomial functions • grow faster than polylogarithmic functions Cutler/Head

More Related