Big-Oh Notation

1. Big-Oh Notation CS 105 Introduction to Data Structures and Algorithms

2. Order Arithmetic • Big-Oh notation provides a way to compare two functions • “f(n) is O( g(n) )” means:f(n) is less than or equal to g(n) up to a constant factor for large values of n

3. Categorizing functions • Big-Oh can be used for categorizing or characterizing functions • For example, the statements: 2n + 3 is O(n) and 5n is O(n)place 2n + 3 and 5n in the same category • Both functions are less than or equal to g(n) = n, up to a constant factor, for large values of n • If the functions are running times of two algorithms, the algorithms are thus comparable

4. Definition of Big-Oh f(n) is O( g(n) ) if there is a real constant c > 0 and an integer constant n0 >= 1 such that f(n) <= c g(n), for n >= n0 less than or equal up to a constant factor for large values of n

5. Example • f(n) = 2n + 5g(n) = n • Consider the condition 2n + 5 <= nwill this condition ever hold? No! • How about if we tack a constant to n? 2n + 5 <= 3nthe condition holds for values of n greater than or equal to 5 • This means we can select c = 3 and n0 = 5

6. Example 3n 2n+5 2n+5 n point where 3n“beats” 2n+5 2n+5 is O( n )

7. Use of Big-Oh notation • Big-Oh allows us to ignore constant factors and lower order (or less dominant) terms 2n2 + 5n – 4 is O( n2 ) constants lower order terms

8. Back to arrayMax example Algorithm arrayMax(A,n): Input: An array A storing n integers. Output: The maximum element in A. currentMax  A[0] for i  1 to n - 1 do if currentMax < A[i] then currentMax  A[i] return currentMax

9. Back to arrayMax example • Depending on what operations we decide to count, running time function f(n) = a n + b • Regardless, f(n) is O(n) • Or, equivalently, the running time of algorithm arrayMax is O(n)

10. Function categories revisited • The constant function: f(n) = 1 • The linear function: f(n) = n • The quadratic function: f(n) = n2 • The cubic function: f(n) = n3 • The exponential function: f(n) = 2n • The logarithm function: f(n) = log n • The n log n function: f(n) = n log n

11. Comparing function categories • Linear (n) is better than quadratic (n2) which is better than exponential (2n) • Are there any function categories better than linear? Yes! • Constant (1) • Logarithmic (log n) • “Better” means resulting values are smaller (slower growth rates)

12. Functions by increasing growth rate • The constant function: f(n) = 1 • The logarithm function: f(n) = log n • The linear function: f(n) = n • The n log n function: f(n) = n log n • The quadratic function: f(n) = n2 • The cubic function: f(n) = n3 • The exponential function: f(n) = 2n

13. Big-Oh in this course • For this course, you will be expected to assess the running time of an algorithm and classify it under one of the categories, using Big-Oh notation • You should be able to recognize, for instance, that, most of the time (not always): • Algorithms with single loops are O(n) • Algorithms with double-nested loops are O(n2)

14. Big-Oh as an upper bound • The statement f(n) is O( g(n) ) indicates that g(n) is an upper bound for f(n) • Which means it is also correct to make statements like: • 3n+5 is O(n2) • 3n+5 is O(2n) • 3n+5 is O(5n + log n - 2) • But the statement 3n+5 is O(n) is the “tightest” statement one can make

15. Relatives of Big-Oh • Big Omega  : lower bound • Big Theta  : the function is both a lower bound and an upper bound • For this course, only Big-Oh notation will be used for algorithm analysis

16. Summary • Big-Oh notation compares functions • It provides for a mechanism to precisely characterize functions according to typical function categories • Running time functions of algorithms are assessed more precisely using Big-Oh • The notation allows us to ignore constants and lower order terms