Analysys & Complexity of Algorithms

1. Analysys & Complexity of Algorithms Big Oh Notation

2. Complexity • Two kinds of complexity • Space complexity • Effects of data type choices on size of data • Effects of amount of data (input and variables) • Time Complexity • Effects of choices of program design • Given 2 algorithms, which is "the best"

3. Big Oh (or Big O) • A measure of complexity related to "n" (problem size=amount of data) e.g.: • # of records to process • #of files • # of numbers or times through a loop • This is asymptotic analysis • associates n, the problem size, with t, the processing time required to solve the problem

4. Big O Examples • x=x+1; • O(1) -> constant degradation • Binary Search of a SORTED set of n elements • O(log n) • for (i=1;i<n;i++) • O(n) -> linear degradation • selection sort, compare two 2D arrays, find duplicates in an UNSORTED list • O(n2) ->quadratic degradation • Generate all premutations of n symbols • O(an) ->'a' is some constant independtent of 'n' • O(l) < O(log n) < O(n) < O(n log n) < O(n2) < O(an)

5. 3 cases • Best case • minimum path lengths • Average case • constant path length • Worst case • maximum path length • most useful!! Leads to better design • answers question: will it be good enough tomorrow??

6. Frequency Counting • Examine a piece of code and predict the number of instructions to be executed • e.g. predict how many times (max) each statement will run.

7. Orderof magnitude • In the previous example: • best case = average case = worst case • Example is based on iteration limit: n • To convert Frequency Count to order of magnitude: • pick the most significant term if polynomial • discard constant terms (like the +1) • disregard coefficients (like the 3) • yields worst case path through algorithm • Big O (represented as O(n)) • O(n) for the previous example

8. Common growth rates

9. Big Oh - Formal Definition • f(n)=O(g(n)), • Thus, g(n) is an upper bound on f(n) • Note:f(n) = O(g(n)) • f(n) has a complexity of g(n)"this is NOT the same as O(g(n)) = f(n) • The '=' is not the usual mathematical "=" operator (it is not reflexive) • We will see more about this in chapter 6