Algorithm Analysis

Algorithm Analysis

Math Review – 1.2 • Exponents • XAXB = XA+B • XA/XB=XA-B • (XA)B = XAB • XN+XN = 2XN ≠ X2N • 2N+2N = 2N+1 • Logarithms • XA=B iff logXB=A • logAB =logCB/logCA; A,B,C > 0, A ≠ 1 • logAB = logA + logB; A, B > 0 • Series N • ∑ 2i = 2N+1-1 i=0 N • ∑ Ai = AN+1-1/A-1 i=0 N • ∑ i= N(N+1)/2 i=0

Running Time • Why do we need to analyze the running time of a program? • Option 1: Run the program and time it • Why is this option bad? • What can we do about it?

Pseudo-Code • Used to specify algorithms • Part English, part code Algorithm (arrayMax(A, n)) curMax = A[0] for i=1 i<n i++ if curMax < A[i] curMax = A[i] return curMax

Counting Operations Algorithm (arrayMax(A, n)) curMax = A[0] for i=1 i<n i++ if curMax < A[i] curMax = A[i] return curMax • Best case – 5n • Worst case – 7n-2 • Average case – hard to analyze

Asymptotic Notation • 7n-2 • n=5 -> 33 • n=100 -> 698 • n=1,000,000 -> 6,999,998 • Running time grows proportionally to n • What happens as n gets large?

Big-Oh • T(N) is O(f(N)) if there are positive constants c and n0such that T(N) <= cf(N) when N>=n0 • 7n2-2 is O(n2) n0>=1 c=8 • Upper bound

Omega • T(N) is Ω(g(N)) if there are positive constants c and n0such that T(N) >= cg(N) when N>=n0 • 7n2-2 is Ω(n2) n0>=1 c=1 • Lower bound

Theta • T(N) is Θ(h(N)) if and only if T(N) = Oh(N) and T(N) = Ωh(N) • 7n2-2 is Θ(n2) • Tightest result possible

Little-Oh • T(N) is o(p(N)) if T(N) = O(p(N)) and T(N) ≠ Θ(p(N)) • 7n2-2 is o(n3) • O when n0>=8 c=1 • Growth rate of T(N) is smaller than growth rate of p(N)

Rules • Rule 1 • If T1(N) = O(f(N) and T2(N) = O(g(N)), then • T1(N) + T2(N) = max(O(f(N)), O(g(N))) • T1(N)* T2(N) = O(f(N)*g(N)) • Rule 2 • If T(N) is a polynomial of degree k, then T(N) = Θ(Nk) • Rule 3 • logkN = O(N) for any constant k. This tells us that logs grow very slowly.

Examples • 87n4 + 7n • 3nlogn + 12logn • 4n4 + 7n3logn

Terminology • Logarithmic – O(logn) • Linear – O(n) • Quadratic – O(n2) • Polynomial – O(nk) k>=1 • Exponential – O(an) a>1

Rule 1 – for loops • The running time of a for loop is at most the running time of the statements inside the for loop (including tests) times the number of iterations. for(i=0; i<5; i++) cout << “hello\n”;

Rule 2 – nested loops • Analyze these inside out. The total running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes if all of the loops. for(i=0; i<n; i++) for(j=0; j<n; j++) cout << “hello\n”;

Rule 3 – consecutive statements • These just add (which means that the max is the one that counts) for(i=0; i<n; i++) a[i] = 0; for(i=0; i<n; i++) for(j=0; j<n; j++) a[i]+=a[j]+i+j;

Rule 4 – if/else • The running time of an if/else statement is never more than the running time of the test plus the larger of the running times of the statements, if(condition) S1 else S2

Example 0 n-1 0 6 4 … 2 • Find maximum number in nxn matrix • Algorithm: 12 3 … 9 … … … … n-1 5 8 … 1

Example • What is the big-oh running time of this algorithm? Algorithm: Input: A, n curMax = A[0][0] for i=0 i<n i++ for j=0 j<n j++ if curMax < A[i][j] curMax = A[i][j] return curMax

Another Example 0 n-1 2 4 … 6 • Determine how many elements of array 1 match elements of array 2 • Algorithm? 0 n-1 6 8 … 3

Another Example 0 n-1 2 4 … 6 Algorithm: Input: A, B, n for i=0 i<n i++ for j=0 j<n j++ if A[i] == A[j] matches++ break • What is the running time of the algorithm? 0 n-1 6 8 … 3

Logs in the Running Time • An algorithm is O(logN) if it takes constant time to cut the problem size by a fraction. • Binary search • Given an integer X and integers A0, A1, …, AN-1, which are presorted and already in memory, find I such that Ai=X, or return i = -1 if X is not in the input.

Binary Search • Algorithm? • Running time is at most ceil(log(N-1))+2 which is O(logN)

Example • 1 – (N-1) - 1 • 2 – (N-1)/2 - 2 • 3 – (N-1)/4 - 4 • 4 – (N-1)/8 - 8 • i – (N-1)/2i-1 -2i-1 = N-1 • difference between high and low is 1 • difference between high and low is 0

Algorithm Analysis