Algorithm Analysis

Algorithm Analysis

Algorithm • An algorithm is a clearly specified set of instructions which, when followed, solves a problem. • recipes • directions for putting together a bicycle • programs

Algorithm Analysis • The process by which we determine the amount of resources used by the algorithm • Resources usually measured • time • memory space • We will concentrate on run time of algorithms

Common functions encountered • Some commonly encountered functions are: • ,linear • ,quadratic • ,cubic • The next slide shows the graphs of these functions

Growth Rates

Running Time T(N)Example 1 Problem: Given an array of N elements, find the smallest. Solution: • key = Integer.Mininum_Value • for( int i=0 ; i< inarray.length ; i++ ) • if ( inarray[i] < key ) key = inarray[i];

Running Time T(N)Example 2 • Suppose we wish to download a file • 2 seconds for setting up the connection • downloading proceeds at 2.8 K/sec • If the downloaded file is N kilobytes, the time to download is T(N) = N/2.8 + 2 • 80K file takes about 31 seconds • 160K file takes about 59 seconds • This is a linear algorithm

Functions in Increasing Order

Measurement of T(N) • Place your timer before/after the algorithm • Run the algorithm multiple times • Take the average of the total time • time = System.currentTimeMillis(); • for (int i = 0; i<num_iteration ; i++) • // Your algorithm comes here • newtime = System.currentTimeMillis(); • output (newTime-time)/num_iteration

Big-O notation • We use big-O to specify growth rates of algorithms • linear functions are specified as O(N) • quadratic functions as O(N2)

Formal Definition of Big-O • Definition: T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 . • Usually T(N) is a complicated function involving N. • We choose F(N) as simple as possible.

Examples of Big-O • Find equivalent Big-Oh functions for the following runtime T(N). • T1(N)=N2 • T2(N)=20*N • T3(N)=N+1000*N2 • T4(N)=N+60 N3 -1000*N2 • T5(N)=N*log(N)+N2

Examples of Big-O • T6(N)=50*N*log(N) + N • T7(N)=N + 60 N3 - N*log(N) • T8(N)= 20*N +100*N2 • T9(N)=90*N2 + 60 N3

Algorithm Analysis • A1: Minimum element in an array • Given an array of N elements, find the smallest. • A2: Closest points in the plane • Given N points in the plane, find the pair of points that are closest together. • A3: Collinear points in the plane • Given N points in the plane, determine if any three are on the same line.

A1: Minimal Element in an Array • Algorithm • Initialize min as the first element • Scan thru the other N-1 elements, updating min as appropriate • There is a fixed amount of work for each element of the array • The running time is O(N)

A2: Closest Point in the Plane • Algorithm • Calculate the distance between each pair of points [there are N(N-1)/2 pairs of points] • find the minimum of these distances • Finding the minimum of N(N-1)/2 distances by this algorithm is O(N2) • There is a O(N log N) algorithm to solve this problem, and one thought to be O(N).

A3: Collinear Points in the Plane • Algorithm • Enumerate all triples of the N points. There are N(N-1)(N-2)/6 of these triples. • check to see if the three points are on the same line • This is a O(N3) algorithm • There is a clever quadratic algorithm for solving this problem

Maximum Contiguous Subsequence Sum Problem • Given (possibly negative) integers A1, A2,…, AN , find (and identify the subsequence corresponding to) the maximum value Ai + Ai+1 +…+ Ak . • The maximum contiguous subsequence sum is zero if all the integers are negative

Examples • With input of [-2, 11, -4, 13, -5, 2], then the maximum sum is 20 using items at positions 2 through 4. • With input of [ 1, -3, 4, -2, -1, 6], then the maximum sum is 7 using the last four items.

Obvious O(N3) Algorithm maxSum = 0;for (i=0; i < N; i++) for (j=i; j < N; j++){ thisSum = 0; for (k=i;k <= j; k++) thisSum += a[k]; if (thisSum > maxSum){ maxSum = thisSum; seqStart = i; seqStop = j; } }return maxSum;

Improving the Algorithm • If we can eliminate one of the loops, we will be able to reduce this algorithm to a O(N2) algorithm. • Observe that Ai + Ai+1 +…+ Ak = (Ai + Ai+1 +…+ Ak-1 ) + Ak . The inner loop is used to calculate Ai + Ai+1 +…+ Ak-1 , then this result is thrown away and then Ai + Ai+1 +…+ Ak is calculated (duplicating work)

Improved O(N2) Algorithm maxSum = 0;for (i=0; i < N; i++){ thisSum = 0; for (j=i; j < N; j++){ thisSum += a[j]; if (thisSum > maxSum){ maxSum = thisSum; seqStart = i; seqStop = j; } }}return maxSum;

Observations • no maximum contiguous subsequence sum begins with a subsequence which has a negative sum • all subsequences which border on the maximum contiguous subsequence must have negative or zero sums

Improving again • Using these observations, any time the current subsequence sum (thisSum) becomes negative, we can begin a new subsequence sum at the next position since the subsequence we are searching for cannot begin with a subsequence with negative sum.

Linear Algorithm maxSum = thisSum = 0;for (i=0, j=0; j < N; j++){ thisSum += a[j]; if (thisSum > maxSum){ maxSum = thisSum; seqStart = i; seqStop = j; } else if (thisSum < 0){ i = j+1; thisSum = 0; }}return maxSum;

Big-O • Definition: T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 . • This means, that for sufficiently large N, T(N) is bounded by some multiple of F(N). • This means that the growth rate of T is no worse than the growth rate of F.

Big- (omega) • Definition: T(N) is (F(N)) if there are positive constants c and N0 such that T(N) cF(N) whenever N N0 . • This means, that for sufficiently large N, T(N) is bounded below by some multiple of F(N). • This means that the growth rate of T is no better than the growth rate of F.

Big- (theta) • Definition: T(N) is (F(N)) if and only if T(N) is O(F(N)) and T(N) is (F(N)). • This means that the growth rate of T is equal to the growth rate of F.

Little-o • Definition: T(N) is o(F(N)) if and only if T(N) is O(F(N)) and T(N) is not (F(N)). • This means that the growth rate of T is strictly better than the growth rate of F

Summary

The Logarithm • Definition: For any B, N > 0, logBN = K if and only if BK = N • In Computer Science, base 2 logarithms are used almost exclusively. • The use of the symbol log indicates log2 unless otherwise indicated

Use of logarithms • Bits in a binary number • The number of bits needed to represent N consecutive integers is log N • Repeated Doubling • Starting with X=1, the number of times needed to double X to reach at least N is log N

Searching • Sequential Search • the average number of comparisons made to find an item in an unordered list of N items is N/2 = O(N) • An unsuccessful search requires testing every item in the list, so is also O(N)

Binary Search • If searching a sorted list of N items, we can do better low = 1; high = N; while (low < high) { mid = (low+high)/2; if (A[mid]< x) low = mid +1; else high = mid;} if (X == A[low]) return low; return -1;

Limitations of Big-O In the following definition of O: T(N) is O(F(N)) if there are positive constants c and N0 such that T(N)cF(N) whenever N N0 . • N0 can be a very large number. • The positive constants c can be a large number. Example : Compare the running time two algorithms Algorithm1 = 10000N Algorithm2 = 2N log(N)

Other Issues of Algorithm Analysis • Memory access • Disk access • Average-case v.s. worst-case running time. • Implementation Difficulties

Algorithm Analysis