Analysis Tools & Asymptotic Notations

Analysis Tools& Asymptotic Notations Data structures & Algorithms

Program Performance • Program performance is the amount of computer memory and time needed to run a program. • How is it determined? • Analytically • performance analysis • Experimentally • performance measurement

Criteria for Measurement • Space • amount of memory program occupies • usually measured in bytes, KB or MB • Time • execution time • usually measured by the number of executions

Complexity • Complexity is the resources required by the algorithms, most commonly known as the time how many steps it needs and the space how much memory it takes • Running Time • Running time — the actual time spent in execution of an algorithm. • It depends on a number of factors • Inputs data • The hardware and software environment. • .

Analysis of Algorithms • There are often many different algorithms which can be used to solve the same problem. Thus, it makes sense to develop techniques that allow us to: • compare different algorithms with respect to their “efficiency” • choose the most efficient algorithm for the problem • The efficiency of any algorithmic solution to a problem is a measure of bye the: • Time efficiency: the time it takes to execute. • Space efficiency: the space (primary or secondary memory) it uses. • We will focus on an algorithm’s efficiency with respect to time.

Components of Program Space • Data space • very much dependent on the computer architecture and compiler • The magnitude of the data that a program works with is another factor char 1 float 4short 2 double 8int 2 long double 10long 4 pointer 2 Unit: bytes See Figure 2.2

Components of Program Space • Data space • Choosing a “smaller” data type has an effect on the overall space usage of the program. • Choosing the correct type is especially important when working with arrays. • How many bytes of memory are allocated with each of the following declarations? double a[100]; int maze[rows][cols];

Time Complexity • Time complexity is the amount of computer time a program needs to run. • Why do we care about time complexity? • Some computers require upper limits for program execution times. • Some programs require a real-time response. • If there are many solutions to a problem, typically we’d like to choose the quickest.

Machine independence • The evaluation of efficiency should be as machine independent as possible. • It is not useful to measure how fast the algorithm runs as this depends on which particular computer, OS, programming language, compiler, and kind of inputs are used in testing • Instead, • we count the number of basic operations the algorithm performs. • we calculate how this number depends on the size of the input. • A basic operation is an operation which takes a constant amount of time to execute. • Hence, the efficiency of an algorithm is the number of basic operations it performs.

Example of Basic Operations • Arithmetic operations: *, /, %, +, - • Assignment statements of simple data types. • Reading of primitive types. • Indexing into an array • Following an object reference • Returning from a method • writing of a primitive types • Simple conditional tests: if (x < 12) ... • a method's return statement • We consider an operation such as ++ , += , and *= as consisting of two basic operations.

Example • Given an algorithm which finds the maximum value in an array, count the number of primitive operations executed in this algorithm.

The number of primitive operations t(n) executed by algorithm arrayMax is: • Best case t(n) = 2+1+n+(2+2)*(n-1)+1 = 5n At least • Worst case t(n) = 2+1+n+(2+2+2)*(n-1)+1 = 7n-2 At most

Best, Average and Worst • Average-Case Analysis—expresses the running time of an algorithm as an • average taken over all possible inputs. • Best-Case Analysis — the shortest running time of an algorithm. • Worst-Case Analysis — the longest running time of an algorithm. • We are usually interested in the worst case complexity

Other way of writing the above algorithm

Result of algorithms analysis will enable to: • determine the running time of a program as a function of its inputs. • determine the total or maximum memory space needed for program data. • determine the total size of the program code. • determine whether the program correctly computes the desired result. • determine the complexity of the program--e.g., how easy is it to read, understand, and modify. • determine the strongest of the program--e.g., how well does it deal with unexpected or erroneous inputs?

Loop Example • Find the exact number of basic operations in the following program fragment: double x, y; x = 2.5 ; y = 3.0; 2 for(int i = 0; i <= n; i++) 1 n+1, 2*n { a[i] = x * y; 2*n x = 2.5 * x; 2*n y = y + a[i]; 3*n } The total number of basic operations is 6 * n + 2 * n + (n + 1) + 3 = 10n + 4

Suppose n is a multiple of 2. Determine the number of basic operations performed by of the method myMethod(): for(int i = 1; i < n; i = i * 2) sum = sum + i + helper(i); is log2n Therefore the number of basic operations is: 1 + 1 + ( log2n) + log2n [2 + 4 + 1 + 1 + (n + 1) + n[2 + 2] + 1] + 1 = 2 + log2n + log2n [10 + 5n] + 1 = 5 n log2n + 11 log2n + 3 static int myMethod(int n) { int sum = 0; for(int i = 1; i < n; i = i * 2) sum = sum + i + helper(i); return sum; } static int helper(int n){ int sum = 0; for(int i = 1; i <= n; i++) sum = sum + i; return sum; }

Operation Count • Worst case count = maximum count • Best case count = minimum count • Average count

Step Count • The operation-count method omits accounting for the time spent on all but the chosen operation • The step-count method count for all the time spent in all parts of the program • A program step is loosely defined to be a syntactically or semantically meaningful segment of a program for which the execution time is independent of the instance characteristics. • 100 adds, 100 subtracts, 1000 multiples can be counted as one step. • However, n adds cannot be counted as one step.

Asymptotic Complexity • Two important reasons to determine operation and step counts • To compare the time complexities of two programs that compute the same function • To predict the growth in run time as the instance characteristic changes • Neither of the two yield a very accurate measure • Operation counts: focus on “key” operations and ignore all others • Step counts: the notion of a step is itself inexact • Asymptotic complexity provides meaningful statements about the time and space complexities of a program

Complexity Example • Two programs have complexities c1n2 + c2n and c3n,respectively • The program with complexity c3n will be faster than the one with complexity c1n2 + c2nfor sufficiently large values of n • For small values of n, either program could be faster  depends on the values of c1, c2 and c3 • If c1 = 1, c2 = 2, c3 = 100, then c1n2 + c2n≤ c3n for n ≤ 98 and c1n2 + c2n> c3n for n > 98 • What if c1 = 1, c2 = 2, and c3 = 3?

Big Oh (O) Notation • Definition of “Big-O” Big-Oh notation represents the growth rate of an algorithm. Big-Oh notation shows the comparison result of the growth rate of two function. Let f(n) and g(n) be functions mapping nonnegative integers to real numbers. We say that f(n) is O(g(n)) if there is a real constant c>0 and an integer constant n0 >= 1 such that f(n)<=cg(n) for every integer n>= n0

Big Oh Examples Justify 7n-2 is O(n). Justification: we need to find a real constant c>0 and an integer n0>=1 such that: 7n-2<=cn for every n>=n0. We can choose c=7, n0=1 and then we have 7n-2<7n when n>=1 Thus 7n-2 is O(n). • Justify 20n3 + 5n2 + 6 O(n3 ) Justification: We chose c=31, n0=1 and then we have 20n3 + 5n2 + 6 ≤ 31n3 when n>=1 Thus 20n3 + 5n2 + 6 is O(n3 )

Common Growth Rate Functions • 1 (constant): growth is independent of the problem size n. • log2N (logarithmic): growth increases slowly compared to the problem size (binary search) • N (linear): directly proportional to the size of the problem. • N * log2N (n log n): typical of some divide and conquer approaches (merge sort) • N2 (quadratic): typical in nested loops • N3 (cubic): more nested loops • 2N (exponential): growth is extremely rapid and possibly impractical.

Practical Complexities Overly complex programs may not be practical given the computing power of the system. READ Chapter 3 of the texbook

Proving Big-O Complexity To prove that f(n) is O(g(n)) we find any pair of values n0 and c that satisfy: f(n) ≤ c * g(n) for  n n0 Note: The pair (n0, c) is not unique. If such a pair exists then there is an infinite number of such pairs. Example: Prove that f(n) = 3n2 + 5 is O(n2) We try to find some values of n and c by solving the following inequality: 3n2 + 5  cn2 OR3 + 5/n2 c (By putting different values for n, we get corresponding values for c)

Analysis Tools & Asymptotic Notations