330 likes | 490 Views
Algorithm Analysis. Chris Kiekintveld CS 2401 (Fall 2010) Elementary Data Structures and Algorithms. Algorithm Analysis. There are many different algorithms to solve the same problem Ask 5 programmers to write a non-trivial program, you will get 5 different solutions Which is best ?
E N D
Algorithm Analysis Chris Kiekintveld CS 2401 (Fall 2010) Elementary Data Structures and Algorithms
Algorithm Analysis • There are many different algorithms to solve the same problem • Ask 5 programmers to write a non-trivial program, you will get 5 different solutions • Which is best? • Correctness • Efficiency Java Programming: Program Design Including Data Structures
Computational Resources • Algorithms require resources to run • Time (processor operations) • Space (computer memory) • Network bandwidth • Programmer time • Two types of costs • Fixed: same every time we run the algorithm • Variable: depends on the size of the input Java Programming: Program Design Including Data Structures
Measuring Resource Use • How can we compare the resources used by different algorithms? • Empirical • Code both algorithms • Run them an record the resources used • You did this in the Fibonacci lab! Java Programming: Program Design Including Data Structures
Empirical Analysis Problems • Depends on code quality/implementation • Better/worse programmers, not the algorithm itself • Depends on computer speed/architecture • Depends on language/compiler efficiency • Depends on the input • E.g. linear search is very fast for some inputs, and very slow for others Java Programming: Program Design Including Data Structures
Analytical Approach • Analyze the algorithm itself • Abstract away from implementation details • How many operations will be executed? • How much memory is used? • Consider different cases (depending on input) • Best • Worst • Average Java Programming: Program Design Including Data Structures
Counting Operations int i = 2; int j = 2; int k = i + j; System.out.println(k); How many operations are there? Assignment: Addition: Print: Total: Java Programming: Program Design Including Data Structures
Counting Operations int i = 2; int j = 2; int k = i + j; System.out.println(i+j+k); How many operations are there? Assignment: Addition: Print: Total: Java Programming: Program Design Including Data Structures
Counting Operations int i = 0; while (i < 10) { System.out.println(i); i++; } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures
Counting Operations for (int i=0; i < 10; i++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures
Counting Operations for (int i=0; i < n; i++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures
Counting Operations for (int i=0; i < n; i++) { for (int j=0; j < n; j++) { System.out.println(i); } } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures
Counting Operations • So far, we have counted every operation • This is quite tedious, especially for infrequent operations • Focus on the most important operation • Most frequent • May need to figure out what this is Java Programming: Program Design Including Data Structures
Another Look at Search Algorithms • We have discussed two ways to search a list • Linear search (unordered data) • Binary search (sorted data) • Data is sorted by “keys” • Unique for each element • Well-defined order Java Programming: Program Design Including Data Structures
Linear (Sequential) Search public int seqSearch(T[] list, int length, T searchItem) { int loc; boolean found = false; for (loc = 0; loc < length; loc++) { if (list[loc].equals(searchItem)) { found = true; break; } } if (found) return loc; else return -1; } Java Programming: Program Design Including Data Structures
Sequential Search Analysis • The statements in the forloop are repeated several times • For each iteration of the loop, the search item is compared with an element in the list • When analyzing a search algorithm, you count the number of comparisons • Suppose that L is a list of length n • The number of key comparisons depends on where in the list the search item is located Java Programming: Program Design Including Data Structures
Sequential Search Analysis (continued) • Best case • The item is the first element of the list • You make only one key comparison • Worst case • The item is the last element of the list • You make n key comparisons • What is the average case Java Programming: Program Design Including Data Structures
Sequential Search Analysis (continued) • To determine the average case • Consider all possible cases • Find the number of comparisons for each case • Add them and divide by the number of cases • Average case • On average, a successful sequential search searches half the list Java Programming: Program Design Including Data Structures
Binary Search public int binarySearch(T[] list, int length, T searchItem) { int first = 0; int last = length - 1; int mid = -1; boolean found = false; while (first <= last && !found) { mid = (first + last) / 2; Comparable<T> compElem = (Comparable<T>) list[mid]; Java Programming: Program Design Including Data Structures
Binary Search (continued) if (compElem.compareTo(searchItem) == 0) found = true; else if (compElem.compareTo(searchItem) > 0) last = mid - 1; else first = mid + 1; } if (found) return mid; else return -1; }//end binarySearch Java Programming: Program Design Including Data Structures
Binary Search Example Figure 18-1 Sorted list for a binary search Table 18-1 Values of first, last, and middle and the Number of Comparisons for Search Item 89 Java Programming: Program Design Including Data Structures
Performance of Binary Search • Suppose that L is a sorted list of size n • And n is a power of 2 (n = 2m) • After each iteration of the forloop, about half the elements are left to search • The maximum number of iteration of the for loop is about m + 1 • Also m = log2n • Each iteration makes two key comparisons • Maximum number of comparisons: 2(m + 1) Java Programming: Program Design Including Data Structures
Comparison: Linear vs BinaryWorst case number of comparison Java Programming: Program Design Including Data Structures
Asymptotic Analysis: Motivation • So far, we have counted operations exactly • We don’t really care about the details • Computers execute billions of operations per second • A few here or there is negligible • Care about overall scalability • As the input size grows, does computation grow quickly or slowly? • Don’t lose the forest for the trees Java Programming: Program Design Including Data Structures
Asymptotic Analysis • Asymptotic means the study of the function f as n becomes larger and larger without bound • Consider functions g(n) = n2 and f(n) = n2 + 4n + 20 • As n becomes larger and larger, the term 4n + 20 in f(n) becomes insignificant • g(1000) = 1,000,000 and f(1000) = 1,004,020 • You can predict the behavior of f(n) by looking at the behavior of g(n) Java Programming: Program Design Including Data Structures
Asymptotic Algorithm Analysis • Identify a function that describes the growth in runtime as the input gets large • An “upper bound” of sorts on the running time • Typically worst-case, but occasionally average case • Describe the number of operations done using a function • Focus only on most important operations • Ignore one time initializations, etc. Java Programming: Program Design Including Data Structures
Common Asymptotic Functions Table 18-4 Growth Rate of Various Functions Java Programming: Program Design Including Data Structures
Common Functions, visual Figure 18-9 Growth Rate of Various Functions Java Programming: Program Design Including Data Structures
Asymptotic Notation: Big-O Notation (continued) Table 18-7 Some Big-O Functions That Appear in Algorithm Analysis Java Programming: Program Design Including Data Structures
Big-Oh Notation (Definition) A function f(n) is O(g(n)) if there exist positive constants c and n0 such that: f(n) ≤ cg(n) for all n ≥ n0 Java Programming: Program Design Including Data Structures
Big-Oh Notation • Translation: After some point, f(n) is always smaller than g(n) • “Some point” refers to increasing problem size • The constant c says that we don’t care about multipliers • So, 2n and n have the same essential growth rate • 2n is O(n) Java Programming: Program Design Including Data Structures
Asymptotic Notation: Big-O Notation (continued) Table 18-8 Number of Comparisons for a List of Length n Java Programming: Program Design Including Data Structures