1 / 33

Algorithm Analysis

Algorithm Analysis. Chris Kiekintveld CS 2401 (Fall 2010) Elementary Data Structures and Algorithms. Algorithm Analysis. There are many different algorithms to solve the same problem Ask 5 programmers to write a non-trivial program, you will get 5 different solutions Which is best ?

max
Download Presentation

Algorithm Analysis

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Algorithm Analysis Chris Kiekintveld CS 2401 (Fall 2010) Elementary Data Structures and Algorithms

  2. Algorithm Analysis • There are many different algorithms to solve the same problem • Ask 5 programmers to write a non-trivial program, you will get 5 different solutions • Which is best? • Correctness • Efficiency Java Programming: Program Design Including Data Structures

  3. Computational Resources • Algorithms require resources to run • Time (processor operations) • Space (computer memory) • Network bandwidth • Programmer time • Two types of costs • Fixed: same every time we run the algorithm • Variable: depends on the size of the input Java Programming: Program Design Including Data Structures

  4. Measuring Resource Use • How can we compare the resources used by different algorithms? • Empirical • Code both algorithms • Run them an record the resources used • You did this in the Fibonacci lab! Java Programming: Program Design Including Data Structures

  5. Empirical Analysis Problems • Depends on code quality/implementation • Better/worse programmers, not the algorithm itself • Depends on computer speed/architecture • Depends on language/compiler efficiency • Depends on the input • E.g. linear search is very fast for some inputs, and very slow for others Java Programming: Program Design Including Data Structures

  6. Analytical Approach • Analyze the algorithm itself • Abstract away from implementation details • How many operations will be executed? • How much memory is used? • Consider different cases (depending on input) • Best • Worst • Average Java Programming: Program Design Including Data Structures

  7. Counting Operations int i = 2; int j = 2; int k = i + j; System.out.println(k); How many operations are there? Assignment: Addition: Print: Total: Java Programming: Program Design Including Data Structures

  8. Counting Operations int i = 2; int j = 2; int k = i + j; System.out.println(i+j+k); How many operations are there? Assignment: Addition: Print: Total: Java Programming: Program Design Including Data Structures

  9. Counting Operations int i = 0; while (i < 10) { System.out.println(i); i++; } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures

  10. Counting Operations for (int i=0; i < 10; i++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures

  11. Counting Operations for (int i=0; i < n; i++) { System.out.println(i); } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures

  12. Counting Operations for (int i=0; i < n; i++) { for (int j=0; j < n; j++) { System.out.println(i); } } How many operations are there? Assignment: Comparison: Increment: Print: Total: Java Programming: Program Design Including Data Structures

  13. Counting Operations • So far, we have counted every operation • This is quite tedious, especially for infrequent operations • Focus on the most important operation • Most frequent • May need to figure out what this is Java Programming: Program Design Including Data Structures

  14. Another Look at Search Algorithms • We have discussed two ways to search a list • Linear search (unordered data) • Binary search (sorted data) • Data is sorted by “keys” • Unique for each element • Well-defined order Java Programming: Program Design Including Data Structures

  15. Linear (Sequential) Search public int seqSearch(T[] list, int length, T searchItem) { int loc; boolean found = false; for (loc = 0; loc < length; loc++) { if (list[loc].equals(searchItem)) { found = true; break; } } if (found) return loc; else return -1; } Java Programming: Program Design Including Data Structures

  16. Sequential Search Analysis • The statements in the forloop are repeated several times • For each iteration of the loop, the search item is compared with an element in the list • When analyzing a search algorithm, you count the number of comparisons • Suppose that L is a list of length n • The number of key comparisons depends on where in the list the search item is located Java Programming: Program Design Including Data Structures

  17. Sequential Search Analysis (continued) • Best case • The item is the first element of the list • You make only one key comparison • Worst case • The item is the last element of the list • You make n key comparisons • What is the average case Java Programming: Program Design Including Data Structures

  18. Sequential Search Analysis (continued) • To determine the average case • Consider all possible cases • Find the number of comparisons for each case • Add them and divide by the number of cases • Average case • On average, a successful sequential search searches half the list Java Programming: Program Design Including Data Structures

  19. Binary Search public int binarySearch(T[] list, int length, T searchItem) { int first = 0; int last = length - 1; int mid = -1; boolean found = false; while (first <= last && !found) { mid = (first + last) / 2; Comparable<T> compElem = (Comparable<T>) list[mid]; Java Programming: Program Design Including Data Structures

  20. Binary Search (continued) if (compElem.compareTo(searchItem) == 0) found = true; else if (compElem.compareTo(searchItem) > 0) last = mid - 1; else first = mid + 1; } if (found) return mid; else return -1; }//end binarySearch Java Programming: Program Design Including Data Structures

  21. Binary Search Example Figure 18-1 Sorted list for a binary search Table 18-1 Values of first, last, and middle and the Number of Comparisons for Search Item 89 Java Programming: Program Design Including Data Structures

  22. Performance of Binary Search • Suppose that L is a sorted list of size n • And n is a power of 2 (n = 2m) • After each iteration of the forloop, about half the elements are left to search • The maximum number of iteration of the for loop is about m + 1 • Also m = log2n • Each iteration makes two key comparisons • Maximum number of comparisons: 2(m + 1) Java Programming: Program Design Including Data Structures

  23. Comparison: Linear vs BinaryWorst case number of comparison Java Programming: Program Design Including Data Structures

  24. Asymptotic Analysis: Motivation • So far, we have counted operations exactly • We don’t really care about the details • Computers execute billions of operations per second • A few here or there is negligible • Care about overall scalability • As the input size grows, does computation grow quickly or slowly? • Don’t lose the forest for the trees Java Programming: Program Design Including Data Structures

  25. Asymptotic Analysis • Asymptotic means the study of the function f as n becomes larger and larger without bound • Consider functions g(n) = n2 and f(n) = n2 + 4n + 20 • As n becomes larger and larger, the term 4n + 20 in f(n) becomes insignificant • g(1000) = 1,000,000 and f(1000) = 1,004,020 • You can predict the behavior of f(n) by looking at the behavior of g(n) Java Programming: Program Design Including Data Structures

  26. Asymptotic Algorithm Analysis • Identify a function that describes the growth in runtime as the input gets large • An “upper bound” of sorts on the running time • Typically worst-case, but occasionally average case • Describe the number of operations done using a function • Focus only on most important operations • Ignore one time initializations, etc. Java Programming: Program Design Including Data Structures

  27. Common Asymptotic Functions Table 18-4 Growth Rate of Various Functions Java Programming: Program Design Including Data Structures

  28. Common Functions, visual Figure 18-9 Growth Rate of Various Functions Java Programming: Program Design Including Data Structures

  29. Java Programming: Program Design Including Data Structures

  30. Asymptotic Notation: Big-O Notation (continued) Table 18-7 Some Big-O Functions That Appear in Algorithm Analysis Java Programming: Program Design Including Data Structures

  31. Big-Oh Notation (Definition) A function f(n) is O(g(n)) if there exist positive constants c and n0 such that: f(n) ≤ cg(n) for all n ≥ n0 Java Programming: Program Design Including Data Structures

  32. Big-Oh Notation • Translation: After some point, f(n) is always smaller than g(n) • “Some point” refers to increasing problem size • The constant c says that we don’t care about multipliers • So, 2n and n have the same essential growth rate • 2n is O(n) Java Programming: Program Design Including Data Structures

  33. Asymptotic Notation: Big-O Notation (continued) Table 18-8 Number of Comparisons for a List of Length n Java Programming: Program Design Including Data Structures

More Related