1 / 40

CS2006 - Data Structures I

CS2006 - Data Structures I. Chapter 10 Algorithm Efficiency & Sorting I. Topics. Algorithm analysis & efficiency Big-O Searching Linear search Binary search. Introduction. Important Questions we ask ourselves about algorithm How to measure the cost of a program?

Anita
Download Presentation

CS2006 - Data Structures I

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. CS2006 - Data Structures I Chapter 10 Algorithm Efficiency & Sorting I

  2. Topics • Algorithm analysis & efficiency • Big-O • Searching • Linear search • Binary search

  3. Introduction • Important Questions we ask ourselves about algorithm • How to measure the cost of a program? • How fast does an algorithm run? • Is this algorithm faster than that one?

  4. Introduction • Cost of computer programs: • It is not always possible to minimize both time & space requirements • How to measure the efficiency? • What will affect the time used by the program?

  5. Introduction • Two approaches to measure the efficiency • Analysis • Usually done with algorithm • Count the number of elementary operations • Assume all these operations take the same amount of time • Benchmarking • How?

  6. Introduction • Algorithm analysis: • Focuses on significant differences in algorithm efficiency, that are likely to dominate the overall cost of a solution • Should be independent of specific implementations, computers, & data • Since none of the algorithms covered in this course has significant space requirements, we will focus on time efficiency

  7. Introduction • Big-O: • Stands for “Order of Magnitude” • A theoretical measure of execution of algorithms • Only an approximation • Smaller factors are usually ignored • Given: • Problem size N (usually measured by the number of items processed) • Required: • Amount of resources consumed (usually time and/or memory)

  8. Introduction • Big-O Examples: • Sequential searching of a list: • Average time depends on the number of elements in the list.

  9. Linear Search boolean linearSearch (int key, int data [], int n) { for(int i = 0 ; i < n ; i++) if (data[I]==key) return true; } • Involves analysis of the number of loop iterations • Operations: • Comparison of I to n • Increment I • Compare array value to key • Total 3

  10. Linear Search • Number of operations: • Worse case: n*3 • Best case: 3 • Average case: (n*3)/2

  11. Big - O Notation • Full definition of Big-O notation: • Let f and g be functions on the non-negative integers. • Then f(n) is of order at most g(n) : • denoted f(n) = O(g(n)) • This means that there exists a positive constant C such that: • |f(n)| <= C(g(n)) • for all sufficiently large positive integers n. For all n>k

  12. Efficiency • Example: Linked List Traversal  Node cur = head;  1 assignment (a) while ( cur != NULL )  n+1 comparisons (c) { System.out.print(cur.item);  n writes (w) cur = cur.next;  n assignments (a) } / / end while • The statement requires the following time units: (n+1) * (a + c) + n * w = (a+c) +(a+c+w) * n  proportional to n • Time required to write N nodes is proportional to N • We only need an approximate count of the instructions executed

  13. Efficiency • Example: Find average of N numbers Sum = 0.0; // 1 time assignment i = 0; // 1 time assignment while ( i < n) // n+1 times comparison { sum += a[ i ]; // n times assignment i += 1; // n times assignment } ave = sum / i; // 1 time assignment • Assumption: • Time needed by both assignment & comparison is the same • Efficiency ~ 3n+4 ~ O(n) for large n

  14. Efficiency • Big O Notation (Order of magnitude) • If an algorithm A requires time proportional to g(n), we say that the algorithm is of order g(n) , or O(g(n)) • g(n) is called the algorithm's growth-rate function • Usually only large values of n >= n0 are considered

  15. Algorithm A (n 2 / 5) Seconds Algorithm B (5 * n) 25 N Efficiency • Big O Notation • Time requirement as function of problem size n  • For n < 25, A is better • For n> 25, B is better

  16. Algorithm A (3 * n 2) Seconds Algorithm B ( n 2 – 3 * n * 10) 1 2 3 N Efficiency • Big O Notation • Example: • For n ≥ 2, algorithm A is better • For n > 2, algorithm B is better

  17. Efficiency • Big O Notation • Properties of Growth-Rate Functions • Ignore low-order terms O(n3 + 4 * n2 + 3 * n)  O(n3 ) • Ignore multiplication constant in the high-order term of an algorithm's growth-rate function O( 5 * n3 )  O(n3 ) • Combine growth-rate functions O( f(n) ) + O( g(n) )  O( f(n) + g(n) ) O ( n3 ) + O ( n )  O ( n3 + n )  O ( n3 ) O ( n Log n + n )  O ( n ( log n + 1 ))  O ( n log n )

  18. Efficiency • Common Growth Rate Functions • Constant:1 • Constant time requirements • Time independent of problem’s size • Logarithmic: log2n • Time increases slowly with problem’s size • Time doubles when size is squared • The base of the log doesn’t affect growth rate • Example?

  19. Efficiency • Common Growth Rate Functions • Linear : n • Time increases directly with problem’s size • Time squares when size is squared • n log n: n * log2n • Time increases rapidly than the linear • Typical algorithms divide the problem into smaller problems that are solved separately • Examples: Merge-Sort • Quadratic: n2 • Time increases rapidly with problem’s size • Algorithms use two nested loops • Practical for small problems only • Examples: Selection sort

  20. Efficiency • Common Growth Rate Functions • Cubic: n3 • Time increases more rapidly with problem’s size • Practical for small problems only • Exponential: 2n • Time increases too rapidly with problem’s size • Impractical

  21. Efficiency • Order of growth of some common functions O(1) < O( log2 n) < O( n ) < O( n*log2 n) < O(n2) < O(n3) < O(2n) Function/n10 100 1,000 10,000 100,000 1,000,000 1 1 1 1 1 1 1 log2n 3 6 9 13 16 19 n 10 102 103 104 105 106 . . . 2n 103 1030 10301 103010 1030103 10301030

  22. Graph of Growth Rate Functions

  23. Efficiency Analysis • Evaluation can be calculated for • Best case • Shortest possible time to find an item • Not very informative • Average case • More difficult to perform than worst-case analysis • Worst case • The maximum amount of time that an algorithm can require to solve a problem of size N • An algorithm can require different times to solve different problems of the same size

  24. Efficiency Analysis • Example: ListRetrieve • Array-based implementation: • nth item can be accessed directly O(1) • LL-based implementation: • nth item needs n steps to be retrieved O(n)

  25. Efficiency Analysis • Example: ListRetrieve • Array-based implementation: • nth item can be accessed directly O(1) • LL-based implementation: • nth item needs n steps to be retrieved O(n)

  26. Efficiency Analysis • When choosing ADT's implementation consider • Frequency • Consider how frequently particular ADT operations occur in a given application. • Example: • Word-processor's spelling checker frequently retrieves, but rarely inserts or deletes items. • Critical operations • Some rarely-used but critical operations must be efficient.

  27. Searching & Sorting • Searching: • Looking for a specific item among a collection of data • Sorting: • Organizing a collection of data into either ascending or descending order according to a specific key • Sorting is often performed as an initialization step for certain searching algorithms

  28. Searching & Sorting • Searching algorithms • Linear (Sequential) search • Binary search • Hashing (Chapter 13) • Sorting algorithms • Selection sort • Bubble sort • Insertion sort • Merge sort • Quick sort • Radix (Bucket) sort • Hashing (Chapter 13)

  29. Sequential (Linear) Search • Analysis • Efficiency = 3 n + 2 • Best case: • O(1) -- when? • Worst case: • O(n) ---- when? • Average case • O(n/2)  O(n) boolean linearSearch (int key, int data [], int n) { for(int i = 0 ; i < n ; i++) if (data[I]==key) return true; }

  30. Binary Search • Elements should be sorted. • Much better than sequential search for large arrays. • Idea: • Divide the array in half. • Search only one of the halves. • Continue until: • Success: • Find the desired item. • Failure: • Reach an array of one data element that isn't the desired value.

  31. Binary Search public static int binarySearch(Comparable[ ] anArray, Comparable target) { int index; int left = 0, right = anArray.length - 1; while( left <= right ) { index = (left + right) / 2; if( target.compareTo(anArray[index]) == 0) return index; if( target.compareTo(anArray[index]) > 0) left = index + 1; else right = index - 1; } return -1;// target not found }

  32. 32 16 8 4 2 1 0 Binary Search Analysis • Worst case - target is not found while( left <= right ) { index = (left + right) / 2; • On each loop iteration the array is (ideally) divided in half. • To determine f(n) consider number of items left in the array on each iteration: Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 Pass 6 Pass 7

  33. 32 16 8 4 2 1 0 Pass 1 Pass 2 Pass 3 Pass 4 Pass 5 Pass 6 Pass 7 Binary Search Analysis Number of passes log2n n 5 7 32 4 6 16 3 5 8 2 4 4 1 3 2 ... 2 1

  34. Binary Search Analysis • The relation between n and number of passes is: f(n) = log2n + 2 • And the time complexity is: O(log2n)

  35. Review • Algorithm analysis should be independent of all of the following EXCEPT ______. • the programming style used in the implementation of the algorithm • the computer used to run a program which implements an algorithm • the number of significant operations in an algorithm • the test data used to test a program which implements an algorithm

  36. Review • Assuming a linked list of n nodes, the code fragment: Node curr = head; while (curr != null) { System.out.println(curr.getItem()); curr.setNext(curr.getNext()); } // end while requires ______ comparisons. • n • n – 1 • n + 1 • 1

  37. Review • Which of the following can be used to compare two algorithms? • growth rates of the two algorithms • implementations of the two algorithms • test data used to test programs which implement the two algorithms • computers on which programs which implement the two algorithms are run

  38. Review • Given the statement:Algorithm A requires time proportional to f(n) Algorithm A is said to be ______. • in class f(n) • of degree f(n) • order f(n) • equivalent to f(n)

  39. Review • If a problem of size n requires time that is directly proportional to n, the problem is ______. • O(1) • O(n) • O(n2) • O(2n)

  40. Review • The value of which of the following growth-rate function grows the fastest? • O(n) • O(n2) • O(1) • O(log2n)

More Related