8. Comparison of Algorithms

1. 8. Comparison of Algorithms Yan Shi CS/SE 2630 Lecture Notes Part of this note is from “C++ Plus Data Structure” textbook slides

2. Map to Joe’s Diner

3. Which Cost More to Feed?

4. How efficient is an algorithm? • Efficiency concerns: • CPU usage (time complexity) • memory usage (space complexity) • disk usage • network usage We discuss time complexity in this course.

5. Order of Magnitude of a Function • Theorder of magnitude, orBig-O notation,of a function expresses the computing time of a problem as the term in a function that increases most rapidly relative to the size of a problem. • Big-O is used to express time complexity of an algorithm

6. Example • How many time do we add to sum? • outer loop repeat n times • each iteration add i times to sum • 0+1+2+…+n-1 = n(n-1)/2 • The time required by an algorithm is proportional to the number of “basic operations” that it performs! • Big-O: measure growth • O( 0.5n2 – 0.5n ) = O( n2– n ) = O(n2) int sum = 0, n = 100; for(int i = 0; i < n; ++i) for(int j = 0; j < i; ++j) sum += i * j;

7. Comparison of Two Algorithms sum of consecutive integers 1 to 20 (n) O(1) O(n)

8. Types of Complexity • Best case complexity: • related to minimum # of steps required by an algorithm, given an ideal set of inputs • Worst case complexity: • related to maximum # of steps required by an algorithm, given the worst possible set of inputs • Average case complexity: • related to the average number of steps required by an algorithm, calculated across all possible sets of inputs this is the one we usually use.

9. Big-O Formal Definition • A function T(N) is O(F(N)) if for some constant c and for all values of N greater than some value n0: T(N) <= c * F(N) • Example: T(N) = 4*N2+7  it is O(N2) why? when c = 5, n0 = 2, T(N) <= c * F(N)

10. Finding “John Smith” Best case: O(1) How about algorithm 2? Worst case: O(N) Average case: O(N) average # of steps: (1+2+…+N)/N = (N+1)/2

11. Binary Search Example • Look for 1 in sorted list 1-64. [1-32]  [1-16][1-8][1-4][1-2][1] • we need to look 6 times: 26 = 64 • this is the worst case. • In the worst case, look for an item in a sorted list of size N, we need to look k times • 2k = N  k = log2N  O(log2N)!

12. Names of Orders of Magnitude O(1) bounded (by a constant) time O(log2N) logarithmic time O(N) linear time O(N*log2N) N*log2N time O(N2) quadratic time O( 2N ) exponential time

13. 1 0 0 1 2 2 1 2 4 4 4 2 8 16 16 8 3 24 64 256 16 4 64 25665,536 32 5 160 10244,294,967,296 64 6 384 4096 128 7 896 16,384 N log2N N*log2N N22N Comparison of Orders

14. Comparison of Orders http://www.objc.io/issue-7/collections.html

15. How to Determine Complexities • Sequence of statements: total time = time(statement 1) + time(statement 2) + ... + time(statement k) statement 1; statement 2; ... statement k;

16. How to Determine Complexities • Branch: total time = max(time(sequence 1), time(sequence 2)) if (condition) { sequence of statements 1 } else { sequence of statements 2 }

17. How to Determine Complexities • Loops: total time = N*time(sequence) How about nested loops? for (i = 0; i < N; i++) { sequence of statements }

18. Exercise: what is the Big-O? for ( i = 0 ; i < n ; i++ ) { subtotal = 0; if ( flag == true ) { for( j=0 ; j < i; j++) subtotal += j; tot += subtotal; } else subtotal = -1; } total time = n * ( 1 + max( ( ( 1+2+...+n-1 ) + 1 ), 1 ) )  O(n2)

19. Exercise • Array based sorted list: • find(linear search): • insert: • delete: • Sorted linked list: • find: • insert: • delete: O(n) O(n)+O(n) = O(n) O(n)+O(n) = O(n) O(n) O(n)+O(1) = O(n) O(n)+O(1) = O(n) How about comparing sorted and unsorted list?