RAIK 283: Data Structures & Algorithms

1. RAIK 283: Data Structures & Algorithms Analysis of Algorithms Dr. Ying Lu ylu@cse.unl.edu August 28, 2012 http://www.cse.unl.edu/~ylu/raik283/ Design and Analysis of Algorithms Chapter 2.1

2. RAIK 283: Data Structures & Algorithms • Giving credit where credit is due: • Most of the lecture notes are based on the slides from the Textbook’s companion website http://www.aw-bc.com/info/levitin • Several slides are from Jeff Edmonds of the York University • I have modified them and added new slides Design and Analysis of Algorithms Chapter 2.1

3. The Time Complexity of an Algorithm

4. The Time Complexity of an Algorithm Specifies how the running time depends on the size of the input

5. Purpose Design and Analysis of Algorithms Chapter 2.1

6. Purpose • To estimate how long a program will run. • To estimate the largest input that can reasonably be given to the program. • To compare the efficiency of different algorithms. • To help focus on the parts of code that are executed the largest number of times. • To choose an algorithm for an application. Design and Analysis of Algorithms Chapter 2.1

7. Purpose (Example) • Suppose a machine that performs a million floating-point operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50? • 1) If the algorithm requires n2 such operations: Design and Analysis of Algorithms Chapter 2.1

8. Purpose (Example) • Suppose a machine that performs a million floating-point operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50? • 1) If the algorithm requires n2 such operations: • 0.0025 second Design and Analysis of Algorithms Chapter 2.1

9. Purpose (Example) • Suppose a machine that performs a million floating-point operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50? • 1) If the algorithm requires n2 such operations: • 0.0025 second • 2) If the algorithm requires 2n such operations: • A) Takes a similar amount of time (t < 1 sec) • B) Takes a little bit longer time (1 sec < t < 1 year) • C) Takes a much longer time (1 year < t) Design and Analysis of Algorithms Chapter 2.1

10. Purpose (Example) • Suppose a machine that performs a million floating-point operations per second (106 FLOPS), then how long an algorithm will run for an input of size n=50? • 1) If the algorithm requires n2 such operations: • 0.0025 second • 2) If the algorithm requires 2n such operations: • over 35 years! Design and Analysis of Algorithms Chapter 2.1

11. Time Complexity Is a Function Specifies how the running time depends on the size of the input. A function mapping “size” of input “time” T(n) executed . Design and Analysis of Algorithms Chapter 2.1

12. Definition of Time? Design and Analysis of Algorithms Chapter 2.1

13. Definition of Time • # of seconds (machine, implementation dependent). • # lines of code executed. • # of times a specific operation is performed (e.g., addition). Design and Analysis of Algorithms Chapter 2.1

14. input size running time Number of times basic operation is executed execution time for basic operation Theoretical analysis of time efficiency Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size • Basic operation: the operation that contributes most towards the running time of the algorithm. T(n) ≈copC(n) Design and Analysis of Algorithms Chapter 2.1

15. Input size and basic operation examples Design and Analysis of Algorithms Chapter 2.1

16. Input size and basic operation examples Design and Analysis of Algorithms Chapter 2.1

17. Input size and basic operation examples Design and Analysis of Algorithms Chapter 2.1

18. Input size and basic operation examples Design and Analysis of Algorithms Chapter 2.1

19. Input size and basic operation examples Design and Analysis of Algorithms Chapter 2.1

20. Theoretical analysis of time efficiency Time efficiency is analyzed by determining the number of repetitions of the basic operation as a function of input size Design and Analysis of Algorithms Chapter 2.1

21. Which Input of size n? Efficiency also depends on the particular input • For instance: search a key in a list of n letters • Problem input: a list of n letters • How many different inputs? Design and Analysis of Algorithms Chapter 2.1

22. Which Input of size n? Efficiency also depends on the particular input For instance: search a key in a list of n letters There are 26n inputs of size n.Which do we considerfor the time efficiency C(n)? Design and Analysis of Algorithms Chapter 2.1

23. Best-case, average-case, worst-case • Worst case: W(n) – maximum over inputs of size n • Best case: B(n) – minimum over inputs of size n • Average case: A(n) – “average” over inputs of size n • NOT the average of worst and best case • Under some assumption about the probability distribution of all possible inputs of size n, calculate the weighted sum of expected C(n) (numbers of basic operation repetitions) over all possible inputs of size n. Design and Analysis of Algorithms Chapter 2.1

24. Example: Sequential search • Problem: Given a list of n elements and a search key K, find an element equal to K, if any. • Algorithm: Scan the list and compare its successive elements with K until either a matching element is found (successful search) or the list is exhausted (unsuccessful search) • Worst case • Best case • Average case Design and Analysis of Algorithms Chapter 2.1

25. An example • Compute gcd(m, n) by applying the algorithm based on checking consecutive integers from min(m, n) down to gcd(m, n) • Input size? • Best case? • Worst case? • Average case? Design and Analysis of Algorithms Chapter 2.1

26. Types of formulas for basic operation count • Exact formula e.g., C(n) = n(n-1)/2 • Formula indicating order of growth with specific multiplicative constant e.g., C(n) ≈ 0.5 n2 • Formula indicating order of growth with unknown multiplicative constant e.g., C(n) ≈ cn2 Design and Analysis of Algorithms Chapter 2.1

27. Types of formulas for basic operation count • Exact formula e.g., C(n) = n(n-1)/2 • Formula indicating order of growth with specific multiplicative constant e.g., C(n) ≈ 0.5 n2 • Formula indicating order of growth with unknown multiplicative constant e.g., C(n) ≈ cn2 • Most important: Order of growth within a constant multiple as n→∞ Design and Analysis of Algorithms Chapter 2.1

28. Asymptotic growth rate • A way of comparing functions that ignores constant factors and small input sizes • O(g(n)): • Θ (g(n)): • Ω(g(n)): Design and Analysis of Algorithms Chapter 2.1

29. Asymptotic growth rate • A way of comparing functions that ignores constant factors and small input sizes • O(g(n)): class of functions f(n) that grow no faster than g(n) • Θ (g(n)): class of functions f(n) that grow at same rate as g(n) • Ω(g(n)): class of functions f(n) that grow at least as fast as g(n) Design and Analysis of Algorithms Chapter 2.1

30. Table 2.1 Design and Analysis of Algorithms Chapter 2.1

31. Classifying Functions Giving an idea of how fast a function grows without going into too much detail.

32. Which are more alike? Design and Analysis of Algorithms Chapter 2.1

33. Which are more alike? Mammals Design and Analysis of Algorithms Chapter 2.1

34. Which are more alike? Design and Analysis of Algorithms Chapter 2.1

35. Which are more alike? Dogs Design and Analysis of Algorithms Chapter 2.1

36. Classifying Animals Vertebrates Fish Reptiles Mammals Birds Dogs Giraffe Design and Analysis of Algorithms Chapter 2.1

37. Which are more alike? n1000 n2 2n Design and Analysis of Algorithms Chapter 2.1

38. Which are more alike? n1000 n2 2n Polynomials Design and Analysis of Algorithms Chapter 2.1

39. Which are more alike? 1000n2 3n2 2n3 Design and Analysis of Algorithms Chapter 2.1

40. Which are more alike? 1000n2 3n2 2n3 Quadratic Design and Analysis of Algorithms Chapter 2.1

41. Classifying Functions? Functions Design and Analysis of Algorithms Chapter 2.1

42. Classifying Functions Functions Factorial Constant Polynomial Exponential Logarithmic Poly Logarithmic n! 5 5 log n (log n)5 n5 25n Design and Analysis of Algorithms Chapter 2.1

43. Classifying Functions? Polynomial Design and Analysis of Algorithms Chapter 2.1

44. Classifying Functions Polynomial ? Cubic Quadratic Linear 5n4 5n3 5n 5n2 Design and Analysis of Algorithms Chapter 2.1

45. Logarithmic • log10n = # digits to write n • log2n = # bits to write n = 3.32 log10n • log(n1000) = 1000 log(n) Differ only by a multiplicative constant Design and Analysis of Algorithms Chapter 2.1

46. Poly Logarithmic (log n)5 = log5 n Design and Analysis of Algorithms Chapter 2.1

47. Which grows faster? log1000 n n0.001 Design and Analysis of Algorithms Chapter 2.1

48. Poly Logarithmic << Polynomial For sufficiently large n log1000 n << n0.001 Design and Analysis of Algorithms Chapter 2.1

49. Which grows faster? 10000 n 0.0001 n2 Design and Analysis of Algorithms Chapter 2.1

50. Linear << Quadratic For sufficiently large n 10000 n << 0.0001 n2 Design and Analysis of Algorithms Chapter 2.1