The Time Complexity of an Algorithm

1. CSE 3101Z Design and Analysis of Algorithms The Time Complexity of an Algorithm Specifies how the running time depends on the size of the input.

2. Purpose • To estimate how long a program will run. • To estimate the largest input that can reasonably be given to the program. • To compare the efficiency of different algorithms. • To help focus on the parts of code that are executed the largest number of times. • To choose an algorithm for an application. COSC 3101, PROF. J. ELDER

3. Time Complexity Is a Function • Specifies how the running time depends on the size of the input. • A function mapping “size” of input “time” T(n) executed . COSC 3101, PROF. J. ELDER

4. Example: Insertion Sort COSC 3101, PROF. J. ELDER

5. Example: Insertion Sort COSC 3101, PROF. J. ELDER

6. Example: Insertion Sort COSC 3101, PROF. J. ELDER

7. Example: Merge Sort COSC 3101, PROF. J. ELDER

8. COSC 3101, PROF. J. ELDER

9. Example: Merge Sort COSC 3101, PROF. J. ELDER

10. Example: Merge Sort COSC 3101, PROF. J. ELDER

11. Time Input Size Relevant Mathematics: Classifying Functions

12. Assumed Knowledge See J. Edmonds, How to Think About Algorithms (HTA): http://www.cse.yorku.ca/~jeff/courses/3101/ Chapter 22. Existential and Universal Quantifiers Chapter 24. Logarithms and Exponentials COSC 3101, PROF. J. ELDER

13. Classifying Functions Describing how fast a function grows without going into too much detail.

14. Which are more alike? COSC 3101, PROF. J. ELDER

15. Which are more alike? Mammals COSC 3101, PROF. J. ELDER

16. Which are more alike? COSC 3101, PROF. J. ELDER

17. Which are more alike? Dogs COSC 3101, PROF. J. ELDER

18. Classifying Animals Vertebrates Fish Reptiles Mammals Birds Dogs Giraffe COSC 3101, PROF. J. ELDER

19. T(n) 10 100 1,000 10,000 log n 3 6 9 13 n1/2 3 10 31 100 10 100 1,000 10,000 n log n 30 600 9,000 130,000 n2 100 10,000 106 108 n3 1,000 106 109 1012 2n 1,024 1030 10300 103000 Classifying Functions n n Note: The universe is estimated to contain ~1080 particles. COSC 3101, PROF. J. ELDER

20. Which are more alike? n1000 n2 2n COSC 3101, PROF. J. ELDER

21. End of Lecture 1 Wed, March 4, 2009

22. Which are more alike? n1000 n2 2n Polynomials COSC 3101, PROF. J. ELDER

23. Which are more alike? 1000n2 3n2 2n3 COSC 3101, PROF. J. ELDER

24. Which are more alike? 1000n2 3n2 2n3 Quadratic COSC 3101, PROF. J. ELDER

25. Classifying Functions? Functions COSC 3101, PROF. J. ELDER

26. 25n n5 2 2 Classifying Functions Functions Exp Polynomial Exponential Logarithmic Constant Poly Logarithmic Double Exponential 5 5 log n (log n)5 n5 25n COSC 3101, PROF. J. ELDER

27. Classifying Functions? Polynomial COSC 3101, PROF. J. ELDER

28. Classifying Functions Polynomial Cubic 4th Order Quadratic Linear 5n4 5n3 5n 5n2 COSC 3101, PROF. J. ELDER

29. 25n n5 2 2 Classifying Functions Functions Exp Polynomial Exponential Logarithmic Constant Poly Logarithmic Double Exponential 5 5 log n (log n)5 n5 25n COSC 3101, PROF. J. ELDER

30. Properties of the Logarithm COSC 3101, PROF. J. ELDER

31. Differ only by a multiplicative constant. Logarithmic • log10n = # digits to write n • log2n = # bits to write n = 3.32 log10n • log(n1000) = 1000 log(n) COSC 3101, PROF. J. ELDER

32. 25n n5 2 2 Classifying Functions Functions Exp Polynomial Exponential Logarithmic Constant Poly Logarithmic Double Exponential 5 5 log n (log n)5 n5 25n COSC 3101, PROF. J. ELDER

33. Poly Logarithmic (log n)5 = log5 n COSC 3101, PROF. J. ELDER

34. (Poly)Logarithmic << Polynomial For sufficiently large n log1000 n << n0.001 COSC 3101, PROF. J. ELDER

35. Classifying Functions Polynomial Cubic 4th Order Quadratic Linear 5n4 5n3 5n 5n2 COSC 3101, PROF. J. ELDER

36. Linear << Quadratic For sufficiently large n 10000 n << 0.0001 n2 COSC 3101, PROF. J. ELDER

37. 25n n5 2 2 Classifying Functions Functions Exp Polynomial Exponential Logarithmic Constant Poly Logarithmic Double Exponential 5 5 log n (log n)5 n5 25n COSC 3101, PROF. J. ELDER

38. Polynomial << Exponential For sufficiently large n n1000 << 20.001 n COSC 3101, PROF. J. ELDER

39. 25n n5 2 2 Ordering Functions Functions Exp Polynomial Exponential Logarithmic Constant << << << << << << Poly Logarithmic Double Exponential 5 5 log n (log n)5 n5 25n << << << << << << COSC 3101, PROF. J. ELDER

40. Yes Yes Yes Yes Yes No Which Functions are Constant? • 5 • 1,000,000,000,000 • 0.0000000000001 • -5 • 0 • 8 + sin(n) COSC 3101, PROF. J. ELDER

41. 9 Lies in between 7 Yes Which Functions are “Constant”? The running time of the algorithm is a “Constant” It does not depend significantly on the size of the input. Yes • 5 • 1,000,000,000,000 • 0.0000000000001 • -5 • 0 • 8 + sin(n) Yes Yes No No COSC 3101, PROF. J. ELDER

42. Which Functions are Quadratic? • n2 • … ? COSC 3101, PROF. J. ELDER

43. Which Functions are Quadratic? • n2 • 0.001 n2 • 1000 n2 Some constant times n2. COSC 3101, PROF. J. ELDER

44. Which Functions are Quadratic? • n2 • 0.001 n2 • 1000 n2 • 5n2 + 3n + 2log n COSC 3101, PROF. J. ELDER

45. Which Functions are Quadratic? • n2 • 0.001 n2 • 1000 n2 • 5n2 + 3n + 2log n Lies in between COSC 3101, PROF. J. ELDER

46. Which Functions are Quadratic? • n2 • 0.001 n2 • 1000 n2 • 5n2 + 3n + 2log n Ignore low-order terms Ignore multiplicative constants. Ignore "small" values of n. Write θ(n2). COSC 3101, PROF. J. ELDER

47. Which Functions are Polynomial? • n5 • … ? COSC 3101, PROF. J. ELDER

48. Which Functions are Polynomial? • nc • n0.0001 • n10000 nto some constant power. COSC 3101, PROF. J. ELDER

49. Which Functions are Polynomial? • nc • n0.0001 • n10000 • 5n2 + 8n + 2log n • 5n2 log n • 5n2.5 COSC 3101, PROF. J. ELDER

50. Which Functions are Polynomial? • nc • n0.0001 • n10000 • 5n2 + 8n + 2log n • 5n2 log n • 5n2.5 Lie in between COSC 3101, PROF. J. ELDER