Discrete Mathematics Summer 2006

1. Discrete Mathematics Summer 2006 By Dan Barrish-Flood originally for Fundamental Algorithms For use by Harper Langston in D.M.

2. What is an Algorithm? • A problem is a precise specification of input and desired output • An algorithm is a precise specification of a method for solving a problem.

3. What We Want in an Algorithm • Correctness • Always halts • Always gives the right answer • Efficiency • Time • Space • Simplicity

4. A Typical Problem Sorting • Input: • A sequence b1, b2, ... bn • Output: • A permutation of that sequence b1, b2, ... bnsuch that

5. A Typical Algorithm

6. Running Time tJ is the number of times the “while” loop test in line 5 is executed for that value of j. Total running time is c1 (n) + c2(n-1) +…+c8(n-1)

7. A Better Way to Estimate Running Time • Trying to determine the exact running time is tedious, complicated and platform-specific. • Instead, we’ll bound the worst-case running time to within a constant factor for large inputs. • Why worst-case? • It’s easier. • It’s a guarantee. • It’s well-defined. • Average case is often no better.

8. Insertion Sort, Redux • Outer loop happens about n times. • Inner loop is done j times for each j, worst case. • j is always < n • So running time is upper-bounded by about “Insertion sort runs in q(n2) time, worst-case.”

9. We’ll make all the “about”s and “rough”s precise. • These so-called asymptotic running times dominate constant factors for large enough inputs.

10. Functions • constants • polynomials • exponentials • logarithms and polylogarithms • factorial

11. Polynomials

12. f(n) is monotonically increasing iff I.e. It never “goes down” — always increases or stays the same. For strictly increasing, replace £ in the above with <. I.e. The function is always going up.

13. Are constant functions monotonically increasing? Yes! (But not strictly increasing.) If , then is strictly increasing. So even increases forever to

14. Exponentials

15. Logarithms

16. n = b log b n (definition) logb(xy)=logbx + logby logban= nlogba logbx = logcx / logcb logb(1/a) = -logba logba = 1/(logab) alogbc= clogba if b > 1, then logbn is strictly increasing. Log Facts

17. More About Logs A polylogarithm is just

18. Factorial

19. More About Monotonically Increasing Functions

20. Bounding Functions • We are more interested in finding upper and lower bounds on functions that describe the running times of algorithms, rather than finding the exact function. • Also, we are only interested in considering large inputs: everything is fast for small inputs. • So we want asymptotic bounds. • Finally, we are willing to neglect constant factors when bounding. They are tedious to compute, and are often hardware-dependent anyway.

21. Example

22. Notation for Function Bounds

23. Big-Oh Facts • The “=” is misleading. E.g • It denotes an upper bound, not necessarily a tight bound. Thus, is meaningless. • Transitivity:

24. Examples We’ve seen that 3n + 8 = O(n). Is 10n = O(3n + 8)? Yes: choose c = 10, n0 = 1: 10n£ 30n+ 80

25. More Examples 2n2 + 5n - 8 = O(n2)? Yes: 2n2 + 5n - 8 £ cn2 ¸n2: 2 + 5/n - 8/n2£c Choose c = 3; subtracting 2: 5/n - 8/n2£ 1 Choose no = 5

26. More Examples

27. Lower Bound Notation

28. “Tight” Bound Notation

29. More Facts

30. Examples

31. Another Upper-Bound Notation

32. Functions and Bounds

33. Logarithms

34. Polynomials

35. Comparing Functions