Discrete Mathematics CS 2610

1. Discrete Mathematics CS 2610 September 24, 2008

2. Greedy Algorithms • Greedy algorithms make the “best” choice at each step of the algorithm. • Greedy algorithms do not “backtrack,” or reconsider previous decisions. • Example: Making change in the fewest number of coins (US currency). • Greedy algorithm: Starting with largest coins, use as many as possible before moving to next largest denomination.

3. Greedy Algorithms • Problem: Assign meeting to conference rooms • Policy: In decreasing order of room capacity, assign a meeting to the next largest available room Meeting 1: 70 Meeting 2: 46 Meeting 3: 125 Meeting 4: 110 Meeting 5: 30 Meeting 6: 87 Room A 200 Room B 150 Room C 150 Room D 100 Room E 75 Room F 50 M1, OK M2, OK M3, OK X M5, OK X This is a suboptimal algorithm!

4. Greedy Algorithms • Problem: Assign meeting to conference rooms • Policy: In ascending order of room capacity, assign the largest remaining meeting that can be held in the room. Meeting 1: 70 Meeting 2: 46 Meeting 3: 125 Meeting 4: 110 Meeting 5: 30 Meeting 6: 87 Room F 50 Room E 75 Room D 100 Room B 150 Room C 150 Room A 200 M2, OK M1, OK M6, OK M3, OK M4, OK M5, OK Just one of many algorithms.

5. Order of Growth Terminology O(1) Constant O(log n) Logarithmic (c  v) O(logc n) Polylogarithmic (c  Z+) O(n) Linear O(nc) Polynomial (c  Z+) O(cn) Exponential (c  Z+) O(n!) Factorial Best Worst

6. Complexity of Problems • Tractable: A problem that can be solved with a deterministic polynomial (or better) worst-case time complexity. • Also denoted as P • ExampleS: • Search Problem • Sorting problem • Find the maximum

7. Complexity of Problems • Intractable: Problems that are not tractable. • Example: Traveling salesperson problem 􀂄 • Wide use of greedy algorithms to get an approximate solution. • For example under certain circumstances, you can get an approximation that is at most double the optimal solution.

8. P vs. NP • NP: Solvable problems whose solution can be checkedin polynomial time. • P  NP • The most famous unproven conjecture in computer science is that this inclusion is proper. • 􀂄 P  NP rather than P = NP

9. Complexity of Problems • Not Solvable: Proven to have no algorithm that computes it • Example: Halting problem (Alan Turing) Determine whether an arbitrary given algorithm, will eventually halt for any given finite input. • Corollary: The question of whether or not a program halts for a given input is unsolvable.

10. Big-O Notation • Big-O notation is used to express the time complexity of an algorithm. • It is a statement about an algorithm, not a machine! 􀂄 • We can assume that any operation requires the same amount of time. • The time complexity of an algorithm can be described independently of the software and hardware used to implement the algorithm.

11. Big-O Notation • Def.: Let f , g be functions with domain R≥0 or N and codomain R. f(x) is O(g(x)) if there are constants C and k s.t. x > k, |f (x)| ≤ C ⋅ |g (x)| • We say f (x) is asymptotically dominated by g (x) • C|g(x)| is an upper bound of f(x). • C and k are called witnesses to the relationship between f & g.

12. Big-O Notation Note that |C ⋅ g(x)| does not need to be greater than |f(x)| for all x, only for all x > k for some k.

13. Big-O Notation • To prove that a function f(x) is O(g(x)) • Find values for k and C- not necessarily the smallest one, larger values also work!! • It is sufficient to find a certain k and C that works • In many cases, for all x ≥ 0, if f(x) ≥ 0 then |f(x)| = f(x) Example: f(x) = x2 + 2x + 1 is O(x2) for C = 4 and k = 1

14. Big-O Notation Show that f(x) = x2 + 2x + 1 is O(x2). When x > 1 we know that x ≤ x2 and 1 ≤ x2 then 0 ≤ x2 + 2x + 1 ≤ x2 + 2 x2 + x2 = 4 x2 so, let C = 4 and k = 1 as witnesses, i.e., f(x) = x2 + 2x + 1 < 4 x2 when x > 1 Could try x > 2. Then we have 2x ≤ x2 & 1 ≤ x2 then 0 ≤ x2 + 2x + 1 ≤ x2 + x2 + x2 = 3 x2 so, C = 3 and k = 2 are also witnesses to f(x) being O(x2). Note that f(x) is also O(x3), etc.

15. Big-O Notation Show that f(x) = 7 x2 is O(x3). When x > 7 we know that 7 x2 < x3 (multiply x > 7 by x2) so, let C = 1 and k = 7 as witnesses. Could try x > 1. Then we have 7 x2 < 7 x3 so, C = 7 and k = 1 are also witnesses to f(x) being O(x3). Note that f(x) is also O(x4), etc.

16. Big-O Notation Show that f(n) = n2 is not O(n). Show that no pair of C and k exists such that n2 ≤ Cn whenever n > k. When n > 0, divide both sides of n2 ≤ Cn by n to get n ≤ C. No matter what C and k are, n ≤ C will not hold for all n with n > k! (There is no bound on the input.)

17. Big-O Notation • Observe that g(x) = x2 is O(x2 + 2x + 1) • Def: Two functions f(x) and g(x) have the same order iff g(x) is O(f(x)) and f(x) is O(g(x))

18. Big-O Notation • Also, the function f(x) = 3 x2 + 2x + 3 is O(x3) • What about O(x4) ? • In fact, the function Cg(x) is an upper bound for f(x), but not necessarily the tightest bound. • When Big-O notation is used, g(x) is usually chosen to be as small as possible. This way it carries the most information possible.