Evaluating & Improving Algorithms

1. Evaluating & Improving Algorithms

2. Analyzing US Change While M>0 c <- value of largest coin with value <= M give c coin to customer M <- M-c • Three statements per loop • Loop executes k times, where k is… • Therefore, running time is 3*k

3. Generalized US Change for i <- 0 to d-1 Calculate and give the max number of this coin (c[i]) • How many times does this loop run? (k) • How many operations are in calculate & give max? (x) • Resulting time is k*x

4. General Rules for Calculating Time • Steps in sequence are added • A function is replaced by its # steps (or count each call as 1 step) • Total # of steps in a loop gets multiplied by the # of times the loop executes

5. Summary so far… • Original US change • O(N) where N is # coins given • Generalized change algorithm • O(N) where N is # different coins in system • But this doesn’t work for all cases!

6. Brute Force* Change N <- 1 While (not done) construct a combination of N coins if it adds up to M return it (done=true) else N <- N+1 *Brute Force = try all combinations

7. How many loops? • One loop for every combination with too few coins 1! + 2! + … • One loop for every combination with the right number of coins before the correct one • Best case for this part is 0 • Worst case for this part is k! where k is the number of coins in the solution

8. Summary and Analysis • Original US change • O(N) where N is # coins given (linear) • Generalized change algorithm • O(N) where N is # different coins • O(1) for a fixed coin system (constant) • Brute force change algorithm • O (N * N!) where N is # coins given • (Worse than 2N)!

9. What We’ve Learned • Brute force algorithms (try all possibilities) are guaranteed to work, but… • Brute force algorithms (usually) take too long • We need ways to speed up the work

10. Detour: Recursion • Specify a solution to a problem in terms of solutions to one or more smaller problems • Mathematically elegant

11. Recursive Fibonacci • Mathematical definition • F(0) = 0 • F(1) = 1 • F(N) = F(N-1) + F(N-2) • Implemented directly: • F(4) = F(3)+F(2) = F(2)+F(1)+F(2) = F(1)+F(0)+F(1)+F(2) = 1+1+1+F(2) = 3+F(1)+F(0) = 3+1+1 = 5

12. A Tree of Recursive Calls F(4) F(3) F(2) F(2) F(0) F(1) F(1) 1 1 1 F(1) F(0) 1 1

13. Dynamic Programming • Save intermediate results from which the final result can be computed • Order the computations so you have all saved results before you need them • Example: our Fibonacci Algorithm

14. Dynamic Programming For Fibonacci • Generate answers from smallest to largest • Use previous answers in calculations fibonacci (n) O(n) where n is input f[0] <- 1 f[1] <- 1 for i <- 2 to n-1 n times f[i] <- f[i-1]+f[i-2] 1 step return f[n-1]

15. Recursion Isn’t Always Slower • Recursive US Change (M) if (M>0) c <- value of largest coin with value <= M give c coin to customer Recursive US Change(M-c) • Only one recursive call, at end • This is “tail recursion”, just as fast as the original loop

16. Greedy Algorithms • Order your algorithm to solve the “biggest chunk” first • Example: always give the biggest coin possible in the change problem • Not every greedy algorithm is correct (e.g. generalized change)

17. Branch and Bound • Consider a brute force problem solution as a series of choices • When a single choice is avoided (pruned), all future choices that depend on it are also avoided • AI calls this “heuristic search”

18. Randomized • Make each choice randomly until the problem is solved • Not guaranteed to solve the problem • Technically not even algorithms, if not guaranteed to halt (e.g. by time limit) • Sometimes we can prove that a “good enough” solution is “likely enough” and these are simple to implement

19. Find the Phone (Brute Force) Find the phone: if the phone is at your location return true (phone is found) else for each direction (left, right, forw, back) take a step if (find the phone) return true return false (phone not found here)

20. Find the Phone (Branch & Bound) Find the phone: if the phone is at your location return true (phone is found) else listen to the phone & rule out opposite direction for each remaining direction take a step if (find the phone) return true return false (phone not found here)

21. Find the Phone (Greedy) Find the phone: if the phone is at your location return true (phone is found) else listen & pick best direction (left, right, forw, back) take a step if (find the phone) return true return false (phone not found here) • What if there’s an irreplaceable Ming vase between you and the phone?

22. Find the Phone (Randomized) Find the phone: if the phone is at your location return true (phone is found) else repeat forever pick a random direction (left, right, forw, back) take a step if (find the phone) return true

23. Divide and Conquer • If the problem is small enough, solve it directly • Else • Break the problem down into 2 or more subproblems • Solve each one separately (recursive step) • Combine the solutions • Works best when subproblems are equal size

24. Divide & Conquer Example • Quick Sort • Pick a value from the list (k) • Divide the input into “<k” list and “>k” list • Sort each list separately • Build the result: • Sorted “<k” list followed by k followed by sorted “>=k” list

25. Machine Learning • Given: many instances of input and output • Discover: a function that relates them • Generalize: to previously unseen inputs • Generally need large amounts of representative data • ML algorithms are fairly time consuming

26. Summary of Techniques • Brute Force • Try every possibility • Rarely practical for reasonable size problems • Dynamic programming • Solve subproblems in the right order and save solutions to build the final answer

27. Summary (cont’d) • Greedy Algorithms • Take the “biggest step” first each time. Not always correct. • Branch and Bound • Rule out impossible choices and their successors, but try everything else • Divide and Conquer • Combine results from smaller subproblems

28. Summary (cont’d) • Machine learning • Use large amounts of data to “automatically” generate an appropriate solution algorithm • Randomized algorithms • Try solutions at random until one is found (or “good enough”) • Not always completely random (e.g. genetic algorithms)