Chapter 5 Fundamental Algorithm Design Techniques

1. Chapter 5Fundamental AlgorithmDesign Techniques

2. Overview • The Greedy Method (5.1) • Divide and Conquer (5.2) • Dynamic Programming (5.3)

3. Greedy Outline • The Greedy Method Technique (§5.1) • Fractional Knapsack Problem (§5.1.1) • Task Scheduling (§5.1.2) • Minimum Spanning Trees (§7.3) [future lecture]

4. A New Vending Machine? • Coke machine in the lounge behind the ulab—can we hack it? • Being efficient computer scientists, we want to return the smallest possible number of coins in change

5. The Greedy Method

6. The greedy method is a general algorithm design paradigm, built on the following elements: configurations: different choices, collections, or values to find objective function: a score assigned to configurations, which we want to either maximize or minimize It works best when applied to problems with the greedy-choice property: a globally-optimal solution can always be found by a series of local improvements from a starting configuration. The Greedy Method

7. Problem: A dollar amount to reach and a collection of coin amounts to use to get there. Configuration: A dollar amount yet to return to a customer plus the coins already returned Objective function: Minimize number of coins returned. Greedy solution: Always return the largest coin you can Example 1: Coins are valued $.32, $.08, $.01 Has the greedy-choice property, since no amount over $.32 can be made with a minimum number of coins by omitting a $.32 coin (similarly for amounts over $.08, but under $.32). Example 2: Coins are valued $.30, $.20, $.05, $.01 Does not have greedy-choice property, since $.40 is best made with two $.20’s, but the greedy solution will pick three coins (which ones?) Making Change

8. How to rob a bank…

9. The Fractional Knapsack Problem • Given: A set S of n items, with each item i having • bi - a positive benefit • wi - a positive weight • Goal: Choose items with maximum total benefit but with weight at most W. • If we are allowed to take fractional amounts, then this is the fractional knapsack problem. • In this case, we let xi denote the amount we take of item i • Objective: maximize • Constraint:

10. 10 ml Example • Given: A set S of n items, with each item i having • bi - a positive benefit • wi - a positive weight • Goal: Choose items with maximum total benefit but with weight at most W. “knapsack” • Solution: • 1 ml of 5 • 2 ml of 3 • 6 ml of 4 • 1 ml of 2 Items: 1 2 3 4 5 Weight: 4 ml 8 ml 2 ml 6 ml 1 ml Benefit: $12 $32 $40 $30 $50 Value: 3 4 20 5 50 ($ per ml)

11. The Fractional Knapsack Algorithm • Greedy choice: Keep taking item with highest value (benefit to weight ratio) • Since • Run time: O(n log n). Why? • Correctness: Suppose there is a better solution • there is an item i with higher value than a chosen item j, but xi<wi, xj>0 and vi<vj • If we substitute some i with j, we get a better solution • How much of i: min{wi-xi, xj} • Thus, there is no better solution than the greedy one AlgorithmfractionalKnapsack(S,W) Input:set S of items w/ benefit biand weight wi; max. weight WOutput:amount xi of each item i to maximize benefit w/ weight at most W for each item i in S xi 0 vi bi / wi{value} w 0 {total weight} whilew < W remove item i w/ highest vi xi min{wi , W - w} w  w + min{wi , W - w}

12. Machine 3 Machine 2 Machine 1 1 2 3 4 5 6 7 8 9 Task Scheduling • Given: a set T of n tasks, each having: • A start time, si • A finish time, fi (where si < fi) • Goal: Perform all the tasks using a minimum number of “machines.”

13. Task Scheduling Algorithm • Greedy choice: consider tasks by their start time and use as few machines as possible with this order. • Run time: O(n log n). Why? • Correctness: Suppose there is a better schedule. • We can use k-1 machines • The algorithm uses k • Let i be first task scheduled on machine k • Machine i must conflict with k-1 other tasks • But that means there is no non-conflicting schedule using k-1 machines AlgorithmtaskSchedule(T) Input:set T of tasks w/ start time siand finish time fi Output:non-conflicting schedule with minimum number of machines m 0 {no. of machines} whileT is not empty remove task i w/ smallest si if there’s a machine j for i then schedule i on machine j else m m + 1 schedule i on machine m

14. Example • Given: a set T of n tasks, each having: • A start time, si • A finish time, fi (where si < fi) • [1,4], [1,3], [2,5], [3,7], [4,7], [6,9], [7,8] (ordered by start) • Goal: Perform all tasks on min. number of machines Machine 3 Machine 2 Machine 1 1 2 3 4 5 6 7 8 9

15. 7 2  9 4 2 4 7 9 7  2 2 7 9  4 4 9 7 7 2 2 9 9 4 4 Divide-and-Conquer

16. Outline and Reading • Divide-and-conquer paradigm (§5.2) • Review Merge-sort (§4.1.1) • Recurrence Equations (§5.2.1) • Iterative substitution • Recursion trees • Guess-and-test • The master method • Integer Multiplication (§5.2.2)

17. Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two or more disjoint subsets S1, S2, … Recur: solve the subproblems recursively Conquer: combine the solutions for S1,S2, …, into a solution for S The base case for the recursion are subproblems of constant size Analysis can be done using recurrence equations Divide-and-Conquer

18. Binary Search • I have a number X between c1 and c2… • Divide and conquer algorithm? • Analysis: • Called a Recurrence Equation • How to solve?

19. Solving the Binary Search Recurrence T(n) = T(n/2) + c T(n/2) = T(n/4) + c T(n/4) = T(n/8) + c … ----------- T(n) = T(n/2) + c = [T(n/4) + c] + c = T(n/22) + 2 c = T(n/23) + 3 c [telescope] = T(n/2k) + k c done when k=lg n; then T(n/2k)=T(1) is base case = b + c lg n

20. Merge Sort Recurrence Equation Analysis • Recurrence Equation? • We can analyze by finding a closed form solution. • Method: expand and telescope

21. Solving the Merge Sort recurrence T(n) = 2T(n/2) + b n T(n/2) = 2T(n/4) + b n/2 T(n/4) = 2T(n/8) + b n/4 … ----------- T(n) = 2T(n/2) + b n = 2[2T(n/4) + b n/2] + b n = 22T(n/22) + 2 b n = 23T(n/23) + 3 b n [telescope] = 2kT(n/2k) + k b n done when k=lg n; then T(n/2k)=T(1) is base case = b n + b n lg n

22. Min & Max • Given a set of n items, find the min and max. How many comparisons needed? • Simple alg: n-1 comparisons to find min, n-1 for max • Recursive algorithm: void minmax(int i, int j, int& min0, int& max0) { if (j<=i+1) [handle base case; return] m=(i+j+1)/2; minmax(i,m-1,min1,max1); minmax(m,j,min2,max2); min0 = min(min1,min2); max0 = max(max1,max2); } Recurrence: T(n)=2T(n/2) + 2; T(2)=1 Solution? 1.5 n - 2

23. Solving the Min&Max Recurrence T(n) = 2T(n/2) + 2; T(2)=1 T(n/2) = 2T(n/4) + 2 T(n/4) = 2T(n/8) + 2 … ----------- T(n) = 2T(n/2) + 2 = 2[2T(n/4) + 2] + 2 = 22T(n/22) + 22 + 2 = 23T(n/23) + 23 + 22 + 2 [telescope] = 2kT(n/2k) + = 2kT(n/2k) + 2k+1 - 2 done when k=(lg n) - 1; then T(n/2k)=T(2) is base case = n/2 + n – 2 = 1.5n - 2

24. Hanoi runtime? void hanoi(n, from, to, spare { if (n > 0) { hanoi(n-1,from,spare,to); cout << from << “ – “ << to << endl; hanoi(n-1,spare,to,from); } } T(n) = 2T(n-1) + c T(0) = b Look it up: T(n) = O(2n)

25. Master Method • Another way to solve recurrences: “Look it up” • The Master Theorem solves many of them: If T(n)=c, n<d, and T(n)=aT(n/b)+f(n), n≥d, and for small constants e>0, k>=0, and d<1 then • If f(n) is O( ) then T(n) is Q( ) • If f(n) is Q( ) then T(n) is Q( ) • If f(n) is W( ) and a f(n/b) ≤ df(n), for n ≤ d, then T(n) is Q(f(n))

27. An Important use of Recurrence Relations by Donald Knuth

29. More Divide and Conquer Examples • pow(x,n) • log(n,base) • Point-in-convex-polygon • Fractals

33. http://www.3villagecsd.k12.ny.us/wmhs/Departments/Math/OBrien/fibonacci2.htmlhttp://www.3villagecsd.k12.ny.us/wmhs/Departments/Math/OBrien/fibonacci2.html

34. http://www.religiousforums.com/forum/material-world/80450-fibonacci-sequence.htmlhttp://www.religiousforums.com/forum/material-world/80450-fibonacci-sequence.html

43. Creating Fractals • Koch Snowflake • Fractal Star • Your own…

44. Integer Multiplication • Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • We can then define I*J by multiplying the parts and adding: • So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2). • But that is no better than the algorithm we learned in grade school.

45. An Improved Integer Multiplication Algorithm • Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • Observe that there is a different way to multiply parts: • So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23) [by master theorem]. • Thus, T(n) is O(n1.585).

46. Practice Analyzing Recursive Functions 1. int fun1(int n) { if (n==1) return 50; return fun1(n/2) * fun1(n/2) + 97; } 2. int fun2(int n) { if (n==1) return 1; int sum=0; for (int i=1; i<=n; i++) sum += fun(n-1); return sum; }

47. Dynamic Programming

48. Outline and Reading • Fibonacci • Making Change • Matrix Chain-Product (§5.3.1) • The General Technique (§5.3.2) • 0-1 Knapsack Problem (§5.3.3)

49. Opinions? public class Fib { public static BigIntegerfibonacci(long n) { if (n <= 2) return new BigInteger("1"); return fibonacci(n-1).add(fibonacci(n-2)); } public static void main( String args[] ) { long n = Integer.parseInt(args[0]); System.out.println("Fib(" + n + ") = " + fibonacci(n).toString()); } }

50. Bottom up… public class Fib2 { static BigInteger[] fibs; public static BigInteger fibonacci(int n) { BigInteger result; fibs[1]=new BigInteger("1"); fibs[2]=new BigInteger("1"); for (int i=3; i<=n; i++) fibs[i] = fibs[i-1].add(fibs[i-2]); return fibs[n]; } public static void main( String args[] ) { int n = Integer.parseInt(args[0]); fibs = new BigInteger[n+1]; System.out.println("Fib(" + n + ") = " + fibonacci(n).toString()); } }