1 / 73

Chapter 5 Fundamental Algorithm Design Techniques

Chapter 5 Fundamental Algorithm Design Techniques. Overview. The Greedy Method (5.1) Divide and Conquer (5.2) Dynamic Programming (5.3). Greedy Outline. The Greedy Method Technique (§5.1) Fractional Knapsack Problem (§5.1.1) Task Scheduling (§5.1.2)

jerrod
Download Presentation

Chapter 5 Fundamental Algorithm Design Techniques

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Chapter 5Fundamental AlgorithmDesign Techniques

  2. Overview • The Greedy Method (5.1) • Divide and Conquer (5.2) • Dynamic Programming (5.3)

  3. Greedy Outline • The Greedy Method Technique (§5.1) • Fractional Knapsack Problem (§5.1.1) • Task Scheduling (§5.1.2) • Minimum Spanning Trees (§7.3) [future lecture]

  4. A New Vending Machine? • Coke machine in the lounge behind the ulab—can we hack it? • Being efficient computer scientists, we want to return the smallest possible number of coins in change

  5. The Greedy Method

  6. The greedy method is a general algorithm design paradigm, built on the following elements: configurations: different choices, collections, or values to find objective function: a score assigned to configurations, which we want to either maximize or minimize It works best when applied to problems with the greedy-choice property: a globally-optimal solution can always be found by a series of local improvements from a starting configuration. The Greedy Method

  7. Problem: A dollar amount to reach and a collection of coin amounts to use to get there. Configuration: A dollar amount yet to return to a customer plus the coins already returned Objective function: Minimize number of coins returned. Greedy solution: Always return the largest coin you can Example 1: Coins are valued $.32, $.08, $.01 Has the greedy-choice property, since no amount over $.32 can be made with a minimum number of coins by omitting a $.32 coin (similarly for amounts over $.08, but under $.32). Example 2: Coins are valued $.30, $.20, $.05, $.01 Does not have greedy-choice property, since $.40 is best made with two $.20’s, but the greedy solution will pick three coins (which ones?) Making Change

  8. How to rob a bank…

  9. The Fractional Knapsack Problem • Given: A set S of n items, with each item i having • bi - a positive benefit • wi - a positive weight • Goal: Choose items with maximum total benefit but with weight at most W. • If we are allowed to take fractional amounts, then this is the fractional knapsack problem. • In this case, we let xi denote the amount we take of item i • Objective: maximize • Constraint:

  10. 10 ml Example • Given: A set S of n items, with each item i having • bi - a positive benefit • wi - a positive weight • Goal: Choose items with maximum total benefit but with weight at most W. “knapsack” • Solution: • 1 ml of 5 • 2 ml of 3 • 6 ml of 4 • 1 ml of 2 Items: 1 2 3 4 5 Weight: 4 ml 8 ml 2 ml 6 ml 1 ml Benefit: $12 $32 $40 $30 $50 Value: 3 4 20 5 50 ($ per ml)

  11. The Fractional Knapsack Algorithm • Greedy choice: Keep taking item with highest value (benefit to weight ratio) • Since • Run time: O(n log n). Why? • Correctness: Suppose there is a better solution • there is an item i with higher value than a chosen item j, but xi<wi, xj>0 and vi<vj • If we substitute some i with j, we get a better solution • How much of i: min{wi-xi, xj} • Thus, there is no better solution than the greedy one AlgorithmfractionalKnapsack(S,W) Input:set S of items w/ benefit biand weight wi; max. weight WOutput:amount xi of each item i to maximize benefit w/ weight at most W for each item i in S xi 0 vi bi / wi{value} w 0 {total weight} whilew < W remove item i w/ highest vi xi min{wi , W - w} w  w + min{wi , W - w}

  12. Machine 3 Machine 2 Machine 1 1 2 3 4 5 6 7 8 9 Task Scheduling • Given: a set T of n tasks, each having: • A start time, si • A finish time, fi (where si < fi) • Goal: Perform all the tasks using a minimum number of “machines.”

  13. Task Scheduling Algorithm • Greedy choice: consider tasks by their start time and use as few machines as possible with this order. • Run time: O(n log n). Why? • Correctness: Suppose there is a better schedule. • We can use k-1 machines • The algorithm uses k • Let i be first task scheduled on machine k • Machine i must conflict with k-1 other tasks • But that means there is no non-conflicting schedule using k-1 machines AlgorithmtaskSchedule(T) Input:set T of tasks w/ start time siand finish time fi Output:non-conflicting schedule with minimum number of machines m 0 {no. of machines} whileT is not empty remove task i w/ smallest si if there’s a machine j for i then schedule i on machine j else m m + 1 schedule i on machine m

  14. Example • Given: a set T of n tasks, each having: • A start time, si • A finish time, fi (where si < fi) • [1,4], [1,3], [2,5], [3,7], [4,7], [6,9], [7,8] (ordered by start) • Goal: Perform all tasks on min. number of machines Machine 3 Machine 2 Machine 1 1 2 3 4 5 6 7 8 9

  15. 7 2  9 4 2 4 7 9 7  2 2 7 9  4 4 9 7 7 2 2 9 9 4 4 Divide-and-Conquer

  16. Outline and Reading • Divide-and-conquer paradigm (§5.2) • Review Merge-sort (§4.1.1) • Recurrence Equations (§5.2.1) • Iterative substitution • Recursion trees • Guess-and-test • The master method • Integer Multiplication (§5.2.2)

  17. Divide-and conquer is a general algorithm design paradigm: Divide: divide the input data S in two or more disjoint subsets S1, S2, … Recur: solve the subproblems recursively Conquer: combine the solutions for S1,S2, …, into a solution for S The base case for the recursion are subproblems of constant size Analysis can be done using recurrence equations Divide-and-Conquer

  18. Binary Search • I have a number X between c1 and c2… • Divide and conquer algorithm? • Analysis: • Called a Recurrence Equation • How to solve?

  19. Solving the Binary Search Recurrence T(n) = T(n/2) + c T(n/2) = T(n/4) + c T(n/4) = T(n/8) + c … ----------- T(n) = T(n/2) + c = [T(n/4) + c] + c = T(n/22) + 2 c = T(n/23) + 3 c [telescope] = T(n/2k) + k c done when k=lg n; then T(n/2k)=T(1) is base case = b + c lg n

  20. Merge Sort Recurrence Equation Analysis • Recurrence Equation? • We can analyze by finding a closed form solution. • Method: expand and telescope

  21. Solving the Merge Sort recurrence T(n) = 2T(n/2) + b n T(n/2) = 2T(n/4) + b n/2 T(n/4) = 2T(n/8) + b n/4 … ----------- T(n) = 2T(n/2) + b n = 2[2T(n/4) + b n/2] + b n = 22T(n/22) + 2 b n = 23T(n/23) + 3 b n [telescope] = 2kT(n/2k) + k b n done when k=lg n; then T(n/2k)=T(1) is base case = b n + b n lg n

  22. Min & Max • Given a set of n items, find the min and max. How many comparisons needed? • Simple alg: n-1 comparisons to find min, n-1 for max • Recursive algorithm: void minmax(int i, int j, int& min0, int& max0) { if (j<=i+1) [handle base case; return] m=(i+j+1)/2; minmax(i,m-1,min1,max1); minmax(m,j,min2,max2); min0 = min(min1,min2); max0 = max(max1,max2); } Recurrence: T(n)=2T(n/2) + 2; T(2)=1 Solution? 1.5 n - 2

  23. Solving the Min&Max Recurrence T(n) = 2T(n/2) + 2; T(2)=1 T(n/2) = 2T(n/4) + 2 T(n/4) = 2T(n/8) + 2 … ----------- T(n) = 2T(n/2) + 2 = 2[2T(n/4) + 2] + 2 = 22T(n/22) + 22 + 2 = 23T(n/23) + 23 + 22 + 2 [telescope] = 2kT(n/2k) + = 2kT(n/2k) + 2k+1 - 2 done when k=(lg n) - 1; then T(n/2k)=T(2) is base case = n/2 + n – 2 = 1.5n - 2

  24. Hanoi runtime? void hanoi(n, from, to, spare { if (n > 0) { hanoi(n-1,from,spare,to); cout << from << “ – “ << to << endl; hanoi(n-1,spare,to,from); } } T(n) = 2T(n-1) + c T(0) = b Look it up: T(n) = O(2n)

  25. Master Method • Another way to solve recurrences: “Look it up” • The Master Theorem solves many of them: If T(n)=c, n<d, and T(n)=aT(n/b)+f(n), n≥d, and for small constants e>0, k>=0, and d<1 then • If f(n) is O( ) then T(n) is Q( ) • If f(n) is Q( ) then T(n) is Q( ) • If f(n) is W( ) and a f(n/b) ≤ df(n), for n ≤ d, then T(n) is Q(f(n))

  26. An Important use of Recurrence Relations by Donald Knuth

  27. More Divide and Conquer Examples • pow(x,n) • log(n,base) • Point-in-convex-polygon • Fractals

  28. http://www.3villagecsd.k12.ny.us/wmhs/Departments/Math/OBrien/fibonacci2.htmlhttp://www.3villagecsd.k12.ny.us/wmhs/Departments/Math/OBrien/fibonacci2.html

  29. http://www.religiousforums.com/forum/material-world/80450-fibonacci-sequence.htmlhttp://www.religiousforums.com/forum/material-world/80450-fibonacci-sequence.html

  30. Creating Fractals • Koch Snowflake • Fractal Star • Your own…

  31. Integer Multiplication • Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • We can then define I*J by multiplying the parts and adding: • So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2). • But that is no better than the algorithm we learned in grade school.

  32. An Improved Integer Multiplication Algorithm • Algorithm: Multiply two n-bit integers I and J. • Divide step: Split I and J into high-order and low-order bits • Observe that there is a different way to multiply parts: • So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23) [by master theorem]. • Thus, T(n) is O(n1.585).

  33. Practice Analyzing Recursive Functions 1. int fun1(int n) { if (n==1) return 50; return fun1(n/2) * fun1(n/2) + 97; } 2. int fun2(int n) { if (n==1) return 1; int sum=0; for (int i=1; i<=n; i++) sum += fun(n-1); return sum; }

  34. Dynamic Programming

  35. Outline and Reading • Fibonacci • Making Change • Matrix Chain-Product (§5.3.1) • The General Technique (§5.3.2) • 0-1 Knapsack Problem (§5.3.3)

  36. Opinions? public class Fib { public static BigIntegerfibonacci(long n) { if (n <= 2) return new BigInteger("1"); return fibonacci(n-1).add(fibonacci(n-2)); } public static void main( String args[] ) { long n = Integer.parseInt(args[0]); System.out.println("Fib(" + n + ") = " + fibonacci(n).toString()); } }

  37. Bottom up… public class Fib2 { static BigInteger[] fibs; public static BigInteger fibonacci(int n) { BigInteger result; fibs[1]=new BigInteger("1"); fibs[2]=new BigInteger("1"); for (int i=3; i<=n; i++) fibs[i] = fibs[i-1].add(fibs[i-2]); return fibs[n]; } public static void main( String args[] ) { int n = Integer.parseInt(args[0]); fibs = new BigInteger[n+1]; System.out.println("Fib(" + n + ") = " + fibonacci(n).toString()); } }

More Related