CSC401 – Analysis of Algorithms Chapter 5--1 The Greedy Method

1. CSC401 – Analysis of AlgorithmsChapter 5--1The Greedy Method Objectives • Introduce the Brute Force method and the Greedy Method • Compare the solutions to the same problem using different methods • Fractional Knapsack problem • Task scheduling problem

2. Brute Force • A straightforward approach, usually based directly on the problem’s statement and definitions of the concepts involved • Examples: • Computing an (a > 0, n a nonnegative integer) • Computing n! • Multiplying two matrices • Selection sort • Searching for a key of a given value in a list

3. Brute-Force Sorting Algorithm Selection SortScan the array to find its smallest element and swap it with the first element. Then, starting with the second element, scan the elements to the right of it to find the smallest among them and swap it with the second elements. Generally, on pass i (0  i  n-2), find the smallest element in A[i..n-1] and swap it with A[i]:A[0]  . . .  A[i-1] | A[i], . . . , A[min], . . ., A[n-1] in their final positions Example: 7 3 2 5

4. Analysis of Selection Sort Time efficiency: (n-1)n/2  O(n2) Space efficiency: 2  O(1)

5. Brute-Force String Matching • pattern: a string of m characters to search for • text: a (longer) string of n characters to search in • problem: find a substring in the text that matches the pattern • Brute-force algorithm Step 1 Align pattern at beginning of text Step 2 Moving from left to right, compare each character of pattern to the corresponding character in text until • all characters are found to match (successful search); or • a mismatch is detected Step 3 While pattern is not found and the text is not yet exhausted, realign pattern one position to the right and repeat Step 2

6. Brute-Force String Matching: Examples • Pattern: 001011 Text: 10010101101001100101111010 • Pattern: happy Text: It is never too late to have a happy childhood.

7. Pseudocode and Efficiency Efficiency: O(nm)

8. Brute-Force Polynomial Evaluation • Problem: Find the value of polynomial p(x) = anxn+ an-1xn-1 +… + a1x1 + a0 at a point x = x0 • Brute-force algorithm • Efficiency: O(n2) • p0.0 • for in downto 0 do • power 1 • for j 1 to i do //compute xi • powerpowerx • pp + a[i] power • returnp

9. Polynomial Evaluation: Improvement We can do better by evaluating from right to left: Better brute-force algorithm Efficiency: O(n) • pa[0] • power 1 • fori 1 tondo • powerpower x • p p + a[i] power • returnp

10. Closest-Pair Problem • Find the two closest points in a set of n points (in the two-dimensional Cartesian plane). • Brute-force algorithm • Compute the distance between every pair of distinct points and return the indexes of the points for which the distance is the smallest.

11. Closest-Pair Brute-Force Algorithm (cont.) Efficiency: O(n2) How to make it faster?

12. Convex hull • Problem • Find smallest convex polygon enclosing n points on the plane • Algorithm • Find each pair of points p1 and p2, determine whether all other points lie to the same side of the straight line through p1 and p2 • Efficiency: • O(n3)

13. Brute-Force Strengths and Weaknesses • Strengths • wide applicability • simplicity • yields reasonable algorithms for some important problems(e.g., matrix multiplication, sorting, searching, string matching) • Weaknesses • rarely yields efficient algorithms • some brute-force algorithms are unacceptably slow • not as constructive as some other design techniques

14. Exhaustive Search • A brute force solution to a problem involving search for an element with a special property, usually among combinatorial objects such as permutations, combinations, or subsets of a set. • Method: • generate a list of all potential solutions to the problem in a systematic manner • evaluate potential solutions one by one, disqualifying infeasible ones and, for an optimization problem, keeping track of the best one found so far • when search ends, announce the solution(s) found

15. 2 a b 5 3 4 8 c d 7 Example 1: Traveling Salesman Problem • Given n cities with known distances between each pair, find the shortest tour that passes through all the cities exactly once before returning to the starting city • Alternatively: Find shortest Hamiltonian circuit in a weighted connected graph • Example: TourCost a→b→c→d→a 2+3+7+5 = 17 a→b→d→c→a 2+4+7+8 = 21 a→c→b→d→a 8+3+4+5 = 20 a→c→d→b→a 8+7+4+2 = 21 a→d→b→c→a 5+4+3+8 = 20 a→d→c→b→a 5+7+3+2 = 17

16. Example 2: Knapsack Problem • Given n items: • weights: w1w2 …wn • values: v1v2 … vn • a knapsack of capacity W • Find most valuable subset of the items that fit into the knapsack • Efficiency: O(2n) • Example: Knapsack capacity W=16 item weight value • 2 $20 • 5 $30 • 10 $50 • 5 $10 Subset Weight Value {1} 2 $20 {2} 5 $30 {3} 10 $50 {4} 5 $10 {1,2} 7 $50 {1,3} 12 $70 {1,4} 7 $30 {2,3} 15 $80 {2,4} 10 $40 {3,4} 15 $60 {1,2,3} 17 not feasible {1,2,4} 12 $60 {1,3,4} 17 not feasible {2,3,4} 20 not feasible {1,2,3,4} 22 not feasible

17. Example 3: The Assignment Problem • There are n people who need to be assigned to n jobs, one person per job. The cost of assigning person i to job j is C[i,j]. • Find an assignment that minimizes the total cost. Job 0 Job 1 Job 2 Job 3 Person 0 9 2 7 8 Person 1 6 4 3 7 Person 2 5 8 1 8 Person 3 7 6 9 4 • Algorithmic Plan:Generate all legitimate assignments, compute their costs, and select the cheapest one. • How many assignments are there? Assignment (col.#s) Total Cost 1, 2, 3, 4 9+4+1+4=18 1, 2, 4, 3 9+4+8+9=30 1, 3, 2, 4 9+3+8+4=24 1, 3, 4, 2 9+3+8+6=26 1, 4, 2, 3 9+7+8+9=33 1, 4, 3, 2 9+7+1+6=23 etc.

18. Final Comments on Exhaustive Search • Exhaustive-search algorithms run in a realistic amount of time only on very small instances • In some cases, there are much better alternatives! • Euler circuits • shortest paths • minimum spanning tree • assignment problem • In many cases, exhaustive search or its variation is the only known way to get exact solution

19. The greedy methodis a general algorithm design paradigm, built on the following elements: configurations: different choices, collections, or values to find objective function: a score assigned to configurations, which we want to either maximize or minimize It works best when applied to problems with the greedy-choice property: a globally-optimal solution can always be found by a series of local improvements from a starting configuration. The Greedy Method Technique

20. Problem: A dollar amount to reach and a collection of coin amounts to use to get there. Configuration: A dollar amount yet to return to a customer plus the coins already returned Objective function: Minimize number of coins returned. Greedy solution: Always return the largest coin you can Example 1: Coins are valued $.25, $.10, $0.05, $0.01 Has the greedy-choice property, since no amount over $.25 can be made with a minimum number of coins by omitting a $.25 coin (similarly for amounts over $.10, but under $.25). Example 2: Coins are valued $.30, $.20, $.05, $.01 Does not have greedy-choice property, since $.40 is best made with two $.20’s, but the greedy solution will pick three coins (which ones?) Making Change

21. The Fractional Knapsack Problem • Given: A set S of n items, with each item i having • bi - a positive benefit (value) • wi - a positive weight • Goal: Choose items with maximum total benefit (value) but with weight at most W. • If we are allowed to take fractional amounts, then this is the fractional knapsack problem. • In this case, we let xi denote the amount we take of item i • Objective: maximize • Constraint:

22. 10 ml Example • Given: A set S of n items, with each item i having • bi - a positive benefit • wi - a positive weight • Goal: Choose items with maximum total benefit but with total weight at most W. “knapsack” • Solution: • 1 ml of 5 • 2 ml of 3 • 6 ml of 4 • 1 ml of 2 Items: 1 2 3 4 5 Weight: 4 ml 8 ml 2 ml 6 ml 1 ml Benefit: $12 $32 $40 $30 $50 Value: 3 4 20 5 50 ($ per ml)

23. The Fractional Knapsack Algorithm • Greedy choice: Keep taking item with highest value (benefit to weight ratio) • Since AlgorithmfractionalKnapsack(S,W) Input:set S of items w/ benefit biand weight wi; max. weight W Output:amount xi of each item i to maximize benefit w/ weight at most W for each item i in S xi 0 vi bi / wi{value} w 0 {total weight} whilew < W remove item i with highest vi xi min{wi , W - w} w  w + xi

24. The Fractional Knapsack Algorithm • Running time: Given a collection S of n items, such that each item i has a benefit bi and weight wi, we can construct a maximum-benefit subset of S, allowing for fractional amounts, that has a total weight W in O(nlogn) time. • Use heap-based priority queue to store S • Removing the item with the highest value takes O(logn) time • In the worst case, need to remove all items • Correctness: Suppose there is a better solution • there is an item i with higher value than a chosen item j, but xi<wi, xj>0 and vi<vj • If we substitute some i with j, we get a better solution • How much of i: min{wi-xi, xj} • Thus, there is no better solution than the greedy one

25. Machine 3 Machine 2 Machine 1 1 2 3 4 5 6 7 8 9 Task Scheduling • Given: a set T of n tasks, each having: • A start time, si • A finish time, fi (where si < fi) • Goal: Perform all the tasks using a minimum number of “machines.”

26. Task Scheduling Algorithm • Greedy choice: consider tasks by their start time and use as few machines as possible with this order. AlgorithmtaskSchedule(T) Input:set T of tasks with start time siand finish time fi Output:non-conflicting schedule with minimum number of machines m 0 {no. of machines} whileT is not empty remove task i with smallest si if there’s a machine j fori then schedule i on machine j else m m + 1 schedule i on machine m

27. Task Scheduling Algorithm • Running time: Given a set of n tasks specified by their start and finish times, Algorithm TaskSchedule produces a schedule of the tasks with the minimum number of machines in O(nlogn) time. • Use heap-based priority queue to store tasks with the start time as the priorities • Finding the earliest task takes O(logn) time • Correctness: (proof by contradiction) Suppose there is a better schedule. • We can use k-1 machines • The algorithm uses k • Let i be first task scheduled on machine k • Machine i must conflict with k-1 other tasks • But that means there is no non-conflicting schedule using k-1 machines

28. Machine 3 Machine 2 Machine 1 1 2 3 4 5 6 7 8 9 Example • Given: a set T of n tasks, each having: • A start time, si • A finish time, fi (where si < fi) • [1,4], [1,3], [2,5], [3,7], [4,7], [6,9], [7,8] (ordered by start) • Goal: Perform all tasks on min. number of machines