Greedy Algorithms

Clayton Andrews 2/26/08 Greedy Algorithms

What is an algorithm? • “An algorithm is any well-defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output.” -(Cormen 5)‏

What is a greedy algorithm? • A greedy algorithm is an algorithm that, at each step, is presented with choices, these choices are measured and one is determined to be the best and is selected. • “It makes a locally optimal choice in the hope that this choice will lead to a globally optimal solution” -(Cormen 370)‏

Greedy algorithms do • Choose the largest, fastest, cheapest, etc... • Typically make the problem smaller after each step or choice. • Sometimes make decisions that turn out bad in the long run

Greedy algorithms don't • Do not consider all possible paths • Do not consider future choices • Do not reconsider previous choices • Do not always find an optimal solution

Greedy Algorithms Speed • Because, greedy algorithms do not consider future decisions or reconsider any previous decisions, the time complexity of a greedy algorithm is usually magnitudes better than a brute force algorithm • If a greedy algorithm is proven to find the optimal answer, always, the algorithm usually becomes standard because of speed and simplicity

Types of Solutions Produced by Greedy Algorithms • Optimal Solutions – The best possible answer that any algorithm could find to the problem • Good Solutions – A solution that is near optimal and could be good-enough for some problems • Bad Solutions – A solution that is not acceptable • Worst Possible Solution – The solution that is farthest from the goal

A simple problem • Find the smallest number of coins whose sum reaches a specific goal • Input: The total to reach and the coins usable • Output: The smallest number of coins to reach the total

A greedy solution • 1. Make a set with all types of coins • 2. Choose the largest coin in set • 3. If this coin will take the solution total over the target total, remove it from the set. Otherwise, add it to the solution set. • 4. Calculate how large the current solution is • 5. If the solution set sums up to the target total, a solution has been found, otherwise repeat 2-5

An Example • Goal = 46 cents • 25 • 35 • 45 • 46

Parts of a Greedy Algorithm • A candidate set • A selection function • A feasibility function • An objective function • A solution function

Candidate Set • The set the solution is constructed from. • Members of the candidate set are checked to see if they can help construct the solution.

Candidate Set (Example)‏ • Candidate Set • Solution Set • As you can see, solution set is made of members of the candidate set

Selection function • The function that chooses what member of the candidate set to select • The selection could be based off previous selections or some type of heuristic

Selection Function(Example)‏ • The selection function for the change example chooses the largest coin in the candidate set • As the coins are removed it still chooses the largest of the remaining. • In this case, the selection function simply has to ensure the list is sorted and remove the largest.

Feasibility Function • The function that determines if the choice of the selection function can help construct the solution • The feasibility function may have to perform operations on the selected member to determine if all or part of it is acceptable

Feasibility Function (Example)‏ • For the change example, the feasibility function determines if a coin is too large. • If the difference between the solution and the goal is too small, then the selected coin is not feasible and is removed from the candidate set.

Objective Function • The objective function determines the value of your current solution • This value can be used to determine if the solution satisfies the requirements or if a member of the candidate set is feasible.

Objective Function(Example)‏ • For the change example, the objective function scores the solution set • In the example, the objective function just sums up the coins in the solution set. • If the solution set had a quarter and a dime the objective function would return 35.

Solution Function • The solution function determines if the current solution is a complete solution • If the current solution meets the problem requirements • If all choices are exhausted, then this must be a solution

Solution Function(Example)‏ • In the change example, the solution function would compare the objective function's output with the target total. • If the objective function's output is close enough to the target total, then the solution function ends the algorithm, because a solution has been found

Greedy Algorithm Part Summary • Candidate Set – the set the solution is made from • Selection Function – determines next candidate set member to be selected • Feasibility Function – can the current candidate set be used to help construct the solution • Objective Function – how good is the current solution • Solution Function – is the current solution a complete solution

Examples • Welsh-Powell Algorithm • Kruskal's Algorithm • 0/1 Knapsack

Welsh-Powell Algorithm • The Welsh-Powell Algorithm is a greedy algorithm for finding good solutions to the graph coloring problem. • Gives a good answer on average to the question “How many colors does it take to color this graph so that no neighbors have the same color?”

Welsh-Powell Algorithm • Candidate Set – The nodes of the graph • Selection Function – Remove node with the most edges • Feasibility Function–Does this node have neighbor who is current color? If not, color the node • Objective Function – Determine how many nodes are uncolored • Solution Function – If the candidate set is not empty, repeat. If the candidate set is empty and the number of uncolored nodes is greater than 0 then choose a new color and put all uncolored nodes in the candidate set, then repeat

Kruskal's Algorithm • Kruskal's algorithm finds the minimum spanning tree of a graph. This greedy algorithm always finds the optimal solution to the problem. • Step 1 is to put each node into its own set

Kruskal's Algorithm • Candidate Set – The edges of the graph • Selection Function – Choose the smallest edge and remove it from the candidate set • Feasibility Function – Does the selected edge form a cycle with its set. If not, merge the sets of both nodes of the edge and add the edge to the solution set • Objective Function – Calculate how many sets are remaining, a complete solution has 1 • Solution Function – If the number of sets is equal to 1, the solution is found, otherwise keep removing edges

0/1 Knapsack • Given a series of items with values and weights and a knapsack with a maximum weight it can hold, what is the maximum value of items you can carry. • Assume an item can only be added once • The greedy solution guarantees a solution that is at least ½ the optimal solution. This isn't a very good guarantee, but good enough for some problems.

0/1 Knapsack • Candidate Set – Items available to add • Selection Function – Remove item with greatest Value/Weight ratio • Feasibility Function – Does this item make the solution weight greater than the maximum capacity of the basket? If not, then add the item to the solution • Objective Function – Find how much weight is left in the basket • Solution Function – If there are items left in the candidate set and the basket can hold more weight then repeat, otherwise a solution has been found

Summary • Greedy algorithms are fast and usually easy to make, which makes them very popular. • Greedy algorithms have parts that are common amongst most of them. • Although it is uncommon for a greedy algorithm to find an optimal solution, sometimes a pretty good solution is fine, because the algorithm is so fast.

Homework • What are the 5 parts of a typical greedy algorithm? • What is one thing that a greedy algorithm does not do?

Works Cited • Introduction to Algorithms (Cormen, Leiserson, Rivest and Stein) 2001. • http://www.cs.binghamton.edu/~dima/cs333/greedy.ppt • http://www.cse.ohio-state.edu/~gurari/course/cis680/cis680Ch17.html • http://www.nist.gov/dads/HTML/greedyalgo.html • (Just Images)http://‏www.wikipedia.com • http://www.cs.ucdavis.edu/~krovetz/courses/ecs122a_F01/lect03.txt

Greedy Algorithms