1 / 25

Greedy Algorithms

Learn about greedy algorithms, optimization problems, and how to make locally optimal choices for globally optimal solutions. Explore activity selection and knapsack problems with greedy strategies and analyze Huffman codes for efficient encoding.

cedricks
Download Presentation

Greedy Algorithms

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Greedy Algorithms Peter Schröder

  2. Greedy Algorithms • Optimization problems • Example: dynamic programming • Considers all possible solutions • Exploits optimal subproblem property • Common features? • Choice of steps along the way • What about always picking the best one at any given step? • Greedy algorithms

  3. Greedy Algorithms • Choice • Take best choice at a given moment • Locally optimal • Globally optimal? • For some problems: yes • Example problem • Activity selection

  4. Activity Selection • Problem statement • Given a resource and several competing activities drawing on this resource, how should they be scheduled so that the largest number of activities can be performed? • Scheduling a lecture hall for a number of different classes • Get the most classes scheduled • NOT the most hours… just the most classes • Suggestions…

  5. Activity Selection • Formal setup • Set of activities • Each activity has start time and finish time • Activities are compatible if • Find maximal subset of S such that its members are compatible • Assume activities ordered by finish time

  6. Algorithm • Greedy-Activity-Selection(s,f) • n := length[s] • a := {1} • j := 1 • For i:=2 to n • do if si >= fj • then a := a + {i} • j := I • Return a

  7. Example Run

  8. Analysis • Greedy strategy • What is it here? • What does it maximize? • Correctness proof • Some optimal solution contains element 1 • Remaining problem is subset of original problem

  9. Runtime • What is the runtime of Greedy-Activity-Selection? • What about a dynamic programming approach to this problem? • Give an algorithm • What is its running time?

  10. When does it work? • Elements of the greedy strategy • No general strategy, but… • Greedy choice property • Optimal substructure property

  11. Greedy Choice Property • Locally optimal choices • Make best choice available at a given moment • Solve subproblems after choice has been made • Contrast with dynamic programming • Choice at a given step may depend on solutions to subproblems

  12. Optimal Substructure • When does a problem exhibit optimal substructure? • If an optimal solution to the entire problem contains within it optimal solutions to subproblems • But this is also true of dynamic programming… • How to tell the difference?

  13. Greedy vs. Dynamic • Knapsack problem • 0-1 version • Include or don’t include • Fractional version • May take fraction of an item • Problem statement • Fill knapsack with items of weight wi and value vi; maximize value in knapsack

  14. Knapsack Problem • Optimal substructure property • Take out item from optimal solution and prove that remaining problem is still optimal

  15. 20 30 50 10 60$ 120$ 100$ Knapsack Problem • Solution • Greedy strategy works for fractional version • Prove it • Need dynamic programming for o-1 version

  16. Another Example • Huffman codes • Efficient encoding of a finite alphabet and a given distribution of occurances

  17. Prefix Codes • Desirable for coding • Optimal compression can always be achieved by a prefix code • No codeword is a prefix of another codeword • Easy on the decoder • Representation of the code book • Suggestions?

  18. Optimal Encoding • Finding optimal coding • Search over all complete binary trees with |C| leaves • Cost function • Algorithm • Suggestions?

  19. Huffman’s Algorithm Huffman(C) n := |C| Q := C For i := 1 to n-1 do z := AllocNode() x := left[z] := ExtractMIN(Q) y := right[z] := ExtractMIN(Q) f[z] := f[x] + f[y] Insert(Q,z) Return ExtractMIN(Q)

  20. Analysis • Running time • Need Heap data structure • Remember properties • Cost for construction • Cost for extraction • Cost for insertion

  21. Analysis • Correctness • Greedy choice property • Optimal substructure • Lemma • Let X and Y be two lowest frequency characters; then there exists and optimal prefix code for C in which X and Y have the same length and differ only in the last bit

  22. Proof of Lemma • Idea: Modify existing optimal tree • Assume B and C are siblings at lowest depth in tree • Assume wlog that • Exchange X with B and Y with C • Show that cost of new tree must be the same as cost of old tree • Assertion of Lemma follows

  23. Analysis • What is the greedy choice property here? • Cost of tree is cost of all mergers • Greedy choice is merger of least cost, i.e., lowest frequency symbols at the time • Optimal substructure? • Lemma: Let T be a full binary tree representing an optimal prefix code; consider any two leaves X and Y and parent Z; consider Z as a character and replace it accordingly; the new tree is optimal

  24. Proof of Lemma • Consider component costs • hence • the assertion follows by contradiction • Correctness of Huffman • follows from Lemmas

  25. Questions • Describe tree • for encoding a file, must also encode structure of tree; how many bits? • Can you do better? • will Huffman always give the best possible encoding? • how about a file consisting of random characters?

More Related