Greedy Algorithms

1. Greedy Algorithms Jeff Chastine

2. Greedy Algorithms • Compared to Dynamic Programming • Both used in optimization problems • More efficient • Not always optimal • Top-down instead of bottom up • Make a locally optimal (quick) choice • Still have optimal substructure • No need to dive into sub-problems Jeff Chastine

3. Activity Selection • Given a set of activities , find the maximum # you can schedule • NOT optimal use of time of a room • One activity is active at one time • Activity has start time of and finish of • Order activities by finish time isifi 1 2 3 4 5 6 7 8 9 10 11 1 3 0 5 3 5 6 8 8 2 12 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

4. Using Dynamic Programming • Defining the subproblem: • Set of activities that “fit” in this time chunk • Picking any activity creates two sub-problems Jeff Chastine

5. Visualization Sij aa ac ak ax az ab aw ay Jeff Chastine

6. Visualization Skj Sik Sij • Optimal solution includes aa ac ak ax az ab aw ay Jeff Chastine

7. Visualization Skj Sik Sij • We know • How to choose activity ? aa ac ak ax az ab aw ay Jeff Chastine

8. Visualization Skj Sik Sij • Iterate through all (like Matrix Mults)! aa ac ak ax az ab aw ay Jeff Chastine

9. Recursive Solution RECURSIVE-ACT-SELECT (s, f, i, n) 1 m ← i + 1 2 whilem ≤ n and sm < fi //find 1st activity 3 dom ← m + 1 4 ifm ≤ n 5 then return {am} RECURSIVE-ACT-SELECT(s, f, m, n) 6 else return Jeff Chastine

10. Code Trace a0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

11. First Call a0 a1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

12. Second Call a2 a0 a1 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

13. Second Call a2 a0 a1 a3 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

14. Second Call a2 a0 a1 a4 a3 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

15. Third Call a0 a1 a4 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

16. Third Call a5 a0 a1 a4 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

17. Third Call a5 a0 a1 a4 a6 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

18. Third Call a5 a0 a1 a4 a6 a7 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

19. Third Call a5 a0 a1 a4 a8 a6 a7 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

20. Fourth Call a0 a1 a4 a8 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

21. Fourth Call a9 a0 a1 a4 a8 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

22. Fourth Call a9 a0 a1 a4 a8 a10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

23. Fourth Call a9 a0 a1 a4 a8 a11 a10 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

24. Result a0 a1 a4 a8 a11 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 Jeff Chastine

25. Where Greedy Fails • Knapsack Problem • Steal items of different weight and value • Sack has weight limit • Goal: steal as much as possible • Two flavors of problem • 0-1 version (take all of something) • Fractional (can take only part) Jeff Chastine

26. 0-1 Example 50 item 3 30 item 2 20 item 1 10 $100 $60 $120 Jeff Chastine

27. Greedy – Go for the Gold! 50 item 3 30 item 2 20 item 1 10 $100 $60 $120 Jeff Chastine

28. Greedy – Go for the Gold! 50 item 3 30 item 2 20 item 1 10 10 $100 $60 $120 Bad Idea… Jeff Chastine

29. Greedy – Pick the next best 50 item 3 30 20 item 2 20 item 1 10 10 $100 $60 $120 Total = $160 Jeff Chastine

30. Greedy Fail 50 30 item 3 30 item 2 20 item 1 10 10 $100 $60 $120 Try again! Total = $180 Jeff Chastine

31. Optimal for 0-1 Problem 50 30 item 3 30 item 2 20 20 item 1 10 $100 $60 $120 Total = $220 Jeff Chastine

32. Fractional Example 50 item 3 30 item 2 20 item 1 10 $100 $60 $120 Jeff Chastine

33. Greedy – Go for the Gold! 50 item 3 30 item 2 20 item 1 10 $100 $60 $120 Jeff Chastine

34. Greedy – Go for the Gold! 50 item 3 30 item 2 20 item 1 10 10 $100 $60 $120 Jeff Chastine

35. Greedy – Get next best… 50 item 3 30 20 item 2 20 item 1 10 10 $100 $60 $120 Jeff Chastine

36. Greedy – AHA! Take only part 50 2030 item 3 30 20 item 2 20 item 1 10 10 $100 $60 $120 Jeff Chastine

37. Greedy – Go for the Gold! 50 2030 $80 item 3 30 20 item 2 $100 20 item 1 10 10 $60 $100 $60 $120 Total = $240 Note: optimal for 0-1 was $220 Jeff Chastine

38. Why the Greedy Fail? 50 50 30 item 3 30 20 item 2 20 item 1 10 10 10 $100 $60 $120 Notice the waste of space! Jeff Chastine

39. Huffman Codes • Objective: compress text data • Raw ASCII text file uses 8 bits/char (fixed) • 1000 characters = 8K bits • Book uses 6 characters for 3 bits/char • Trick • Analyze: not all characters are used! • Analyze: frequency of characters • Prefix codes: no binary code is prefix of another Jeff Chastine

40. Analysis a b c d e f Frequency (in thousands) 45 13 12 16 9 5 Fixed-length codeword 000 001 010 011 100 101 Variable-length codeword 0 101 100 111 1101 1100 110001001101 = face //12 bits 01011010 = abba // 8 bits Jeff Chastine

41. How it works • Sort by least frequent e:9 c:12 b:13 d:16 a:45 f:5 Jeff Chastine

42. How it works • Group lowest two e:9 c:12 b:13 d:16 a:45 f:5 Jeff Chastine

43. How it works • Combine and re-sort 14 c:12 b:13 d:16 a:45 1 0 e:9 f:5 Watch how least frequent characters are pushed down the tree! Jeff Chastine

44. How it works • Group lowest two 14 c:12 b:13 d:16 a:45 1 0 e:9 f:5 Jeff Chastine

45. How it works • Re-sort 14 25 d:16 a:45 1 1 0 0 e:9 f:5 c:12 b:13 Jeff Chastine

46. How it works 14 25 d:16 a:45 1 1 0 0 e:9 f:5 c:12 b:13 Jeff Chastine

47. How it works 25 30 a:45 0 1 1 0 14 c:12 b:13 d:16 1 0 e:9 f:5 Notice what’s happening to the first grouping we did Jeff Chastine

48. How it works 25 30 a:45 0 1 1 0 14 c:12 b:13 d:16 1 0 e:9 f:5 Jeff Chastine

49. How it works 55 a:45 0 1 25 30 0 1 1 0 14 c:12 b:13 d:16 1 0 e:9 f:5 Jeff Chastine

50. How it works 100 1 0 55 a:45 0 1 25 30 0 1 1 0 14 c:12 b:13 d:16 1 0 e:9 f:5 Jeff Chastine