1 / 29

Approaches to Problem Solving

Approaches to Problem Solving. greedy algorithms dynamic programming backtracking divide-and-conquer. Problem: Huffman Coding. Def: binary character code = assignment of binary strings to characters. e.g. ASCII code A = 01000001 B = 01000010 C = 01000011

jorged
Download Presentation

Approaches to Problem Solving

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Approaches to Problem Solving • greedy algorithms • dynamic programming • backtracking • divide-and-conquer

  2. Problem: Huffman Coding Def: binary character code = assignment of binary strings to characters e.g. ASCII code A = 01000001 B = 01000010 C = 01000011 … fixed-length code How to decode: ? 01000001010000100100001101000001

  3. Problem: Huffman Coding Def: binary character code = assignment of binary strings to characters e.g. code A = 0 B = 10 C = 11 … variable-length code How to decode: ? 0101001111

  4. Problem: Huffman Coding Def: binary character code = assignment of binary strings to characters e.g. code A = 0 B = 10 C = 11 … variable-length code Def: A code is prefix-free if no codeword is a prefix of another codeword. How to decode: ? 0101001111

  5. Problem: Huffman Coding Def: binary character code = assignment of binary strings to characters e.g. another code A = 1 B = 10 C = 11 … variable-length code Def: A code is prefix-free if no codeword is a prefix of another codeword. How to decode: ? 10101111

  6. Problem: Huffman Coding Def: Huffman coding is an optimal prefix-free code. • Optimization problems • Input: • Output: • Objective:

  7. Problem: Huffman Coding Def: Huffman coding is an optimal prefix-free code. • Huffman coding • Input: • Output: • Objective: an alphabet with frequencies a prefix-free code minimize expected number of bits per character

  8. Problem: Huffman Coding Example: A 60% B 20% C 10% D 10% Is fixed-width coding optimal ? • Huffman coding • Input: • Output: • Objective: an alphabet with frequencies a prefix-free code minimize expected number of bits per character

  9. Problem: Huffman Coding Example: A 60% B 20% C 10% D 10% Is fixed-width coding optimal ? NO, exists a prefix-free code using 1.6 bits per character ! • Huffman coding • Input: • Output: • Objective: an alphabet with frequencies a prefix-free code minimize expected number of bits per character

  10. Problem: Huffman Coding • Huffman ( [a1,f1],[a2,f2],…,[an,fn]) • if n=1 then • code[a1] “” • else • let fi,fj be the 2 smallest f’s • Huffman ( [ai,fi+fj],[a1,f1],…,[an,fn] ) • omits ai,aj • code[aj]  code[ai] + “0” • code[ai]  code[ai] + “1” • Huffman coding • Input: • Output: • Objective: an alphabet with frequencies a prefix-free code minimize expected number of bits per character

  11. Problem: Huffman Coding Lemma 1: Let x,y be the symbols with frequencies fx > fy. Then in an optimal prefix code length(Cx)  length(Cy).

  12. Problem: Huffman Coding Lemma 1: Let x,y be the symbols with frequencies fx > fy. Then in an optimal prefix code length(Cx)  length(Cy). Lemma 2: If w is a longest codeword in an optimal code then there exists another codeword of the same length.

  13. Problem: Huffman Coding Lemma 1: Let x,y be the symbols with frequencies fx > fy. Then in an optimal prefix code length(Cx)  length(Cy). Lemma 2: If w is a longest codeword in an optimal code then there exists another codeword of the same length. Lemma 3: Let x,y be the symbols with the smallest frequencies. Then there exists an optimal prefix code such that the codewords for x and y differ only in the last bit.

  14. Problem: Huffman Coding Lemma 1: Let x,y be the symbols with frequencies fx > fy. Then in an optimal prefix code length(Cx)  length(Cy). Lemma 2: If w is a longest codeword in an optimal code then there exists another codeword of the same length. Lemma 3: Let x,y be the symbols with the smallest frequencies. Then there exists an optimal prefix code such that the codewords for x and y differ only in the last bit. Theorem: The prefix code output by the Huffman algorithm is optimal.

  15. Greedy Algorithms • make decisions “greedily”, previous decisions are never reconsidered • e.g. Huffman coding • another example: the divisible version of KNAPSACK KNAPSACK Input: Output: Objective: a number W and a set of n items, the i-th item has a weight wi and a cost ci a subset of items with total weight · W maximize cost Version 1: Items are divisible.

  16. KNAPSACK – divisible: a greedy solution • KNAPSACK-DIVISIBLE(n,c,w,W) • sort items in decreasing order of ci/wi • i = 1 • currentW = 0 • while (currentW + wi < W) { • take item of weight wi and cost ci • currentW += wi • i++ • } • take W-currentW portion of item i Correctness: Running time:

  17. KNAPSACK – indivisible Version 2: Items are indivisible. Does previous algorithm work for this version of KNAPSACK?

  18. BackTracking • “brute force”: try all possibilities • running time?

  19. Dynamic Programming • idea: remember values for smaller instances, use it for computing the value for a larger instance Example: Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, …

  20. Dynamic Programming • idea: remember values for smaller instances, use it for computing the value for a larger instance Example: Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, … A recursive solution: • Fib(n) • if n < 0 then RETURN “undefined” • if n · 1 then RETURN n • RETURN Fib(n-1) + Fib(n-2)

  21. Dynamic Programming • idea: remember values for smaller instances, use it for computing the value for a larger instance Example: Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, … A recursive solution: • Fib(n) • if n < 0 then RETURN “undefined” • if n · 1 then RETURN n • RETURN Fib(n-1) + Fib(n-2) Can do better (faster) ?

  22. Dynamic Programming • idea: remember values for smaller instances, use it for computing the value for a larger instance Example: Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, … A recursive solution: • Fib(n) • if n < 0 then RETURN “undefined” • if n · 1 then RETURN n • RETURN Fib(n-1) + Fib(n-2) Can do better (faster) ? YES ! Just store Fib[k] in an array !

  23. Dynamic Programming • idea: remember values for smaller instances, use it for computing the value for a larger instance Example: Fibonacci sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, … • FibDynProg(n) • Fib[0] = 0 • Fib[1] = 1 • for i=2 to n do • Fib[i] = Fib[i-1] + Fib[i-2] • RETURN Fib[n] The heart of the solution: Fib[k] = the k-th Fibonacci number Can do better (faster) ? YES ! Just store Fib[k] in an array !

  24. KNAPSACK – indivisible: a dyn-prog solution A recursive approach (returns only the best cost, not the set of items): • KNAP-IND-REC(n,c,w,W) • if n·0 • return 0 • if W < wn • withLastItem = -1 // undefined • else • withLastItem =cn+KNAP-IND-REC(n-1,c,w,W-wn) • withoutLastItem = KNAP-IND-REC(n-1,c,w,W) • return max{withLastItem,withoutLastItem}

  25. KNAPSACK – indivisible: a dyn-prog solution The heart of the algorithm: S[k][v] =

  26. KNAPSACK – indivisible: a dyn-prog solution The heart of the algorithm: S[k][v] = maximum cost of a subset of the first k items, where the weight of the subset is at most v

  27. KNAPSACK – indivisible: a dyn-prog solution The heart of the algorithm: S[k][v] = maximum cost of a subset of the first k items, where the weight of the subset is at most v • KNAPSACK-INDIVISIBLE(n,c,w,W) • init S[0][v]=0 for every v=0,…,W • init S[k][0]=0 for every k=0,…,n • 3. for v=1 to W do • for k=1 to n do • S[k][v] = S[k-1][v] • if (wk· v) and • (S[k-1][v-wk]+ck > S[k][v]) • then • S[k][v] = S[k-1][v-wk]+ck • RETURN S[n][W]

  28. KNAPSACK – indivisible: a dyn-prog solution The heart of the algorithm: S[k][v] = maximum cost of a subset of the first k items, where the weight of the subset is at most v • KNAPSACK-INDIVISIBLE(n,c,w,W) • init S[0][v]=0 for every v=0,…,W • init S[k][0]=0 for every k=0,…,n • 3. for v=1 to W do • for k=1 to n do • S[k][v] = S[k-1][v] • if (wk· v) and • (S[k-1][v-wk]+ck > S[k][v]) • then • S[k][v] = S[k-1][v-wk]+ck • RETURN S[n][W] Running time:

  29. KNAPSACK – indivisible: a dyn-prog solution The heart of the algorithm: S[k][v] = maximum cost of a subset of the first k items, where the weight of the subset is at most v • KNAPSACK-INDIVISIBLE(n,c,w,W) • init S[0][v]=0 for every v=0,…,W • init S[k][0]=0 for every k=0,…,n • 3. for v=1 to W do • for k=1 to n do • S[k][v] = S[k-1][v] • if (wk· v) and • (S[k-1][v-wk]+ck > S[k][v]) • then • S[k][v] = S[k-1][v-wk]+ck • RETURN S[n][W] How to output a solution ?

More Related