A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding

1. A Generic Top-Down Dynamic-Programming Approach to Prefix-Free coding YalinZhang zhangyl@tamu.edu

2. Huffman Coding • Given a probability distribution over a set of n words, find a minimal-cost prefix-free coding for these words. • Input: alphabet , probability • Output: code • Cost =

3. Huffman Coding • Complexity greedy algorithm for binary case. • Complexity P is sorted, r-ary case. What about extra constraints or generalization?

4. Variations of Huffman Coding • Length-limited coding • Unequal-cost coding • Mixed-radix coding • Reserved-length coding • One-ended coding

5. Variations of Huffman Coding Standard binary tree Mix-radix tree Reserved-length tree General approach: Dynamic Programming to build an optimal coding tree.

6. Top-Down DP for GMR coding • Generalized-Mixed-Radix problem • Both the arity of an internal node and the length of an edge depends on its level. • Given a sequence of arities R = (r1, r2, . . .) and a sequence of edge lengths C = (c1, c2 . . .), a generalized mixed radix (GMR) tree T satisfying R, C is a tree in which internal node v at level i − 1 has at most ri children and and the length of an edge from v to any of its children is ci

7. Top-Down DP for GMR coding • The depth of a node on level i of the tree will be the sum of the lengths of the edges on the path from the root to level i. • Cost(T) = • The problem to be solved is, given P and R,C, find a min-cost tree T∗ with n leaves Cost(T∗) = min {Cost(T) | T has n leaves}

8. Top-Down Dynamic Program • In , ≥ ≥ … ≥ ≥ 0; • For i > n, = 0; • There is a full tree T∗ with the same cost as the optimal tree for n leaves. T∗ has n′ leaves where max(n, rℓ(T∗)) ≤ n′ ≤ n + rℓ(T∗) − 1 • The cost of a tree is fully determined by the number of leaves on each level.

9. Top-Down Dynamic Program • Build optimal trees top-down, starting with a root; • Expand tree level-by-level by making some nodes on the bottom level internal. • Until a tree whose bottom level contains no internal nodes.

10. Top-Down Dynamic Program • i-level signature is an ordered pair in which m = b = |{}| Partial cost of T is

11. Top-Down Dynamic Program • P={1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12}

12. Top-Down Dynamic Program • Let T′ be an (i − 1)-level tree with. Expand T′ to a full i-level tree by adding all nodes on level i. • (m’, b’)->(m,b), where m = m’ + b’ri – b • (0, 1)->…->(n’, 0)

13. Top-Down Dynamic Program • Objective: • Complexity? • Speedup by batching approach

14. Batching for Speed up

15. Batching for Speed up • If • Else

16. Batching for Speed up • For fixed i, we can fill all valid signatures (m, b), where m+b=d, in time. • d is bounded by n. • We can fill all valid signatures in level i in time. • The overall time complexity is

17. Results