1 / 36

Minimum Cost Binary Trees

Minimum Cost Binary Trees. Speaker: Dana Moshkovitz. Outline. Minimum Cost Binary Trees: The problem’s description The Garsia-Wachs algorithm Kingston’s proof for the Garsia-Wachs algorithm. Minimum Cost Binary Trees: Description.

sanjiv
Download Presentation

Minimum Cost Binary Trees

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Minimum Cost Binary Trees Speaker: Dana Moshkovitz

  2. Outline • Minimum Cost Binary Trees: The problem’s description • The Garsia-Wachs algorithm • Kingston’s proof for the Garsia-Wachs algorithm

  3. Minimum Cost Binary Trees: Description Given a list of n weights (nonnegative real numbers), p1,...,pn, let us look at the class of binary trees, which these weights are assigned to their leaves in the same order given. we would like to find the tree with the minimum weighted external path length, i.e. that tree T, s.t is minimal (hi, namely the level of pi, is the number of arcs in T along the path from the root to the leaf, whose weight is pi) Let’s observe an example...

  4. Minimum Cost Binary TreesPresenting the problem Example 10 7 2 4 12 6 10 7 2 4 12 6 Suppose these are the weights we get: We can construct the following tree: Now let us add to each leave its weighted external path length Total: 106 20 14 6 12 36 18

  5. Minimum Cost Binary TreesPresenting the problem Example Total: 106 Clearly, this tree was not in fact minimal: 20 14 10 7 6 12 36 18 2 4 12 6 20 24 12 10 12 6 This is because we can construct a “better” tree. Such tree is this one: 21 Total: 101 7 8 16 2 4

  6. Minimum Cost Binary TreesSolving the problem Intuition Our first observation is that the smaller the weight, the lower it should be in the tree. The above obvious observation is all we needed in the unordered version. Let us recall Huffman’s simple greedy algorithm. The general idea behind his solution was to construct the tree from the bottom and always choose the two smallest weights to be siblings. Let us demonstrate...

  7. Minimum Cost Binary TreesSolving the problem Intuition - Back to Huffman’s Algorithm 10 7 2 4 12 6 41 24 12 17 6 We choose the two less expensive fatherless weights and create them a father Again we have these weights: Total: 100

  8. Minimum Cost Binary TreesSolving the problem Intuition The question arising here is whether and how can we apply the same idea to the ordered case. The problem now is that we cannot arbitrarily choose the two cheapest leaves. The two leaves we choose must be neighbors. Unfortunately, this demand makes the naive greedy algorithm incorrect (For example, suppose the weights are 4,3,4,4. The correctness of the result is depended on whether we choose <4,3> or <3,4> to be the minimal pair). So, what do we do??

  9. Minimum Cost Binary TreesSolving the problem Toward a Solution • The feel is that we have some pretty useful ideas: • Constructing the tree from the bottom • Finding siblings in each iteration • However, as we have already seen, they still need some refinement.

  10. Minimum Cost Binary TreesSolving the problem Toward a Solution We need an additional observation: Suppose we have weights p1,...,pn and levels h1,...,hn. There is only one binary tree (at most) determined by this data, and that tree can be computed efficiently. Let’s demonstrate and explain this:

  11. Minimum Cost Binary TreesSolving the problem Toward a Solution 10 7 2 4 12 6 2 3 4 4 2 2 Given feasible weights p1,...,pn and levels h1,...,hn, let us see how can we construct the binary tree they represent. Algorithmically we can obtain this by preserving an array, which will hold in its i-th entry a pointer to a node in the i-th level lacking a right son.

  12. Minimum Cost Binary TreesSolving the problem Toward a Solution We conclude, that it suffices to solve the unordered version under the constraint that the level of each leaf in the resulting tree equals the level of the corresponding leaf of an ordered solution. What we need to explain now is how do we solve the unordered problem under this constraint. The question is how do we choose the right pair of neighbors.

  13. Minimum Cost Binary TreesSolving the problem Right Minimal Pairs Remark: From now on, we shall treat our list of weights as though it consists of additional two sentinels, namely and +1, placed in the leftmost and rightmost positions in the list respectively. This is done for the ease of the discussion, proving the modification does not alter the solution is rather trivial (see Gilbert and Moore’s result). Let us define the following: A pair of leaves pi-1,pi is right minimal (briefly R.M.) if (i) 1<in (ii) pi-2+pi-1  pi-1+pi (iii) pi-1+pi < pj-1+pj for all j>i In other words, two neighbors are right minimal if their sum is minimal among the sums to their right, but this does not hold for the pairs to their left.

  14. Minimum Cost Binary TreesSolving the problem Some Interesting Facts about R.M. Pairs 3 4 4 5 10 6 7 < 8 < 9 < 15 < 16 Lemma 1: Suppose we have a sequence of at least three weights pa,pa+1,...,pb, s.t. pj-1+ pj< pj+ pj+1, for a<j<b, then ha ha+1... hb-1 in every minimal tree containing these weights.

  15. Minimum Cost Binary TreesSolving the problem Some Interesting Facts about R.M. Pairs pj-1 R pj pj-1 pj R Proof. Suppose hj-1<hj for some a<j<b. pj must be a left child. Then the transformation: is both legal and less expensive: since pj+1 must be in R, |R|pj+1>pj-1. Therefore the lemma holds. 

  16. Minimum Cost Binary TreesSolving the problem Some Interesting Facts about R.M. Pairs Lemma 2: if pi-1,pi is the rightmost R.M. pair, then hi-1 hi... hn in every minimal tree. Proof. Directly follows from the choice of the pair and from lemma 1.  Further explanation: notice that when a pair is the rightmost R.M pair, the sequence of pairs to its right constructs a monotone series. (Otherwise the pair that “breaks” the monotony would be the rightmost R.M pair).

  17. Minimum Cost Binary TreesSolving the problem Some Interesting Facts about R.M. Pairs pi R pi-1 pi-1 pi R Lemma 3: if pi-1,pi is the rightmost R.M. pair, then hi-1= hi in some minimal tree. Proof. By lemma 2 it suffices to show that hi-1 hi in some minimal tree. Suppose we have a minimal tree such that hi-1>hi . In this tree pi-1is a right child, so we can use the following transformation to get another minimal tree: Again, |R| pi-2 pi, so the new tree is necessarily minimal. 

  18. Minimum Cost Binary TreesSolving the problem ... And finally... After we clarified the most important notion for the purpose of this lecture, namely R.M. pairs, and proved some interesting properties related to such, let us finally observe the desired algorithm. Afterwards we shall prove the correctness of our solution.

  19. Minimum Cost Binary TreesSolving the problem The Algorithm Execute the following two steps n-1 times: (1) Locate the rightmost R.M pair of entries. (2) Find the first entry to its right, which is greater than/equals it, and move the pair to the left of this entry.

  20. Minimum Cost Binary TreesSolving the problem Simulating the algorithm 10 7 2 4 12 6 41 2 3 4 4 2 2 23 23 13 13 6 6 18 18 10 7 2 4 12 6 10 7 2 4 12 6 We start with the same input The heights yield the tree we have already seen: We locate the rightmost R.M pair, pair them and move them to their right place.

  21. Minimum Cost Binary TreesProving the Solution Lemma 4: Let pi+k+1 be the first node to the right of the rightmost R.M pair pi-1,pi, s.t pi+k+1> pi-1+pi. In some minimal tree T for which hi-1= hi, either (a) hi+k= hi-1,or (b) hi+k= hi, and pi+k is a right child. Proof. Begin with T the minimal tree of lemma3. By lemma 2 we merely need to show that hi+k hi-1 in T. Let us suppose hi+k< hi-1. Let pm (mi+k) be the first node to the right of pi, s.t. hm< hi-1. pm<pi-1+pi.

  22. Minimum Cost Binary TreesProving the Solution pm . . . p q . . . pm p q There are two possibilities: (a) pi-1,pi are siblings in T (b) pi,pi+1 are siblings in T Let p,q denote the two siblings. We can transform T as follows and improve the cost by decreasing p+q-pm pi-1+pi-pm>0. This contradicts the hypothesis T was minimal. Hence hi+k hi-1.

  23. Minimum Cost Binary TreesProving the Solution . . . . . . pi+k+1 pi-1 pi pi+k pi+k+1 pi+k pi-1 pi It remains to prove, that when hi+k= hi, pi+k can be made a right child. If pi+k is already a right child, we are done. Otherwise, if k>0, we can transform T:

  24. Minimum Cost Binary TreesProving the Solution While if k=0, we can use this transformation: pi+k+1 pi-1 pi pi+k+1 pi-1 pi In both cases, our claim holds. 

  25. Minimum Cost Binary TreesProving the solution Lemma 5: For every rearrangement done in an iteration of the algorithm, the cost for the new series cannot exceed the cost for the original series of weights. Proof. Let k be the number of places we move the R.M. pair in a specific iteration. We will exhibit a tree T’ for the new arrangement, whose leaves has the same levels as in T, the minimum tree of lemma 4. This - of-course - will prove the lemma. If k=0, pi-1 and pi are siblings in T, so we may take T’=T.

  26. Minimum Cost Binary TreesProving the solution pi-1 pi pi+1 pi+1 pi-1 pi Otherwise k>0. First we will transform T, so pi-1 and pi (the current rightmost R.M. pair) are siblings (If they they are not so already): Lemma 4 states, that they have the same level Lemma 4 states pi+1 is a right child

  27. Minimum Cost Binary TreesProving the solution . . . pi+k . . . pi+k pi-1 pi pi-1 pi . . . . . . pi-1 pi pi+k pi+k pi-1 pi Since k>0, the first weight greater than/equal to pi-+pi, namely pi+k+1, is still to the right of pi. By lemma 4, there are merely two possibilities: hi+k=hi or hi+k=hi-1. In each case all we need to do is “slid” the new node to the right until it passes pi+k. 

  28. Minimum Cost Binary TreesProving the solution Lemma 6: For every rearrangement done in an iteration of the algorithm, the cost for the new series is at least the cost for the original series of weights, and if equality holds, then the two solutions have corresponding levels. Proof. Now - given a minimum tree T’ for the new arrangement - we need to construct a minimum tree T for the original series (while preserving the levels). Again we denote the number of places we move the pair by k. If k=0, T=T’.

  29. Minimum Cost Binary TreesProving the solution pi+1 Tx Tx . . . . . . pi-1 pi . . . pi-1 pi pi+1 Otherwise k>0. We know that pi+k<pi-1+pipi+k+1, so we can use lemma 1 in T’ with pi+1,...,pi+k,px, pi+k+1 (px is the father of the R.M. pair) to obtain: hi+1... hi+khx in T’. Hence, either or In the first case the treatment is simple: we need to move Tx to the left.

  30. Minimum Cost Binary TreesProving the solution (a) Tx Tx . . . . . . . . . . . . pi-1 pi pi-1 pi pi+1 pi+1 The treatment of the second case consists of two phases: (a) Slid Tx to the left across all the leaves in its level. (b) Rotate all the nodes two places to the right.

  31. Minimum Cost Binary TreesProving the solution How do we perform the rotation? A simulation Tx pi-1 pi pi-1 pi+1 pi+1 pi pi-1 pi+1 pi

  32. Minimum Cost Binary TreesProving the solution For every level the pair drops downward (possibly 0), some other pair moves up one level. But we have taken a R.M. pair (for which all the pairs with greater indices weight more than it does). This implies that w(T’)=w(Ta)w(T), so the weight of T’ is at least the weight of a minimum cost tree for the original series of weights. If equality holds, the level of the pair is not changed by the rotation, so T is a minimum cost tree for the original series, which preserves the levels of T’.

  33. Minimum Cost Binary TreesProving the solution Those two final lemmas allow us to finally state the following: The algorithm we presented is correct. 

  34. Minimum Cost Binary TreesSolving the problem Another way to view this algorithm 10 7 2 4 12 6 41 18 23 2 3 4 4 2 2 10 13 18 10 7 6 18 10 7 6 12 6 0 1 0 0 0 2 2 0 3 0 1 0 2 0 3 1 1 4 3 1 0 2 4 1 1 0 0 1 2 1 0 1 1 1 2 0 10 7 2 4 12 6 Which yield the tree we have already seen: We start with the same input We locate the rightmost R.M pair, pair them and move them to their right place.

  35. Minimum Cost Binary TreesReferences This overview was based on: [1] J. H. Kingston. A new proof for the Garcia-Wachs algorithm. J. of Algorithms, 9:129-136,1988 The Garsia-Wachs algorithm: [2] A. M. Garsia and M. L. Wachs, A new algorithm for minimum cost binary trees. Sicomp, 6(4), 622-642, 1977 Huffman’s algorithm: [3] D. Huffman, A method for the construction of minimum- redundancy codes, Proc. IRL, 1098-1101, 1952

  36. Minimum Cost Binary TreesReferences You might be interested in previous results concerning this problem: The original O(n3) algorithm: [4] E. N. Gilbert and E.F. Moore, Variable length binary encodings, Bell Systems Tech, 38, 933-968, 1977. An O(n2) algorithm: [5] D. E. Knuth, Optimum binary search trees, Acta Inform. 1, 14-25, 1971. The Hu-Tucker algorithm (O(nlogn)): [6] T.C. Hu and C. Tucker, Optimum computer search trees, SIAM J. Appl. Math., 514-532, 1971. [7] T.C. Hu, A new proof of the T-C algorithm, SIAM J. Appl. Math. 25, 83-94, 1973.

More Related