1 / 20

Lossless Compression Multimedia Systems (Module 2 Lesson 2)

Summary: Adaptive Coding Adaptive Huffman Coding Sibling Property Update Algorithm Arithmetic Coding Coding and Decoding Issues: EOF problem, Zero frequency problem. Sources: The Data Compression Book, 2 nd Ed., Mark Nelson and Jean-Loup Gailly.

ciqala
Download Presentation

Lossless Compression Multimedia Systems (Module 2 Lesson 2)

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Summary: Adaptive Coding Adaptive Huffman Coding Sibling Property Update Algorithm Arithmetic Coding Coding and Decoding Issues: EOF problem, Zero frequency problem Sources: The Data Compression Book, 2nd Ed., Mark Nelson and Jean-Loup Gailly. Introduction to Data Compression, by Sayood Khalid The Sqeeze Page at SFU Lossless CompressionMultimedia Systems (Module 2 Lesson 2)

  2. Adaptive Coding Motivations: • The previous algorithms (both Shannon-Fano and Huffman) require the statistical knowledge which is often not available (e.g., live audio, video). • Even when it is available, it could be a heavy overhead. • Higher-order models incur more overhead. For example, a 255 entry probability table would be required for a 0-order model. An order-1 model would require 255 such probability tables. (A order-1 model will consider probabilities of occurrences of 2 symbols) The solution is to use adaptive algorithms. Adaptive Huffman Coding is one such mechanism that we will study. The idea of “adaptiveness” is however applicable to other adaptive compression algorithms.

  3. ENCODER Initialize_model(); do { c = getc( input ); encode( c, output ); update_model( c ); } while ( c != eof) DECODER Initialize_model(); while ( c = decode (input)) != eof) { putc( c, output) update_model( c ); } Adaptive Coding • The key is that, both encoder and decoder use exactly the same initialize_model and update_model routines.

  4. The Sibling Property The node numbers will be assigned in such a way that: • A node with a higher weight will have a higher node number • A parent node will always have a higher node number than its children. In a nutshell, the sibling property requires that the nodes (internal and leaf) are arranged in order of increasing weights. The update procedure swaps nodes in violation of the sibling property. • The identification of nodes in violation of the sibling property is achieved by using the notion of a block. • All nodes that have the same weight are said to belong to one block

  5. Flowchart of the update procedure • The Huffman tree is initialized with a single node, known as the Not-Yet-Transmitted (NYT) or escape code. This code will be sent every time that a new character, which is not in the tree, is encountered, followed by the ASCII encoding of the character. This allows for the de-compressor to distinguish between a code and a new character. Also, the procedure creates a new node for the character and a new NYT from the old NYT node. • The root node will have the highest node number because it has the highest weight. START First appearance of symbol Yes NYT gives birth To new NYT and external node No Increment weight of external node and old NYT node; Adjust node numbers Go to symbol external node No Node number max in block? Switch node with highest numbered node in block Go to old NYT node Yes Increment node weight Is thisthe rootnode? No Go toparent node Yes STOP

  6. Example NYT #0 Initial Huffman Tree Counts: (number of occurrences) B:2 C:2 D:2 E:10 Root W=16 #8 W=6 #6 E W=10 #7 W=2 #4 W=4 #5 B W=2 #1 NYT C W=2 #2 D W=2 #3 #0 Example Huffman tree after some symbols have been processed in accordance with the sibling property

  7. Example Counts: (number of occurrences) A:1 B:2 C:2 D:2 E:10 Root W=16+1 #10 W=6+1 #8 E W=10 #9 W=2+1 #6 W=4 #7 B W=2 #3 W=1 #2 C W=2 #4 D W=2 #5 NYT A W=1 #1 #0 A Huffman tree after first appearance of symbol A

  8. Increment Counts: A:1+1 B:2 C:2 D:2 E:10 Root W=17+1 #10 W=7+1 #8 E W=10 #9 W=3+1 #6 W=4 #7 B W=2 #3 W=1+1 #2 C W=2 #4 D W=2 #5 NYT A W=1+1 #1 #0 An increment in the count for A propagates up to the root

  9. Swapping Another increment in the count for A results in swap Counts: A:2+1 B:2 C:2 D:2 E:10 Root W=18 #10 W=8 #8 E W=10 #9 W=4 #6 W=4 #7 W=2 #2 B W=2 #3 Swap nodes 1 and 5 C W=2 #4 D W=2 #5 A W=2 #1 NYT #0 Counts: A:3 B:2 C:2 D:2 E:10 Root W=18+1 #10 W=8+1 #8 E W=10 #9 W=4 #6 W=4+1 #7 W=2 #2 B W=2 #3 C W=2 #4 A W=2+1 #5 D W=2 #1 NYT #0

  10. Swapping … contd. Counts: A:3+1 B:2 C:2 D:2 E:10 Root W=19+1 #10 W=9+1 #8 E W=10 #9 W=4 #6 W=5+1 #7 B W=2 #3 W=2 #2 C W=2 #4 A W=3+1 #5 D W=2 #1 NYT #0 Another increment in the count for A propagates up

  11. Swapping … contd. Another increment in the count for A causes swap of sub-tree Counts: A:4+1 B:2 C:2 D:2 E:10 Root W=20 #10 W=10 #8 E W=10 #9 W=4 #6 W=6 #7 B W=2 #3 W=2 #2 C W=2 #4 A W=4 #5 D W=2 #1 NYT #0 Swap nodes 5 and 6

  12. Swapping … contd. Further swapping needed to fix the tree Counts: A:4+1 B:2 C:2 D:2 E:10 Root W=20 #10 W=10 #8 E W=10 #9 W=6 #7 A W=4+1 #6 C W=2 #4 W=4 #5 B W=2 #3 W=2 #2 D W=2 #1 NYT #0 Swap nodes 8 and 9

  13. Swapping … contd. Counts: A:5 B:2 C:2 D:2 E:10 Root W=20+1 #10 E W=10 #8 W=10+1 #9 W=6 #7 A W=5 #6 C W=2 #4 W=4 #5 B W=2 #3 W=2 #2 D W=2 #1 NYT #0

  14. Arithmetic Coding Arithmetic coding is based on the concept of interval subdividing. • In arithmetic coding a source ensemble is represented by an interval between 0 and 1 on the real number line. • Each symbol of the ensemble narrows this interval. • As the interval becomes smaller, the number of bits needed to specify it grows. • Arithmetic coding assumes an explicit probabilistic model of the source. • It uses the probabilities of the source messages to successively narrow the interval used to represent the ensemble. • A high probability message narrows the interval less than a low probability message, so that high probability messages contribute fewer bits to the coded ensemble

  15. Arithmetic Coding: Description • In the following discussions, we will use M as the size of the alphabet of the data source, • N[x] as symbol x's probability, • Q[x] as symbol x's cumulative probability (i.e., Q[i]=N[0]+N[1]+...+N[i]) • Assume we know the probabilities of each symbol of the data source, • we can allocate to each symbol an interval with width proportional to its probability, and each of the intervals does not overlap with others. • This can be done if we use the cumulative probabilities as the two ends of each interval. Therefore, the two ends of each symbol x amount to Q[x-1] and Q[x]. • Symbol x is said to own the range [Q[x-1], Q[x]).

  16. Arithmetic Coding: Encoder We begin with the interval [0,1) and subdivide the interval iteratively. • For each symbol entered, the current interval is divided according to the probabilities of the alphabet. • The interval corresponding to the symbol is picked as the interval to be further proceeded with. • The procedure continues until all symbols in the message have been processed. • Since each symbol's interval does not overlap with others, for each possible message there is a unique interval assigned. • We can  represent the message with the interval's two ends [L,H). In fact, taking any single value in the interval as the encoded code is enough, and usually the left end L is selected.

  17. Arithmetic Coding Algorithm L = 0.0; H = 1.0; While ( (x = getc(input)) != EOF ) { R = (H-L); H = L + R * Q[x]; L = L + R * Q[x-1]; } Output(L); R is the interval range, and H and L are two ends of the current code interval. x is the new symbol to be encoded. H and L are initialized to 0 and 1 respectively 

  18. 0.7 0.67 0.6286 0.634 C B B A 0.4 0.61 0.61 0.6196 Arithmetic Coding: Encoder example 1 String: BCAB Code sent: 0.6196 0

  19. Decoding Algorithm • When decoding the code v is placed on the current code interval to find the symbol x so that Q[x-1] <= code < Q[x]. The procedure iterates until all symbols are decoded. v = input_code(); for (;;) { x = find_symbol_straddling_this_range(v); putc(x); R = Q[x] – Q[x-1]; v = (v – Q[x-1])/R; }

  20. Arithmetic Coding: Issues • The zero-frequency problem: Each symbol's predicted probability must not be zero or the interval will become zero and interval renormalization would fail. This is called the zero-frequency problem. Models that adapt online may encounter such problem when decaying. • The EOF problem: • Assume we pick the lower end of the interval as the encoded code. Two messages may yield the same code if one message is identical to the other, except for a sequence of finite number of the first symbol(first in table, not in the sequence) as a suffix. • For e.g., Both BCAB, BCABA, BCABAA, BCABAAA will have the same lower interval but different upper intervals. (try it) • The simplest solution is to let the decoder know the length of the encoded message. The decoder will know if the message size is fixed or can be transmitted at first. However this is not plausible if the data size is not known beforehand, such as live broadcasting data; or it's too costly to do so, such as tapes whose size is unknown at the beginning. • There is another solution if we introduce a special EOF symbol to the alphabet. The symbol takes a small interval and is used only at the end of the message. When the decoder detects the EOF symbol it knows the end of the message is reached.

More Related