SWE 423: Multimedia Systems

1. SWE 423: Multimedia Systems Chapter 7: Data Compression (3)

2. Outline • Entropy Encoding • Arithmetic Coding

3. Entropy Encoding: Arithmetic Coding • Initial idea introduced in 1948 by Shannon • Many researchers worked on this idea • Modern arithmetic coding can be attributed to Pasco (1976) and Rissanen and Langdon (1979) • Arithmetic coding treats the whole message as one unit • In practice, the input data is usually broken up into chunks to avoid error propagation

4. Entropy Encoding: Arithmetic Coding • A message is represented by a half-open interval [a,b), where a,b. • General idea of encoding • Map the message into an open interval [a,b) • Find a binary fractional number with minimum length that belongs to the above interval. This will be the encoded message • Initially, [a,b) = [0,1) • When the message becomes longer, the length of the interval shortens and the # of bits needed to represent the interval increases

5. Entropy Encoding: Arithmetic Coding • Coding Algorithm Algorithm ArithmeticCoding // Input: symbol: Input stream of the message terminator: terminator symbol // : Low[] and High[]: all symbols’ ranges // Output: binary fractional code of the message low = 0; high = 1; range = 1; while (symbol != terminator) { get (symbol); high = low + range * High(symbol); low = low + range * Low(symbol); range = high – low; } return CodeWord(low,high);

6. Entropy Encoding: Arithmetic Coding • Binary code generation Algorithm CodeWord // Input: low and high // Output: binary fractional code code = 0; k = 1; while (value(code) < low) { assign 1 to the kth binary fraction bit; if (value(code) > high) replace the kth bit by 0; k++; }

7. Entropy Encoding: Arithmetic Coding • Example: Assume S = {A,B,C,D,E,F,$}, where $ is the terminator symbol. In addition, assume the following probabilities for each character: • Pr (A) = 0.2 • Pr(B) = 0.1 • Pr(C) = 0.2 • Pr(D) = 0.05 • Pr(E) = 0.3 • Pr(F) = 0.05 • Pr($) = 0.1 Generate the fractional binary code of the message CAEE

8. Entropy Encoding: Arithmetic Coding • It can be proven that log2 (1/ Pi) is the upper bound on the number of bits needed to encode a message • In our case, the maximum is equal to 12. • When the length of the message increases, the range decreases and the upper bound value ...... • Generally, arithmetic coding outperforms Huffman coding • Treats the whole message as one unit vs. an integral number of bits to code each character in Huffman coding • Redo the previous example CAEE$ using Huffman coding and notice how many bits are required to code this message.

9. Entropy Encoding: Arithmetic Coding • Decoding Algorithm Algorithm ArithmeticDecoding // Input: code: binary code // : Low[] and High[]: all symbols’ ranges // Output: The decoded message value = convert2decimal(code); Do { find a symbol s so that Low(s) <= value < High(s); output s; low = Low(s); high = High(s); range = high – low; value = (value – low) / range; } while s is not the terminator symbol;

10. Entropy Encoding: Arithmetic Coding • Example