CSc 461/561 Multimedia Systems Part B: 1. Lossless Compression

1. CSc 461/561Multimedia Systems Part B: 1. Lossless Compression

2. Summary (1) Information (2) Types of compression (3) Lossless compression algorithms • Shannon-Fano Algorithm • Huffman coding • Run-length coding • LZW compression • Arithmetic Coding (4) Example: Lossless image compression

3. 1. Information (1) Information is decided by three parts: • The source • The receiver • The delivery channel We need a way to measure information: • Entropy: a measure of uncertainty; min bits • alphabet set {s1, s2, …, sn} • probability {p1, p2, …, pn} • entropy: - p1 log2 p1 - p2 log2 p2 - … - pn log2 pn

4. 1. Entropy examples (2) • Alphabet set {0, 1} • Probability: {p, 1-p} • Entropy: H = - p log2 p - (1-p) log2 (1-p) • when p=0, H=0 • when p=1, H=0 • when p=1/2, Hmax=1 • 1 bit is enough!

5. 2. Types of compression (1) • Lossless compression: no information loss • Lossy compression: otherwise

6. 2. Compression Ratio (2) • Compression ratio • B0: # of bits to represent before compression • B1: # of bits to represent after compression • compression ratio = B0/B1

7. 3.1 Shannon-Fano algorithm (1) • Fewer bits for symbols appear more often • “divide-and-conquer” • also known as “top-down” approach • split alphabet set into subsets of (roughly) equal probabilities; do it recursively • similar to building a binary tree

8. 3.1 Shannon-Fano: examples (2)

9. 3.1 Shannon-Fano: results (3) • Prefix-free code • no code is a prefix of other codes • easy to decode

10. 3.1 Shannon-Fano: more results (4) • Encoding is not unique • roughly equal Encoding 2 Encoding 1

11. 3.2 Huffman coding (1) • “Bottom-up” approach • also build a binary tree • and know alphabet probability! • start with two symbols of the least probability • s1: p1 • s2: p2 • s1 or s2: p1+p2 • do it recursively

12. combine combine Sort Sort combine Sort combine Sort 0 0.6 0.4 1 1 1 0.4 1 a2 (0.4) a1(0.2) a3(0.2) a4(0.1) a5(0.1) 1 0.4 01 0.2 00 01 01 0.4 1 0.6 000 0.2 0.2 000 000 0.4 01 0010 0010 0.2 0.2 001 0011 Assign code 0011 3.2 Huffman coding: examples (2) • Encoding not unique; prefix-free code • Optimality: H(S) <= L < H(S)+1

13. 3.3 Run-length coding • Run: a string of the same symbol • Example • input: AAABBCCCCCCCCCAA • output: A3B2C9A2 • compression ratio = 16/8 = 2 • Good for some inputs (with long runs) • bad for others: ABCABC • how about to treat ABC as an alphabet?

14. 3.4 LZW compression (1) • Lempel-Ziv-Welch (LZ77, W84) • Dictionary-based compression • no a priori knowledge on alphabet probability • build the dictionary on-the-fly • used widely: e.g., Unix compress • LZW coding • if a word does not appear in the dictionary, add it • refer to the dictionary when the word appears again

15. 3.4 LZW examples (2) • Input • ABABBABCABABBA • Output • 1 2 4 5 2 3 4 6 1

16. 3.5 Arithmetic Coding (1) • Arithmetic coding determines a model of the data -- basically a prediction of what patterns will be found in the symbols of the message. The more accurate this prediction is, the closer to optimality the output will be. • Arithmetic coding treats the whole message as one unit.

17. 3.5 Arithmetic Coding (2)

18. 3.5 Arithmetic Coding (3)

19. 3.5 Arithmetic Coding (4)

20. 3.5 Arithmetic Coding (5)

21. 4. Lossless Image Compression (1)

22. 4. Lossless Image Compression (2)

23. 4. Lossless JPEG NNeighboring Pixels for Predictors in Lossless JPEG NeighPredictors for Lossless JPEG