1 / 75

Lecture 3

Lecture 3. Mei-Chen Yeh 2010/03/16. Announcements (1). Assignment formats: A word or a pdf file Subject ( 主旨 ): Multimedia System Design-Assignment #X-your student id-your name Multimedia System Design-Assgnment #2-697470731- 游宗毅 File name ( 檔名 )

ona
Download Presentation

Lecture 3

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Lecture 3 Mei-Chen Yeh 2010/03/16

  2. Announcements (1) • Assignment formats: • A word or a pdf file • Subject (主旨): Multimedia System Design-Assignment #X-your student id-your name Multimedia System Design-Assgnment #2-697470731-游宗毅 • File name (檔名) Assignment #X-your student id-your name Assignment #2-697470731-游宗毅.doc

  3. Announcements (2) • For the assignment#2… • If you did not use either a pdf or a doc file, please re-send your report to TA using the format. • Due 03/16 (today) …and based on TA’s clock

  4. Announcements (3) • The reading list is finally released! • Sources: • Proceedings of ACM MM 2008 • Proceedings of ACM MM 2009 • The best paper in MM 2006 and MM 2007 • Interesting papers not on the list? Let me know!

  5. Announcements (4) • So you need to… • Browse the papers • Discuss with your partners about the paper choice • …and do it as soon as possible! I love that paper! That paper sucks…

  6. How to access the papers? • The ACM digital library • http://portal.acm.org/ • Should be able to download the papers if you connect to the site on campus • MM08 paper on the web • http://mpac.ee.ntu.edu.tw/multimediaconf/acmmm2008.html • Google search

  7. Next week in class • Bidding papers! (論文競標) • Each team will get a ticket, where you put your points. Ticket # 7 Team name: 夜遊隊 Team members: 葉梅珍游宗毅 ------------------------------------------------------------------- paper1 paper2 paper3 paper4 … paper 25 501015205 Total : 100 points

  8. Bidding rules • The team with the most points gets the paper. • Every team gets one paper. • When a tie happens… …and the winner takes the paper.

  9. More about the bidding process • Just, fair, and open! • I will assign a paper to teams in which no one show up for the bid. 公平 公正 公開 Questions

  10. Multimedia Compression (1)

  11. Outline • Introduction • Information theory • Entropy (熵) coding • Huffman coding • Arithmetic coding

  12. Why data compression? • Transmission and storage • For uncompressed video • CD-ROM (650MB) could store 650MB x 8 / 221Mbps ≈ 23.5 seconds • DVD-5 (4.7GB) could store about 3 minutes Approximate Bit Rates for Uncompressed Sources

  13. What is data compression? • To represent information in a compact form (as few bits as possible) • Technique Compressed data Reconstruction Compression Original Reconstructed data Codec = encoder + decoder

  14. Example: image codec Lossy! Encoder Decoder source Lossless? Not necessarily true! Technique (cont.) • Lossless • The reconstruction is identical to the original. • Lossy • Involves loss of information • Do not send money! • Do now send money!

  15. Performance MeasuresHow do we say a method is good or bad? • The amount of compression • How closely the reconstruction is • How fast the algorithm performs • The memory required to implement the algorithm • The complexity of the algorithm • …

  16. Two phases: modeling and coding Original Compressed data • Modeling • Discover the structure in the data • Extract information about any redundancy • Coding • Describe the model and the residual (how the data differ from the model) Encoder Fewer bits!

  17. Example (1) • 5 bits * 12 samples = 60 bits • Representation using fewer bits?

  18. n Example: Modeling

  19. Original data Model 10 11 12 13 14 15 16 17 18 19 20 Residual 0 1 0 -1 1 -1 0 1 -1 -1 1 1 Example: Coding • {-1, 0, 1} • 2 bits * 12 samples = 24 bits (compared with 60 bits before compression) We use the model to predict the value, then encode the residual!

  20. Another Example • Morse Code (1838) Shorter codes are assigned to letters that occur more frequently!

  21. A Brief Introduction to Information Theory

  22. b determines the unit of information Information Theory (1) • A quantitative (量化的) measure of information • You will win the lottery tomorrow. • The sun will rise in the east tomorrow. • Self-information [Shannon 1948] P(A): the probability that the event A will happen The amount of surprise or uncertainty in the message

  23. Information Theory (2) • Example: flipping a coin • If the coin is fair P(H) = P(T) = ½ i(H) = i(T) = -log2(½) = 1 bit • If the coin is not fair P(H) = 1/8, P(T)=7/8 i(H) = 3 bits, i(T) = 0.193 bits The occurrence of a HEAD conveys more information!

  24. Information Theory (3) • For a set of independent events Ai • Entropy (the average self-information) • The coin example • Fair coin (1/2, 1/2): H=P(H)i(H) + P(T)i(T) = 1 • Unfair coin (1/8, 7/8): H=0.544

  25. Information Theory (4) • Entropy • The best a lossless compression scheme can do • Not possible to know for a physical source • Estimate (guess)! • Depends on our assumptions about the structure of the data

  26. Estimation of Entropy (1) • 1 2 3 2 3 4 5 4 5 6 7 8 9 8 9 10 • Assume the sequence is i.i.d. • P(1)=P(6)=P(7)=P(10)=1/16 P(2) =P(3)=P(4)=P(5)=P(8)=P(9)=2/16 • H = 3.25 bits • Assume sample-to-sample correlation exists • Model: xn = xn-1 + rn • Residuals: 1 1 1 -1 1 1 1 -1 1 1 1 1 1 -1 1 1 • P(1)=13/16, P(-1)=3/16 • H = 0.7 bits

  27. Estimation of the Entropy (2) • 1 2 1 2 3 3 3 3 1 2 3 3 3 3 1 2 3 3 1 2 • One symbol at a time • P(1) = P(2) = ¼, P(3) = ½ • H = 1.5 bits/symbol • 30 (1.5*20) bits are required in total • In blocks of two • P(1 2) = ½, P(3 3)=½ • H = 1 bit/block • 10 (1*10) bits are required in total

  28. Coding

  29. letter codeword _ alphabet code Coding (1) • The assignment of binary sequences to elements of an alphabet • Rate of the code: average number of bits per symbol • Fixed-length code and variable-length code

  30. Decodable With one-symbol delay Ambiguous Not uniquely decodable Prefix code instantaneous

  31. Coding (3) • Example of not uniquely decodable code Letters Code a1 0 a2 1 a3 00 a4 11 100 a2 a3 => 100 a2 a1 a1 => back

  32. a2 Coding (4) • Not instantaneous, but uniquely decodable code Oops! a2 a3 a3a3a3a3a3a3a3

  33. Prefix Codes • No codeword is a prefix to another codeword • Uniquely decodable

  34. Huffman Coding • Basic algorithm • Extended form • Adaptive coding

  35. Huffman Coding Huffman procedure: Two least frequent symbols differ only in the last bit. Ex: m0, m1 • Observations of prefix codes • Frequent symbols have shorter codewords • The two symbols that occur leastfrequently have the same length

  36. Algorithm 0 (1) a’1(0.6) 0 a’3(0.4) 1 1 1 0 a’4(0.2) 0 1 a4(0.1) a5(0.1) a3(0.2) a1(0.2) a2 (0.4) 1 01 0011 0010 000 A = {a1, a2, a3, a4, a5} P(a1) = 0.2, P(a2) = 0.4, P(a3) = 0.2, P(a4) = 0.1, P(a5) = 0.1

  37. Algorithm (1) 0 a’1(0.6) (1) 0 0 1 a’2(0.6) a’3(0.4) 1 1 1 1 0 0 a’1(0.4) a’4(0.2) a’4(0.2) 1 1 0 0 0 1 a2(0.4) a1(0.2) a4(0.1) a3(0.2) a5(0.1) a2(0.4) 00 10 010 011 11 a5(0.1) a4(0.1) a1(0.2) a3(0.2) Which code is preferred? The one with minimum variance! 1 01 0011 0010 000 Entropy: 2.122 bits/symbol Average length: 2.2 bits/symbol

  38. Exercise e (30.5) h (13.4) o (16.1) p (5.0) w (2.4) l (9.8) t (22.8)

  39. Length of Huffman Codes entropy of the source average code length • A source S with A = {a1,…,ak} and {P(a1),…,P(ak)} • Average codeword length: • Lower and upper bounds

  40. Extended Huffman Codes (1) 1 bit / letter No compression! 0.645 bit / letter • Consider small alphabet and skewed probabilities • Example: • Block multiple symbols together

  41. Extended Huffman Codes (2) H = 0.816 bits/symbol = 1.2 bits/symbol H = 1.6315 bits/block = 0.816 bits/symbol = 1.7228 / 2 = 0.8614 bits/symbol Another example:

  42. Adaptive Huffman Coding (1) Maintain a Huffman Tree! • No initial knowledge of source distribution • One-pass procedure • Based on statistics of encountered symbols

  43. Adaptive Huffman Coding (2) • Huffman tree • Node (id, weight) • Sibling property • w(parent) = sum of w(children) • ids are ordered with non-decreasing weights weight: # of occurrence Id: 1 2 3 4 5 6 7 8 9 10 11 w: 2 3 5 5 5 6 10 11 11 21 32 Non-decreasing!

  44. Adaptive Huffman Coding (3) • NYT (not yet transmitted) code • w(NYT) = 0 • Transmitted when seeing a new letter • Smallest id in the tree • Uncompressed code (ex: m letters)

  45. Adaptive Huffman Coding: Encode (1) r 10001 a 00000 a 1 NYT 0 Output: 00000 1 010001 Input: [a a r d v a r k] (Alphabet: 26 lowercase letters) Initial tree

  46. Adaptive Huffman Coding: Encode (2) v 1011 NYT 00 d 00011 NYT 000 5 3 2 1 Output: 0000010100010000011 001011 Input: [a a r d v a r k] (Alphabet: 26 lowercase letters)

  47. Adaptive Huffman Coding: Encode (3) Tree update 5 3 2 5 3 Swap 47 and 48 2 1

  48. Adaptive Huffman Coding: Encode (4) Tree update 5 3 2 50 49 48 47 45 46 Swap 49 and 50 43 44

More Related