1 / 59

Multimedia Compression

Multimedia Compression. Audio, image and video require vast amounts of data 320x240x8bits grayscale image: 77Kb 1100x900x24bits color image: 3MB 640x480x24x30frames/sec: 27.6 MB/sec Low network’s bandwidth doesn't allow for real time video transmission

eron
Download Presentation

Multimedia Compression

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimedia Compression • Audio, image and video require vast amounts of data • 320x240x8bits grayscale image: 77Kb • 1100x900x24bits color image: 3MB • 640x480x24x30frames/sec: 27.6 MB/sec • Low network’s bandwidth doesn't allow for real time video transmission • Slow storage or processing devices don't allow for fast playing back • Compression reduces storage requirements Multimedia Compression

  2. Classification of Techniques • Lossless: recover the original representation • Lossy: recover a representation similar to the original one • high compression ratios • more practical use • Hybrid: JPEG, MPEG, px64 combine several approaches Multimedia Compression

  3. Compression Standards Furht at.al. 96 Multimedia Compression

  4. Lossless Techniques Furht at.al. 96 Multimedia Compression

  5. Lossy Techniques Furht at.al. 96 Multimedia Compression

  6. JPEG Modes of Operation • Sequential DCT: the image is encoded in one left-to-right, top-to-bottom scan • Progressive DCT: the image is encoded in multiple scans (if the transmission time is long, a rough decoded image can be reproduced) • Hierarchical: encoding at multiple resolutions • Lossless : exact reproduction Multimedia Compression

  7. JPEG Block Diagrams Furht at.al. 96 Multimedia Compression

  8. JPEG Encoder • Three main blocks: • Forward Discrete Cosine Transform (FDCT) • Quantizer • Entropy Encoder • Essentially the sequential JPEG encoder • Main component of progressive, lossless and hierarchical encoders • For gray level and color images Multimedia Compression

  9. Sequential JPEG • Pixels in [0,2p-1] are shifted in [-2p-1,2p-1-1] • The image is divided in 8x8 blocks • Each 8x8 block is DCT transformed Multimedia Compression

  10. DCT Coefficients • F(0,0) is the DC coefficient: average value over the 64 samples • The remaining 63 coefficients are the AC coefficients • Pixels in [-128,127]: DCTs in [-1024,1023] • Most frequencies have 0 or near to 0 values and need not to be encoded • This fact achieves compression Multimedia Compression

  11. Quantization Step • All 64 DCT coefficients are quantized • Fq(u,v) = Round[F(u,v)/Q(u,v)] • Reduces the amplitude of coefficients which contribute little or nothing to 0 • Discards information which is not visually significant • Quantization coefficients Q(u,v) are specified by quantization tables • A set of 4 tables are specified by JPEG Multimedia Compression

  12. Quantization Tables Furht at.al. 96 • for (i=0; i < 64; i++) for (j=0; j < 64; j++) Q[i,j] = 1 + [ (1+i+j) quality]; • quality = 1: best quality, lowest compression • quality = 25: poor quality, highest compression Multimedia Compression

  13. AC Coefficients Furht at.al. 96 • The 63 AC coefficients are ordered by a “zig-zag” sequence • Places low frequencies before high frequencies • Low frequencies are likely to be 0 • Sequences of such 0 coefficients will be encoded by fewer bits Multimedia Compression

  14. DC Coefficients • Predictive coding of DC Coefficients • Adjacent blocks have similar DC intensities • Coding differences yields high compression Multimedia Compression

  15. Entropy Encoding • Encodes sequences of quantized DCT coefficients into binary sequences • AC: (runlength, size) (amplitude) • DC:(size, amplitude) • runlength: number consecutive 0’s, up to 15 • takes up to 4 bits for coding • (39,4)(12) = (15,0)(15,0)(7,4)(12) • amplitude: first non-zero value • size: number of bits to encode amplitude • 0 0 0 0 0 0 476: (6,9)(476) Multimedia Compression

  16. Huffman coding • Converts each sequence into binary • First DC following with ACs • Huffman tables are specified in JPEG • Each (runlength, size) is encoded using Huffman coding • Each (amplitude) is encoded using a variable length integer code • (1,4)(12) => (11111101101100) MultimediaCompression

  17. Example of Huffman table Furht at.al. 96 Multimedia Compression

  18. JPEG Encoding of a 8x8 block Furht at.al. 96 Multimedia Compression

  19. Compression Measures • Compression ratio (CR): increases with higher compression • CR = OriginalSize/CompressedSize • Root Mean Square Error (RMS): better quality with lower RMS • Xi: original pixel values • xi: restored pixel values • n: total number of pixels Multimedia Compression

  20. Furht at.al. 96 Multimedia Compression

  21. JPEG Decoder • The same steps in reverse order • The binary sequences are converted to symbol sequences using the Huffman tables • F’(u,v) = Fq(u,v)Q(u,v) • Inverse DCT Multimedia Compression

  22. Progressive JPEG • When image encoding or transmission takes long there may be a need to produce an approximation of the original image which is improved gradually Furht at.al. 96 Multimedia Compression

  23. Progressive Spectral Selection • The DCT coefficients are grouped into several bands • Low-frequency bands are first • band1: DC coefficient only • band2: AC1,AC2 coefficients • band3: AC3, AC4, AC5, AC6 coefficients • band4: AC7, AC8 coefficients Multimedia Compression

  24. Lossless JPEG • Simple predictive encoding Furht at.al. 96 prediction schemes Multimedia Compression

  25. Hierarchical JPEG • Produces a set of images at multiple resolutions • Begins with small images and continues with larger images (down-sampling) • The reduced image is scaled-up to the next resolution and used as predictor for the higher resolution image Multimedia Compression

  26. Encoding • Down-sample the image by 2a in each x, y • Encode the reduced size image (sequential, progressive ..) • Up-sample the reduced image by 2 • Interpolate by 2 in x, y • Use the up-sampled image as predictor • Encode differences (predictive coding) • Go to step 1 until the full resolution is encoded Multimedia Compression

  27. Furht at.al. 96 Multimedia Compression

  28. JPEG for Color images • Encoding of 3 bands (RGB, HSV etc.) in two ways: • Non-interleaved data ordering: encodes each band separately • Interleaved data ordering: different bands are combined into Minimum Coded Units (MCUs) • Display, print or transmit images in parallel with decompression Multimedia Compression

  29. Interleaved JPEG • Minimum Coded Unit (MCU): the smallest group of interleaved data blocks (8x8) Furht at.al. 96 Multimedia Compression

  30. Video Compression • Various video encoding standards: QuickTime, DVI, H.261, MPEG etc • Basic idea: compute motion between adjacent frames and transmit only differences • Motion is computed between blocks • Effective encoding of camera and object motion Multimedia Compression

  31. MPEG • The Moving Picture Coding Experts Group (MPEG) is a working group for the development of standards for compression, decompression, processing, and coded representation of moving pictures and audio • MPEG groups are open and have attracted large participation • http://mpeg.telecomitalialab.com Multimedia Compression

  32. MPEG Features • Random access • Fast forward / reverse searches • Reverse playback • Audio – visual synchronization • Robustness to errors • Auditability • Cost trade-off Multimedia Compression

  33. MPEG -1, 2 • At least 4 MPEG standards finished or under construction • MPEG-1: storage and retrieval of moving pictures and audio on storage media • 352x288 pixels/frame, 25 fps, at 1.5 Mbps • Real-time encoding even on an old PC • MPEG-2:higher quality, same principles • 720x576 pixels/frame, 2-80 Mbps Multimedia Compression

  34. MPEG-4 • Encodes video content as objects • Based on identifying, tracking and encoding object layers which are rendered on top of each other • Enables objects to be manipulated individually or collectively on an audiovisual scene (interactive video) • Only a few implementations • Higher compression ratios Multimedia Compression

  35. MPEG-7 • Standard for the description of multimedia content • XML Schema for content description • Does not standardize extraction of descriptions • MPEG1, 2, and 4 make content available • MPEG7 makes content semantics available Multimedia Compression

  36. MPEG-1,2 Compression • Compression of full motion video, interframe compression, stores differences between frames • A stream contains I, P and B frames in a given pattern • Equivalent blocks are compared and motion vectors are computed and stored as P and B frames Furht at.al. 96 Multimedia Compression

  37. Frame Structures • I frames: self contained, JPEG encoded • Random access frames in MPEG streams • Low compression • P frames: predicted coding using with reference to previous I or P frame • Higher compression • B frames: bidirectional or interpolated coding using past and future I or P frame • Highest compression Multimedia Compression

  38. Example of MPEG Stream Furht at.al. 96 • B frames 2 3 4 are bi-directionally coded using I frame 1 and P frame 5 • P frame 5 must be decoded before B frames 2 3 4 • I frame 9 must be decoded before B frames 6 7 8 • Frame order for transmission: 1 5 2 3 4 9 6 7 8 Multimedia Compression

  39. MPEG Coding Sequences • The MPEG application determines a sequenceof I, P, B frames • For fast random access code the whole video as I frames (MJPEG) • High compression is achieved by using large number of B frames • Good sequence: (IBBPBBPBB)(IBBPBBPBB)... Multimedia Compression

  40. Motion Estimation • The motion estimator finds the best matching block in P, B frames • Block: 8x8 or16x16 pixels • P frames use only forward prediction: a block in the current frame is predicted from past frame • B frames use forward or backward or prediction by interpolation: average of forward, backward predicted blocks Multimedia Compression

  41. Motion Vectors block: 16x16pixles Furht at.al. 96 • One or two motion vectors per block • One vector for forward predicted P or B frames or backward predicted B frames • Two vectors for interpolated B frames Multimedia Compression

  42. MPEG Encoding • I frames are JPEG compressed • P, B frames are encoded in terms of future or previous frames • Motion vectors are estimated and differences between predicted and actual blocks are computed • These error terms are DCT encoded • Entropy encoding produces a compact binary code • Special cases: static and intracoded blocks Multimedia Compression

  43. MPEG encoder JPEG encoding Furht at.al. 96 Multimedia Compression

  44. MPEG Decoder Furht at.al. 96 Multimedia Compression

  45. Motion Estimation Techniques • Not specified by MPEG • Block matching techniques • Estimate the motion of an nxm block in present frame in relation to pixels in previous or future frames • The block is compared with a previous or forward block within a search area of size (m+2p)x(n+2p) • m = n = 16 • p = 6 Multimedia Compression

  46. Block Matching Furht at.al. 96 • Search area in block matching techniques • Typical case: n=m=16, p=6 • F: block in current frame • G: search area in previous (or future) frame Multimedia Compression

  47. Cost functions • The block has moved to the position that minimizes a cost function • Mean Absolute Difference (MAD) • F(i,j) : a block in current frame • G(i,j) : the same block in previous or future frame • (dx,dy) : vector for the search location • dx=(-p,p), dy=(-p,p) Multimedia Compression

  48. More Cost Functions • Mean Squared Difference (MSD) • Cross-Correlation Difference (CCF) Multimedia Compression

  49. More cost Functions • Pixel Difference Classification (PDC) • t: predefined threshold • each pixel is classified as a matching pixel (T=1) or a mismatching pixel (T=0) • the matching block maximizes PDC Multimedia Compression

  50. Block Matching Techniques • Exhaustive: very slow butaccurate • Approximation: faster but less accurate • Three-step search • 2-D logarithmic search • Conjugate direction search • Parallel hierarchical 1-D search (not discussed) Pixel difference classification (not discussed here) Multimedia Compression

More Related