Overview of Fine Granularity Scalability in MPEG-4 Video Standard

1. Overview of Fine Granularity Scalability in MPEG-4 Video Standard Weiping Li Presented by : Brian Eriksson

2. Introduction - Problem • Tradition System • Video encoded close to channel capacity • Decoded using all of the bits • Internet Video System • Channel capacity changes • Partially decodable within a bit range

3. Received Quality = Dialup Rate Good = Broadband Rate Bad = Layered Coding Low High = Desired Coding Channel Bitrate Introduction – Video Coding Performance = Distortion-Rate Curve

4. Received Quality Good Bad = Layered Coding Low High Channel Bitrate Previous Layered Coding Methods

5. MPEG-2 Overview • I-Frame – Intracoded Picture • Contains all frame information • P-Frame – Predictive-Coded Picture • Uses previous frame and motion information to reconstruct frame • B-Frame – Bidirectional-Coded Picture • Uses previous frame, next frame, and motion information to reconstruct the frame

6. Temporal Scalability • Video is encoded into two layers with equal resolution but using different prediction. • Base layer has lower frame rate Base Enh. Enh. Base Enh. Enh. Base Enh. Enh. With Enhancement Layer : Frame Rate = 30 fps Base X X Base X X Base X X Without Enhancement Layer : Frame Rate = 10 fps

7. MPEG-2 SNR Scalability Enhancement Layer Stream Video Output IDCT + + Base Layer Stream Motion Compensation • Motion prediction includes enhancement information • Efficiency dependent on two factors • Encoder uses enhancement layer (drift can occur) • Decoder receives enhancement layer

8. + IDCT Enhancement Bitstream Enhancement Layer Video Upsampler Base Layer Video Base Bitstream (Motion Compensated) Spatial Scalability • Same frame rate, different resolutions • Base frame = 128x128, enhancement = 256x256 • Enhancement Layer not in prediction loop

9. Received Quality Good Bad Low High Channel Bitrate MPEG-4 Fine Granularity Scalability Technique = Distortion-Rate Curve = Desired Coding

10. Bitplane Coding Technique • Given vector X: • X = [-12, -53, 62, -7, 31,…,180,-43,…,5] • Convert to signed magnitude • Separate into sign bit and absolute values

11. Sign(X) = [0, 0, 1, 0, 1,…,1,0,…,1] • Abs(X) = [12, 53, 62, 7,31,…,180,43,…,5] • To find the number of bitplanes needed, find the maximum value of abs(X) • Max(abs(X)) = 180 = 10110100b • 8 bitplanes • Sign bit plane • 1 bitplane Bitplane Coding Technique

12. Sign(X) = [0, 0, 1, 0, 1,…,1,0,…,1] X = [-12, -53, 62, -7, 31,…,180,-43,…,5] c 0,0,1,0,1,…,1,0,…,1 0,0,0,0,0,…,1,0,…,0 0,0,0,0,0,…,0,0,…,0 0,1,1,0,0,…,1,1,…,0 0,1,1,0,1,…,1,0,…,0 1,0,1,0,1,…,0,1,…,0 1,1,1,1,1,…,1,0,…,1 0,0,1,1,1,…,0,1,…,0 0,1,0,1,1,…,0,1,…,1 Bitplane Coding Technique

13. Run Level Encoding • Symbol : (RUN, EOP) • RUN = Number of Consecutive Zeros before a 1 • EOP = 0 if there are more ones • EOP = 1 if the rest of the line are zeros. • Example: • {1,0,1,0,0,0,1,0,0,0,…} • = (0,0),(1,0),(3,1) • {0,0,0,1,0,0,0,…} • = (3,1)

14. MPEG-4 Bitplane FGS Technique • Base layer reaches lower bound of bit-range • Divide image into 8x8 DCT blocks • Divide blocks into Y,U,V color components • Use bitplane run-level coding to encode/decode

15. Original Signal DCT Bitplane Encoding + - Enhancement Layer Signal Base Layer Signal (Motion Compensated) Fine Granularity Scalability Encoder

16. Fine Granularity Scalability Decoder Bitplane Decoding Enhancement Layer Video IDCT + Base Layer Video (Motion Compensated)

17. c c c c Number of Truncated Layers = 2 Received X = [-12, -52, 60, -4,28,…,180,-40,…,-4] Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5] Number of Truncated Layers = 3 Received X = [-8, -48, 56, -0,24,…,176,-40,…,-0] Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5] Number of Truncated Layers = 0 Received X = [-12, -53, 62, -7,31,…,180,-43,…,-5] Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5] Number of Truncated Layers = 1 Received X = [-12, -52, 62, -6,30,…,180,-42,…,-4] Original X = [-12, -53, 62, -7,31,…,180,-43,…,-5] 0,0,1,0,1,…,1,0,…,1 0,0,0,0,0,…,1,0,…,0 0,0,0,0,0,…,1,0,…,0 0,0,1,0,1,…,1,0,…,1 0,0,0,0,0,…,1,0,…,0 0,0,1,0,1,…,1,0,…,1 0,0,1,0,1,…,1,0,…,1 0,0,0,0,0,…,1,0,…,0 0,0,0,0,0,…,0,0,…,0 0,0,0,0,0,…,0,0,…,0 0,0,0,0,0,…,0,0,…,0 0,0,0,0,0,…,0,0,…,0 0,1,1,0,0,…,1,1,…,0 0,1,1,0,0,…,1,1,…,0 0,1,1,0,0,…,1,1,…,0 0,1,1,0,0,…,1,1,…,0 0,1,1,0,1,…,1,0,…,0 0,1,1,0,1,…,1,0,…,0 0,1,1,0,1,…,1,0,…,0 0,1,1,0,1,…,1,0,…,0 1,0,1,0,1,…,0,1,…,0 1,0,1,0,1,…,0,1,…,0 1,0,1,0,1,…,0,1,…,0 1,0,1,0,1,…,0,1,…,0 1,1,1,1,1,…,1,0,…,1 0,0,0,0,0,…,0,0,…,0 1,1,1,1,1,…,1,0,…,1 1,1,1,1,1,…,1,0,…,1 0,0,0,0,0,…,0,0,…,0 0,0,0,0,0,…,0,0,…,0 0,0,1,1,1,…,0,1,…,0 0,0,1,1,1,…,0,1,…,0 0,0,0,0,0,…,0,0,…,0 0,0,0,0,0,…,0,0,…,0 0,0,0,0,0,…,0,0,…,0 0,1,0,1,1,…,0,1,…,1 Truncated Bitplane

18. Advanced Bitplane Techniques

19. Frequency Weighted Bitplane Normal Bitplane DCT Index DCT Index Frequency Weighting in Bitplane

20. Selectively Enhanced Bitplane • Detect visually significant area • Shift upward in bitplane • More likely to be included in truncated bitstream

21. Other Methods • Error Resilience in Bitplane • Random burst errors in the bitstream • Resynchronization markers are used to resynch. • Temporal Scalability in Bitplane • Uses FGS to encode/decode the entire temporal enhancement frame

22. Simulation - DCT

23. Simulation - DWT

24. Selectively Enhanced / Weighted Bitplane Setup

25. 1 Layer Removed 2 Layers Removed 4 Layers Removed 5 Layers Removed 6 Layers Removed 3 Layers Removed Simulation – Enhanced Weighted Bitplane

26. Simulation – Li

27. Conclusions • FGS Bitplane method allows for quality parallel to distortion-rate curve • A wavelet-based approach may yield better results • Fairly simple implementation

28. Questions?