Direction-Adaptive Partitioned Block Transform for Color Image Coding

1. Mina Makar & Sam Tsai Stanford University EE 398 Final Project Direction-Adaptive Partitioned Block Transform for Color Image Coding

2. Overview • Introduction • DA-PBT on color images • Choosing color space & sub-sampling chroma • Color quality along directions, comparing with DCT, DCT-8 • Rate allocation on color components • Extensions for encoder improvement • Quantization matrix or not? • Joint Cb/Cr mode decision, faster mode selection • Effect of post-filtering • Framework for DA-PBT on color images • Conclusions

3. Color Separation Spatial Filtering XYZ Representation CIELAB Calculation Introduction • What is DA-PBT? • Apply DCT on partitioned blocks along different directions • Visual Quality of Color Images • CIELAB ΔE metric • Spatial extension to CIELAB: Spatial blurring in human vision system • Higher value  Higher distortion [C. -L. Chang & B. Girod ’08]

4. Choosing Color Space • YCbCr vs. RGB • YCbCr  Better visual quality for same rate • No Sub-sampling for Cb & Cr Components • Gives better flexibility in rate allocation • Sub-sampling  No improvement for reasonable bit rates

5. Color Performance with Directions • Experiment Setup • Better visual quality in all directions • DA-PBT significantly outperforms DCT-8 for directions near its supported directions

6. DA-PBT 0° 25° 45° 90° 65° 135°

7. DCT-8 0° 25° 45° 90° 65° 135°

8. DA–PBT vs. Other Transforms • Image set of 5 images • DCT-8 • DCT (adaptive block size) • DA-PBT • DA-PBT (spatial prediction) • Results for ‘Lena’ (Equal Q)

9. Rate Allocation • Rate allocation across color spaces for optimal visual quality • Experiment: Different Y:Cb:Cr ratios  Lowest ΔE • Using DA-PBT • Image size: 256x256 • Q step size range: 4~100 • Rates: 1~0.5 bpp • Best results around 4:1:1 • Larger Q for Y-component

10. Quantization Matrices ? • Higher Q for higher frequencies • No frequency localization in DA-PBT • Experiment: • Encode Y in DA-PBT • Encode Cb/Cr in DCT (adaptive block size)‏ • Quantization matrices  Extended from JPEG default matrix • No improvement in ΔE • Encoding all components in DA-PBT with fixed Q for the whole color component is better

11. Joint Cb/Cr Mode Decision • Apply same direction and block size to both Cb/Cr • Mode decision: J = (DCb + DCr) + λ*(RCb + RCr + Ro) • One overhead (Ro) for Cb/Cr

12. Faster Mode Selection • Luma optimal block sizes used as indication when deciding Chroma modes • Luma 16x16  Chroma 16x16 • Luma 16x8 or 8x16  Chroma no smaller than 16x8 or 8x16 • Luma 8x8  Chroma no smaller than 8x8 • Less than 3% sub-optimal blocks compared to optimal mode decisions • Experiment: 463 sub-optimal / 21760 total

13. Effect of Post-filtering • Low rates  Blocking artifacts • Apply ‘Unblock’ software on low bit rate images ΔE [J. Costella ’06]

14. Encoder Y Cb Cr Transform Decoder Rate Allocation QY Y DA-PBT R G B Transform Modes QC Post-filtering Bit-stream Bit-stream [Cb, Cr] or [Joint Cb/Cr] Proposed Image Coding Framework

15. Joint encoding for min ΔE QY = 73, QCb,Cr = 38 RY = 0.301, RCb,Cr = 0.199 Faster mode selection 7 sub-optimal / 256 total Framework Results min ΔE (0.5 bpp) Equal Q (0.5 bpp) Original

16. Conclusions • “DA-PBT on Color Images” summary • Working in YCbCr color space with no Sub-sampling • Less color degradation along edges • Outperforms DCT & DCT-8 • 4:1:1 ratio  Choosing larger Q for Y-component • No quantization matrix. Fixed Q for each color component • Proposed Image Coding Framework • Rate allocation to minimize ΔE • Joint encoding for Cb/Cr to reduce overhead • Use Y mode decisions as indication for Cb/Cr modes • Deblocking filter to reduce blocking artifacts (Low bit rate)‏