1 / 33

Perception-motivated High Dynamic Range Video Encoding

INFORMATIK. Perception-motivated High Dynamic Range Video Encoding. Rafal Mantiuk, Grzegorz Krawczyk, Karol Myszkowski, Hans-Peter Seidel. High Dynamic Range. LDR Video Intended for existing displays Relative pixel brightness. HDR Video Intended for the human eye

senalda
Download Presentation

Perception-motivated High Dynamic Range Video Encoding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. INFORMATIK Perception-motivated High Dynamic Range Video Encoding Rafal Mantiuk, Grzegorz Krawczyk,Karol Myszkowski, Hans-Peter Seidel

  2. High Dynamic Range

  3. LDR Video Intended for existing displays Relative pixel brightness HDR Video Intended for the human eye Photometric or radiometric units [cd/m2, Watt/m2sr] High vs Low Dynamic Range Video

  4. High Dynamic Range Video • Goal: Efficient encoding of full dynamic range of luminance perceived by the human observer 1st demo

  5. Overview • HDR Pipeline • HDR Video Encoding • Luminance Quantization • Edge Coding • Results • vs. MPEG-4 • vs. OpenEXR • Demo & Applications

  6. Related Work • HDR Pipeline Acquisition  Storage  Display

  7. Related Work • HDR Pipeline Acquisition  Storage Display • Global Illumination • HDR Cameras • HDRC (IMS Chips) • Lars III (Silicon Vision) • Autobrite (SMal Camera Technologies) • LM9628 (National) • Digital Pixel System (Pixim) • Technology overview [Nayar2003] HDRC – IMS Chips

  8. Related Work • HDR Pipeline Acquisition Storage  Display • Still images • Radiance – RGBE [Ward91] • OpenEXR [Bogart2003] • logLuv TIFF [Ward98] • HDR JPEG [Ward2004] • Video • No video format

  9. Related Work • HDR Pipeline Acquisition Storage Display • LDR Displays • But Tone Mapping necessary • HDR displays start to appear • University of British Columbia [Seetzen2004]

  10. HDR Encoding Framework • Detail level 1: Input & Output bitstream LDR Video encoder HDR White: MPEG Orange: HDR Encoder

  11. HDR Encoding Framework • Detail level 2: Color Transform LDR bitstream YCrCb Color Video Transform Encoder L u'v' HDR p White: MPEG Orange: HDR Encoder

  12. HDR Encoding Framework • Detail level 3: Edge Coding DCT Variable Coding length LDR bitstream Color Motion Tran. Comp. HDR Edge Run- Coding length White: MPEG Orange: HDR Encoder

  13. DCT Variable Coding length LDR RGB bitstream Color Motion Tran. Comp. HDR XYZ Edge Run- Coding length Encoding of Color

  14. Encoding of Color • How to represent color data? • Floating Points – ineffective compression • Integers – ok, but require quantization • How to quantize color data? • Quantization errors < threshold of perception • Use uniform color space (L*u*v*, L*a*b*) [Ward98] • Find minimum number of bits • Color(u*v*) – 8 bits are enough

  15. Encoding of Luminance • How to quantize luminance? • Gamma correction? • Logarithm? 8 6 log(Y)? 4 2 log Luminance Y 0 -2 -4 Integer representation

  16. Threshold Versus Intensity • Psychophysical measurements • The smallest perceivable difference Y for a certain adaptation level YA • tvi [Ferwerda96, CIE 12/2.1] Y log Threshold Y YA - Adaptation Luminance log Adaptation Luminance YA

  17. tvi ( Y ) f e max Luminance Quantization Just below threshold of perception Maximum quantization error log Luminance Y Integers Lp

  18. tvi ( Y ) f e max y d ( l ) - - = × × y 1 l L in L space 2 f tvi ( ( l )) P dl y - ( l ) L to Y mapping P - f threshold decrease Luminance Quantization Just below threshold of perception • Capacity function [Ashkihmin02] • Grayscale Standard Display Function [DICOM03] Maximum quantization error log Luminance Y 10 – 11 bits are enough Integers Lp

  19. Luminance QuantizationsComparison 2 cvi 11-bit percep. quant. 32-bit LogLuv 0 RGBE log Contrast Threshold -2 -4 -4 -2 0 2 4 6 8 log Adapting Luminance

  20. DCT Variable Coding length LDR RGB bitstream Color Motion Tran. Comp. HDR XYZ Edge Run- Coding length Edge Coding

  21. Edge Coding: Motivation • HDR video can contain sharp contrast edges • Light sources, shadows • DCT coding of sharp contrast may cause high frequency artifacts DCT coding Edge coding

  22. Edge Coding: Solution • Solution: Encode sharp edges in spatial domain, the rest in frequency domain Run-length encoding DCT encoding

  23. Edge Coding: Algorithm original I horizontal decomposition edge block horiz. edges II horizontal DCT III vertical decomposition edge block vert. edges IV vertical DCT

  24. Edge Coding: Algorithm original I horizontal decomposition edge block horiz. edges II horizontal DCT III vertical decomposition edge block vert. edges IV vertical DCT

  25. Edge Coding: Algorithm original I horizontal decomposition edge block horiz. edges II horizontal DCT III vertical decomposition edge block vert. edges IV vertical DCT

  26. Edge Coding: Algorithm original I horizontal decomposition edge block horiz. edges II horizontal DCT III vertical decomposition edge block vert. edges IV vertical DCT

  27. Edge Coding: Algorithm original I horizontal decomposition edge block horiz. edges II horizontal DCT III vertical decomposition edge block vert. edges IV vertical DCT

  28. Results • 2x size of tone-mapped MPEG-4 video • 20-30x saving compared to intra-frame compression (OpenEXR) Bit-stream Size

  29. Demo & Applications • Display dependent rendering • Choice of tone-mapping • Extended postprocessing

  30. Conclusions • HDR video compression • Modest changes to MPEG-4 • Lpu’v’ color space • Luminance quantization (10-11 bits) • Edge coding • Applications • On-the-fly tone mapping • Blooming, motion blur, night vision • Tuned for display • LDR / HDR Display

  31. Acknowledgments Comments and help • Volker Blanz • Scott Daly • Michael Goesele • Jeffrey Schoner • HDR Images and Sequences • Paul Debevec • SpheronVR • Jozef Zajac • Christian Fuchs • Patrick Reuter • HDR Camera • HDRC(R) VGAx courtesy of IMS CHIPSwww.hdrc.com

  32. Thank you http://www.mpi-sb.mpg.de/resources/hdrvideo/

More Related