1 / 21

Current Work on Video Coding

Current Work on Video Coding. Raj Kumar. Overview. Fine-Grained Scalability (FGS) What is it and why do we need it What are its drawbacks Our new scheme (FGS+) B-Frame only scheme All-Frame Scheme Future Directions. Enhancement Layer. I. B. B. P. B. Base Layer.

verna
Download Presentation

Current Work on Video Coding

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Current Work on Video Coding Raj Kumar

  2. Overview • Fine-Grained Scalability (FGS) • What is it and why do we need it • What are its drawbacks • Our new scheme (FGS+) • B-Frame only scheme • All-Frame Scheme • Future Directions

  3. Enhancement Layer I B B P B Base Layer Fine Grained Scalability • What is it • What are the pros and cons

  4. I B B P B Enhancement Layer I B B P B Base Layer Traditional Vs Fine-grained Encoding

  5. Temporal Enhancement Layer Spatial Enhancement Layer I P Base Layer Temporal FGS

  6. Problem with Temporal FGS • Reference Frames are poor quality • Doesn’t use motion-prediction well • Compression ratio suffers compared to traditional encoders (approx 2-2.5 DB)

  7. Insight • FGS provides two degrees of freedom • SNR • Temporal • Is there an optimal path that will maximize overall quality. • How do we determine it • How do we take advantage of it

  8. Idea • Can we use a part of the enhancement layer, in addition to the base layer, to predict motion? • We can if we know how much of the base layer will be present at the decoder when the temporal layers are introduced • So if we build in rules about when temporal layers are introduced relative to spatial quality, we can!

  9. Implementation • Increase SNR quality to a predetermined point • Then improve temporal quality • Further improve SNR quality at the new frame-rate • And so on….

  10. Temporal Enhancement Layer Spatial Enhancement Layer I P Base Layer FGS+ Scheme (B-frame only)

  11. Two issues • When (at what SNR quality) do we introduce new temporal-frames? • How much extended-reference do we use

  12. Introducing Temporal Frames • We conducted a study where users chose preferred frame-rate at different bit-rates: • As the bit-rate goes up, people prefer better frame-rates • High-motion videos (Stefan, Coastguard) require a quicker transition

  13. Determining the size of the reference • Plot SNR performance as a function of enhancement at various bit-rates • Choose enhancement corresponding to best quality

  14. Performance • Improvement over FGS varies from 0.19 dB to 1.28 dB • At low bit-rates simple videos benefit (Coastguard) • At high bit-rates complex videos benefit (Mobile)

  15. FGS+ for All-Frames • We can improve performance further by using enhancement for base-layer P-frames also • But P-frames are present at all bit-rates. How much enhancement do we use?

  16. FGS+ Scheme (All-frames) Temporal Enhancement Layer Spatial Enhancement Layer I P Base Layer

  17. Improvement in Performance • At 3 bit-planes • Performance varies from –0.03 dB to 0.59 dB • Performance will degrade at low bit-rates, due to incomplete references

  18. Conclusion • FGS+ uses a spatio-temporal notion of video-quality • Improves performance of traditional FGS from 0.16 to 1.61 dB

  19. Current Issues • Spatio-Temporal preference curves vary for different videos • How do we choose the correct curve automatically • Use Motion-Vectors, Image-Complexity

  20. Future Directions • Factors other than video-characteristics do affect spatio-temporal preferences • Environment (resolution, display-type, lighting) • User preferences (moods, some prefer spatial-quality always) • How do we account for these other dimensions ? • Large dimensional space requires many (thousands) of sample points

  21. Future Directions • Build online experiment • Interactive web-based tool that lets user watch many videos • Allow user to change spatio-temporal settings • Record data corresponding to changes (video-characteristics, display resolution and type, user information)

More Related