A New Content-Based Hybrid Video Transcoding Method

1. A New Content-Based Hybrid Video Transcoding Method YongQing Liang YapPeng Tan Presented by Robert Hung

2. Introduction • Video Content Descriptors • Proposed Selection Method • Experimental Results • Summary

3. Introduction Aim: Bitrate reduction of a compressed video. Construct a system consists of three common reduction techniques in transform coding • Requantization (RQ) • Spatial resolution downsampling (SD) • Temporal resolution downsampling (TD) Problem: Integrates the three techniques and selected accordingly • Strategy on the selection of the techniques (No mention on why needs three techniques, Not addressing the issue on the use of two or three techniques together at the same frame, a hole needed to be filled, maybe, indeed, no need to consider, but why?)

4. Introduction • Proposed Solution: • Define two descriptors of the video contents as the input parameter of the selection method. • a) Motion activity descriptor (MA) • b) Spatial activity descriptor (SA) • The input parameters to the selection method including • two video content descriptors, (MA, SA) • target bitrate (TB) • the original frame rate (FR) • New Selection Method based on some heuristic rules

5. Introduction • The system is illustrated below TB MA Processed video Decoded video SELECTOR SA FR TD RQ SD

6. Video Content Descriptors Motion Activity Descriptor - MA = average magnitude of the motion vector of a frame - intracoded block has the predefined maximum motion vector (Not a motion compensation block) - notcoded block has zero motion vector (Block has the same pattern as the previous one at the same location) -AMA = average motion activity over several consecutive frames -200 p-frames statistics shown in the next slide, the correlation coefficient is 0.92. (only one set of data)

7. MA against bits of the frame MA against the bits of the frame

8. Video Content Descriptors Spatial Activity Descriptor -SA = Mean Quantization of the frame - The rationale is that “a video frame contains a lot of spatial details, more bits are required to code the frame with fixed quantization scales. If the bitrate is fixed, larger quantization scales will be used to code the frame.” (The measure is used as reference for the next frame)

9. Proposed selection Method • Two main categories of frame • Low frame rate • High frame rate • Some reasoning behind the selection method:

10. Proposed selection Method Reasoning: High spatial activity High motion activity In the consecutive frame Rule 1. If RQ can achieve the requirement, do RQ Rule 2. If Low motion activity, do RQ Rule 3. If motion activity is high, SD is applied. Otherwise RQ since previously used SD. Rule 4. If Both spatial and motion activity are high, SD is applied Low Frame rate is the average motion activity of the several consecutive frames is the average quanatization scale of the previous frame

11. Proposed selection Method the original quantization scale of the current frame Target bit rate High Frame Rate The actual bitrate of the previous frame

12. Proposed selection Method Rule 1: Previous selection is on RQ > => TD selected is small Reasons: low motion activity spatial detail can be retained > 30 => SD selected > is high Reasons: SD is selected instead of TD because of the high motion activity ,

13. Proposed selection Method Rule 2: Previous selection is on SD > => RQ selected is small Reasons: low motion activity spatial detail can be retained

14. Proposed selection Method Rule 3: Previous selection is on TD < => RQ selected Reasons: A little adjustment can achieve the target bitrate > 30 is high => SD selected > Reasons: SD is selected instead of TD because of the high motion activity

15. Experimental Results HVT Transcode the video of “volley ball” from 636Kbits/s to 140Kbits/s, HVT is 1dB higher in average.

16. Experimental Results Very low PSNR for Requantization Transcode the video of “skating” from 112Kbits/s to 50Kbits/s, HVT is 0.5 and 6.0 dB higher in average than RQ and SD respectively .

17. Summary • A method on selecting the bitrate reduction techniques • Simple rules • Simple measurements • Good result • Motion activity and Spatial activity descriptor are defined to characterize the video contents

18. Comment • Rules should be expressed more precisely. (If the authors could lay down the Pro and Con of each techniques, and derive the conditions for employing them.) • Good results, but the implementation is difficult to follow, no mention of which implementation of RQ,SD and TD. • No mention on how it is switching from one techniques to another. E.g. how to handle the change in resolution between frames. • It is the current state of art in hybrid video transcoder