Selected Papers from ICIP 2004

1. Selected Papers from ICIP 2004 Presented by Peter

2. Secure Media Streaming & Secure Adaptation for Non-scalable Video (Invited Paper) John G.Apostolopoulos Hewlett-Packard Labs, Palo Alto, CA

3. Summary • Targets • Adapting the media for the time-varying available network bandwidth for non-scalable video • Protect the security of the media • Media data is encrypted while R-D information is unencrypted • Each P-frame is encoded into one packet • Some P-frame is more important than others that give less distortion for when being dropped

4. Total Distortion Left: Carphone, Foreman Right: MrthrDhtr, Salesman

5. Secure Adaptive Streaming using a Secure-Media Rate-Distortion Hint Track (SM-RDHT) • SM-RDHT for untrusted streaming server • RD information embedded in hint track • Media itself is encrypted

6. SM-RDHT

7. Secure Transcoding at a Mid-Network Node using Secure Scalable Packets • H.264 video is packetized into secure scalable packets • Unencrypted packet headers provide R-D information – the importance of each packet • Mid-network transcoder can perform R-D optimized adaptation across multiple packets of a single stream or across packets of multiple different strems • 1 byte R-D information for each packet (one frame)

8. Secure Transcoding at a Mid-Network Node using Secure Scalable Packets

9. Comments • Packet size is limited to one frame that reduce the flexibility • Distortion highly depends on method of error concealment methods used at the decoder that is unknown at the streaming server and Mid-Network nodes

10. Discussions

11. Simple AVC-Based Codecs with Spatial Scalability R. Lange, Ł. Błaszak and M. Domański Poznań University of Tachnology, Poland

12. Summary • Spatial scalability for AVC-Based Codecs • Base layer is fully AVC-compliant • Prediction using the interpolation based layer and previous frame of enhancement layer • Improved motion vector encoding • Codec complexity is comparable to the complexity of a pair of codec used for simulcast coding of two layer • Proposed to the AVC standard

13. To Layers Structure

14. Improved Encoding of Motion Vectors in the Enhancement Layer • Optimum motion vectors are used in both layers • Previous proposals used motion vectors in the same layer to predict the current MV • Improved version includes up-scaled base layer MV to predict current MV • Prediction residuals are encoded using CABAC • Directional prediction is used as AVC standard for 8x16 blocks

16. Improved results from MV Predcition

17. Prediction in the Enhancement Layer • Two additional reference frames • Interpolated from decoded current base-layer frame • Average of latter and last temporal reference frame • Scalable coding is efficient if temporal prediction and base layer prediction are mixed with substantial prob. For each mode. • Edge-adaptive interpolation technique is used that improve the performance by 1dB

18. Experimental Results

19. Comments • The proposed scalable codec is not compliant with AVC • Only 2 layers are presented, Not Finegranularity scalable in spatial resolution • Coding efficiency improvement is small comparing to simulcast

20. Discussions

21. Mode Mapping Method for H.264/AVC Spatial Downscaling Transcoding P. Zhang, Y. Lu, Q. Huang and W. Gao Chinese Academy of Science, Microsoft Research Asia

22. Summary • Focused only on mode decision part • Cascaded Pixel-Domain transcoder (CPDT) • Mode mapping only for 16x16 predict mode for I frames and 8x8 prediction mode for P frames • Save about 50% time cost • High correlation between the four modes of original MB and the corresponding MB at half resolution • Two modes are proposed: • Simple Mode (SimMap) • Use motion vectors information (MapMV)

25. Simple Mode Mapping Method (SimMap) • For I frames • If more than 1 MB use I4x4 mode => use I4x4 in downsized frame • Otherwise use I16x16 • For P frames • If all MB are I16x16 => I16x16 • If more than 1MB are intra mode => I4x4 • If all are P16x16 or skip mode => P16x16, otherwise P8x8 is selected • 4 sub-modes are decided by direct mapping

27. Mode Mapping with Motion Vector (MapMV) • For the SimMap, the P16x8 and P8x16 are not utilized • When P8x8 is selected from SimMap: • Compute the distance between all MVs • If all distance < Th, => P16x16 • If D(MV1, MV3) and D(MV2,MV4) <Th, => P16x8 • If D(MV1, MV2) and D(MV3,MV4) <Th => P8x16 • Else P8x8

28. Experimental Results

29. Comments • PSNR loss more than 2dB at low bit rate

30. Discussions

31. A New Rate Control Scheme for H.264 Video Coding P. Yin and J. Boyce Corporate Research, Thomson Inc. Princeton

32. Summary • A new constant bit rate control method based on TMN8 • Use simple preprocessing to achieve the target bit rate • Better target bit rate, bit allocation, buffer management • Adoption of virtual frame skipping • Frame level and MB level rate control • Simulations show that the method meet the target bitrate even for scene changes and scene transitions

33. Preprocessing Stage • Chicken and egg problem in RDO: • Quantization parameter QP is needed for RDO such as mode decision • Residue signal is needed to determine the QP, i.e. mode has to be decided to obtain the QP • For I frames, residue signals are estimated using original pixels • For P frames, rate constrained 16x16, 1 reference frame ME is performed to obtain the estimated residue • Average QP of previously coded picture is used the determine the 

34. Frame-layer rate control • Determine the frame QPf • Use GOP layer rate control • GOP length = 1s • Constraint the number of bit allocated for GOP to prevent buffer overflow • Unused bits are distributed over several following GOPs • Allocate more target bits for P pictures at the beginning of GOP for better references • Virtual buffer level is used to prevent QP deceases very quickly

35. MB-layer Rate Control • For I picture, a higher distortion is given to MBs with less detail • For P picture, a higher distortion is given to the MBs with more residue errors • Better perceptual quality is maintained for I picture and can be propagated to following P pictures • QP variation within a frame is limited to QPf 2

36. Virtual Frame Skipping • After a frame is encoded, if buffer level > 90% of total buffer, next frame is virtual skipped until buffer level is less than 90% • Virtual skipped is achieved by code every MB in P picture to skip mode • Increase the frame QP by 2 after a frame is skipped

37. Experimental Results

39. Comments • An implemented rate control system

40. Discussions

41. Video Encoder Complexity Reduction by Estimating Skip Mode Distortion I. Richardson and Y. Zhao The Robert Gordon University

42. Summary • Predicts MBs that are likely to be skipped by the encoder • By estimating the increase in distortion due not skipping • Complexity reduced as motion estimation, FDCT, quantization and VLC is skipped for a skipped MB

43. Macroblock Distortion and Skip Prediction • SAE is used as distortion measure: • The difference between the SAEskip and SAEnoskip is used to determine whether a MB will be skipped ot not • SAEnoskip is not available at the encoder, estimated by SAEnoskip of previous frames

44. Experimental Results

45. Experimental Results

46. Experimental Results

47. Comments

48. Discussions

49. An Improved Rate-Quantization Model for Rate Control in Real-Time Video Encoding B. Xie and W. Zeng PacketVideo Corporation, San Diego University of Missouri-Columbia

50. Rate-Distortion Optimized Video Coding with Stopping Rules: Quality and Complexity M. Moecke and R. Seara Federal University of Santa Catarina