1 / 73

Video Compression Standards

Video Compression Standards. Wen Gao Peking University. Outline. Technology issues IPR issues Effort in RF or low price IPR Future standards Summary. Video coding standards – how many aspects need to look at?. At least two Technical achievement on removing redundancy

wshort
Download Presentation

Video Compression Standards

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Video Compression Standards Wen Gao Peking University

  2. Outline • Technology issues • IPR issues • Effort in RF or low price IPR • Future standards • Summary http://www.avs.org.cn

  3. Video coding standards – how many aspects need to look at? • At least two • Technical achievement on removing redundancy • IPR policy movement http://www.avs.org.cn

  4. Three issues on video coding technology • Where the redundancy come from, how many kind of redundancy can be removed • What is the up-bound for video coding (lossless/lossy) • Why standard progress is always in traditional way (keeping in hybrid coding) http://www.avs.org.cn

  5. Answer to issue 1 • Spatial redundancy • From over sampling • Can be removed by • Transform/prediction • Analysis/reconstruction (inpainting, texture coding, …) • Temporal redundancy • From duplication of no change region • Can be removed by • Prediction (motion) coding • Coding redundancy • From normal bit allocation for sparse data • Can be removed by entropy coding • VLC/arithmetic coding http://www.avs.org.cn

  6. Major tools in hybrid video coding Transform coding Input Video Signal Split into Macroblocks 16x16 pixels Coder Control Control Data Transform/Quant coeffs - Entropy coding Prediction coding Inv Quant/ Inv Transform Entropy Coding De-blocking Filter Intra-frame Prediction Output Video Signal Motion- Compensation Intra/Inter Motion Data Motion Estimation http://www.avs.org.cn 6

  7. 1970s 1960s Ahmed, 1974 Andrews, 1968 Pratt, 1969 Chen, 1973 Fourier Transform Hadamard Transform Slant Transform Cosine Transform Pratt 1969 Huang 1966 Statistical Coding Zonal Coding Habibi, 1971 Tasto 1971 Block Classification Huang 1963 Max, 1960 Adaptive Block Coding Non-Uniform Quantizer Block Quantization Fano 1949 Seyler 1962 Huffman 1952 Chen 1981 VLC Video Variable-Length Coding VLC Scene Adaptive Coder Woods 1969 Dead-Zone Quantizer Uniform Quantizer Pratt 1969 Tescher, 1976 Threshold Coding Pratt, 1970 ZZ-Run-Length Coding Run-Length Coding Tescher, 1975 Zigzag Scan Tescher, 1977 C Reader, 2002 Rate Buffer Spatial Coding Tools http://www.avs.org.cn

  8. 1960s 1970s Limb 1975 Netravali 1979 Seyler 1962 Brofferio 1970 Motion Measurement PRA Motion Estimation Temporal Statistics Motion Modeling Seyler 1962 Candy 1971 Netravali 1979 Conditional Update (Spatial; Pixel) Motion Compensated DPCM (Pixel) 3D (Temporal) DPCM H.120 Harrison 1952 Rocca 1969 Koga 1981 Conditional Update (Spatial; Block) DPCM Motion Compensated DPCM (Block) Giorda 1975 Taki 1974 Variable – Size BMA Motion Est. BMA Motion Estimation Jain 1979 H.261 MPEG1 MPEG2 Etc. Schroeder 1970 Transform in DPCM Loop with MC Hybrid Coding Roese 1975 Netravali 1979 Reader 1973 Pratt 1968 Motion Compensated Transform Hybrid Coding (Transform) Conditional Update (Transform) Transform Coding (Intra) Jones 1978 Interlaced Field/Frame Coding MPEG2 C Reader, 2002 Temporal Coding Tools http://www.avs.org.cn

  9. Answer to issue 2 • Given a 1080P(30fps) video data, what is the up-bound of lossless compression? • Typical MPEG-2 compression is in 20Mbps • 1080x1920x8x3x30 ≈ 1500Mbps • 150 : 20 ≈ 75 : 1 • Typical AVC/H.264 compression is in 10Mbps • 1080x1920x8x3x30 = ≈ 1500Mbps • 1500 : 10 ≈ 150 : 1 • Can HVC/H.265 do at 300:1? • Can people achieve at 1000:1 or even higher? http://www.avs.org.cn

  10. If someone find an ideal transform, then we can get 1920:1 If we can find a ideal transform http://www.avs.org.cn

  11. Up-bound? compression 2000 One possible up-bound 1500 AVS/H.264 1000 MPEG-1/MPEG-2 500 VCD HDTV DVD http://www.avs.org.cn

  12. Real up-bound？ • By that up-bound, we can have • DVD(720x576), 720 : 1 • HDTV(1920x1080), 1920 : 1 • XDTV(4000x2000), 4000 : 1 • If that is true, considering temporal factor, then we should have at least 1 order higher that just deal that in one frame • Please let me know, if you get the real up-bound http://www.avs.org.cn

  13. Next G AVS MPEG-4 AVC MPEG-4 MPEG-2 MPEG-1 2009 2017 2013 2005 1997 2001 1989 1993 Standards still have enough space to work • ISO/IEC MPEG • MPEG-1/2/4/AVC/HVC • ITU-T VCEG • H.261/2/3/4/5/??? http://www.avs.org.cn

  14. Answer to issue 3 • After big success of MPEG-2, we do need to face legacy resource (content, product in use, technology, …) • In simple word, industry want to keep their early investment valuable http://www.avs.org.cn

  15. History of Video coding Standards • ISO/IEC recommended MPEG series standards • MPEG-1, MPEG-2, MPEG-4, MPEG-4 AVC • ITU-T recommended H.26X series standards • H.261, H.262, H.263, H.263+, H.263++, H.264 http://www.avs.org.cn

  16. Tools in H.261, MPEG-1, MPEG-2 Input Video Signal Split into Macroblocks 16x16 pixels Coder Control Control Data Transform/Quant coeffs - Inv Quant/ Inv Transform Entropy Coding De-blocking Filter Intra-frame Prediction Output Video Signal Motion- Compensation Intra/Inter Motion Data Motion Estimation http://www.avs.org.cn 16

  17. Tools in AVC/H.264, AVS Input Video Signal Split into Macroblocks 16x16 pixels Coder Control Control Data Transform/Quant coeffs - Inv Quant/ Inv Transform Entropy Coding De-blocking Filter Intra-frame Prediction Output Video Signal Motion- Compensation Intra/Inter Motion Data Motion Estimation http://www.avs.org.cn 17

  18. 1950s 1960s 1970s 1980s 1990s 2000s Entropy Coding 1949-1976 DPCM 1952-1980 Motion Compensated Prediction 1972-1989 Transform Coding 1965-1980 H.261 1984-1990 JPEG 1984-1992 MPEG1 MPEG2 H.263 MPEG4 AVC/H.264 VC-1 AVS Main Tools & Standards Modified From C Reader http://www.avs.org.cn

  19. Video coding standards IPR issue • What base to pay • Device base by end user/manufacture/chip company/software company • Usage base by content provider/operator • How much to pay • Pricing, by percentage/fixed rate • How long have to pay • Patent life time, 17year/20year http://www.avs.org.cn

  20. Who control MPEG IPR policy • Non-MPEG organization • Profitable company • Maintain MPEG-2, MPEG-4, MPEG-4 AVC, VC-1,… http://www.avs.org.cn

  21. MPEG-X/H.26x IPR policy • MPEG-2 licensing term weakness – fixed price • IPR policy for patents in MPEG-2 is good in beginning, but not good now • 4.0$/device => 2.5$/device (after 2002) • Now cost for MPEG-2 STB is under 15$, IPR charge is over 14% • In normal sense, patent charge should be within 5-10% • MPEG-4 licensing term, charge to operator • Against by broadcasting operators from Japan, USA, Europe, … • IPR policy for patents in MPEG-4 is bad, almost kill MPEG-4 • IPR policy for patents in MPEG-4 AVC/H.264 is still with question, maybe have problems later on http://www.avs.org.cn

  22. Participation Fees Where End User pays Subscription Title-by-Title Codec Manufacturers Internet Broadcast Participation Fees where Remuneration is from Other sources Free Television AVC/H.264 licensing structure Source:AVC/H.264 license terms http://www.avs.org.cn

  23. Case estimation for operator • C1 (Beijing Cable TV, a pay TV operator) • They have 3M users, if every user watch 2 title each day • Then they might need to collect from end-users • 3M * (0.02 * 2 * 365)/year = 43.8M USD/year • Paying in Cap : 4.25M(in 2009); 5.00M(in 2010) • C2(China mobile, a mobile operator with 31 subsidiary company) • If each subsidiary has over 100K VOD/TimeShifting service subscribers • Then they may need to pay amount of cap • 4.25M*31= 131.75M USD/yr, in 2009 • 5M*31=155M USD/yr, after 2009 • C3(CMMB operator, a mobile TC operator in China, with 150 subsidiary company now, maybe over 600 after 2011) • Paying in Cap • 4.25M*150= 627.5M USD/yr, in 2009 • 5M*150=750M USD/yr, after 2009 • 5M*600=3B USD/yr, after 2011 http://www.avs.org.cn

  24. Solution? • Industry looking for more selection/replacement • RF MPEG-4 AVC/H.264 baseline • VC-1 • AVS • OMS • With same/better coding performance than H.264/AVC • Suitable for big operator like China Telecom, China Netcom, China Mobile, CMMB, Telefonica http://www.avs.org.cn

  25. Video Object Planes Variable Block-size Motion Compensation Macroblocks Transform Coding Multiple Reference DPCM P-frames B-frames Generic B-pictures DCT 1950 1999 ~1989 Advanced Deblocking Filter Hybrid Coding Scene Adaptive Coder Integer Transform Interlace Huffman Coding Motion Vector Prediction Block Motion Estimation Object-Based Scalability Error Resilience Modified from C Reader, 2002 Using for free Need to pay Periods of key video coding tools contributed http://www.avs.org.cn

  26. Effort in RF or low price IPR • FR baseline @ MPEG-4 AVC/H.264 • VC-1 @ Microsoft • AVS @ China • RF video codec @ Sun Microsystems http://www.avs.org.cn

  27. MPEG-4 AVC/H.264 FR baseline • The foundation of AVC/H.264 IPR policy was laid in early 2001 • Inherited by JVT when it was formed in December 2001 • JVT-C110 [Sullivan, Microsoft] JVT IPR Status Report • JVT-C085 [Kogure+, Matsushita] IPR WG establishment for JVT • JVT-C123 [van der Meer, Philips] JVT Licensing issue • JVT-C124 [van der Meer, Philips] Information on JVT patents • JVT-C149 [Yagasaki, Sony] Licensing issues on JVT • JVT-C150 [Lindbergh, Polycom] Support for JVT Royalty Free Baseline • By May 2002, FR Baseline supported by companies including • Apple Computer, Inc. (USA), British Telecommunications plc (UK), Broadcom Corp. (USA), Cisco Systems, Inc. (USA), Conexant Systems, Inc. (USA), Deutsche Telekom AG (Germany), FastVDO LLC (USA), Nokia Corp. (Finland), Polycom, Inc. (USA), RADVISION, Inc. (Israel), SANDVIDEO (USA), Siemens AG (Germany), Sun Microsystems, Inc. (USA), Tandberg (Norway), Telenor (Norway), Teles AG (Germany), Texas Instruments, Inc. (USA), UBVideo (Canada), VCON (Israel)VideoLocus Inc. (Canada), ViXS Systems Inc. (Canada), VWeb Corp. (USA) • Activity slow down in 2003 • some key contributors to H.264 that supported the RF baseline had donated their IP to the standard, so it could not be used defensively against the IP holders that opposed the RF baseline. • The promise to license RF that accompanied every contribution had a key weakness – the promise was conditional on all others also licensing RF • MPEGLA licensed H.264 with a single license for all profiles, not distinguishing the baseline in any way. The fee however was far lower than the MPEG2 fee http://www.avs.org.cn

  28. VC-1 • In 2001, Microsoft create VC-9 • Microsoft promise in low IPR charge • VC-9 approved and recommended by STMPT, becoming a US standard • After VC-9 open to public, Microsoft faced a hard time • In the end, Microsoft agreed to license their patents via same channel as AVC/H.264 http://www.avs.org.cn

  29. AVS • Starting in 2002 • Promise in very low IPR price http://www.avs.org.cn

  30. Making a good AVS for industry We need Good IPR policy Good coding technologies Good industry promotion Three horses cart AVS WG AVS PPC AVSA http://www.avs.org.cn 30

  31. AVS Working Group • Set up by MII (Ministry of Information Industry) in June of 2002 • In charge of national standardization of audio-video coding for Chinese A/V industry • Also in charge of organizing the delegation of China National Body to MPEG(ISO/IEC JTC1/SC29/WG11) http://www.avs.org.cn

  32. AVS members • Total 192 members (by Dec. 30, 2007) • CE, 12%； • Com, 12%； • Comp and Soft, 44%； • IC, 14%； • Local research entities, 25% • Worldwide research entities, 30% http://www.avs.org.cn

  33. AVS Multinational members http://www.avs.org.cn

  34. AVS Local Members http://www.avs.org.cn

  35. AVS industry Alliance May 25, 2005, announced at People’s Great Hall http://www.avs.org.cn

  36. Amoisonic Broadcom Changhong Haier Highsense Huaju Foundation Huawei Langchao NSCC … Panstar Semi SVA Skyworth Spectrum Communication Sunnywell TCL USstarcom ZTE … AVS Industry Alliance (32 by 2008) http://www.avs.org.cn

  37. AVS IPR Experts Group • Late 2003 the President of AVS asked that a group of experts be assembled to create a recommendation for an IPR policy for AVS • Create policies that: • reflect and respect China law and culture • reflect WTO requirements • reflect successful global practices • balance the rights of the inventor and needs of the implementers • innovative and forward looking • evolves with practice and law in China http://www.avs.org.cn

  38. AVS IPR Experts Group • Experienced members:经验丰富的成员 • Patent Attorneys专利律师 • Technical Experts技术专家 • Contract Attorneys处理合同的律师 • Previous experience in setting up patent pools具有建立专利池相关经验的成员 • Members are from:成员来自 • Asia 亚洲 • Europe 欧洲 • North America 北美 • Members represent: 成员代表了 • Consumer Electronics 消费电子产品界 • Information Technology 信息技术界 • Microelectronics微电子产业界 http://www.avs.org.cn

  39. AVS IPR Experts group • Key members • BroadCom • IBM • Intel • Matsushita/Panasonic • Microsoft • Nokia • Sony • Sun Microsystems http://www.avs.org.cn

  40. Licensing Obligations • Non-Participant: • RAND RF • POOL • RAND • No License • Participant: • RAND RF • POOL • RAND • Join AVS • Sign Member Agreement • Declare default option • Participate in Subgroup • Commit to license • License options • RAND RF • POOL • RAND • Submit Contribution • Disclose patents • Declare license option • China Patent: • RAND RF • POOL • Worldwide Patent: • RAND RF • POOL • RAND http://www.avs.org.cn

  41. Disclosure Obligations • When making a contribution • Unpublished, published and granted patents • Ongoing • When a patent is discovered • When a Patent Application is published • During Final Draft Standard Review • 90 days to declare licensing option, OR: • Default applies http://www.avs.org.cn

  42. AVS Licensing • 1 RMB concept • Only one charge, to device/software/chip • No charge to operator • No charge to end-user • Will be handled by AVS PPA • PPA is a non-profit organization • Founded By #1 institute of MII and ICT of CAS, in 2005 http://www.avs.org.cn

  43. HDTV/SDTV broadcasting and Storage applications Part2: Video Part1: system Part3: Audio Part4: DRM Mobile Applications Part7: Mobile Video Part7: Mobile Video X profile AVS Standardization Roadmap http://www.avs.org.cn

  44. AVS Video Technical Aspect • High efficiency • Similar with H.264/MPEG-4 AVC • 2 to 3 times higher than MPEG-2 • Lower complexity • 2 reference frames • 8*8 block but not 4*4 • No useless tools for SD and HD video coding • 70% for decoder as that of H.264 • 30% for encoder • Compatible with MPEG-2 at system level http://www.avs.org.cn

  45. AVS1-P2 diagram and special techniques Input Video Signal Split into Macroblocks 16x16 pixels Coder Control Control Data Transform/Quant coeffs - Inv Quant/ Inv Transform Entropy Coding De-blocking Filter Intra-frame Prediction Output Video Signal Motion- Compensation Intra/Inter Motion Data Motion Estimation http://www.avs.org.cn 46

  46. Performance- HD (1280x720) http://www.avs.org.cn

  47. Performance – SD (720x576) http://www.avs.org.cn

  48. Performance – CIF (352x288) http://www.avs.org.cn

  49. AVS1-P2 Subjective Testing • April-Sept., 2005, AVS subjective testing has been executed by SARFT testing center • The testing is based on GY/T 134-1998 standard as well as related ITU testing standard http://www.avs.org.cn

  50. AVS video standard In Feb. 2006, AVS video issued as national standard GB/T 20090.2-2006 Effective from March 2006 http://www.avs.org.cn 51

More Related