Bericht für den ITG Fachausschuss 3.2 Treffen am 27.6.2008, RWTH Aachen

1. MPEG Video Subgroup and JVT Activities83rd and84th MPEG Meeting, Antalya & Archamps26th and 27th JVT meeting, Antalya & Geneva Bericht für den ITG Fachausschuss 3.2 Treffen am 27.6.2008, RWTH Aachen

2. MPEG-2 & MPEG-4 • MPEG-2: New Level for 1080@50/60p (Amd.3 – currently FPDAM) • MPEG-4 part 2: New Levels for Studio Profiles (re-definition of 1080p, support for 4Kx2K) • Also conformance and software updates

3. 15938-3 MPEG-7 Visual • MPEG-7 Visual (ISO/IEC 15938-3) Amd.3 Image Signature Tools • Signature for global image features (based on trace transform) – Finding same images from large databases also under various modifications: Good performance unless heavy cropping, heavy translation • Status: FPDAM; FDAM expected for October • Currently extension of this method investigated, applying it locally around significant feature points • Other Core Experiments for MPEG-7 New Visual Extensions • VCE6: Localized signature under more difficult modifications of images • VCE7: Preparation of CfP for video signatures • Previous VCE5 (face recognition with infrared) discontinued, need better understand the purpose and whether this is relevant • Preliminary Call for Proposals on Video Signature Tools • More sequences sought, final CfP next meeting • Will probably be next amendment

4. Decoder Description Decoder Toolbox Decoding Solution Decoder Formulation Mechanism Reconfigurable Video Coding 23001-4 23002-4 Decoder Description Decoder Decoded Video Data Encoded Video Data Receiver

5. Reconfigurable Video Coding - Status • ISO/IEC FCD 23001-4 Codec Configuration Representation • Normative specification of reduced CAL (RVC-CAL) included • Specification of BSDL profiling for RVC • Annex C (listing "supported standards") removed – this is solely a matter of tool library definitions and completeness • ISO/IEC FCD 23002-4 Video Tool Library • Various updates • Support for all tools from MPEG-2 MP, MPEG-4 SP, AVC BP (some tests ongoing) + potentially SVC BP • Overall section structure, such that new tools, software and conformance can easily be added in • WD 1 of ISO/IEC 23002-4/Amd.1 (Conformance and Reference Software) • WD 1 of ISO/IEC 23002-4/Amd.2 (Tools for MPEG-4 ASP, AVC HP and SVC)

6. RVC – Other documents • Description of Core Experiments in RVC (N9777) • Further tools and parser developments (CABAC parser will certainly be a dedicated functional unit, bitstream conformance not testable by BSDL) • RVC Work Plan and FU Development Status (N9778) • RVC-SPS v1.0 (Simulation, Profiling and Synthesis Tools) (N9779) • Intersting support e.g. CAL->VHDL, CAL->C++ • Expected to become a TR (non-normative) in MPEG-C later • Process for Extending the MPEG RVC Toolbox (N9780) • Description of Exploration Experiments on Toolbox Extension Exercise (N9781) • Developing conformance testing for non-MPEG tools (AVS) • Status Report on using RVC Framework with non-MPEG Tools and Standards (N9782) • Summarizes achievements made so far in context of AVS example

7. 3D Video • What is focus of this work? Support for N-view displays (various types) where only low amount of views (1-3) and associated depth map(s) is encoded N M Kx video plus depth

8. 3D Video • Basic principle:Stereoscopicand multiviewdisplays capableto present variousviews • Maximum angle between leftmost and rightmost position expected to be around 20 degrees for the upcoming generations of displays

9. 3D Video • Possible approach of evaluating technology – proponents would be asked to provide • Output views (quasi continuous) • Encoded representation of depth + video (bitstream at given rates) • Decoder that generates the output views from bitstream (for proof) • What would we be going to compare against? • Original views vs. interpolated views (planned for Hannover) • MVC/AVC coding of video plus MVC/AVC coding of depth plus available view interpolation as probably "lower anchor" (this will be prepared jointly by the group after Hannover) • Concrete output views are randomly selected by the time when "expert viewing" evaluation is made

10. 3D Video • Time line (ambitious): • until 08/07: Generate depth maps and interpolated view sequences from the old and new test sets • 08/07: Perform experts viewing with different stereoscopic displays, decide for number of cameras and view distance useful for the MVC-plus-depth anchors • 08/07: Draft CfP (internal) • Until 08/10: Perform experiments to decide about bit rates for the MVC-plus-depth anchors, selection of sequences • 08/10: Experts viewing again, preliminary CfP • Until 09/01: Refinement of test conditions • 09/01: Final CfP • 09/04: Responses due

11. New Challenges in Video Coding • First Thoughts on New Challenges in Video Coding Standardization(N9785) • Summary of the discussion points from Archamps • Planning follow-up brainstorming workshop, soliciting contributions on • Is there need for higher compression efficiency? How mature are technical solutions? • Studies about performance of our standards in new application domains (including new display types for high resolution, high dynamic range etc.) • Studies about application domains where proprietary solutions might be considered as better (including reasons why this is the case) • What are criteria to evaluate performance in dedicated application domains? • What could be limits of using RVC for extending or developing standards? • Is it possible and useful to define performance indicators for encoders, delivery and decoders, would normative encoders be beneficial for certain application domains?

12. JVT Outputs of April/May 2008 • Study Text of ISO/IEC 14496-4:2004/FPDAM 30 Conformance Testing for New Profiles for Professional Applications(N9754) • Study Text of ISO/IEC 14496-4:2004/FPDAM 31 Conformance Testing for Scalable Video Coding(N9755) • Disposition of Comments on ISO/IEC 14496-5:2001/FPDAM 18(N9758) • Text of ISO/IEC 14496-5:2001/FDAM 18 Reference Software for New Profiles for Professional Applications(N9759)

13. Status of AVC development • 14496-10:200X edition (including SVC) was finalized • Text of ISO/IEC 14496-10:200X/DCOR 1 (N9574) • Various small bug fixes and typo corrections • Clarifications on SVC SEI messages and level constraints • Conformance and software amendments for SVC • Text of ISO/IEC 14496-4:2004/FPDAM 31 Conformance Testing for Scalable Video Coding (N9568 / N9569) • Text of ISO/IEC 14496-5:2001/FPDAM 19 Reference Software for Scalable Video Coding (N9572 / N9573)

14. SVC • Technology under further consideration • Bit-depth scalability • Color format scalability • Simplified FGS • No clear evidence about near-term need for action • Further study in AHGs

15. SVC Development • Report on SVC Verification Tests (N9577) • Over-simplified conclusion: • SVC enables scalability (temporal, spatial, and quality, and combinations) with not more than a 10% bit rate increase(relative to single-layer AVC) • In some configurations, quality parity was achieved with no increase of bit rate

16. What we did • Subjective Performance Evaluation of the SVC Extension of AVC • Several test cases for spatial, quality and combined scalability • Three application scenarios: TV broadcasting, Conversational, Movie Production • Three profiles tested: Scalable Baseline, Scalable High, Scalable High Intra • SVC compared to AVC either at 10% additional bit rate or at the same bit rate • Visual quality evaluated in subjective tests

17. Test Report • SVC Verification Test Report N9577 • 3 pages of introduction and summary • Details about • Results • Test scenarios and test conditions • Subjective Testing can be found in Annexes • Excel-Sheet containing all results • Bit streams are public available • For the first time (all) bit streams will be available • For some test cases even encoders and configuration files will be available

18. Results - quoting The test results show that scalability of video bit streams can be provided by SVC when a low cost in terms of bit rate overhead is accepted. No substantial quality penalty was found in the tested SVC application scenarios in comparison to the corresponding AVC results. This was true for spatial scalability as well as for SNR scalability and combined spatial/temporal scalability. For nearly all tested sequences and test cases, SVC with a 10% bit rate increase provided approximately the same or better subjective quality than AVC. In many test cases the same conclusion would appear likely to have been drawn even if a smaller bit rate overhead margin had been applied for the SVC encodings. Among the individual comparison points that showed a significant difference in quality for SVC with a 10% bit rate increase relative to a corresponding AVC encoding, the number of cases where the SVC quality surpassed the AVC quality exceeded the number of cases where the opposite was the case.

19. Results - quoting Therefore, it can be concluded that for the tested scenarios the maximum bit rate increase necessary to achieve scalability with comparable quality is 10%, a margin which had been an important benchmark goal of the SVC standardization project. For the Scalable High Profile and Scalable High Intra Profile test cases, typically parity of visual quality was reached for SVC and AVC even at the same bit rates. In the tested scenarios, the bit rate savings of SVC in comparison to simulcast of several single-layer AVC bit streams ranged between 17% and 40%. These bit rate savings relative to simulcast are particularly important for applications in which video must be provided with different spatial resolutions, for which simulcast would previously have been the only available AVC-based standardized solution.

20. Results Example • Baseline Broadcasting SNR Scalability QVGA - QVGA Uncertainty Area EL BL

21. Multiview Video Coding • Disposition of Comments on ISO/IEC 14496-10:200X/PDAM 1 (N9575) • Text of ISO/IEC 14496-10:200X/FPDAM 1 Multiview Video Coding(N9576) • Only contains changes above macroblock level • Only one profile (multiview high) proposed • Level definitions need more clarification, adopt constraint set flag proposal (JVT-Z049) • Improved high-level syntax (JVT-Z038) • Overview of Multiview Video Coding(N9580) • Update for MPEG Technologies web page

22. Multiview Video Coding • Technology in FPDAM gives approx. 20% bit rate reduction compared to simulcast • Sufficient to enable applications with minimal implementation difficulty • Macroblock-level tools investigated could give another 10% additionally • Not clear yet (in terms of applications/profiles) whether this is valuable • Expect more improvement and clarification about usefulness from AHG study (potential phase 2) • Joint Multiview Video Model (JMVM) 7(N9578) adopts • JVT-Z021: Fine motion matching for motion skip • JVT-Z032: Clarification Motion skip enable flag • JMVM 7 Software(N9579) • Testbed for studying improved coding tools • Reference software will be separated by next meeting

23. AVC – Other Developments • Bitstream splicing: • Three input proposal documents (JVT-Z040, JVT-Z041,JVT-Z042) • Incoming liaison statement from ITU-T SG 9 (M15072) • Sending reply to liaison statement; establishing AHG to study • Adaptive motion vector precision exploration proposal (JVT-Z022 / M15185) • Coding efficiency improvement and reduction of memory bandwidth reported • Results are preliminary, but some coding efficiency gain reported • Further investigation encouraged

24. JVT Outputs of April / May • Study Text of ISO/IEC 14496-10:2008/FPDAM 1 Multiview Video Coding(N9760) • Working Draft 1 of Multiview Video Coding Reference Software(N9761) • Joint Multiview Video Model (JMVM) 8(N9762) • JMVM 8 Software(N9762) • Overview of Scalable Video Coding (SVC)(N9792) • Further work on Text of ISO/IEC 14496-10:200X/DCOR 1 (N9574) • Clarifications on AVC and SVC • The term "Constrained Baseline bitstream" will designate a baseline bitstream with constraint_set1_flag set equal to one – not another profile (from the consensus reached in JVT in reaction to NB positions)