1 / 50

MPEG for the past, present and future of television

MPEG for the past, present and future of television. Leonardo Chiariglione Keynote Speech at ATSC Annual Meeting Washington, DC – 2012/05/08. MPEG for the past of television. System Transport Stream DSM-CC Timed metadata Video MPEG-2 High Profile Audio MPEG-1 Audio Layer II

malo
Download Presentation

MPEG for the past, present and future of television

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. MPEG for the past, present and future of television Leonardo Chiariglione Keynote Speech at ATSC Annual Meeting Washington, DC – 2012/05/08

  2. MPEG for the past of television • System • Transport Stream • DSM-CC • Timed metadata • Video • MPEG-2 High Profile • Audio • MPEG-1 Audio Layer II • MPEG-2 AAC MPEG for the past, present and future of TV

  3. An assessment • MPEG has provided the means for the television business on air, cable and satellite to migrate from the analogue to the digital age • It has been a collective efforts • Representatives from the entire spectrum of the broadcasting world provided their requirements • Packages of patent licences were created (outside of MPEG) • Today there is virtually no broadcasting system that is not based, at least partially, on MPEG standards MPEG for the past, present and future of TV

  4. MPEG for the present of television • Systems • Carriage of MPEG-4 on MPEG-2 TS • Dynamic Adaptive Streaming over HTTP (DASH) • Video • MPEG-4 Advanced Video Coding (AVC) • 3D extensions • Audio • Various forms of Advanced Audio Coding (AAC) • Universal Speech and Audio (USAC) MPEG for the past, present and future of TV

  5. An assessment • The deployment of 2nd generation digital broadcasting systems continues successfully • Use of the AVC on MPEG-2 TS allows packing twice as many programs as MPEG-2 Video in an analogue channel • With IPTV the notion of broadcasting is expanding • MPEG is helping the broadcasting industry • To enrich the broadcasted user experience • Through standard technologies that serve multiple fields MPEG for the past, present and future of TV

  6. MPEG for the future of television • Well, that’s what I am supposed to talk about… MPEG for the past, present and future of TV

  7. Broadcasting is a great business to be in… • Broadcasting is a proven business with a constant revenue flow • Many technologies appear that • Can help new competing businesses but • Can also be used to extend the broadcasting business • MPEG is a neutral body devoted to the development of standard technologies for user communities to exploit MPEG for the past, present and future of TV

  8. MPEG-H: a video standard for future broadcasting • ISO/IEC 23008 High Efficiency Coding and Media Delivery in Heterogeneous Environments • A suite of standards for media coding and delivery • Currently scheduled to appear in 3 parts • MPEG Media Transport • High Efficiency Video Coding • 3D Audio • Timeline MPEG for the past, present and future of TV

  9. Service composition in MMT MMT Package composition information friendly with html5 <subtitle> Bird is flying… UD contents Bird is flying… 3D Earth legacy broadcasting future broadcasting MPEG for the past, present and future of TV

  10. Smart delivery in Clouds Clouds video file Web • Service Component List • Component location • Composition relationship • QoE requirement • How to deliver • How to consume pull pull MMT Encapsulator MMT Frame • Configuration • Adaptive delivery • Sync, Delay, Loss control

  11. Mash-up service Audio Video1 Video2 widget Web page MMT assets Service 1 (MMT Package1) Service 2 (MMT Package2) MPEG for the past, present and future of TV

  12. Second screen management MMT Package composition information Issue <device 1> … <device2> … Video content on 2nd device Additional Information with Live Content (HTML) MPEG for the past, present and future of TV

  13. MPEG-H part 2 High Efficiency Video Coding • Primary goal: ≥ 50% better compression than AVC • For large displays (e.g. 4kx2k) • For mobile (e.g. WVGA) • Developed jointly with ITU-T by Joint Collaborative Team (JCT) on Video Coding • Technically in the same stream of past MPEG Video Coding standards (AVC and prior standards) • Currently at Committee Draft level MPEG for the past, present and future of TV

  14. HEVC vs AVC Subjective Performance Measurements • Preliminary subjective tests carried out for HEVC Committee Draft • Reasonably small and non-overlapping confidence intervals • Average rate savings at same (interpolated) MOS points • HEVC with 67% lower rate in class B (Full HD) • HEVC with 49% lower rate in class C (WVGA) • HEVC with 58% lower rate overall MPEG for the past, present and future of TV

  15. A performance sample MPEG for the past, present and future of TV

  16. For those technology aware… • Block-based/Variable block sizes • Block motion compensation • Fractional-pel motion vectors • Spatial intra prediction • Spatial transform of residual difference • Integer-based transform designs • Arithmetic or VLC-based entropy coding • In-loop filtering to form final decoded picture MPEG for the past, present and future of TV

  17. Where we are (May 2012) • Activity: level of 1,000 documents per meeting crossed • Very diverse participation from industry/academia • Significant technical advance over prior standards • Challenge: computational/implementation complexity • Deliverables: • Spec • Reference software • Conformance • Currently only one profile MPEG for the past, present and future of TV

  18. HEVC Scalability Extensions/1 • Serve the needs of heterogeneous environment of future digital TV distribution • Multiple devices: SHDTVs, set-top boxes, tablets, smart phones, PCs etc. • Wide range of processing powers, display sizes and power consumption needs • Multiple networks: Cable, Satellite, Terrestrial, CDN, Cable Modem, xDSL, WiFi, 4G • Wide range of channel bandwidths and QoS • Multiple protocols: MPEG-2 TS, DASH, HLS, RTP, UDP • Multiple services: Broadcasting, On demand, Streaming, Over The Top MPEG for the past, present and future of TV

  19. HEVC Scalability Extensions/2 • Options for backward compatible migration paths • From 720p60 to 1080p60 • From1080p60 to 4kx2kx60 • From AVC to HEVC • Time Line • May 2012 Preliminary Call for Proposals (public) • July 2012 Final Call for Proposals (public) • October 2012 • Evaluation of proposals & start of collaborative design phase • Development of the time line associated with standardisation phases (WD, CD/PDAM, FDIS/FDAM, DIS/AMD) MPEG for the past, present and future of TV

  20. Service Transitions • 1950s: Color TV • Analog, backward compatible • 1990s: Digital TV • New infrastructure required • Transitions from SD to HD • 2010s: 3D • Mixed results from services introduction is • Not a single format across all services MPEG for the past, present and future of TV

  21. Current Status of 3D Video • Increasing investments in 3D production and services • Increasingly capable 3D displays in the market (many competing and emerging technologies) • Market is primarily stereo(mix of different formats being deployed right now) • Focus of current MPEG work: development of a new 3D format and associated compression techniques that could facilitate generation of multiview output MPEG for the past, present and future of TV

  22. Existing 3DV Coding Formats • Pack pixels from left and right views into a single frame (loss of resolution) • Leverage existing infrastructure and equipment, only require additional signaling to de-interleave frame • Embraced by broadcasters for first phase of 3D broadcast services • Full-resolution coding of stereo and MV video as extension of AVC • High coding efficiency achieved via inter-view prediction techniques • 2D compatibility supported • Adopted as format for 3D Blu-ray Disc • Being considered for second phase of broadcast standards Left Right MPEG for the past, present and future of TV

  23. Targets of Future 3DV Format MPEG for the past, present and future of TV

  24. 3DV Reference Framework Limited # Video Inputs (e.g., 2 or 3 views) Depth Estimation Video/Depth Encoder Decoder and View Synthesis Larger # Output Views 1010001010001 + Binary Representation & Reconstruction Process MPEG for the past, present and future of TV

  25. Overview of Call For Proposals • Test Material & Conditions • 8 test sequences (1920x1080 & 1024x768) • 4 target bit rates per sequence • 2-view and 3-view test scenarios • AVC and HEVC compatible test categories • 23 responses (12 AVC + 11 HEVC) • Evaluation in Nov 2011 • Objective quality (PSNR of synthesized views) • Subjective assessment (stereo and auto-stereo) MPEG for the past, present and future of TV

  26. 3DV Testing Scenarios 2-view Configuration 3-view Configuration MPEG for the past, present and future of TV

  27. Tool Categories • Texture coding • Independent of depth • E.g., inter-view prediction of color view, inter-view prediction of motion parameters and residual data • Using depth data • E.g., view synthesis prediction, motion prediction • Depth coding • Independent of texture • E.g., depth modeling modes, weighted prediction, reduced resolution coding • Using texture data • E.g., motion parameter inheritance, intra prediction MPEG for the past, present and future of TV

  28. 3DV Current Status • Extensive subjective evaluation of 20+ proposals to the 3DV CfP • Considered both AVC and HEVC compatible • Many new coding tools proposed for MV texture and depth coding • Substantial rate savings compared to capabilities of existing standards • AVC and HEVC extensions standardisation under wayd • Add support for depth • New AVC-based coding tools • MV extensions of HEVC • Hybrid architectures MPEG for the past, present and future of TV

  29. Hybrid Architectures • From a pure compression efficiency point of view, it is always best to use the most advanced codec • However, when introducing new services, providers must also consider capabilities of existing receivers and transition plan • Use Case: • Many terrestrial broadcast systems based on MPEG-2 • US cable systems based on mix of MPEG-2 and AVC • May not be easy to simply switch codecs in near term MPEG for the past, present and future of TV

  30. Multiview Extensions of HEVC • HEVC Stereo/MV extensions being considered • Extensions expected to be completed approximately 12 months after the base specification, i.e., Jan 2014 • Reportedly simple extensions of HEVC can achieve 30-40% bit rate reduction compared to HEVC simulcast • Further gains expected if block-level changes to codec are considered MPEG for the past, present and future of TV

  31. MPEG-H part 3: 3D Audio • Well, there is no Systems and Video without Audio… MPEG for the past, present and future of TV

  32. 3D Audio requirements/1 • High quality: quality of decoded sound perceptually transparent • Localization and Envelopment: Accurate sound localization and very high sense of sound envelopment within a targeted listening area (“sweet spot”) • Flexible Loudspeaker Placement: the transmitted audio program to be mapped to a rendering setup in which loudspeakers are in alternate locations • Interactivity: Interactive modification of the sound scene rendered from the coded representation MPEG for the past, present and future of TV

  33. 3D Audio requirements/2 • Rendering on setups with fewer loudspeakers (including headphones): ability to derive a signal from the transmitted program material for reproduction with fewer loudspeakers • Audio/visual alignment & consistency: ability to adapt the rendered acoustic scene to be consistent with the visual • Efficiency for decoding on different setups: ability to be rendered on all reproduction loudspeaker setups or headphone MPEG for the past, present and future of TV

  34. 3D Audio requirements/3 • Appropriate computational complexity and system latency for the target application (Broadcasting, Spatial two-(n-)way communication / telepresence • Transcoding for low bandwidth devices: A lower bandwidth signal can be extracted from the original program material • Backward compatibility: (e.g. to 5.1 channel programs and decoders or transmission of HRTF encoded signals) MPEG for the past, present and future of TV

  35. Envisioned Architecture • Inputs • Audio Channels • “normal” content • Audio Objects • “helicopter overflight” • Audio Scene • Ambisonics • Outputs • Loudspeakers • Headphones • Spatialized sound • In-frame loudspeakers • “Sound Bar” • Channel • 500 kb/s primary channel • 64 kb/s cellular channel MPEG for the past, present and future of TV

  36. Audio, part of an Audio-visual Experience • What is new in video? • Larger, high-resolution displays • Greater sense of envelopment • Possibly closer viewing distance • More efficient compression • Wireless transmission to portable devices • Immersive and enveloping audio experience MPEG for the past, present and future of TV

  37. Use Case 1Home Theatre • Many loudspeakers • 10.1 • 22.2 • “3D” Experience • Height loudspeakers for greater envelopment • Sense that audio objects are near or distant • When near, they are consistent with video image MPEG for the past, present and future of TV

  38. 22.2 multichannel sound system (NHK) MPEG for the past, present and future of TV

  39. Home Theater – Issues • Maintain compatibility with legacy systems: 5.1, stereo • Will content providers really step up to 22.2? • Will consumers adopt 22.2?: 2 front high • How to compress 22.2 for transmission? • MPEG AAC 64 kb/s/channel: -> 1.5 mb/s • How to render for legacy setups? • Relatively slow-growth market MPEG for the past, present and future of TV

  40. Use Case 2Personal TV • Small but high-resolution display • “super-tablet” • Local wireless communications • To cable or fiber home hub • Possible audio presentation • Headphones • Loudspeakers around perimeter of display • Issue: To what extent can the user get an enveloping experience from only “front” loudspeakers? MPEG for the past, present and future of TV

  41. Use Case 3Mobile TV • Hand-held display: Smartphone • Headphone listening: • Stereo, perhaps with binauralisation • Fastest growing market: Quick rollout and adoption of standard • What is new? • USAC for stereo; MPEG Surround for binauralisation • What is the “Wow” factor? • To spur adoption of new technology MPEG for the past, present and future of TV

  42. User Experience Spatialized on headphones Home Theatre Tablet TV MPEG for the past, present and future of TV

  43. Flexible Rendering • Content providers create 22.2 program • Consumed on stereo, 5.1, 10.1, 22.2 layouts • Consumed on “wrong” layouts • Mis-positioned surrounds • Missing surrounds • Non-standard layouts • 2 front high, 5.1 mid MPEG for the past, present and future of TV

  44. 3D Audio Test Platform • NHK “Loudspeaker Array Frame” (LAF) • Suggestions for • Alternative platform for assessing “Home Theatre” use case • Model for assessing “Personal TV” use case MPEG for the past, present and future of TV

  45. Summary • Provide compelling immersive audio experience for audio/visual programs • Create content once, present on many different loudspeaker layouts or on headphones • Provide high compression and high-quality audio presentation MPEG for the past, present and future of TV

  46. MPEG-V: Humans have more than two senses MPEG for the past, present and future of TV

  47. Augmented Reality Application Format (ARAF) Local Real World Environment Remote Real World Environment Remote Sensors & Actuators Local Sensors & Actuators ARAF file or stream MARBle User Media Servers Service Servers MPEG for the past, present and future of TV

  48. Augmentation Region in ARAF AR service provider A User A Broadcaster User B Augmentation Region AR service provider B MPEG for the past, present and future of TV

  49. The past, the present and the future, again • 25 years ago MPEG selected the digital media way bringing innovation and interoperability …and thrived • This year the broadcasting industry has started a move in future broadcast services with interoperability at its core …and it is bound to thrive • MPEG looks forward to continuing a cooperation with the broadcasting industry providing the necessary standard technologies MPEG for the past, present and future of TV

  50. http://mpeg.chiariglione.org/ MPEG for the past, present and future of TV

More Related