1 / 35

Internet Media Delivery Systems: Current Design, Open Issues, Research Problems.

Internet Media Delivery Systems: Current Design, Open Issues, Research Problems. G. S. Greenbaum, Ph.D RealNetworks – Codec Group Stanford University Feb 13, 2001. Outline. Introduction to Internet-based Media Distribution Systems Components

paiva
Download Presentation

Internet Media Delivery Systems: Current Design, Open Issues, Research Problems.

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Internet Media Delivery Systems: Current Design, Open Issues, Research Problems. G. S. Greenbaum, Ph.D RealNetworks – Codec Group Stanford University Feb 13, 2001

  2. Outline • Introduction to Internet-based Media Distribution Systems • Components • A Real Example of a Media System: RealSystem iQ • RealProducer 8.5 • RealSystem Sever 8 and Proxy 8 with Neuralcast Technologies • RealPlayer 8 • Open Problems • source/channel coding • server storage/complexity tradeoffs • network traffic optimization problems

  3. Introduction:: Ambitions of a Media Delivery System • The Overall Goal is to Maximize the End-User Subjective Experience to Streaming Multimedia • A Successful Internet Media Delivery System Must Provide: • Tools for the easy re-purposing of new and extant content distribution over digital networks • Robust, Distributed, Scalable, Secure, and Flexible Architecture for content distribution • Ubiquitous client distribution supporting diverse data types

  4. Introduction:: Components Capture Encoding Serving Internet distribution Playback IP network Real Player Real Server Real Producer SDK SDK SDK Plug-ins Plug-ins Plug-ins Real Proxy Plug-ins

  5. Introduction:: Components • Content Creation: Digitization and Encoding • Most general purpose content starts life in analog form which needs conversion to digital form • Rich multimedia data types: audio, video, image, VRML, etc. • Lossy and lossless modes of compression • Download vs Broadcast (streamed) modes • Encoding tools typically have little knowledge of the distribution network or computational resources of the client playback device • Metadata for efficient search and retrieval • Rights of content provider must be insured

  6. Introduction:: Components • Transmission and Distribution • Live and pre-recorded Broadcast and VOD modes of operation • Redundant, Intelligent, and Reliable architecture needed • Redundant:: Zero points of failure at each critical node of the distribution chain • Intelligent: Load balancing (dynamic capacity allocation) for efficient distribution • Reliable: Stream distributions that traverse public and private network segments; fortified against loss and congested • Architecture usually contains splitters/repeaters and caches for efficient usage of network/gateway capacity working in both a push and pull mode

  7. Introduction:: Components • Client-side Player residing on different appliances and diverse networks • DSL, Cable Modem, Intranets/firewalls, POTS, ISDN, etc. • PC’s, cellular devices, PDA’s, set-top boxes, digital VCRs, etc.

  8. RealSystem iQ:: RealProducer 8.5 • State of the Art Narrowband and Broadband Audio and Video Codecs: RealVideo 8 and RealAudio 8 • Pre-processing: • Noise removal • Resampling • Inverse-Telecine • Deinterlacing • Rate control: • Two-pass, VBR • Channel adaptation: • SureStream, forward unequal ECC, data interleaving

  9. “Storage” Broadcast SLTA library File System File Format Producer Live Text Live Pix Remote Broadcast Library Internet/Intranet

  10. RealSystem iQ:: Server 8 • Adaptive serving: • Bandwidth detection • Stream switching • Thinning • multicast • Designed for distributed delivery networks in mind: • proxy plug-ins • edge-server plug-ins • distributed licensing • Features: • Ad Insertion, Authentication extensions, remote administration, firewall proxy, etc. (see www.realnetworks.com for more details)

  11. “Storage” Network Services Server Registry RealSystem iQ:: Server 8 BroadcastApplications Admin, monitoring Applications Broadcast Plug-in File Format Plug-in File System Plug-in Generic Plug-ins (logging, monitoring, etc.) Server Core Packet Sink Plug-in Allowance Plug-in Internet/Intranet

  12. Server Clustering Originate & Deliver from Any Where An architecture free from ‘origins’ and ‘edges’

  13. Distributed Architectures

  14. RealSystem iQ:: Server 8 Neuralcast Technologies • NeuralCast Communications Protocol • The ability for a network of RealSystem Servers to become self-aware by exchanging information and making decisions. Current abilities include capacity sharing and capacity fail over in the event of a network or equipment issue. • NeuralCast Live Distribution • The intelligent and reliable delivery of broadcasts through a network or RealSystem Servers. Current abilities include multi-protocol transmission between servers, error correcting methods for streams, terrestrial and satellite multicast support. • NeuralCast Live Redundancy • From encoder to server or from server to server, the ability to send redundant streams, providing a fail-over feed in the event of a network or equipment outage.

  15. RealSystem iQ:: Server 8 Neuralcast Technologies • NeuralCast Live Distribution • Push and Pull models • Re-Transmission requests made at receiver not originating transmitter • FEC configurable • Multiple Transport Support allows the same broadcast to be transmitted via conventional unicast UPD or TCP across non-multicast enabled segments

  16. RealSystem iQ:: Server 8 Neuralcast Technologies • NeuralCast Live Redundancy • Stateless protocol decouples live broadcast packet contents from the transmitting network • Live broadcast streams that are sent redundantly over different networks may be re-converged at the Server receiver Encoder/ SLTA

  17. RealSystem iQ:: Live Webcast Deployment Server Splitter Relays

  18. RealSystem iQ:: Real Proxy 8 • Access Provider Deployment Goals • Optimize gateway bandwidth for streaming media • Eliminate redundant requests for media from subscribers for content that is back hauled from the public Internet • Improve playback quality experience by migrating content closer to the viewing subscriber • Deployed Technologies • RealSystem Proxy near Telco/Cable cross connects or at data center

  19. RealSystem iQ:: Real Proxy 8 Deployment Traffic Management Cache Acquisition (Cold)

  20. RealSystem iQ:: Real Proxy 8 Deployment Cache Playback (Hot) Live Splitting

  21. RealSystem iQ:: RealPlayer 8 • Post-Filters • Sharpening • color correction • graphic equalizer • Rescaling • Bandwidth Simulator • Reverb • Network Aware • Monitors Network connection • Auto-update enabled to obtain latest player components • Auto-configures for best transport method • Features • RV8, RA8, Visualizations, Radio Tuner, Media Search, localizations for different languages, etc. see www.real.com for a complete list.

  22. “Storage” Network Services Audio Services Client Registry Video Surface RealSystem iQ::Real Client Architecture RealPlayer RealJukebox ActiveX Netscape Plug-in Client Core Rendering Plug-in File Format Plug-in File System Plug-in Generic Plug-ins Internet/Intranet

  23. Open Problems... Overall minimum-distortion optimization Source/Channel coding Internet traffic problems IP network Real Player Real Server Real Producer Real Proxy Traffic optimization in dedicated delivery network Server storage vs. complexity tradeoff

  24. Source Coding Issues:The Challange • Video: no adequate distortion measure • Human retina - an ultimate 120:1 lossy compressor Human eye has around 6M cones + 120M rods; but only around 1M ganglion fibers in the optical nerve.

  25. Source Coding Issues:The Challange • Some other facts from physiology: • visual recognition requires power of ~30% of the cortex • hearing -- uses just 3% of the cortex • Ultimate video decoder = 1/3 of a human (adequately educated and trained)

  26. Source Coding Issues • Constraints: • Bounded delay (1-10 s and/or MB constraint) • Targeting many discrete, fixed bit rate connections • Random-access is necessary • Seek/rewind on pre-recorded media • Ability to join live stream • Packetization: Each packet must contain sufficient information to be decoded. • Low complexity decoding • Real-time encoding • Need to assume inherently lossy network transport

  27. Source Coding Issues • Bounded Delay • Not as much an issue for Downloadable Media • Leveraged to solve inhomogeneity in content bit allocation (VBR), network jitter, random access, re-transmission requests, etc. • Imposes constraints on look-ahead dependencies • Imposes constraints on interleaving techniques for error mitigation • Cannot replicated the “remote-control” paradigm of broadcast television.

  28. Source Coding Issues • Bandwidth Scalability • Layered Scalability, e.g. FGS, annex O. • Introduces additional delay • Hit in coding efficiency • Embedded techniques, e.g. FGS, wavelets • Introduces additional delay • Possible hit in coding efficiency • Multi-Encoding e.g. SureStream • Increased file size and possible delivery costs • Optimal coding efficiency for targeted rate • ?

  29. Source Coding Issues • How to facilitate random access decoding? • independent encoding • pre-encoded transitions • pre-encoded correlated fragments combined with Slepian-Wolf encoding • use of multiple-description codes • something else??? ?

  30. Source Coding Issues • Packetization • Not an issue for downloadable media • Imposes constraints on dependencies for streaming content leading to a decrease in coding efficiency • Due to IP and packet reconstruction overhead, packet size should be nearly as large as the packet fragmentation limit imposed by routers, 1500 bytes. => increased latency of stream • Complexity Scalability • How does the algorithm scale on limited resource devices for the encoder and decoder? • Scalable Post-filters • Limited decode: B-frames dropped or I-frame only • Short-cuts: limited ME/MC

  31. Channel Coding Issues • IP have been designed to relieve engineers from thinking about physical connection and channel coding problems • Ironically enough, with streaming media we now have to go back to basics… • UDP is mainly used=> packet erasures • ARQ and ECC • Client side mitigation techniques • Ideally, we want the server to do adaptive channel coding/ transmission. Cost is reduction in server load. • How much of this work can be done ahead of the time by encoder? • Is there a way to store all necessary information in a compact form? Is SureStream the ideal?

  32. Source/Channel Coding Issues • Separation Theorem may not hold: • packets have finite lengths • packets can be sent via multiuser channels • Should we use joint source/channel coding? • How to combine distortion due to lossy compression and one due to channel errors • can we design a universal lossy joint source/channel code?

  33. Source/Channel coding problems • Another way to deal with IP packets is to use multiple-description coding: • Ozarow (1980) The achievable set of quantuples (R1,R2 ,d1,d2,d3) is given by:

  34. Summary • Several embodiments of Media Delivery Systems exist today, but the technology is still in its toddler years. • Many more issues, problems, and questions than answers. • Plenty of opportunities for grad students, post-docs, and professors! (and job security for those outside of academia)

More Related