1 / 61

Introduction and Architectures

This course provides an overview of multimedia applications, challenges, architectures, media on demand, machine internals, and video server structures. It covers topics such as network content architectures, file systems, media data distribution protocols, and resource scheduling.

sisbell
Download Presentation

Introduction and Architectures

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. INF5070 – Media Storage and Distribution Systems: Introduction and Architectures 29/8 - 2005

  2. Overview • Intro • about the course • multimedia applications and challenges • Architectures • Media (Video) on Demand • Machine internals • Video server structures • Examples

  3. INF5070:The Course

  4. Lecturers • Carsten Griwodz • email: griff @ ifi • office: 3343 • Pål Halvorsen • email: paalh @ ifi • office: 3345

  5. Network Network Network Network Content architectures file systems media data distribution protocols resource scheduling topologies

  6. Content - lectures • System architectures(server and system designs) • Media and user behavior (wire and file formats, codecs, processing multimedia data, user interactivity) • Resource management(CPU and memory management) • Protocols with and without Quality of Service (QoS)(specific and generic QoS approaches)

  7. Content - lectures • Storage systems (management of multimedia files, retrieval) • Distribution(use of caches and proxy servers) • Peer-to-Peer(various clients, different amount of resources) • Guest lecture 7/11: The FAST searching system (architecture: resource utilization and performance, storage and distribution of data, parallelism, etc.)

  8. Content - student assignment • Mandatory student assignment (will be presented more in-depth later): • write a project plan describing your assignment (3/10) • write a report describing the test and its results, and give a presentation to the class at the end of the course (28/11) • for example (examples from earlier): • TCP friendliness variations • New TCP implementations • Network emulators • Comparison of Linux schedulers • File system benchmarking (different OSes and file systems) • Comparison of open source video servers (Helix vs. Darwin) • Comparison of methods for network performance monitoring • …

  9. Goals • Media servers and distribution system • media characteristics • architectures • system support • protocols • distribution mechanisms • … • Be able to evaluate any combination of these mechanisms, e.g., • data placement on disk and memory caching algorithm • video popularity and proxy caching algorithm • …

  10. Exam • Prerequisite: approved presentation of student assignment • Oral exam (??/12-2005): • all transparencies from lectures • content of your own student assignment

  11. Applications and Challenges

  12. Applications • Multimedia enriches the user interface giving new applications • Broadcast – server is only VCR substitute (e.g., MTV Europe) • Media-on-Demand (MoD) • Network-based, user controlled distribution of media content,e.g.: • Video-on-Demand (VoD) (e.g., Bell Atlantic (‘93)~1000 users, 700 choices) • News-on-Demand (NoD) (e.g., CNN, BBC  web today, BAD quality) • Learning-on-Demand (LoD) (e.g., the OMODIS project) • Virtual worlds (typically live, not on-Demand) • video conferencing (e.g., USIT’s electronic classroom) • games (e.g., Anarchy online, Quake, …)

  13. Requirements • Application • QoS – time sensitivity • resource capabilities – support interactive streaming of multimedia content • Business • scalability • reliability • Architectural • topology • cost vs. performance

  14. Technical Challenges • User end system • real-time processing of streams (1000 MIPS for an MPEG-II decoder) • request/response delay (< 150 ms for videophones) • high data rates, e.g., MPEG-II DVD quality: • average video rate of 3.5 Mbps • average transport stream of 4 – 8 Mbps (video, audio, headers, error protection) • max. total video data rate of 10.08 Mbps • max. user rate of 11.08 Mbps (all included like control signals) • Network • real-time transport of continuous media data • TCP fairness • mobility • …

  15. Technical Challenges • Servers (and proxy caches) • storage • real-time retrieval of contiguous media streams, e.g.: • 4000 movies * 90 minutes * 15 Mbps (HDTV) = 40.5 TB • 2000 CDs * 74 minutes * 1.4 Mbps = 1.4 TB • I/O • many concurrent clients expecting continuous playout of DVD-quality video (~ 4Mbps) • disk: Seagate X15 - ~400 Mbps • network: Gb Ethernet (1/10 Gbps) • bus(ses): - PCI 64-bit, 133Mhz (8 Gbps) - PCI-Express (2.5 Gbps each direction/lane) • computing in real-time • encryption • adaptation • transcoding • …

  16. Media-on-Demand (MoD) Systems • Classification parameters • interaction • structure of movies • presentation form • Common directions • analog  digital media • distribution  interaction • media broadcasting  media multicasting  personalized media • linear movies  branched movies  variable movies • Evolution • broadcast - traditional, no user control • near MoD - same media distributed in intervals (pay-per-view) • true MoD - full user control, VCR capabilities, bidirectional

  17. Television (Broadcast) time channels sender receiver(s) • analog or digital • traditionally, one program per channel • analog use frequency division multiplexing only • digital may additionally use time division multiplexing inside one frequency (several programs per channel) • linear movies

  18. Near Video-on-Demand (NVoD) time channels sender receiver(s) • analog or digital broadcasting • one program over multiple channels • time-slotted emission of the program • linear movies

  19. (True) Video-on-Demand (VoD) time movies sender receiver(s) • digital uni- or multicasting • control channels • linear movies

  20. Comparison: NVoD vs. TVoD

  21. “Interactive Vision” time movie sender receiver(s) • digital uni- or multicasting • control channels • fixed non-linear movies

  22. “Cyber Vision” time sender receiver(s) • digital uni- or multicasting • control channels • variable non-linear “movies”, e.g., • - games, virtual reality, …

  23. Application Classification Overview Cyber Vision variable Interactive Vision movie structure branched HDTV VoD digital TV NVoD linear presentationform unidirectional VIDEO interaction analog bidirectional

  24. Media (Video) on Demand

  25. Challenges • VoD goals: • network-based distribution of media content to consumers • bring control to users • VoD in LANs is solved: OVERPROVISIONING works • established in studio business • established in small area (hotel/hospital/plane/…) businesses Network

  26. Challenges • VoD in WANs • assumptions: • overprovisioning of resources will NOT work • no central control of delivery system • programs: • need for interoperability – not from a single source • need for co-operative distribution systems • amount of data: • estimated 65000 movies made in 1995  260 TB MPEG-2 data • additionally, data from TV-series, sport clips, news, … • historically: • much attention as “interactive TV (ITV)” some years ago • many (not successful) field trials • now: interest turned to Internet-based systems

  27. ITV Network Architecture Approaches • WAN backbones • SONET • ATM • Local distribution network • ADSL (asymmetric digital subscriber line) • FTTC (fiber to the curb) • FTTH (fiber to the home) • HFC (hybrid fiber coax) (=cable modem) • E-PON (Ethernet passive optical network) • … • Internet WAN • Diffserv over MPLS (multi-protocol layer switching) • point-to-point Gbps ethernet • Internet local • IP over the old distribution networks ATM / SONETbackbone network wireless Internet based systems ADSL telephone cable

  28. Concerns: Internet-Based VoD Systems • Can technical problems be mastered? • broadband communication to every home • user-friendly end systems • server technology • Market success? • what prices will consumers accept? • high equipment (HW & SW) costs • data costs • will it be competitive? • to existing TV programs • to video rentals • what is the consequence of ITV field trials in USA and Europe? • no big success • only a few private consumers willing to participate • trials were cancelled or switched from TV-based to PC-based platforms

  29. Driving Forces • Hardware/software (IT) companies • computer (e.g., IBM, HP, Sun, Microsoft, …) • consumer electronics (e.g., Sony, Philips, …) • Network companies • telephone (e.g., Telenor, Telia, BT, AT&T, …) • cable TV (e.g., UPC, Time-Warner Cable, …) • Content companies • media • movies (e.g., Time-Warner, Disney, …) • TV programs (e.g., NRK, TV2, TV3, …) • hyper media information bases (e.g., Springer, Bertelsmann, …) • home-shopping (e.g., Amazon, Ebay, …) • video games (e.g., Nintendo, Sega, Microsoft, …)

  30. VoD Deployment Status – I • Digital Video Broadcast (DVB) • no VoD • cable, antenna, or satellite broadcast • some NVoD scheduling approaches (e.g., TV 1000) • Digital Audio-Visual Council (DAVIC) • defines interfaces only • no standardization of algorithms for interoperation • closed project • Internet Engineering Task Force (IETF) • currently no large-scale, wide-area video distribution • sporadic use of cooperative web caching • starting AV caching considerations • defines protocols and inspires interoperability testing Broadcast world suited for Near VoD Broadcast world suited for True VoDInternet world suited for True VoD MPEG 4 suited for Interactive Vision

  31. VoD Deployment Status – II

  32. VoD System Architecture local distribution network backbone network local distribution network local distribution network

  33. master servers regional servers local servers end-systems VoD Storage Hierarchy completeness of available content • Use hierarchies • Popularity of movies:not all are equally popular – most request directed to only a few (Zipf distribution) • Straight forward hierarchy: • popular videos replicated and kept close to clients • locality vs. communication vs. server costs

  34. VoD Components • Servers • Networks • backbone • local networks • Intermediate nodes • routers • proxy cache servers • End-systems • PCs • TV sets with set-top boxes

  35. Traditional Server Machine Internals

  36. application file system communication system General Operating System Structure and Retrieval Data Path user space kernel space

  37. Example: Intel Hub Architecture (850 Chipset) – I Intel D850MD Motherboard: RDRAM connectors CPU socket RDRAM interface system bus hub interface PCI bus Memory Controller Hub I/O Controller Hub PCI connectors

  38. application file system communication system disk network card I/O controller hub memory controller hub file system communication system application network card disk Example: Intel Hub Architecture (850 Chipset) – II Note:these transfers only show data movement between sub-systems and not the commands themselves. Additionally, data touching operations within a sub-system will require that data is moved from memory and to the CPU, e.g.: - checksum calculation - encryption - data encoding - forward error correction Pentium 4 Processor registers cache(s) system bus (64-bit, 400/533 MHz~24-32 Gbps) RDRAM RDRAM RAM interface (two 64-bit, 200 MHz ~24 Gbps) RDRAM RDRAM hub interface (four 8-bit, 66 MHz 2 Gbps) PCI slots PCI slots PCI bus (32-bit, 33 MHz 1 Gbps) PCI slots

  39. application CPU CPU file system communication system L1 L1 disk network card (four 64-bit, 400 MHz  ~95 Gbps) (eight 32-bit, 400 MHz  ~95 Gbps) (four 64-bit, 400 MHz  ~95 Gbps) RAM RAM RAM PCI busses (32/64-bit, 33/66 MHz  1-4 Gbps) GX bus (two 32-bit, 600 MHz  ~35 Gbps) RIO bus (two 8-bit, 500 MHz  ~7 Gbps) Example: IBM POWER 4 POWER 4 chip Note:Again, data touching operations add movement operations core interface switch L2 fabric controller file system L3 memory controller GXcontroller L3 controller communication system application network card PCI slots remote I/O(RIO) bridge PCI host bridge PCI-PCI bridge disk PCI slots PCI host bridge PCI-PCI bridge

  40. application file system communication system disk network card Example: AMD Opteron & Intel Xeon MP 4P servers • Know your hardware – different configuration may have different bottlenecks

  41. Server Internals • Data retrieval from disk and push to network • buffer requirements • bus transfers • CPU usage • concurrent streams (can be merged??) • storage (disk) system: • scheduling – ensure that data is available in time • block placement – contiguous, interleaving, striping • … • Stable operations: • redundant HW • multiple nodes • Much more, e.g., caching/prefetching, admission control, …

  42. Video Server Structure

  43. Internal content directory: External content directory: IBM TigerShark HP, DEC, Novell, … HP, DEC, Novell, …. switched network network attachment switched network switch switch content directory/memory management switch switch switch switch file system switch switch switch storage management IBM TigerShark switch switch controller switch switch storage device Video Server:Server Components & Switches [Tetzlaff & Flynn 94] IP, … RPC in application, … NFS, … AFS, CODA, … distributed OS, … Disk arrays (RAID), …

  44. data control Video Server:Simple General Server Architecture [Sitaram & Dan 00] • Storage subsystem: • stores data • different devices • Processor subsystem: • executing part • management and operations • Network subsystem: • transmit MM data storagesubsystem processor subsystem network subsystem control server data server clients application server • Data server: • data delivery • “specialized file system” • buffer manager • data importer/exporter • Application server: • user interface • billing • content database • user database • service gateways • Control server: • administrator • admission control • optimization  A similar architecture is used in SGI/Kassena Media Base & IBM VideoCharger

  45. data data Network Network content content Video Server:Directory Access & Data Retrieval • Two-step retrieval: • “challenge”:resource management • Request redirection: • “challenge”:client gets data from another machine e.g., Real-time Transport Protocol (RTP),maybe RTSP give permission e.g., Hypertext Transfer Protocol (HTTP), Session Initiation Protocol (SIP),maybe Real-Time Streaming Protocol (RTSP) e.g., TCP handoveror handoff

  46. server server client client Video Server:Directory Access & Data Retrieval • Pull model: • client sends several requests • deliver only small part of data • fine-grained client control • favors high interactivity • suited for editing, searching, etc. • Push model • client sends one request • streaming delivery • favors capacity planning • suited for retrieval, download, playback, etc. e.g., Priority Progress Streaming (PPS) e.g., HTTP, File Transfer Protocol (FTP),RTP + RTSP

  47. Single server easy to implement scales poorly Partitioned server users divided into groups content : assumes equal groups location : store all data on all servers load imbalance Network Network Network Video Server:Server Topology – I

  48. Externally switched servers use network to make server pool manages load imbalance(control server directs requests) still data replication problems (control server doesn’t need to be a physical box - distributed process) Fully switched server server pool storage device pool additional hardware costs e.g., Oracle, Intel, IBM data Network data data control data I/O switch Network data control Video Server:Server Topology – II

  49. Video Distribution Server:Typical In the Internet Today • Push systems(pull in video editing/database systems) • Traditional (specialized) file systems – not databases – for data storage • No in-band control (control and data information in separate streams) • External directory services for data location(control server + data pump) • Request redirection for access control • Single stand-alone servers  (fully) switched servers

  50. Server Examples

More Related