Tema 0: Transmisión de Datos Multimedia

1. Computer Networking: A Top Down Approach Featuring the Internet, 3rd edition. Jim Kurose, Keith RossAddison-Wesley, July 2004. Tema 0: Transmisión de Datos Multimedia Clases de aplicaciones multimedia Redes basadas en IP y QoS

2. What is multimedia? • Definition of multimedia • Hard to find a clear-cut definition • In general, multimedia is an integration of text, graphics, still and moving images, animation, sounds, and any other medium where every type of information can be represented, stored, transmitted and processed digitally • Characteristics of multimedia • Digital – key concept • Integration of multiple media type, usually including video or/and audio • May be interactive or non-interactive

3. Various Media Types • Text, Graphics, image, video, animation, sound, etc. • Classifications of various media types • Captured vs. synthesized media • Captured media (natural) : information captured from the real world • Example: still image, video, audio • Synthesized media (artificial) : information synthesize by the computer • Example: text, graphics, animation • Discrete vs. continuous media • Discrete media: space-based, media involve the space dimension only • Text, Image, Graphics • Continuous media: time-based, media involves both the space and the time dimension • Video, Sound, Animation

4. Sound Video Animation Continuous Continuous Graphics Text Image Discrete Discrete Captured From real world Synthesized By computer Classification of Media Type

5. Text • Plain text • Unformatted • Characters coded in binary form • ASCII code • All characters have the same style and font • Rich text • Formatted • Contains format information besides codes for characters • No predominant standards • Characters of various size, shape and style, e.g. bold, colorful

6. Plain Text vs. Rich Text An example of Plain text Example of Rich text

7. Graphics • Revisable document that retains structural information • Consists of objects such as lines, curves, circles, etc • Usually generated by graphic editor of computer programs Example of graphics (FIG file)

8. Digital still image Computer software Synthesized image Capture and A/D conversion Scanned image Images • 2D matrix consisting of pixels • Pixel—smallest element of resolution of the image • One pixel is represented by a number of bits • Pixel depth– the number of bits available to code the pixel • Have no structural information • Two categories: scanned vs. synthesized still image Camera

9. Gray-scale image color image Binary image Images (cont.) • Examples of images • Binary image – pixel depth 1 • Gray-scale – pixel depth 8 • Color image – pixel depth 24

10. Video vs. Animation • Both images and graphics can be displayed as a succession of view which create an impression of movement • Video – moving images or moving pictures • Captured or Synthesized • Consists of a series of bitmap images Each image is called a frame Frame rate: the speed to playback the video (frame per second) • Animation – moving graphics • Generated by computer program (animation authoring tools) • Consists of a set of objects • The movements of the objects are calculated and the view is updated at playback

11. Sound • 1-D time-based signal • Speech vs. non-speech sound • Speech – supports spoken language and has a semantic content • Non-speech – does not convey semantics in general • Natural vs. structured sound • Natural sound – Recorded/generated sound wave represented as digital signal • Example: Audio in CD, WAV files • Structured sound – Synthesize sound in a symbolic way • Example: MIDI file

12. Networked Multimedia • Local vs. networked multimedia • Local: storage and presentation of multimedia information in standalone computers • Sample applications: DVD • Networked: involve transmission and distribution of multimedia information on the network • Sample applications: videoconferencing, web video broadcasting, multimedia Email, etc. Image server A scenario of multimedia networking Internet Video server

13. Consideration of Networked Multimedia • Requirements of multimedia applications on the network • Typically delay sensitive • end-to-end delay • delay jitter: • Jitter is the variability of packet delays within the same packet stream • Quality requirement • Satisfactory quality of media presentation • Synchronization requirement • Continuous requirement (no jerky video/audio) • Can tolerant some degree of information loss • Challenges of multimedia networking • Conflict between media size and bandwidth limit of the network • Conflict between the user requirement of multimedia application and the best-effort network • How to meet different requirements of different users?

14. Technologies of Multimedia Networking • Media compression – reduce the data volume Address the1st challenge • Image compression • Video compression • Audio compression • Multimedia transmission technology Address the 2nd and 3rd challenges • Protocols for real-time transmission • Rate / congestion control • Error control

15. Multimedia Networking Systems • Live media transmission system • Capture, compress, and transmit the media on the fly (example?) • Send stored media across the network • Media is pre-compressed and stored at the server. This system delivers the stored media to one or multiple receivers. (example?) • Differences between the two systems • For live media delivery: • Real-time media capture, need hardware support • Real-time compression– speed is important • Compression procedure can be adjusted based on network conditions • For stored media delivery • Offline compression – better compression result is important • Compression can not be adjusted during transmission

16. Classes of multimedia applications • Streaming stored audio and video • Streaming live audio and video • Real-time interactive audio and video

17. 2. video sent 3. video received, played out at client 1. video recorded network delay streaming: at this time, client playing out early part of video, while server still sending later part of video Streaming Stored Multimedia: What is it? t>0 100% Cumulative data time

18. Streaming vs. Download of Stored Multimedia Content • Download: Receive entire content before playback begins • High “start-up” delay as media file can be large • ~ 4GB for a 2 hour MPEG II movie • Streaming: Play the media file while it is being received • Reasonable “start-up” delays • Reception Rate >= playback rate. Why?

19. Streaming Stored Multimedia: Interactivity • VCR-like functionality: client can pause, rewind, FF, push slider bar • 10 sec initial delay OK • 1-2 sec until command effect OK • RTSP often used (more later) timing constraint for still-to-be transmitted data: in time for playout

20. client video reception constant bit rate video playout at client variable network delay buffered video client playout delay Streaming Multimedia: Client Buffering • Client-side buffering, playout delay compensate for network-added delay, delay jitter constant bit rate video transmission Cumulative data time

21. Streaming Multimedia: Client Buffering • Client-side buffering, playout delay compensate for network-added delay, delay jitter constant drain rate, d variable fill rate, x(t) buffered video

22. Interactive, Real-Time Multimedia applications: IP telephony, video conference, distributed interactive worlds • end-end delay requirements: • audio: < 150 msec good, < 400 msec OK • includes application-level (packetization) and network delays • higher delays noticeable, impair interactivity • session initialization • how does callee advertise its IP address, port number, encoding algorithms?

23. Internet multimedia: simplest approach • audio or video stored in file • files transferred as HTTP object • received in entirety at client • then passed to player audio, video not streamed: • no, “pipelining,” long delays until playout!

24. Progressive Download • browser GETs metafile • browser launches player, passing metafile • player contacts server • server downloads audio/video to player

25. Streaming from a streaming server • This architecture allows for non-HTTP protocol between server and media player • Can also use UDP instead of TCP.

26. Multimedia Over Today’s Internet • TCP/UDP/IP: “best-effort service” • no guarantees on delay, loss • But multimedia apps requires QoS and level of performance to be effective! • Today’s Internet multimedia applications use application-level techniques to mitigate (as best possible) effects of delay, loss

27. Streaming Multimedia: UDP or TCP? UDP • server sends at rate appropriate for client (oblivious to network congestion!) • often send rate = encoding rate = constant rate • then, fill rate = constant rate - packet loss • short playout delay (2-5 seconds) to compensate for network delay jitter • error recover: time permitting TCP • send at maximum possible rate under TCP • fill rate fluctuates due to TCP congestion control • larger playout delay: smooth TCP delivery rate • HTTP/TCP passes more easily through firewalls

28. QoS network provides application with level of performance needed for application to function. Multimedia, Quality of Service: What is it? Multimedia applications: network audio and video (“continuous media”)

29. Improving QOS in IP Networks • Thus far: “making the best of best effort” • Future: next generation Internet with QoS guarantees • RSVP: signaling for resource reservations • Differentiated Services: differential guarantees • Integrated Services: firm guarantees • simple model for sharing and congestion studies:

30. Principles for QOS Guarantees • Example: 1Mbps IPphone, FTP share 1.5 Mbps link. • bursts of FTP can congest router, cause audio loss • want to give priority to audio over FTP Principle 1 packet marking needed for router to distinguish between different classes; and new router policy to treat packets accordingly

31. Principles for QOS Guarantees (more) • what if applications misbehave (audio sends higher than declared rate) • policing: force source adherence to bandwidth allocations • marking and policing at network edge: • similar to ATM UNI (User Network Interface) Principle 2 provide protection (isolation) for one class from others

32. Principles for QOS Guarantees (more) • Allocating fixed (non-sharable) bandwidth to flow: inefficient use of bandwidth if flows doesn’t use its allocation Principle 3 While providing isolation, it is desirable to use resources as efficiently as possible

33. Principles for QOS Guarantees (more) • Basic fact of life: can not support traffic demands beyond link capacity Principle 4 Call Admission: flow declares its needs, network may block call (e.g., busy signal) if it cannot meet needs

34. Summary of QoS Principles