Chapter 19/20: Real-time and Multimedia Systems

1. Chapter 19/20: Real-time and Multimedia Systems Inclusions from Tanenbaum, Boss, Modern Operating Systems, 2014

2. Multimedia and Real-Time Systems • Multimedia files • Compression • Requirements of Multimedia Kernels • CPU Scheduling • Disk Scheduling • Network Management

3. What is Multimedia? • Document or file containing two or more continuous media. • Media that must be accessed and played back at a specific rate or over a specific time interval.

4. What is Multimedia • Multimedia data is stored in the file system like other ordinary data • Key characteristics of multimedia: • Multimedia files can be large! • 100 minute movie (mpeg) requires 1.125 GB • Multimedia uses high data transfer rates • Comes from the nature of visual and acoustic information • Eye and ear can process prodigious amounts of information and must be fed at a constant rate. • Multimedia requires real-time playback • Once a file is delivered to a client, it must continue at a certain rate of playback.

5. Definitions: Streaming • Streaming is delivering a multimedia file from a server to a client - typically the deliver occurs over a network connection • There are two different types of streaming: 1. Progressive download - the client begins playback of the multimedia file as it is delivered. The file is ultimately stored on the client computer 2. Real-time streaming - the multimedia file is delivered to - but not stored on - the client’s computer

6. Definitions: Real-time Streaming • There are two types of real-time streaming: (1) Live streaming - used to deliver a live event while it is occurring (2) On-demand streaming - used to deliver media streams such as movies, archived lectures, etc. The events are not delivered in real-time

7. Multimedia Operating System Issues The operating system must guarantee the specific data rate and timing requirements of continuous media QOS : Quality of Service Guarantee specific rate and timing requirements of media Guarantee clear picture/voice synchronization Influences OS tasks such as • Compression and decoding may require significant CPU • Disk Scheduling: File systems must be efficient to meet rate requirements • CPU Scheduling: multimedia task must be scheduled with certain priorities to ensure meeting deadline requirements of continuous media. • Network protocols must support bandwidth requirements

8. Multimedia Files • Multimedia files are… more than one…

9. Multimedia Files : Video • For example, video must be displayed at 24-30 frames per second. Multimedia video data must be delivered at a rate which guarantees 24-30 frames/second. Experiments show that people notice a flicker less than 24 frames / second SO, to increase frame rate Progressive technique - use more bandwidth to send more frames Interlacing: (analog) - show odd scan lines then show even ones

10. Multimedia Files : Video

11. Multimedia Files : Video • For example, video must be displayed at 24-30 frames per second. Multimedia video data must be delivered at a rate which guarantees 24-30 frames/second. Experiments show that people notice a flicker less than 24 frames / second SO, to increase frame rate Progressive technique - use more bandwidth to send more frames Interlacing: (analog) - show odd scan lines then show even ones • Repaint the screen (digital)

12. Multimedia Files : Audio Audio waves are converted to binary by quantizing the sample of the sine wave

13. Multimedia Files: Compression Multimedia files are big. • Because of the size and rate requirements of multimedia systems, multimedia files are often compressed into a smaller form • Compression requires 2 algorithms: encoding - decoding • JPEG – (Joint Photographic Experts Group) • Quantizes blocks of the picture so that each shows the amount by which it differs from the corresponding element in the previous block. • Linearizes the elements to produces the bit string file.

14. Multimedia Files: Compression MPEG – (Motion Picture Experts Group) • 3 different kinds of frames • I (Intra-coded) frames: Self-contained JPEG pictures • P (predictive) frames: • Block by block differences with last frame • B (Bidirectional) frames: • Differences from the last and next frame.

15. Multimedia Files • Multimedia files are… more than one…

16. Multimedia Files : Video Definitions: Flicker is determined by the number of times the screen is painted/sec. A still image painted 20 frames/sec will not show jerky motion but it will flicker because the image will decay from the retina. Smoothness of motion is determined by the number of different images /sec A movie with 20 different frames/sec, each painted 4x in a row will not flicker, but the motion will appear jerky

17. Multimedia File Systems • File placement • Contiguous Interleave video, audio and text in a single contiguous file per movie • Block model

18. Multimedia File Systems • Use a different paradigm than traditional file systems • Traditional: open > read (block) > read (unrelated block) … doesn’t work Pull Server Push Server • Predictable next buffer • Timing deadlines • Rate requirements

19. Disk Scheduling • Disk scheduling algorithms must be optimized to meet the timing deadlines and rate requirements of continuous media • Earliest-Deadline-First (EDF) Scheduling • SCAN-EDF Scheduling

20. Disk Scheduling (Cont) • The EDF scheduler uses a queue to order requests according to the time it must be completed (its deadline)

21. Disk Scheduling (Cont) • SCAN-EDF scheduling is similar to EDF except that requests with the same deadline are ordered according to a SCAN policy

22. Deadline and cylinder requests for SCAN-EDF scheduling

23. CPU Scheduling • Multimedia systems require hard realtime scheduling to ensure critical tasks will be serviced within timing deadlines • Most hard realtime CPU scheduling algorithms assign realtime processes static priorities that do not change over time

24. Requirement of Multimedia Operating Systems • There are three levels of QoS (1) Best-effort service - the system makes a best effort with no QoS guarantees (2) Soft QoS - allows different traffic streams to be prioritized, however no QoS guarantees are made (3) Hard QoS - the QoS rquirements are guaranteed

25. Further QoS Issues • Different processes must run at different frequencies with different amounts of work with different deadlines • Scheduling of multiple competing processes, some or all with different deadlines to be met  real-time scheduling

26. Real-Time CPU Scheduling • Periodic processes require the CPU at specified intervals (periods) • p is the duration of the period • d is the deadline by when the process must be serviced • t is the processing time

27. 3 periodic processes

28. Multimedia Disk Scheduling

29. Rate Montonic Scheduling • A priority is assigned based on the inverse of its period • Shorter periods = higher priority; • Longer periods = lower priority • P1 is assigned a higher priority than P2.

30. Earliest Deadline First Scheduling • Priorities are assigned according to deadlines:the earlier the deadline, the higher the priority; the later the deadline, the lower the priority

31. Network Management • Three general methods for delivering content from a server to a client across a network: (1) Unicasting - the server delivers the content to a single client (2) Broadcasting - the server delivers the content to all clients, regardless whether they want the content or not (3) Multicasting - the server delivers the content to a group of receivers who indicate they wish to receive the content

32. RealTime Streaming Protocol (RTSP) • Standard HTTP is stateless whereby the server does not maintain the status of its connection with the client

33. Streaming media from a conventional web server

34. Realtime Streaming Protocol

35. RTSP States • SETUP - the server allocates resources for a client session • PLAY - the server delivers a stream to a client session • PAUSE - the server suspends delivery of a stream • TEARDOWN - the server breaks down the connection and releases the resources allocated for the session

36. RTSP state machine

37. End of Chapter 20

39. Exercise 20.10