Multimedia Storage Servers: A Tutorial

1. Multimedia Storage Servers: A Tutorial D. James Gemmell, Harrick M. Vin, Dilip D. Kandlur, P. Venkat Rangan, Lawrence A. Rowe (IEEE Computer, May 1995) Presented by Hoang Nguyen CS59KN – 2nd Presentation

2. Introduction & Motivation Server Client Buffers Buffers Graphics/video hardware Buffers Buffers Audio hardware Storage Network

3. Requirements • Real-time Storage and Retrieval • Recording • CM recording devices generate continuous stream of media quanta that must be stored in realtime. • Playback - Reverse operation of recording • Media Quanta must be presented using the same timing sequence with which they were captured. • High Data Transfer Rate and Large Storage Space • HDTV quality - 81Mbytes/sec • NTSC quality - 27Mbytes/sec

4. Continuous Media Recording and Playback Quanta Read from server disk Quanta Buffered Quanta Consumed (data must be displayed in a smooth and timely fashion) Time Server Challenge - to ensure continuous retrieval of media stream

5. Playback • Single Stream Playback • Continuous playback assured by buffering entire stream before playback - Inefficient !!! • To efficiently service a single stream, solve 3 problems • Prevent Starvation • Minimize Buffer Space Requirement • Minimize initiation latency • Multistream Playback - Server must process • Retrieval requests for several streams simultaneously. Possible solutions: • dedicate a disk head to each stream - inefficient. • Multiplex streams per disk • Multiple retrieval requests for the same stream

6. Supporting Continuous Media • Goal - Constant and timely retrieval of data • Approaches: • Special disk scheduling algorithms • Buffering • Avoid starvation and jitter • For maximum consumption • Admission control • Proper management of multimedia disk storage • results in optimized organization (placement) of multimedia files on disk

7. Outline • Goal - Constant and timely retrieval of data • Approaches: • Special disk scheduling algorithms • Buffering • Avoid starvation and jitter • For maximum consumption • Admission control • Proper management of multimedia disk storage • Implementing Multimedia File System

8. Disk Scheduling Algorithm • Goal of scheduling in traditional filesystems • reduce cost of seek time • achieve high throughput • provide fair disk access • Goal of scheduling in MM Filesystems • meet deadlines of all time critical tasks • keep necessary buffer requirements low • serve many streams concurrently • find balance between time constraints and efficiency

9. EDF Scheduling Algorithm • Problem • poor throughput and excessive seek time • Preemptive algorithm - hence has high overhead • EDF must be combined or adapted with file system strategies 3 30 2 16 3 50 t 2 42 3 24 1 45 3 30 2 16 3 30 1 12 2 16 3 50 3 50 3 50 2 40 2 42 3 50 2 42 1 45 2 42 1 22 1 45 1 12 50 2 40 1 12 16 2 40 2 40 42 2 40 40 1 22 45 12 22 20

10. SCAN-EDF Scheduling Algorithm Deadline + N_i/Nmax • Combination of SCAN with EDF • With EDF - request with earliest deadline is served • Among requests with the same deadline, choose based on SCAN direction 3.30 30 2.16 16 3.50 50 t 2.42 42 3.30 30 1.45 45 2.16 16 3.50 50 3.50 50 1.12 12 3.50 50 2.42 42 2.42 42 2.42 42 2.40 40 1.45 45 2.42 42 2.40 40 1.12 12 2.40 40 1.22 22 2.40 40 42 1.45 45 2.40 40 16 1.22 22 40 1.22 22 45 Deadline 2 22 12 Deadline 1 Nmax = 100

11. SCAN-EDF Enhancements (N_i-N)/Nmax if (N_i  N) Head moves upwards N_i 0 N_max N_i N (Nmax-N_i)/Nmax if (N_i < N) Head moves downwards N_i/Nmax if (N_i  N) N_i 0 N_max N_i N (N-N_i)/Nmax if (N_i < N)

12. Trade-off between round-length and latency

13. Group Sweeping Algorithm • Group Sweeping Algorithm • Streams are divided into groups. • Groups are served in round-robin manner. • SCAN for each group. • Server can balance the trade-off

14. Group Sweeping Algorithm (cont) 3.4 24 3.3 30 Disk access requests in one cycle with deadline deadline blocknumbers 2.0 16 3.3 50 2.2 42 1.2 45 1.4 12 Deadline 3.3 2.4 40 Deadline 1.1 Deadline 2.0 1.1 22 2.0 16 3.4 24 1.4 12 2.2 42 3.3 30 1.2 45 2.4 40 3.3 50 1.1 22 Group 2 Group 3 Group 1 scan scan scan 42 40 16 24 30 30 12 22 45

15. Buffering • Avoid starvation and jitter • Sufficient blocks per round • Enough for longest possible round length • Minimize round length • For Maximum Consumption • Topped up strategy • Variable read amount

16. Admission Control • Deterministic Admission Control • All deadlines guaranteed to be met. • Consider worst case scenarios when admitting new streams • Low utilization • Statistical Admission Control • Deadlines guaranteed to be met with certain probability • Consider system’s statistical behavior • Background • No guarantee for meeting deadlines

17. Managing MM Storage

18. Data Placement Schemes Contiguous Placement Constrained Placement Log-structured Placement

19. RAID Controller Sector 0 Sector 1 Sector 2 Logical Sector 0 Data striping • Scatter data across multiple disks • Achieves load balance • Can use RAID (Redundant Array of Inexpensive Disks) technology • allows for parallel access to an array of disks • Disks are spindle synchronized and operate in lock-step parallel mode • Physical and logical blocks have identical access time and transfer rate increases • Striping improves bandwidth requirements, does not improve seek time or rotational latency

20. Data Interleaving • Blocks are interleaved across the disk array with successive file blocks stored on different disks • Simple interleaved pattern • stores blocks cyclically across an array of N disks • Difference to striped data • interleaved data are not spindle synchronized • Data interleaving operates independently

21. File-retrieval structure • Fundamental issue • keep track of which disk block belongs to each file and keep a map of how to travel from block to block in a file • With continuous files - contiguous placement we don’t need maps. • Scattered files need file mapping • Linked Lists - random access is highly inefficient • File Allocation Table (FAT) • each table entry maintains a pointer to the next block of a file. If FAT is in memory, random access is fast • Indexing • index a file and store an index for each file separately -

22. Hybrid solution • For real-time playback • implement a linked list • achieve random seek through indexing • overhead in keeping link and index pointers up to date

23. Fast Forward and Rewind • Approaches • Playback media at a higher rate - impractical • Playback at normal rate but skip frames • Play every 3d or 5th frame - skip step • index file useful to find blocks when skipping • can get complicated when interdependencies exist for synchronization. • Approaches for FF via data skipping • Create a separate and highly compressed file • Categorize block as relevant or irrelevant • Intelligent arrangement of blocks for fast forward

24. Comments • Pros • A good picture of MM storage server • Both concept and implementation are addressed • Cons • Some enhancements are not mentioned (e.g. Enhanced SCAN-EDF, Mixed strategy Disk Scheduling) • Caching • No citation

25. References • Multimedia Storage Servers: A Tutorial, D.J. Gemmel et al. • Multimeda System, K. Nahrstedt, R. Steinmetz

26. Mixed Scheduling Example Status of Buffers Balanced Buffers . . . . . . Buffers SSTF Disk Scheduling