Multimedia Systems Lecture 10 – Media Servers and File Management

1. Multimedia Systems Lecture 10 – Media Servers and File Management

2. Buffering Strategies in Client-Server Systems Read encoded frames from VOD Disk Packetization Protocol Stack Processing network VOD Server VOD Client De-Packetization Protocol Stack Processing playout Decoder

3. Client/Server Video-on-Demand System

4. Memory Management • Virtual memory versus real memory paradigm. • In VM – paging introduces unpredictable delays. • Multimedia timing requirements suggest that no paging is desired. • Multimedia applications may want to pin pages into memory which include their time-sensitive code.

5. Memory Reservation Concept • Admitted storage – memory size from the global reserve that has been already allocated to accepted processes. • Available storage – free memory from global reserve that is available for new RT processes. • GlobalReserve:= AdmittedStorage + AvailableStorage;

6. Reservation Protocol • Step 1: SRT (Soft Real-Time) Process declares ‘RequestedStorage’ to Memory Broker. • Step 2: Admission Control performed in broker. • Step 3: If Admission positive, broker builds memory reserve of size ‘RequestedStorage’, allocated to SRT process during time critical run. • Step 4: Memory Broker • provides reservation ID to SRT process, • creates entry in reservation table, and • informs memory controller

7. Reservation Protocol (cont.) • Step 5: SRT process uses its reservation ID during execution to get memory allocated from memory controller. • Step 6: Memory controller • checks the validity of the reservation, and if positive, • allocates shared memory to SRT from global reserve pool; • gives key of shared segment to application which then attaches the shared memory segment to its own address space

8. Reservation Protocol (cont.) • Step 7: After SRT finishes, it • Frees shared memory segment from its application space, • Sends an event to memory controller with key to shared memory • Step 8: Memory controller • Frees shared memory • Increases available storage information for broker • AvailableStorage: = AvailableStorage + FreeStorage

9. Servers Classification • Media Servers • Push Servers – file servers with streaming model • Servers push data towards users • Traditional File Servers • Pull Servers - FTP servers • Users pull data in one block at a time by repeatedly calling read to get one block after another • RPC (Remote Procedure Calls)

10. Media Server Requirements • Real-time storage and retrieval • Media quanta must be presented using the same timing sequence with which they were captured. • High-Data Transfer Rate and Large Storage Space • HDTV quality: 1280x720 pixels/frame; 24 bits/pixel -> 81 Mbytes per second • NTCS quality: 640x480 pixels/frame; 24 bits/pixel ->27MBytes per seconds

11. Playback • Single Stream Playback • Possible approach – buffer the whole stream • Possible approach – prefetch just short video part • Problem: • Prevent starvation • Minimize buffer space requirement • Minimize initiation latency

12. Playback (cont.) • Multiple Streams Playback • Possible approach – dedicate a disk to each stream • Possible approach – multiple streams per disk

13. Support for Continuous Media • Proper management of multimedia disk storage • Optimal placement of data blocks on disk • Usage of multiple disks • Role of tertiary storage • Special disk scheduling algorithms and sufficient buffers to avoid jitter

14. Media Server Architecture Delivered data Incoming request Network Attachment Content Directory Memory Management File System Storage management Disk controller Storage device

15. Disk Layout Zoned Disk (ZBR – Zone Bit Recording) Traditional Random Access Disk Layout Track Sector Advantage: Sector size same Rotation speed constant; efficient Usage of space Advantage: Easy mapping of location Information to head movement and disk rotation Problem: loss of storage space

16. ZBR Disk Design • Stores more sectors per track on outer tracks than on inner tracks. • Is called also Zone Constant Angular Velocity (ZCAV). • Place most popular movies in disk outer zones and aggregate “hot” movies together to take advantage of high bandwidth in outer zones. • ZBR’s drive controller varies rate at which it reads and writes – faster on outer tracks. • Products that use ZBR • HD DVD-RW • Most hard drives since 1990s

17. Storage Management • Storage access time to read/write disk block is determined by 3 components: • Seek Time • Time required for the movement of read/write head. • Rotational Time (Latency Time) • Time during which transfer cannot proceed until the right block or sector rotates under read/write head. • Data Transfer Time • Time needed for data to copy from disk into main memory

18. Placement of MM Data Blocks on Single Disk

19. Intra-file Seek Time • Intra-file seek – can be avoided in scattered layout if the amount read from a stream always evenly divides block . • Solution: select sufficient large block and read one block in each round • If more than one block is required to prevent starvation prior to next read, deal with intra-file seek

20. Placement of Multiple MM Files on Single Disk • Popularity concept among multimedia content . • Take popularity into account when placing movies on disk. • Model of popularity distribution – Zipf’s Law • Movies are kth ranked • if their probability of customer usage is C/k, • C = normalization factor • Condition holds: C/1 + C/2 + … C/N = 1, • N is number of customers

21. Example • Assume N = 5 movies • Problem: what is the probability that the next customer picks 3rd ranked movie? • Solution: • Solve C from the equation • C/1 + C/2 + C/3 + C/4 + C/5 = 1 • C * (1+1/2+1/3+1/4+1/5) =1 • C = 0.437 • Probability to pick 3rd ranked movie is C/3 = 0.437/3 = 0.1456

22. Need for Multiple Disks Solutions for Media Server • Limitation of Single Disk: Disk Throughput • Approach: 1 Maintain multiple copies of the same file on different disks • Very expensive • Approach 2: Spread multimedia file across multiple disks

23. Storage/Disk Management • Disk access – slow and costly. • Reduce disk access • Use block caches (anticipate future reads or writes) • Reduce disk arm motion • Blocks accesses in sequence (continuously) , place together on one cylinder • Interleaved vs. non-interleaved storage

24. Data Interleaving Data Interleaving On single disk (consecutive blocks are placed on The same cylinder But in interleaved way Data Interleaving On Multiple Disks (Disks are not Synchronized)

25. Disk Scheduling Policies • Goal of Scheduling in Traditional Disk Management • Reduce cost of seek time • Achieve high throughput • Provide fair disk access • Goal of Scheduling in Multimedia Disk Management • Meet deadline of all time-critical tasks • Keep necessary buffer requirements low • Serve many streams concurrently • Find balance between time constraints and efficiency

26. EDF (Earliest Deadline First) Disk Scheduling • Each disk block request is tagged with deadline • Policy: • Schedule disk block request with earliest deadline • Excessive seek time – high overhead • Pure EDF must be adapted or combined with file system strategies

27. SCAN-EDF Scheduling Algorithm • Combination of SCAN and EDF algorithms. • Each disk block request tagged with augmented deadline • Add to each deadline perturbation • Policy: • SCAN-EDF chooses the earliest deadline • If requests with same deadline, then choose request according to scan direction

28. Mixed Scheduling SSTF (Shortest Seek Time First) + Balanced Strategy

29. Multimedia File Systems • Real-time Characteristics • Read operation must be executed before well-defined deadline with small jitter • Additional buffers smooth data • File Size • Can be very large even those compressed • Files larger than 232 bytes • Multiple Correlated Data Streams • Retrieval of a movie requires processing and synchronization of audio and video streams

30. Placement of Mapping Tables • Fundamental Issue: keep track of which disk blocks belong to each file. • For continuous files/contiguous placement • don’t need maps • For scattered files • Need maps • Linked lists (inefficient for multimedia files) • File allocation tables (FAT)

31. Indexing and FAT Higher Level Index Table Per File File Allocation Table Block I1 Location PTR I Frame Block I2 Location PTR P Frame Block I3 Location PTR B Frame Block P11 Location PTR P Frame Block P12 Location PTR ……….. Block B1 Location PTR Block P21 Location PTR Block P22 Location PTR …………..

32. Constant and Real-time Retrieval of MM Data • Retrieve index in real-time. • Retrieve block information from FAT. • Retrieve data from disk in real-time . • Real-time playback • Implement linked list • Random seek (Fast Forward, Rewind) • Implement indexing • MM File Maps • include metadata about MM objects: creator of video, sync info

33. Fast Forward and Rewind • Play back media at higher rate • Not practical solution • Continue playback at normal rate, but skip frames • Define skip steps, (e.g. skip every 3rd, or 5th frame) • Be careful about interdependencies within MPEG frames • Approaches for FF: • Create a separate and highly compressed file • Categorize each frame as relevant or irrelevant • Intelligent arrangement of blocks for FF

34. Block Size Issues in File Organization • Small Block Sizes • Use smaller block sizes, smaller than average frame size • Organization Strategy: Constant Time Length • Need Metadata structure, called Frame Index • Frame means a time frame within a movie • Under the time frame read all blocks (audio, video, text) belonging to this time frame Movie Time line Frame index ……… A A A V V V V T T V V

35. Block Size Issues • Large Block Size • Use large blocks (e.g., 256 KB) which include multiple audio/video/text frames • Organization Strategy: Constant Data Length • Need Metadata structure, called Block Index • Each block contains multiple movie frames

36. Tradeoffs • Frame index : needs large RAM usage while movie is playing, however little disk wastage. • Block index (if frames are not split across blocks): need low RAM usage, but major disk wastage. • Block index(if frames are split across blocks): need low Ram usage, no disk wastage, extra seek times

37. The End