CGMB324: multimedia system design

1. CGMB324: multimedia system design Chapter 10: Multimedia Hardware, Storage Devices, Software & Applications

2. Objectives Upon completing this chapter, you should be able to: • understand the important requirements of a multimedia workstation • identify the common input and output technologies related to multimedia • discuss the concept related to optical storage media • identify important characteristics of a multimedia operating system • identify the common types of multimedia software and applications

3. Multimedia Hardware • Multimedia Workstation Requirements • Processor – distribute the work among different processors • Bus – must have dedicated, synchronized bus for continuous multimedia data • Multimedia devices – for I/O of multimedia data

4. Multimedia Hardware • Primary storage – for copying operation • Secondary storage – to balance between high storage density, low access time • Operating system – distinguish between processes for continuous and discrete computation

5. Multimedia Hardware • Input / output technologies • Pen input • Video and image display systems • Print output • Image scanners • Digital voice and audio capture • Video images and animation • Digital camera

6. Multimedia Storage Requirements • MM systems require storage for large capacity objects such as video, audio, animation and images. • Depending on the compression scheme, video bandwidth requirements range from 1.5 Mbit/s (MPEG-1) to 10 Mbit/s (MPEG-2) and up to 1 Gbit/s (HDTV)!! • Depending on the fidelity and compression, audio may consume from 8 Kb/s  all the way up to 384 Kb/s. • In applications that require multiple multimedia streams, these numbers increase accordingly.

7. Multimedia Storage Requirements • Some of the major distinguishing characteristics of multimedia objects are: • multimedia objects are large (large file sizes), • multimedia streams have stringent real-time requirements, • multimedia streams represent long and fairly constant (predictable) loads, • multimedia streams usually have high bandwidth requirements, • multimedia objects tend to exhibit sequential access patterns within a given medium.

8. Multimedia Storage Requirements • Multimedia objects have real-time playback and, in many cases, similar recording requirements. • Video is somewhat scalable in the sense that its frame rate, and to a certain extent, its window size, may be varied to compensate for response time fluctuations without significantly perceptible adverse effects. • Audio on the other hand, has stringent timing requirements and needs virtually constant playback speed for acceptable quality.

9. Multimedia Storage Requirements • The file systems of earlier operating systems were designed to handle bursty traffic with no real-time requirements, and they usually have rather poor throughput for continuous load. • Indexing and retrieval of multimedia objects poses some very different and demanding requirements for navigation on database systems; • e.g. multimedia databases need to support searches by media content, such as video clips and image attributes, on top of processing the traditional text-based queries. • There is still much work being done on this.

10. Multimedia Storage Requirements • Mass storage devices are required to overcome the high demand for storage space for multimedia systems. • Among the mass storage devices, optical and magnetic media technology are some of the cheapest solutions as compared to internal storage devices like DRAM, SDRAM etc.

11. Usage of Magnetic Media In Multimedia • Multimedia Objects • Volatile Objects – audio and video input. They are sometimes not retrieved from storage but directly captured from the source. • Nonvolatile Objects – stored in a storage device. Two issues arises: • Capacity • Transfer Speed / Bandwidth

12. Usage of Magnetic Media In Multimedia Note the difference between the requirements of various multimedia objects

13. Usage of Magnetic Media In Multimedia A/V (Audio/Video) Ready Magnetic Drive • Temporal requirements of some multimedia objects for sustained delivery had created a need for a device with guaranteed delivery. • The A/V ready drive was developed to address this need.

14. Usage of Magnetic Media In Multimedia • Features of A/V ready drive: • Multi Segmented Caching – different cache for different objects • Write Behind Caching and Write Coalescing – data to be written is buffered in cache first, and coalesces (blends) multiple write requests in a single disk revolution. • Tagged Command Queuing – tagging I/O request and queue them to reduce latencies.

15. Usage of Magnetic Media In Multimedia • Fast ECC (Error Correcting Codes)– auto-correcting soft errors. • Guaranteed minimum sustained rate: 3 Mbytes/sec • Fast Drive Speed (at least 5400 rpm) • Synchronized Spindles – Supports RAID Configurations

16. Optical Storage Media

17. General Concepts of Optical Storage Media • The optical technology is based on a light source • A sharply focused laser beam is directed onto the surface of a spinning disk • Physical variations in the surface (pits and landings) are arranged in concentric tracks of the disk • The pits and landings deflect the focused beam toward an optical receiver to signify a 1 or disperse the beam away to signify a 0 • Optical storage is normally read-only

18. General Concepts of Optical Storage Media • Locating a specific address involves • moving the head to a general area • adjusting the rotation speed • reading the address • making minor adjustment to find and access the specific sector

19. Review of Optical Storage Media • CD-DA - Compact Disk Digital Audio • Stores digitized audioinformation (1982) • CD-ROM - Compact Disk Read-Only Memory • Stores computer data (1985) • Later became the High Sierra format (ISO9660) • CD-I - Compact Disk Interactive • Stores audio, video, graphics, text and machine code on CD-ROM (1986) • DVI - Digital Video Interactive • Stores digitized, compressedrepresentation of audio/video information on CD (1987) • CD-XA – CD eXtended Architecture • Contains Multiple tracks differentiated by mode (1988) • WORM - Write-Once Read-Many • A disk which is more easily written than CD (1990s)

20. Review of Optical Storage Media • Erasable Optical Disk • Optical disk that can be easily erased and rewritten (1990s) • DVD – Digital Video/Versatile Disk (1996) • A single-layer DVD can store 4.7 GB (4.38 GiB), which is around seven times as much as a standard CD-ROM. • BD – Blu-ray disc (Blu-ray Disc Association BDA) • next-generation format for high-definition video and high-density data. • A single-layer disc can fit 23.3, 25, or 27 GB (enough for approximately four hours of high-definition video with audio) • supports 25GB for one layer, 50GB for two and 100GB for four, • HD-DVD (Toshiba) • A single layer capacity of 15 GB and a dual-layer capacity of 30 GB • Triple-layer disc is in development, which would offer 45GB of storage.

21. Optical Storage Media CD-ROM • Based on laser beam (780nm) reflection, focused at 1m area • 120mm diameter, 1.2mm thickness, 15 mm hole • track-to-track spacing 1.6 microns • Record-able width 32.55 mm • number of tracks 32,550/1.6 = 20,344 • tracks are arranged in a spiral • storage capacity about 774.57 MBytes (unformatted) • Some CD-ROMs have a formatted storage capacity of up to 870 MB, due to writing in the ‘lead-in’ / ‘lead out’ area of the disc

22. Optical Storage Media CD-ROM • The CD-ROM is derived from the original CD audio music disc. • Because audio has a constant data rate, e.g. 44100 samples per second, the datarate you get when reading the disc has to be constant as well. • A typical audio CD spins at about 500 rpm when the laser is at the middle (center) of the disc and slows to about 250 rpm when reading the outer area of the disc. • This is because the laser reads a fixed number of centimetres of data track each second (CLV – Constant Linear Velocity), and this data is packed the same way whether at the middle or edge of the disc. • A CD audio disc delivers about 150 KB/s (1x speed). • CLV is used for drives under 12x usually.

23. Optical Storage Media CD-ROM • Now, with fast CD-ROM speeds these days (e.g. 40x, 50x), reading data using CLV would require changing speeds between 20,000 rpm at the center of the disc and 10,000 rpm at the edge. • This requires too much power when searching back and forth on a disc for a file. • So, since it isn’t really crucial what speed data (unlike music) is read from a CD these days, CD-ROMs use CAV (Constant Angular Velocity). • This means the disc spins at a fixed rate and the reading data rate varies. So, if the speed was fixed at 10,000 rpm, it would only deliver 40x speed at the outer edge (more data there) and 20x near the center (less data). • This is why data that is regularly accessed (like in certain games) is usually placed at the edge of the disc, to ensure it’s read as fast as possible.

24. Optical Storage Media CD-ROM • Three major advantages • Information storage capacity is large • It can be mass-replicated cheaply • It is removable • Disadvantages • Read-only and cannot be updated • Longer access time, typically half a second

25. Optical Storage Media Erasable Optical Disk • Can be repeatedly written and overwritten • Uses magneto-optical technology • A laser beam and magnetic field is used to record and erase information • Laser beam heats a spot on the medium • While it is still hot, a magnetic field changes the orientation of that spot • Uses constant angular velocity (CAV) • Principal advantages • High capacity - 650 Mbytes • Portability - Removable • Reliability

26. Optical Storage Media Digital Video Disk (DVD) • made from a 0.6 mm thick disc of polycarbonate plastic coated with a much thinner reflective layer of aluminium or gold. • Two such discs are glued together to form a 1.2 mm disc that can be designed to be read from one side (single sided) or both sides (double sided). • can be used for data storage, including movies with high video and sound quality. • DVDs resemble compact discs as their physical dimensions are the same (12 cm or occasionally 8 cm in diameter) but they are encoded in a different format and at a much higher density. • A single-layer DVD can store 4.7 GB (4.38 GiB), which is around seven times as much as a standard CD-ROM. • DVD format is 47 % more efficient with respect to CD-ROM

27. Multimedia Operating Systems

28. Multimedia Operating Systems • Notion of real time • Real Time process = process which delivers the results of the processing in a given time span. • Dead lines • Divided into soft deadline and hard deadline. • Soft deadline can be violated without any adverse effect but not hard deadlines!! • Real time features: • Predictably fast response to critical events • High degree of resource utilization • Stability under transient (temporary) load

29. Multimedia Operating Systems • MM systems have a different set of real-time requirements: • Fault tolerance requirements are less strict • Some multimedia applications can afford to miss a deadline without any severe failure • All time critical processing are periodic rather than sporadic (irregular instances)

30. Multimedia Operating Systems • Resource Management • Resources • Can be active or passive • Exclusive or shared • Single or multiple instance • Multimedia requirements • Throughput determined by the needed data rate • Local and global (end-to-end) delay • Jitter – maximum variance • Reliability – error detection and correction (QoS – quality of service)

31. Multimedia Operating Systems • Resource Management • Components and phases • Client makes a reservation request to the resource manager component. • Resource manager will see whether the request can be guaranteed or not • Allocation schemes • Pessimistic approach (reservations for worst case) • Optimistic approach (for average workload)

32. Multimedia Operating Systems • Process Management • Real time processing requirements for scheduling multimedia tasks • two conflicting goals must be satisfied • Uncritical process should not starve • Time critical process should never be subject to priority inversion. • File System Management • MM systems tends to have more continuous data than discrete data, which are different in terms of: • Real time characteristics (time dependent delivery) • Large file size • Multiple data streams

33. Multimedia Operating Systems • Other OS issues • IPC (inter-process communication) and synchronization • Memory management • Device management

34. Multimedia Software & Applications

35. Multimedia Software • Software for Media Composition • Text and Graphics Editors • Image Editors • Animation Editors • Sound Editors • Video Editors • Editors often allow you create media as well, but not always. If not, you will need a separate application to create media, like for video capture.

36. Multimedia Software • Software for Media Integration • Multimedia Editors • Hypermedia / Hypertext Editors • Authoring Tools

37. Multimedia Applications • Interactive • Conversational – video conferencing, video phone, Computer supported cooperative work (CSCW). • Messaging Service. E.g. MIME (Multipurpose Internet Mail Extensions) – send/receive multimedia files via e-mail • Retrieval Service – Video Server, WWW, Gopher • Tele-action Service – Tele-robotics with kinesthetic feedback • Tele-operation Service –remote education, Tele-office, Tele-collaboration, telemedicine

38. Multimedia Applications • Distribution • Pay-per-view – similar to cable service • Near Video on Demand (Near VoD) – multichannel broadcast + deferred airing • True Video on Demand (VoD) – e.g. interactive TV, Cyber Vision

39. Multimedia Applications • Media Consumption • Viewing Multimedia Documents –browsing or detailed media consumption • Electronic Books, Proceedings and Newspapers • Info Kiosks • Tele-shopping

40. Multimedia Applications • Media Entertainment • Virtual Reality (VR) • Interactive Video - interactive TV, VoD. • Interactive Audio - CD on Demand • Games – Tele-games

41. Multimedia Applications • Trends • From reengineering to establishing new application domains • From single PC user to multi-user or personalized environment • Application becomes less local, more distributed • Media Consumption mode from passive to active interaction • From unidirectional to bi-directional information flow • Technical improvements improve productivity through better collaboration and visualization.

42. List of References • http://en.wikipedia.org/wiki/Blu-ray_Disc