ICS 218 -Multimedia Systems and Applications

1. ICS 218 -Multimedia Systems and Applications Lecture 1 - Introduction to Multimedia Prof. Nalini Venkatasubramanian nalini@ics.uci.edu Introduction to Multimedia

2. Course logistics and details • Course Web page - • http://www.ics.uci.edu/~ics218 • Lectures - MW 2:00-3:20p.m, ELH 110 • Course Laboratories - machines on 3rd floor CS labs • ICS 218 Textbook: Multimedia: Computing, Communications and Applications Ralf Steinmetz and Klara Nahrstedt, Prentice-Hall Inc. • Other reading material • Technical papers and reports Introduction to Multimedia

3. Course logistics and details • Homeworks and Assignments • 3 homeworks in the quarter of which 1 or 2 may be programming assignments. • Tests • Final Exam - as per UCI course catalog • Course Project • Maybe done individually, in groups of 2 or 3(max) • Potential projects on webpage • Data representation, multimedia systems and networks, multimedia applications Introduction to Multimedia

4. ICS 218 Grading Policy • Homeworks - 30% • (3 homeworks each worth 10% of the final grade). • Class Project - 40% of the final grade • Final exam - 30% of the final grade • Final assignment of grades will be based on a curve. Introduction to Multimedia

5. Lecture Schedule • Weeks 1 and 2: • Multimedia Representation • Introduction to Multimedia • Audio/Image/Video Representation • Weeks 3 and 4: • Multimedia Compression • Encoding and Compression Techniques • Image Compression (JPEG) • Video Compression (MPEG/MPEG2/MPEG4) • Week 5 and 6: • Multimedia Resource Management • Multimedia Quality of Service and Server Design • Multimedia Operating Systems - Process Management, Filesystems and buffer management Introduction to Multimedia

6. Course Schedule • Weeks 7 and 8: • Multimedia Communication Systems • MM Networking • MM Communication • Week 9: • Multimedia Synchronization • Multimedia Synchronization Concepts • Multimedia Synchronization Enforcement • Week 10: • Multimedia Applications and Services • Multimedia Video Conferencing • Multimedia Entertainment Applications Introduction to Multimedia

7. Introduction • Multimedia Description • Why multimedia systems? • Classification of Media • Multimedia Systems • Data Stream Characteristics Introduction to Multimedia

8. Multimedia Description • Multimedia • is an integration of continuous media (e.g. audio, video) and discrete media (e.g. text, graphics, images) through which digital information can be conveyed to the user in an appropriate way. • Multi • many, much, multiple • Medium • An interleaving substance through which something is transmitted or carried on Introduction to Multimedia

9. Why Multimedia Computing? • Application driven • e.g. medicine, sports, entertainment, education • Information can often be better represented using audio/video/animation rather than using text, images and graphics alone. • Information is distributed using computer and telecommunication networks. • Integration of multiple media places demands on • computation power • storage requirements • networking requirements Introduction to Multimedia

10. Multimedia Information Systems • Technical challenges • Sheer volume of data • Need to manage huge volumes of data • Timing requirements • among components of data computation and communication. • Must work internally with given timing constraints - real-time performance is required. • Integration requirements • need to process traditional media (text, images) as well as continuous media (audio/video). • Media are not always independent of each other - synchronization among the media may be required. Introduction to Multimedia

11. High Data Volume of Multimedia Information Introduction to Multimedia

12. Technology Incentive • Growth in computational capacity • MM workstations with audio/video processing capability • Dramatic increase in CPU processing power • Dedicated compression engines for audio, video etc. • Rise in storage capacity • Large capacity disks (several gigabytes) • Increase in storage bandwidth,e.g. disk array technology • Surge in available network bandwidth • high speed fiber optic networks - gigabit networks • fast packet switching technology Introduction to Multimedia

13. Application Areas • Residential Services • video-on-demand • video phone/conferencing systems • multimedia home shopping (MM catalogs, product demos and presentation) • self-paced education • Business Services • Corporate training • Desktop MM conferencing, MM e-mail Introduction to Multimedia

14. Application Areas • Education • Distance education - MM repository of class videos • Access to digital MM libraries over high speed networks • Science and Technology • computational visualization and prototyping • astronomy, environmental science • Medicine • Diagnosis and treatment - e.g. MM databases that provide support for queries on scanned images, X-rays, assessments, response etc. Introduction to Multimedia

15. Classification of Media • Perception Medium • How do humans perceive information in a computer? • Through seeing - text, images, video • Through hearing - music, noise, speech • Representation Medium • How is the computer information encoded? • Using formats for representing and information • ASCII(text), JPEG(image), MPEG(video) • Presentation Medium • Through which medium is information delivered by the computer or introduced into the computer? • Via I/O tools and devices • paper, screen, speakers (output media) • keyboard, mouse, camera, microphone (input media) Introduction to Multimedia

16. Classification of Media (cont.) • Storage Medium • Where will the information be stored? • Storage media - floppy disk, hard disk, tape, CD-ROM etc. • Transmission Medium • Over what medium will the information be transmitted? • Using information carriers that enable continuous data transmission - networks • wire, coaxial cable, fiber optics • Information Exchange Medium • Which information carrier will be used for information exchange between different places? • Direct transmission using computer networks • Combined use of storage and transmission media (e.g. electronic mail). Introduction to Multimedia

17. Media Concepts • Each medium defines • Representation values - determine the information representation of different media • Continuous representation values (e.g. electro-magnetic waves) • Discrete representation values(e.g. text characters in digital form) • Representation space determines the surrounding where the media are presented. • Visual representation space (e.g. paper, screen) • Acoustic representation space (e.g. stereo) Introduction to Multimedia

18. Media Concepts (cont.) • Representation dimensions of a representation space are: • Spatial dimensions: • two dimensional (2D graphics) • three dimensional (holography) • Temporal dimensions: • Time independent (document) - Discrete media • Information consists of a sequence of individual elements without a time component. • Time dependent (movie) - Continuous media • Information is expressed not only by its individual value but also by its time of occurrence. Introduction to Multimedia

19. Multimedia Systems • Qualitative and quantitative evaluation of multimedia systems • Combination of media • continuous and discrete. • Levels of media-independence • some media types (audio/video) may be tightly coupled, others may not. • Computer supported integration • timing, spatial and semantic synchronization • Communication capability Introduction to Multimedia

20. Data Streams • Distributed multimedia communication systems • data of discrete and continuous media are broken into individual units (packets) and transmitted. • Data Stream • sequence of individual packets that are transmitted in a time-dependant fashion. • Transmission of information carrying different media leads to data streams with varying features • Asynchronous • Synchronous • Isochronous Introduction to Multimedia

21. Data Stream Characteristics • Asynchronous transmission mode • provides for communication with no time restriction • Packets reach receiver as quickly as possible, e.g. protocols for email transmission • Synchronous transmission mode • defines a maximum end-to-end delay for each packet of a data stream. • May require intermediate storage • E.g. audio connection established over a network. • Isochronous transmission mode • defines a maximum and a minimum end-to-end delay for each packet of a data stream. Delay jitter of individual packets is bounded. • E.g. transmission of video over a network. • Intermediate storage requirements reduced. Introduction to Multimedia

22. Data Stream Characteristics • Data Stream characteristics for continuous media can be based on • Time intervals between complete transmission of consecutive packets • Strongly periodic data streams - constant time interval • Weakly periodic data streams - periodic function with finite period. • Aperiodic data streams • Data size - amount of consecutive packets • Strongly regular data streams - constant amount of data • Weakly regular data streams - varies periodically with time • Irregular data streams • Continuity • Continuous data streams • Discrete data streams Introduction to Multimedia

23. Classification based on time intervals Strongly periodic data stream T Weakly periodic data stream T1 T2 T3 T T Aperiodic data stream T1 T2 Introduction to Multimedia

24. Classification based on packet size D1 T t Strongly regular data stream D1 D1 T D2 Weakly regular data stream D3 t D1 D2 D3 D1 D2 Irregular data stream t D3 Dn Introduction to Multimedia

25. Classification based on continuity Continuous data stream D1 D2 D3 D4 D D1 D2 D3 D4 D Discrete data stream Introduction to Multimedia

26. Logical Data Units • Continuous media consist of a time-dependent sequence of individual information units called Logical Data Units (LDU). • a symphony consists of independent sentences • a sentence consists of notes • notes are sequences of samples • Granularity of LDUs • symphony, sentence, individual notes, grouped samples, individual samples • film, clip, frame, raster, pixel • Duration of LDU: • open LDU - duration not known in advance • closed LDU - predefined duration Introduction to Multimedia

27. Granularity of Logical Data Units Film Clip Frame Blocks Pixels Introduction to Multimedia