1 / 70

ICS 542 Multimedia Computing

ICS 542 Multimedia Computing . Spring Semester 2007 - 2008 (072) King Fahd University of Petroleum & Minerals Information & Computer Science Department. Important Preliminaries. Instructor: Dr. Wasfi Al-Khatib وصفي الخطيب Office: (22) 133-1 Office hours: TBA By Appointment.

elana
Download Presentation

ICS 542 Multimedia Computing

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. ICS 542Multimedia Computing Spring Semester 2007 - 2008 (072) King Fahd University of Petroleum & Minerals Information & Computer Science Department

  2. Important Preliminaries • Instructor: Dr. Wasfi Al-Khatib وصفي الخطيب • Office: (22) 133-1 • Office hours: • TBA • By Appointment. • Phone: 1715 • email: wasfi@kfupm.edu.sa

  3. General Description • This course gives an overview of the different issues in the representation and management of multimedia data in the context of content-based retrieval. The emphasis will be on textual data, video data, audio data, multimedia documents and multimedia datamining. However, students interested in other related topics will have the opportunity to explore them through their projects and possibly research paper presentations.

  4. Feedback Slip

  5. Course Material • Prerequisites: Instructor Consent. • Text Book • Ze-Nian Li & Mark S. Drew, Fundamentals of Multimedia, ISBN 0-13-061872-1, Prentice-Hall, 2004. • References • Guojun Lu, Multimedia Database Management Systems, Artech House.. • Michael Berry & Murray Browne, Understanding Search Engines: Mathematical Modeling and Text Retrieval, ISBN 0-89871-437-0, SIAM, 1999. • Select research papers and articles

  6. Grading Policy (1)

  7. Grading Policy (2) • The project's 40% is distributed as follows: • Project Proposal: 1% • Project Progress Report 1: 3% • Project Progress Report 2: 5% • Final Project Report and Prototype: 30% • In Class Project Presentation: 5%

  8. Important Dates

  9. Remarks (1) • Homeworks are due by the time announced on Blackboard. Late homeworks are NOT accepted. • Quizzes: 25-40 minute. Each covers material covered since the last quiz or major exam. • The Major exam will cover material covered from the beginning of the semester until the last lecture before the day of the exam.

  10. Remarks (2) • Project proposal, progress report 1, progress report 2, and final report Late Submission Policy: • First day costs 10%. • Multiply previous day cost by 2 to get the percentage cost of the current day. • You will receive zero points if not submitted before the fifth day. • Format of each of the above deliverables will be given later.

  11. Research Paper Presentations Policy (1) • Each student will present one research papers that will be pre-assigned two weeks before the presentation time. • Each presentation will take between 20-30 minutes total (including questions and answers) • Handouts for the presentations must be available by 4:00pm on Blackboard the same day of presentation. • I expect you to “master” the paper.

  12. Research Paper Presentations Policy (2) • The presentation must include the following: • Title and author of the paper. • Brief summary: describe the main points of the paper. • Strengths: List 2-3 positive contributions of the paper. • Weaknesses: List 2-3 weaknesses of the paper. • Expected Research Impact: Possible utilization of the work in different and/or other areas. • Overall Assessment: Was it worth reading? (Excellent, Very Good, Good, Fair, Poor)

  13. Your 24-Hour Right • One has 24 hours to object to the grade of a quiz or a major exam starting from the end of the class time in which the graded exam papers have been distributed. • i.e. if you were not present in class during the distribution of exam papers, you lose this right. • If for some reason you cannot see me in person, within this period, send me an email requesting an appointment. The email, though, should be sent within the 24-hour time period.

  14. Code of Ethics • KFUPM regulations and standards will be enforced

  15. The Term Multimedia • Multi • Prefix • From Latin “Multus” meaning numerous • Media • Root • Plural form of the Latin word “Medium”. • “Medium” is a noun meaning middle or center.

  16. The Term Media • Media: A means to distribute and present information

  17. Text Data vs. Multimedia Data

  18. Attributes of Media • Perception Media • Representation Media • Presentation Media • Storage Media • Transmission Media • Information Exchange Media • Presentation Spaces and Presentation Values • Presentation Dimensions

  19. Brief History of Multimedia Systems

  20. Multimedia is still at its infancy • Cannot avoid fuzziness in scope, multiplicity of definitions and non-stabilized terminology.

  21. Great Impact of Multimedia • Integrating all media in the computer allows using the existing computer power to represent information interactively. • This can be stored in mass storage devices. • This can, also, be transmitted over computer networks.

  22. Terminology • Monomedia object: Object containing data of a single type • Multimedia object: Object containing data of multiple media • Hypertext document: Nonlinear text document (with links) • Hypermedia document: Nonlinear multimedia document

  23. Issues in Multimedia Database Management Systems (1) • Development of formal semantic modeling techniques for multimedia information. • Design of powerful indexing, searching, and organization methods. • Development of models for specifying the media synchronization/integration requirements.

  24. Issues in Multimedia Database Management Systems (2) • Designing formal multimedia query languages. • Development of efficient data-placement schemas for physical storage management. • Design and development of suitable architecture and operating system support. • Management of distributed multimedia databases.

  25. User Interface Interactive Layer (3) Multimedia Query Interface Navigation Tool Media Editing Interactive Icons Multimedia Object Management and Query Processing Layer (2) Multimedia Information/Media Composition & Integration Meta-Model/Query Processing Text DBMS Image DBMS Audio DBMS Video DBMS Monomedia Database Management Layer (1) Records Text Images Audio Video Reference Architecture

  26. Our Focus • Information Extraction from Multimedia Data, aka Content-Based Retrieval (Projects) • Which information to extract? • How to extract it? • Multimedia Data Representation and Indexing Techniques (Lectures and Projects)

  27. Outline of the Course • Multimedia datatypes, their representations and properties: text, audio, image, and video. • Mono-media indexing and Retrieval (Text, Image, Audio and Video). • Multimedia document indexing and retrieval. • Multimedia Compression. • Multimedia Datamining. … + select techniques from research papers.

  28. Multimedia Data Types & Formats How computers “see” or store the data? • Raw data. • Structured data. • Compressed.

  29. Text • Plain Text • Structured Text • Latex • XML • Compressed Text • Huffman Coding • Run-Length Coding • LZW Coding

  30. Example Latex Document \documentclass[ece,plain]{puthesis} \usepackage{rotating,graphicx,tabularx,epsfig,psfig,subfigure,alg,amssymb,hhline} \title{Data Modeling and Querying in Video Databases} \author{Wasfi Al-Khatib}{Al-Khatib, Wasfi} \degree{Doctor of Philosophy}{Ph.D.}{May}{2001} \majorprof{Arif Ghafoor} \begin{document} \include{front} \include{ch1} \include{ch2} \include{ch3} \include{ch4} \include{ch5} \include{ch6} \bibliography{../refs/refs} \include{vita} \end{document}

  31. Example XML Document 1 <?xml version="1.0"?> <Schema xmlns="urn:schemas-microsoft-com:xml-data" xmlns:dt="urn:schemas-microsoft-com:datatypes"> <ElementType name="first-name" content="textOnly"/> <ElementType name="last-name" content="textOnly"/> <ElementType name="name" content="textOnly"/> <ElementType name="price" content="textOnly" dt:type="fixed.14.4"/> <ElementType name="author" content="eltOnly" order="one"> <group order="seq"> <element type="name"/> </group> <group order="seq"> <element type="first-name"/> <element type="last-name"/> </group> </ElementType> <ElementType name="title" content="textOnly"/>

  32. Example XML Document 1 (Cont.) <AttributeType name="genre" dt:type="string"/> <ElementType name="book" content="eltOnly"> <attribute type="genre" required="yes"/> <element type="title"/> <element type="author"/> <element type="price"/> </ElementType> <ElementType name="bookstore" content="eltOnly"> <element type="book"/> </ElementType> </Schema>

  33. Example XML Document 2 <?xml version='1.0'?> <!-- This file represents a fragment of a book store inventory database --> <bookstore> <book genre="autobiography"> <title>The Autobiography of Benjamin Franklin</title> <author> <first-name>Benjamin</first-name> <last-name>Franklin</last-name> </author> <price>8.99</price> </book> … </bookstore>

  34. Computer Graphics • Vector Graphics • Bitmap Images • Combining vectors and bitmaps

  35. Models of Bitmapped Graphics • Modeled by an array of small picture elements (pixels). • Intensity or color of pixels are stored in a pixel-based graphics file. • Bitmapped graphics does not use bitmaps (except for pure monochrome images). It use pixel maps

  36. Models of Vector Graphics • Stored as a mathematical description of a collection of individual lines, curves and shapes making up the image • e.g. line = two end points • For example, xfig in unix, Adobe Illustrator in Windows. • Displaying a vector image • requires some computation to be performed in order to interpret the model and generate an array of pixels to be displayed • The process of interpreting the vector description is known as rasterizing

  37. Vector Graphics • Scalable • Resolution independent • No background • Cartoon-like • Inappropriate for photo-realistic images • Metafiles contain both raster and vector data

  38. Xfig Example

  39. # FIG 3.2 Landscape Center Inches Letter 100.00 Single -2 1200 2 6 4200 4425 6825 4575 4 0 0 50 0 0 12 0.0000 4 135 510 4200 4567 include\001 4 0 0 50 0 0 12 0.0000 4 90 210 5100 4545 src\001 4 0 0 50 0 0 12 0.0000 4 135 225 5775 4567 bin\001 4 0 0 50 0 0 12 0.0000 4 135 285 6525 4567 data\001 -6 2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 4575 1950 3825 2625 2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 4650 1950 5325 2625 2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 5400 3000 5400 3525 2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 5400 3825 4500 4350 2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 5400 3825 5250 4350 2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 5400 3825 5850 4350 2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 5400 3825 6600 4350 2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 3825 3000 3825 3525 2 1 0 1 0 7 50 0 -1 0.000 0 0 -1 0 0 2 5400 3825 7275 4275 4 0 0 50 0 0 12 0.0000 4 165 855 4200 1875 CS400_600\001 4 0 0 50 0 0 12 0.0000 4 180 570 5100 2850 Projects\001 4 0 0 50 0 0 12 0.0000 4 135 885 3450 2850 Homeworks\001 4 0 0 50 0 0 12 0.0000 4 180 375 5250 3750 proj1\001 4 0 0 50 0 0 12 0.0000 4 135 315 3675 3750 hw1\001 4 0 0 50 0 0 12 0.0000 4 135 165 7275 4575 all\001

  40. Computer Animations • Animation is produced by sequential rendering of frames of graphics.

  41. Audio • Audio is a wave resulting from air pressure disturbance that reaches our eardrum generating the sound we hear. • Humans can hear frequencies in the range 20-20,000 Hz. • ‘Acoustics’ is the branch of physics that studies sound

  42. Characteristics of Audio • Audio has normal wave properties • Reflection • Refraction • Diffraction • A sound wave has several different properties: • Amplitude (loudness/intensity) • Frequency (pitch) • Envelope (waveform)

  43. Audio Amplitude • Audio amplitude is often expressed in decibels (dB) • Sound pressure levels (loudness or volume) are measured in a logarithmic scale (deciBel, dB) used to describe a ratio • Suppose we have two loudspeakers, the first playing a sound with power P1, and another playing a louder version of the same sound with power P2, but everything else (how far away, frequency) is kept the same. • The difference in decibels between the two is defined to be 10 log10 (P2/P1) dB

  44. Audio Amplitude • In microphones, audio is captured as analog signals (continuous amplitude and time) that respond proportionally to the sound pressure, p. • The power in a sound wave, all else equal, goes as the square of the pressure. • Expressed in dynes/cm2. • The difference in sound pressure level between two sounds with p1 and p2 is therefore 20 log10 (p2/p1) dB • The “acoustic amplitude” of sound is measured in reference to p1 = pref = 0.0002 dynes/cm2. • The human ear is insensitive to sound pressure levels below pref.

  45. Audio Amplitude

  46. Audio Frequency • Audio frequency is the number of high-to-low pressure cycles that occurs per second. • In music, frequency is referred to as pitch. • Different living organisms have different abilities to hear high frequency sounds • Dogs: up to 50KHz • Cats: up to 60 KHz • Bats: up to 120 KHz • Dolphins: up to 160KHz • Humans: • Called the audible band. • The exact audible band differs from one to another and deteriorates with age.

  47. Audio Frequency • The frequency range of sounds can be divided into • Infra sound 0 Hz – 20 Hz • Audible sound 20 Hz – 20 KHz • Ultrasound 20 KHz – 1 GHz • Hypersound 1 GHz – 10 GHz • Sound waves propagate at a speed of around 344 m/s in humid air at room temperature (20 C) • Hence, audio wave lengths typically vary from 17 m (corresponding to 20Hz) to 1.7 cm (corresponding to 20KHz). • Sound can be divided into periodic (e.g. whistling wind, bird songs, sound from music) and nonperiodic (e.g. speech, sneezes and rushing water).

  48. Audio Frequency • Most sounds are combinations of different frequencies and wave shapes. Hence, the spectrum of a typical audio signal contains one or more fundamental frequency, their harmonics, and possibly a few cross-modulation products. • Fundamental frequency • Harmonics • The harmonics and their amplitude determine the tone quality or timbre.

  49. Audio Envelope • When sound is generated, it does not last forever. The rise and fall of the intensity of the sound is known as the envelope. • A typical envelope consists of four sections: attack, decay, sustain and release.

  50. Audio Envelope • Attack: The intensity of a note increases from silence to a high level • Decay: The intensity decreases to a middle level. • Sustain: The middle level is sustained for a short period of time • Release: The intensity drops from the sustain level to zero.

More Related