1 / 37

Multimedia- and Web-based Information Systems

Multimedia- and Web-based Information Systems. Lecture 4. Multimedia: Audio. Generation of Sound. Sound is a physical phenomenon Shock wave fluctuations of the air Caused by vibration of some material e.g. string of a violin Perceived as sound by human ear. Audiotechnology.

zuriel
Download Presentation

Multimedia- and Web-based Information Systems

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Multimedia- and Web-based Information Systems Lecture 4

  2. Multimedia: Audio

  3. Generation of Sound • Sound is a physical phenomenon • Shock wave fluctuations of the air • Caused by vibration of some material • e.g. string of a violin • Perceived as sound by human ear

  4. Audiotechnology • Processing of acoustic signals • Sound • Music (e.g. MIDI) • Voice processing (synthesis and analysis) • A lot of Multimedia applications utilize the medium Audio as Music and/or voice

  5. Fundamentals • <Drawing> • Frequency • f = 1/T • Pitch of a tone • Frequency domains • Perceivable sound: 20 Hz – 20 kHz • Audio = acoustically perceivable signals • Amplitude • Volume

  6. Psycho acoustics • Research of spatial sound perception • Interaural Time Difference (ITD) • Interaural Intensity Difference (IID) • Doppler-Effect • Masking of sound

  7. Representation of audio in a computer • Continual sound wave can not be represented in a computer • Measure amplitude in regular intervals • Sequence of samples (sampled values) • Sampling Rate • <Drawing>

  8. Nyquist‘s Sampling Theorem • Bandwidth of a digitally sampled audio signal is about half as large as the sampling rate • CD • Sampling rate 44.100 Hz • Bandwidth: 0 Hz – 22.050 Hz

  9. Generation of Samples • Quantisation • Projection of a discrete sampled value onto X bit • X-bit-Quantisation • The lower the quantisation the worse the resulting sound quality • 8 bits with the telephone, 16 bits with the CD

  10. Audio-Coding • PCM • Pulse Code Modulation • DPCM • Differential PCM • DM • Delta Modulation • ADPCM • Adaptive DPCM

  11. Companding • Projection of „Signals“ • Emphasis / neglect of important/less important areas of the signal • <Drawing>

  12. Music - MIDI • MIDI = Music Instrument Digital Interface • Transmission of coded music signals between electronic musical instruments (and computers) • Instrument-based Representation • Instead of sampling values • Compact representation

  13. MIDI – Fundamentals • Hardware • Connection of the equipment (MIDI-Port and MIDI-cable) • Data format • MIDI-Message • Coding of the instrument, start and finish of a note, base frequency and amplitude

  14. MIDI - Devices • An instrument following the MIDI-Standard is a MIDI-device • 16 channels • 128 instruments • Properties of instruments (e.g. can it play multiple notes simoultaneously?) • Control by a computer (processing) • Sequencer (Storage and changing the MIDI-data)

  15. MIDI and SMPTE-Timing-Standards • MIDI-Clock • Receipient is able to synchronize to the time of the sender, 24 marks per quarter of a note • SMPTE-Timecode (Society of Motion Picture and Television Engineers) • Hours:Minutes:Seconds:Frames (30 Frames/s) • MIDI Time Code (Simplification) • Do not transmit the whole time representation with every frame

  16. Voice signals • Processing by human and machine • Synthetic generation of voice • Voice input

  17. Demands in voice output • Voice output: automated generation of voice • Real time • Understandable • Natural

  18. Voice synthesis • Transformation of a text into acoustic signals Transfer of sounds Dictionary Transcription Voice Synthesis Text Phonetic Transcription

  19. Voice Input Voice input Speaker recognition Voice recognition ... What? How? Who? Verification, Identification

  20. Principles of voice recognition Memory of references (features) Feature extraction (Storage) Voice Analysis (feature extraction) Pattern Recognition (Comparison with reference, Decision) Voice (Specialized Chips) (Main Processor) Recognized Voice

  21. Components of voice recognition sound-patterns, word models semantics syntax Acoustic and phonetic analysis Syntactic analysis Semantic analysis Voice Recognized sentence

  22. Multimedia: Images, Figures and Animation

  23. Images and Figures • Image • N rows with M points each • Figure • Specified trough graphical primitives with properties • Lines, rectangles, ellipses, ... (Primitives) • Width, Color, ... (Properties)

  24. Images • Pixel = Picture Element • Color coding • 8-Bit-PCM-Quantisation • 2^8*2^8*2^8 = ca. 16 million colors • 256 shades of grey • Digital Images require a lot of memory • 1024*768*3 = 2.3 MByte

  25. Image formats • Recording Image Format • Spatial Resolution • Pixel x Pixel • Color coding • Image Storage Format • Two-dimensional matrix (bitmap) • Color information (RGB-Value, Index) • Compressed / Uncompressed

  26. Popular Image Storage Formats • GIF (Graphics Interchange Format) • 256 Colors, Animated GIFs • LZW-Algorithm (Lempel-Ziv-Welch) • TIFF (Tagged Image File Format) • Various uses, e.g. fax machines • BMP (Bitmap) • Color depth 1, 4, 8, 24 bit • JPEG (more details later)

  27. Computerized Processing • Computer graphics • Synthesis of real or imaginary pictures from computer-based models • Image Processing • Analysis of scenes or reconstruction of models

  28. Image Analysis • Extraction of descriptions from images • Picture Improvement • Removal of noise • Pattern recognition • OCR (Optical Character Recognition) • Analysis of scenes • Reconstruction of 3D-models

  29. Features used with image analysis • Color • Color histogram • Grey-value histogram • Texture • Surface structures of small extent • Edges

  30. Computer-based Animation • Literal meaning: „bring to life“ • Dynamics of movement • Dynamics of Change • Camera position and orientation • Created, changed and generated with the help of a computer • Animations as a „Medium“

  31. Input process • Extreme or characteristic positions • Key frames • Single pictures of an animation called frames

  32. Arrangement of the animation • Combination of Fore- and Background • Animated cartoons • Displacement and zoom

  33. Generation of intermediate frames • Interpolation of an objects movement • Specification of intermediate frames • Interpolation • Linear Interpolation • Splines • More natural movements • <Drawing>

  34. Changes in Color • Color Lookup Table (CLUT) • Determines the color values of single pixels • Cyclic change of the CLUT • Simple and fast

  35. Specification of animations • Linear lists • Higher-level programming language • Visual animation languages

  36. Methods of controlling animation • Open explicit control • Every single event is described • Procedural Control • Information Interchange between objects • Control on the basis of varying conditions • Restrictions that need to be fulfilled • Dynamics of real bodies and material properties

  37. Methods of controlling animation • Control by analysis of real movement • Motion capturing • Transfer of the analysis of real events onto objects that are to be animated • Cinematic and dynamic control • Applying physical laws • Cinematics and gravity

More Related