1 / 31

CS 414 – Multimedia Systems Design Lecture 4 – Visual Perception and Digital Image Representation

This lecture covers the concepts of visual perception and digital image representation in multimedia systems design. Topics include signal-to-noise ratio, human visual system, digital image sampling and quantization, and more.

williamrwilliam
Download Presentation

CS 414 – Multimedia Systems Design Lecture 4 – Visual Perception and Digital Image Representation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 414 – Multimedia Systems DesignLecture 4 – Visual Perception and Digital Image Representation Klara Nahrstedt Spring 2014 CS 414 - Spring 2014

  2. Administrative • Groups are formed and names have been sent to Engineering IT and Barb Leisner • We will inform you about group directories as soon as we have information from Engineering IT CS 414 - Spring 2014

  3. Administrative • Leasing Process from Barb Leisner • Lease one Logitech camera - two cameras within one group to start MP1, and then for MP2/MP3. • Leasing process starts on January 31 • Pick up the camera from Barb Leisner office, 2312 SC • Bring your student ID to sign for the camera • Each cs414 group is responsible for his/her own camera • if you loose it (or badly damage) and you don’t have police report, you pay for it (charged to your student account at the end of the semester) • Hours to pick up camera: Monday –Friday 9am-5pm • No camera pickup on Saturday and Sunday CS 414 - Spring 2014

  4. Today Introduced Concepts • Important Metric for Digital Audio • Signal-to-Noise Ratio (dB) • Human Visual System • Digital Images • Sampling • Quantization • Spatial Resolution CS 414 - Spring 2014

  5. Signal-to-Noise Ratio(metric to quantify quality of digital audio) CS 414 - Spring 2014

  6. A 0 -A Signal To Noise (SNR) Ratio • Measures strength of signal to noise SNR (in DB)= • Given sound form with amplitude in [-A, A] • Signal energy = CS 414 - Spring 2014

  7. Modeling of Noise - Quantization Error • Difference between actual and sampled value • amplitude between [-A, A] • quantization levels = N • e.g., if A = 1,N = 8, = 1/4 CS 414 - Spring 2014

  8. Compute Signal to Noise Ratio • Signal energy = ; Noise energy = ; • Noise energy = • Signal-to-Noise = • SNR depends on number of bits (number of quantization levels) assigned to signal • Every bit increases SNR by ~ 6 decibels

  9. Integrating Aspects of Multimedia Audio/Video Presentation Playback Image/Video Capture Audio/Video Perception/ Playback Image/Video Information Representation Transmission Transmission Compression Processing Audio Capture Media Server Storage Audio Information Representation A/V Playback CS 414 - Spring 2014

  10. Human Visual System • Eyes, optic nerve, parts of the brain • Transforms electromagnetic energy

  11. Human Visual System • Image Formation • cornea, sclera, pupil,iris, lens, retina, fovea • Transduction • retina, rods, and cones • Retina has photosensitive receptors at back of eye • Processing • optic nerve, brain

  12. Contain photo-pigment Respond to low energy Enhance sensitivity Concentrated in retina, but outside of fovea One type, sensitive to grayscale changes Contain photo-pigment Respond to high energy Enhance perception Concentrated in fovea, exist sparsely in retina Three types, sensitive to different wavelengths Rods vs Cones (Responsible for us seeing brightness and color) Cones Rods CS 414 - Spring 2014

  13. Tri-stimulus Theory • 3 types of cones (6/7 Mil. of them) • Red = L cones, Green = M cones, Blue = S cones • Ratio differentiates for each person • E.g., Red (64%), Green (32%), rest S cones • E.g., L(50.6%), M(44.2%), rest S cones • Each type most responsive to a narrow band electro-magnetic waves • red and green absorb most energy, blue the least • Light stimulates each set of cones differently, and the ratios produce sensation of color CS 414 - Spring 2014

  14. Color and Visual System • Color refers to how we perceive a narrow band of electromagnetic energy • source, object, observer • Visual system transforms light energy into sensory experience of sight

  15. Color Perception (Color Theory) Hue Scale • Hue • Refers to pure colors • dominant wavelength of the light • Saturation • Perceived intensity of a specific color • how far color is from a gray of equal intensity • Brightness (lightness) • perceived intensity Original Saturation lightness CS 414 - Spring 2014 Source: Wikipedia

  16. Digitalization of Images – Capturing and Processing CS 414 - Spring 2014

  17. Capturing Real-World Images W2 W1 r W3 F r= function of (W1/W3); s=function of (W2/W3) s Picture – two dimensional image captured from a real-world scene that represents a momentary event from the 3D spatial world CS 414 - Spring 2014

  18. Image Concepts - Sampling • An image is a function of intensity values over a 2D plane I(r,s) • Sample function at discrete intervals to represent an image in digital form • matrix of intensity values for each color plane • intensity typically represented with 8 bits • Sample points are called pixels CS 414 - Spring 2014

  19. Digital Image Sampling • Sample = pixel • Image Size (in pixels) • Image Size = Height x Width (in pixels) • 320x240 pixels • 640x480 pixels • 1920x1080pixels CS 414 - Spring 2014

  20. Digital Images - Quantization • Quantization = number of bits per pixel • Example: if we would sample and quantize standard TV picture (525 lines) by using VGA (Video Graphics Array), • video controller creates matrix 640x480pixels, and • each pixel is represented by 8 bit integer (256 discrete gray levels) CS 414 - Spring 2014

  21. Image Representations • Black and white image • single color plane with 2 bits • Grey scale image • single color plane with 8 bits • Color image • three color planes each with 8 bits • RGB, CMY, YIQ, etc. • Indexed color image • single plane that indexes a color table • Compressed images • TIFF, JPEG, BMP, etc. 4 gray levels 2gray levels

  22. Digital Image Representation (3 Bit Quantization) CS 414 - Spring 2014

  23. Color QuantizationExample of 24 bit RGB Image 24-bit Color Monitor CS 414 - Spring 2014

  24. Image Representation Example 24 bit RGB Representation (uncompressed) Color Planes

  25. Graphical Representation CS 414 - Spring 2014

  26. Image Properties (Color) CS 414 - Spring 2014

  27. Color Histogram CS 414 - Spring 2014

  28. Spatial domain refers to planar region of intensity values at time t Frequency domain think of each color plane as a sinusoidal function of changing intensity values refers to organizing pixels according to their changing intensity (frequency) Spatial and Frequency Domains CS 414 - Spring 2014

  29. Spatial Resolution and Brightness • Spatial Resolution (depends on: ) • Image size • Viewing distance • Brightness • Perception of brightness is higher than perception of color • Different perception of primary colors • Relative brightness: green:red:blue= 59%:30%:11% CS 414 - Spring 2014 Source: wikipedia

  30. Image Size (in Bits) • Image Size = Height x Width X Bits/pixel • Example: • Consider image 320x240 pixels with 8 bits per pixel • Image takes storage 7680 x 8 bits = 61440 bits or 7680 bytes CS 414 - Spring 2014

  31. Summary • Important Image Processing Functions (see Computer Vision/Image Processing classes) • Filtering • Edge detection • Image segmentation • Image recognition • Formatting • Conditioning • Marking • Grouping • Extraction • Matching • Image synthesis CS 414 - Spring 2014

More Related