1 / 33

Visual Perception and Digital Image Representation

This lecture covers the concepts of visual perception and digital image representation in multimedia systems. Topics include signal-to-noise ratio, human visual system, color perception, and digitalization of images.

williamsgeorge
Download Presentation

Visual Perception and Digital Image Representation

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. CS 414 – Multimedia Systems DesignLecture 4 – Visual Perception and Digital Image Representation Klara Nahrstedt Spring 2012 CS 414 - Spring 2012

  2. Administrative • Groups are formed and names have been sent to TSG and Barb Leisner • We will inform you about group directories as soon as we have information from TSG CS 414 - Spring 2012

  3. Administrative • Leasing Process from Barb Leisner • Lease one Logitech camera - two cameras within one group to start MP1, and then for MP2/MP3. • Leasing process starts on January 25 • Pick up the camera from Barb Leisner office, 2312 SC • Bring your student ID to sign for the camera • Each cs414 student is responsible for his/her own camera • if you loose it (or badly damage) and you don’t have police report, you pay for it (charged to your student account at the end of the semester) • Hours to pick up camera: Monday –Friday 9am-5pm • No camera pickup on Saturday and Sunday CS 414 - Spring 2012

  4. Today Introduced Concepts • Two Important Metrics for Digital Audio • Signal-to-Noise Ratio (dB) • Digital Audio Data Rates (bits per second) • Human Visual System • Digital Images • Sampling • Quantization • Spatial Resolution CS 414 - Spring 2012

  5. Signal-to-Noise Ratio(metric to quantify quality of digital audio) CS 414 - Spring 2012

  6. A 0 -A Signal To Noise (SNR) Ratio • Measures strength of signal to noise SNR (in DB)= • Given sound form with amplitude in [-A, A] • Signal energy = CS 414 - Spring 2012

  7. Modeling of Noise - Quantization Error • Difference between actual and sampled value • amplitude between [-A, A] • quantization levels = N • e.g., if A = 1,N = 8, = 1/4 CS 414 - Spring 2012

  8. Compute Signal to Noise Ratio • Signal energy = ; Noise energy = ; • Noise energy = • Signal-to-Noise = • SNR depends on number of bits (number of quantization levels) assigned to signal • Every bit increases SNR by ~ 6 decibels CS 414 - Spring 2012

  9. Audio Data Rate • Data rate = best sample rate * quantization (bits per sample) * channel • Derived from • Nyquist Data Rate = 2*H log2N, where H is basic sampling rate, N is number of quantization levels • Compare rates for CD vs. mono audio • 8000 samples/second * 8 bits/sample * 1 channel= 8 kBytes / second • 44,100 samples/second * 16 bits/sample * 2 channel = 176 kBytes / second ~= 10MB / minute CS 414 - Spring 2012

  10. Integrating Aspects of Multimedia Audio/Video Presentation Playback Image/Video Capture Audio/Video Perception/ Playback Image/Video Information Representation Transmission Transmission Compression Processing Audio Capture Media Server Storage Audio Information Representation A/V Playback CS 414 - Spring 2012

  11. Human Visual System • Eyes, optic nerve, parts of the brain • Transforms electromagnetic energy

  12. Human Visual System • Image Formation • cornea, sclera, pupil,iris, lens, retina, fovea • Transduction • retina, rods, and cones • Retina has photosensitive receptors at back of eye • Processing • optic nerve, brain

  13. Contain photo-pigment Respond to low energy Enhance sensitivity Concentrated in retina, but outside of fovea One type, sensitive to grayscale changes Contain photo-pigment Respond to high energy Enhance perception Concentrated in fovea, exist sparsely in retina Three types, sensitive to different wavelengths Rods vs Cones (Responsible for us seeing brightness and color) Cones Rods CS 414 - Spring 2012

  14. Tri-stimulus Theory • 3 types of cones (6/7 Mil. of them) • Red = L cones, Green = M cones, Blue = S cones • Ratio differentiates for each person • E.g., Red (64%), Green (32%), rest S cones • E.g., L(50.6%), M(44.2%), rest S cones • Each type most responsive to a narrow band electro-magnetic waves • red and green absorb most energy, blue the least • Light stimulates each set of cones differently, and the ratios produce sensation of color CS 414 - Spring 2012

  15. Color and Visual System • Color refers to how we perceive a narrow band of electromagnetic energy • source, object, observer • Visual system transforms light energy into sensory experience of sight

  16. Color Perception (Color Theory) Hue Scale • Hue • Refers to pure colors • dominant wavelength of the light • Saturation • Perceived intensity of a specific color • how far color is from a gray of equal intensity • Brightness (lightness) • perceived intensity Original Saturation lightness CS 414 - Spring 2012 Source: Wikipedia

  17. Color Perception (Hue) Relation between “hue” of colors (spectrum of colors) with maximal saturation in HSV and HSL with Their corresponding RGB coordinates HSV = Hue, Saturation, Value (Lightness) HSL = Hue, Saturation, Lightness CS 414 - Spring 2012

  18. Digitalization of Images – Capturing and Processing CS 414 - Spring 2012

  19. Capturing Real-World Images W2 W1 r W3 F r= function of (W1/W3); s=function of (W2/W3) s Picture – two dimensional image captured from a real-world scene that represents a momentary event from the 3D spatial world CS 414 - Spring 2012

  20. Image Concepts - Sampling • An image is a function of intensity values over a 2D plane I(r,s) • Sample function at discrete intervals to represent an image in digital form • matrix of intensity values for each color plane • intensity typically represented with 8 bits • Sample points are called pixels CS 414 - Spring 2012

  21. Digital Image Sampling • Sample = pixel • Image Size (in pixels) • Image Size = Height x Width (in pixels) • 320x240 pixels • 640x480 pixels • 1920x1080pixels CS 414 - Spring 2012

  22. Digital Images - Quantization • Quantization = number of bits per pixel • Example: if we would sample and quantize standard TV picture (525 lines) by using VGA (Video Graphics Array), • video controller creates matrix 640x480pixels, and • each pixel is represented by 8 bit integer (256 discrete gray levels) CS 414 - Spring 2012

  23. Image Representations • Black and white image • single color plane with 2 bits • Grey scale image • single color plane with 8 bits • Color image • three color planes each with 8 bits • RGB, CMY, YIQ, etc. • Indexed color image • single plane that indexes a color table • Compressed images • TIFF, JPEG, BMP, etc. 4 gray levels 2gray levels

  24. Digital Image Representation (3 Bit Quantization) CS 414 - Spring 2012

  25. Color QuantizationExample of 24 bit RGB Image 24-bit Color Monitor CS 414 - Spring 2012

  26. Image Representation Example 24 bit RGB Representation (uncompressed) Color Planes

  27. Graphical Representation CS 414 - Spring 2012

  28. Image Properties (Color) CS 414 - Spring 2012

  29. Color Histogram CS 414 - Spring 2012

  30. Spatial domain refers to planar region of intensity values at time t Frequency domain think of each color plane as a sinusoidal function of changing intensity values refers to organizing pixels according to their changing intensity (frequency) Spatial and Frequency Domains CS 414 - Spring 2012

  31. Spatial Resolution and Brightness • Spatial Resolution (depends on: ) • Image size • Viewing distance • Brightness • Perception of brightness is higher than perception of color • Different perception of primary colors • Relative brightness: green:red:blue= 59%:30%:11% CS 414 - Spring 2011 Source: wikipedia

  32. Image Size (in Bits) • Image Size = Height x Width X Bits/pixel • Example: • Consider image 320x240 pixels with 8 bits per pixel • Image takes storage 7680 x 8 bits = 61440 bits or 7680 bytes CS 414 - Spring 2012

  33. Summary • Important Image Processing Functions (see Computer Vision/Image Processing classes) • Filtering • Edge detection • Image segmentation • Image recognition • Formatting • Conditioning • Marking • Grouping • Extraction • Matching • Image synthesis CS 414 - Spring 2012

More Related