Image and Video Processing

1. Image and Video Processing Dr. V. Këpuska

2. Light and Image Perception

3. Illumination • Quantification of human Psycho-visual sensation: • Peak spectral sensitivity of a human observer at 555 nm wavelength is normalized to unity. • Human spectral sensitivity drops to 4/10,000 relative to peak at the two ends of the spectrum at 400 nm and 735 nm. Veton Këpuska

4. Visible Spectrum • We percieve electromagnetic energy having wavelengths in the range 400-700 nm as visible light (slide taken from groups.csail.mit.edu/.../Lecture02/spectrum.gif ). Veton Këpuska

5. Brightness • Brightness is proportional to logarithm of the luminous flux as defined by Fecher’s Law: • Consider a point source which emits luminous flux along radial lines: Ω – measured in steradians Ω A r Veton Këpuska

6. Luminous Intensity • Luminous Intensity I at a point source is defined as • Thus, luminous flux incident on area dA from a source of intensity I is: Veton Këpuska

7. Illumination • The luminous flux per unit area falling on a surface is called illumination E and is given by: Veton Këpuska

8. Digital Image Representation

9. Digital Image Representation Image(y,x) ≡ Matrix(row, colum) Veton Këpuska

10. What is an Image • Collection of elements organized in a MATRIX form. • Each Matrix element represent smallest part of a picture element = pixel • Images are thus characterized by number of pixels • Larger the number of pixels the better the quality of image. • Larger the number of pixels the larger the storage space required to store the image. • More on this latter. Veton Këpuska

11. Representation of Pixels • What information is contained in pixels: • If the information is represented as a 4-bit code than there could be only 24 = 16 different representations. • Typically 8 bits are used to represent each pixel. This defines a 28 = 256 levels of representation. • What does this level represent: • Gray Scale • Color – there are several ways to represent color that gives raise to various standards discusses next. Veton Këpuska

12. Pixel Information and its Representation • Gray Scale Example produced with MATLAB: • Two dimensional example from wikipedia with 212=4096 gray scale levels Veton Këpuska

13. Pixel Information and its Representation • Monochromatic Scale • The idea behind Gray scale image representation is to represent the image in a monochromatic scale. • Defined as “quality” of a single “color” • Color • From theory of light and the theories how we perceive color information there are a number of standards that have been developed to represent color. • They all address the Chromaticity – defined as the quality of a color as determined by its "purity" and dominant wavelength. Veton Këpuska

14. Color and How it is Perceived • In the study of the perception of color, one of the first mathematically defined color spaces was the CIE XYZ color space (also known as CIE 1931 color space), created by the International Commission on Illumination (CIE) in 1931. • The human eye has receptors for short (S), middle (M), and long (L) wavelengths, also known as blue, green, and red receptors. That means that one, in principle, needs three parameters to describe a color sensation. A specific method for associating three numbers (or tristimulus values) with each color is called a color space: the CIE XYZ color space is one of many such spaces. However, the CIE XYZ color space is special, because it is based on direct measurements of the human eye, and serves as the basis from which many other color spaces are defined. • The CIE XYZ color space was derived from a series of experiments done in the late 1920s by W. David Wright (Wright 1928) and John Guild (Guild 1931). Their experimental results were combined into the specification of the CIE RGB color space, from which the CIE XYZ color space was derived. This article is concerned with both of these color spaces. Veton Këpuska

15. Color and How it is Perceived • Since the human eye has three types of color sensor that respond to different ranges of wavelengths, a full plot of all visible colors is a three-dimensional figure. • However, the concept of color can be divided into two parts: brightness and chromaticity. For example, the color white is a bright color, while the color grey is considered to be a less bright version of that same white. In other words, the chromaticity of white and grey are the same while their brightness differs. • The CIE XYZ color space was deliberately designed so that the Y parameter was a measure of the brightness or luminance of a color. The chromaticity of a color was then specified by the two derived parameters x and y which are functions of all three tristimulus values X, Y, and Z: Veton Këpuska

16. Color and How it is Perceived Veton Këpuska

17. Color Model • A color model is an abstract mathematical model describing the way colors can be represented as tuples of numbers, typically as three or four values or color components (e.g. RGB and CMYK are color models). Veton Këpuska

18. Pixel Information and its Representation • The HSV (Hue, Saturation, Value) model, also known as HSB (Hue, Saturation, Brightness), defines a color space in terms of three constituent components: • Hue, the color type (such as red, blue, or yellow): • Ranges from 0-360 (but normalized to 0-100% in some applications) • Saturation, the "vibrancy" of the color: • Ranges from 0-100% • Also sometimes called the "purity" by analogy to the colorimetric quantities excitation purity and colorimetric purity • The lower the saturation of a color, the more "grayness" is present and the more faded the color will appear, thus useful to define desaturation as the qualitative inverse of saturation • Value, the brightness of the color: • Ranges from 0-100% Veton Këpuska

19. HSV-HSB Color Model • Example of HSV Color Model Veton Këpuska

20. RGB • RGB is shorthand for Red, Green, Blue. • An RGB color space is any additive color space based on the RGB color model. • RGB is a convenient color model for computer graphics because the human visual system works in a way that is similar—though not quite identical—to an RGB color space. The most commonly used RGB color spaces are sRGB and Adobe RGB (which has a significantly larger gamut). Adobe has recently developed another color space called Adobe Wide Gamut RGB, which is even larger, in detriment to gamut density. • As of 2007, sRGB is by far the most commonly used RGB color space, particularly in consumer grade digital cameras, because it is considered adequate for most consumer applications, and its design simplifies previewing on the typical computer display. Adobe RGB is being built into more medium-grade digital cameras, and is favored by many professional graphic artists for its larger gamut. • RGB spaces are generally specified by defining three primary colors and a white point. In the table below the three primary colors and white points for various RGB spaces are given. The primary colors are specified in terms of their CIE 1931 color space chromaticity coordinates (x,y). Veton Këpuska

21. RGB • RGB Color Space Veton Këpuska

22. CMYK • It is possible to achieve a large range of colors seen by humans by combining cyan, magenta, and yellow transparent dyes/inks on a white substrate. These are the subtractiveprimary colors. Often a fourth black is added to improve reproduction of some dark colors. This is called "CMY" or "CMYK" color space. • The cyan ink will reflect all but the red light, the yellow ink will reflect all but the blue light and the magenta ink will reflect all but the green light. This is because cyan light is an equal mixture of green and blue, yellow is an equal mixture of red and green, and magenta light is an equal mixture of red and blue. Veton Këpuska

23. CMYK • CMYK Color Space: Veton Këpuska

24. Color Image Representation • Wide Screen Computer Monitor: • 1920x1080 pixels • 1,296,000≈1,3 M pixels • 32-bit RGB Representation • Total: 1,296,000x32x3 = 124,416,000 ≈124 Mbits or • 24,883,200 ≈ 25 MBytes Veton Këpuska

25. Video Standards • NTSC - NTSC is the analog television system in use in Canada, Japan, South Korea, the Philippines, the United States, and some other countries, mostly in the Americas (see map). It is named for the National Television Standards Committee, the U.S. standardization body that adopted it. • PAL was developed by Walter Bruch at Telefunken in Germany. The format was first unveiled in 1963, with the first broadcasts beginning in the United Kingdom and Germany in 1967. [1] Telefunken was later bought by the French electronics manufacturer Thomson. • SECAM Compagnie Générale de Télévision where Henri de France developed SECAM, historically the first European colour television standard. Veton Këpuska

26. Video Standards • The term "PAL" is often used informally to refer to a 625-line/50 Hz (576i, principally European) television system, and to differentiate from a 525-line/60 Hz (480i, principally North American/Central American/Japanese) "NTSC" system. Accordingly, DVDs are labelled as either "PAL" or "NTSC" (referring informally to the line count and frame rate) even though technically neither of them have encoded PAL or NTSC composite colour. • The NTSC format is used with the M format (see broadcast television systems), which consists of 29.97 interlaced frames of video per second. Each frame consists of 484 lines out of a total of 525 (the rest are used for sync, vertical retrace, and other data such as captioning). • PAL uses 625 lines, and so has a better picture quality. The NTSC system interlaces its scanlines, drawing odd-numbered scanlines in odd-numbered fields and even-numbered scanlines in even-numbered fields, yielding a nearly flicker-free image at its approximately 59.94 hertz (nominally 60 Hz/100.1%) refresh frequency. Veton Këpuska

27. Video Standards • NTSC’s refresh compares favorably to the 50 Hz refresh rate of the PAL and SECAM video formats used in Europe, where 50 Hz alternating current is the standard; flicker was more likely to be noticed when using these standards until modern PAL TV sets began using 100 Hz refresh rate to eliminate flicker. This produces a far more stable picture than native NTSC and PAL had, effectively displaying each frame twice. This did, at first, cause some motion problems, so it was not universally adopted until a few years ago. Interlacing the picture does complicate editing video, but this is true of all interlaced video formats, including PAL and SECAM. Veton Këpuska

28. Resolution • DVD images have 720 by 480 (NTSC) pixels or 720 by 576 (PAL) pixels Veton Këpuska

29. Image Processing Areas

30. Image Processing Areas • Acquisition, Storage and Retrieval • Digitization (sampling and quantization) • Image compression • Expansion of compressed images • Enhancement • Image equalization • Image enhancement • Image filtering • Image De-noising • Transformation • Translation • Scaling • Rotation • Object Recognition • Comparison and Abstraction • Classification Veton Këpuska

31. Geometric Transformations of an Image • Simple geometric transformations like translations, rotations and torsions are not trivial because of integer nature of the pixels Veton Këpuska

32. Image Transformations • Image Transformation can be done via Matrixes. • Matrixes transform vectors from one space to the other. • In geometry, an affine transformation or affine map (from the Latin, affinis, "connected with") between two vector spaces (strictly speaking, two affine spaces) consists of a linear transformation followed by a translation: Veton Këpuska

33. Image Transformations • This operation can be simplified by adopting the following notation for a generalized transformation matrix A and extension of input vector • This form is called homogenous coordinate system represention. Veton Këpuska

34. Examples of Transformation Matrixes • Translation: Veton Këpuska

35. Examples of Transformation Matrixes • Rotation: Veton Këpuska

36. Rotation Transformation Veton Këpuska

37. Scaling Transformation • Scaling Veton Këpuska

38. Shearing Transformation • Shear parallel to x axis Veton Këpuska

39. Shearing Transformation • Shear parallel to y axis Veton Këpuska

40. Reflection Transformation • Reflection: To reflect a vector about a line that goes through the origin, let (ux, uy) be a unit vector in the direction of the line; then: Veton Këpuska

41. Example Using “Lena” Veton Këpuska

42. Image Enhancement

43. Image Equalization • Histogram Based Equalization • Any gray-level modification technique is based on creating a mapping of the gray levels in the original image to the gray levels in the modified image. • Let x represent the original image as a gray-level. • Let y represent the modified image. y=T(x) Veton Këpuska

44. Histogram Flattening • Histogram Computation • PDF of the histogram for the image of the size of MxN • CDF from the PDF Veton Këpuska

45. Histogram Flattening • Transformation Mapping Veton Këpuska

46. Example: Lena Original Image pdf cdf Trans Veton Këpuska

47. Linear Filtering of Images • Definition of 2D-Linear Filtering: • It is the operation that it is applied to the image x(k,l) to obtain the image y(k,l) defined by: • The sequence h(k,l) dependent on to indexes that defines a filter, is called the Point Spread Function (PSF). Veton Këpuska

48. Linear Filtering of Images • As was the case with 1D (e.g., sequences of time) filtering • the convolution operation is linear and space-invariant. • h(k,l) is the equivalent of the impulse response of the one dimensional case. Veton Këpuska

49. Convolution – Multiplication Equivalence • If x, h and y are related by • The X,H and Y the Discrete Frequency Representations are related by • X,H and Y are 2D DFTs of x,h, and y respectively. Veton Këpuska

50. Separable Filters • A filter is said to be separable if its PSF has the following property: • In case of finite PSF represented by matrix h of elements h(k,l) and if hx and hy are the vectors with respective components hx(k) and hy(l) then the relation above is equivalent to: Veton Këpuska