Lecture 23: Imaging, Color CAP 5415

1. Lecture 23: Imaging, Color CAP 5415

2. Working with Intensity Values • Simple Non-Linearities can have a great effect on image appearance

3. Working with Intensity Values • Simple Non-Linearities can have a great effect on image appearance

4. Working with Intensity Values • Simple Non-Linearities can have a great effect on image appearance

5. Working with Intensity Values • This is not just linear scaling

6. Working with Intensity Values • Can go the other way

7. Gamma correction • Model this non-linearity using this type of curve:

8. Cameras and Gamma • Consumer cameras put a similar non-linearity into photos • Makes images look better • You (as a vision person) don't want nice-looking images • You want ACCURATE images • Need to calibrate camera to remove gamma

9. Method 1: Use a target

10. Method 2: Use multiple images of same scene

11. Which leads to a second problem • Most camera only give you 8 bits of precision • Hard to capture both bright and dim regions • Solution: • Use multiple images to create a High Dynamic Range Image • Can be solved as a least-squares problem if you account for saturation and dimness

12. Now that we can get accurate intensities, what about color? • Engineering problem: • I have sensor that records the amount of light at different pixels • How do I get a color image instead of a black and white image?

13. Solutions • Three sensors • One sensor with a color mask • Each pixel records one wavelength • A common pattern for the mask is the Bayer pattern:

14. Mosaicing • So, if I took a picture of this edge • My sensor would record this image

15. Demosaicing • I have 1 color at each pixel • I need three • Easy solution: Interpolate +

16. Problem! This smooths across the edge • Because the different pixels are used to red and green, the smoothing may be different +

17. Result: Color Fringing

18. Color Fringing (Results from Brainard et al)

19. Fast Solution • The fringing occurs when the correlation between the color channels is incorrectly estimated • One measure of this correlation is the color difference • Can fix errors using median filtering

20. Simple Demosaicing Algorithm (Freeman) • Use linear interpolation to get first estimate • Compute difference images between color channels • Median filter these difference images • Use filtered difference images to reconstruct

21. (Slide by Freeman)

22. (Slide by Forsyth)

23. (Slide by Forsyth)

24. (Slide by Forsyth)

25. Color matching experiments - I • Show a split field to subjects; one side shows the light whose color one wants to measure, the other a weighted mixture of primaries (fixed lights). • Each light is seen in film color mode. (Slide by Forsyth)

26. Color matching experiments - II • Many colors can be represented as a mixture of A, B, C • write M=a A + b B + c C where the = sign should be read as “matches” • This is additive matching. • Gives a color description system - two people who agree on A, B, C need only supply (a, b, c) to describe a color. (Slide by Forsyth)

27. Subtractive matching • Some colors can’t be matched like this: instead, must write M+a A = b B+c C • This is subtractive matching. • Interpret this as (-a, b, c) • Problem for building monitors: • Choose R, G, B such that positive linear combinations match a large set of colors (Slide by Forsyth)

28. The principle of trichromacy • Experimental facts: • Three primaries will work for most people if we allow subtractive matching • Exceptional people can match with two or only one primary. • This could be caused by a variety of deficiencies. • Most people make the same matches. • There are some anomalous trichromats, who use three primaries but make different combinations to match. (Slide by Forsyth)

29. Grassman’s Laws • For colour matches made in film colour mode: • symmetry: U=V <=>V=U • transitivity: U=V and V=W => U=W • proportionality: U=V <=> tU=tV • additivity: if any two (or more) of the statements U=V, W=X, (U+W)=(V+X) are true, then so is the third • These statements are as true as any biological law. They mean that color matching in film color mode is linear.

30. RGB: primaries are monochromatic energies are 645.2nm, 526.3nm, 444.4nm. CIE XYZ: Primaries are imaginary, but have other convenient properties. Color coordinates are (X,Y,Z), where X is the amount of the X primary, etc. Usually draw x, y, where x=X/(X+Y+Z) y=Y/(X+Y+Z) Linear color spaces • A choice of primaries yields a linear color space --- the coordinates of a color are given by the weights of the primaries used to match it. • Choice of primaries is equivalent to choice of color space.

31. CIE xy

32. HSV hexcone

33. Perceptually Linear Color Spaces • When doing thing like segmentation or edge detection, we would like to compare colors in a way that is similar to how humans judge similarity • A perceptually-linear space tries to match the numerical distances to our perceptual differences.

34. CIE L*a*b* • Can be computed from CIE XYZ Space • What about RGB? • Need a standard for what R,G,and B are • sRGB or Adobe RGB are examples

35. RGB cannot express all possible colors