Recap from Friday

1. Recap from Friday • Pinhole camera model • Perspective projections • Lenses and their flaws • Focus • Depth of field • Focal length and field of view

2. What is wrong with this picture?

3. What is wrong with this picture?

4. Capturing Light… in man and machine Many slides by Alexei A. Efros CS 195g: Computational Photography James Hays, Brown, Spring 2010

5. Image Formation Digital Camera Film The Eye

6. Digital camera • A digital camera replaces film with a sensor array • Each cell in the array is light-sensitive diode that converts photons to electrons • Two common types • Charge Coupled Device (CCD) • CMOS • http://electronics.howstuffworks.com/digital-camera.htm Slide by Steve Seitz

7. Sensor Array CMOS sensor

8. Sampling and Quantization

9. Interlace vs. progressive scan http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz

10. Progressive scan http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz

11. Interlace http://www.axis.com/products/video/camera/progressive_scan.htm Slide by Steve Seitz

12. The Eye • The human eye is a camera! • Iris - colored annulus with radial muscles • Pupil - the hole (aperture) whose size is controlled by the iris • What’s the “film”? • photoreceptor cells (rods and cones) in the retina Slide by Steve Seitz

13. The Retina

14. Light Retina up-close

15. Two types of light-sensitive receptors Cones cone-shaped less sensitive operate in high light color vision Rods rod-shaped highly sensitive operate at night gray-scale vision © Stephen E. Palmer, 2002

16. Rod / Cone sensitivity The famous sock-matching problem…

17. Distribution of Rods and Cones Night Sky: why are there more stars off-center? © Stephen E. Palmer, 2002

18. Eye Movements • Saccades • Microsaccades • Ocular microtremor (OMT)

19. Electromagnetic Spectrum Human Luminance Sensitivity Function http://www.yorku.ca/eye/photopik.htm

20. Visible Light Why do we see light of these wavelengths? …because that’s where the Sun radiates EM energy © Stephen E. Palmer, 2002

21. The Physics of Light Any patch of light can be completely described physically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm. © Stephen E. Palmer, 2002

22. The Physics of Light Some examples of the spectra of light sources © Stephen E. Palmer, 2002

23. Yellow Blue Purple Red 400 700 400 700 400 700 400 700 The Physics of Light Some examples of the reflectance spectra of surfaces % Photons Reflected Wavelength (nm) © Stephen E. Palmer, 2002

24. mean area variance The Psychophysical Correspondence There is no simple functional description for the perceived color of all lights under all viewing conditions, but …... A helpful constraint: Consider only physical spectra with normal distributions © Stephen E. Palmer, 2002

25. # Photons Wavelength The Psychophysical Correspondence Mean Hue © Stephen E. Palmer, 2002

26. # Photons Wavelength The Psychophysical Correspondence Variance Saturation © Stephen E. Palmer, 2002

27. # Photons Wavelength The Psychophysical Correspondence Area Brightness © Stephen E. Palmer, 2002

28. Physiology of Color Vision Three kinds of cones: • Why are M and L cones so close? • Why are there 3? © Stephen E. Palmer, 2002

29. More Spectra metamers

30. Color Sensing in Camera (RGB) • 3-chip vs. 1-chip: quality vs. cost • Why more green? Why 3 colors? http://www.cooldictionary.com/words/Bayer-filter.wikipedia Slide by Steve Seitz

31. Practical Color Sensing: Bayer Grid • Estimate RGBat ‘G’ cells from neighboring values http://www.cooldictionary.com/words/Bayer-filter.wikipedia Slide by Steve Seitz

32. RGB color space • RGB cube • Easy for devices • But not perceptual • Where do the grays live? • Where is hue and saturation? Slide by Steve Seitz

33. HSV • Hue, Saturation, Value (Intensity) • RGB cube on its vertex • Decouples the three components (a bit) • Use rgb2hsv() and hsv2rgb() in Matlab Slide by Steve Seitz

34. Project #1 • How to compare R,G,B channels? • No right answer • Sum of Squared Differences (SSD): • Normalized Correlation (NCC):

35. Image half-sizing This image is too big to fit on the screen. How can we reduce it? How to generate a half- sized version?

36. Image sub-sampling 1/8 1/4 • Throw away every other row and column to create a 1/2 size image • - called image sub-sampling Slide by Steve Seitz

37. Image sub-sampling 1/2 1/4 (2x zoom) 1/8 (4x zoom) Aliasing! What do we do? Slide by Steve Seitz

38. Gaussian (lowpass) pre-filtering G 1/8 G 1/4 Gaussian 1/2 • Solution: filter the image, then subsample • Filter size should double for each ½ size reduction. Why? Slide by Steve Seitz

39. Subsampling with Gaussian pre-filtering Gaussian 1/2 G 1/4 G 1/8 Slide by Steve Seitz

40. Compare with... 1/2 1/4 (2x zoom) 1/8 (4x zoom) Slide by Steve Seitz

41. Gaussian (lowpass) pre-filtering G 1/8 G 1/4 Gaussian 1/2 • Solution: filter the image, then subsample • Filter size should double for each ½ size reduction. Why? • How can we speed this up? Slide by Steve Seitz

42. Image Pyramids • Known as a Gaussian Pyramid [Burt and Adelson, 1983] • In computer graphics, a mip map [Williams, 1983] • A precursor to wavelet transform Slide by Steve Seitz

43. A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose Figure from David Forsyth

44. What are they good for? • Improve Search • Search over translations • Like project 1 • Classic coarse-to-fine strategy • Search over scale • Template matching • E.g. find a face at different scales • Pre-computation • Need to access image at different blur levels • Useful for texture mapping at different resolutions (called mip-mapping)

45. Gaussian pyramid construction filter mask • Repeat • Filter • Subsample • Until minimum resolution reached • can specify desired number of levels (e.g., 3-level pyramid) • The whole pyramid is only 4/3 the size of the original image! Slide by Steve Seitz