CSCE 641: Computer Graphics Image Formation & Plenoptic Function

1. CSCE 641: Computer GraphicsImage Formation & Plenoptic Function Jinxiang Chai

2. Meeting Schedule • MW (1:40pm-2:55pm)?

3. Are They Images?

4. Outline • Image representation • Color representation • Pin-hole Camera • Projection matrix • Plenoptic function

5. Outline • Image representation • Color representation • Pin-hole Camera • Projection matrix • Plenoptic function

6. Image Representation An image is a 2D rectilinear array of Pixels - A width X height array where each entry of the array stores a single pixel

7. Image Representation A pixel stores color information Luminance pixels - gray-scale images (intensity images) - 0-1.0 or 0-255 - 8 bits per pixel Red, green, blue pixels (RGB) - Color images - Each channel: 0-1.0 or 0-255 - 24 bits per pixel

8. Image Representation An image is a 2D rectilinear array of Pixels - A width X height array where each entry of the array stores a single pixel - Each pixel stores color information (255,255,255)

9. Outline • Image representation • Color representation • Pin-hole Camera • Projection matrix • Plenoptic function

10. Color Representation • Why do we use RGB to encode pixel color? • Can we use RGB to represent all colors? • What are other color representations?

11. Human Vision Model of human vision

12. Human Vision Model of human vision • Vision components: • Incoming light • Human eye

13. Electromagnetic Spectrum Visible light frequencies range between: • Red: 4.3X1014 hertz (700nm) • Violet: 7.5X1014 hertz (400nm)

14. Visible Light The human eye can see “visible” light in the frequency between 400nm-700nm

15. Visible Light The human eye can see “visible” light in the frequency between 400nm-700nm 400nm 700nm

16. Visible Light The human eye can see “visible” light in the frequency between 400nm-700nm 400nm 700nm • Not strict boundary • Some colors are absent (brown, pink)

17. Spectral Energy Distribution Three different types of lights

18. Spectral Energy Distribution The six spectra below look the same purple to normal color-vision people

19. Color Representation? Why not all ranges of light spectrum are perceived? So how to represent color? - unique - compact - work for as many visible lights as possible 400nm 700nm

20. Human Vision Photoreceptor cells in the retina: - Rods - Cones

21. Light Detection: Rods and Cones Rods: -120 million rods in retina -1000X more light sensitive than Cones - Discriminate B/W brightness in low illumination - Short wave-length sensitive Cons: - 6-7 million Cones in the retina - Responsible for high-resolution vision - Discriminate Colors - Three types of color sensors (64% red, 32%, 2% blue) - Sensitive to any combination of three colors

22. Tristimulus of Color Theory Spectral-response functions of each of the three types of cones

23. Tristimulus of Color Theory Spectral-response functions of each of the three types of cones Can we use them to match any spectral color?

24. Tristimulus of Color Theory Spectral-response functions of each of the three types of cones Color matching function based on RGB - any spectral color can be represented as a linear combination of these primary colors

25. Tristimulus of Color Theory Spectral-response functions of each of the three types of cones Color matching function based on RGB - any spectral color can be represented as a linear combination of these primary colors

26. Tristimulus of Color Theory Spectral-response functions of each of the three types of cones Color matching function based on RGB - any spectral color can be represented as a linear combination of these primary colors

27. Tristimulus Color Theory So, color is psychological - Representing color as a linear combination of red, green, and blue is related to cones, not physics - Most people have the same cones, but there are some people who don’t – the sky might not look blue to them (although they will call it “blue” nonetheless) - But many people (mostly men) are colorblind, missing 1,2 or 3 cones (can buy cheaper TVs)

28. Additive and Subtractive Color RGB color model CMY color model White: [0 0 0]T Green: [1 0 1]; White: [1 1 1]T Green: [0 1 0]; Complementary color models: R=1-C; G = 1-M; B=1-Y;

29. RGB Color Space blue green red RGB cube • Easy for devices • Can represent all the colors? • Where is brightness, hue and saturation?

30. Tristimulus • Since 3 different cones, the space of colors is 3-dimensional. • We need a way to describe color within this 3 dimensional space. • We want something that will let us describe any visible color with additive combination…

31. The CIE XYZ system • CIE – Comission Internationale de l’Eclairage - International Commission on Illumination - Sets international standards related to light • Defined the XYZ color system as an international standard in 1931 • X, Y, and Z are three Primary colors. - imaginary colors - all visible colors can be defined as an additive combination of these three colors. - defines the 3 dimensional color space

32. Chromaticity Diagram • Project the X+Y+Z=1 slice along the Z-axis • Chromaticity is given by the x, y coordinates

33. CIE Perceptual Space Which colors can RGB monitor displays?

34. Monitor/Print/Scanner Gamut

35. HSV Color Model Perceptually appropriate: - Hue: the color type (0-360 deg) - Saturation: the intensity of the color (0-100%) - Brightness: the brightness of color (0-100%) Nonlinear transform between the HSV and RGB space

36. Outline • Image representation • Color representation • Pin-hole Camera • Projection matrix • Plenoptic Function

37. How Do We See the World? Let’s design a camera: idea 1: put a piece of film in front of camera Do we get a reasonable picture?

38. Pin-hole Camera • Add a barrier to block off most of the rays • This reduces blurring • The opening known as the aperture • How does this transform the image?

39. Camera Obscura • The first camera • Known to Aristotle • Depth of the room is the focal length • Pencil of rays – all rays through a point

40. Camera Obscura How does the aperture size affect the image?

41. Shrinking the Aperture • Why not make the aperture as small as possible? • Less light gets through • Diffraction effects…

42. Shrink the Aperture: Diffraction A diffuse circular disc appears!

43. Shrink the Aperture

44. The Reason of Lenses

45. “circle of confusion” Adding A Lens • A lens focuses light onto the film • There is a specific distance at which objects are “in focus” • other points project to a “circle of confusion” in the image • Changing the shape of the lens changes this distance

46. Changing Lenses 50 mm 28 mm 70 mm 210 mm

47. Outline • Image representation • Color representation • Pin-hole Camera • Projection matrix • Plenoptic Function

48. Projection Matrix • What’s the geometric relationship between 3D objects and 2D images?

49. Modeling Projection: 3D->2D The coordinate system • We will use the pin-hole model as an approximation • Put the optical center (Center Of Projection) at the origin • Put the image plane (Projection Plane) in front of the COP • The camera looks down the negative z axis

50. Modeling Projection: 3D->2D Projection equations • Compute intersection with PP of ray from (x,y,z) to COP • Derived using similar triangles (on board)