3-D Computer Vision CSc 83020

1. 3-D Computer VisionCSc 83020 Image Formation and Optics

2. Image: 2D projection of a 3D scene. We need to understand Geometric & Radiometric relations between the scene and its image. Image Formation & Optics

3. Topics: • Pinhole & Perspective Projection. • Image Formation using Lenses. • Lens Related Issues. • Image Formation in the Eye. • Our Visual World.

4. Pinhole & the Perspective Projection (x,y) SCREEN SCENE Is there an image being formed on the screen?

5. Image plane Image Pinhole Object Pinhole camera Pinhole Camera • “Camera obscura” – known since antiquity

6. Perspective Camera From Trucco & Verri r’ r (x,y,z) (X,Y,Z) Center of Projection r =[x,y,z]T r’=[X,Y,Z]T r/f=r’/Z x=f * X/Z y=f * Y/Z z=f f: effective focal length: distance of image plane from O.

7. Magnification From Trucco & Verri (x,y) (X,Y,Z) d Center of Projection d’ (x+dx,y+dy) (X+dX,Y+dY,Z) (x+dx)/f=(X+dX)/Z (y+dy)/z=(Y+dY)/Z dx/f=dX/Z dy/f=dY/Z x/f=X/Z y/f=Y/Z =>

8. Magnification From Trucco & Verri (x,y) (X,Y,Z) d Center of Projection d’ (x+dx,y+dy) (X+dX,Y+dY,Z) Magnification: |m|=||d’||/||d||=|f/Z| or m=f/Z m is negative when image is inverted…

9. Magnification • Area(image)/Area(scene)=? • m can be assumed to be CONSTANT if range of scene depth (ΔZ) is much smaller than average scene depth (Z).

10. Implications For Perception* Same size things get smaller, we hardly notice… Parallel lines meet at a point… * A Cartoon Epistemology: http://cns-alumni.bu.edu/~slehar/cartoonepist/cartoonepist.html

11. Vanishing Points (from NALWA)

12. Consequences: Parallel lines meet • There exist vanishing points Marc Pollefeys

13. Vanishing points VPL H VPR VP2 VP1 Different directions correspond to different vanishing points VP3 Marc Pollefeys

14. Question • How many vanishing points are there in an image?

15. Approximations • Linear approximation to perspective equations. • Orthographic: (m=1 => x=X, y=Y).

16. Approximations • Linear approximation to perspective equations. • Weak-Perspective: m is CONSTANT. x=f*X/Z  f*X/Zavg(Zavg average distance of points from camera) y=f*Y/Z  f*Y/Zavg • Possible when Zavg is much smaller than ΔZ (relative distance of points along the optical axis).

17. Weak-Perspective Cont. From Trucco & Verri OBJECT POINTS Zavg Zavg: average distance of points along the optical axis.

18. Approximations Weak Perspective Para Perspective Ioannis Stamos – CSc 83020 Spring 2007

19. Pictorial Comparison Weak perspective Perspective  Marc Pollefeys

20. Problems with Pinholes • Pinhole size (aperture) must be small. • The smaller the size, the less light goes through. • If pinhole is comparable to wavelength λ of light DIFFRACTION effects blur image. • Pinhole diameter d=2*sqrt(f*λ) for sharp images: If f=50mm and λ=600nm (red light) then d=0.36mm.

21. Pinhole cameras

22. Lenses Used to avoid problems associated with pinholes. Ideal Lens: Same projection, but gathers more light! From Trucco & Verri f: point of convergence of rays that come from infinity.

23. Thin Lens: Projection optical axis Image plane f z Spherical lense surface: Parallel rays are refracted to single point

24. Thin Lens: Projection optical axis Image plane f f z Spherical lense surface: Parallel rays are refracted to single point

25. Thin Lens: Properties • Any ray entering a thin lens parallel to the optical axis must go through the focus on other side • Any ray entering through the focus on one side will be parallel to the optical axis on the other side

26. Lenses Used to avoid problems associated with pinholes. Ideal Lens: Same projection, but gathers more light! Ray of light From Trucco & Verri Optical Axis Ioannis Stamos – CSc 83020 Spring 2007

27. Lenses Gaussian Lens Formula for thin lenses: 1/Ž + 1/ž = 1/f Ž=Z+f, ž=z+f f: focal length of lens: ability to bend light Ray of light Optical Axis From Trucco & Verri Example: if f=50mm, Ž=300mm, then image distance ž =60mm.

28. Blur Circle (Defocus) IMAGE PLANE APERTURE Blur Circle w/ diameter b P p d (aperture) OPTICAL AXIS Ž ž ž’ Ž’

29. Blur Circle (Defocus) IMAGE PLANE APERTURE Blur Circle w/ diameter b P p d OPTICAL AXIS 3D SCENE ž’ Ž’

30. Blur Circle (Defocus) IMAGE PLANE APERTURE Blur Circle w/ diameter b OPTICAL AXIS 3D SCENE Ž ž ž’ Ž’

31. Blur Circle (Defocus) IMAGE PLANE APERTURE Blur Circle w/ diameter b P p d (aperture) OPTICAL AXIS Ž ž ž’ Ž’

32. b=? 1/ ž+1/ Ž=1/f 1/ ž’+1/ Ž’=1/f ž=Ž*f/(Ž-f) ž’=Ž’*f/(Ž’-f) (ž’- ž)=[f/(Ž’-f)]*[f/(Ž-f)]*(Ž- Ž’) Blur Circle Diameter b= | ž’-ž | d / ž’ (from similar triangles) --------------------------------------------------------------------------- Depth of Field Range of object distances (Ž- Ž’) over which image is “sufficiently well” focused. i.e. b is less than resolution of imaging sensor. Note that b is proportional to d (aperture).

33. Aperture & DOF d= (From KODAK) Ioannis Stamos – CSc 83020 Spring 2007

34. Blur Circle (Defocus) IMAGE PLANE APERTURE Blur Circle w/ diameter b P p d (aperture) OPTICAL AXIS Ž ž ž’ Ž’ Ioannis Stamos – CSc 83020 Spring 2007

35. Focusing • Defocused image can be made focused by: • Moving image plane. • Moving the lens. • Both as a single unit. Ioannis Stamos – CSc 83020 Spring 2007

36. Two Lens System From Shree Nayar’s notes Ž2 Ž1 ž2 ž1 Magnification: m=x’’/x=(i2/o2)*(i1/o1) Zooming: Varying magnification without moving object or image plane. Example: Move LENS2 to change m and then move LENS1 and LENS2 together to re-focus. ZOOMING=CHANGING EFFECTIVE FOCAL LENGTH

37. Thick Lens From Horn Ioannis Stamos – CSc 83020 Spring 2007

38. Vignetting From Horn

39. Vignetting Effect: Darkens pixels near the image boundary

40. From Shree Nayar’s notes Ioannis Stamos – CSc 83020 Spring 2007

41. Distortion magnification/focal length different for different angles of inclination pincushion (tele-photo) barrel (wide-angle) Can be corrected! (if parameters are know) Marc Pollefeys

42. Chromatic Aberration rays of different wavelengths focused in different planes cannot be removed completely Marc Pollefeys

43. Image Formation in the Eye • Optics in the Eye: Iris, Lens, Retina… • Defects in the Eye’s Lens: • Myopia (Near-Sighted) • Hyperopia (Far-Sighted) • Accomodation (Focusing)

44. Ioannis Stamos – CSc 83020 Spring 2007

45. THE HUMAN EYE! Ioannis Stamos – CSc 83020 Spring 2007

47. Ioannis Stamos – CSc 83020 Spring 2007

49. Ioannis Stamos – CSc 83020 Spring 2007

50. Our Visual World • Image Formation: 3D => 2D • Can we recover 3D Scene from 2D Image? • We live in a special world! • Medium (Air): Transparent & Homogeneous. • Objects: Opaque & Reflective. • We need to recover surfaces, not volumes. • Is one image enough?