Digital Image Processing Additional Material : Imaging Geometry 11 September 2006

1. Digital Image Processing Additional Material: Imaging Geometry 11 September 2006

2. Imaging Geometry: Perspective Transformation (X,Y,Z) = world coordinate Image plane y,Y x,X (X,Y,Z) l z,Z Lens center (x,y) (x,y,z) = Camera coordinate system l = focal length Eq. 1.1

3. Imaging Geometry: Perspective Transformation (cont.) Question: How can we project the real world object at (X,Y,Z) onto the image plane (such as photographic film)? Answer: Relation between camera coordinate (x,y,z) and world coordinate (X,Y,Z) are given by Eq. 1.2 Since on the image plane z is always zero, z=0, we consider only (x,y) while z is neglected.

4. Imaging Geometry: Perspective Transformation (cont.) Cartesian coordinate Homogeneous coordinate Equation 1.2 is not linear because of Z in the dividers so we introduce the homogeneous coordinate to solve this problem. k = nonzero constant To convert from the homogeneous coordinate wh to the Cartesian coordinate w, we divide the first 3 components of whby the fourth component.

5. Imaging Geometry: Perspective Transformation (cont.) The perspective transformation matrix for the homogeneous coordinate: Perspective transformation becomes: Eq. 1.3

6. Imaging Geometry: Perspective Transformation (cont.) From homogeneous coordinate We get camera coordinate in the image plane:

7. Imaging Geometry: Inverse Perspective Transformation Eq. 1.4 where

8. Inverse Perspective Transformation (cont.) For an image point (x0,y0), since on the image plane z=0, we have We get the world coordinate : Since the perspective transformation maps 3-D coordinates to 2-D Coordinates, we cannot get the inverse transform unless we have additional information.

9. Inverse Perspective Transformation (cont.) To find the solution, let We get Eq. 1.5

10. Inverse Perspective Transformation (cont.) From Eq. 1.5, Eq. 1.6 We get Substituting Eq. 1.6 into Eq.1.5, we get Eq. 1.7 Equations 1.7 show that inverse perspective transformation requires information of at least one component of the world coordinate of the point.

11. Inverse Perspective Transformation (cont.) (X1,Y1,Z1) (X2,Y2,Z2) (X3,Y3,Z3) (x,y) These equations: show that points on Line L in the world coordinate map to a single point in the image plane. Line L Image plane y,Y x,X l z,Z Lens center Points (X1,Y1,Z1) , (X2,Y2,Z2), and (X3,Y3,Z3) map to Point (x,y) in the image plane.

12. Stereo Imaging: How we get depth information from 2 eyes (x1,y1) y y Optical axis x x Image 1 Image 2 Left lens center B l (x2,y2) right lens center World point (X,Y,Z)

13. Stereo Imaging: How we get depth information from 2 eyes (cont.) Problem: we know camera coordinates of the object on left and right image planes (x1,y1) and (x2,y2) and want to how far from the camera the object is located. Note: when y-axis is parallel to the ground, we have y1 = y2 X Image 1 (x1,y1) Z l Origin of world coordinate system w B Image 2 l (x2,y2) Plane of constant Z

14. Stereo Imaging: How we get depth information from 2 eyes (cont.) 1. From the inverse perspective transform, we compute X1 and X2: 2. Z1 and Z2 must be equal, we get 3. Since left and right lenses are separated by distance B, we have 4. From 1, 2 and 3, we get Solving Z yields

15. Stereo Imaging: How we get depth information from 2 eyes (cont.) We can locate the object if we know positions of the object in left and right image planes using Equation: Question: While the equation is so simple but why it is very difficult to built an automatic stereo vision system that can reconstruct 3-D scene from images obtained from 2 cameras? Answers: for a computer, locating the corresponding points on left and right images is the most difficult task.

16. Imaging Geometry : Affine Transformations 1. Translation 2. Scaling 3. Rotating

17. Image Geometry: Translation of Object Displace the object by vector (X0,Y0,Z0)with respect to its old position. (X+X0,Y+Y0,Z+Z0) Z (X0,Y0,Z0) (X,Y,Z) Y X

18. Image Geometry: Translation of Frame Z* Y * X * Translate the origin point of the frame by (X0,Y0,Z0) with respect to the old frame Z (X0,Y0,Z0) Y (0,0,0) X The object still stays at the same position. Only the frame is moved.

19. Image Geometry: Scaling Scale by factors Sx, Sy, Sz along X, Y, and Z axes. E D G F e d g f C c b a B A Note: Origin point is unchanged.

20. Image Geometry: Rotating an object about X-axis Rotate an object about X-axis by qx in a counterclockwise direction. Z Y qx D X=X * Note : In this case the object is moved. Only y and z are changed while x stills the same.

21. Image Geometry: Rotating a frame about X-axis Z* Y * Rotate the frame about X-axis by qx in a counterclockwise direction. Z qx Y D X=X * Note : In this case the object is not moved. The frame is rotated instead.

22. Image Geometry: Rotating an object about Y-axis Rotate an object about Y-axis by qy in a counterclockwise direction. Z D qy Y= Y * X Note : In this case the object is moved. Only x and z are changed while y stills the same.

23. Image Geometry: Rotating a frame about Y-axis Z* X * Rotate the frame about Y-axis by qy in a counterclockwise direction. Z qy D Y= Y * X Note : In this case the object is not moved. The frame is rotated instead.

24. Image Geometry: Rotating an object about Z-axis Rotate an object about Z-axis by qz in a counterclockwise direction. D Z qz Y X Note : In this case the object is moved. Only x and y are changed while z stills the same.

25. Image Geometry: Rotating a frame about Z-axis Rotate the frame about Z-axis by qz in a counterclockwise direction. D Z= Z* Y * Y qz X X * Note : In this case the object is not moved. The frame is rotated instead.

26. z Camera Coordinate System Z x y Y World Coordinate System X How to compute a point on an image plane from the world coordinate Problem: we know the location of the object and want to know where it will be projected on the film (image plane). Answer: 1. Transform the world coordinate to the camera coordinate 2. Perform the perspective transformation

27. Y y Image plane (z=0) (X,Y,Z) X x l Z z Lens center (x,y) (x,y,z) = Camera coordinate, (X,Y,Z) = World coordinate How to compute a point on an image plane from the world coordinate (cont.) Before using the perspective transformation, the world axes X-Y-Z must coincide with the camera axes x-y-z, (we need some transformations).

28. Compute the camera coordinate fromthe world coordinate z 3 Camera Coordinate System Z r 2 x 4 y w0 1 Y Gimbal center X World Coordinate System Steps from Gonzalez’s book 1. Translate* by w0 2. Pan the camera* (rotate about Z-axis) 3. Tilt the camera* (rotate about X-axis) 4. Translate* by z = r 5. Compute the perspective Tr. Note : *perform on the frame

29. Compute the camera coordinate from the world coordinate (cont.) World coordinate Perspective tr. Translate to the gimbal center w0 Translate to the image plane Center by z = r Rotate by Pan (qz) and Tilt (qx) Formula from Gonzalez’s book Camera coordinate

30. Compute the camera coordinate from the world coordinate (cont.) z 3 Camera Coordinate System Z 2 x 4 w1 1 y Y X World Coordinate System General case 1. Translate* by w1 2. Pan the camera* 3. Tilt the camera* 4. Twist the camera* (rotate about Y-axis) Gimbal center 5. Compute the perspective Tr. Note : *perform on the frame

31. Compute the camera coordinate from the world coordinate (cont.) World coordinate Perspective tr. Rotate by Pan (qz), Tilt (qx), Twist (qy) Translate to the image plane center w1 General case Camera coordinate