1 / 75

The Image Formation Pipeline

The Image Formation Pipeline. Computer Graphics. Output. Image. Model. Synthetic Camera. (slides courtesy of Michael Cohen). Computer Vision. Output. Model. Real Scene. Real Cameras. (slides courtesy of Michael Cohen). Combined. Output. Image. Real Scene. Model. Synthetic

hue
Download Presentation

The Image Formation Pipeline

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The Image Formation Pipeline Computer Vision : CISC4/689

  2. Computer Graphics Output Image Model Synthetic Camera (slides courtesy of Michael Cohen) Computer Vision : CISC4/689

  3. Computer Vision Output Model Real Scene Real Cameras (slides courtesy of Michael Cohen) Computer Vision : CISC4/689

  4. Combined Output Image Real Scene Model Synthetic Camera Real Cameras (slides courtesy of Michael Cohen) Computer Vision : CISC4/689

  5. Abstract camera model - box with a small hole in it Pinhole cameras work in practice Pinhole cameras Computer Vision : CISC4/689

  6. Distant objects are smaller Computer Vision : CISC4/689

  7. Consequences: Parallel lines meet • There exist vanishing points Marc Pollefeys Computer Vision : CISC4/689

  8. The Effect of Perspective Computer Vision : CISC4/689

  9. Vanishing Points Vertical Line Vanishing Point • Parallel scene lines meet at a vanishing point in the image. Horizontal Line Vanishing Point Andrew C. Gallagher CRV 2005 Computer Vision : CISC4/689

  10. Vanishing points VP1 VP2 VP3 Different directions correspond to different vanishing points Marc Pollefeys Computer Vision : CISC4/689

  11. each set of parallel lines (=direction) meets at a different point The vanishing point for this direction Sets of parallel lines on the same plane lead to collinear vanishing points. The line is called the horizon for that plane If lines are parallel to an axis, corresponding VPs are called axis vanishing points. Good ways to spot faked images scale and perspective don’t work vanishing points behave badly supermarket tabloids are a great source. Vanishing points Computer Vision : CISC4/689

  12. Computer Vision : CISC4/689 Slide credit: David Jacobs

  13. Properties of Projection(Perspective) • Points project to points • Lines project to lines • Vanishing points for parallel lines • Parallel lines parallel to image plane donot converge • Closer objects appear bigger • Angles are not preserved • Degenerate cases • Line through focal point projects to a point. • Plane through focal point projects to line Computer Vision : CISC4/689

  14. Pinhole Camera Terminology Image plane Optical axis Principal point/ image center Focal length Camera center/ pinhole Camera point Image point Computer Vision : CISC4/689

  15. The equation of projection Computer Vision : CISC4/689

  16. Cartesian coordinates: We have, by similar triangles, that (x, y, z) -> (f x/z, f y/z, -f) Ignore the third coordinate, and get The equation of projection Computer Vision : CISC4/689

  17. Turn previous expression into HC’s HC’s for 3D point are (X,Y,Z,T) HC’s for point in image are (U,V,W) The camera matrix Computer Vision : CISC4/689

  18. Issue perspective effects, but not over the scale of individual objects collect points into a group at about the same depth, then divide each point by the depth of its group Adv: easy Disadv: wrong Weak perspective Computer Vision : CISC4/689

  19. Weak Perspective Projection Z Reduction of height by same amount even though they are at different distances. O -x Z Z f Computer Vision : CISC4/689

  20. The Equation of Weak Perspective(scaled Orthographic) • s is constant for all points. • Parallel lines no longer converge, they remain parallel. Computer Vision : CISC4/689 Slide credit: David Jacobs

  21. Generalization of Orthographic Projection When the camera is at a (roughly constant) distance from the scene, take m=1. Computer Vision : CISC4/689 Marc Pollefeys

  22. The projection matrix for orthographic projection Computer Vision : CISC4/689

  23. Pictorial Comparison Weak perspective Perspective Computer Vision : CISC4/689  Marc Pollefeys

  24. Summary: Perspective Laws • Perspective • Weak perspective • Orthographic Computer Vision : CISC4/689

  25. Pros and Cons of These Models • Weak perspective has simpler math. • Accurate when object is small and distant. • Most useful for recognition. • Pinhole perspective much more accurate for scenes. • Used in structure from motion. • When accuracy really matters, we must model the real camera • Use perspective projection with other calibration parameters (e.g., radial lens distortion) Computer Vision : CISC4/689 Slide credit: David Jacobs

  26. Affine cameras Computer Vision : CISC4/689

  27. Issue camera may not be at the origin, looking down the z-axis extrinsic parameters one unit in camera coordinates may not be the same as one unit in world coordinates intrinsic parameters - focal length, principal point, aspect ratio, angle between axes, etc. Camera parameters Note the matrix dimensions Computer Vision : CISC4/689

  28. Issues: what are intrinsic parameters of the camera? what is the camera matrix? (intrinsic+extrinsic) General strategy: view calibration object identify image points obtain camera matrix by minimizing error obtain intrinsic parameters from camera matrix Error minimization: Linear least squares easy problem numerically solution can be rather bad Minimize image distance more difficult numerical problem solution usually rather good, start with linear least squares Numerical scaling is an issue Camera calibration Computer Vision : CISC4/689

  29. Outline • Vector, matrix basics • 2-D point transformations • Translation, scaling, rotation, shear • Homogeneous coordinates and transformations • Homography, affine transformation Computer Vision : CISC4/689

  30. Notes on Notation • Vectors, points: x, v (assume column vectors) • Matrices: R, T • Scalars: x, a • Axes, objects: X, Y, O • Coordinate systems: W, C • Number systems: R, Z • Specials • Transpose operator: xT (as opposed to x0) • Identity matrix: Id • Matrices/vectors of zeroes, ones: 0, 1 Computer Vision : CISC4/689

  31. Block Notation for Matrices • Often convenient to write matrices in terms of parts • Smaller matrices for blocks • Row, column vectors for ranges of entries on rows, columns, respectively • E.g.: If A is 3 x 3 and : Computer Vision : CISC4/689

  32. 2-D Transformations • Types • Scaling • Rotation • Shear • Translation • Mathematical representation Computer Vision : CISC4/689

  33. 2-D Scaling Computer Vision : CISC4/689

  34. 2-D Scaling Computer Vision : CISC4/689

  35. 2-D Scaling sx 1 Horizontal shift proportional to horizontal position Computer Vision : CISC4/689

  36. 2-D Scaling sy 1 Vertical shift proportional to vertical position Computer Vision : CISC4/689

  37. 2-D Scaling Computer Vision : CISC4/689

  38. Matrix form of 2-D Scaling Computer Vision : CISC4/689

  39. 2-D Scaling Computer Vision : CISC4/689

  40. 2-D Rotation Computer Vision : CISC4/689

  41. µ 2-D Rotation Computer Vision : CISC4/689

  42. µ 2-D Rotation Computer Vision : CISC4/689

  43. µ Matrix form of 2-D Rotation (this is a counterclockwise rotation; reverse signs of sines to get a clockwise one) Computer Vision : CISC4/689

  44. µ Matrix form of 2-D Rotation Computer Vision : CISC4/689

  45. 2-D Shear (Horizontal) Computer Vision : CISC4/689

  46. 2-D Shear (Horizontal) Horizontal displacement proportional to vertical position Computer Vision : CISC4/689

  47. 2-D Shear (Horizontal) (Shear factorh is positive for the figure above) Computer Vision : CISC4/689

  48. 2-D Shear (Horizontal) Computer Vision : CISC4/689

  49. 2-D Shear (Vertical) Computer Vision : CISC4/689

  50. 2-D Translation Computer Vision : CISC4/689

More Related