1 / 177

Computer and Robot Vision II

Computer and Robot Vision II. Chapter 15 Motion and Surface Structure from Time Varying Image Sequences. Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw 指導教授 : 傅楸善 博士. 15.1 Introduction.

eagan
Download Presentation

Computer and Robot Vision II

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Computer and Robot Vision II Chapter 15Motion and Surface Structure from Time Varying Image Sequences Presented by: 傅楸善 & 王林農 0917 533843 r94922081@ntu.edu.tw 指導教授: 傅楸善 博士

  2. 15.1 Introduction • Motion analysis involves estimating the relative motion of objects with respect to each other and the camera given two or more perspective projection images in a time sequence. DC & CV Lab. CSIE NTU

  3. 15.1 Introduction (cont’) • Real-world applications: industrial automation and inspection, robot assembly, autonomous vehicle navigation, biomedical engineering, remote sensing, general 3D-scene understanding DC & CV Lab. CSIE NTU

  4. 15.1 Introduction (cont’) • object motion and surface structure recovery from: • observed optic flow • point correspondences DC & CV Lab. CSIE NTU

  5. 15.2 The Fundamental Optic Flow Equation • (x, y, z): 3D point on moving rigid body • (u, v): perspective projection on the image plane • f: camera constant • (u, v): velocity of the point (u, v) . . DC & CV Lab. CSIE NTU

  6. 15.2 The Fundamental Optic Flow Equation (cont’) • take time derivatives of both sides • yields the fundamental optic flow equation: . . . . . . . . . . . DC & CV Lab. CSIE NTU

  7. 15.2 The Fundamental Optic Flow Equation (cont’) • general solution: (λ is a free variable) . . . . . DC & CV Lab. CSIE NTU

  8. 15.2.1 Translational Motion • Known: • N-point optic flow field: • Unknown: • corresponding unknown 3D points: • all points moving with same but unknown velocity • (x, y, z) • can be solved up to a multiplicative constant . . . . . DC & CV Lab. CSIE NTU

  9. 15.2.2 Focus of Expansion and Contraction • Known: • 3D motion is translational • one 2D projected point (u, v) has no motion: • thus translational motion is in a direction along the ray of sight . . DC & CV Lab. CSIE NTU

  10. 15.2.2 Focus of Expansion and Contraction (cont’) • focus of expansion (FOE): if 3D point field moving toward camera • FOE: motion-field vectors radiate outward from that point • focus of contraction (FOC): if 3D point field moving away from camera • FOC: vectors radiate inward toward diametrically opposite point flow pattern of the motion field of a forward-moving observer DC & CV Lab. CSIE NTU

  11. DC & CV Lab. CSIE NTU

  12. 15.2.3 Moving Line Segment • Known: • fixed distance between two unknown 3D points • translational motion with common velocity (x, y, z) • corresponding optic flow: . . . DC & CV Lab. CSIE NTU

  13. 15.2.3 Moving Line Segment (cont’) • Unknown: • : two unknown 3D points • common velocity: (x, y, z) . . . DC & CV Lab. CSIE NTU

  14. 15.2.3 Moving Line Segment (cont’) • From the perspective projection equations: • From the optic flow equation: DC & CV Lab. CSIE NTU

  15. 15.2.3 Moving Line Segment (cont’) • From the known length of the line segment: • The optic flow equation (15.9) permits us to obtain a least squares solution for z in terms of z1 and z2, from DC & CV Lab. CSIE NTU

  16. 15.2.3 Moving Line Segment (cont’) • We obtain • Substituting this back into the equation, we can solve z2 in terms of z1:z2=kz1 DC & CV Lab. CSIE NTU

  17. 15.2.3 Moving Line Segment (cont’) • Substitute the relations for (x1, y1, z1) from equations into Eq.(15.10) to obtain • Hence: DC & CV Lab. CSIE NTU

  18. 15.2.4 Optic Flow Acceleration Invariant . • Since • differentiating general solution in Sec 15.2 and solve for (x, y, z) .. .. .. . .. .. . .. . .. .. .. DC & CV Lab. CSIE NTU

  19. joke DC & CV Lab. CSIE NTU

  20. 15.3 Rigid-Body Motion • Rigid-body motion: no relative motion of points w.r.t. (with respect to) one another • Rigid-body motion: points maintain fixed position relative to one another • Rigid-body motion: all points move with the body as a whole DC & CV Lab. CSIE NTU

  21. 15.3 Rigid-Body Motion (cont’) • R(t): rotation matrix • T(t): translation vector • p(0): initial position of given point • R(0)=I, T(0)=0 • p(t): position of given point at time t DC & CV Lab. CSIE NTU

  22. 15.3 Rigid-Body Motion (cont’) • Rigid-body motion in displacement vectors: • velocity vector: time derivative of its position: . . . DC & CV Lab. CSIE NTU

  23. 15.3 Rigid-Body Motion (cont’) • Since • (a) translational-motion field under projection onto hemispherical surface only translational-component motion useful in determining scene structure • (b) rotational-motion field under projection onto hemispherical surface rotational-motion field provides no information about scene structure . . . DC & CV Lab. CSIE NTU

  24. 15.3 Rigid-Body Motion (cont’) DC & CV Lab. CSIE NTU

  25. 15.3 Rigid-Body Motion (cont’) • we can describe rigid-body motion in instantaneous velocity by . DC & CV Lab. CSIE NTU

  26. 15.3 Rigid-Body Motion (cont’) • :angular velocities in three axes • : translational velocities in three axes • from rigid-body-motion equation . . . DC & CV Lab. CSIE NTU

  27. 15.3 Rigid-Body Motion (cont’) • and perspective projection equation • we can determine an expression for z: . . DC & CV Lab. CSIE NTU

  28. 15.3 Rigid-Body Motion (cont’) • after simplification . . DC & CV Lab. CSIE NTU

  29. 15.3 Rigid-Body Motion (cont’) • image velocity: expressed as sum of translational field and rotational field • (x, y, z): 3D coordinate before rigid-body motion in displacement vectors • (x’, y’, z’): 3D coordinate after rigid-body motion in displacement vectors • : rotation angles in three axes • : translation in three axes DC & CV Lab. CSIE NTU

  30. 15.3 Rigid-Body Motion (cont’) • Rigid-body motion in displacement vectors: DC & CV Lab. CSIE NTU

  31. 15.3 Rigid-Body Motion (cont’) • motion in displacement vector and instantaneous velocity is different: • e.g. moon encircling earth • instantaneous velocity: first order approximation of displacement vector • first order approximation: when small, DC & CV Lab. CSIE NTU

  32. 15.3 Rigid-Body Motion (cont’) • first order approximation: when time=1 thus • x=(x’ - x)/1 • first order approximation: . DC & CV Lab. CSIE NTU

  33. joke DC & CV Lab. CSIE NTU

  34. 15.4 Linear Algorithms for Motion and Surface Structure from Optic Flow • 15.4.1 The Planar Patch Case • : arbitrary object point on planar patch at time t • : central projective coordinates of p(t) onto image plane z= f DC & CV Lab. CSIE NTU

  35. 15.4.1 The Planar Patch Case . . • : instantaneous velocity of moving image point • : optic flow image point • : instantaneous rotational angular velocity • : instantaneous translational velocity . . DC & CV Lab. CSIE NTU

  36. 15.4.1 The Planar Patch Case (cont’) • unit vector n(t): orthogonal to moving planar patch • rigid planar patch motion represented by rigid-motion constraint: . DC & CV Lab. CSIE NTU

  37. 15.4.1 The Planar Patch Case (cont’) • from above two equations: • Let • Rigid-motion constraint could be written as DC & CV Lab. CSIE NTU

  38. 15.4.1 The Planar Patch Case (cont’) • denote the 3 x 3 matrix by W and its three row vectors by • W:called planar motion parameter matrix • since skew symmetric DC & CV Lab. CSIE NTU

  39. 15.4.1 The Planar Patch Case (cont’) . • above equation can be written as • from perspective projection equations: • taking time derivatives of these equations we have . . . . . . . . DC & CV Lab. CSIE NTU

  40. 15.4.1 The Planar Patch Case (cont’) • substitute equations into above equations: • from third row • substitute z to obtain optical flow-planar motion equation . . . . . . . . . DC & CV Lab. CSIE NTU

  41. 15.4.1 The Planar Patch Case (cont’) • we have 2N linear equations: n=1,…,N: • optic flow-planar motion recovery: first solve W then find . . DC & CV Lab. CSIE NTU

  42. 15.4.2 General Case Optic Flow-Motion Equation • 1. set up optic flow-motion equation not involving depth information • 2. solve it by using linear least-squares technique DC & CV Lab. CSIE NTU

  43. 15.4.3 A Linear Algorithm for Solving Optic FlowMotion Equations DC & CV Lab. CSIE NTU

  44. 15.5.4 Mode of Motion, Direction of Translation, and Surface Structure • mode of motion: whether translation k=0 or not • direction of translation: direction of k • surface structure: relative depth when k 0 DC & CV Lab. CSIE NTU

  45. 15.4.5 Linear Optic Flow-Motion Algorithm and Simulation Results • motion and shape recovery algorithms should answer three questions: • minimum number of points to compute motion and shape • what set of optic flow points violate rank assumption e.g. collinearity… • What’s the accuracy of estimated motion from noisy optic flow? DC & CV Lab. CSIE NTU

  46. joke DC & CV Lab. CSIE NTU

  47. 15.5 The Two View-Linear Motion Algorithm DC & CV Lab. CSIE NTU

  48. 15.5.1 Planar Patch Motion Recovery from Two Perspective Views: A Brief Review • Two View-Planar Motion Equation • imaging geometry for two view-planar motion • rigid planar patch in motion in half-space z< 0 DC & CV Lab. CSIE NTU

  49. DC & CV Lab. CSIE NTU

  50. 15.5.1 Planar Patch Motion Recovery from Two Perspective Views: A Brief Review (cont’) • : arbitrary object point before motion • : same object point after motion • : central projective coordinates of • f : camera constant DC & CV Lab. CSIE NTU

More Related