1 / 47

Rank Conditions in Multiple View Geometry: Applications and Conclusions

This study explores the theory and algorithms of multiple view geometry, focusing on point features, multilinear constraints, rank conditions, and applications like structure from motion. The formulation involves camera modeling with multiple images, generalization to higher dimensions, and a review of literature. The study includes a review of recent books on the topic, a discussion of rank conditions, and the implications for point features in various views. The text provides insights into line, plane, and space representations, offering geometric interpretations and concluding with ongoing work in the field.

skinnerk
Download Presentation

Rank Conditions in Multiple View Geometry: Applications and Conclusions

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Rank Conditions in Multiple View Geometry Rene Vidal (EECS.UCB), Kun Huang (ECE.UIUC) Jana Kosecka (CS.GMU), Robert Fossum (Math.UIUC) by Yi Ma Perception & Decision Laboratory Decision & Control Group, CSL Image Formation & Processing Group, Beckman Electrical & Computer Engineering Dept., UIUC http://black.csl.uiuc.edu/~yima

  2. FORMULATION: camera model and multiple images POINT FEATURE: multilinear constraints v.s. rank conditions GENERALIZATION: line, plane, space of higher dimensions APPLICATIONS: matching, transfer, structure from motion CONCLUSIONS AND ON-GOING WORK

  3. FORMULATION: camera model and multiple images POINT FEATURE: multilinear constraints v.s. rank conditions GENERALIZATION: line, plane, space of higher dimensions APPLICATIONS: matching, transfer, structure from motion CONCLUSIONS AND ON-GOING WORK

  4. FORMULATION - Fundamental Geometric Problem Input: Corresponding images (of “features”) in multiple images. Output: Camera motion, camera calibration, object structure. Jana’s apartment Image courtesy of Jana Kosecka

  5. curve & surface plane line algorithm projective point algebra affine geometry Euclidean perspective orthographic 2 views omni-directional 3 views 4 views m views FORMULATION – Orthodox View (State of the Art)

  6. FORMULATION – Literature Review “Multiple” view geometry theory • Two views: Kruppa’13, Longuet-Higgins’81, Huang & Faugeras’89, … • Three views: Spetsakis & Aloimonos’90, Shashua’94, Hartley’94, … • Four views: Triggs’95, Shashua’00, … • Multiple views: Heyden & Astrom’97’98, … • Higher dimension: Wolf & Shashua’01 Multiple view geometry algorithms • Euclidean: Maybank’93, Weng, Ahuja & Huang’93, … • Affine: Quan & Kanade’96, … • Projective: Triggs’96, … • Orthographic: Tomasi & Kanade’92, … Recent books on multiple view geometry 1. Multiple view geometry in computer vision, Hartley & Zisserman’00. 2. Geometry of multiple images, Faugeras, Luong & Papadopoulo’01.

  7. Rank Conditions FORMULATION – A Provocative Stand: One Theorem for All & More? curve & surface plane line algorithm projective point algebra affine geometry Euclidean perspective orthographic 2 views omni-directional 3 views 4 views m views

  8. FORMULATION – A Little Notation: Hat Operator

  9. FORMULATION: camera model and multiple images POINT FEATURE: multilinear constraints v.s. rank conditions GENERALIZATION: line, plane, space of higher dimensions APPLICATIONS: matching, transfer, structure from motion CONCLUSIONS AND ON-GOING WORK

  10. POINT FEATURE – Pinhole Camera Model Homogeneous coordinates of a 3-D point Homogeneous coordinates of its 2-D image Projection of a 3-D point to an image plane

  11. POINT FEATURE – Multiple View Structure From Motion Given corresponding images of points: recover everything else from equations associated to each: . . .

  12. POINT FEATURE – Conventional Multilinear (Multifocal) Constraints For images of the same 3-D point : (leading to the conventional approach) Multilinear constraints among 2, 3, 4-wise views

  13. POINT FEATURE – Rank Condition on the Multiple View Matrix WLOG, choose camera frame 1 as the reference Multiple View Matrix Lemma [Rank Condition for Point Features] (generic) (degenerate) Let then and are linearly dependent.

  14. POINT FEATURE – Rank Condition Implies Bilinear Constraints Fact: Given non-zero vectors Hence, we have These constraints are only necessary but NOT sufficient!

  15. POINT FEATURE – Rank Condition Implies Trilinear Constraints Fact: Given non-zero vectors Hence, we have • These constraints are only necessary but NOT sufficient! • However, there is NO further relationship among quadruple wise • views. Quadrilinear constraints hence are redundant!

  16. POINT FEATURE – Uniqueness of Pre-image (Bilinear Constraints) “Bilinear means pair-wise coplanar”: except in a rare coplanar case: Rectilinear motion Trifocal plane

  17. POINT FEATURE – Uniqueness of Pre-image (Trilinear Constraints) “Trilinear means triple-wise incidental”: except in a rare collinear case:

  18. POINT FEATURE – Uniqueness of Pre-image (Multiple View Matrix) Proposition [Uniqueness of Pre-image] Given vectors with respect to camera frames, they correspond to a unique point in the 3-D space if the rank of the matrix is1. If the rank is 0, the point is determined up to a line on which all the camera centers must lie. “point incidence condition” . . .

  19. POINT FEATURE – Geometric Interpretation is the “depth” of the point relative to the camera center. Points that give the same matrix are on a sphere of radius

  20. FORMULATION: camera model and multiple images POINT FEATURE: multilinear constraints v.s. rank conditions GENERALIZATION: line, plane, space of higher dimensions APPLICATIONS: matching, transfer, structure from motion CONCLUSIONS AND ON-GOING WORK

  21. GENERALIZATION – Line Feature Homogeneous representation of a 3-D line Homogeneous representation of its 2-D co-image Projection of a 3-D line to an image plane

  22. GENERALIZATION – Multiple View Matrix: Line v.s. Point Point Features Line Features

  23. GENERALIZATION – Rank Conditions: Line v.s. Point

  24. GENERALIZATION – Incidence Relations Among Features Incidence conditions: inclusion, intersection, and restriction. “nothing but incidence conditions” . . .

  25. Multi-nonlinear constraints • among 3, 4-wise images. • Multi-linear constraints • among 2, 3-wise images. GENERALIZATION – What is the Matrix? Theorem 1 [The Universal Rank Condition] for images of a point on a line:

  26. Examples: Case 1: a line reference Case 2: a point reference GENERALIZATION – Implications of the Rank Condition • All previously known constraints are the theorem’s instantiations. • It implies more constraints and is now complete. • Degenerate configurations if and only if a drop of rank.

  27. GENERALIZATION – Global Multiple-View Analysis (Example)

  28. GENERALIZATION – A Family of Incidental Lines (A Corollary) each can randomly take the image of any of the lines: Nonlinear constraints among up to four views . . .

  29. Corollary [Coplanar Features] Rank conditions on the new extended remain exactly the same! GENERALIZATION – Restriction to a Plane (A Corollary) Homogeneous representation of a 3-D plane

  30. In addition to previous constraints, it simultaneously gives homography: GENERALIZATION – Multiple View Matrix: Coplanar Features Given that a point and line features lie on a plane in 3-D space:

  31. Corollary [Point/Line Duality] For planar features, points and lines are hence equivalent! GENERALIZATION – Coplanar Point/Line Duality On the plane any two points determine a line and any two lines determine a point.

  32. Multi-quadratic equations in given images GENERALIZATION – Intrinsic Rank Condition for Planar Features 4 coplanar points + 3 virtual points = 7 effective points

  33. Before: Now: Time base This is a perspective projection from <n to <2. GENERALIZATION – Euclidean Imbedding of Dynamic Scenes

  34. Theorem 2 [Generalized Rank Condition for from <n to <k] GENERALIZATION – Rank Condition in Space of High Dimension

  35. GENERALIZATION – Rank Conditions for Curves & Surfaces Differentiating the matrix of a point (moving) along a curve: gives rise to a rank condition for curve. intensity level sets region boundaries . . .

  36. GENERALIZATION – From Tangent to Point-wise Correspondence gives rise to a set of ordinary differential equations: • The rank deficiency condition for relates points and tangent • lines of a curve. • Solving these equations establishes point-wise correspondence for • image curves and in fact eventually for surface as well. • gives constraints on curvature and normals of image curves.

  37. FORMULATION: camera model and multiple images POINT FEATURE: multilinear constraints v.s. rank conditions GENERALIZATION: line, plane, space of higher dimensions APPLICATIONS: matching, transfer, structure from motion CONCLUSIONS AND ON-GOING WORK

  38. APPLICATIONS – Multiple View Matching Test Given the projection matrix associated to camera frames. Then for vectors

  39. APPLICATIONS – Transferring Images to a New View Given the projection matrix associated to camera frames. Then for given vectors So given images, rank deficiency adds a linear constraint on the image. Computing the kernel of gives the new image.

  40. APPLICATIONS – Motion and Structure from Multiple Views Given images of points:

  41. APPLICATIONS – SVD Based Four Step Algorithm for SFM

  42. APPLICATIONS – Utilize All Incidence Conditions (Example) Three edges intersect at each vertex. . . .

  43. APPLICATIONS – Utilizing All Incidence Conditions (Simulations)

  44. APPLICATIONS – Landing a Helicopter (Experiments) Images courtesy of Omid Shakernia, UCB

  45. FORMULATION: camera model and multiple images POINT FEATURE: multilinear constraints v.s. rank conditions GENERALIZATION: line, plane, space of higher dimensions APPLICATIONS: matching, transfer, structure from motion CONCLUSIONS AND ON-GOING WORK

  46. CONCLUSIONS AND ON-GOING WORK • Rank condition simplifies, unifies and completes existing algebraic • results on multiview constraints (no tensor and algebraic geometry). • Rank condition is for all features, all incidence relations, all number • of views, all types of projection, all (linear) spaces of arbitrary • dimensions. • Rank condition intrinsically ties together geometry and algebra. • Metric multiple view geometry. • Consistent, optimal and robust algorithms for correspondence, • image-based view synthesis, and reconstruction of motion & structure. • Apply to dynamical scenes: human motion; sensor networks… • Real-time algorithms for autonomous navigation and robotic control.

  47. Rank Conditions in Multiple View Geometry Rene Vidal (EECS.UCB), Kun Huang (ECE.UIUC) Jana Kosecka (CS.GMU), Robert Fossum (Math.UIUC) by yima@uiuc.edu “Rank conditions on the multiple view matrix”, submitted to IJCV. “General rank conditions in multiple view geometry”, submitted to D&CG “An invitation to 3-D vision”, Ma, Soatto, Kosecka, and Sastry, 2002.

More Related