1 / 46

Self-calibration Class 13

Self-calibration Class 13. Read Chapter 6. Assignment 3. Collect potential matches from all algorithms for all pairs Matlab ASCII format, exchange data Implement RANSAC that uses combined match dataset Compute consistent set of matches and epipolar geometry

alec
Download Presentation

Self-calibration Class 13

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Self-calibrationClass 13 Read Chapter 6

  2. Assignment 3 • Collect potential matches from all algorithms for all pairs • Matlab ASCII format, exchange data • Implement RANSAC that uses combined match dataset • Compute consistent set of matches and epipolar geometry • Report thresholds used, match sets used, number of consistent matches obtained, epipolar geometry, show matches and epipolar geometry (plot some epipolar lines). Due next Tuesday, Nov. 2 naming convention: firstname_ij.dat chris_56.dat http://www.unc.edu/courses/2004fall/comp/290/089/assignment3/ [F,inliers]=FRANSAC([chris_56; brian_56; …])

  3. Papers • Each should present a paper during 20-25 minutes followed by discussion. Partially outside of class schedule to make up for missed classes. (When?) • List of proposed papers will come on-line by Thursday, feel free to propose your own (suggestion: something related to your project). • Make choice by Thursday, assignments will be made in class. • Everybody should have read papers that are being discussed.

  4. Papers http://www.unc.edu/courses/2004fall/comp/290b/089/papers/

  5. 3D photography course schedule

  6. Ideas for a project?

  7. Dealing with dominant planar scenes (Pollefeys et al., ECCV‘02) • USaM fails when common features are all in a plane • Solution: part 1 Model selection to detect problem

  8. Dealing with dominant planar scenes (Pollefeys et al., ECCV‘02) • USaM fails when common features are all in a plane • Solution: part 2 Delay ambiguous computations until after self-calibration (couple self-calibration over all 3D parts)

  9. Non-sequential image collections Problem: Features are lost and reinitialized as new features 3792 points Solution: Match with other close views 4.8im/pt 64 images

  10. Relating to more views • For every view i • Extract features • Compute two view geometry i-1/i and matches • Compute pose using robust algorithm • Refine existing structure • Initialize new structure For every view i Extract features Compute two view geometry i-1/i and matches Compute pose using robust algorithm For all close views k Compute two view geometry k/i and matches Infer new 2D-3D matches and add to list Refine pose using all 2D-3D matches Refine existing structure Initialize new structure Problem: find close views in projective frame

  11. Determining close views • If viewpoints are close then most image changes can be modelled through a planar homography • Qualitative distance measure is obtained by looking at the residual error on the best possible planar homography Distance =

  12. Non-sequential image collections (2) 2170 points 3792 points 9.8im/pt 64 images 4.8im/pt 64 images

  13. Hierarchical structure and motion recovery • Compute 2-view • Compute 3-view • Stitch 3-view reconstructions • Merge and refine reconstruction F T H PM

  14. Stitching 3-view reconstructions Different possibilities 1. Align (P2,P3) with (P’1,P’2) 2. Align X,X’ (and C,C’) 3. Minimize reproj. error 4. MLE (merge)

  15. Refining structure and motion • Minimize reprojection error • Maximum Likelyhood Estimation (if error zero-mean Gaussian noise) • Huge problem but can be solved efficiently (Bundle adjustment)

  16. P1 P2 P3 M U1 U2 W U3 WT V 3xn (in general much larger) 12xm Sparse bundle adjustment Non-linear min. requires to solve Jacobian of has sparse block structure im.pts. view 1 Needed for non-linear minimization

  17. U-WV-1WT WT V 3xn 11xm Sparse bundle adjustment • Eliminate dependence of camera/motion parameters on structure parameters Note in general 3n >> 11m Allows much more efficient computations e.g. 100 views,10000 points, solve 1000x1000, not 30000x30000 Often still band diagonal use sparse linear algebra algorithms

  18. Self-calibration • Introduction • Self-calibration • Dual Absolute Quadric • Critical Motion Sequences

  19. Motivation • Avoid explicit calibration procedure • Complex procedure • Need for calibration object • Need to maintain calibration

  20. Motivation • Allow flexible acquisition • No prior calibration necessary • Possibility to vary intrinsics • Use archive footage

  21. Projective ambiguity Reconstruction from uncalibrated images  projective ambiguity on reconstruction

  22. Stratification of geometry Projective Affine Metric 15 DOF 7 DOF absolute conic angles, rel.dist. 12 DOF plane at infinity parallelism More general More structure

  23. Constraints ? Scene constraints • Parallellism, vanishing points, horizon, ... • Distances, positions, angles, ... Unknown scene  no constraints • Camera extrinsics constraints • Pose, orientation, ... Unknown camera motion  no constraints • Camera intrinsics constraints • Focal length, principal point, aspect ratio & skew Perspective camera model too general  some constraints

  24. Euclidean projection matrix Factorization of Euclidean projection matrix Intrinsics: (camera geometry) (camera motion) Extrinsics: Note: every projection matrix can be factorized, but only meaningful for euclidean projection matrices

  25. Constraints on intrinsic parameters Constant e.g. fixed camera: Known e.g. rectangular pixels: square pixels: principal point known:

  26. Self-calibration Upgrade from projective structure to metric structure using constraintsonintrinsic camera parameters • Constant intrinsics • Some known intrinsics, others varying • Constraints on intrincs and restricted motion (e.g. pure translation, pure rotation, planar motion) (Faugeras et al. ECCV´92, Hartley´93, Triggs´97, Pollefeys et al. PAMI´99, ...) (Heyden&Astrom CVPR´97, Pollefeys et al. ICCV´98,...) (Moons et al.´94, Hartley ´94, Armstrong ECCV´96, ...)

  27. A counting argument • To go from projective (15DOF) to metric (7DOF) at least 8 constraints are needed • Minimal sequence length should satisfy • Independent of algorithm • Assumes general motion (i.e. not critical)

  28. Outline • Introduction • Self-calibration • Dual Absolute Quadric • Critical Motion Sequences

  29. The Dual Absolute Quadric The absolute dual quadric Ω*∞ is a fixed conic under the projective transformation H iff H is a similarity • 8 dof • plane at infinity π∞ is the nullvector of Ω∞ • Angles:

  30. Absolute Dual Quadric and Self-calibration Eliminate extrinsics from equation Equivalent to projection of Dual Abs.Quadric Dual Abs.Quadric also exists in projective world Transforming world so that reduces ambiguity to similarity

  31. * * projection constraints Absolute Dual Quadric and Self-calibration Projection equation: Translate constraints on K through projection equationto constraints on * Absolute conic = calibration object which is always present but can only be observed through constraints on the intrinsics

  32. Constraints on * #constraints condition constraint type

  33. Linear algorithm (Pollefeys et al.,ICCV´98/IJCV´99) Assume everything known, except focal length Yields 4 constraint per image Note that rank-3 constraint is not enforced

  34. Linear algorithm revisited (Pollefeys et al., ECCV‘02) Weighted linear equations assumptions

  35. Projective to metric Compute T from using eigenvalue decomposition of and then obtain metric reconstruction as

  36. Alternatives: (Dual) image of absolute conic • Equivalent to Absolute Dual Quadric • Practical when H can be computed first • Pure rotation(Hartley’94, Agapito et al.’98,’99) • Vanishing points, pure translations, modulus constraint, …

  37. Note that in the absence of skew the IAC can be more practical than the DIAC!

  38. Kruppa equations Limit equations to epipolar geometry Only 2 independent equations per pair But independent of plane at infinity

  39. Refinement • Metric bundle adjustment Enforce constraints or priors on intrinsics during minimization (this is „self-calibration“ for photogrammetrist)

  40. Outline • Introduction • Self-calibration • Dual Absolute Quadric • Critical Motion Sequences

  41. Critical motion sequences (Sturm, CVPR´97, Kahl, ICCV´99, Pollefeys,PhD´99) • Self-calibration depends on camera motion • Motion sequence is not always general enough • Critical Motion Sequences have more than one potential absolute conic satisfying all constraints • Possible to derive classification of CMS

  42. Critical motion sequences:constant intrinsic parameters Most important cases for constant intrinsics Note relation between critical motion sequences and restricted motion algorithms

  43. Critical motion sequences:varying focal length Most important cases for varying focal length (other parameters known)

  44. Critical motion sequences:algorithm dependent Additional critical motion sequences can exist for some specific algorithms • when not all constraints are enforced (e.g. not imposing rank 3 constraint) • Kruppa equations/linear algorithm: fixating a point Some spheres also project to circles located in the image and hence satisfy all the linear/kruppa self-calibration constraints

  45. Non-ambiguous new views for CMS (Pollefeys,ICCV´01) • restrict motion of virtual camera to CMS • use (wrong) computed camera parameters

  46. Next class: shape from silhouettes

More Related