1 / 43

A dvanced C omputer V ision Feature-based Alignment

A dvanced C omputer V ision Feature-based Alignment. Lecturer: Lu Yi & Prof. Fuh l ynn. luyi@gmail.com CSIE NTU. Content. Pre-requisites: mainly about camera model Alignment Algorithm RANdom Sample Consensus (RANSAC ) Pose Estimation. Sources.

duante
Download Presentation

A dvanced C omputer V ision Feature-based Alignment

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Advanced Computer VisionFeature-based Alignment Lecturer: Lu Yi & Prof. Fuh lynn.luyi@gmail.com CSIE NTU

  2. Content • Pre-requisites: mainly about camera model • Alignment Algorithm • RANdomSample Consensus (RANSAC) • Pose Estimation

  3. Sources • EGGN 512 – Computer Vision course by Prof. Willliam Hoff • http://inside.mines.edu/~whoff/courses/EENG512/ • Playlist@Youtube: • http://www.youtube.com/playlist?list=PL4B3F8D4A5CAD8DA3 • Digital Visual Effects course by Prof. Yung-Yu Chuang • http://www.csie.ntu.edu.tw/~cyy/courses/vfx/14spring/news/ • Both are highly recommended

  4. Taught Way • Cover the main parts of textbook chapter 6 • Lecture mostly depended on EGGN 512 course videos

  5. Frames of Reference • Image frames are 2D; other are 3D • The “pose” (position and orientation) of a 3D rigid body has 6 degrees of freedom

  6. Pre-requisites • Image Formation • Geometric Transformations (2D-2D, 3D-3D, 3D-2D) • Camera Model (Camera Matrix = Intrinsic + Extrinsic camera matrix) • Allrelated contents arein textbook chapter 2: Image formation

  7. Pre-requisites • Review • Projective Transformation • Camera Matrix • EGGN 512 – Lecture 5-1 3D-2D Transforms • http://www.youtube.com/watch?v=DFNjOUMuecU&index=11&list=PL4B3F8D4A5CAD8DA3 • EGGN 512 – Lecture 5-2 3D-2D Transforms • http://www.youtube.com/watch?v=5gesrLgNuQo&list=PL4B3F8D4A5CAD8DA3

  8. Alignment Algorithm • EGGN 512 – Lecture 14-1 Alignment • http://www.youtube.com/watch?v=UcU4814hvR8&list=PL4B3F8D4A5CAD8DA3 • EGGN 512 – Lecture 14-2 Alignment • http://www.youtube.com/watch?v=XxEKMecNZk0&list=PL4B3F8D4A5CAD8DA3 • EGGN 512 – Lecture 14-3 Alignment • http://www.youtube.com/watch?v=lSXfv4baMwk&list=PL4B3F8D4A5CAD8DA3 • Slides • http://inside.mines.edu/~whoff/courses/EENG512/lectures/15-AlignmentNonlinear.pdf

  9. Alignment Algorithm • Feature-based methods: only use feature points to estimate parameters • Features: SIFT, SURF, Harris .etc.

  10. Determine pairwise alignment • p’=Mp, where M is a transformation matrix, p and p’ are feature matches • It is possible to use more complicated models such as affine or perspective • For example, assume M is a 2x2 matrix • Find M with the least square error

  11. Determine pairwise alignment • Overdeterminedsystem

  12. Normal Equation • Given an overdeterminedsystem • the normal equation is that which minimizes the sum of the square differences between left and right sides • Why?

  13. Normal Equation nxm, n equations, m variables

  14. Normal Equation

  15. → Normal Equation

  16. Normal Equation

  17. Normal Equation

  18. Determine pairwise alignment • p’=Mp, where M is a transformation matrix, p and p’ are feature matches • For translation model, it is easier. • What if the match is false? Avoid impact of outliers. (Ans: RANSAC)

  19. Application: Image Stitching • Stitching = alignment + blending photometric registration geometrical registration

  20. Application: Image Stitching • Panorama – the whole picture • Compact Camera FOV = 50 x 35°

  21. Application: Image Stitching • Panorama – the whole picture • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135°

  22. Application: Image Stitching • Panorama – the whole picture • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135° • Panoramic Mosaic = 360 x 180°

  23. Application: Image Stitching • Example: • http://graphics.stanford.edu/courses/cs178/applets/projection.html

  24. Application: Face Alignment • Face alignment is the key preprocessing step for face recognition

  25. Application: Face Alignment • Examples from LFW dataset:

  26. RANSAC Algorithm • RANSAC = RANdom Sample Consensus • An iterative method to estimate parameters of a mathematical model from a set of observed data which contains outliers [Wiki] • Compare to robust statistics • Given N data points xi, assume that majority of them are generated from a model with parameters , try to recover . • Fischler, M. A., & Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6), 381-395.

  27. How many times? How big? Smaller is better How to define? Depends on the problem. RANSAC Algorithm Run k times: (1) draw n samples randomly (2) fit parameters  with these n samples (3) for each of other N-n points, calculate its distance to the fitted model, count the number of inlier points, c Output  with the largest c

  28. n samples are all inliers a failure failure after k trials RANSAC Algorithm (How to determine K) p: probability of real inliers P: probability of success after k trials for P=0.99

  29. RANSAC Algorithm (Example: line fitting)

  30. RANSAC Algorithm (Example: line fitting) n=2

  31. RANSAC Algorithm (Example: line fitting)

  32. RANSAC Algorithm (Example: line fitting)

  33. RANSAC Algorithm (Example: line fitting) c=3

  34. RANSAC Algorithm (Example: line fitting) c=3

  35. RANSAC Algorithm (Example: line fitting) c=15

  36. RANSAC Algorithm • EGGN 512 – Lecture 27-1 RANSAC • http://www.youtube.com/watch?v=NKxXGsZdDp8&list=PL4B3F8D4A5CAD8DA3&index=54

  37. RANSAC Algorithm (Example: Homography)

  38. RANSAC Algorithm (Example: Homography)

  39. RANSAC Algorithm (Example: Homography)

  40. Pose Estimation • EGGN 512 – Lecture 16-1 Pose Estimation • http://www.youtube.com/watch?v=kq3c6QpcAGc&list=PL4B3F8D4A5CAD8DA3 • EGGN 512 – Lecture 17-1 Pose from Lines • http://www.youtube.com/watch?v=D_4eUoqgWdc&list=PL4B3F8D4A5CAD8DA3

  41. Pose Estimation • EGGN 512 – Lecture 18-1 SVD • http://www.youtube.com/watch?v=C852P1JrHXI&list=PL4B3F8D4A5CAD8DA3 • EGGN 512 – Lecture 18-2 SVD • http://www.youtube.com/watch?v=aIkzK4CdYes&list=PL4B3F8D4A5CAD8DA3

  42. Pose Estimation • EGGN 512 – Lecture 19-1 Linear Pose Estimation • http://www.youtube.com/watch?v=HojSSrxsB4Q&list=PL4B3F8D4A5CAD8DA3 • EGGN 512 – Lecture 19-2 Linear Pose Estimation • http://www.youtube.com/watch?v=ik8dFybnyPY&list=PL4B3F8D4A5CAD8DA3 • EGGN 512 – Lecture 19-3 Linear Pose Estimation • http://www.youtube.com/watch?v=OS5b-3Xfn1M&list=PL4B3F8D4A5CAD8DA3

  43. END

More Related