1 / 63

Recognising Panoramas

Recognising Panoramas. M. Brown and D. Lowe, University of British Columbia. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35°. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35° Human FOV = 200 x 135°. Introduction.

rogan-cardenas
Download Presentation

Recognising Panoramas

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Recognising Panoramas M. Brown and D. Lowe, University of British Columbia

  2. Introduction • Are you getting the whole picture? • Compact Camera FOV = 50 x 35°

  3. Introduction • Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135°

  4. Introduction • Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135° • Panoramic Mosaic = 360 x 180°

  5. Why “Recognising Panoramas”?

  6. Why “Recognising Panoramas”? • 1D Rotations (q) • Ordering  matching images

  7. Why “Recognising Panoramas”? • 1D Rotations (q) • Ordering  matching images

  8. Why “Recognising Panoramas”? • 1D Rotations (q) • Ordering  matching images

  9. 2D Rotations (q, f) • Ordering  matching images Why “Recognising Panoramas”? • 1D Rotations (q) • Ordering  matching images

  10. 2D Rotations (q, f) • Ordering  matching images Why “Recognising Panoramas”? • 1D Rotations (q) • Ordering  matching images

  11. 2D Rotations (q, f) • Ordering  matching images Why “Recognising Panoramas”? • 1D Rotations (q) • Ordering  matching images

  12. Why “Recognising Panoramas”?

  13. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  14. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  15. Overview • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  16. Overview • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  17. Invariant Features • Schmid & Mohr 1997, Lowe 1999, Baumberg 2000, Tuytelaars & Van Gool 2000, Mikolajczyk & Schmid 2001, Brown & Lowe 2002, Matas et. al. 2002, Schaffalitzky & Zisserman 2002

  18. SIFT Features • Invariant Features • Establish invariant frame • Maxima/minima of scale-space DOG  x, y, s • Maximum of distribution of local gradients q • Form descriptor vector • Histogram of smoothed local gradients • 128 dimensions • SIFT features are… • Geometrically invariant to similarity transforms, • some robustness to affine change • Photometrically invariant to affine changes in intensity

  19. Overview • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  20. Nearest Neighbour Matching • Find k-NN for each feature • k  number of overlapping images (we use k = 4) • Use k-d tree • k-d tree recursively bi-partitions data at mean in the dimension of maximum variance • Approximate nearest neighbours found in O(nlogn)

  21. Overview • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  22. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  23. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  24. Overview • Feature Matching • Image Matching • RANSAC for Homography • Probabilistic model for verification • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  25. Overview • Feature Matching • Image Matching • RANSAC for Homography • Probabilistic model for verification • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  26. RANSAC for Homography

  27. RANSAC for Homography

  28. RANSAC for Homography

  29. Overview • Feature Matching • Image Matching • RANSAC for Homography • Probabilistic model for verification • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  30. Probabilistic model for verification

  31. Probabilistic model for verification • Compare probability that this set of RANSAC inliers/outliers was generated by a correct/false image match • ni = #inliers, nf = #features • p1 = p(inlier | match), p0 = p(inlier | ~match) • pmin = acceptance probability • Choosing values for p1, p0 and pmin

  32. Finding the panoramas

  33. Finding the panoramas

  34. Finding the panoramas

  35. Finding the panoramas

  36. Overview • Feature Matching • Image Matching • RANSAC for Homography • Probabilistic model for verification • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  37. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  38. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  39. Overview • Feature Matching • Image Matching • Bundle Adjustment • Error function • Multi-band Blending • Results • Conclusions

  40. Overview • Feature Matching • Image Matching • Bundle Adjustment • Error function • Multi-band Blending • Results • Conclusions

  41. Error function • Sum of squared projection errors • n = #images • I(i) = set of image matches to image i • F(i, j) = set of feature matches between images i,j • rijk = residual of kth feature match between images i,j • Robust error function

  42. Homography for Rotation • Parameterise each camera by rotation and focal length • This gives pairwise homographies

  43. Bundle Adjustment • New images initialised with rotation, focal length of best matching image

  44. Bundle Adjustment • New images initialised with rotation, focal length of best matching image

  45. Overview • Feature Matching • Image Matching • Bundle Adjustment • Error function • Multi-band Blending • Results • Conclusions

  46. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  47. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  48. Multi-band Blending • Burt & Adelson 1983 • Blend frequency bands over range l

  49. 2-band Blending Low frequency (l > 2 pixels) High frequency (l < 2 pixels)

  50. Linear Blending

More Related