1 / 62

Automatic Image Stitching using Invariant Features

Automatic Image Stitching using Invariant Features. Matthew Brown and David Lowe, University of British Columbia. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35°. Introduction. Are you getting the whole picture? Compact Camera FOV = 50 x 35°

knut
Download Presentation

Automatic Image Stitching using Invariant Features

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automatic Image Stitching using Invariant Features Matthew Brown and David Lowe, University of British Columbia

  2. Introduction • Are you getting the whole picture? • Compact Camera FOV = 50 x 35°

  3. Introduction • Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135°

  4. Introduction • Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135° • Panoramic Mosaic = 360 x 180°

  5. Recognising Panoramas

  6. Recognising Panoramas • 1D Rotations (q) • Ordering  matching images

  7. Recognising Panoramas • 1D Rotations (q) • Ordering  matching images

  8. Recognising Panoramas • 1D Rotations (q) • Ordering  matching images

  9. 2D Rotations (q, f) • Ordering  matching images Recognising Panoramas • 1D Rotations (q) • Ordering  matching images

  10. 2D Rotations (q, f) • Ordering  matching images Recognising Panoramas • 1D Rotations (q) • Ordering  matching images

  11. 2D Rotations (q, f) • Ordering  matching images Recognising Panoramas • 1D Rotations (q) • Ordering  matching images

  12. Recognising Panoramas

  13. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  14. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  15. Overview • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  16. Overview • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  17. Invariant Features • Schmid & Mohr 1997, Lowe 1999, Baumberg 2000, Tuytelaars & Van Gool 2000, Mikolajczyk & Schmid 2001, Brown & Lowe 2002, Matas et. al. 2002, Schaffalitzky & Zisserman 2002

  18. SIFT Features • Invariant Features • Establish invariant frame • Maxima/minima of scale-space DOG  x, y, s • Maximum of distribution of local gradients q • Form descriptor vector • Histogram of smoothed local gradients • 128 dimensions • SIFT features are… • Geometrically invariant to similarity transforms, • some robustness to affine change • Photometrically invariant to affine changes in intensity

  19. Overview • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  20. Nearest Neighbour Matching • Nearest neighbour matching • Use k-d tree • k-d tree recursively bi-partitions data at mean in the dimension of maximum variance • Approximate nearest neighbours found in O(n log n) • Find k-NN for each feature • k  number of overlapping images (we use k = 4) [ Beis Lowe 1997, Nene Nayar 1997, Gray Moore 2000, Shakhnarovich 2003 ]

  21. K-d tree

  22. K-d tree

  23. Overview • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  24. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  25. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  26. Overview • Feature Matching • Image Matching • RANSAC for Homography • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  27. Overview • Feature Matching • Image Matching • RANSAC for Homography • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  28. RANSAC for Homography

  29. RANSAC for Homography

  30. RANSAC for Homography

  31. least squares line RANSAC: 1D Line Fitting

  32. RANSAC: 1D Line Fitting

  33. RANSAC line RANSAC: 1D Line Fitting

  34. The RANSAC Algorithm function H = RANSAC(points, nIterations) { bestInliers = 0; bestH = zeros(3, 3); for (i = 0; i < nIterations; i++) { samplePoints = RandomSample(points); H = ComputeTransform(samplePoints); nInliers = Consistent(H); if (nInliers > bestInliers) { bestInliers = nInliers; bestH = H; } // end if } // end for } // end RANSAC

  35. 2D Transforms • Linear (affine) • Homography

  36. Finding the panoramas

  37. Finding the panoramas

  38. Finding the panoramas

  39. Finding the panoramas

  40. Connected Components

  41. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  42. Bundle Adjustment • Adjust rotation, focal length of each image to minimise error in matched features

  43. Bundle Adjustment • Adjust rotation, focal length of each image to minimise error in matched features

  44. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  45. Overview • Feature Matching • Image Matching • Bundle Adjustment • Multi-band Blending • Results • Conclusions

  46. Multi-band Blending • Burt & Adelson 1983 • Blend frequency bands over range l

  47. 2-band Blending Low frequency (l > 2 pixels) High frequency (l < 2 pixels)

  48. Linear Blending

  49. 2-band Blending

More Related