1 / 61

Automatic Panoramic Image Stitching using Local Features

This presentation introduces the concept of recognizing and stitching panoramas automatically using local features. The usage of local features, such as SIFT features and nearest neighbor matching, aids in image alignment and rendering, resulting in compelling panoramic mosaics. The process involves feature matching, motion models, RANSAC, bundle adjustment, and automatic straightening to enhance the stitching accuracy. Gain compensation techniques and multi-band blending are also discussed, emphasizing the importance of geometric and photometric invariance for seamless panoramic images.

aacree
Download Presentation

Automatic Panoramic Image Stitching using Local Features

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Automatic Panoramic Image Stitching using Local Features Matthew Brown and David Lowe, University of British Columbia

  2. Introduction • Are you getting the whole picture? • Compact Camera FOV = 50 x 35°

  3. Introduction • Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135°

  4. Introduction • Are you getting the whole picture? • Compact Camera FOV = 50 x 35° • Human FOV = 200 x 135° • Panoramic Mosaic = 360 x 180°

  5. Why Use Local Features?

  6. Why Use Local Features? • 1D Rotations (q) • Ordering  matching images

  7. Why Use Local Features? • 1D Rotations (q) • Ordering  matching images

  8. Why Use Local Features? • 1D Rotations (q) • Ordering  matching images

  9. 2D Rotations (q, f) • Ordering  matching images Why Use Local Features? • 1D Rotations (q) • Ordering  matching images

  10. 2D Rotations (q, f) • Ordering  matching images Why Use Local Features? • 1D Rotations (q) • Ordering  matching images

  11. 2D Rotations (q, f) • Ordering  matching images Why Use Local Features? • 1D Rotations (q) • Ordering  matching images

  12. Recognising Panoramas [ Brown and Lowe ICCV 2003, IJCV 2007 ]

  13. Automatic Stitching • Feature Matching • Image Matching • Image Alignment • Rendering • Results • Conclusions

  14. Automatic Stitching • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Image Alignment • Rendering • Results • Conclusions

  15. Invariant Features • Schmid & Mohr 1997, Lowe 1999, Baumberg 2000, Tuytelaars & Van Gool 2000, Mikolajczyk & Schmid 2001, Brown & Lowe 2002, Matas et. al. 2002, Schaffalitzky & Zisserman 2002

  16. SIFT Features • Invariant Features • Establish invariant frame • Maxima/minima of scale-space DOG  x, y, s • Maximum of distribution of local gradients q • Form descriptor vector • Histogram of smoothed local gradients • 128 dimensions • SIFT features are… • Geometrically invariant to similarity transforms, • some robustness to affine change • Photometrically invariant to affine changes in intensity

  17. Automatic Stitching • Feature Matching • SIFT Features • Nearest Neighbour Matching • Image Matching • Image Alignment • Rendering • Results • Conclusions

  18. Nearest Neighbour Matching • Nearest neighbour matching • Use k-d tree • k-d tree recursively bi-partitions data at mean in the dimension of maximum variance • Approximate nearest neighbours found in O(n log n) • Find k-NN for each feature • k  number of overlapping images (we use k = 4) [ Beis Lowe 1997, Nene Nayar 1997, Gray Moore 2000, Shakhnarovich 2003 ]

  19. K-d tree

  20. K-d tree

  21. Automatic Stitching • Feature Matching • Image Matching • Motion Model • RANSAC • Image Alignment • Rendering • Results • Conclusions

  22. 2D Motion Models • Linear (affine) • Homography

  23. Homography for Rotation • Projection equation • → pairwise homographies set t = 0 for a pair where

  24. Automatic Stitching • Feature Matching • Image Matching • Motion Model • RANSAC • Image Alignment • Rendering • Results • Conclusions

  25. RANSAC for Homography

  26. RANSAC for Homography

  27. RANSAC for Homography

  28. least squares line RANSAC: 1D Line Fitting

  29. RANSAC: 1D Line Fitting

  30. RANSAC line RANSAC: 1D Line Fitting

  31. Match Verification

  32. Finding the panoramas

  33. Finding the panoramas

  34. Finding the panoramas

  35. Finding the panoramas

  36. Automatic Stitching • Feature Matching • Image Matching • Image Alignment • Bundle Adjustment • Automatic Straightening • Rendering • Results • Conclusions

  37. Motion Model Revisited • Recall our image motion model • Parameterise each camera by rotation and focal length

  38. Bundle Adjustment • Sum of squared projection errors • n = #images • I(i) = set of image matches to image i • F(i, j) = set of feature matches between images i,j • rijk = residual of kth feature match between images i,j • Huber (robust) error function

  39. Bundle Adjustment • Adjust rotation, focal length of each image to minimise error in matched features

  40. Bundle Adjustment • Adjust rotation, focal length of each image to minimise error in matched features

  41. Automatic Stitching • Feature Matching • Image Matching • Image Alignment • Bundle Adjustment • Automatic Straightening • Rendering • Results • Conclusions

  42. Automatic Straightening

  43. Automatic Straightening • Heuristic: user does not twist camera relative to horizon • Up-vector perpendicular to plane of camera x vectors

  44. Automatic Straightening

  45. Automatic Stitching • Feature Matching • Image Matching • Image Alignment • Rendering • Results • Conclusions

  46. Gain Compensation • No gain compensation

  47. Gain Compensation • Gain compensation • Single gain parameter gi for each image ( Better solution = HDR [ Debevec 1997 ] )

  48. Multi-band Blending • No blending

  49. Multi-band Blending • Linear blending • Each pixel is a weighted sum

  50. Multi-band Blending • Multi-band blending • Each pixel is a weighted sum (for each band)

More Related