1 / 60

Image Stitching: Intro to RANSAC and Midquiz Slides

This presentation covers the basics of image stitching, including keypoint detection, local descriptor computation, homography estimation using RANSAC, and projection onto a surface. The lecture also provides an overview of the image stitching algorithm and different projection surfaces. The slides feature examples and explanations to help understand the concepts.

reinaldol
Download Presentation

Image Stitching: Intro to RANSAC and Midquiz Slides

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Stitching, Intro to RANSAC • Midquiz, March 5 Slides originally by Derek Hoeim, image above from JPL

  2. Last Class: Keypoint Matching 1. Find a set of distinctive key- points 2. Define a region around each keypoint A1 B3 3. Extract and normalize the region content A2 A3 B2 B1 4. Compute a local descriptor from the normalized region 5. Match local descriptors K. Grauman, B. Leibe

  3. Last Class: Summary • Keypoint detection: repeatable and distinctive • Local Max (spatially and scale) of feature score • Descriptors: robust and selective • SIFT: spatial histograms of gradient orientation

  4. Today: Image Stitching • Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish

  5. Homography Reminder • p = K [R t] P • p’ = K’ [R’ t’] P’ • t=t’=0 • p‘=Hpwhere H = K’ R’ R-1 K-1 • Typically only R and f will change (4 parameters), but, in general, H has 8 parameters . X x x' f f'

  6. Views from rotating camera Camera Center

  7. Image Stitching Algorithm Overview • Detect keypoints • Match keypoints • Estimate homography with four matched keypoints (using RANSAC) • Project onto a surface and blend

  8. Image Stitching Algorithm Overview • Detect/extract keypoints (e.g., DoG/SIFT) • Match keypoints (most similar features, compared to 2nd most similar) • Compute Homography. • Declare Success

  9. Homography math reminder: Hp = p’ Write out the lines of this matrix equation. And remember which variables are unknown. Computer Vision, Robert Pless

  10. Ax=b Matlab: x = A\b Then make your homographymatrix H by rearranging x into a 3 x 3 matrix Computer Vision, Robert Pless

  11. Computing homographies in the real world • Assume we have matched points with outliers: How do we compute homographyH? Automatic Homography Estimation with RANSAC

  12. Computing homography • Assume we have matched points with outliers: How do we compute homographyH? Automatic Homography Estimation with RANSAC • Choose number of samples N • Choose 4 random potential matches • Compute H • Project points from x to x’ for each potentially matching pair: • Count points with projected distance < t • E.g., t = 3 pixels • Repeat steps 2-5 N times • Choose H with most inliers How many samples do we need? HZ Tutorial ‘99

  13. Reprise: Automatic Image Stitching • Compute interest points on each image • Find candidate matches • Estimate homographyH using matched points and RANSAC with normalized DLT • Project each image onto the same surface and blend

  14. Choosing a Projection Surface Many to choose: planar, cylindrical, spherical, cubic, etc.

  15. Planar Mapping x x f f For red image: pixels are already on the planar surface For green image: map to first image plane

  16. Planar vs. Cylindrical Projection Planar Photos by Russ Hewett

  17. Spherical Mapping x x f f For red image: compute h, theta on cylindrical surface from (u, v) How can you do this from the homography equation?

  18. Cylindrical Mapping x x f f For red image: compute h, theta on cylindrical surface from (u, v) How can you do this from the homography equation?

  19. Why does it matter?

  20. Planar vs. Cylindrical Projection Planar Photos by Russ Hewett

  21. Planar vs. Cylindrical Projection Cylindrical

  22. Simple gain adjustment

  23. Stitching bigger sets of images together. • … or the incredible beauty of associative groups

  24. Recognizing Panoramas Brown and Lowe 2003, 2007 Some of following material from Brown and Lowe 2003 talk

  25. Recognizing Panoramas Input: N images • Extract SIFT points, descriptors from all images • Find K-nearest neighbors for each point (K=4) • For each image • Select M candidate matching images by counting matched keypoints (M=6) • Solve homographyHij for each matched image

  26. Recognizing Panoramas Input: N images • Extract SIFT points, descriptors from all images • Find K-nearest neighbors for each point (K=4) • For each image • Select M candidate matching images by counting matched keypoints (M=6) • Solve homographyHij for each matched image • Decide if match is valid (ni > 8 + 0.3 nf ) # keypointsin overlapping area # inliers

  27. RANSAC for Homography Initial Matched Points

  28. RANSAC for Homography Final Matched Points

  29. Verification

  30. RANSAC for Homography

  31. Recognizing Panoramas (cont.) (now we have matched pairs of images) • Find connected components

  32. Finding the panoramas

  33. Finding the panoramas

  34. Finding the panoramas

  35. What’s left? Fancier blending (really a graphics problem  ) • Gain compensation: minimize intensity difference of overlapping pixels • Blending • Pixels near center of image get more weight • Multiband blending to prevent blurring

  36. Multi-band Blending (Laplacian Pyramid) • Burt & Adelson 1983 • Blend frequency bands over range l

  37. Multiband blending

  38. Blending comparison (IJCV 2007)

  39. Blending Comparison

  40. Tips and Photos from Russ Hewett

  41. Capturing Panoramic Images • Tripod vs Handheld • Help from modern cameras • Leveling tripod • Gigapan • Or wing it • Image Sequence • Requires a reasonable amount of overlap (at least 15-30%) • Enough to overcome lens distortion • Exposure • Consistent exposure between frames • Gives smooth transitions • Manual exposure • Makes consistent exposure of dynamic scenes easier • But scenes don’t have constant intensity everywhere • Caution • Distortion in lens (Pin Cushion, Barrel, and Fisheye) • Polarizing filters • Sharpness in image edge / overlap region

  42. Pike’s Peak Highway, CO Photo: Russell J. Hewett Nikon D70s, Tokina 12-24mm @ 16mm, f/22, 1/40s

  43. Pike’s Peak Highway, CO Photo: Russell J. Hewett (See Photo On Web)

  44. 360 Degrees, Tripod Leveled Photo: Russell J. Hewett Nikon D70, Tokina 12-24mm @ 12mm, f/8, 1/125s

  45. Howth, Ireland Photo: Russell J. Hewett (See Photo On Web)

  46. Handheld Camera Photo: Russell J. Hewett Nikon D70s, Nikon 18-70mm @ 70mm, f/6.3, 1/200s

  47. Handheld Camera Photo: Russell J. Hewett

  48. Les Diablerets, Switzerland Photo: Russell J. Hewett (See Photo On Web)

  49. Macro Photo: Russell J. Hewett & Bowen Lee Nikon D70s, Tamron 90mm Micro @ 90mm, f/10, 15s

  50. Side of Laptop Photo: Russell J. Hewett & Bowen Lee

More Related