1 / 65

Image Stitching

Image Stitching. Tamara Berg CSE 590 Computational Photography. Many slides from Alyosha Efros & Derek Hoiem. How can we align two pictures?. What about global matching?. How can we align two pictures?. Global matching? But what if Not just translation change, but rotation and scale?

marcus
Download Presentation

Image Stitching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Image Stitching Tamara Berg CSE 590 Computational Photography Many slides from AlyoshaEfros & Derek Hoiem

  2. How can we align two pictures? • What about global matching?

  3. How can we align two pictures? • Global matching? • But what if • Not just translation change, but rotation and scale? • Only small pieces of the pictures match?

  4. Keypoint Matching 1. Find a set of distinctive key- points 2. Define a region around each keypoint A1 B3 3. Extract and normalize the region content A2 A3 B2 B1 4. Compute a local descriptor from the normalized region 5. Match local descriptors K. Grauman, B. Leibe

  5. Main challenges • Change in position, scale, and rotation • Change in viewpoint • Occlusion • Articulation, change in appearance

  6. Question • Why not just take every patch in the original image and find best match in second image?

  7. Goals for Keypoints Detect points that are repeatable and distinctive

  8. Key trade-offs B3 A1 A2 A3 B2 B1 Localization More Repeatable More Points Robust to occlusion Works with less texture Robust detection Precise localization Description More Selective More Robust Minimize wrong matches Deal with expected variations Maximize correct matches

  9. Keypoint Localization • Goals: • Repeatable detection • Precise localization K. Grauman, B. Leibe

  10. Which patches are easier to match? ?

  11. Choosing interest points • If you wanted to meet a friend would you say • “Let’s meet on campus.” • “Let’s meet on Green street.” • “Let’s meet at Green and Wright.” • Or if you were in a secluded area: • “Let’s meet in the Plains of Akbar.” • “Let’s meet on the side of Mt. Doom.” • “Let’s meet on top of Mt. Doom.”

  12. Choosing interest points • Corners • “Let’s meet at Green and Wright.” • Peaks/Valleys • “Let’s meet on top of Mt. Doom.”

  13. Many Existing Detectors Available Hessian & Harris[Beaudet ‘78], [Harris ‘88] Laplacian, DoG[Lindeberg ‘98], [Lowe 1999] Harris-/Hessian-Laplace [Mikolajczyk & Schmid ‘01] Harris-/Hessian-Affine [Mikolajczyk & Schmid ‘04] EBR and IBR [Tuytelaars & Van Gool ‘04] MSER[Matas ‘02] Salient Regions [Kadir & Brady ‘01] Others… K. Grauman, B. Leibe

  14. Harris Detector [Harris88] Intuition: Search for local neighborhoods where the image content has two main directions. K. Grauman, B. Leibe

  15. Harris Detector – Responses [Harris88] Effect: A very precise corner detector.

  16. Harris Detector – Responses [Harris88]

  17. So far: can localize in x-y, but not scale

  18. Automatic Scale Selection How to find corresponding patch sizes? K. Grauman, B. Leibe

  19. Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

  20. Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

  21. Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

  22. Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

  23. Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

  24. Automatic Scale Selection • Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

  25. What Is A Useful Signature Function? • Difference of Gaussian = “blob” detector K. Grauman, B. Leibe

  26. DoG – Efficient Computation • Computation in Gaussian scale pyramid Sampling withstep s4=2 s s s s Original image K. Grauman, B. Leibe

  27. Results: Lowe’s DoG K. Grauman, B. Leibe

  28. p 2 0 Orientation Normalization [Lowe, SIFT, 1999] • Compute orientation histogram • Select dominant orientation • Normalize: rotate to fixed orientation T. Tuytelaars, B. Leibe

  29. Available at a web site near you… • For most local feature detectors, executables are available online: • http://robots.ox.ac.uk/~vgg/research/affine • http://www.cs.ubc.ca/~lowe/keypoints/ • http://www.vision.ee.ethz.ch/~surf K. Grauman, B. Leibe

  30. How do we describe the keypoint?

  31. Local Descriptors • The ideal descriptor should be • Robust • Distinctive • Compact • Efficient • Most available descriptors focus on edge/gradient information • Capture texture information • Color rarely used K. Grauman, B. Leibe

  32. Local Descriptors: SIFT Descriptor • Histogram of oriented gradients • Captures important texture information • Robust to small translations / affine deformations [Lowe, ICCV 1999] K. Grauman, B. Leibe

  33. What to use when? Detectors • Harris gives very precise localization but doesn’t predict scale • Good for some tracking applications • DOG (difference of Gaussian) provides ok localization and scale • Good for multi-scale or long-range matching Descriptors • SIFT: good general purpose descriptor

  34. Things to remember • Keypoint detection: repeatable and distinctive • Corners, blobs • Harris, DoG • Descriptors: robust and selective • SIFT: spatial histograms of gradient orientation

  35. Image Stitching • Combine two or more overlapping images to make one larger image Add example Slide credit: Vaibhav Vaish

  36. Panoramic Imaging • Higher resolution photographs, stitched from multiple images • Capture scenes that cannot be captured in one frame • Cheaply and easily achieve effects that used to cost a lot of money

  37. Pike’s Peak Highway, CO Photo: Russell J. Hewett Nikon D70s, Tokina 12-24mm @ 16mm, f/22, 1/40s

  38. Pike’s Peak Highway, CO Photo: Russell J. Hewett (See Photo On Web)

  39. 360 Degrees, Tripod Leveled Photo: Russell J. Hewett Nikon D70, Tokina 12-24mm @ 12mm, f/8, 1/125s

  40. Howth, Ireland Photo: Russell J. Hewett (See Photo On Web)

  41. Capturing Panoramic Images • Tripod vs Handheld • Help from modern cameras • Leveling tripod • Or wing it • Image Sequence • Requires a reasonable amount of overlap (at least 15-30%) • Enough to overcome lens distortion • Exposure • Consistent exposure between frames • Gives smooth transitions • Manual exposure • Caution • Distortion in lens (Pin Cushion, Barrel, and Fisheye) • Motion in scene

  42. Handheld Camera Photo: Russell J. Hewett Nikon D70s, Nikon 18-70mm @ 70mm, f/6.3, 1/200s

  43. Handheld Camera Photo: Russell J. Hewett

  44. Les Diablerets, Switzerland Photo: Russell J. Hewett (See Photo On Web)

  45. Macro Photo: Russell J. Hewett & Bowen Lee Nikon D70s, Tamron 90mm Micro @ 90mm, f/10, 15s

  46. Side of Laptop Photo: Russell J. Hewett & Bowen Lee

  47. Ghosting and Variable Intensity Photo: Russell J. Hewett Nikon D70s, Tokina 12-24mm @ 12mm, f/8, 1/400s

  48. Photo: Russell J. Hewett

  49. Ghosting From Motion Photo: Bowen Lee Nikon e4100 P&S

  50. Motion Between Frames Photo: Russell J. Hewett Nikon D70, Nikon 70-210mm @ 135mm, f/11, 1/320s

More Related