1 / 39

Local features and image matching

Local features and image matching. October 1 st 2015 Devi Parikh Virginia Tech.

annettev
Download Presentation

Local features and image matching

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Local features and image matching October 1st 2015 Devi Parikh Virginia Tech Disclaimer: Many slides have been borrowed from Kristen Grauman, who may have borrowed some of them from others. Any time a slide did not already have a credit on it, I have credited it to Kristen. So there is a chance some of these credits are inaccurate.

  2. Announcements • PS2 due Monday • PS3 out • Due October 19th 11:55 pm Slide credit: Kristen Grauman

  3. Topics overview • Features & filters • Grouping & fitting • Multiple views and motion • Homography and image warping • Local invariant features • Image formation • Epipolar geometry • Stereo and structure from motion • Recognition • Video processing Slide credit: Kristen Grauman

  4. Numerical Issues • When computing H • Say true match is [50 100 1]  [50 100] • [50.5 100 1] • [50.5 100] • [50 100 1.5] • [33 67] • Scale co-ordinates to lie between 0 and 2

  5. Topics overview • Features & filters • Grouping & fitting • Multiple views and motion • Homography and image warping • Local invariant features • Image formation • Epipolar geometry • Stereo and structure from motion • Recognition • Video processing Slide credit: Kristen Grauman

  6. Last time • Image mosaics • Fitting a 2D transformation • Affine, Homography • 2D image warping • Computing an image mosaic

  7. Robust feature-based alignment Source: L. Lazebnik

  8. Robust feature-based alignment • Extract features Source: L. Lazebnik

  9. Robust feature-based alignment • Extract features • Compute putative matches Source: L. Lazebnik

  10. Robust feature-based alignment • Extract features • Compute putative matches • Loop: • Hypothesize transformation T (small group of putative matches that are related by T) Source: L. Lazebnik

  11. Robust feature-based alignment • Extract features • Compute putative matches • Loop: • Hypothesize transformation T (small group of putative matches that are related by T) • Verify transformation (search for other matches consistent with T) Source: L. Lazebnik

  12. Robust feature-based alignment • Extract features • Compute putative matches • Loop: • Hypothesize transformation T (small group of putative matches that are related by T) • Verify transformation (search for other matches consistent with T) Source: L. Lazebnik

  13. Today How to detect which features to match?

  14. Detecting local invariant features • Detection of interest points • Harris corner detection • Scale invariant blob detection: LoG (next time) • Description of local patches (next time)

  15. Local features: main components • Detection: Identify the interest points • Description:Extract vector feature descriptor surrounding each interest point. • Matching: Determine correspondence between descriptors in two views Kristen Grauman

  16. Local features: desired properties • Repeatability • The same feature can be found in several images despite geometric and photometric transformations • Saliency • Each feature has a distinctive description • Compactness and efficiency • Many fewer features than image pixels • Locality • A feature occupies a relatively small area of the image; robust to clutter and occlusion

  17. Goal: interest operator repeatability • We want to detect (at least some of) the same points in both images. • Yet we have to be able to run the detection procedure independently per image. No chance to find true matches!

  18. Goal: descriptor distinctiveness • We want to be able to reliably determine which point goes with which. • Must provide some invariance to geometric and photometric differences between the two views. ?

  19. Local features: main components • Detection: Identify the interest points • Description:Extract vector feature descriptor surrounding each interest point. • Matching: Determine correspondence between descriptors in two views

  20. What points would you choose?

  21. “flat” region:no change in all directions “edge”:no change along the edge direction “corner”:significant change in all directions Corners as distinctive interest points • We should easily recognize the point by looking through a small window • Shifting a window in anydirection should give a large change in intensity Slide credit: AlyoshaEfros, Darya Frolova, Denis Simakov

  22. Corners as distinctive interest points 2 x 2 matrix of image derivatives (averaged in neighborhood of a point). Notation:

  23. What does this matrix reveal? First, consider an axis-aligned corner:

  24. What does this matrix reveal? First, consider an axis-aligned corner: This means dominant gradient directions align with x or y axis Look for locations where bothλ’s are large. If either λ is close to 0, then this is notcorner-like. What if we have a corner that is not aligned with the image axes?

  25. What does this matrix reveal? Since Mis symmetric, we have The eigenvalues of M reveal the amount of intensity change in the two principal orthogonal gradient directions in the window.

  26. “flat”region “edge”: “corner”: Corner response function 1 >> 2 1 and 2 are small; 1 and 2 are large,1 ~ 2; 2 >> 1 2 1

  27. Harris corner detector • Compute M matrix for each image window to get their cornernessscores. • Find points whose surrounding window gave large corner response (f> threshold) • Take the points of local maxima, i.e., perform non-maximum suppression

  28. Example of Harris application Kristen Grauman

  29. Example of Harris application • Compute corner response at every pixel. Kristen Grauman

  30. Example of Harris application Kristen Grauman

  31. Properties of the Harris corner detector • Rotation invariant? • Scale invariant? Yes

  32. Properties of the Harris corner detector • Rotation invariant? • Scale invariant? Yes No All points will be classified as edges Corner !

  33. Harris Detector: Steps Slide credit: Kristen Grauman

  34. Harris Detector: Steps Compute corner response f Slide credit: Kristen Grauman

  35. Harris Detector: Steps Find points with large corner response: f > threshold Slide credit: Kristen Grauman

  36. Harris Detector: Steps Take only the points of local maxima of f Slide credit: Kristen Grauman

  37. Harris Detector: Steps Slide credit: Kristen Grauman

  38. Summary • Image warping to create mosaic, given homography • Interest point detection • Harris corner detector • Next time: • Laplacian of Gaussian, automatic scale selection

  39. Questions? • See you Tuesday!

More Related