1 / 74

Local invariant features

Local invariant features. Cordelia Schmid INRIA, Grenoble. ( ). Local features. local descriptor. Several / many local descriptors per image Robust to occlusion/clutter + no object segmentation required Photometric : distinctive

castillod
Download Presentation

Local invariant features

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Local invariant features Cordelia Schmid INRIA, Grenoble

  2. ( ) Local features local descriptor Several / many local descriptors per image Robust to occlusion/clutter + no object segmentation required Photometric : distinctive Invariant : to image transformations + illumination changes

  3. Local features: interest points

  4. Local features: Contours/segments

  5. Local features: segmentation

  6. Application: Matching Find corresponding locations in the image

  7. Matching algorithm • Extract descriptors for each image I1 and I2 • Compute similarity measure between all pairs of descriptors • Select couples according to different strategies • All matches above a threshold • Winner takes all • Cross-validation matching • Verify neighborhood constraints • Compute the global geometric relation (fundamental matrix or homography) robustly 6. Repeat matching using the global geometric relation

  8. Selection strategies • Winner takes all • The best matching pairs (with the highest score) is selected • All matches with the points and are removed • Cross-validation matching • For each point in image 1 keep the best match • For each point in image 2 keep the best match • Verify the matches correspond both ways

  9. Illustration – Matching Interest points extracted with Harris detector (~ 500 points)

  10. Illustration – Matching Matching Interest points matched based on cross-correlation (188 pairs)

  11. Illustration – Matching Global constraints Global constraint - Robust estimation of the fundamental matrix 89 outliers 99 inliers

  12. Application: Panorama stitching Images courtesy of A. Zisserman.

  13. Application: Instance-level recognition Search for particular objects and scenes in large databases …

  14. Difficulties Finding the object despite possibly large changes in scale, viewpoint, lighting and partial occlusion  requires invariant description Scale Viewpoint Occlusion Lighting

  15. Difficulties • Very large images collection  need for efficient indexing • Flickr has 2 billion photographs, more than 1 million added daily • Facebook has 15 billion images (~27 million added daily) • Large personal collections • Video collections, i.e., YouTube

  16. Applications Search photos on the web for particular places ...in these images and 1M more Find these landmarks

  17. Applications • Take a picture of a product or advertisement  find relevant information on the web [Pixee – Milpix]

  18. Applications • Finding stolen/missing objects in a large collection …

  19. Applications • Copy detection for images and videos Search in 200h of video Query video

  20. Sony Aibo – Robotics Recognize docking station Communicate with visual cards Place recognition Loop closure in SLAM Applications 20 K. Grauman, B. Leibe Slide credit: David Lowe

  21. Instance-level recognition: Approach • Extraction of invariant image descriptors • Matching descriptors between images • Matching of the query images to all images of a database • Speed-up by efficient indexing structures • Geometric verification • Verification of spatial consistency for a short list

  22. Local features - history • Line segments [Lowe’87, Ayache’90] • Interest points & cross correlation [Z. Zhang et al. 95] • Rotation invariance with differential invariants [Schmid&Mohr’96] • Scale & affine invariant detectors [Lindeberg’98, Lowe’99, Tuytelaars&VanGool’00, Mikolajczyk&Schmid’02, Matas et al.’02] • Dense detectors and descriptors [Leung&Malik’99, Fei-Fei& Perona’05, Lazebnik et al.’06] • Contour and region (segmentation) descriptors [Shotton et al.’05, Opelt et al.’06, Ferrari et al.’06,Leordeanu et al.’07]

  23. Local features 1) Extraction of local features • Contours/segments • Interest points & regions • Regions by segmentation • Dense features, points on a regular grid 2) Description of local features • Dependant on the feature type • Segments  angles, length ratios • Interest points  greylevels, gradient histograms • Regions (segmentation)  texture + color distributions

  24. Line matching • Extraction de contours • Zero crossing of Laplacian • Local maxima of gradients • Chain contour points (hysteresis) • Extraction of line segments • Description of segments • Mi-point, length, orientation, angle between pairs etc.

  25. Experimental results – line segments images 600 x 600

  26. Experimental results – line segments 248 / 212 line segments extracted

  27. Experimental results – line segments 89 matched line segments - 100% correct

  28. Experimental results – line segments 3D reconstruction

  29. Problems of line segments • Often only partial extraction • Line segments broken into parts • Missing parts • Information not very discriminative • 1D information • Similar for many segments • Potential solutions • Pairs and triplets of segments • Interest points

  30. Overview • Harris interest points • Comparison of IP: SSD, ZNCC, SIFT • Scale & affine invariant interest points (student presentation) • Evaluation and comparison of different detectors • Region descriptors and their performance

  31. Harris detector [Harris & Stephens’88] Based on the idea of auto-correlation Important difference in all directions => interest point

  32. Harris detector Auto-correlation function for a point and a shift

  33. Harris detector Auto-correlation function for a point and a shift { → uniform region small in all directions → contour large in one directions → interest point large in all directions

  34. Harris detector

  35. with first order approximation Harris detector Discret shifts are avoided based on the auto-correlation matrix

  36. the sum can be smoothed with a Gaussian Harris detector Auto-correlation matrix

  37. Harris detector • Auto-correlation matrix • captures the structure of the local neighborhood • measure based on eigenvalues of this matrix • 2 strong eigenvalues • 1 strong eigenvalue • 0 eigenvalue => interest point => contour => uniform region

  38. Interpreting the eigenvalues Classification of image points using eigenvalues of M: 2 “Edge” 2 >> 1 “Corner”1 and 2 are large,1 ~ 2;E increases in all directions 1 and 2 are small;E is almost constant in all directions “Edge” 1 >> 2 “Flat” region 1

  39. Corner response function α: constant (0.04 to 0.06) “Edge” R < 0 “Corner”R > 0 |R| small “Edge” R < 0 “Flat” region

  40. Reduces the effect of a strong contour Harris detector • Interest point detection • Treshold (absolut, relatif, number of corners) • Local maxima • Cornerness function

  41. Harris Detector: Steps

  42. Harris Detector: Steps Compute corner response R

  43. Harris Detector: Steps Find points with large corner response: R>threshold

  44. Harris Detector: Steps Take only the points of local maxima of R

  45. Harris Detector: Steps

  46. Harris detector: Summary of steps • Compute Gaussian derivatives at each pixel • Compute second moment matrix M in a Gaussian window around each pixel • Compute corner response function R • Threshold R • Find local maxima of response function (non-maximum suppression)

  47. Harris - invariance to transformations • Geometric transformations • translation • rotation • similitude (rotation + scale change) • affine (valide for local planar objects) • Photometric transformations • Affine intensity changes (I  a I + b)

  48. Harris Detector: Invariance Properties • Rotation Ellipse rotates but its shape (i.e. eigenvalues) remains the same Corner response R is invariant to image rotation

  49. Intensity scale: I  aI R R threshold x(image coordinate) x(image coordinate) Harris Detector: Invariance Properties • Affine intensity change • Only derivatives are used => invariance to intensity shift I I+b Partially invariant to affine intensity change, dependent on type of threshold

  50. Harris Detector: Invariance Properties • Scaling Corner All points will be classified as edges Not invariant to scaling

More Related