1.4k likes | 1.66k Views
Mehdi Faraji farajimhd@gmail.com. Computer Vision. Local Invariant Features. SLIDES have been prepared by: Dr. Ghassabi. Outline. Why do we care about matching features? Problem Statement Properties of features Types of invariance Introduction to feature matching
E N D
Mehdi Faraji farajimhd@gmail.com Computer Vision Local Invariant Features SLIDES have been prepared by: Dr. Ghassabi
Outline • Why do we care about matching features? • Problem Statement • Properties of features • Types of invariance • Introduction to feature matching • Matching using invariant descriptors • Feature Detection • Corner Detection • Moravec, harris • Harris properties (rotation, intensity, scale invariance) • Low’s key point • Feature description • SIFT (Scale Invariant Feature Transform) • SIFT Extensions: PCA-SIFT, GLoH ,SPIN image, RIFT, • Feature matching • Applications (examples) • Future Works • Conclusion Outline Motivation Problems statement How we solve it Future Work Reference
Motivation • Why do we care about matching features? • image stitching, • object recognition, • Indexing and database retrieval, • Motion tracking • … Others Outline Motivation Problems statement How we solve it Future Work Reference
Example: How do we build panorama? We need to match (align) images
Matching with Features Detect feature points in both images
Matching with Features Detect feature points in both images Find corresponding pairs
Matching with Features Detect feature points in both images Find corresponding pairs Use these pairs to align images
Types of variance • Illumination • Scale • Rotation • Affine • Full Perspective • Problems statements • Properties of good features Outline Motivation Problems statement How we solve it Future Work Reference
Types of variance • Outline • Motivation • Problems statement • Types of variance • Problem1 • Problem2 • Properties of good features • How we solve it • Future Work • Reference • Illumination
Types of variance • Outline • Motivation • Problems statement • Types of variance • Problem1 • Problem2 • Properties of good features • How we solve it • Future Work • Reference • Illumination • Scale
Types of variance • Outline • Motivation • Problems statement • Types of variance • Problem1 • Problem2 • Properties of good features • How we solve it • Future Work • Reference • Illumination • Scale • Rotation
Types of variance • Outline • Motivation • Problems statement • Types of variance • Problem1 • Problem2 • Properties of good features • How we solve it • Future Work • Reference • Illumination • Scale • Rotation • Affine
Types of variance • Outline • Motivation • Problems statement • Types of variance • Problem1 • Problem2 • Properties of good features • How we solve it • Future Work • Reference • Illumination • Scale • Rotation • Affine • Full Perspective
Problems statement • Problem 1: • Detect the same point independently in both images • Outline • Motivation • Problems statement • Types of variance • Problem1 • Problem2 • Properties of good features • How we solve it • Future Work • Reference no chance to match! We need a repeatable detector How to find landmarks to match across two images? How achieve landmarks invariance to scale, rotation, illumination distortions?
Problems statement • Problem 2: • For each point correctly recognize the corresponding one • Outline • Motivation • Problems statement • Types of variance • Problem1 • Problem2 • Properties of good features • How we solve it • Future Work • Reference ? We need a reliable and distinctive descriptor How to distinguish one landmark from another?
Properties of features • Distinctiveness • Invariance • Invariance to illumination, scale, Rotation, Affine, full perspective Good features should be robust to all sorts of distortions that can occur between images. • Outline • Motivation • Problems statement • Types of variance • Problem1 • Problem2 • Properties of good features • How we solve it • Future Work • Reference
Methods using invariant descriptors • Methods using invariant descriptorsInvarianceto: transformation change in illumination image noiseDistinctiveness • Local features • Feature Detector • Feature descriptor • Feature-matching • Outline • Motivation • Problems statement • How we solve it • Methods of Feature matching • Invariant descriptors • Future Work • Reference
Methods using invariant descriptors • Local features • Feature Detector • Point detector • Corner detectors • Moravec, harris, SUSAN, Trajkovic operators • Low’s key point • Region detector • Harris-Laplase, Harris affine, Hessian affine, edge-based, Intensity-based, salient region detectors • Feature descriptor • Feature-matching • Outline • Motivation • Problems statement • How we solve it • Methods of Feature matching • Invariant descriptors • Future Work • Reference
Methods using invariant descriptors • Local features • Feature Detector • Feature descriptor • Filter-based • Steerable filters • Gabor filters • Complex filters • Distribution-based • Local • SIFT, PCA-SIFT, GLOH, Spin image, RIFT,, SURF • global • Shape context • Textons • Derivative-based • Others • Moment-based, Phase-based, Color-based • Feature-matching • Outline • Motivation • Problems statement • How we solve it • Methods of Feature matching • Invariant descriptors • Future Work • Reference
Moravec corner detector (1980):Idea • We should easily recognize the point by looking through a small window • Shifting a window in anydirection should give a large change in intensity
Moravec corner detector:Idea flat no change in all directions
Moravec corner detector:Idea flat edge no change along the edge direction
Moravec corner detector:Idea corner isolated point flat edge significant change in all directions
Moravec corner detector:Idea • Feature detection: the math • Consider shifting the window W by (u,v) • how do the pixels in W change? • compare each pixel before and after bysumming up the squared differences (SSD) • this defines an SSD “error” of E(u,v): W
Window function Shifted intensity Intensity E u v Moravec corner detector:Idea Change of intensity for the shift [u,v]: Four shifts: (u,v) = (1,0), (1,1), (0,1), (-1, 1) Look for local maxima in min{E}
Problems of Moravec detector • Noisy response due to a binary window function • Only a set of shifts at every 45 degree is considered • Only minimum of E is taken into account • Harris corner detector (1988) solves these problems.
Harris corner detector : the math Noisy response due to a binary window function • Use a Gaussian function
Harris corner detector : the math Equivalently, for small shifts [u,v] we have a bilinear approximation: , where M is a 22 matrix computed from image derivatives: • You can move the center of the green window to anywhere on the blue unit circle • Which directions will result in the largest and smallest E values? • We can find these directions by looking at the eigenvectors of M
Harris corner detector Intensity change in shifting window: eigenvalue analysis 1, 2 – eigenvalues of M direction of the fastest change Ellipse E(u,v) = const direction of the slowest change (max)-1/2 (min)-1/2
Harris corner detector Classification of image points using eigenvalues of M: 2 edge 2 >> 1 Corner 1 and 2 are large,1 ~ 2;E increases in all directions 1 and 2 are small;E is almost constant in all directions edge 1 >> 2 flat 1
Selecting Good Features l1 and l2 are large
Selecting Good Features large l1, small l2
Selecting Good Features small l1, small l2
Harris corner detector: the math Responds too strong for edges because only minimum of E is taken into account • A new corner measurement
Harris corner detector: the math Measure of corner response: (k – empirical constant, k = 0.04-0.06) • The Algorithm: • Find points with large corner response function R (R > threshold) • Take the points of local maxima of R
Harris Detector 2 “Edge” “Corner” • R depends only on eigenvalues of M • R is large for a corner • R is negative with large magnitude for an edge • |R| is small for a flat region R < 0 R > 0 “Flat” “Edge” |R| small R < 0 1
Harris detector: summary • Average intensity change in direction [u,v] can be expressed as a bilinear form: • Describe a point in terms of eigenvalues of M:measure of corner response • A good (corner) point should have a large intensity change in all directions, i.e. R should be large positive