200 likes | 355 Views
Special Topic on Image Retrieval. Local Feature Matching Verification. Geometric Verification. Motivation Remove false matches by checking geometric consistency. Red line : geometric consistent match Blue line : geometric inconsistent match. Global Verification: RANSAC.
E N D
Special Topic on Image Retrieval Local Feature Matching Verification
Geometric Verification • Motivation • Remove false matches by checking geometric consistency Red line: geometric consistent match Blue line: geometric inconsistent match
Global Verification: RANSAC • Take RANSAC as an example • Check geometric consistency from matched feature pairs. Random sampling
Local Geometric-Verification • Locally nearest neighbors (Video Goole, cvpr’03) • Matched regions should have a similar spatial layout. • For each match define its search area • Region in the search area that also matches casts a vote for the image • Reject matches with no support • Drawback • Sensitive to clutter
Hamming Embedding (ECCV’08) • Introduced as an extension of BOV [Jegou 08] • Combination of • A partitioning technique (k-means) • A binary code that refine the descriptor • Representation of a descriptor x • Vector-quantized to q(x) as in standard BOV • short binary vector b(x) for an additional localization in the Voronoi cell • Two descriptors x and y match iif
Hamming Embedding • Binary signature generation • Off-line learning • Random matrix generation • Descriptor projection and assignment • Median values of projected descriptors • On-line binarization • Quantization assignment • Descriptor projection • Computing the signature:
Local Geometric-Verification • Bundled feature (CVPR’09) • Group local features in local MSER region. • Increase discriminative power of visual words. • Allowed to have large overlap error. • Bundle comparison: • Mm(q; p): number of common visual words between two bundles • Mg(q; p): inconsistency of geometric order in x- and y- direction. • Drawbacks: Infeasible for rotated bundles.
Local Geometric-Verification Bundled feature (CVPR’09) • Visual words are bundled in MSER regions. • Spatial consistency for bundled features is utilized to weight visual words. # of shared visual words • Great performance for partial-dup detection in over 1 M database • Drawbacks: Infeasible for rotated bundles. Spatial consistency • Z. Wu, J. Sun, and Q. Ke, “Bundling Features for Large Scale Partial-Duplicate Web Image Search,” CVPR 09
Global Verification: RANSAC • RANSAC: remove outliers by inlier classification • Inliers: true matched features • Outliers: false matched features • Assumption of RANdomSAmpleConsensus (RANSAC) • The original data consists of inliers and outliers. • A subset of inliers can estimate a model to optimally explain the inliers. • Estimate the affine transformation by RANSAC • Procedure: Iteratively select a random subset as hypothetical inliers • A model is fitted to the hypothetical inliers. • All other data are tested against the fitted model for inlier classification. • The model is re-estimated from all hypothetical inliers. • The model is evaluated by estimating the error of the inliers relative to the model. • Drawbacks: Computationally expensive, not scalable Fischler, et al., RANdomSAmpleConsensus: a paradigm for model fitting with applications to image analysis and automated cartography, Comm. of the ACM, 24:381-395, 1981
Spatial Coding for Geometric Verification (ACM MM’ 10) • Motivation • Encode local features’ relative positions into compact binary maps • Check spatial consistency of local matches for geometric verification • Spatial coding maps • Relative spatial positions between local features. • Very efficient and high precision Zhou & Tian, Spatial Coding for large scale partial-duplicate image search. ACM Multimedia 2010.
Spatial Map Generation • In previous case, each quadrant has one part • Consider each quadrant is uniformly divided into two parts. = Rotate 45 degree counterclockwise
Spatial Map Generation • Generalized spatial map: GX and GY • Each quadrant is uniformly divided into r parts. X-map X-map X-map Y-map Y-map Y-map k=0 k=1 … … k=r-1
Generalized Spatial Coding • Spatial coding maps: • Each quadrant uniformly divided into r parts. • Decompose the division into r sub-division. • Rotate each sub-division to align the axis. New featurelocations after rotation : Generalized spatial maps:
Spatial Verification • Verification with spatial maps GX and GY • Compare the spatial maps of matched features: • k=0, …, r-1; i, j=1, …, N; N: number of matched features • Find and delete the most inconsistent matched pair, recursively: Vx: inconsistent degree in X-map Vy: inconsistent degree in Y-map Identify i* and remove
5 y 1 2 1 2 3 3 4 4 x 5
Geometric Verification with Coding Maps 5 1 y 2 1 2 3 3 4 4 x 5 SUM
4 4 4 Image Plane Division (TOMCCAP’ 10) 3 3 3 2 2 2 1 1 1 5 5 5 4 4 4 (c) (a) (b) 3 3 3 2 2 2 5 5 5 1 1 1 (f) (d) (e)
4 Geometric Square Coding 3 2 • Coordinate adjustment • Square coding map 1 5 4 3 2 5 Generalized map: 1
Geometric Fan Coding • Fan coding maps • Coordinate adjustment • Generalized coding maps
Geometric Verification • Compare the fan coding maps of matched features: • Inconsistency measurement from geometric fan coding: • Inconsistency measurement from geometric square coding: • Inconsistency matrix: