SVD-SIFT FOR WEB NEAR-DUPLICATE IMAGE DETECTION

1. SVD-SIFT FOR WEB NEAR-DUPLICATE IMAGE DETECTION Image Processing (ICIP), 2010 17th IEEE International Conference on September 26-29, 2010

2. Outline • Near-duplicate image • Scale Invariant Feature Transform (SIFT) & SURF • Introduction • Singular Value Decomposition (SVD) Theorem • Low-rank matrix approximation • SVD-SIFT • Experiment Result • Conclusion

3. Near-duplicate image Near-Duplicate Image Detection

4. Web near-duplicate image Near-duplicate image type demonstration and Detection effect based on SIFT descriptor

5. SIFT & SURF 593 feature points 34 pairs of feature points match 782 feature points

6. Introduction • Two kinds of Methods for near-duplicate image detection : • Global features • Performance good when image format, size and quality change, but poor for the complex edition techniques such as inserting, forging, etc. • Local features • It has good detection effect on the near-duplicate images processed with different formats, size transformation, complex edition, post processing, etc. However, the disadvantage of local descriptor methods is the high computational cost.

7. Introduction • SIFT is the kind of method based on local features. • In this paper, we propose SVD-SIFT feature, it uses the property of SIFT feature itself to improve the matching speed. Near-duplicate image type demonstration and Detection effect based on SIFT descriptor

8. SVD Theorem • The matrix SVD theorem can be described as follows: • If M∈ Rm×n(m > n), rank(M) = r, then there exists two orthogonal matrices U, V , and a diagonal matrix makes the establishment of the following equation: Where U = [u1, u2, u3, ..., um] ∈ Rm×m, UT =U-1, UUT=UTU=I ; V = [v1, v2, v3, ..., vn] ∈ Rn×n , VT =V-1, VVT= VTV = I ; Σ = diag(σ1, σ2, ..., σr, 0, ..., 0) ∈ Rm×n, σ1 ≥ σ2 ≥ ... ≥ σr . = mxn mxm mxn nxn

9. SVD Theorem mxn mxn = • The singular value decomposition and theeigenvalue decomposition are closely related. Namely: • The left singular vectors of M are eigenvectors of MMT . • The right singular vectors of M are eigenvectors of MTM . • The non-zero singular values of Σ are the square roots of the non-zero eigenvalues of MTM or MMT . v1 v2 u1 u2...um ... vn The left singular vectors of M The right singular vectors of M

10. SVD Theorem • The matrix singular value has the following characteristics: • Characteristic 1: Transposition and replacement invariance • That is to say, after transposition or row-column replacement operation of the matrix, its singular value is invariable. • Characteristic 2: Energy concentricity • The matrix M can be approximately restructured by the first k largest singular values of M. It can be proved that the matrix corresponding to the first k largest singular values of A is the closest to matrix M under the Frobenius norm. (Low-rank matrix approximation)

11. Low-rank matrix approximation • Consider a matrix M ∈ Rm×nand itssingular value decomposition M = UΣVT, then the matrix Min the Frobeniusnorm of rank k(k ≤ min(m, n)), the bestapproximation matrix can be expressed using the followingformula: Where U = [u1, u2, u3, ..., um] ∈ Rm×m V = [v1, v2, v3, ..., vn] ∈ Rn×n = diag(σ1, σ2, ..., σk, 0, ..., 0) ∈ Rm×n U = [u1, u2, u3, ..., um] ∈ Rm×m V = [v1, v2, v3, ..., vn] ∈ Rn×n Σ= diag(σ1, σ2, ..., σr, 0, ..., 0) ∈ Rm×n

12. Low-rank matrix approximation

13. SVD-SIFT • The image SIFT feature is one set that contains many local feature points, each feature point is described by a 128-dimensional vector. Therefore, the SIFT feature points set of one image can be represented with a matrix. n SVD-SIFT extraction and matching based on SIFT feature points set.

14. SVD-SIFT • According to Characteristic 1: • The singular value of the image SIFT feature matrix is not related to the position of SIFT feature point. Then, we do not need to care about the position of the SIFT points when matching. • According to Characteristic 2: • We can use the energy concentricity of the first k largest singular values of the image SIFT feature matrix to greatly reduce the matching time cost.

15. SVD-SIFT • Suppose that A and B represent two images containing m and n SIFT feature points respectively, the matching algorithm has the following four steps : • Step 1: • Matrix A128×m= (A1,A2, ...,Am) represents the feature point set of image A and matrix B128×n = (B1,B2, ...,Bn) represents the feature points set of image B, respectively. • Step 2: • A d-dimensional linear subspace of A and B is represented by an orthonormal basis matrix PA ∈ A128×dand PB∈ B128×drespectively subject to AATPAΛAPATandBBTPBΛBPBT, where ΛAand ΛBare the eigenvalue diagonal matrices of the d largest eigenvalues, PA and PBare the eigenvector matrices of the d largest eigenvalues.

16. SVD-SIFT • Step 3: • Using the first k largest values of ΛAand ΛB respectively to constitute the SIFT singular value feature vector of images A and B. • Step 4: • Two measurement methods can be chosen when matching: • The L1 distance is used to compute the similarity of k dimensional SIFT singular value characteristic; • The singular value decomposition is carried out for PATPB∈ Rd×d, so PATPB= USVT, then the similarity between point sets A and B can be measured by the trace of the singular value matrix S, sim(A,B) = trace(S).

17. Experiment Result • Ordinal Intensity Signature(OIS) • Global features • L1 distance matching • SIFT • Local features • Best-Bin-First-Tree matching • SVD-SIFT • Global features • L1 distance matching

18. Experiment Result • Precision (P) • Recall (R)

19. Experiment Result • The detection time includes the time for feature extraction and pre-processing of the query image.

20. Conclusion • The computational costs of directly matching two SIFT feature points sets is high. In this paper, SVD-SIFT has been proposed and it has been theoretically proven. • According to energy concentricity of matrix singular value, the first k largest singular values of image SIFT feature matrix maintains the original characteristic of the matrix well, experimental results demonstrate that the method can obtain a better tradeoff between the effectiveness and efficiency for detection.