SURF: Speeded Up Robust Features

1. SURF: Speeded Up Robust Features 授課教授: 連震杰 教授 Group number: 20 Advisor: Tzuu-Hseng S. Li Group members: E24956552 何雅芳 E24951099 蕭信揚 N26984224 李佳樺 aiRobots Laboratory, Department of Electrical Engineering, National Cheng Kung University, Tainan, Taiwan, R.O.C.

2. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

3. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

4. Introduction • The task of finding point correspondences between two images of the same scene or object is part of many computer vision applications. • This article presents a novel scale- and rotation-invariant detector and descriptor, coined SURF (Speeded-Up Robust Features). • SURF approximates or even outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared muchfaster.

5. Most valuable property: Repeatability (whether it reliably finds the same interest points under different viewing condition.) Introduction (cont’d) • The search for discrete image point correspondences can be divided into three main steps. Step1. Detector Interest points are selected Step2. Descriptor Extract the vector for matching Focus on scale and image rotation invariant. Step3. Match Often based on a distance between the vector

6. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

7. Related Work • Interest Point Detection => (1) Using the determinant of the Hessian matrix rather than its trace (the Laplacian) seems advantageous, (2) approximations like the DoG can bring speed at a low cost in terms of lost accuracy.

8. Related Work (cont’d) • Interest Point Description => The SIFT descriptor still seems to be the most appealing descriptor for practical uses, and hence also the most widely used nowadays.

9. Integral image: Property… B-D D B Σ C-D (x,y) A C (1)Fast implementation of box type convolution filters (2)Independent of its size Related Work (cont’d) • Our approach Step1. Fast-Hessian detector Based on the Hessian matrix but use a very basic approximation – DoG + Integral image (reduce the computation time) Step2. SURF Descriptor Describes a distribution of Haar-wavelet Responses within the interest point neighborhood Answer… + Integral image (reduce the computation time) ★Question: Why can this method reduce the computation time? Step3. Match Present a new indexing step based on the sign of the Laplacian (Speed up & increase the robustness)

10. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

11. Gaussian second order derivative x-dir y-dir xy-dir Fast-Hessian Detector • Hessian matrix H( x , σ) in x at scale σ is defined as • Approximation LoG with box filters => DoG Box filters (instead of Gaussian) x-dir y-dir xy-dir 9x9 box filter with σ=1.2

12. 27 x 27 (σ=3.6) 51 x 51 (σ=6.8) 21 x 21 (σ=2.8) 39 x 39 (σ=5.2) scale 15 x 15 (σ=2.0) 27 x 27 (σ=2.8) 15 x 15 (σ=2.0) 9 x 9 (σ=1.2) Octave2 (increase:12) Octave1 (increase:6) Fast-Hessian Detector (cont’d) • The scale space is analysed by up-scaling the filter size rather than iteratively reducing the image size. • The scale space is divided into octaves. An octave represents a series of increasing filter response maps. For each new octave, the filter size increase is doubled. (going from 6 to 12 to 24…) It is selected as the interest point only if it is larger than all of these neighbors.

13. Q & A (Fast-Hessian Detector) • Question1. 以放大filter的size代替將圖片縮小，有什麼好處? • Answer1. 因為integral image的使用，使得計算量不會隨filter的size增加，且沒有將圖片縮小，圖片就不會失真。

14. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

15. SURF Descriptor Orientation Assignment Fixing a reproducible orientation based on information from a circular region around the interest point. Descriptor Components Construct a square region aligned to the selected orientation, and extract the SURF descriptor from it. Interest point Features

16. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

17. =B-C-E+F F E D a =A-B-D+E C B A b Σ=-A+2B-C+D-2E+F 4s a-b dy 6s (dx1,dy1) π/3 dx (dx2,dy2) Image Orientation Assignment • Haar Wavelet • Orientation dx dy Orientation Interest point

18. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

19. 20s Haar wavelet (filter size 2s) Vertical direction Horizontal direction Descriptor Components • Constructing a square region centered around the interest point, and oriented along the orientation. • The region is split up regularly into smaller 4 × 4 square sub-regions. • (4x4)x4=> a 64 dimensional vector

20. Q&A(SURF Descriptor) • Question1. Why to use Σ|dx| and Σ|dy| ? • Answer1. • Question2. Why to use Haar wavelet response? • Answer2.

21. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

22. Matching • For fast indexing during the matching stage, the sign of the Laplacian (i.e. the trace of the Hessian matrix) for the underlying interest point is included. • In the matching stage, we only compare features if they have the same type of contrast.

23. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

24. Experiments • Scale variant + rotation

25. Experiments (cont’d) • Rotation

26. Experiments (cont’d) • Blurred

27. Experiments (cont’d) • Photometric deformations

28. Outline • Introduction • Related Work • Fast-Hessian Detector • SURF Descriptor • Orientation Assignment • Descriptor Components • Matching • Experiments • Conclusion

29. Conclusion • SURF outperforms previously proposed schemes with respect to repeatability, distinctiveness, and robustness, yet can be computed and compared much faster. • Future work will aim at optimizing the code for additional speed up.

30. ★ Thanks for your attention!!