Real Time Object Recognition Using the SURF Algorithm

1. Real Time Object Recognition Using the SURF Algorithm Frank Bergschneider February 21, 2014 Presented to National Instruments

2. Introduction • Object Recognition • Real Time, Online Applications • Applications: Security, Shopping, Augmented Reality • Speeded Up Robust Features Algorithm • OpenCVLibrary

3. Object Recognition • 3 General Steps • 1. Interest Point Detection • 2. Interest Point Description Feature Vector Extraction • 3. Feature Vector Matching Between Two Images

4. Object Recognition Load training image Detect training interest points Extract training interest point descriptors Initialize match object Initialize and open camera feed While (Not User Exit) Grab video frame Detect interest points Extract descriptors Match query points with training points If (Matching Points > Threshold) Compute Homography Transform Box Draw Box on Object and Display Else Continue End While

5. Interest Point Detection • Approximating the determinant of the Hessian matrix • If determinant is local max -> Interest Point • Hessian approx. using Integral Images and Box Filters • Big decreases in calculation time (constant time calcs!)

6. Interest Point Detection • Integral Image • Hessian Matrix

7. Interest Point Detection • Interest Points found at different scales • Box Filter scaled up, instead of down sampling image The first octave interest points are on the left side, and the second and third octave interest points are the right side for clarity

8. Interest Point Detection • Calculate Integral Image, I • Loop For Each Point • Loop For Octave • Calculate dxx, dyy, dxy with Box Filter • Normalize Responses • Calculate Determinant • If (Determinant > Threshold) • Store Point as Interest Point at Scale • End If • End Loop Octave • End Loop For Each Point • Suppress Non Maximum Interest Points • Interpolate Interest Points Between Octaves • Output: Interest Points

9. Feature Vector Description • Interest Point Characterized with Haar Wavelets • Orientation Information Extracted • Square Area Around IP in Direction of Orientation • Spatial Intensity Distribution

10. Feature Vector Description

11. Feature Vector Description • For Each Interest Point • Calculate Orientation • Window Area of 20s around Point • Divide Window into 4x4 subareas • For Each Subarea • Calculate Haar Wavelet • Smooth with Gaussian • Form Feature Vector • End For Each Subarea • Store Feature Vector • End For Each Interest Point • Output: Feature Vector

12. Feature Vector Matching • k-Nearest Neighbors • Fast Library for Approximated Nearest Neighbors (FLANN) • 2 Nearest Neighbors Found • Distance Ratio Rule for Good Matches • If Number of Good Matches > Threshold Then Object Recognized

13. Feature Vector Matching

14. Results • Key Parameter: Min Determinant of Hessian • Balance Btw Detector and Descriptor at det(H)=4500 • Best: det(H)=2000, d_Ratio=0.6, Good Matches=8 • Frames Per Second: 2.9

15. Conclusion • SURF Algorithm implemented and test with OpenCV • Robust to Scale Change, Rotation, and Noise • Sufficiently Fast for Real Time Object Recognition • Integral Image and Box Filter yield huge performance • Algorithm prime for FPGA/GPU implementation