Extension of M-VOTE: Improving Feature Detection

1. Extension of M-VOTE: Improving Feature Detection Presented by Wyman Group Meeting 24 Oct 2005

2. Outline • Feature Detection • KLT detector • SUSAN detector • Rotational motion detection • How zooming of camera enhances interaction with user

3. Feature Detection • To detect points of interest (feature) out of an image • Requirement on points of interest • High contrast and different from nearby blocks • Corners are used in motion analysis • Motion is ambiguous at an edge but unambiguous at a corner • Corner detectors • Plessey detector • KLT detector • Susan detector

4. What Is a Corner? • Corners are local image features characterised by locations where variations of intensity function f(x, y) in both X and Y directions are high • Intersection points between two or more edge segments

5. Corner Detectors • Corner detection should satisfy a number of important criteria: • All the true corners should be detected • No false corners should be detected • Corner points should be well localized • Corner detector should be robust with respect to noise • Corner detector should be efficient

6. Families of Corner Detectors • Work directly with the values of brightness of images • Usually based on the study of derivatives (orientation, magnitude) of grey-level or color image • Extract object boundaries first then analyze its shape • Use edge detectors first then analyze the curvature of boundaries • Less reliable

7. KLT Corner Detector • Search for points where variations in two orthogonal directions are large • Based on local structure matrix • Derivatives of the intensity function f(x, y) are calculated in each point • Then, the entries of the matrix are obtained • Each of the entries is smoothed by Gaussian filter

8. KLT Corner Detector • The local structure matrix Cstr can be diagonalised by rotation of the coordinate axes. The diagonal entries will be the two eigenvalues • The eignevalues are nonnegative • Assume λ1>=λ2>=0 • For a perfectly uniform image: • Cstr = 0 and λ1=λ2=0 • For a perfectly black-and-white step edge: • λ1>0, λ2=0, where the eigenvector associated with λ1 is orthogonal to the edge • For a corner of black square against a white background: • λ1>=λ2>0. The higher contrast in that direction, the larger the eigenvalue • A corner is identified by two strong edges. That is, a corner is a location where the smaller eigenvalue,λ2 , is large enough

9. KLT Corner Detector • The KLT corner detector has two parameters: the threshold on λ2 denoted by λthr, and the linear size of a square window D • The algorithm • Compute fx and fy over the entire image f(x, y) • For each image point p: • form the matrix Cstr over a DxD neighbourhood of p; • compute λ2, the smaller eigenvalue of Cstr; • if λ 2 >λthr, save the p into a list, L. • Sort L in decreasing order of λ2. • Scan the sorted list from top to bottom. For each current point, p, delete all points appearing further in the list, which belong to the neighbourhood of p

10. KLT Corner Detector • The threshold λthr can be estimated from the histogram of λ2 • There is no simple criterion for the window size D. Values between 2 and 10 are adequate in most practical cases • But some corners which are close to each other may be lost for larger D

11. Example of corner detection

12. The SUSAN Detector • Proposed by Smith and Brady in 1995 • It doesn’t use any derivatives • SUSAN stands for Smallest “Univalue Segment Assimilating Nucleus” • It is based on the fact that each point within an image has associated with it a local area of comparable brightness • The Principle • It generates a circular mask around a given point in an image • It compares the intensity of neightbouring pixels with that of the centre pixel (nucleus of the mask), the area with similar intensity to the nucleus is called USAN area • Repeat the procedure for each pixel within the image

13. The SUSAN Detector • USAN area varies within the image depending on its location with respect to special features of the image • The USAN area is maximum within the rectangular area but falls to a minimum at an edge and to an even smaller value corresponding to a local minimum at a corner • This is the property upon which the corner finder algorithm is based, hence the name SUSAN

14. The SUSAN Detector • The algorithm • Determine a circular mask, typically of 37 pixels around a nucleus for each point within the image • Calculate the difference in brightness between each pixel of the mask and that of its nucleus and • Sum the number of pixels within the circular mask which have similar intensity levels to that of the nucleus

15. The SUSAN Detector • Compare n with g, the geometric threshold which is set to half of the maximum value that n can be (nmax/2) • At a perfect corner (where two straight edges intersect) the USAN area will always be less than half the size of the mask area, and will be a local minimum

16. The SUSAN Detector (Improved) • However, the above process may generate “false” corners • There are a couple of experiments to remove false positives arising from strong edges and noise • The first test consists in finding the centre of gravity of the USAN area • Only those pixels falling on a straight line from the nucleus of the mask to its centre of gravity and forming part of the USAN area will constitute a proper corner

17. The SUSAN Detector • The initial corner response image is subjected to non-maximal suppression • Local maxima are searched for in smaller five-by-five pixel regions • Find a pixel which has a response greater than a given threshold is also greater than all its immediate neighbors • In other words, if at least one of those neighboring pixels has a magnitude higher than that of the examined pixel then the latter is assigned a zero value

18. The SUSAN Detector • SUSAN has better accuracy, reliability and speed than Plessey corner detector Output of the Plessey corner detector (o =2.0) given the test image. (3.5 sec) Output of the SUSAN corner detector (t=10) given the test image. (0.3 sec)

19. SUSAN Corner Detection Result

20. Advantages of SUSAN Detector • It performs well even in the presence of noise • No image derivatives are used • Integrating effect and non-linear response give strong noise rejection • The use of controlling parameters is much simpler and less arbitrary

21. Problems • Are these corner detection algorithms fast enough for mobile devices? • Floating point operation consideration • How to extend the algorithms to detect corners in color image? • Robust feature matching across widely separated color images. Proceedings of the 17th International Conference on Pattern Recognition (ICPR’04)

22. Reference • S.M. Smith. SUSAN - a new approach to low level image processing. Internal Technical Report TR95SMS1, Defence Research Agency, Chobham Lane, Chertsey, Surrey, UK, 1995. • J. Shi and C. Tomasi. Good features to track. In Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR94), Seattle, June 1994.