Advanced Computer Vision Introduction

1. Advanced Computer VisionIntroduction Lecture 02 Roger S. Gaborski Roger S. Gaborski

2. Corner Detection: Basic Idea We should easily recognize the point by looking through a small window Shifting a window in anydirection should give a large change in intensity “flat” region:no change in all directions “edge”:no change along the edge direction “corner”:significant change in all directions Roger S. Gaborski Source: A. Efros

3. Corner Detection: Mathematics Window function Shifted intensity Intensity Window function w(x,y) = or 1 in window, 0 outside Gaussian Change in appearance for the shift [u,v]: Roger S. Gaborski Source: R. Szeliski

4. Corner Detection: Mathematics Change in appearance for the shift [u,v]: I(x, y) E(u, v) E(3,2) E(0,0) Roger S. Gaborski Source: R. Szeliski

5. Corner Detection: Mathematics Change in appearance for the shift [u,v]: We want to find out how this function behaves for small shifts Second-order Taylor expansion of E(u,v) about (0,0) (local quadratic approximation): Roger S. Gaborski Source: R. Szeliski

6. Corner Detection: Mathematics M The quadratic approximation simplifies to where M is a second moment matrixcomputed from image derivatives: Roger S. Gaborski Source: R. Szeliski

7. Interpreting the second moment matrix First, consider the axis-aligned case (gradients are either horizontal or vertical) λ1 and λ2 will be proportional to the principal curvature of autocorrelation function. If either eigenvalue λ is close to 0, then this is not a corner, so look for locations where both are large. Roger S. Gaborski

8. Interpreting the eigenvalues Classification of image points using eigenvalues of M: 2 “Edge” 2 >> 1 “Corner”1 and 2 are large,1 ~ 2;E increases in all directions 1 and 2 are small;E is almost constant in all directions “Edge” 1 >> 2 “Flat” region 1 Roger S. Gaborski

9. Defining Corner Response Function, R Recall: A = [ a b; c d] Det(A) = ad – bc Then: R = det(M)- α trace(M)2 = λ1λ2 - α (λ1 + λ2) Where α = .04 to .06 Roger S. Gaborski

10. Corner Response Function R < 0 R > 0 Edge Corners |R| small R < 0 “Flat” region Edge Roger S. Gaborski

11. Harris Detector Algorithm • Compute Gaussian Derivatives at each point • Compute Second Moment Matrix M • Compute Corner Response Function • Threshold R • Find Local Maxima Roger S. Gaborski

12. Harris Corner Detector • Reference: C.G. Harris and M.J. Stephens “A Combined Corner and Edge Detector” • Code inspired by Peter Kovesi • Derivative Masks: dx = [-1, 0, 1;-1, 0, 1;-1, 0, 1] • dy = dx’ • Image Derivatives; • Ix = imfilter(im, dx, 'conv',‘same’); • Iy = imfilter(im, dy, 'conv',‘same’); • Gaussian Filter • g = fspecial(‘gaussian’, 6*sigma, sigma); Roger S. Gaborski

13. Smooth squared image derivative • Ix2 = imfilter (Ix.^2, g, 'conv', ‘same’); • Iy2 = imfilter (Iy.^2, g, 'conv', ‘same’); • IxIy = imfilter (Ix .* Iy, g, 'conv', ‘same’); c = (Ix2.*Iy2 – IxIy.^2)./(Ix2+Iy2).^2; Roger S. Gaborski

14. Non-maximal Suppression and Threshold • Extract local maxima – gray scale morphological dilation • size = 2*radius+1; %radius is parameter • mx = imdilate(c,ones(size)); %gray scale dilate • cc = (c==mx)&(c>thresh); %find maxima • [r,c] = find(cc) %find row, col coordinates • figure, imagesc( im), colormap(gray) • hold on • plot(c,r, ‘rs’), title(‘Corners) Roger S. Gaborski

15. 100x100 Grid background =1, lines = 1 Roger S. Gaborski

16. Roger S. Gaborski

17. Image rotated 45 Degrees Roger S. Gaborski

18. Image rotated 45 Degreessame parameters Roger S. Gaborski

19. Roger S. Gaborski

20. Roger S. Gaborski

21. Porsche Image Roger S. Gaborski

22. Harris Points Roger S. Gaborski

23. 1983 Porsche Roger S. Gaborski

24. HW#2 – Due Tuesday, noon • Work in teams of 2 or 3 • Write a Harris Detector Function (do not simply copy one from web, write your own) • Experiment with ‘grid image’ and Flower2 image and two ‘interesting’ images of your choice • Goals: - Find all intersections on grid image • Detect all petal end points on flower image – better results that class lecture slide • Email: 1- write up including result images, observations and 2-MATLAB code Roger S. Gaborski

25. Object Recognition • Issues: • Viewpoint • Scale • Deformable vs. rigid • Clutter • Occlusion • Intra class variability Roger S. Gaborski

26. Current Work • Fix: • Viewpoint • Scale • Rigid • Explore affects of: • Intra class variability • Clutter • Occlusion Roger S. Gaborski

27. Goal • Locate all instances of automobiles in a cluttered scene Roger S. Gaborski

28. Acknowledgements • Students: • Tim Lebo • Dan Clark • Images used in presentation: • ETHZ Database, UIUC Database Roger S. Gaborski

29. Object Recognition Approaches • For specific object class: • Holistic • Model whole object • Parts based • Simple parts • Geometric relationship information Roger S. Gaborski

30. Training Images and Segmentation Roger S. Gaborski

31. Implicit Shape Model • Patches – local appearance prototypes • Spatial relationship – where the patch can be found on the object • For a given class w: ISM(w) = (Iw ,Pw ) where Iw is the codebook containing the patches and Pw is the probability distribution that describes where the patch is found on the object • How do we find ‘interesting’ patches? Roger S. Gaborski

32. Harris Point Operator • what is it? Roger S. Gaborski

33. Roger S. Gaborski

34. Harris Points Roger S. Gaborski

35. Segmented Training Mask Segmented mask ensures only patches containing valid car regions are selected A corresponding segmentation patch is also extracted Roger S. Gaborski

36. Selected Patches Roger S. Gaborski

37. How is spatial information represented? • Estimate the center of the object using the centroid of the segmentation mask • Displacement between: • Center of patch • Centroid of segmentation mask Roger S. Gaborski

38. Individual Patch and Displacement Information Roger S. Gaborski

39. Typical Training Example Roger S. Gaborski

40. Typical Training Example Roger S. Gaborski

41. Extracted Training Patches Roger S. Gaborski

42. Cluster Patches • Many patches will be visually similar • Normalized Grayscale Correlation is used to cluster patches • All patches within a certain neighborhood defined by the NGC are grouped together • The representative patch is determined by mean of the patches • The geometric information for each patch in the cluster is assigned to the representative patch Roger S. Gaborski

43. Patches Roger S. Gaborski

44. Wheel Patch Example Roger S. Gaborski

45. Clusters Opportunity for better clustering method Roger S. Gaborski

46. Clusters Roger S. Gaborski

47. Roger S. Gaborski

48. Object Detection • Harris point operator to find interesting points • Extract patches • Match extracted patches with model patches • Spatial information predicts center of object • Create voting space Roger S. Gaborski

49. Ideal Voting Space Example Roger S. Gaborski

50. Multiple Votes Multiple geometric interpretations Roger S. Gaborski