650 likes | 801 Views
Advanced Computer Vision Introduction. Lecture 02 Roger S. Gaborski. Corner Detection: Basic Idea. We should easily recognize the point by looking through a small window Shifting a window in any direction should give a large change in intensity.
E N D
Advanced Computer VisionIntroduction Lecture 02 Roger S. Gaborski Roger S. Gaborski
Corner Detection: Basic Idea We should easily recognize the point by looking through a small window Shifting a window in anydirection should give a large change in intensity “flat” region:no change in all directions “edge”:no change along the edge direction “corner”:significant change in all directions Roger S. Gaborski Source: A. Efros
Wikipedia: Corner Detection-No shortage of corner detectors • Moravec • Harris • Laplacian of gaussians, Difference of Gaussian and Determinant of the Hessian • SIFT • SUSAN Corner Detector • Trajkovic – Hedley Corner Detector Roger S. Gaborski
Corner Detection: Weighted Sum of Differences Window function Shifted intensity Intensity Window function w(u,v) = Area of patch: (u,v) or 1 in window, 0 outside Gaussian Change in appearance for the shift [x,y]: E(x,y) = ΣΣ w(u,v) (I(u+x, v+y) – I(u,v))2 u v Roger S. Gaborski Source: Wikipedia
Corner Detection: Mathematics Change in appearance for the shift [x,y]: I(u,v) E(x,y) E(3,2) E(0,0) Roger S. Gaborski Source: R. Szeliski
Corner Detection Change in appearance for the shift [x,y]: • E(x,y) = ΣΣ w(u,v) (I(u+x, v+y) – I(u,v))2 We want to find out how this function behaves for small shifts Taylor expansion of I(u+x, v+y)= I(u,v)+Ix(u,v)x+Iy(u,v)y Ix and Iy are partial derivatives of x and y Results in E(x,y) = ΣΣ w(u,v) (Ix(u,v)x+Iy(u,v)y)2 Roger S. Gaborski
Corner Detection: Mathematics The approximation simplifies to E(x,y) = (x,y) A (x,y)T Where Ais a second moment matrixcomputed from image derivatives: A = ΣΣ w(u,v) Ix2IxIy IxIy Iy2 Roger S. Gaborski
Interpreting the second moment matrix Analyze eigenvalues of A to determine if a corner exists If λ1and λ2= 0, then pixel (x,y) is not of interest If one eigenvalue λ is close to 0, and the other is large, (x,y) is on an edge If both eigenvalues λ are large, (x,y) is a corner Roger S. Gaborski
Interpreting the eigenvalues Classification of image points using eigenvalues of M: 2 “Edge”2 >> 1 “Corner”1 and 2 are large,1 ~ 2;E increases in all directions 1 and 2 are small;E is almost constant in all directions “Edge”1 >> 2 “Flat” region 1 Roger S. Gaborski
Defining Corner Response Function, R Instead of calculating eigenvalues, Harris defined the following function: R = det(A)- α trace(A)2 = λ1λ2 - α (λ1 + λ2) Where α = .04 to .06 Recall: A = [ a b; c d] Det(A) = ad – bc Roger S. Gaborski
Corner Response Function R < 0 R > 0 Edge Corners |R| small R < 0 “Flat” region Edge Roger S. Gaborski
Harris Detector Algorithm • Compute Derivatives at each point • Compute Second Moment Matrix A • Compute Corner Response Function • Threshold R • Find Local Maxima Roger S. Gaborski
Harris Corner Detector • Reference: C.G. Harris and M.J. Stephens “A Combined Corner and Edge Detector” • Code inspired by Peter Kovesi • Derivative Masks: dx = [-1, 0, 1;-1, 0, 1;-1, 0, 1] • dy = dx’ • Image Derivatives; • Ix = imfilter(im, dx, 'conv',‘same’); • Iy = imfilter(im, dy, 'conv',‘same’); • Gaussian Filter • g = fspecial(‘gaussian’, 6*sigma, sigma); Roger S. Gaborski
Smooth squared image derivative • Ix2 = imfilter (Ix.^2, g, 'conv', ‘same’); • Iy2 = imfilter (Iy.^2, g, 'conv', ‘same’); • IxIy = imfilter (Ix .* Iy, g, 'conv', ‘same’); c = (Ix2.*Iy2 – IxIy.^2)./(Ix2+Iy2).^2; Roger S. Gaborski
Non-maximal Suppression and Threshold • Extract local maxima – gray scale morphological dilation • size = 2*radius+1; %radius is parameter • mx = imdilate(c,ones(size)); %gray scale dilate • cc = (c==mx)&(c>thresh); %find maxima • [r,c] = find(cc) %find row, col coordinates • figure, imagesc( im), colormap(gray) • hold on • plot(c,r, ‘rs’), title(‘Corners) Roger S. Gaborski
100x100 Grid background =1, lines = 1 Roger S. Gaborski
Image rotated 45 Degrees Roger S. Gaborski
Image rotated 45 Degreessame parameters Roger S. Gaborski
Porsche Image Roger S. Gaborski
Harris Points Roger S. Gaborski
1983 Porsche Roger S. Gaborski
HW#3 – Due Tuesday, 12/13 10am • Work in teams of 2 or 3 • Write a Harris Detector Function (do not simply copy one from web, write your own) • Experiment with ‘grid image’ and Flower2 image and two ‘interesting’ images of your choice. NOTE: Create your own grid image using Matlab (for example, a matrix of all zeros with white lines, pixels = 1) • Goals: - Find all intersections on grid image • Detect all petal end points on flower image – better results that class lecture slide • Email: rsg.advcv@gmail.com • 1- write up including result images, observations and 2- MATLAB code Roger S. Gaborski
Object Recognition • Issues: • Viewpoint • Scale • Deformable vs. rigid • Clutter • Occlusion • Intra class variability Roger S. Gaborski
Goal • Locate all instances of automobiles in a cluttered scene Roger S. Gaborski
Acknowledgements • Students (Thesis in RIT Library): • Tim Lebo • Dan Clark • Images used in presentation: • ETHZ Database, UIUC Database Roger S. Gaborski
Object Recognition Approaches • For specific object class: • Holistic • Model whole object • Parts based • Simple parts • Geometric relationship information • We could use a similar approach to match patches representing different image categories (‘sand’ patches located on lower half of beach scenes) Roger S. Gaborski
Training Images and Segmentation Roger S. Gaborski
Implicit Shape Model • Patches – local appearance prototypes • Spatial relationship – where the patch can be found on the object • For a given class w: ISM(w) = (Iw ,Pw ) where Iw is the codebook containing the patches and Pw is the probability distribution that describes where the patch is found on the object • How do we find ‘interesting’ patches? Roger S. Gaborski
Harris Point Operator Roger S. Gaborski
Harris Points Roger S. Gaborski
Segmented Training Mask Segmented mask ensures only patches containing valid car regions are selected A corresponding segmentation patch is also extracted Roger S. Gaborski
Selected Patches Roger S. Gaborski
How is spatial information represented? • Estimate the center of the object using the centroid of the segmentation mask • Displacement between: • Center of patch • Centroid of segmentation mask Roger S. Gaborski
Individual Patch and Displacement Information Roger S. Gaborski
Typical Training Example Roger S. Gaborski
Typical Training Example Roger S. Gaborski
Extracted Training Patches Roger S. Gaborski
Cluster Patches • Many patches will be visually similar • Normalized Grayscale Correlation is used to cluster patches • All patches within a certain neighborhood defined by the NGC are grouped together • The representative patch is determined by mean of the patches • The geometric information for each patch in the cluster is assigned to the representative patch Roger S. Gaborski
Patches Roger S. Gaborski
Wheel Patch Example Roger S. Gaborski
Clusters Opportunity for better clustering method Roger S. Gaborski
Clusters Roger S. Gaborski
Object Detection • Harris point operator to find interesting points • Extract patches • Match extracted patches with model patches • Spatial information predicts center of object • Create voting space Roger S. Gaborski
Ideal Voting Space Example Roger S. Gaborski
Multiple Votes Multiple geometric interpretations Roger S. Gaborski
Resolving False Detections Roger S. Gaborski