1 / 72

Locating and Describing Interest Points

season
Download Presentation

Locating and Describing Interest Points

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

    1. Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem

    2. What is “object recognition”?

    3. 1. Identify a Specific Instance General objects Challenges: rotation, scale, occlusion, localization Approaches Geometric configurations of keypoints (Lowe 2004) Works well for planar, textured objects

    4. 1. Identify a Specific Instance Faces Typical scenario: few examples per face, identify or verify test example What’s hard: changes in expression, lighting, age, occlusion, viewpoint Basic approaches (all nearest neighbor) Project into a new subspace (or kernel space) (e.g., “Eigenfaces”=PCA) Measure face features Make 3d face model, compare shape+appearance (e.g., AAM)

    5. 2. Detect Instance of a Category Much harder than specific instance recognition Challenges Everything in instance recognition Intraclass variation Representation becomes crucial

    6. 2. Detect Instance of a Category Template or sliding window Works well when Object fits well into rectangular window Interior features are discriminative

    7. 2. Detect Instance of a Category Parts-based

    8. 3. Assign a label to a pixel or region Stuff Materials, object regions, textures, etc. Approaches Label patches + CRF Segmentation + Label Regions

    9. Next two lectures Object instance recognition Interest points (today) Detecting Representing Simple matching Recognizing objects with interest points (Thurs) Geometric verification RANSAC + Hough voting Efficient matching

    10. General Process of Object Recognition

    11. General Process of Object Recognition

    12. General Process of Object Recognition

    13. General Process of Object Recognition

    14. Overview of Keypoint Matching K. Grauman, B. Leibe

    15. Main challenges Change in position and scale Change in viewpoint Occlusion Articulation

    16. Goals for Keypoints Detect points that are repeatable and distinctive

    17. Key trade-offs

    18. Keypoint localization Goals: Repeatable detection Precise localization Interesting content

    19. Keypoint Localization Goals: Repeatable detection Precise localization Interesting content K. Grauman, B. Leibe

    20. Keypoint Localization Goals: Repeatable detection Precise localization Interesting content ? Look for two-dimensional signal changes K. Grauman, B. Leibe

    21. Choosing interest points If you wanted to meet a friend would you say “Let’s meet on campus.” “Let’s meet on Green street.” “Let’s meet at Green and Wright.” Corner detection Or if you were in a secluded area: “Let’s meet in the Plains of Akbar.” “Let’s meet on the side of Mt. Doom.” “Let’s meet on top of Mt. Doom.” Blob (valley/peak) detection

    22. Choosing interest points Corners “Let’s meet at Green and Wright.” Peaks/Valleys “Let’s meet on top of Mt. Doom.”

    23. Many Existing Detectors Available K. Grauman, B. Leibe Hessian & Harris [Beaudet ‘78], [Harris ‘88] Laplacian, DoG [Lindeberg ‘98], [Lowe 1999] Harris-/Hessian-Laplace [Mikolajczyk & Schmid ‘01] Harris-/Hessian-Affine [Mikolajczyk & Schmid ‘04] EBR and IBR [Tuytelaars & Van Gool ‘04] MSER [Matas ‘02] Salient Regions [Kadir & Brady ‘01] Others…

    24. Hessian Detector [Beaudet78] Hessian determinant K. Grauman, B. Leibe

    25. Hessian Detector [Beaudet78] Hessian determinant K. Grauman, B. Leibe

    26. Hessian Detector – Responses [Beaudet78]

    27. Hessian Detector – Responses [Beaudet78]

    28. Harris Detector [Harris88] Second moment matrix (autocorrelation matrix) K. Grauman, B. Leibe

    29. Harris Detector [Harris88] Second moment matrix (autocorrelation matrix) K. Grauman, B. Leibe

    30. Harris Detector [Harris88] Second moment matrix (autocorrelation matrix) 30 K. Grauman, B. Leibe

    31. Harris Detector [Harris88] Second moment matrix (autocorrelation matrix) 31

    32. Harris Detector [Harris88] Second moment matrix (autocorrelation matrix) 32

    33. Harris Detector: Mathematics

    34. Harris Detector – Responses [Harris88]

    35. Harris Detector – Responses [Harris88]

    36. So far: can localize in x-y, but not scale

    37. Automatic Scale Selection K. Grauman, B. Leibe

    38. Automatic Scale Selection Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

    39. Automatic Scale Selection Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

    40. Automatic Scale Selection Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

    41. Automatic Scale Selection Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

    42. Automatic Scale Selection Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

    43. Automatic Scale Selection Function responses for increasing scale (scale signature) K. Grauman, B. Leibe

    44. What Is A Useful Signature Function? Laplacian-of-Gaussian = “blob” detector K. Grauman, B. Leibe

    45. Laplacian-of-Gaussian (LoG) Local maxima in scale space of Laplacian-of- Gaussian K. Grauman, B. Leibe

    46. Results: Laplacian-of-Gaussian K. Grauman, B. Leibe

    47. Difference-of-Gaussian (DoG) Difference of Gaussians as approximation of the Laplacian-of-Gaussian K. Grauman, B. Leibe

    48. DoG – Efficient Computation Computation in Gaussian scale pyramid K. Grauman, B. Leibe

    49. Results: Lowe’s DoG K. Grauman, B. Leibe

    50. T. Tuytelaars, B. Leibe Orientation Normalization Compute orientation histogram Select dominant orientation Normalize: rotate to fixed orientation

    51. Harris-Laplace [Mikolajczyk ‘01] Initialization: Multiscale Harris corner detection

    52. Harris-Laplace [Mikolajczyk ‘01] Initialization: Multiscale Harris corner detection Scale selection based on Laplacian (same procedure with Hessian ? Hessian-Laplace) K. Grauman, B. Leibe

    53. Maximally Stable Extremal Regions [Matas ‘02] Based on Watershed segmentation algorithm Select regions that stay stable over a large parameter range K. Grauman, B. Leibe

    54. Example Results: MSER 54 K. Grauman, B. Leibe

    55. Comparison

    56. Available at a web site near you… For most local feature detectors, executables are available online: http://robots.ox.ac.uk/~vgg/research/affine http://www.cs.ubc.ca/~lowe/keypoints/ http://www.vision.ee.ethz.ch/~surf K. Grauman, B. Leibe

    57. Local Descriptors The ideal descriptor should be Robust Distinctive Compact Efficient Most available descriptors focus on edge/gradient information Capture texture information Color rarely used K. Grauman, B. Leibe

    58. Local Descriptors: SIFT Descriptor

    59. Details of Lowe’s SIFT algorithm Run DoG detector Find maxima in location/scale space Remove edge points Find all major orientations Bin orientations into 36 bin histogram Weight by gradient magnitude Weight by distance to center (Gaussian-weighted mean) Return orientations within 0.8 of peak Use parabola for better orientation fit For each (x,y,scale,orientation), create descriptor: Sample 16x16 gradient mag. and rel. orientation Bin 4x4 samples into 4x4 histograms Threshold values to max of 0.2, divide by L2 norm Final descriptor: 4x4x8 normalized histograms

    60. Matching SIFT Descriptors Nearest neighbor (Euclidean distance) Threshold ratio of nearest to 2nd nearest descriptor

    61. SIFT Repeatability

    62. SIFT Repeatability

    63. SIFT Repeatability

    64. SIFT Repeatability

    65. Local Descriptors: SURF K. Grauman, B. Leibe

    66. Record distribution of points relative to each point Record distribution of points relative to each point

    68. Choosing a detector What do you want it for? Precise localization in x-y: Harris Good localization in scale: Difference of Gaussian Flexible region shape: MSER Best choice often application dependent Harris-/Hessian-Laplace/DoG work well for many natural categories MSER works well for buildings and printed things Why choose? Get more points with more detectors There have been extensive evaluations/comparisons [Mikolajczyk et al., IJCV’05, PAMI’05] All detectors/descriptors shown here work well

    69. Comparison of Keypoint Detectors

    70. Choosing a descriptor Again, need not stick to one For object instance recognition or stitching, SIFT or variant is a good choice

    71. Things to remember Keypoint detection: repeatable and distinctive Corners, blobs, stable regions Harris, DoG Descriptors: robust and selective spatial histograms of orientation SIFT

    72. Next time Recognizing objects using keypoints

More Related