1 / 144

Features: Digital Visual Effects with Yung-Yu Chuang

Explore the features in digital visual effects with Yung-Yu Chuang, with slides by Trevor Darrell and contributions from Cordelia Schmid, David Lowe, Darya Frolova, Denis Simakov, Robert Collins, and Jiwon Kim. Learn about feature detection, description, and matching, with a focus on the Harris corner detector.

harkless
Download Presentation

Features: Digital Visual Effects with Yung-Yu Chuang

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript


  1. Features Digital Visual Effects Yung-Yu Chuang with slides by Trevor DarrellCordelia Schmid, David Lowe, Darya Frolova, Denis Simakov, Robert Collins and Jiwon Kim

  2. Outline • Features • Harris corner detector • SIFT • Extensions • Applications

  3. Features

  4. ? Features • Also known as interesting points, salient points or keypoints. Points that you can easily point out their correspondences in multiple images using only local information.

  5. Desired properties for features • Distinctive: a single feature can be correctly matched with high probability. • Invariant: invariant to scale, rotation, affine, illumination and noise for robust matching across a substantial range of affine distortion, viewpoint change and so on. That is, it is repeatable.

  6. Applications • Object or scene recognition • Structure from motion • Stereo • Motion tracking • …

  7. Components • Feature detection locates where they are • Feature description describes what they are • Feature matching decides whether two are the same one

  8. Harris corner detector

  9. Moravec corner detector (1980) • We should easily recognize the point by looking through a small window • Shifting a window in anydirection should give a large change in intensity

  10. Moravec corner detector flat

  11. Moravec corner detector flat

  12. Moravec corner detector flat edge

  13. Moravec corner detector corner isolated point flat edge

  14. window function shifted intensity intensity Moravec corner detector Change of intensity for the shift [u,v]: Four shifts: (u,v) = (1,0), (1,1), (0,1), (-1, 1) Look for local maxima in min{E}

  15. Problems of Moravec detector • Noisy response due to a binary window function • Only a set of shifts at every 45 degree is considered • Only minimum of E is taken into account • Harris corner detector (1988) solves these problems.

  16. Harris corner detector Noisy response due to a binary window function • Use a Gaussian function

  17. Harris corner detector Only a set of shifts at every 45 degree is considered • Consider all small shifts by Taylor’s expansion

  18. Harris corner detector Only a set of shifts at every 45 degree is considered • Consider all small shifts by Taylor’s expansion

  19. Harris corner detector Equivalently, for small shifts [u,v] we have a bilinear approximation: , where M is a 22 matrix computed from image derivatives:

  20. Harris corner detector (matrix form)

  21. Harris corner detector Only minimum of E is taken into account • A new corner measurement by investigating the shape of the error function represents a quadratic function; Thus, we can analyze E’s shape by looking at the property of M

  22. Harris corner detector High-level idea: what shape of the error function will we prefer for features? flat edge corner

  23. Quadratic forms • Quadratic form (homogeneous polynomial of degree two) of n variables xi • Examples =

  24. Symmetric matrices • Quadratic forms can be represented by a real symmetric matrix A where

  25. Eigenvalues of symmetric matrices Brad Osgood

  26. Eigenvectors of symmetric matrices

  27. Eigenvectors of symmetric matrices

  28. Harris corner detector Intensity change in shifting window: eigenvalue analysis 1, 2 – eigenvalues of M direction of the fastest change Ellipse E(u,v) = const direction of the slowest change (max)-1/2 (min)-1/2

  29. Visualize quadratic functions

  30. Visualize quadratic functions

  31. Visualize quadratic functions

  32. Visualize quadratic functions

  33. Harris corner detector Classification of image points using eigenvalues of M: 2 edge 2 >> 1 Corner 1 and 2 are large,1 ~ 2;E increases in all directions 1 and 2 are small;E is almost constant in all directions edge 1 >> 2 flat 1

  34. Harris corner detector Only for reference, you do not need them to compute R Measure of corner response: (k – empirical constant, k = 0.04-0.06)

  35. Harris corner detector

  36. Another view

  37. Another view

  38. Another view

  39. Summary of Harris detector • Compute x and y derivatives of image • Compute products of derivatives at every pixel • Compute the sums of the products of derivatives at each pixel

  40. Summary of Harris detector • Define the matrix at each pixel • Compute the response of the detector at each pixel • Threshold on value of R; compute nonmax suppression.

  41. Harris corner detector (input)

  42. Corner response R

  43. Threshold on R

  44. Local maximum of R

  45. Harris corner detector

  46. Harris detector: summary • Average intensity change in direction [u,v] can be expressed as a bilinear form: • Describe a point in terms of eigenvalues of M:measure of corner response • A good (corner) point should have a large intensity change in all directions, i.e. R should be large positive

  47. Now we know where features are • But, how to match them? • What is the descriptor for a feature? The simplest solution is the intensities of its spatial neighbors. This might not be robust to brightness change or small shift/rotation. 1 2 3 4 5 6 7 8 9 ) ( 1 2 3 4 5 6 7 8 9

  48. threshold R R x(image coordinate) x(image coordinate) Harris detector: some properties • Partial invariance to affine intensity change • Only derivatives are used => invariance to intensity shift I I+b • Intensity scale: I  aI

  49. Harris Detector: Some Properties • Rotation invariance Ellipse rotates but its shape (i.e. eigenvalues) remains the same Corner response R is invariant to image rotation

  50. Harris Detector is rotation invariant Repeatability rate: # correspondences# possible correspondences

More Related