1 / 37

Stanford CS223B Computer Vision, Winter 2006 Lecture 7 Optical Flow

Stanford CS223B Computer Vision, Winter 2006 Lecture 7 Optical Flow. Optical Flow: Outline. Examples Formal definition, 1D case From 1D to 2D: Aperture Problem Course motion and pyramids. Optical Flow. * Picture from Khurram Hassan-Shafique CAP5415 Computer Vision 2003.

mackenzie-levine
Download Presentation

Stanford CS223B Computer Vision, Winter 2006 Lecture 7 Optical Flow

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Stanford CS223B Computer Vision, Winter 2006Lecture 7 Optical Flow

  2. Optical Flow: Outline • Examples • Formal definition, 1D case • From 1D to 2D: Aperture Problem • Course motion and pyramids

  3. Optical Flow * Picture from Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  4. Optical Flow: Outline • Examples • Formal definition, 1D case • From 1D to 2D: Aperture Problem • Course motion and pyramids • Flow Segmentation

  5. Optical Flow Image tracking 3D computation Image sequence (single camera) Tracked sequence 3D structure + 3D trajectory

  6. Optical Flow Velocity vectors Common assumption: The appearance of the image patches do not change (brightness constancy) What is Optical Flow?

  7. Optical Flow Assumptions:Brightness Constancy * Slide from Michael Black, CS143 2003

  8. Optical Flow Assumptions: * Slide from Michael Black, CS143 2003

  9. Optical Flow Assumptions: * Slide from Michael Black, CS143 2003

  10. { Because no change in brightness with time Ix v It Optical Flow: 1D Case Brightness Constancy Assumption:

  11. ? Tracking in the 1D case:

  12. Temporal derivative Spatial derivative Assumptions: • Brightness constancy • Small motion Tracking in the 1D case:

  13. Temporal derivative at 2nd iteration Can keep the same estimate for spatial derivative Tracking in the 1D case: Iterating helps refining the velocity vector Converges in about 5 iterations

  14. For all pixel of interest p: • Compute local image derivative at p: • Initialize velocity vector: • Repeat untilconvergence: • Compensate for current velocity vector: • Compute temporal derivative: • Update velocity vector: Requirements: • Need access to neighborhood pixels round p to compute • Need access to the second image patch, for velocity compensation: • The pixel data to be accessed in next image depends on current velocity estimate (bad?) • Compensation stage requires a bilinear interpolation (because v is not integer) • The image derivative needs to be kept in memory throughout the iteration process Algorithm for 1D tracking:

  15. Optical Flow: Outline • Examples • Formal definition, 1D case • From 1D to 2D: Aperture Problem • Course motion and pyramids • Flow Segmentation

  16. 2D: From 1D to 2D tracking 1D: Shoot! One equation, two velocity (u,v) unknowns…

  17. From 1D to 2D tracking We get at most “Normal Flow” – with one point we can only detect movement perpendicular to the brightness gradient. Solution is to take a patch of pixels Around the pixel of interest. * Slide from Michael Black, CS143 2003

  18. How does this show up visually?Known as the “Aperture Problem”

  19. Aperture Problem Exposed Motion along just an edge is ambiguous

  20. Aperture Problem: Example

  21. Aperture Problem in Real Life

  22. From 1D to 2D tracking The Math is very similar: Window size here ~ 5x5 or 11x11

  23. More Detail:Solving the aperture problem • How to get more equations for a pixel? • Basic idea: impose additional constraints • most common is to assume that the flow field is smooth locally • one method: pretend the pixel’s neighbors have the same (u,v) • If we use a 5x5 window, that gives us 25 equations per pixel! * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  24. Solution: solve least squares problem • minimum least squares solution given by solution (in d) of: • The summations are over all pixels in the K x K window • This technique was first proposed by Lukas & Kanade (1981) • described in Trucco & Verri reading Lukas-Kanade flow • Prob: we have more equations than unknowns * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  25. Conditions for solvability • Optimal (u, v) satisfies Lucas-Kanade equation • When is This Solvable? • ATA should be invertible • ATA should not be too small due to noise • eigenvalues l1 and l2 of ATA should not be too small • ATA should be well-conditioned • l1/ l2 should not be too large (l1 = larger eigenvalue) * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  26. Edge • large gradients, all the same • large l1, small l2 * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  27. Low texture region • gradients have small magnitude • small l1, small l2 * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  28. High textured region • gradients are different, large magnitudes • large l1, large l2 * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  29. Observation • This is a two image problem BUT • Can measure sensitivity by just looking at one of the images! • This tells us which pixels are easy to track, which are hard • very useful later on when we do feature tracking... • Once suggestion: Track Harris Corners! * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  30. Optical Flow, Example Harris Corners David Stavens, Andrew Lookingbill, David Lieb (CS223B 2004)

  31. Optical Flow, Example Optical flow David Stavens, Andrew Lookingbill, David Lieb (CS223B 2004)

  32. Optical Flow: Outline • Examples • Formal definition, 1D case • From 1D to 2D: Aperture Problem • Course motion and pyramids

  33. Revisiting the small motion assumption • Is this motion small enough? • Probably not—it’s much larger than one pixel (2nd order terms dominate) • How might we solve this problem? * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  34. Reduce the resolution! * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

  35. u=1.25 pixels u=2.5 pixels u=5 pixels u=10 pixels image It-1 image It-1 image I image I Gaussian pyramid of image It-1 Gaussian pyramid of image I Coarse-to-fine optical flow estimation

  36. warp & upsample run iterative L-K . . . image J image It-1 image I image I Gaussian pyramid of image It-1 Gaussian pyramid of image I Coarse-to-fine optical flow estimation run iterative L-K

  37. Multi-resolution Lucas Kanade Algorithm

More Related